The problem, or one of them, is that the physical sciences, at least as they are presently constituted, haven't even got the necessary concepts available to them. If you ask a physicist for a definition of "red" he will give you a particular wavelength of electromagnetic radiation, but there is nothing in his lexicon which corresponds to what Joe Bloggs means by "red".
It's not a conceptual problem, but an epistemological problem. Science is a necessarily objective enterprise, and so cannot directly address the essence of subjective experience; there's a philosophical divide of the same order as the interaction problem (the dualist notion of the immaterial influencing the material). Science can only describe qualia as the subjective experience of certain patterns of neural activity.
But a physicist is the wrong person to ask for a biologically relevant definition of what is perceived as 'red'. Neuroscientists can tell you that it isn't only related to a specific range of frequencies of light, but also to a whole range of other visual factors; it is affected by the colour nature (hue, value, tint, shade, and saturation) of the overall illumination; with the colour nature of the surrounding areas, with the 3D structure of the environment, and so-on. See
Color Illusions for some illustrations. Paul Churchland's paper '
Chimerical Colors' (
Philosophical Psychology Vol. 18, No. 5, October 2005, pp. 527–560 - now, sadly, behind a paywall) demonstrated how a simple triplet circuit of opposing colour processing neurons in the primary visual pathway could give rise to the full colour space we perceive, and account for context relative colour changes. More importantly, this neural model predicted there should be three types of colours possible to experience
outside the natural colour spindle (i.e. 'chimerical', 'self-luminous', and 'hyperbolic' colours), that are never seen in nature. He went further and gave colour plates with instructions for how to see examples of these literally extraordinary colours.
This kind of work, and the work on visual anomalies (colour blindness, tetrachromacy, etc.), means we know in great detail
how our colour perceptions are generated, and
why they vary between individuals, and with context, the way they do; but the essential subjective quality of the experience ('qualia') is outside the scientific remit.
Personally, I suspect it's an 'empty' question, like asking why you are
you, in
your part of the world, rather than someone else in
another part of the world.