Not exactly; colours are mental representations of the world
based on wavelengths of light; what we perceive is adjusted for colour constancy according to expectations based on past experience, so we might perceive the same wavelengths differently depending on the context. We only agree on the colour of a stoplight because the frequencies of its light are similar to the frequencies of light from other things that we call red.
There's neither wavelength nor colour in the brain, just neurons firing in patterns. It's been shown that the output of a small groups of cells in the visual cortex (the Hurvich–Jameson (H–J) opponent-process network) determines the perceived colour, and it's possible to manipulate the visual input to these cells so as to produce outputs that are outside the normal/natural range of outputs, and so enable the perception of colours that you wouldn't otherwise see (
Chimerical Colors: Some Phenomenological Predictions from Cognitive Neuroscience by Paul Churchland - I have a pre-paywall copy of the PDF if you're interested).
It's only red because that's what we call the colour of things that match it visually. People with various forms of colour-blindness, and tetrachromats, will see a greater or smaller range of colours than normal, so won't always even agree on what matches and what doesn't...