Remove colour 'shades'/shadows - c++

Question

What OpenCV functions can be used to ignore/filter out colour 'variation'/shades of a colour (shadows, reflections, etc.)?
Shouldn't removing the Value/Intensity channel from HSV images create colour blocks and reduce/eliminate 'colour variance'/shades of white due to light?
If you look at the following image, the *walls are painted one solid consistent colour of cream/white. But it's colour varies alot because of shadows and light reflections. *Referring to the white walls above the lockers.
I thought that if I convert the image to HSV then remove the Value/Intensity channel that I can filter out those wall reflections and shadows, colour variation - ie, the light. Then I just colour reduce the image and I should have a large colour block for the wall (above the lockers)? Ie, see the wall in it's true form/colour as one solid colour block.
But from my image above you can see the wall is not one solid/consistent colour after removing the V channel and after colour reducing.
void removeIntensity()
{
Mat hsv, hs, reducedHs;
Mat image = imread("../../Book_Tutorials/images/11.jpg");
if (image.cols > 300) {
float scale = 300.0 / (float)image.rows;
resize(image, image, { int(scale * image.cols), 300 });
}
cvtColor(image, hsv, CV_BGR2HSV);
std::vector<Mat> hsvChannels;
split(hsv, hsvChannels);
// Set value/intensity to constant value: can I remove those channels completely?
hsvChannels[2] = 0;
merge(hsvChannels, hs);
reduce(hs, reducedHs, 6);
imshow("image", image);
imshow("hsv", hsv);
imshow("hs", hs);
imshow("reducedHs", reducedHs);
}
void reduce(const Mat& hsv, Mat& reduced, int nColours)
{
int n = hsv.rows * hsv.cols;
std::vector<int> labels;
Mat centres;
Mat collapsedImage = hsv.reshape(1, n);
collapsedImage.convertTo(collapsedImage, CV_32F);
kmeans(collapsedImage, nColours, labels, TermCriteria(TermCriteria::EPS + TermCriteria::COUNT, 10, 1.0),
3, KMEANS_PP_CENTERS, centres);
for (int i = 0; i < n; i++) {
collapsedImage.at<float>(i, 0) = centres.at<float>(labels[i], 0);
collapsedImage.at<float>(i, 1) = centres.at<float>(labels[i], 1);
collapsedImage.at<float>(i, 2) = centres.at<float>(labels[i], 2);
}
reduced = collapsedImage.reshape(3, hsv.rows);
reduced.convertTo(reduced, CV_8U);
}