Usual vertex normals:
Modified vertex normals:
What do you think what is the best way to convert colour to normals? Currently, I check if colour is black or white and if it is normals remain untouched. If it is not RGB is converted to XYZ so that 0.5 if subtracted from each channel and normalised. For example if you want Z up normals you need to paint vertices with R = 0.5, G = 0.5, B = 1