Sounds like it's not really a difficulty selector since it doesn't affect the combat difficulty of the game itself. It's just there to give a good message on the current state of affairs. Unless i'm missing something and skin color will affect other aspects of the game but I can't think of what those aspects would be outside something like less money to buy equipment. It's been awhile since I played SoT so I kinda forgot how the game systems work.
I believe it reduces to amount of in-game cash you get from encounters, as well as changes dialogue options with NPCs