It's inevitable that the obvious difference will be mentioned when decribing someone. I get tired of white hand-wringing over this; I couldn't begin to count the number of times I've overheard someone with slightly more melanin in their skin mention that so-and-so was "white" as a way of sort of setting the scene, or narrowing down the identity of someone who's name they've forgotten, etc. This is generally done with no malice or contempt whatsoever, and I'm not offended by it as a "white" man. Pussyfooting around it is just a pretense of righteousness, and in my mind it's a form of racism. You can mention someone's age, gender, height, weight, wealth, clothing, etc., but God forbid you describe their race...?!
We're all brown, some more than others... there are historical and cultural complexities to it,and a few secondary physical characteristics, but it's really just a matter of melanin. I don't think I'll ever fully understand what all the fuss is about, but I guess many people are still driven by some ancient tribal urge to mistrust the "others"...