I am sorry for just dropping a bunch of text (feels like a smarmy thing to do). I used to hear that often. It drives me nuts. It does not mean what one thinks it means or what one wants it to mean.

“[S]aying you are colorblind is not helping anyone— it is not helping you; it is not helping me. First, you do see color… yeah, you really do… yeaaaaah, you really, really do.
More to the point, “I don’t see color” has always been such an odd olive branch of “relatability.” Odd because it is untrue. Odd because of its dissonance — its actual meaning is the opposite of its intended meaning. Odd because those tropists seem to miss, in their private teachable moments, the huge reveal “colorblind” brings with it.
Oh, I understand its purpose. Believe me, we all do. We are just being nice when we let you explain it to us for the fifty-eleventh time this week. At the risk of landing on the wrong side of humble, I am not, shall we say, untutored. I do not strike even the most casual of observers as lacking emotional literacy. But even if I were both dumb and tone deaf, this isn’t the utility of the Superconducting Super Collider we’re contemplating here.
The speaker of the phrase “I don’t see color” aims for “I respect you, I accept you as a person and I do so with no judgment.” The speaker lands on “I do not consider your race so that I do not judge you.” The speaker drives right past “Seeing your race triggers me to say I do not see your race, which I must erase so as not to judge you.”
I know that you employ this phrase because you fear that we judge you and find you lacking. Okay, we judge you — good call. So, here’s a pro tip: If you center around you a thing you are doing for me, you’re doing it wrong. If this is just for me, you need only convey, in word or deed, “I respect you” and leave it there. You don’t, of course. You keep speaking, yet you do so without knowing you are talking to yourself about yourself for yourself. You don’t really need us for your pep talk, darlin. For this, there is definitely no “we.” In fact, for this the “we” is all you and none of me.”