So, the problem I have with this thinking comes from two wrongs don't make a right.
If you allow the government to make decisions based on skin color, it is still racism.
Yes, my parents owned a house for a while. There was no wealth I got out of that. They could have easily been renting and my position in life would have been the same.
Just like any one else, if it can be proven that the government discriminated against an individual, then by all means that individual should be compensated.
If any laws prevent fair housing or it can be shown that an individual has received prejudicial at the hands of a business/mortgage lender then they should have a remedy for that wrong regardless of the color of their skin.
If any law can be shown to be racist in nature, then by all means, lets all get behind rectifying that.
However if you are going to justifying the government discriminating against people because of their skin color, you are no different than those people who justified discrimination in the past.