There's cultural implications beyond basic racism in this situation, as well. It would have been different if he were, for example, the CEO of a company that manufactures basketballs and sells them to the players. But he is not in a business relationship with these players, he is the
owner of the team, with all the
possessive baggage that implies.
Plus, the woman who recorded him, his sometime-girlfriend, is mixed race herself. He doesn't like black people, he just likes screwing them. Part of the transcript had him telling her that she was welcome to bring her friend Magic Johnson to the house, "entertain him, feed him,
fuck him--just don't bring him to the games." Another layer of cultural baggage, that slaves were so often used for sex.
It's like when a homophobe is outed as being gay--the outrage is greater because it's not just about the discrimination, it's about the hypocrisy. Americans will respect a racist's right to free speech, but not a hypocritical racist.