Zip code can absolutely be a proxy for race.

There’s a reason that banks have compliance departments that have to review any statistical models inputs and assumptions for implicit violations of the Fair Lending / Fair Housing acts.