I've started treating "How are you?" or "How's it going?" as a greeting in that context, and just responding with another greeting, like, "Hi! Lovely weather we're having!" or something similarly irrelevant and disengaged.
People probably assume I didn't hear them or something, but not once has anyone reiterated the question.
My personal path was first learning to lie about it - then later on understanding that nobody was lying about it per se, but they might appreciate the offer of concern without actually wanting to take advantage of it. Or they might just not want to confide in a casual acquaintance.
I think the tricky and important part is actually navigating levels of intimacy, e.g. by phrasing questions in a way that doesn't ask to cross too many boundaries all at once. Conversation is like a dance, it goes better when everyone is giving and taking feedback on where it goes. In this sense "how are you" is just an invitation to dance. Don't be surprised when most people don't want to start dancing to a mournful tune!
People probably assume I didn't hear them or something, but not once has anyone reiterated the question.