So really you're betting on what "allsides.com" thinks, not if an article is actually right or left leaning (since there is no objective measure of right/left leaning).
Somewhat. Allsides has a whole system[1] as to how they assign bias which is fairly robust, in my opinion. You're not betting on what the moderator at allsides think a source bias is, you're betting on what all of the patrons at allsides think the source bias is which is a little better.
But you are right in that no one can "objectively" measure partisanship since there is no such thing.
