Is that not a fair thing to do?

Can we not assume someone with access to a Facebook account is authentically the owner of that Facebook account for all of our intents and purposes?

As it is right now, yes. But imagine a scenario where Facebook might implement a child account where a parent has access to monitor the usage.

Now there are two people with authorisation to access this Facebook account so the process no longer unique authenticates a single individual.

Of course this is a contrived example and I'm sure there are better examples. But this is why OAuth is authorisation and not authentication and why something like OpenID Connect exists on top of OAuth2.

Authentication is to prove who you are - authorization is to just have permission to do something, as a subset of authenticated rights.

You would generally be much less rigorous in the Facebook example between giving someone access to your shared photo albums than to your account settings. Having an oauth token does not make you signed into Facebook at all, but just says that you have valid rights to do something.