They also say, 'Liberty', 'In God We Trust', & 'E Pluribus Unum'. Lots of writing on government-issued materials is false, ceremonial, outdated, or overridden by later acts.
If it really bothers you, you can file that off your pennies before spending them at 5¢ value. We won't mind.
The $2 bills are currency. He recombines them to create funny & thought-provoking situations, including encounters with local police & Secret Service, that test the boundaries of what's allowed & what's trusted.
Unless you are committing fraud, like falsely trying to claim it has a value other than what it actually does, defacing currency isn't illegal. So trying to pass off a penny as being worth 5 cents today by filing it down would violate that law, but if they really were worth 5 cents, it would be fine.
But they say "ONE CENT" on them.