In a fascist dictatorship, they have a lot more in common than opposition.
But if the dictatorship is a communist one they have more in common with the nazis! Or if your country is invaded by Russia you might find yourself fighting side by side with the Azov battalion.
There are libertarians who genuinely care about free speech and might make useful allies on those issues.
Just because someone is the enemy of your enemy, or an occasionally useful ally, doesn’t mean you want to unify with them.
I was considering putting scare quotes around “communism”, but refrained in order to avoid an argument about what is and isn’t really communism. Yet here we are. So much for left unity! ;D