You have the context: they are still showing their support for a fascist state, and this is worthy of harassment, as a matter of community defense.

Would you defend a woman wearing her grandpa’s swastika pin, for any reason, or is that enough context, in that case?

We, as a society, are swinging back to making displays of fascism unacceptable. That’s all the needed context.