The comparison is fine when it's appropriate, but it shouldn't be misused.
When I first saw the Blues Brothers, I thought the screenwriters were being lazy by making the villains "Illinois Nazis." I thought that the idea of Nazis in Illinois was outlandish and ridiculous, and I thought they were just thoughtlessly latching onto characters that the audience would recognize as the villains without having to do much to establish them that way.
... but then later on, I discovered that Illinois Nazis in the 70s were a real thing:
National Socialist Party of America v. Village of Skokie - Wikipedia
Suddenly, I realized that what I had originally taken as a tackish screenwriter trope was actually a pretty on-point response to very current events and issues.
Short version: make sure you know why someone is using Hitler or Nazi analogies before you criticize them for doing it.
I think in The Blues Brothers, they were using it for comedic effect more than anything else.
Although somewhat unrelated, I have seen those complain about using the Nazis as a source of comedy, such as those who complain about Hogan's Heroes. Many thought it was in poor taste and considered it offensive.