I tend to agree with you, though I've heard it come up in the past, where criticism of Israel has been equated with anti-Semitism, at least in US political discourse. Although it seems it was more prevalent in the past than in more recent years. I recall when actress Vanessa Redgrave made some remarks about Israel at the Academy Awards during the late 1970s, and she was criticized as being anti-Semitic.
Speaking as a Brit who has lived in the US and travelled in the Middle East a fair bit, I must I say I encounter this confusion fairly regularly among Americans. There is in the US a powerful Israel lobby, which does its best to mix the two things up as much as possible. Walt and Mearsheimer wrote a book about it, which I have read and which rang very true to me. (The most amusing thing about that book was that one of the authors got told (1) there was no such thing as an "Israel lobby" and (2) if he continued to write about it, they would make sure he never got another academic post!
)
The trouble is that some of these lobby organisations claim, quite wrongly, to speak on behalf of all Jewish people, which, if one is not careful, actually
causes antisemitism, because people can be tricked into thinking that's true.
We get the same sort of phenomenon in the UK, with muslim organisations that claim to speak for British muslims, but have been taken over by political islam, which is actually just a political project and not part of the religion at all.