In my Pagan days, I only had issue once.
I worked in a group home, and one day one of the other staff asked me about Paganism and what it is. A nurse that had never cared for me overheard and interrupted, stating "It means she sacrifices chickens on her altar." Very calmly, I told him he was being ridiculous. He had been working with me enough years to know I was vegetarian, and should by now realize I sacrifice veggie burgers on my altar.
He didn't see it coming, and laughed and laughed, and contemplated what a vegetarian sacrifice would look like...
We got along fine from that point on.