I'm Israeli-American, but I have no Hebrew accent in my English speech, and my Hebrew is not quite fluent. I was in Israel a few years back, taking a bus from Tel Aviv to Jerusalem. I was wearing a sleeveless t-shirt and shorts, because it was very hot. Yes, this is relevant.
I wanted to be sure I was taking the right bus. Since the person in front of me in line asked the driver a question in English and the driver responded fluently, I also asked my question in English.
I got on the bus, two seats behind the driver. The guy in the seat in front of me started talking to the driver about me, more specifically about which body parts he liked - my legs, my butt - and which ones he would prefer augmented (guess which).
I let the jerk go on and on. The driver just listened, occasionally responding to his friend with non-committal grunts or head movements - in Israel, the roads are insane, and the drivers really need to focus on the driving. When it was time for me to leave the bus, I casually said, in Hebrew, as I passed the jerk in the seat ahead of mine, "It's not polite to talk about people in a language that you think they don't speak - and it makes you look silly when they understand you."
The driver and the other passengers in earshot started howling with laughter. The jerk turned beet red and muttered something under his breath. I just smiled and walked off.
But damn, I do like the "Habla Espanol, bitch" line.