I just came back from Spain where being topless on the beach is completely normalised. And while being topless myself not once I saw anyone staring at me or having an inappropriate reaction.
Just because some men will get an erection from seeing breasts, that doesn't make them inherently sexual. In most situations, they are just body parts. And in countries where people don't automatically assume nudity or semi-nudity equals sex that's the norm.
I do agree though that dress codes are tricky and it's not easy to draw lines on what is and isn't acceptable or when. But this shouldn't stop people from having a conversation around them.