peeling_oranges offers some first-hand knowledge:
As a gay man, I consider myself something of an authority on the asshole -- and your doctor is a schmuck. If too much anal sex had that kind of effect, by this point in my life I'd be able to turn myself inside out like a Popple***. My sphincter muscle is actually MUCH stronger (and therefore tighter) than it was years ago.
This is amusing on it's own, but it's the footnote that takes the cake
***If you don't know what a Popple is, then you're too young to be reading about ass-stretching, so go away.
EDIT: For those unfamiliar with Popples. They were small plush dolls that could be turned inside-out, putting the doll inside a pouch, basically.