What about pseudohermaphrodites? I used to know a girl who by all characteristics and external appearances was feminine, had a vagina, etc. but when she was a teenager she and her family discovered that she was actually Xy when she developed cancer in gonadal tissue. She continued to identify and live as a female.
I actually consider hermaphrodites a legitimate third sex. It was something she was born with, not something that was sewn or stitched on later.
There's no question there are effeminate males and butch females. And I do agree that there are inclinations that can go either way in terms of sexuality.
I've always felt you deal with what nature gave you, and while there's nothing wrong with trying to improve your appearances or be something different then what you are genetically, there is no question that some people can accomplish it more than others.
I've known a few very passable transvestites and transsexuals, even attracted to, but unfortunately that's not the norm for a greater majority of people.
On a positive note, I think this could change over time with the advances in science and technology to where in a future generation it won't really matter anymore and people can actually be comfortable with themselves and others.
I just don't think we're quite there just yet.