"So basically sex is binary, but we're not so desperate that we can't recognize that the non-binary cases exist."

Totally agree. For example, look how many people today are mixed race. Because there's no "Thai-AfricanAmerican" box on an EEOC form. that doesn't mean Tiger Woods doesn't exist, eh?