I’ve always considered myself more of a libertarian than a straight-up conservative, which is why I’m surprised to find myself at odds with recent libertarian writers covering the topic of feminism, and more specifically sex difference.

In Reason, Sharon Presley critiques Christina Hoff Sommers’ book Freedom Feminism, focusing most of her criticism on the idea of innate sex differences. Presley writes:

But the bigger problem is “maternal feminism,” by which she means a belief that women and men are “different but equal.” This is essentially a code for claiming that various stereotypes about women are true: that we are more caring and more nurturing, that most of us would stay at home and be moms if we could, that many of us don’t really care much about careers, that we would be happier accepting our “differences.” Sommers writes, for example, that “the paradox of egalitarian feminism” is that “when women are liberated from the domestic sphere…and no longer sequestered in the role of nurturer, many, perhaps most, persist in giving priority to the domestic sphere.” What kind of “paradox” is this? Of course most women (and men) want families, but that doesn’t mean they can’t want careers too.

Sommers does not provide evidence that “many, perhaps most” women feel this way. A 2012 Gallup poll showed that 63 percent of women, including those with children at home, work outside the home. And while some of those might prefer to stay at home were it not for economic necessity, work outside the home is more common among women with college or graduate degrees and women with higher incomes. Seventy-five percent of women with a college degree have jobs. For women with postgraduate education, the number is 84 percent. Low-income mothers are far less likely to be employed than are upper-income mothers (45 percent vs. 77 percent). Does she seriously think it possible that “most” women do not want careers? That most prefer “domestic life” to careers, as if they were either/or? She even suggests that most women prefer what she calls “pink collar” jobs, such as fashion design and nursing, even though women are now around 47.3 percent of students in medical schools, 47.2 percent in law schools, and the majority (58 percent) in college.

I’ve had a similar debate recently with Cathy Reisenwitz. First, I think that everyone ought to acknowledge that no one believes there is an “either or” choice when it comes to career or family life. Christina Hoff Sommers doesn’t feel this way–she is after all, both a successful academic and a mom—and I don’t either. The question is whether men and women have the same preference for how much time to dedicate to career relative to family and other pursuits.

Presley uses as evidence most women want careers by showing that most women, including most moms, are currently working, and that more educated moms are more likely to work.  She asknowledges some of these moms may work out of necessity, but doesn’t take that any further.  Yet there actually is data on the work arrangements that mothers would prefer: This Pew study found 37 percent of working mothers prefer full-time work arrangements. Note that half preferred part-time work, while just one in ten didn’t want to work at all. That’s an important point—most women want to work and want careers, but most ideally would work less than they do now so that they could spend more time with their families. This likely also holds a key to why educated women are more likely to be in the labor force than less educated moms: Professional jobs, beyond being more interesting and personally fulfilling, lend themselves to more flexible work arrangements that allow women to better balance their desires for a fulfilling career and a family life.

At the same time, fathers had a much stronger stated preference for full-time work: “Fully 75% of fathers with children under age 18 say working full time is ideal for them. Some 15% say working part time would be ideal, and 10% say they would prefer not to work at all.”  Certainly social expectations and realities play a role in this. Men undoubtedly feel more pressure than do women to earn more, and it is more socially awkward for them to take time out from work. There are a growing number of stay-at-home Dads, but they remain a minority and therefore have less of access than stay-at-home moms to social support.

Yet it seems extreme to presume that all of women’s stated preferences for spending time with their children is the result of societal brainwashing, rather than at least in part attributable to a different genetic makeup. Yet Presley goes pretty far in dismissing the idea of sex-based differences rooted in biology:

Nor does she back up her claims about the “differences” between women and men. In fact, the consensus among most serious scientists who do research—neuroscientistLise Eliot, psychologist Janet Hyde, neuropsychologist Melissa Hines, and biologist Anne Fausto-Sterling, for example—is that the behavioral and cognitive differences between women and men are not nearly as great as the average person (or Sommers) imagines. There is almost certainly a small genetic component, but it is less overall than the contributions of multitudes of cultural, family, and individual environmental influences. Anthropological research on sex bears this out; scholars such as Peggy Reeves Sanday have shown there is more variation in gender roles than Sommers apparently assumes. From a feminist point of view—and from an individualist one—Sommers’ stereotyping is unacceptable. We should be looking at the merits and choices of each person as an individual, not as a member of a particular biological group.

A consensus among “serious” scientists? This is the kind of sloppy, demagoguery I would hope “serious” libertarian thinkers would reject. Just because someone’s ideas differ than one’s own doesn’t mean they aren’t “serious.” Louann Brizendine, Leonard Sax, Steven Rhoads, and many others who write on sex differences are serious academics and their work shouldn’t be dismissed.

There is a rich and growing field of study on the role biology has in determining our traits. Books like this book by economist Bryan Caplan provides fascinating evidence (mostly from twin studies) that we are much more nature than nurture. His book doesn’t focus on sex-based differences—rather on why parents shouldn’t put so much pressure on themselves—but does suggest biology is a powerful force. Such work should be encouraged and we should be ready to review whatever evidence emerges.

The unsettling accusation that perceiving sex difference is “stereotyping”—which hints at plain old “sexism”—is meant to discourage thoughtful discussion.  But it isn’t stereotyping to admit women have a different biology that may impact our perspectives and preferences.  Women carry babies and can nurse them, releasing different hormones that make being with babies pleasuable; we have different chemicals that color our perception of the world from the time we are born.  It would be surprising indeed if these realities had no impact, but I’m interested to see more evidence to that end if it really emerges.

Some women seem to fear acknowledging innate sex differences would be to cast women as lesser.  But that really seems unfounded in today’s world.  No one—certainly not Christina Hoff Sommers or any other writer I know of—thinks that sex is determinative. It doesn’t mean a woman can’t be just as good at math or fulfill any job just as well as a man can. And yes, we should work to make sure our society isn’t sending messages that some activities are inherently “female” or “male,” so that our sons and daughters feel limited in what they should try. But we also shouldn’t presume that something is wrong if women and men, on average, exhibit different preferences for how to spend their time.

Presley seems to presume the social pressure on women only works one way—pushing us to quit our jobs and have babies. But some women certainly feel as though they were mislead in the other direction.

That’s why conversations about this are important. Women shouldn’t be told that having babies is their only option, but they also shouldn’t be pushed to believe their desire to do so is the result of the patriarchy. We want women to make informed choices, and should avoid dismissing women who make different choices—or who have different perspectives—as undeserving of the label “feminist.”