As an anthropologist who has spent decades doing research on and with women in different communities in the Middle East, I have found myself increasingly troubled by our obsession with Muslim women.

Ever since 2001, when defending the rights of Muslim women was offered as a rationale for military intervention in Afghanistan, I have been trying to reconcile what I know from experience about individual women’s lives, and what I know as a student of the history of women and of feminism in different parts of the Muslim world, with the stock images of Muslim women that bombard us here in the West.

Over the past decade, from the girls and women like Nujood Ali, whose best-selling memoir I Am Nujood, Age 10 and Divorced was co-written, like so many of the others, by a Western journalist, to Malala Yousafzai, they have been portrayed as victims of the veil, forced marriage, honor crimes or violent abuse. They are presented as having a deficit of rights because of Islam.

But they don’t always behave the way we expect them to, nor should they.

Take the veil, for example.

We were surprised when many women in Afghanistan didn’t take them off after being “liberated,” seeing as they had become such symbols of oppression in the West.

We were confusing veiling with a lack of agency. What most of us didn’t know is that 30 years ago the anthropologist Hanna Papanek described the burqa as “portable seclusion” and noted that many women saw it as a liberating invention because it enabled them to move out of segregated living spaces, while still observing the requirements of separating and protecting women from unrelated men.

People all over the globe, including Americans, wear the appropriate form of dress for their socially shared standards, religious beliefs and moral ideals. If we think that U.S. women live in a world of choice regarding clothing, we need to look no further than our own codes of dress and the often constricting tyrannies of fashion.

As for Malala, she was subjected to horrible violence by the Taliban, but education for girls and Islam are not at odds, as was suggested when atheist Sam Harris praised Malala for standing up to the “misogyny of traditional Islam.”

Across the Muslim world girls have even been going to state schools for generations.

In Pakistan, poverty and political instability undermine girls’ schooling, but also that of boys. Yet in urban areas, girls finish high school at rates close to those of young men, and they are only fractionally less likely to pursue higher education.

In many Arab countries, and in Iran, more women are in university than men.

In Egypt, women make up a bigger percentage of engineering and medical faculties than women do in the US.

A language of rights cannot really capture the complications of lives actually lived.

If we were to consider the quandaries of a young woman in rural Egypt as she tries to make choices about who to marry or how she will make a good life for her children in trying circumstances, perhaps we would realize that we all work within constraints.

It does not do justice to anyone to view her life only in terms of rights or that loaded term, freedom. These are not the terms in which we understand our own lives, born into families we did not choose, finding our way into what might fulfill us in life, constrained by failing economies, subject to the consumer capitalism, and making moral mistakes we must live with.

There is no doubt that Western notions of human rights can be credited for the hope for a better world for all women. But I suspect that the deep moral conviction people feel about the rightness of saving the women of that timeless homogeneous mythical place called Islamland is fed by something else that cannot be separated from our current geopolitical relations.

Blinded to the diversity of Muslim women’s lives, we tend to see our own situation too comfortably.

Representing Muslim women as abused makes us forget the violence and oppression in our own midst. Our stereotyping of Muslim women also distracts us from the thornier problem that our own policies and actions in the world help create the (sometimes harsh) conditions in which distant others live.

Ultimately, saving Muslim women allows us to ignore the complex entanglements in which we are all implicated and creates a polarization that places feminism only on the side of the West.

MORE: Saudi Cleric Says Driving Hurts Women’s Ovaries

(MORE: Forbidden to Drive: A Saudi Woman on Life Inside the Kingdom)

(MORE: Brides Before Bombs: Nigerian City Fights Terrorism With Mass Weddings)