Image credit: Sodahead.com

Years ago when I was living in Boulder, Colorado, I volunteered as a legal advocate with the Safehouse Progressive Alliance for Nonviolence (SPAN) and learned about catastrophic cases of domestic violence. As part of my training to become a legal advocate, I became familiar with the federal and state-specific legislation covering domestic violence. Colorado is one of 22 states that apply mandatory or preferred arrest in cases of domestic violence. When they enacted the mandatory arrest law in 1994, legislation fostered by the federal Violence Against Women Act (VAWA), policymakers had the best of intentions to protect former and potential victims from repeated intimate partner violence.

Mandatory arrest laws were enacted to protect women from experiences like that of Tracey Thurman, a Connecticut woman who was left for dead in the driveway by her husband who had beaten and stabbed her multiple times in 1984. Against all odds, Tracey not only survived her husband’s attack, but successfully sued the City of Torrington, Connecticut for failing to protect her after she had repeatedly reported her husband’s violent episodes to authorities. Unsurprisingly, after Thurman’s case, domestic violence advocates nationwide pushed legislators to pass mandatory and pro-arrest laws that transfer the decision to press charges from women to law enforcement officers.

However, empirical evidence in the last decade has caused some researchers and practitioners to raise their eyebrows. Before mandatory arrest laws became popular, police had full discretion in calling the shots with regard to domestic violence incidence. The implementation of the mandatory arrest laws in the 1990s removed law enforcement’s discretion, directing police officers to arrest the identified perpetrator or in some cases, perpetrators. But, has the legal pendulum swung so far that it is time to reevaluate the statutes? Are mandatory arrest laws effective or, by implementing them, do we create more harm for victims because police are ineffectively trained to enforce the laws and knowing this, victims become too scared to call the police when they have experienced domestic violence?

Former Harvard researcher Radha Iyengar tracked homicide rates in states with mandatory arrest laws compared to states without such laws. Iyengar’s findings revealed that in the 22 states that mandate arrest, homicides have increased by 50%, while in states that employ police discretion, homicides have actually declined. Iyengar attributes the increase in homicide rates to the reluctance by victims of abuse to put their partners behind bars. The fear of seeing their partners go to jail has in fact led victims to contact police less frequently than in previous years. Iyengar notes:

The mandatory arrest laws were intended to impose a cost on abusers. But because of psychological, emotional, and financial ties that often keep victims loyal to their abusers, the cost of arrest is easily transferred from abusers to victims. Victims want protection, but they do not always want to see their partners be put behind bars.

Victims of intimate partner violence want their voices to be heard. But it appears that mandatory arrest laws, however well-intentioned, may instead be silencing victims and, more distressing still, triggering retaliatory violence.

One of the most significant consequences of mandatory arrest laws has been the subsequent rise of dual arrests when police respond to a domestic violence call. A 2008 publication produced by the National Institute of Justice commented that not only were “arrest rates in intimate partner cases 97% higher in states with mandatory arrest laws, compared to states with discretionary arrest laws,” but also, “mandatory, but not preferred, arrest laws increased the likelihood that police would arrest both parties”. Similarly, Iyengar’s research finds that victims avoid calling the police out of fear that they will be arrested for acting in self-defense. She also notes the possibility of dual arrest most concerns victims with children at home.

The National Institute of Justice report revealed that dual arrest is more likely in domestic violence cases where the victim is unclear. This was particularly true in incidents involving simple rather than aggravated assault. In aggravated assault situations, the increased physical violence and evidence help police more clearly distinguish between perpetrators and victims. In same-sex cases, dual arrest rates were ten times the rate observed in incidents between female victims and male offenders. Police officers in mandatory arrest law states may not be properly trained to address same-sex domestic violence, leading to an increase in dual arrests.

Research in recent decades supports greater discretion in police intervention in domestic violence cases. The enactment of mandatory and preferred arrest laws in the 1980s and 1990s was meant to provide increased support for victims. Both decades witnessed a rise in the number of fatal domestic violence incidents, prompting legislators and victim advocates to utilize these tragic cases as a platform to promote punitive reform in domestic violence laws. There is little doubt that these laws were developed with the victim in mind. However, contemporary research questions the capacity of mandatory arrest laws to provide victims with the protection that legislators and victim advocates had expected.

Mandatory arrest laws may have once seemed necessary to draw attention to the growing domestic violence problem in the United States. However, in recent years, it has become clear that these laws have failed not only to provide victims with greater protection but led to more victim arrests by police unable to make discretionary decisions based on probable cause. As it stands, states with mandatory arrest laws may not be appropriately equipped to deal with domestic violence until they understand the intricacies of the legislation and work to promote victims’ rights and eliminate gender inequality embedded within law enforcement’s patriarchal nature. Until then, the unintended consequences of the laws may be outweighing the anticipated benefits. The available research offers evidence to promote a middle ground approach between “discretionary” and “mandatory” arrest.

One recent afternoon, a group of DePaul researchers—students, staff, and faculty—met in the SSRC conference room for the second session in our Research Ethics Workshop Series for lunch and discussion about what is and isn’t acceptable research practice, where grey areas exist, and what fundamental structures of academic research might actually contribute to poor or even fraudulent research.

That conversation, aided by the slides above, ranged from the practical to the principled. I’ll try to recapture that conversation here.

The presentation began with the case of Dutch social psychologist Diederik Stapel, who was recently found guilty of extensively fabricating data over many years in dozens of studies, at least 30 articles, and up to 14 dissertations. The universities Stapel had been affiliated with acted swiftly and thoroughly, publishing a preliminary report that outlined his methods, the effects, the academic environment that had contributed to his behavior, and issuing some recommendations. This fascinating report raises points about academic fraud and deception revealed through the tale of a charismatic swindler who abused the trust, admiration, and ambition of his colleagues and students to spin research gold from invented data, apparently in the name of selfless scientific progress.

The slides also illustrate the story of evolutionary biologist Marc Hauser, whose research assistants discovered coding inconsistencies that led to his conviction on eight counts of academic misconduct by his home institution (Harvard, no less). Though less extensive than Stapel’s fraud, Hauser ended up leaving his position at Harvard (after the faculty there voted overwhelmingly to bar him from teaching). The exact nature of his misconduct is a mystery since Harvard (hoping to protect its reputation?) has not released any information.

The workshop ended with a quiz-style round of questions related to a recent study of psychologists about their research practices. We debated the prevalence and defensibility of potentially dubious ethical practices and compared our conclusions with the study’s findings. Some discrepancies took us by surprise; others seemed the result of an over-generous interpretation of the survey questions.

A recurring theme in our conversation was the primacy transparency should have in every aspect of the research process, from collaborating with colleagues, to study design and methodology, to publishing findings. Making data and detailed information about research methodology available after publication would facilitate not only replication but peer review. We agreed that even in cases of academic misconduct, transparency as practiced by the Dutch universities in Stapel’s case was preferable to Harvard’s secrecy about what its transgressor had done—if only to address and ward against future misconduct. Transparency becomes especially important when students and faculty or others in relationships involving a power differential are working together on an ongoing basis. From the start, parties must be explicit about their expectations concerning workload, authorship, and data ownership.

Support for graduate students became a key theme in our own exploration of fraud. Academic training in a PhD program is more than scholastic education; it’s also professional training, which should offer students hands-on opportunities to develop solid (and ethical) research practices. The Dutch committees found that Stapel’s students were isolated from both each other and other faculty—unaware that Stapel’s practice of “collecting” data without their participation was unusual. Stapel abused their trust and robbed them of the opportunity to learn research skills through practice. The investigating committees emphasized the importance of having a neutral party within institutions to whom students can report unethical research practices or consult about what is acceptable.

There are many forms of “currency” in academia, not the least of which is data, earned through rigorous research practice. Stapel ran a tidy counterfeiting operation for years, trading his “data” (which he himself was said to call “gold”) for status, reputation, and international influence. Many people writing about the Stapel case have lamented the data- hoarding that too often characterizes social psychology research, and other fields. How available should data be at each stage of research? Numerous organizations are moving towards open access to data (after publication).

Publications are another form of research currency (this one needed for tenure). Authorship can be slippery to define in collaborative research.  We talked about the ethical grey areas where a mutually beneficial relationship may exist between advisers (especially those with some prestige in their field) and students whose work would go unpublished or unnoticed without the adviser’s prominent authorship. Professional societies include ethical guidelines about authorship. The American Sociological Association, for instance, specifically states that students should have lead authorship on works substantially derived from their dissertation work, a practice our participants said is not always followed. Students may feel that they benefit when their work is “brought to light” by the authorship of a more prominent researcher, but is it ethical? Is it fair?

Further reading:

Retraction Watch As the name suggests, this site tracks retractions throughout scientific scholarship.

John, L., Loewenstein, G.F. and Prelec, D. (January 31, 2012). Measuring the Prevalence of Questionable Research Practices with Incentives for Truth-Telling.

Garner, R. (2012). [Review of the book Plastic Fantastic: How the Biggest Fraud in Physics Shook the Scientific World]. Science & Society: Vol. 76, No. 1, pp. 115-135. doi: 10.1521/siso.2012.76.1.115 Available from Guilford Journals.

Friday, February 10, 1 pm: Faculty Seminar
“Development and Evaluation of a Sexual Identity Measure for the National Health Interview Survey (NHIS)”
Miller will describe the use of qualitative research in developing a precise sexual identity measure for a large-scale quantitative survey and the resulting complications.

Monday, February 13, daytime: Lab Visits
Faculty are invited to schedule appointments to meet with Miller to discuss their research, questionnaire design, or other research questions. 

Monday, February 13, 6 – 7:30 pm: Public Lecture
“Question Evaluation at the National Center for Health Statistics”
This lecture, open to the public, will center on Miller’s work at the CDC and will consider examples of questions that inadvertently compromised data quality through a lack of rigorous evaluation.

I talked with Kristen today to learn more about what she does and why it matters.

What do you do at the Question Design Research Laboratory?  What does an ordinary day look like?

KM: We’re the National Center for Health Statistics, we’re the main data collectors for the Department of Health and Human Services. That’s what our agency does, and it’s through surveys. My staff, the QDRL, is a consulting piece of NCHS.

Basically what we do is we evaluate questions and we develop methodologies to be able to evaluate questions. Specifically, when I talk about evaluating questions, I’m talking about validity, and are the questions really capturing what analysts believe that they are capturing.

What are some of the methods that you employ to evaluate any particular question?

KM: The main thing we do here, and the main thing that the Federal Statistical System uses, is called cognitive interviewing. First you ask them the survey question, they give their answer, then you interview them to figure out what they were thinking and why they answered the way they did.

From that then, you do your analysis and you can figure out specifically what the constructs are that the questions are capturing.

So it’s an interesting combination of using qualitative methods to improve a quantitative methodology.

KM: Absolutely, and that’s very much how we see it. At this point, if you go into the literature and you look at cognitive interviewing methods, you’ll find that the literature is rather [sparse]. What we’re working on is applying more of a sociological perspective, and then pulling in the qualitative literature. So we very much talk about Glaser and Strauss, doing grounded theory, and that sort of thing.

When you are looking at any particular survey, what are the 3 most frequent critiques that you at the QDRL have for survey design? What do you suggest to people most often to improve questions?

KM: I really can’t answer that question. Where we’re coming from is that no one can write a survey question. No one knows how it’s going to be taken. Pulling in our sociological background, people from different social locations can really interpret questions differently. If you understand it that way, one single person can’t sit there and write a question and know how everybody’s going to understand it. So what we believe is that the only way to realy do that is to perform these qualitative studies and really find out how people from all of these different social locations go about understanding questions. Our main issue is that you can’t just sit there and critique a question and change a question. You need to actually study how it performs. Only after you study how it performs, is when you can start tinkering with questions.

Because we do population surveys, we started thinking about all the different types of minority groups. I think that the biggest group that we have problems with are low-socioeconomic-status (SES) people. These are the people who really have difficulty understanding a question as question designers intended it. It makes sense, right?  Because there are people with Ph.D.’s sitting in Washington writing these questions with specific intentions—”Do you have coronary heart disease?” Well, people who don’t have good health care, they’re not going to know what that is. We’ve done studies in rural Mississippi and people will say, “No, I don’t have that,” and you’ll look at the guy’s health record and you’ll see that not only does he have coronary heart disease but he’s got congestive heart failure. I’m really concerned that the federal statistical system is not getting equal quality data, and that’s one of my most passionate subjects, and I try to shed light on that and to improve it.

What specifically are some of the dangers that you see from a survey that doesn’t take seriously some of these design considerations, that isn’t anticipating the different kinds of groups that are going to be responding to the survey? What’s the big danger here?

KM: There’s multiple dangers. The big danger is that you’re not collecting the data that you think you’re collecting and that there’s actually bias in the data that you’re collecting. Some groups are under-reporting, some groups are over-reporting, it’s just like what I talked about. But I also think that this leads to ethical issues. We’re the federal government, we’re supposed to be democratic, we’re supposed to be making sure that not one group is privileged over another group. And I also think that it’s an obligation for survey methodologists to be able to say, or to prove, or to give evidence that they are indeed capturing what they say they are capturing. I think it should be a best practice in survey research to be able to say, “This is what this measure gets and this is how I know.”  And that’s just not done.

And so we have these dual concerns  of both empirical validity, but also a democratic ethical obligation.

KM: Correct. And you know, there’s all these standards for surveys to show their response rate, sampling error, and go through all of these other issues related to survey research, but it has not been addressed regarding item-response error and validity. It’s such a huge piece of the puzzle.

I know that you’ll be talking (at DePaul) about your work designing a measure for sexual identity for national surveys. Could you talk a little bit  about what some of your concerns were?

KM: We were having huge problems getting good measures for that question. We looked in our own surveys and asked about it, but even if you looked internationally, there were so many people that would say “Don’t know” or there was non-response.  People didn’t know these terms. People didn’t know what heterosexual meant. People didn’t know what homosexual was, so they just wouldn’t answer. And what was happening, and what does happen, is that analysis would just go in and lop off the missings and then look at the data from there. But what was happening was that the missings were not random, they were largely low-SES people and Hispanics.  And so when you’re doing a cross-tab of obesity and sexual identity, what one of our surveys was finding was that lesbians were more likely to be obese than any other sexual identity group. Well, what was happening was that they really weren’t. The next year, we actually fixed the question and it turned out that lesbians were actually the least likely to be obese. Because what was happening was low-educated Hispanic heterosexual respondents, so you’re taking a particular group out of the analysis. So what I’ll be talking about are all the design problems we knew about and our understanding of the response errors that we were seeing, and how we made a question, tested the question, and this is the question that is in the field now.

There’s a near consensus among scholars that schools distribute punitive measures in thoroughly racialized ways, punishing black students most often (Keleher 2000; Gregory, Skiba, and Noguera 2010; Welch and Payne 2010). Being suspended or expelled from school substantially increases the likelihood that a young person will be arrested and incarcerated later in their lives (Wald and Losen 2003; Simmons 2006; Weissman, Cregor, and Gainsborough 2008). Racial disparities in suspension and expulsion rates, then, contribute to racial disparities in educational outcomes and life chances of young people in the United States. Harsh punitive measures by school officials do not lead to the improvement of pupils’ behavior or  reduce school violence and increase school safety (Imich 1994; Skiba and Peterson 1999; Skiba et el. 2008).

Map by Dan Cooper

To quantify one dimension of this problem, Mariame Kaba of Project NIA, and Frank Edwards of the SSRC at DePaul this week released “Policing Chicago Public Schools,” a report detailing the scope and character of arrests occurring on Chicago Public Schools’ (CPS) property. Data on policing activities in public schools is pretty difficult to get. School officials, CPS in particular, are very reluctant to release information about school discipline, and the Chicago Police Department has only recently made useful data on crime and arrests easily accessible to the public. This report is an effort to make what is going on in schools more transparent.

Key findings of the report include:

• More than 5,500 arrests of young people under 17 years old took place on CPS properties in 2010.
• Black youth are disproportionately targeted for arrest at school. In 2010, while they represented 45% of CPS students, black youth accounted for 74% of juvenile school-based arrests .
• Young men are much more likely to be arrested on CPS property than their female counterparts.
• Nearly a third of school-based arrests in 2010 were for simple battery (fighting).
• Certain police districts on Chicago’s South Side have far greater rates of arrest at schools than other police districts.

Notably, this data suggests that arrests on CPS property account for about 20 percent of all juvenile arrests in  Chicago. The bulk of these arrests are for relatively minor infractions such as simple battery and disorderly conduct.  Theorists, empirical scholars and activists have argued for some time that as zero-tolerance policies and policing have replaced traditional school disciplinary practices (e.g., a trip to the principal’s office), the rates of suspension, expulsion and arrest at schools have skyrocketed, suggesting that we might best view these efforts as creating a school-to-prison pipeline. This report provides empirical precision to help quantify the magnitude of the problem in Chicago. The results unfortunately confirm that this pipeline is indeed operating here and is contributing to the production of the racial bias scholars detect operating throughout the juvenile and criminal justice systems.

Frank Edwards contributed to this post.

This week, Scientific American posted a short interview with NASA’s chief climate scientist, Jim Hansen, often referred to as “the father of global warming”. The interview centers on Hansen’s activism in favor of policy changes to curb CO₂ emissions and fossil fuel dependency to curtail climate change. In recent years, Hansen has intentionally been leaving the lab to attend protests and rallies—even getting arrested multiple times, most recently this summer at the White House protesting an oil pipeline.

Suffragette Emmeline Pankhurst being arrested, via the Nationaal Archief, Netherlands

Asked at the top of the interview whether his credibility as a scientist has suffered due to his political activism around his subject of study, Hansen says:

If I was not publishing papers in the peer reviewed literature, then that would be a valid criticism. But I am still publishing. I’m trying to make that science clear to the public. It’s not easy: The scientific evidence has really become very clear, and we’re not doing a very good job of communicating that.

Hansen’s answer gets at the heart of something we talk about all the time at the SSRC: Getting socially-relevant research findings with the potential to impact policy into public discussions.

Scientific inquiry rests on a foundation of objectivity (a concept that could spiral into a post-modernist free-fall if we let it; luckily, we won’t), but most of that objectivity is confined to the lab. In developing our research designs and hypotheses, and analyzing our data, we strive for a bias-free open-mindedness that will allow the findings to surprise or even disappoint us. But how far does the persona of the dispassionate, objective, scientist extend? What happens in the case of findings that are critically important to society and  vital (by which I mean life-or-death) to policy decision-making?

To my knowledge, the ethical statements and policies of research institutions and professional organizations typically don’t address the dissemination of findings to the public in this regard. What if they did? Do researchers have an ethical obligation to make sure that their findings are not only published in the academic literature, but also are understood by the general public—especially in the case of critically important and socially relevant research? Is it the role of the researcher to act merely as an expert giving testimony to policymakers, or are activism and advocacy like Jim Hansen’s necessary?

I don’t think climate change is the only arena in which credibility and objectivity might be called into question when a researcher engages in political activism (though that’s a particularly hot potato these days), but I do think it’s telling that the very first question put to a respected researcher is whether his willingness to fight for his findings damages his credibility. Would you be willing to risk arrest for your research? How far will you go to make sure that the public hears and understands your findings in the context of a political and public debate? Is that even the role of the researcher?

This is a centerpiece of our work at the SSRC; we’re eager to know what you have to say about whether and how research findings can or should be “translated” to the layperson and findings disseminated to the public at large. Any thoughts?

Photo by Jeffrey Beall

The root of my newest obsession: this article.  The author, Seth Stephens-Davidowitz, a PhD student in economics at Harvard, innovatively uses location-specific Google search terms (Google Insights) to estimate that Obama lost 5 percentage points in 2008 because of “racial animus” or racism.  How does he do this? Well, since self-reported surveys poorly capture racism, he looks at use of the n-word in Google searches as a proxy for an area’s level of “racial animus.” He then compares an area’s racially charged search terms to its votes for Obama, controlling for its votes for John Kerry in 2004. Although we could definitely argue whether Google searches including the n-word adequately capture racial animus, there appears to be something to this. To a methodologist, it presents a fascinating example of attempting to measure something that appears to be unmeasurable.

Davidowitz isn’t the only one using Google search terms for social research. NPR recently did a piece on sociologist Phillip Cohen’s Google Correlate explorations.  Cohen has some interesting analyses examining the correlations among search terms and geographic space, such as how Google searches for certain political commentators such as Rachel Maddow or Rush Limbaugh correlate strongly with other “odd” search terms (like fennel salad) and also adhere to red/blue-state patterns. Is your interest piqued? Here are more of Cohen’s interesting Google correlations.

Guess which terms correlate more strongly with “cadmium” (a chemical element in the periodic table)? Here’s the answer.  Hmmmm….interesting?

In sum, I agree with Cohen that “Someone — probably not me — should get serious about using this kind of data to connect search behavior with demographic trends, politics, culture, and other aggregate patterns of social behavior.” Any takers?

The Social Media Research Foundation’s mission is to understand, map, and measure social media. The SMRF tools will allow scholars to create data visualizations in addition to data collection and analysis.

One of the most fascinating rapidly emerging open-source tools provided by SMRF is NodeXL. This template can be downloaded as a Microsoft Excel add-on and will enable users to easily graph data collected via social networking sites. The social network analysis feature supports a number of networking data sets including personal emails, twitter, flickr, and facebook. Still confused by how this works? Tutorial handbooks used by professional instructors who teach social media analysis are available to guide you through this process.

The Social Media Research Foundation may not have the answers to all of your questions, however they are pioneers in a relatively new field of research contributing to a large body of work. Explore their website; you may find something of interest to you.

By Lars Plougmann@Flickr.com

Following the release of the U.S. Census Bureau’s 2010 Census, several demographers and research institutes are worth noting for the helpful graphs and maps they have created to highlight changes in the racial and ethnic landscape of America. These data snapshots, based on the 2010 U.S. Census and the Census Bureau’s American Community Survey, offer a quick and easy overview of the country’s racial and ethnic make-up in terms of residential segregation, income segregation, population changes, and other demographic shifts over the past few decades.

The Census Bureau has also put out some great interactive maps and data profiles of interest to both researchers and the general public. Its interactive population map, for example, allows users to follow demographic changes from state to state, right down to census blocks within cities. You can easily locate your own census track or block by a search feature and find out the demographic make-up of your neighborhood.

Other interesting data snapshots are the work of sociologist and demographer Dr. William H. Frey, which can be accessed through his website and the University of Michigan’s Social Science Data Analysis Network (SSDAN). His visual depiction of population changes includes each state, going back to the 1900s. These charts and graphs illustrate segregation rates over the past 30 years by state to provide an overview of residential changes over time. Brown University’s US 2010 Research Project is another useful tool for examining levels of segregation. The site lets users access information on segregation for Metropolitan Statistical Areas using measures that include the index of dissimilarity and exposure indices.

Together, these tools offer a general visual understanding of America’s racial and ethnic population changes and its residential orientation in a Post-Civil Rights era.

Photo by Per Ola Wiburg

We’ll be taking a short blogging break while DePaul is closed for Winter Break.

Stay warm and cozy and we’ll see you in 2012. Happy New Year!

Photo via James Vaughn

I recently stumbled upon one of the most ingenious resources available on the internet and want to share it with you. Personally, I don’t think any post is worth more than one that allows the public to access something so marvelous, so productive, so stimulating, and so superbly edifying. Perhaps I’m exaggerating a bit, but truth be told, having an online lecture course by some of the most renowned scholars at your fingertips is quite impressive. While you may not be able to attend their classes, you can certainly sit quietly at your computer, plug in your headphones, and watch lecturers talk about a particular subject or listen to their lectures over iTunes. Is that great or what? And it is all free.

Open Culture, an online source for free cultural and educational media, has approximately 400 online “courses”  by well-known scholars available. I’m talking about scholars such as David Harvey, Richard Wolff, Clayborne Carson, Harvey Molotch, Michael Burawoy, and many more. Each of these scholars has taught graduate and undergraduate courses over their academic careers which they have recorded and now made available to the public. For instance, those interested in reading Karl Marx’s Capital, Volume 1: A Critique of Political Economy, may find David Harvey’s lecture series helpful while reading Marx’s book. Harvey has recorded 13 sessions of lectures covering Capital after teaching it for 30 years. Because this particular text can be confusing, having a renowned scholar like David Harvey help a reader understand Marx is a great luxury. Other scholars I’ve mentioned have also made their lectures public, in many cases, through online lectures organized by their universities and colleges, such as UC Berkeley, Yale, and MIT. This seems like a growing trend, with implications for pedagogy.

Additionally, a great stock of foreign language courses is also available, including a fascinating Spanish language course created by the BBC. It provides an interactive feature that includes watching a sort of virtual movie while you learn basic Spanish.  Need to learn the basics of a language for your research project (not uncommon)? Why spend money when you can do it for zilch?

In any event, check out Open Culture and immerse yourself in one of these lectures. I sure you will find something worth your time. Enjoy!