Baldeep Singh, MD, with staff at Samaritan House

Stanford researchers CLEA SARNQUIST and MICHAEL BAIOCCHI, PHD (center left and right), work with Kenyan researchers and data colletors to refine their latest survey.

A Project to Reduce Rape of Young Kenyans

Stanford researchers CLEA SARNQUIST and MICHAEL BAIOCCHI, PHD (center left and right), work with Kenyan researchers and data colletors to refine their latest survey.

A Project to Reduce Rape of Young Kenyans

The topic is daunting, even unbelievable in our world, and the complexities that surround it are hard to grasp. How do you teach girls aged 12 to 14 to fight off a sexual assault — in Kenya — in slums where regular meals and clean water are not assured? Moreover, almost as important, how can you know whether the lessons actually worked?

Reliable survey data indicate that as many as 46 percent of Kenyan women experience sexual assault as children. For the most part, these girls do not report rapes or assaults, even to their parents, as the risks are too great.

The nonprofit group No Means No Worldwide, founded by Lee Paiva from San Francisco, has been working to reduce the incidence of rape in young girls and women in Kenya since 2010. Anecdotal reports about the prevention program have been positive, with the girls being inspired by an educational intervention that increases their self-esteem and teaches them defensive tactics.

The reports of the girls successfully avoiding attempted rapes and sexual assaults have been rewarding to those involved in the program. But objective data had been missing, leaving them to wonder if the time and money being spent are having the desired result. To gather those data, Stanford researchers, led by Michael Baiocchi, PhD, tackled the challenging job of designing a randomized controlled trial that compares the rate of rape in trained girls with that in untrained girls.

The Intervention
The intervention is taught in school by local women and introduces four pathways to preventing sexual assault. The girls are introduced to situational awareness, where they learn to recognize dangerous situations and to look around for who or what can help them. They are taught that their own thoughts and feelings are valued and thus they learn to be empowered to make themselves heard in dangerous situations.

They learn what to say — to shout — in such a situation. And they learn physical skills for defending themselves. Not only do they learn to fight off an attack, often by family members or boyfriends, but they also learn how to report those attacks so the situation can be improved.

Challenges of Randomization
The team decided that the most ethical way to learn the relative effectiveness of the intervention and, critically, to collect objective data on outcomes is to use a delayed-treatment study design. Girls would be randomized into two groups: one taught the intervention immediately, the other taught the intervention later. The two groups complete surveys at three time points, measuring the difference in the number of rapes in both groups of girls over two years.

Baiocchi, assistant professor of medicine in the Stanford Prevention Research Center, is the principal investigator of the trial.

Although randomized controlled trials are considered the gold standard for measuring differences between two groups, as a statistician Baiocchi immediately recognized issues that might compromise the results of the trial and devised ways to either avoid or account for them.

Problems and Solutions
Having learned about some specific problems from their earlier, smaller study of girls in 28 schools, Baiocchi and his colleagues — statistics PhD students Rina Friedberg and Evan Rosenman — created statistical tools that would let them avoid a false-negative result. A study with a false-negative result, which would incorrectly show no benefit from an intervention that really does work, can be devastating as it can cripple an otherwise valuable line of research.

The first statistical problem was spillover, which is a major problem for behavioral interventions. In Nairobi the schools the girls attended were close enough to one another that girls who were taught the intervention might share what they learned with friends who were in the delayed intervention group. After several months of such sharing, the trial could have 500 trained girls in the intervention group, another 100 trained girls in the supposedly ‘untrained’ group, and only 400 truly untrained girls. This spillover between trained and untrained groups could jeopardize the result. “Even if your intervention is working and it’s doing a really good job,” explains Baiocchi, “if it spills over in ways that you’re not anticipating you get a fake null result.”

The topic is daunting, even unbelievable in our world, and the complexities that surround it are hard to grasp. How do you teach girls aged 12 to 14 to fight off a sexual assault — in Kenya — in slums where regular meals and clean water are not assured? Moreover, almost as important, how can you know whether the lessons actually worked?

Reliable survey data indicate that as many as 46 percent of Kenyan women experience sexual assault as children. For the most part, these girls do not report rapes or assaults, even to their parents, as the risks are too great.

The nonprofit group No Means No Worldwide, founded by Lee Paiva from San Francisco, has been working to reduce the incidence of rape in young girls and women in Kenya since 2010. Anecdotal reports about the prevention program have been positive, with the girls being inspired by an educational intervention that increases their self-esteem and teaches them defensive tactics.

The reports of the girls successfully avoiding attempted rapes and sexual assaults have been rewarding to those involved in the program. But objective data had been missing, leaving them to wonder if the time and money being spent are having the desired result. To gather those data, Stanford researchers, led by Michael Baiocchi, PhD, tackled the challenging job of designing a randomized controlled trial that compares the rate of rape in trained girls with that in untrained girls.

The Intervention
The intervention is taught in school by local women and introduces four pathways to preventing sexual assault. The girls are introduced to situational awareness, where they learn to recognize dangerous situations and to look around for who or what can help them. They are taught that their own thoughts and feelings are valued and thus they learn to be empowered to make themselves heard in dangerous situations. They learn what to say — to shout — in such a situation. And they learn physical skills for defending themselves. Not only do they learn to fight off an attack, often by family members or boyfriends, but they also learn how to report those attacks so the situation can be improved.

Challenges of Randomization
The team decided that the most ethical way to learn the relative effectiveness of the intervention and, critically, to collect objective data on outcomes is to use a delayed-treatment study design. Girls would be randomized into two groups: one taught the intervention immediately, the other taught the intervention later. The two groups complete surveys at three time points, measuring the difference in the number of rapes in both groups of girls over two years.

Baiocchi, assistant professor of medicine in the Stanford Prevention Research Center, is the principal investigator of the trial. Although randomized controlled trials are considered the gold standard for measuring differences between two groups, as a statistician Baiocchi immediately recognized issues that might compromise the results of the trial and devised ways to either avoid or account for them.

Problems and Solutions
Having learned about some specific problems from their earlier, smaller study of girls in 28 schools, Baiocchi and his colleagues — statistics PhD students Rina Friedberg and Evan Rosenman — created statistical tools that would let them avoid a false-negative result. A study with a false-negative result, which would incorrectly show no benefit from an intervention that really does work, can be devastating as it can cripple an otherwise valuable line of research.

The first statistical problem was spillover, which is a major problem for behavioral interventions. In Nairobi the schools the girls attended were close enough to one another that girls who were taught the intervention might share what they learned with friends who were in the delayed intervention group. After several months of such sharing, the trial could have 500 trained girls in the intervention group, another 100 trained girls in the supposedly ‘untrained’ group, and only 400 truly untrained girls. This spillover between trained and untrained groups could jeopardize the result. “Even if your intervention is working and it’s doing a really good job,” explains Baiocchi, “if it spills over in ways that you’re not anticipating you get a fake null result.”

The fix for this problem, says Baiocchi, was to develop a framework for “weighted-design randomized trials where you can either create a lot of spillover or no spillover at all. For interventions that have a social component, such as the Kenyan girls playing together, the framework is useful for defining indirect effects.”

The second problem was imbalances between the arms of randomized trials. Statistically, a randomized trial with 5,000 flips of a coin is very likely to have groups that are similar, whereas a trial with 28 flips of a coin is quite likely to have imbalances. In their initial trial of 28 schools, imbalance hit the study hard. One of the two groups had a rape rate of 11 percent at baseline while the other had a rape rate of 7 percent; such an imbalance at baseline can challenge drawing strong results from the trial. “To overcome this,” says Baiocchi, “we developed a sensitivity analysis that asks how imbalanced arms of the trial have to be before your conclusions are suspect. Our framework helps researchers who use cluster-randomized trials understand how much imbalance is too much imbalance. This framework is a win for public health randomized trials.”

Adapting the New Trial
The current trial includes girls in 94 schools: Girls in 48 of the schools receive the training immediately while 46 schools will have the intervention at a later date. The researchers have been careful to put schools with tight social bonds in the same cohort, therefore avoiding having the intervention spill over from trained to untrained girls. Friedberg explains that “just dividing everyone geographically might result in two populations that are materially different, and then you have another problem.”

Baiocchi adds that to avoid both the spillover and imbalance problems “we selected schools that were far enough apart that we didn’t believe the girls would form friendship bonds but close enough that the schools looked very similar.”

An Unexpected Study
Baiocchi and his graduate students have an opportunity to measure the impact of their training in a completely unanticipated study. Rosenman describes a new project with political beginnings. “Because of Kenya’s disputed presidential election in 2017 and the wave of violence that ensued, our data collection was disrupted for months. That gave us the opportunity to think about how political violence relates to sexual violence, and so we are comparing two cohorts, one from before the election and one after.”

Baiocchi further explains how this study will help them: “We would expect to see an uptick of violence against vulnerable populations during this period. Now we have a chance to learn whether our intervention performed better or worse during those months.” This project may provide useful, empirical evidence for developing interventions to reduce rates of sexual assault in active conflict zones — the topic of the 2018 Nobel Peace Prize.

The fix for this problem, says Baiocchi, was to develop a framework for “weighted-design randomized trials where you can either create a lot of spillover or no spillover at all. For interventions that have a social component, such as the Kenyan girls playing together, the framework is useful for defining indirect effects.”

The second problem was imbalances between the arms of randomized trials. Statistically, a randomized trial with 5,000 flips of a coin is very likely to have groups that are similar, whereas a trial with 28 flips of a coin is quite likely to have imbalances. In their initial trial of 28 schools, imbalance hit the study hard. One of the two groups had a rape rate of 11 percent at baseline while the other had a rape rate of 7 percent; such an imbalance at baseline can challenge drawing strong results from the trial. “To overcome this,” says Baiocchi, “we developed a sensitivity analysis that asks how imbalanced arms of the trial have to be before your conclusions are suspect. Our framework helps researchers who use cluster-randomized trials understand how much imbalance is too much imbalance. This framework is a win for public health randomized trials.”

Adapting the New Trial
The current trial includes girls in 94 schools: Girls in 48 of the schools receive the training immediately while 46 schools will have the intervention at a later date. The researchers have been careful to put schools with tight social bonds in the same cohort, therefore avoiding having the intervention spill over from trained to untrained girls. Friedberg explains that “just dividing everyone geographically might result in two populations that are materially different, and then you have another problem.”

Baiocchi adds that to avoid both the spillover and imbalance problems “we selected schools that were far enough apart that we didn’t believe the girls would form friendship bonds but close enough that the schools looked very similar.”

An Unexpected Study
Baiocchi and his graduate students have an opportunity to measure the impact of their training in a completely unanticipated study. Rosenman describes a new project with political beginnings. “Because of Kenya’s disputed presidential election in 2017 and the wave of violence that ensued, our data collection was disrupted for months. That gave us the opportunity to think about how political violence relates to sexual violence, and so we are comparing two cohorts, one from before the election and one after.”

Baiocchi further explains how this study will help them: “We would expect to see an uptick of violence against vulnerable populations during this period. Now we have a chance to learn whether our intervention performed better or worse during those months.” This project may provide useful, empirical evidence for developing interventions to reduce rates of sexual assault in active conflict zones — the topic of the 2018 Nobel Peace Prize.

 Save as PDF