This blog is meant to clarify what we might call “flawed” understanding of a USAID spokesman regarding the BARR survey methodology and how the earth quake death toll was arrived at.
As part of the US government’s effort to discredit a survey that it commissioned and for which it reviewed and approved the methodology, the Assistant Administrator for Latin America and the Caribbean for USAID issued this statement on June 4th,
“Mark Feierstein of the U.S. Agency for International Development said the report is problematic because the authors used a statistical sampling that was not representative. The study didn’t include data from heavily damaged areas in Haiti’s countryside or from the number of houses that collapsed and killed people, he said.
‘Those are all serious flaws,’ Feierstein, USAID’s assistant administrator for Latin America and the Caribbean, told The Associated Press in a telephone interview.”
What’s flawed here is USAID’s understanding of the methodology that it reviewed and approved. The fact is that, as USAID has indicated elsewhere, the survey was commissioned specifically to justify how 100’s of millions of dollars of US Government money was spent. But as it turned out, the data also contradicted the erroneous suppositions that every major NGO, UN, US and Haitian Government organization has participated in promoting. Suppositions that have encouraged enormous expenditures and donations from well meaning US and European citizens but with disappointing results.
For the record, here is a detailed response to what can only be called USAIDs flawed understanding of the survey.
There are two points upon which the estimation is said to be flawed.
1) the study didn’t include data from the number of houses that collapsed and killed people.
2) the study didn’t include data from heavily damaged areas in Haiti’s countryside
Here are clarifications:
1) The BARR survey did include data from houses that collapsed and killed people.
The University educated BARR surveyors visited all homes in the randomly selected sample areas. In the case of collapsed or abandoned residences, the surveyors gathered basic demographic data from neighbors. Specifically, they asked neighbors how many people were living in the house at the time of the earthquake, number killed, number of survivors, and current location of survivors.
As for people killed in schools, businesses, or on the street: From a methodological standpoint what was so effective about the BARR survey technique is that interviewers asked people, not “how many people were killed in this home?” but rather, “how many people who lived in this home were killed?”
In other words, the BARR survey captured data on people killed wherever they might have been and it included everyone who lived in the selected sample area at the time of the earthquake, whether their home had collapsed or not.
Note that everyone has a home and it was homes that were the sampling units and this is why the BARR technique is much more powerful than having, for example, looked at how many people died in a sample of buildings that included businesses or schools. The point being that what BARR surveyors were interested in was not the homes, so much as the people who lived in them.
2) Data from heavily damaged areas in Haiti’s countryside were not included in the survey:
Those areas were deliberately omitted from the original sampling frames specifically because it was known that they were not as hard hit as metropolitan Port-au-Prince.
However, they were included in the final estimates based on what was found in hard hit metropolitan Port-au-Prince.
In other words, the overwhelming likelihood is that the BARR estimates for total fatalities from the earthquake are higher than the true death toll.
More specifically, the study was based on that part of metropolitan Port-au-Prince that was hardest hit by the earthquake and which, according to official declarations from the Government of Haiti, suffered 90% of all damage wrought by the seismic event.
The population of that region, using census figures, is about 2 million. The population that conclusions were extrapolated to was 1 million, for a total population of 3 million–the United Nations defined earthquake impacted population.
Again, BARR sampled the 2 million hardest hit people, and then extrapolated the findings to the total population of 3 million that the UN had defined as impacted.
At the risk of redundancy, the 1 million people to which the findings were extrapolated lived in areas not as severely struck by the earthquake.
This means that intuitively the survey would have overestimated the number of people killed.
For those interested, here are the specifics. Areas such as Cite Soley (a neighborhood of some 300,000), the Department of Sudest where Jacmel is located (and where a majority portion of the 500,000 people in that region are included in the earthquake strike zone), and upper Petion Ville and Kenscoff (the mountainous area above Port-au-Prince zone which includes another 200,000 plus people), were not included in the sample. The reason for this is because it was known that those areas were not especially hard hit. In the case of Cite Soley, the many wooden, light cement, and tin roofed homes meant that the population suffered very little from the earthquake. Upper Petion Ville and Kenskoff escaped with even less damage. Indeed, in most of that region one would not be able to detect that an earthquake hit.
Jacmel, although receiving a great deal of media attention, ended up with an official total fatality figure of 300 to 400 people (based on the Mayor’s office).
Among the only significantly impacted regions not included in the survey was Leogane, the epicenter of the earthquake. That’s an area of over 200,000 people. And while the Central government reported 20,000 to 30,000 deaths; Leogane authorities reported 3,364.
And so even Leogane did not warrant inclusion.
And once again, it was to these areas that the rates from Port-au-Prince were extrapolated to arrive at the estimations. Estimations that it was clear would overestimate, not underestimate, the death toll had they been included in the original representative sample.
As for outlying rural areas. The rural areas suffered far less damage than metropolitan Port-au-Prince. In the rural areas people live in tin and thatch roofed homes built of waddle and dab, wood, and to a much lesser degree of light cement construction. USAID knows this. ACDI/VOCA, that organization with the largest USAID contract in the Jacmel region had 500 employees and volunteers working in the Jacmel rural area at the time of the earthquake. None were killed. New Missions in Leogane, where no fewer than four international organizations based relief efforts after the earthquake, reported having 5,000 children on sponsorship and an unknown number of teachers, pastors and church members. On their website they listed 6 fatalities.
The point I am making here is that it was upon this type of data that the survey decisions were made.
And while I have no vested interests in the survey conclusions one way or another, I do have an interest in defending my reputation and logic upon which the methodology was developed and the analysis made.
And now, I am perplexed as to why one of the highest ranking directors of the same USG organization that commissioned the survey and approved the methodology is undermining or does not understand what those methods were.
