[casi-analysis] lancet figures: fuller government response is still rubbish

[Daniel O'Huiginn <do227@DELETETHIShermes.cam.ac.uk>]

[ This message has been sent to you via the CASI-analysis mailing list ]


The FCO has just released a slightly more detailed response to the
Lancet report. I've pasted it below, or you can read it at
http://www.fco.gov.uk/servlet/Front?pagename=OpenMarket/Xcelerate/ShowPage&c=Page&cid=1007029391629&a=KArticle&aid=1100183680513

To summarise...

It begins with a general introduction ('the security context'). The main
argument here is 'we have a UN-given right to Iraq, so any violence is the
fault the terrorists who dare oppose us. The way you can tell they're
terrorists is that they oppose us'.

Then there's an odd rejection of the case that the government should be doing
body counts: "The Lancet study suggests that there is an obligation deriving
from Article 27 of the Fourth Geneva Convention for the multinational force
(MNF) itself to have a reckoning of the number of civilian casualties it has
caused. There is nothing in Article 27, or elsewhere in the Fourth Geneva
Convention, to support this suggestion."

The Lancet report's actual reference to the Geneva Convention was a little more
nuanced:

"The Geneva Conventions have clear guidance about the responsibilities of
occupying armies to the civilian population they control. The fact that more
than half the deaths reportedly caused by the occupying forces were women and
children is cause for concern. In particular, Convention IV, Article 27 states
that protected persons “. . . shall be at all times humanely treated, and shall
be protected especially against acts of violence . . .”. It seems difficult to
understand how a military force could monitor the extent to which civilians are
protected against violence without systematically doing body counts or at least
looking at the kinds of casualties they induce." [p. 7]

This is probably not the best legal reference around [doesn't 'protected
persons' basically mean PoWs?], but then Roberts &co aren't lawyers. The basic
argument, that monitoring of casualty figures is needed to make sure you're
taking due care is a valid one.

Moreover, it's an argument that the government itself makes quite strongly in
other cases. Take a look, for example, at DFID's 'country assistance plan' from
February [http://www.dfid.gov.uk/pubs/files/capiraqenglishfull.pdf]. There,
DFID thinks that one of the best services it can offer the the Iraqi government
is helping it collect decent statistics, and so develop and monitor policy:

"Our priority will be to ensure that economic
policies take account of the needs of the poor and
vulnerable and that appropriate social safety nets
are put in place. We will offer help to the Central
Statistics Office to provide reliable data as the basis
for deciding policy, and monitoring impact" [p. 10]

If DFID thinks monitoring is essential for government planning, why does the
FCO not think monitoring is essential for military planning?

Then we get the argument that health ministry figures and IBC differ from the
Lancet figures. We've been through all this before, and the government doesn't
have a leg to stand on, so I'll let it be. Interesting quote: "hospitals in
Iraq have no obvious reason to under-report the number of the dead and injured"
[the assumption being that by virtue of being hospitals, they have perfect
knowledge of every death in Iraq. Silly; among other things, the Lancet authors
point out that families rarely had death certificates for children, suggesting
that the hospitals don't have good records of infant deaths.]

Finally, some comments on the Lancet methodology itself. The basic method here
is to break down the figures into the smallest subgroups possible, and then
argue that they are ridiculous, or that they show that nothing was the
coalition's fault. It should be obvious that if you break down figures to the
point where you're basing conclusions on only a few deaths, you're going to get
some odd results. That doesn't discredit the overall death figures. I'm pretty
sure there's a comment in the report to the effect that you can't treat the
study as giving reliable information on sub-groups. If that comment isn't
there, it's something that should be obvious to any reader. And if you do play
with subgroups, you can come up with all kinds of obscene but unreliable
figures (for example Glen managed to extrapolate the child mortality figures to
indicate 38,000 excess child deaths).

There are some other fairly dubious arguments based on the statistics. People
other than me are in a better position to analyse them properly, but the
following seemed odd:
- "The figures derived from the survey’s data on Fallujah would have resulted
in an estimated 200,000 excess deaths within Fallujah alone" Is this what the
extrapolation from Fallujah figures means? I thought it was an extra 200,000
nationwide, but I could be wrong.
- no comment on the implications of discounting Fallujah/the tendency of
cluster sample surveys to underestimate (see
http://www.crookedtimber.org/archives/002858.html for a decent explanation)
- assuming that deaths by heart attack, road accident, etc, are unrelated to
the invasion (this perhaps links to a general perception that the coalition are
only responsible for people they actually shoot, rather than the wider group of
those who have died as a result of the invasion)

I hope a list member with a better grip on the statistical side of things can
go a bit further with this.

Also, a wonderful argument: "Since 58 of the 61 deaths attributed to Coalition
forces were said to have been caused by “helicopter gunships, rockets or other
forms of aerial weaponry”, it cannot have been possible for the families in
every case to have known for certain who was responsible. It is also possible
that they would have been afraid to have blamed the deaths of their relatives
on the insurgents." So the 'insurgents' are now operating helicopter gunships?
Scary!


And, as a tangent, a couple of statements which could be useful in campaigning
on other issues: a) "This obligation [to avoid excessive harm to civilians]
under international humanitarian law has been fully complied with by the United
Kingdom in respect of all military operations in Iraq." b) "The MNF
[multi-national force] is currently providing food and other supplies to
civilians in the Fallujah area, and working closely with the Iraqi Ministry of
Health to ensure medical supplies reach there."


[http://www.fco.gov.uk/servlet/Front?pagename=OpenMarket/Xcelerate/ShowPage&c=Page&cid=1007029391629&a=KArticle&aid=1100183680513]

WRITTEN MINISTERIAL STATEMENT RESPONDING TO A LANCET STUDY ON IRAQI CASUALTY
FIGURES (17/11/04)

Written Ministerial Statement (17 November 2004)

Iraq: Casualty Estimates: Lancet Study

The Secretary of State for Foreign and Commonwealth Affairs: This Statement
provides a response to the article .Mortality before and after the 2003
invasion of Iraq: cluster sample survey. published in the Lancet on 29 October
concerning civilian casualties since the beginning of military action in Iraq
in March 2003.

The Security Context

It is important to recall the background to the current violence in Iraq.  In
the period of major combat activities in Iraq between the coalition and Iraqi
forces loyal to Saddam Hussein, there were inevitably civilian casualties
caused by military action by both sides. Every effort was made, on the part of
the coalition, to minimise the civilian casualties as required by international
humanitarian law.

Casualties . civilian and military . which have occurred since major combat
activities ended on 1 May 2003 have done so directly as a result of those
determined to undermine the political process. Security Council Resolution
1546,  adopted on 8 June 2004, noted the request of the Iraqi Interim
Government that the Multi-National Force should remain to help the sovereign
Government of Iraq to ensure security and reaffirmed its authorisation on that
basis.  The Multi-National Force has been acting under that mandate, in support
of the Iraqi security forces, to ensure the maintenance of security and
stability in Iraq.  The mandate specifically authorises action against
terrorists.

The Iraqi government continues to face a violent insurgency.  Every day brings
grim evidence that the targets of this insurgency are not merely a supposed
occupation force.  They include members of Iraqi civil institutions and
security forces, ordinary Iraqis and foreigners working to build a better
future for Iraq. They include children killed when co-ordinated bombs were
detonated near a water treatment plant in Baghdad, Iraqi army trainees and
police butchered with premeditation and foreigners executed in front of the
cameras. The scale of the threat has prompted international humanitarian
agencies to close their operations in the country.

In those many parts of Iraq where there have been no terrorist incidents, there
have been no casualties. If the terrorists and insurgents gave up their
campaign, the violence in Iraq would cease.

However, while the insurgents continue in their attempts to destroy the
political process leading to a constitutionally elected government, and to
attack those rebuilding Iraq.s infrastructure and public services, the
Multi-National Force will support the efforts of the Iraqi government and
security forces to defeat them.  That may include military action if there is
no alternative.

The Multi-National and Iraqi forces continue to act so as to minimise civilian
casualties. This is despite the fact that the insurgents have shown no
compunction in using mosques, schools and hospitals as defensive bases.  The
MNF is currently providing food and other supplies to civilians in the Fallujah
area, and working closely with the Iraqi Ministry of Health to ensure medical
supplies reach there.

The Iraqi Government and its international partners remain committed to
defeating the terrorists and insurgents in Iraq.  I pay tribute to all those
who are working to build a safe and democratic country.

The legal context

The Lancet study suggests that there is an obligation deriving from Article 27
of the Fourth Geneva Convention for the multinational force (MNF) itself to
have a reckoning of the number of civilian casualties it has caused. There is
nothing in Article 27, or elsewhere in the Fourth Geneva Convention, to support
this suggestion.

The basic obligations under international humanitarian law as regards civilian
casualties in an armed conflict are set out in Additional Protocol 1 to the
Geneva Conventions, which also reflects customary international law.  In
particular, indiscriminate attacks are prohibited, and this includes any
.attack which may be expected to cause incidental loss of civilian life, injury
to civilians, damage to civilian objects, or a combination thereof, which would
be excessive in relation to the concrete and direct military advantage
anticipated..  This obligation under international humanitarian law has been
fully complied with by the United Kingdom in respect of all military operations
in Iraq.

Casualty Estimates

In many cases it would be impossible to make a reliably accurate assessment
either of the civilian casualties resulting from any particular attacks or of
the overall civilian casualties of a conflict.  This is particularly true in
the conditions that exist in Iraq.  However, since 5 April 2004 the Iraqi
Ministry of Health has sought to collect casualty data.  Explaining the
procedure, the Iraqi Minister of Health stated on 29 October: .Every hospital
reports daily the number of civilians (which may include insurgents) who have
been killed or injured in terrorist incidents or as a result of military
action.  All casualties are likely to be taken to hospital in these
circumstances except for some insurgents (who may fear arrest) and those with
minor injuries.  The figures show that between 5 April 2004 and 5 October 2004,
3,853 civilians were killed and 15,517 were injured.  I am satisfied that this
information is the most reliable available..  We share this view.  The
Ministry.s figures do not of course cover the whole of the period since
military action was taken, but they do include the months of April and August,
when casualty figures were particularly high.

Other figures are proposed by NGOs.  www.iraqbodycount.net, an NGO-based
website, provides a running estimate of civilian casualties based on media and
other reports.  This has suggested that from March 2003 to the present there
have been between 14,284 and 16,419 civilian deaths. This is an estimate
relying on media reports, and which we do not regard as reliable.  It includes
civilian deaths at the hands of terrorists as well as of the Coalition forces.
It relies on media reporting to decide who is a civilian and who is not.  It
does help to show however that the Iraqi Ministry of Health figures are not the
only ones to differ widely from the Lancet.s estimate.

The Lancet

The article estimates that between 8,000 and 194,000 more people died following
the invasion of Iraq than previous rates of mortality would have predicted,
with the .most likely. figure being 98,000 extra deaths. 73 of the total of 142
deaths recorded by the survey in the period since the invasion are judged to
have occurred through violence from the coalition forces or from terrorists.
Other deaths recorded were the result for example of heart attacks or road
accidents, not of coalition or terrorist action.

The Lancet.s researchers acknowledge that they encountered no evidence of
widespread wrongdoing on the part of individual Multi-National Force (MNF)
soldiers on the ground.

The design of the Lancet study and its statistical methodology passed the
process of peer review before publication and is similar to that followed in
cases where the data are difficult to obtain.   But that should not mask the
fact that any methodology critically depends on the accuracy of the data
subject to its analysis.

As the authors of the study themselves acknowledge, it was carried out under
exceptionally difficult conditions which in particular restricted the size of
the samples surveyed. As the Lancet article says, the estimate of deaths is
based on an extrapolation from an increase from 46 deaths in 110,538
person-months before the conflict, to 142 deaths in 138,439 person-months after
the conflict. Of those, 61 deaths were attributed to Coalition forces, most of
them in Fallujah, a sample which the authors admit is an .extreme statistical
outlier..

The figures derived from the survey.s data on Fallujah would have resulted in
an estimated 200,000 excess deaths within Fallujah alone over the past 18
months.  This would amount to almost two-thirds of the total population of the
town - which is just not credible.  The authors of the study understandably
discounted the data. In general they have noted that the data on which they
based their projections was of .limited precision.. This limited precision is
reflected in the very large range which they use for their estimate of excess
mortality (8,000 . 194,000).  Although the levels of probability vary across
its range, any figure within this range is consistent with the data.

We doubt the survey.s attribution of 61 violent deaths to action by Coalition
forces.  Only 2 deaths are attributed in the survey to .anti-coalition forces..
This is an astonishingly small proportion of those said to have died from
violence, given the large numbers of Iraqis we know have died in individual
incidents at the hands of terrorists.  Since 58 of the 61 deaths attributed to
Coalition forces were said to have been caused by .helicopter gunships, rockets
or other forms of aerial weaponry., it cannot have been possible for the
families in every case to have known for certain who was responsible.  It is
also possible that they would have been afraid to have blamed the deaths of
their relatives on the insurgents.
The authors also acknowledge that .many of the Iraqis reportedly killed by US
forces could have been combatants..  The greatest increase in deaths which they
report was among 15-59 year old men, while for instance among the elderly in
the survey there was effectively no increase in the death rate at all.

Other questions about the Lancet study relate to the significant differences
between its estimate and other evidence, notably the figures on casualties
produced by the Iraqi Ministry of Health and quoted above.

If the Lancet survey is accurate we could have expected Iraqi Ministry of
Health figures, compiled by hospitals, to show many more  times the number of
people killed and wounded over that period than they in fact do. Hospitals in
Iraq have no obvious reason to under-report the number of dead and injured.
The Lancet article does not explain this discrepancy.

So while recognising the bravery and professionalism of those conducting the
Lancet study, the Government does not accept its central conclusion, and
continues to believe that the most reliable figures for casualties in Iraq are
those provided by Iraqi hospitals to the Iraqi Ministry of Health.

------------
Daniel O'Huiginn
do227@cam.ac.uk
07745 192426
24, Priory Road, Cambridge
------------



_______________________________________
Sent via the CASI-analysis mailing list
To unsubscribe, visit http://lists.casi.org.uk/mailman/listinfo/casi-analysis
All postings are archived on CASI's website at http://www.casi.org.uk