Why you should dismiss the CARRS-Q bicycle helmet research
The Cochrane Review
This is the second post in a series (Read Part One) looking at the non-peer reviewed CARRS-Q publication entitled “Bicycle Helmet Research“, published in 2011 and widely relied upon to support mandatory helmet laws for cyclists. This publication was commissioned by the State Government to support its policy of mandatory helmet laws in Queensland, Australia. This post considers helmet efficacy research conducted by the Cochrane Collaboration, which was heavily relied upon by the CARRS-Q publication.
The CARRS-Q publication’s key finding is that:
“[c]urrent bicycle helmet wearing rates are halving the number of head injuries experienced by Queensland cyclists.”
In its publication CARRS-Q goes so far as to say that helmets reduce the risk of head injury by 69%, and serious brain injury by 74% (and these figures are being quoted, unquestioned, by the media). While these figures are far in excess of the findings of much of the research it reviews, CARRS-Q presents these high figures by relying heavily on just the one report published by a company called the Cochrane Collaboration (Thompson et al: Helmets for preventing head and facial injuries in bicyclists, Cochrane Database of Systematic Reviews 1999).
The Cochrane Collaboration is a UK based company, listed as a charity, that presents itself as an international network of volunteer contributors whose writings on issues of health and welfare are published through a medium called the Cochrane Library. The Cochrane Collaboration is predominately supported by national and trans-national government funding. Typically a Cochrane Review is considered the ‘gold standard’ with respect to medical meta-analyses but this doesn’t imply that Cochrane Reviews are infallible or will be interpreted correctly.
The Cochrane review used data that compared bicyclists who crashed and sustained head injury against bicyclists who crashed and did not sustain head injury, and then assessed helmet wearing rates amongst both groups.
The authors of the Cochrane Review did not collect any new data but instead relied on pre-existing data. Of all the academic data available about helmet use, the authors of the Cochrane Review found only seven studies that the authors claimed were:
“… conducted to the most rigorous scientific standards.” (CARRS-Q, 2011, page 11)
Four of the seven studies used were previous studies conducted by one of the authors themselves, and all the chosen studies supported bicycle helmet use.
The first study utilised in the Cochrane review related to 1,040 Emergency Department visits to a hospital in Cambridge, UK in 1992 (Maimaris et al: Injury patterns in cyclists attending an accident and emergency department: a comparison of helmet wearers and non-wearers, British Medical Journal 1994).
This study found that head injuries were suffered by:
- 8.1% of non-helmeted cyclists who owned a helmet,
- 9.2% of non-helmeted cyclists who didn’t own a helmet, and
- 3.5% of helmeted cyclists.
Even though helmets were not mandatory in the environment where this research was gathered, no attempt was made to investigate or exclude any other factors influencing certain cyclists choosing to wear a helmet that may also influence the likelihood of injury. For example, with no mandatory laws requiring helmets, it is very likely that people who voluntary choose to wear helmets are more risk-averse or belong to a data sub-set less likely to have a cycling related injury (for example age, riding location, riding environment, etc.)
The second study utilised in the Cochrane review related to 1,710 cyclists treated in two Melbourne hospitals during a period spanning 1987 and 1989, prior to the introduction of mandatory helmet laws. (McDermott et al: The effectiveness of bicyclist helmets: a study of 1,710 casualties, Journal of Trauma 1993).
This second study concluded that, prior to the introduction of mandatory helmet laws, 21.1% of helmeted cyclists suffered head injury compared to 34.4% of unhelmeted cyclists.
However this study also found that helmeted cyclists were more likely to collide with motor vehicles than non-helmeted cyclists (17.6% to 14.5%), and importantly that there were no significant differences in mortality rates between helmeted and non-helmeted cyclists.
This Melbourne study also identified an increase in neck injuries that is attributable to helmet use, but this is not mentioned at all in the CARRS-Q publication. This finding is important, because it gels with other more recent research discussed elsewhere in this series of posts.
This Melbourne study is also discussed further in the next post. It is also relied upon by CARRS-Q to further support its findings, despite the Melbourne study being one of the studies utilised in the Cochrane Review itself.
The third study utilised by the Cochrane Review related to 445 children cyclists under 15 years old who were treated at two Brisbane hospitals during a period spanning 1991 and 1992. (Thomas et al: Effectiveness of bicycle helmets in preventing head injury in children: case control study, British Medical Journal 1994).
However this Brisbane-based study included any injury to the skull or scalp in its definition of head injury. Any scratch or bruise no matter how inconsequential chalked up a result in favour of helmets. Brain injury was determined as instances where the child reportedly lost consciousness.
The fourth study was conducted by one of the authors of the Cochrane Review, and related to 668 cyclists treated for head injuries at hospitals in Seattle, USA during a period spanning 1986 and 1987. (Thompson et al: A case control study on the effectiveness of bicycle safety helmets, New England Journal of Medicine 1989)
This Seattle-based study also included superficial contusions, abrasions and lacerations in its definition of head injury, as well as skull fractures and brain injuries.
The fifth study was also conducted by one of the authors of the Review, concurrently with the above study, and related to 531 cyclists treated for facial injuries at hospitals in Seattle, USA during the same period spanning 1986 and 1987. (Thompson et al: Effectiveness of bicycle safety helmets in preventing serious facial injury, Journal of the American Medical Association 1996)
This study related to facial injuries only, not head injuries. The results of this study was that helmets had little or no protective effect on overall risk of facial injury, but did protect against serious upper facial injury.
The sixth and seventh studies, also conducted by the same author, were conducted concurrently from 1992 to 1994 and also related to cyclists treated at Seattle, USA hospitals for facial injuries and head injuries. (Thompson et al: Effectiveness of bicycle safety helmets in head injuries: a case control study, Journal of the American Medical Association 1996; Thompson et al: Effectiveness of bicycle safety helmets in preventing serious facial injury, Journal of the American Medical Association 1996)
Only one of the seven studies used in the Cochrane Review included deaths at the scene of the crash or those who were declared dead on arrival to the hospital. (Thompson et al: Effectiveness of bicycle safety helmets in head injuries: a case control study, Journal of the American Medical Association 1996).
All of the above raises three very major concerns with the Cochrane Review:
- The Cochrane Review mixes results from children-focussed research with all-age research. It therefore cannot be relied upon if children have a different propensity for injuries compared to other types of cyclists. Other research discussed in other posts in this series indicates this is true.
- The Cochrane Review cannot be relied upon if, where helmets are not mandatory, there are any significant differences in riding patterns between cyclists who voluntarily choose to wear a helmet and those that do not. For example, some people who choose to ride safely or in safer circumstances may also choose to wear a helmet.
- The Cochrane Review cannot be relied upon for any findings relating any effect that helmets may have on injuries serious enough to possibly cause death. Excluding the most serious injuries from the research will overemphasize the impact that superficial injuries will have on the findings. It will increase the statistical relevance of scratches, bumps and bruises.
The Cochrane review concludes that helmet wearing reduces the likelihood of head injury by 69% where a cyclist was involved in a crash with a motor vehicle, and 68% in all other types of crashes. Despite other research, available to CARRS-Q, discussed in other posts in this series arriving at significantly smaller numbers, this headline-grabbing statistic is the one CARRS-Q relies upon.
The Cochrane Review also states that over 90% of cyclist deaths are caused by collisions with motor vehicles.
From the reports above, it could equally be determined that people who are likely to own a helmet are the type of people less likely to have an accident. It could also be determined that if one wears a helmet, one’s risk of a collision with a motor vehicle is increased.
Very recent research by CARRS-Q has even confirmed this, finding that cyclists, notably teenagers, who choose to not wear helmets (despite mandatory laws) may elicit riskier behaviour and are ten times more likely to be involved in a ‘conflict’, such as swerving or emergency braking to avoid a collision.
While such findings may appear spurious, the potential for the Cochrane Review findings to support such conclusions is important. As will be discussed in the other posts in this series, there is additional significant evidence that reductions in head injury have more to do with helmet laws reducing the number of cyclists particularly risk-taking teenagers, than any actual protection helmets offer in the event of a crash. There is also additional significant evidence that wearing a helmet makes cyclists feel safer, thereby taking more risks, and that motorists take more risks when driving amongst helmeted cyclists.
One thing is extremely clear from the research used in the Cochrane Review, that is not highlighted in the CARRS-Q publication: these reports all support the contention that many, many more deaths and injuries can be prevented by taking steps to ensure motor vehicles don’t collide with cyclists, than could ever be prevented by helmets.
The CARRS-Q publication relied on other research as well as the Cochrane Review. Other posts in this series critically discuss much of this other research, and also highlights some of the research ignored by CARRS-Q.
“8.1% of non-helmeted cyclists who owned a helmet,
9.2% of non-helmeted cyclists who didn’t own a helmet, and
3.5% of helmeted cyclists.”So if they are using this difference to claim a 60%+ reduction in head injuries due to helmet wearing, then simply owning but not wearing a helmet reduces head injury rates by 14%. Those hats really are magic!