Product launch (finally!) and the science
This post is about science and bad science.
But first, most of you reading this already know that three days prior to this post, BlueSkies (where I am the founder) finally launched our product on indiegogo. Some of you may have seen our post featured on the asthma blog of Dr. Ann Wu, a pediatrician at Boston Children's Hospital and a Public Medicine researcher at Harvard Medical School with a focus on asthma. Pretty much she is awesome. You can read some of the science details there, and I won't rehash the links. But please go to the indiegogo campaign and support and share it if you haven't already!
Science and bad science.
Traffic pollution almost certainly is one of several causes of asthma. Journal article after journal article has shown this. Yet some journal articles somehow show that traffic pollution doesn't cause asthma. What gives?
Bad science can be alright science that disguises itself as great science, leaving lay readers to not know what is just alright, and what is great.
Here is my favorite thing: ripping apart people who don't know what they are doing, but they like to write like they do.
The best example is this article here that showed no link between air pollution and asthma rates.
Except they literally didn't measure air pollution during the study. They did the study in 1994-2008, and then measured air pollution in 2009. They used 2009 data to estimate pollution levels in the years prior. This is bad science. But they did the science, so they should publish it. They should have put caveats in the abstract. But no one should be citing this study. Because it wasn't a great study.
Better studies measure air pollution directly, and then compare it to asthma rates. Every well-conducted study I've seen shows a link between air pollution and asthma rates. Every study I've seen published that doesn't show a link... well, these all have gaping holes in the science.
Not measuring air pollution, and then trying to associate air pollution levels with asthma rates, is the equivalent of saying "The economy in the US in 2011 was not great. I see we have low unemployment rates. It must be because of that, because housing prices have been steady for the last year," when we all damn well know that the housing bubble, fueled by wall-street repacking the housing debt, caused the collapse.
Conclusion: this paper was just okay. It should never be cited when there are papers that actually measured pollution when they wanted to know what the pollution levels are.
How is a non-scientist to know which of these articles are correct?
You can't. This particular detail of the research seems super tiny. The researchers themselves probably didn't realize it was a problem because they are not atmospheric chemists. If you send the articles to a pediatrician studying childhood asthma, they'd probably not pick out the distinction I just made between these studies measuring vs. modeling pollution. The only reason I can do it is because I more or less have a PhD in measuring vs. modeling pollution. The only way for you to know it is wrong is to go to an expert and ask them.
One final, very confusing note
Some studies measure pollution directly in one area, and then adjust around that measured pollution level by using models. Say, for instance, I am measuring air pollution near a freeway in New York City. I know that traffic is producing pollution there, and I also know that Central Park has no traffic (cause it's a park)
The result? If I were making a map of air pollution in NYC, and I was only measuring pollution from near highways, I would know that I had to lower the pollution in the park. I would measure the pollution in the park at several different intervals, and then make a model of why it differs from the pollution near the highway. The key is that I've measured both at the highway, and at the park. I measure at the highway always, I measured at the park several times, but I could not afford to measure at the park always. I create a relationship. This means I have a good understanding of pollution levels in the park, even when I'm not measuring it, because I know how those levels relate to pollution near the highway. If I measured both at all times, I'd have a strongly good understanding of pollution levels in both places.
Yes, strongly good is not a real phrase.
Thanks for reading!
- Jason Munster