The "Groundbreaking" Trans Study That Wasn't
No amount of studying will erase men's competitive advantage in sports.
Did you guys hear about this “groundbreaking” new study? It’s a systematic review, actually. Complete with a meta-analysis.
Turns out, if you roll up dried cat shit and smoke it, it can cure diabetes! Isn’t that wild?
I mean, okay, maybe the foil shouldn’t come off our champagne just yet...
The “meta-analysis” didn’t have much data to work with. Almost nobody has ever tried rolling up cat shit in a blunt wrap and smoking it. Almost none of the people who have have diabetes. It’s also not clear that any of them were “cured,” or that cat shit was the thing that cured them.
A lot of the studies reviewed lacked a control group. Since few smoke cat shit to remedy their diabetes, it was hard to form a cohort of cat shit smokers to compare with non cat shit smokers.
Besides, “curing diabetes” isn’t technically possible. All you can really do is track other things happening in the body if you want to see how well cat shit manages the condition. For example, diabetes causes extreme thirst and frequent urination. It also causes fatigue, effects vision, and slows healing.
One study in the meta-analysis found that participants who smoked cat shit were less thirsty, and peed less frequently. Another found that people could see better, had more energy, and healed faster (granted, this was all based on self-reports). If these things are true though, it follows that smoking cat shit must cure diabetes, since it seems to ameliorate all the symptoms.
That’s how it works, right?
Or does our “groundbreaking” new cat shit study have some flaws?
Drawing on David Hume as an influence, astronomer Carl Sagan coined an adage that became known as The Sagan Standard: “Extraordinary claims require extraordinary evidence.”
“Smoking cat shit cures diabetes,” would be an extraordinary claim. Are the data we’re using to support this claim also extraordinary? Are our findings “groundbreaking?”
Or actually, is our evidence rather weak? Is it really evidence at all?
Trans Women In Sports
Here’s another extraordinary claim:
“Biologically male bodies do not have a competitive advantage over biologically female bodies in sports.”
This claim is extraordinary because it flies in the face of observations that are so obvious, we don’t need academic studies to record them. We watch the Olympics. We watch the NCAA. We watch high school volleyball. If boys and men didn’t have a sports edge over girls and women, we’d have about 600 ways of knowing that, and none of them would require peer review.
And yet, Spanish language newspaper El Pais is touting a “groundbreaking” new study that claims to have turned our observations on their heads:
The article’s subject is a Brazilian paper published in the British Journal of Sports Medicine: Body composition and physical fitness in transgender versus cisgender individuals: a systematic review with meta-analysis.
We’re going to pick through this paper, and trace its journey from root to flower. It’ll offer us an excellent glance at how real scientific data get misused by bad faith researchers, who launder it through a gullible, ideologically-captured press, and spread it via gullible, lazy internet posters.
It feels to me like a lot of people don’t actually bother reading, or even reading about these things when they drop. The person I got this from had plainly not skimmed past the headline, which did not stop him from roasting the bigots for failing to take heed.
Folks don’t dig in. Either because the athletic advantages to being biologically male are clear to them without a study, or because they’re zealous ideologues who don’t care. It’s a shame that more don’t peek behind the curtain though.
The cobbling together of these nonsensical papers represents a scandal in its own right, and one that far outshines the presence of biologically male players in women’s sports.
These are the people we count on to tell us which medicines to take. They tell us which cancers can kill us and which can be treated. They tell us how. They tell us what we can do to stay healthy, and what will cause us to nosedive. For most of us, there may be no more consequential field of scientific research.
If the field is this easy to compromise - if all it takes is a little social pressure and doctors will tell us that men don’t reliably outperform women in athletics; that smoking cat shit is good for us, actually - how can we trust them?
How can we trust anything?
The Role Of The Media & Tech
The scientific community doesn’t bear all of the blame for this unfortunate set of conditions. They couldn’t work their dirty magic without a major assist from a friendly media that knows how to game search engine optimization (SEO) to grow its reach.
“Groundbreaking,” for example, is not a word used in the BJSM study. It was an editorial choice made by El Pais, and it’s one that mattered.
Just over a year ago, I wrote a piece lamenting the current of dishonesty running through the trans sports debate. In it, while examining another garbage “study",” I discussed the one-two punch of media + tech in misleading curious bystanders:
What I have done is what I thought any ordinary layperson might do if they heard that “studies” proved it was okay for trans women to compete in women’s sports: I Googled it.
I looked for the “studies,” and this is what Google’s algorithm thought I should see. Which is pretty unimpressive. Google has about a 90% market share in the world of search engines, so if people are asking for scholarly research, and Google is showing them research that boasts poor scholarship, it’s a problem.
And unfortunately, things aren’t much better if we check in with the NIH, the ACLU, Human Rights Campaign, Gender Justice, or the NIH again.
Seriously, go down the link rabbit hole on any one of those. You will find no serious study anywhere that backs up the claims made in the headlines, and no indication at all that this issue is really much ado about nothing.
How, I wonder, has the explosive language used in El Pais’s headline impacted what laypeople will find if they search this topic now. Let’s find out:
When I ran a search that explicitly used the word “threat,” (also used in the El Pais subtitle) Google returned the El Pais article as the number two entry. When I ran a more neutrally-worded search - “trans women in womens sports” - it was still in fifth place.
So a sloppy, weak paper, benefiting from a sloppy, mendacious write-up is now topping the charts for anyone who wants to learn more about this subject.
El Pais itself follows an all too familiar pattern in their coverage. Bold claim, followed by an admission that the claim couldn’t really be substantiated, followed by a suggestion that the claim be accepted anyway, because bigotry.
Every time.
“This refutes the logic behind blanket bans on transgender women in sports,” argues Bruno Gualano, a physician and researcher at the University of São Paulo in Brazil, who co-authored the study.
…
“The 52 studies on which this analysis is based have different designs and methodologies, Gualano acknowledges. The body of scientific evidence, therefore, is not entirely conclusive and is of heterogeneous quality. “It’s not perfect, but it’s the best scientific evidence available.”
…
The expert calls for analyzing the debate within a broader context, taking into account the exclusion and violence faced by the trans community. “We believe the debate should be guided by values fundamental to sport itself, such as fairness, inclusion, and human dignity, rather than sweeping bans,” he concludes.
I’ve been around this block enough times to know that there was no way 52 studies could have had good enough data to underpin a systematic review about trans athletic performance. And indeed, there wasn’t. So what then is included in the 52? El Pais keeps quoting people who complain about the paucity of trans athletes, so how do we get to n=6,485?
Measuring Athletic Performance
As we’ll see, we get there by including in our systematic review an enormous number of studies that don’t actually look at athletic performance. Per my own scan of the studies’ objectives, of the 52, only 8 were explicitly about sports or exercise, and I’m counting two questionables:
One looked at fitness testing for trans service members (Chiccarelli, et al 2023). It’s not about athletic competition, per se, but I think it’s fair to count it. It reviews benchmarks like sit-ups and push-ups per one minute, and mile run times. A lot of the studies in the review are like this. They measure things that are certainly adjacent to athletic performance, but that don’t directly answer the question of whether overall, there’s enough of a reduction in men’s performance after “gender affirming hormone therapy” (GAHT) to fairly allow trans women into women’s sports.
Another study that I perhaps shouldn’t have counted towards our eight is Yamada, et al 2023, which looked at the effects of aerobic exercise on arterial stiffness in patients who received GAHT. I included it because it was directly about exercise, just not about whether being trans helps you perform it better. And anyway, it dealt with trans men, who really aren’t at issue in the debate over athletic fairness.
My mini-review was actually a decent microcosm of a problem inherent to this body of research. We have a specific question: is it fair for trans women to compete in women’s sports? We have a problem in answering this question: not many people are trans, and not many trans people are athletes. How do we reconcile this?
What most of these meta-researchers do is, they look to characteristics that they think would probably help a person perform well at sports: strength, lung capacity, fat distribution, bone density. Then they make the intuitive leap that these metrics, when stacked together, act as a kind of stand-in for overall athletic ability.
But do they? Is that really what these metrics can tell us?
It’s not that these are bad proxies for athletic potential, but performance in an actual sport requires a complex, multifactorial interplay between them. Being strong is probably better than not being strong, but are the strongest baseball players always the heaviest hitters? Do the strongest quarterbacks have the best arms?
The reason it’s so important to actually, directly study sports is because real ability on the court or field can’t be predicted like this. You can’t Frankenstein your way to a star athlete just by fitting them with the necessary parts.
There’s also a lot missing here. What about frame size? What about limb proportion? Joint angles? Balance? Center of mass? Coordination? Hand size? Foot size? Grip surface? Psychological fortitude?
There’s no reason at all to think that GAHT can change any of these characteristics, which is presumably why none of the studies in the systematic review looked at them. They absolutely matter though. Without them, we’re back to claiming that cat shit cures diabetes, because somebody said their foot stopped tingling after smoking it.
Oh and by the way, a lot of the studies don’t even say what the analysts say they say.
Download the full list of studies covered by the BJSM review here: Link.
Pants On Fire
Behold this opening paragraph from El Pais:
The inclusion of transgender women in women’s sports has become a battleground in a larger culture war. Positions are often based on ideological or moral convictions. However, a medical and scientific debate also underlies this issue — a debate that is now closer to being resolved. A scientific team from Brazil has conducted a meta-analysis encompassing 52 studies and 6,485 participants, analyzing the body composition and physical fitness of transgender and cisgender women. While transgender women showed greater lean mass — indicating greater muscle mass — they did not exhibit greater physical capacity, such as strength or aerobic fitness, than cisgender women. (emphasis mine)
This is a straight-up lie. Right out of the gate.
This is a lie about the exact subject the article is covering. It’s the author gazing up at a clear blue sky, then returning to his laptop to type, “The sky is orange.”
We’ll get to what makes this a lie in a moment, and we’ll explore some ways we could change it to make it not a lie. But first, we should review just how shaky this ground is already.
