When is a Drug Safe and Effective Enough?
Friedreich’s Ataxia (sometimes shortened to FA) is a disease that slowly damages the nervous system over time. Often, FA results in difficulty controlling muscle movements, impaired sensory function, spinal curvature, and a host of other serious health concerns. Symptoms of FA usually begin during early or late childhood and last the duration of the victim’s lifetime (which is often shortened). While FA is rare (about 5,000 Americans currently have it), those with it can only receive treatment for symptoms.
Fortunately, the FDA granted fast-track designation for an experimental FA treatment named Omaveloxolone in 2022 after it demonstrated the ability to slow the disease’s progression in clinical trials. As an article published by the Children’s Hospital of Philadelphia summarized, “omaveloxolone has a potentially beneficial effect in slowing the progression of Friedreich’s ataxia out to more than two years... [and] it appears that this drug truly provides benefit to patients at all stages.”
The FDA has a different assessment.
After nearly a year of clinical trials, the agency determined that omaveloxolone’s ability to help patients was not clinically significant enough to advance in the approval process. Why not? The FDA’s standard cut-off level of significance requires a p-value of 0.01 or less (indicating a 99 percent chance results of treatment were not due to random chance). Results from omaveloxolone’s clinical trial resulted in a p-value of 0.014.
That 0.004 point difference means omaveloxolone’s manufacturer must either perform additional clinical trials in hopes of reaching the desired level of significance or abandon hope for approval. Both options prevent thousands of patients with FA from accessing their best hopes to control the disease and live a better life.
Is the difference between 0.01 and 0.014 a big enough difference to withhold beneficial but somewhat unproven treatments from reaching suffering patients? It mostly depends on who gets to decide.
As a bureaucracy tasked with determining how safe and effective drugs and other treatments need to be before reaching patients, the FDA has little reason to relax its standards. Doing so increases the risk of approving a potentially ineffective or harmful product—which can create a public backlash. Consequently, the FDA decision-making process emphasizes the risk of harm or failure rather than the potential benefits of new treatments.
Conversely, patients might strongly consider receiving a treatment that might not help or even harm them depending on their state of health and other factors that aren’t considered in aggregated cost-benefit analyses performed by regulators. As economist Robert Higgs explained in his woefully underappreciated article entitled Banning a Risky Product Cannot Improve Any Consumer’s Welfare (Properly Understood), with Applications to FDA Testing Requirements, different risk assessments between regulators and patients only exemplify that choices contain an inescapable element of risk. He also wisely notes:
To ban a good because a third party believes it to be in some sense riskier than the consumer believes it to be or because a third party values risk-avoidance more than the consumer does is simply to impose the third party’s preferences on the actual consumer.
Different p-values from clinical trial results can be the difference between statistical significance and non-significance. Based on a 99 percent significance threshold, omaveloxolone came up short.
But questions of what level of significance should be standard or when a drug’s potential benefits outweigh its potential costs are matters that have to be answered by people. And the different willingness to accept risk depends on personal circumstances. P-values cannot capture these considerations—neither can the FDA.