Why we haven't made breakthrough medical progress in the internet era, part 2: clinical trials
Success rates are falling, but some pharmacos outperform and the disruptors are coming
Continuing on my series on why we haven’t made breakthrough medical progress, after an investigation of funding issues, my focus turns to clinical trials: starting with some definitions, moving on to current trends, the drivers of success and failure, some outlier performers in clinical trials, and the disruptors looking to change the game.
Clinical trials are the human testing phase gates that all medications/drugs need to pass before being approved by the FDA for sale in the US market.
Trends and problems in clinical trials
There are a lot of problems and reasons drugs fail across clinical trial phase boil down to four major reasons per recent University of Michigan and BMS research: lack of clinical efficacy (40-50% typically in phase 2/3 trials), unmanageable toxicity (30% typically in phase 1 trials), poor drug-like properties (10-15%) and lack of commercial needs and strategic planning (10%)
Even if a drug gets through all phases successfully, it takes on average 10+ years from a phase 1 clinical trial initiation to approval.
As larger prevalence disease areas (“primary care” diseases like heart disease, diabetes, etc.) have been addressed by existing approved drugs, pharma companies have to go after harder to treat niche (i.e. lower prevalence) diseases, which can yield lower ROIs at an opportunity cost weighted average R&D cost of $3B per approved drugs and a declining overall likelihood of approval at an average of 7.6% over the last 14 years (the recent uptick in 2023 rates may be inflated due to boundary effects).
Delays in clinical trials are quite common with 80% of clinical trials failing to meet their enrollment timelines, and with an average cost of $37K / day to operate a clinical trial and $8M in lost revenue per day (assuming an opportunity cost of an approved $3B revenue drug) [Source]. Clinical trial enrollment delays are extremely costly.
Why does the FDA matter so much? Can’t pharma companies just go to another geography or regulatory authority to get an approval? Well the FDA is considered to be the top regulatory body in biopharma around the world, and its approvals hold the keys to the largest market for drugs in the world by near an order of magnitude in the US at $631B in annual sales (as of 2022).
While overall success rates are declining, the breakdown by disease areas shows some signs of hope though a majority of industry focus has been on Oncology, and success rates have been going down across most conditions when comparing 2021 rates to 2016.
Some of the biggest innovations in technology and modality / mechanisms of action have helped to drive higher success rates, particularly with in Oncology with CAR-T therapeutics, a type of immunotherapy using modified T cells to treat blood cancers, at a 3x overall success rate compared to Oncology as a whole. Another hot development in oncology delivery mechanisms, ADCs or antibody-drug conjugates, have a 2x overall success rate compared to Oncology as a whole.
Ultimately what matters to the health of innovation in the biopharma industry is the successful launch of novel active substances. New launches are relatively flat on a 5 year horizon, and slightly up on a 10 year horizon, with a plurality of launches in the Oncology space in recent years.
Success & cost drivers
What drives the success of clinical trials? Based on the above data cuts, there are obvious areas like the drug indication disease/therapeutic area (oncology at 5.5% significantly underperforms metabolic at 15.5%) and technology (CAR-T, with a low sample size, at 17.3% significantly outperforms antisense tech at 5.2%).
This intuition is verified by machine learning research like that conducted by Andrew Lo, Kien Wei Siah and Chi Heem Wong from MIT and separately by BIO & QLS Advisors (a hedge fund formed by Andrew Lo to trade on alpha from clinical trial prediction models). Below is a feature importance chart showing indication is unsurprisingly the most important factor in predicting phase approval success rates, and a feature importance table from the MIT research showing importance of features like trial outcome (obvious) and actual patient recruitment (accrual). Some features that are a bit more interesting are the trial sponsor’s previous success, an area I will explore in the next section.
Unfortunately, as is the case in much of the biopharma world, there is a high rate of missing data as can be seen by the below table from the MIT research.
Some recent attempts to get over missing data include imputation via traditional ML techniques and use of LLMs like Mistral-7B to enrich features based on learning from existing trial information like RPI and Illinois researchers did with TrialEnroll in order to predict trial enrollment success; remember actual patient enrollment (accrual) is a key predictive factor in clinical trial success as determined by the previously referenced MIT study. An ablation study on trial enrollment features showed that inclusion criteria count (criteria for enrollment) and a lower maximum age resulted in worse performance in enrollment, which is probably an intuitive result.
New datasets like clinical trials outcomes (CTO) dataset will probably lead to much more robust ML predictive research augmented by LLM predictions on trial outcomes based on research abstracts.
On a related, but separate topic, costs are a primary consideration in trials. While not representative of current absolute costs, a 2016 study from the US DHHS shows that relative cost drivers for clinical trials are primarily driven by clinical procedure costs (~20% across phases), administrative staff costs (11-20%, increasing by phase), and site retention costs (~10% across phases).
Outlier performers in clinical development success
ML models cited above show that trial sponsor success can breed more success. Two outlier pharma clinical development performers Alnylam and Vertex, have both returned 3-5x to shareholders in the last 10 years with incredible clinical development success rates (62% overall for Alnylam, 85% rate of advancing drugs to phase 3 for Vertex).
As of December 2022, Alnylam has an astounding 62% overall clinical development success rate. Depending on your point of comparison this is almost 10x the rate of success overall across all therapeutic areas/modalities, and 4.6x the rate of RNAi success. Alnylam leans into genetic evidence screening of patients and its RNAi technology that regulates gene expression. As noted in the Trends section above, RNAi (and siRNA) therapeutics have an overall 13.5% likelihood of approval based on candidates advanced via clinical development. Technology advantage and specialization are key success drivers of outlier performance, but it is also hard to draw out success factors in an explainable and causal impact way like ML models can. Founding CEO John Maraganore has a very compelling account of how Alnylam was able to create a RNAi behemoth.
Vertex, through recent R&D strategy decisions by former CEO Jeff Leiden, has been able to consistently outperform peers across a variety of therapeutic areas. It has a 85% success rate of advancing to phase 3 clinical trials (!) and attributes much of its success to a disease-first approach to its R&D, where the causal human biology and pathway is well-known. Vertex initially was known for Hepatitis C success, and subsequent failure, but reclaimed success in Cystic Fibrosis approvals via a robust patient foundation collaboration (Kalydeco, with the Cystic Fibrosis Foundation), innovations in bringing the first approved CRISPR medication (Casgevy, with CRISPR Therapeutics) to market, and more therapeutics across more niche conditions.
Outside of these outlier performers, some under-performers like Pfizer have made 10x gains in clinical development yield rates, increasing their overall clinical development success rates from 2%(!) to 20%.
Sound strategic planning, experienced management and a technical edge are all key drivers of success in the current FDA - clinical trial industrial complex, but what can be done to disrupt the status quo either by improving performance in the system or find a new way to bring drugs to market outside of the system?
Questions to challenge the status quo, and the disruptors trying to do so
Why should patient recruiting take so long and fail to meet targets, and operations cost so much? Several startups are looking to address these recruitment issues like Power, Verily, Formation Bio, Vial. As mentioned in my previous piece on the TechBio idea maze, the latter two companies Formation and Vial are taking their (perceived?) strength in clinical trial operations to become clinical development companies that in-license or develop their own drugs more effectively.
Why do clinical trials need to be held at specific academic medical sites? Decentralized clinical trials are one attempt to change the game by enabling remote patient enrollment and decentralized clinical trial operations including Science 37, Hawthorne Health, and others. This can lead to faster enrollment, higher retention, and reduced costs via geographic diversity.
Why do we need to wait for INDs to allow patients to try drugs that are considered generally recognized as safe for conditions they can’t or won’t wait 10 years for an approved drug to emerge? Biohacking may be a new trend that becomes mainstream over time per Shelby’s recent exploration, and with more permissive right to try laws in various states like Montana but also including the right to try for people to try drugs that have passed phase 1 trials across 41+ states. Repurposing is an area I’m particularly bullish on as small biotechs like Viking Therapeutics and nonprofits like Every Cure use ML techniques to discover existing drugs that could be used for indications they were not originally approved for. Maybe we even need a new place for biotech innovators to run trials and give access to medications in permissive regulatory environments like Vitalia in Roatán, and Dubai in the future.
Why do we even need clinical trials, can’t we simulate human response to drugs via computation models? This is a hard question to answer, as human biology simulation is incredibly complex. Not only do we not have the right data, at the right precision/resolution and diversity, we also don’t have the available compute to model such a complex multi-organ system as humans. The most recent work in this space is on virtual cells, check out a good overview here from CZI. There are also some fascinating ways to visualize the underlying complexity of human biology using tools like Mol*. I think human biology simulation is an under-invested area, and there needs to be more investment in measurement advancements (beyond Cryo-EM, etc.), longitudinal enriched data studies over very long periods of time, spatial biology, network systems biology and more. If we can simulate a human body and its various diseased and normal states, why would we need so many clinical trials? There are some (more rudimentary) advances with clinical trial digital twin studies, which help to reduce recruitment requirements and improve success rates as Unlearn has been able to do.
Why wait for cheaper medications when you can make them? Compounding medications that are still on patent, but where shortages exist, offers an interesting avenue for decreasing patient cost for high cost brand name medications. One example is GLP-1 compounding and Hims & Hers (among others like Sesame), where GLP-1 ingredient shortages have persisted including for Lilly’s GLP-1 agonist tirzepatide, though may be short-lived.
If we can predict clinical trial success, is the most market efficient path to trade in the public markets on this information? In some ways, yes, and as mentioned above Andrew Lo has done this with QLS Advisors ($7.2M AUM reported, likely much larger), and via a collaboration with BlackRock and renowned scientists / entrepreneurs to bring systematic investing to LPs and retail investors. This is a highly attractive approach especially for single asset biotech companies where an approval can make or break that company’s fortune. Cash earned from trading can in turn be used to spin up biotech companies, however may be less disruptive than other approaches listed above and fewer biotech companies are able to raise money in the public markets to fund clinical trials.
Thinking ahead to next time: understanding the cause of disease
Despite the challenges in clinical trials and funding overall, the last big stumbling block to medical breakthroughs that I will explore in my next post is our understanding of the causes of disease and the perturbations that can improve someone’s diseased status to [less diseased, normal / cured]. The biggest diseases we have yet to solve include cancer and neurodegenerative diseases like dementia / Alzheimer’s, and chronic diseases of aging overall. Even when we know most of the mechanisms, diseases can evolve to constantly outmaneuver our best weapons like in the case of cancer. Sometimes we lack a fundamental understanding in how a disease forms like in the case of dementia, where the broader biotech and science community have been futilely chasing long-held beliefs such as the beta amyloid plaque hypothesis. We have found ways to treat metabolic diseases like heart disease and diabetes, but so far have very few avenues to reverse their course and cure ourselves of these common (slow) killers and diseases of aging. Maybe AI can help us, but maybe we need a more comprehensive understanding and measurement of how our bodies work.
Great article! Keytruda isn't a CAR-T therapy though fyi