Can New Test Be Trusted?

Arenz booked his free MRI.

The results left him heavy: stage I kidney cancer. The Galleri test had missed it.

Arenz received his free Galleri test through a cancer screening program funded by the city of Mesa at Vincere Cancer Center in Scottsdale. Under the leadership of radiation oncologist and Vincere co-owner Vershalee Shukla, MD, the program screens first responders in more than 10 Arizona cities at no cost to them.

Vincere began using Galleri shortly after the test launched for consumers in June 2021. Since then, the first responder program has become the largest commercial user of the test in North America.

But Galleri’s ability to identify cancer and, perhaps more importantly, the fallout from wrong results, have been under scrutiny since the test’s launch. Galleri, which has not yet been FDA-approved, is still so new that few know what incorrect results look like in practice and how often they might occur.

“Cancer screening is a very complicated issue,” she says. And “being honest, the tests are good but are not ready yet [for wider use].”

Arenz was not the only firefighter who faced a surprise after taking a Galleri test.

In nearby Phoenix, 51-year-old Mike Curtis knew his risk for cancer was high, but he wasn’t that worried. Curtis had been running into fires since he was 17, and his dad, also a firefighter, had died of cancer at age 58.

He’d taken the Vincere Cancer Center up on every free screening service since the program began in late 2018 — well before Shukla started using Galleri in 2021. His most recent lung scan was clear. But he did the Galleri test just to stay vigilant.

His result was a shock. The test detected signs of cancer.

Curtis decided to tell no one, not even his wife. He’d bear the bad news alone until he was certain.

Shukla, however, doubted the blood test results right away. She rushed to do several follow-up tests. One week, and scans of Curtis’s belly and pelvis later, her hunch was confirmed. The Galleri test was wrong, Curtis did not have cancer.

The price of his peace of mind: an extensive work-up with a $4,000 price tag. Fortunately, the bill was covered by the screening program.

Overall, in just over 18 months of using the blood test, Shukla has only encountered one other false positive out of about 2,000 Galleri results.

She also discovered two positive signals for cancer using Galleri, which were confirmed with follow-up tests. One was a chordoma, a rare type of bone cancer, and the other was a squamous cell carcinoma of the head and neck. Galleri caught both remarkably early, in time for treatment.

But for Shukla, false negatives were particularly “horrible.” Arenz’s was just one of 28 cancers that the blood test missed. And because 500 negative tests are yet to be validated, the 28 false negatives may be an underestimate.

Grail Senior Medical Director Whitney Jones, MD, agrees that the test is not meant to be a standalone when screening for cancer. The goal of Galleri is to “complement other screenings, not replace them,” he says.

According to an analysis of Galleri data and Shukla’s experience, the test’s specificity was over 99%. That means the test successfully minimizes false positives.

But the test’s sensitivity was much lower. Shukla’s data in first responders measured the sensitivity to be 6.7%. That means the test misses about 93 out of every 100 cancers. According to Grail’s latest data in more than 6,300 people over age 50, the test’s sensitivity was 29%.

Specificity and sensitivity are metrics used to credential a test and demonstrate confidence in its ability to detect the target disease. A test with high specificity can correctly identify patients without the condition in question, while a test with high sensitivity can correctly identify patients who do have the disease. But there are tradeoffs between sensitivity and specificity. One value is increased at the expense of the other.

It’s normal for a cancer screening test to prioritize specificity, says Aparna Parikh, MD, an oncologist at Massachusetts General Hospital Cancer Center in Boston. In a test like Galleri, which is meant to be used with other screening tests, “at least we are seeing a good specificity, which is important because we don’t want false positives, where the downstream impact on the patient can be high.”

Overall, Jones says, Grail Bio’s aim is to build a test that’s sensitive enough to catch the most dangerous cancers without flooding the health system with false positives. In addition, he says, the sensitivity varies by cancer type and tends to be lower for cancers that already have other available screening tests as well as for earlier-stage disease.

While there’s no definitive threshold for sensitivity, miss rates as high as 93% and 71% “provide little confidence in the [accuracy of the] test,” Lee says.

Positive and negative predictive values, however, are more clinically relevant measures of a screening test. These numbers inform how likely a patient’s results are true, and therefore how worried they should be about a positive result and how much they should trust a negative .

In first responders, Shukla found that only half of positive Galleri tests were confirmed cases of cancer. And an analysis of Grail’s data found only 38% of positive Galleri tests – 35 of 92 people – had a validated cancer diagnosis.

In the general population, if only 38% of positive Galleri results truly have cancer, the test is “not quite useful to make a decision for the patient or the providers,” Lee says.

Galleri may also be a costly prospect for patients, no matter the result, says Electra Paskett, PhD, an epidemiologist and cancer screening expert at Ohio State University. A positive Galleri test requires a cascade of follow-up diagnostic tests without the promise of payer coverage. For a negative result, Galleri recommends the patient be screened again in a year, an annual cost of $950 plus any follow-up testing when Galleri does pick something up.

“If a provider wants to offer the Galleri test, all those things need to be made abundantly clear, in my opinion,” Paskett says.

After a negative Galleri test, Arenz’s cancer didn’t slip through the cracks because he received other advanced imaging free of charge. But whether all doctors will go to such lengths to back up Galleri results, even in negative patients, is unknown.

A negative result can give patients “a huge false sense of security,” says Shukla. And if a test is positive, the workup isn’t simple, she says. Chasing cancer, especially one that’s not really there, can be nerve-racking and expensive.