Theranos Facts Oct. 22 2015 press release by Theranos Lyrics
On October 15 and 16, The Wall Street Journal published two front-page articles about Theranos. We have heard the questions that have followed these stories and want to provide answers.
As we provide these answers, it is important not to lose sight of a few simple truths:
Theranos' technology is reviewed by regulators, proven in the field, and praised by leaders in the industry and doctors and individuals that we serve.
We provide blood tests faster, requiring far less blood and patient discomfort, than for any test previously available—or available today from any other laboratory.
We have run more than 3.5 million tests, with patient satisfaction scores from tens of thousands of patients that consistently rate Theranos, on average, over 4.8 out of 5. To our knowledge, we are the first laboratory to publish our prices, lab proficiency-testing scores, customer satisfaction scores, guest visit times, and more on our website.
We are confident in the reliability of our tests, because we comprehensively validate the accuracy of every test we run. In addition, we are the only laboratory that has committed to submitting all of our laboratory-developed tests, including our technology, procedures, and methods, to FDA for review and clearance. FDA has already cleared one of those tests, including our underlying test systems and Nanotainer™ tubes, for use in detecting the herpes simplex virus (HSV-1). The decision summary, which describes the rigorous science behind the clearance, is available online.
The faster speed and lower cost of our tests mean that consumers, and their doctors, can monitor their health so that problems are identified early enough to be addressed by medical professionals. That lower prices can reduce the cost to insurers, Medicare, and Medicaid by billions of dollars, and can give individuals access to critical health data they could not otherwise afford.
Our proprietary devices are making it possible to run finger-stick samples for tests that could never be run on finger-stick before. We began using our proprietary devices in our lab at the launch of our retail operations. And we initiated filings with FDA two years ago—by choice, not necessity—because we are seeking to create a new model for laboratory testing standards, and have championed FDA oversight ever since. It is the right thing, which is also the hard thing.
We are the same company offering the same services as we were last week.
Now, let us tell you what the reporter got wrong and why.
What The Reporter Got Wrong—And Our Responses
The stories say or imply many things that are simply wrong. The following are examples of the reporter's incorrect points—and the truth.
Accuracy and Reliability: The stories suggest that Theranos' tests are not accurate. That is wrong: Theranos' tests are accurate and reliable. Here are the facts.
Satisfied customers: Theranos has provided accurate and reliable tests to tens of thousands of satisfied customers. You can see testimonials from some of those guests on our website: https://www.theranos.com/guest-stories
We gave a selection of testimonials in writing to the reporter on July 1, 2015.
FDA's clearance: Just this past July, FDA cleared Theranos' finger-stick test for HSV-1, which included a rigorous review of our underlying test systems and finger-stick technology.
In order to clear that technology, FDA rigorously reviewed our test systems and Nanotainer™ tubes. To view the studies that show the accuracy and reliability of Theranos' HSV-1 test, take a look at FDA's decision summary, available here: http://www.accessdata.fda.gov/cdrh_docs/reviews/K143236.pdf.
The FDA's decision provides independent validation of the ground breaking Theranos System upon which the HSV-1 test is run. FDA's decision summary describes how Theranos' clearance is for use for HSV-1 "on the Theranos System" and performed on "fingerstick ...obtained with the...Nanotainer™ Tubes." See http://www.accessdata.fda.gov/cdrh_docs/reviews/K143236.pdf, at page 2, under "Intended Use," et seq.
The studies provided to FDA were detailed and rigorous. Among other things:
Theranos provided comprehensive data to FDA on our test systems and methods, the chemistry, the hardware, and the software in the company's foundational system and Laboratory Developed Test (LDT) application.
Theranos provided study data from 818 subjects of varying age and ethnicity, demonstrating that our devices could be run accurately using only a finger-stick as well as a traditional venous draw across large numbers of Theranos devices, all compared against an FDA cleared, commercially available reference method.
Theranos' HSV-1 studies included matrix comparison studies across 69 Theranos devices. Use of such a large number of devices in the studies is unique, as such studies are often run on a single or small number of devices.
The larger number of devices allows for comprehensive characterization of variation across devices, which can be a source of variance in data across laboratories. Theranos' HSV-1 clinical studies also included sensitivity and specificity studies with hundreds of human samples – both capillary and venous, CDC panel testing – which demonstrated 100% agreement with the results provided by CDC, low-prevalence population tests, and other studies.
Theranos also completed full submissions for our HSV-1 IgG test on venous serum, venous plasma, capillary plasma and capillary whole blood, to demonstrate equivalence on our system across the different sample types, in contrast to conventional tests which have only been cleared for venous samples.
We told this information to the reporter on July 3, 2015.
The reporter did not ask a single question about Theranos' FDA clearance or CLIA waiver.
CMS and CLIA: To earn and maintain CLIA certification, Theranos must ensure the accuracy and reliability of our tests. CLIA's purpose is to ensure quality laboratory testing and ensure accurate and reliable test results.
Theranos is a full-service laboratory that is subject to regulation by the Centers for Medicare & Medicaid Services (CMS). CMS regulates all laboratory testing (except research) performed on humans in the U.S. through the Clinical Laboratory Improvement Amendments (CLIA).
Before we offered any test to anyone, anywhere, regardless of the machines or methods used, we had to scientifically verify the accuracy and reliability of that test. We have done that for each and every one of our tests.
Theranos publishes our proficiency-testing results and certain validation data on our website. To our knowledge, none of our competitors are willing to be so transparent. See https://www.theranos.com/our-lab.
We told this to the reporter on several occasions, including July 3, 2015.
People who know: In addition to our regulators, our business partners have spoken in support of Theranos' technology.
We gave quotes and sources to the reporter over several occasions, including on July 8 and July 17, 2015. He did not use any of those quotes or sources in his article.
Finger-stick tests in Theranos' clinical lab: The article implies that Theranos said that all our tests are performed with finger-sticks. That's not true: We have consistently said in public statements, on our website, and to our customers in Wellness Centers that some of our tests are performed on venous draws, in part because we have been becoming a full-service laboratory. In December 2014, Theranos offered more than 80 of the tests on our online test menu via finger-stick. Also, of the finger-stick tests that Theranos offered, all ran using proprietary technologies—technologies that are not commercially available—except a few routinely used point-of-care tests. Here are the facts.
Finger-stick tests: Because Theranos has been working to transition to an FDA framework, we have moved from use of our laboratory-developed tests (LDTs) to FDA-cleared and approved assays. At the same time, Theranos has been increasing the number of tests we offer, in order to become a full-service lab. Theranos offered tests on venous samples—run in our own laboratory—from the time we launched, and have expanded that capability to be able to offer rare and specialty tests at our unprecedented low prices, as those are the tests that often cost the most. Additionally, because of this offering, most patients would not have to go to multiple labs to fill lab orders.
As Theranos transitioned to the FDA framework, and added specialty and esoteric tests to our laboratory offerings, the percentages of guests getting finger-stick based tests evolved. For example, by the fourth quarter of 2014, 57% of guests got lab tests run on finger-stick samples. This transition was by choice—not by necessity. In December of 2014, more than 80 tests on Theranos' online test menu were offered via finger-stick and performed using proprietary technologies (other than a few routinely used point-of-care tests). Theranos has multiple proprietary technologies that run in our laboratory, none of which are commercially available.
Theranos' website: Since around the time of Theranos' launch, our website has said that Theranos processes both finger-sticks and venous draws.
Other statements: Additionally, on dozens of occasions, Theranos has explained that we are a full reference lab that can conduct a broad array of tests, and are using venipuncture. For example:
Forbes (October 2015): http://www.forbes.com/video/4543269142001/
Fortune (July 16, 2015): http://fortune.com/2015/07/16/fda-clears-theranos-to-do-test-outside-lab/
Fortune (May 7, 2015): http://fortune.com/2015/05/07/theranos-jump-starts-consumer-lab-testing/
The San Francisco Business Times (May 5, 2015): http://www.bizjournals.com/sanfrancisco/blog/biotech/2015/05/theranos-elizabeth-holmes-quest-labcorp-lh-dgx.html
Theranos told this in writing to the reporter three times: On June 30, July 26, and August 23, 2015.
What about the changes to Theranos' website? First of all, Heather King is misquoted as saying that the changes to Theranos' website were made for "marketing accuracy." Ms. King said no such thing. Instead, we explained to the reporter that the changes to the website were simply done to emphasize that, regardless of whether Theranos takes a venous sample or a finger-stick draw, our blood draws are smaller and less painful, improving the overall patient experience. Theranos updates our website regularly. As explained to the reporter, this update was not done at the request, complaint, or recommendation of FDA.
In addition to telling this to the reporter, we even put this in writing on October 11, 2015.
Capabilities of Theranos' devices: The article implies that Theranos' proprietary devices were only capable of running a limited number of tests. First, "Edison" is only one of many proprietary devices used as part of Theranos proprietary technologies. In total, Theranos research and development has developed hundreds of tests for finger-stick samples using our proprietary devices. Here are the facts.
Theranos' filings with FDA show the versatility of Theranos devices, and our confidence in the results of our tests: Theranos has publicly advocated for FDA regulation of laboratory-developed tests (LDTs), and over 120 of the tests developed for use on our devices used as part of Theranos proprietary technologies have been submitted in pre-submissions to FDA. These pre-submissions span a broad range of specialties, including hematology, microbiology, clinical chemistry, and molecular biology. And Theranos' proprietary systems, including the Nanotainer™ tube, have already received FDA clearance for the HSV-1 test. This voluntary engagement with FDA is not only a statement about our confidence in our technologies, but this summer's clearance and CLIA waiver would not be possible if our devices did not work with those tests.
We told this to the reporter on several occasions, including on July 3, 2015.
Even basic things are misrepresented: Theranos has invented much more than just the "Edison." In fact, "Edison" is an earlier version of just one of our proprietary devices used as part of Theranos proprietary technologies.
We told this to the reporter on October 11, 2015.
The Journal rejected our offer for a demonstration: On October 8, Theranos offered to bring one of our devices to The Journal for demonstration purposes, to show the capabilities of Theranos' technology so they could compare our results to any other lab of their choosing. We made this offer with no strings attached. The Journal declined.
Capillary versus finger-stick: The reporter writes that "Lab experts say finger-pricked blood samples can be less pure than those drawn from a vein because finger-pricked blood often mixes with fluids from tissue and cells that can interfere with tests." This is misleading: With each FDA filing, Theranos is showing that our finger-stick tests are just as accurate as venous draws, starting with our first FDA clearance this summer. Here are the facts.
Theranos' FDA submissions and FDA's clearance: As part of its test validation and before making submissions to FDA, Theranos generates validation data showing equivalence between results from capillary and venous blood samples run using our proprietary technologies. Indeed, FDA determined that Theranos' finger-stick test and associated device for HSV-1 performed just as accurately and reliably as a predicate venous test system. You can read that decision summary here: http://www.accessdata.fda.gov/cdrh_docs/reviews/K143236.pdf
We sent FDA's decision summary to the reporter on July 17, 2015.
This thesis comes from Theranos' competitors: This quote is nearly the same as the one that Quest Diagnostics, one of Theranos' biggest competitors, gave to The New Yorker in December 2014 for its profile of Theranos. Nigel Clarke, one of Quest's senior scientific directors, argued in that article "that finger-stick blood tests aren't reliable for clinical diagnostic tests; because the blood isn't drawn from a vein, the sample can be contaminated by lanced capillaries or damaged tissues." http://www.newyorker.com/magazine/2014/12/15/blood-simpler
February 2014 experiment: The reporter questions the accuracy of Theranos' tests based on a February 2014 experiment using leftover proficiency-testing samples. This is wrong and misleading: the reporter does not explain that the experiment was not designed to assess the accuracy of any Theranos tests, nor was it proficiency testing. Theranos has shared its proficiency testing methods with our regulators, and our proficiency testing methods meet the regulatory requirements. Here are the facts.
What is proficiency testing? Proficiency testing is used to monitor laboratories' continuing performance by comparing a laboratory's test results against other laboratories using similar methods, reagents, and/or instruments. Proficiency-testing samples are often modified by the proficiency-testing providers with processes like freeze-drying and adding preservatives, so that the samples will be stable when they're shipped all around the country. These samples can return different results when they are run on different machines, which is why lab proficiency test results are compared against other laboratories running the same methods and instruments—a peer group.
What the experiment was: In February 2014, some Theranos employees conducted an internal experiment using leftover samples from a previous proficiency-testing event.
According to guidance from industry standard-setting organizations, it is common practice for labs to use leftover proficiency-testing samples for experiments, after the results have been submitted.
In this case, the samples were run for proficiency testing in early February 2014, and the data from these samples was reported back shortly thereafter. Separately, the lab employees above conducted the experiment on the leftover samples.
That experiment was not "proficiency testing" because Theranos had already run proficiency testing on the proper machines and the results of this experiment were never intended to be, and never were, submitted to any of the accredited organizations that do proficiency-testing grading.
The experiment was intended to compare the results that would be produced when using those proficiency-testing samples on different devices.
Matrix effects: We explained to the reporter that proficiency-testing samples can behave differently on different platforms. This phenomenon is called "matrix effects," and it is well known in the industry. This is the reason why proficiency testing programs use "peer groups" for grading test results, namely comparing a lab's test results against other laboratories running the same methods and instruments.
Here are some of the sources that we gave to the reporter in writing on July 3, 2015 that explain matrix effects, and that the reporter to this day has never disclosed to the public:
Rex Astles, CLIA Proficiency Testing – Criteria for Acceptable Performance at 16–17 (Sept. 1, 2010), available at https://wwwn.cdc.gov/cliac/pdf/Addenda/cliac0910/Addendum%20K_Astles.pdf ("'Matrix effect,' refers to an analytical effect, inherent to the interaction between the PT material and the test system, which results in an analytical bias in PT results. Affected PT results tend to agree with PT results obtained using the same test system, but they will not agree with results from other, unaffected test systems. ... Biases are not necessarily predictable or correctable, and the PT test results cannot be compared with results from a reference method or all-methods mean.")
Daniel C. Edson et al., Proficiency Testing: A Guide to Maintaining Successful Performance, 38 LAB MEDICINE 184, 185 (Mar. 2007), available at http://labmed.ascpjournals.org/content/38/3/184.full.pdf ("Assignment to the wrong peer group can result in unsatisfactory performance because of matrix effects (bias introduced by artificial constituents added to a material). Because matrix effects impact all users of the same instrument/reagent group similarly, peer group grading mitigates this influence.").
Clinical & Laboratory Standards Institute, Using Proficiency Testing and Alternative Assessment to Improve Medical Laboratory Quality; Third Edition—Draft Guideline at 32 (Dec. 3, 2014) ("It is important to note that most PT samples are made with a different matrix than normal body fluids; hence, the results may not be commutable. This limits the ability to use PT material for assessing the trueness of a new analyzer and is the reason that most PT material are graded based on the peer group.").
Vitamin D: Vitamin D is an analyte for which the proficiency-testing providers have observed matrix effects, resulting in significant differences across different peer groups. The experiment in February 2014 in which Theranos employees ran experiments with leftover proficiency-testing samples included this very same analyte—Vitamin D.
There are data for Vitamin D proficiency testing available on API's website here: https://www.api-pt.com/pdsselect.aspx?te=2015CHEA. These data show differences between peer groups. In some cases, there is more than 75% difference—and in one case, it appears, more than 100% difference—in proficiency-testing values between peer groups, demonstrating matrix effects.
What the experiment showed: "Matrix effects" are why Theranos does alternative assessment procedures for proficiency testing on our systems, as described below.
What the experiment did not show: These results did not show that there was any issue with the accuracy of Theranos' technology.
We explained all of this—including the circumstances of the experiment and the existence of matrix effects—to the reporter at an in-person meeting on June 23, 2015; and in on-the-record statements of July 3, 2015, and August 23, 2015.
Proficiency testing: The reporter raises questions regarding whether Theranos' proficiency-testing protocols comply with regulations. But the approach that the reporter describes is unremarkable: It's just a description of Theranos' alternative assessment procedures. Theranos has explained our process to our regulators, and proficiency testing at Theranos meets the regulatory requirements. Here are the facts.
We described our proficiency testing and alternative assessment procedures to the reporter on at least four occasions—in a four-hour meeting with one of our laboratory directors on June 23, 2015; and in detailed, lengthy on-the-record statements of July 3, July 26, and August 23, 2015.
How does proficiency testing work? Proficiency testing is used to monitor laboratories' continuing performance by comparing a laboratory's test results against other laboratories using similar methods, reagents, and/or instruments. Proficiency-testing samples are often modified by the proficiency-testing providers with processes like freeze-drying and adding preservatives, so that the samples will be stable when they're shipped all around the country. These samples can return different results when they are run on different machines, which is why lab proficiency test results are compared against other laboratories running the same methods and instruments—a peer group.
Here are some of the sources that we gave to the reporter on July 3, 2015, that describe peer groups and why they are required:
Daniel C. Edson et al., Proficiency Testing: A Guide to Maintaining Successful Performance, 38 LAB MEDICINE 184, 185 (Mar. 2007), available at http://labmed.ascpjournals.org/content/38/3/184.full.pdf ("Assignment to the wrong peer group can result in unsatisfactory performance because of matrix effects (bias introduced by artificial constituents added to a material). Because matrix effects impact all users of the same instrument/reagent group similarly, peer group grading mitigates this influence.").
College of American Pathologists, Proficiency Testing Manual at 14 (2015), available at http://www.cap.org/apps/docs/proficiency_testing/surveys-excel-manual.pdf (emphasizing that "participants' results are combined into comparable method/instrument groups").
American Proficiency Institute, Frequently Asked Questions, available at https://www.api-pt.com/PersonalAssistant.aspx?r=Default.aspx ("We need a method for most tests that you are submitting results for. Choosing a method ensures that your results are compared to those of your individual peer group.").
What happens when there is no peer group? When no peer group is available—i.e., when no other lab is running exactly the same method and the same platform—the procedure for proficiency testing, as dictated by industry standards from the Clinical and Laboratory Standards Institute (CLSI) and others, may be different. It is called alternative assessment procedures, or AAP. This is provided for in CLIA regulations (42 C.F.R. § 493.1236(c)(2)), as we told the reporter, including in writing.
Here are some of the sources that we gave to the reporter on this procedure on July 3, 2015:
42 C.F.R. § 493.1236(c)(2) (providing that a laboratory must verify the accuracy of tests "for which compatible proficiency testing samples are not offered by a CMS-approved proficiency testing program" at least twice annually).
Centers for Medicare & Medicaid Services, State Operations Manual Appendix C – Survey Procedures and Interpretive Guidelines for Laboratories and Laboratory Services at D5219 (2015), available at https://www.cms.gov/Regulations-and-Guidance/Legislation/CLIA/Downloads/App-C_Survey-Procedures-IGs-for-Labs-Labs-Svcs-Final.pdf ("Laboratory tests or procedures that are not compatible [under 42 C.F.R. § 493.1236(c)(2)] may include new or emerging technologies for which PT is not yet available.") (emphasis added).
Clinical & Laboratory Standards Institute, Using Proficiency Testing and Alternative Assessment to Improve Medical Laboratory Quality; Third Edition—Draft Guideline at 27 ("In the absence of a comparable peer group to use for evaluation, the participant should consider the options listed for an AAP.").
When we told the reporter about CLSI—the industry standard-setting organization—he did not know what it was.
How does Theranos conduct proficiency testing? In accordance with industry standards, Theranos conducts standard proficiency testing when a peer group is available through organizations like CAP, API and the State of New York. This is done for our commercially available machines that other labs are also running, and that Theranos uses to process patient samples. For our proprietary technologies—for which there is no peer group—we verify the accuracy of our tests through alternative assessment procedures (AAP), in accordance with industry guidance, as follows:
Theranos frequently compares the performance of our own proprietary technology and the performance of commercially available analyzers we use for testing venous samples in our lab.
We do so using real clinical samples.
In other words, for each analyte, Theranos collects two paired samples from the same subject—one capillary and one venous. We run the capillary sample using our proprietary technologies. We run the venous sample using the commercially available device that Theranos runs for venous sample testing for that analyte—the same devices that have already gone through the ordinary proficiency-testing process (for that analyte). We repeat this test procedure for multiple subjects. Then, we compare all these data to assess the accuracy of our tests.
We keep the data from our AAP using Theranos proprietary technologies on file for regulators to see, as we are supposed to.
This process, including the fact that Theranos does AAP, has been disclosed to and discussed with regulators. Theranos' proficiency testing process meets the regulatory requirements.
Theranos explained this to the reporter in lengthy on-the-record statements, including with citations to hundreds of pages of industry sources, on June 23, 2015; July 3, 2015; July 26, 2015; and August 23, 2015.
What about the emails? First, the reporter cites claims of anonymous former employees that Sunny Balwani, our president and COO, ordered them to "report only the result from instruments bought from other companies." This is nothing more than a description of Theranos' proficiency-testing process and alternative assessment procedures. Among other things, what is missing from this explanation is that, in accordance with regulations, Theranos keeps AAP data on hand, so that CLIA surveyors can review it when they visit Theranos.
Second, the reporter omitted important content, and falsely characterized Mr. Balwani's email. Among other things, Mr. Balwani's email explicitly directed employees to follow lab standard operating procedures; discussed training of staff on Theranos' proficiency testing and alternative assessment procedures; and said that outside counsel and regulators are always available to answer any questions.
Third, the reporter selectively quotes from an email regarding an employee reading regulations. What the email actually says is that after "reading through the regulations more finely," the employee concluded that Theranos would have to conduct AAP under the regulations.
Please note: the reporter has taken this quote out of context.
The reporter interpreted this email, and published misleading excerpts from it, without even showing us a copy.
What about the complaint? The reporter referred to a purported complaint filed by a former employee with the New York State Department of Health in March 2014. This anecdote is irresponsibly reported.
Even the reader has no way of knowing the contents of the purported complaint, since the reporter then immediately cites only the response of Stephanie Schulman. This leaves the misleading impression that whatever this former employee "described" was what Theranos actually did. The reporter did not show Theranos a copy of this document.
The reporter also irresponsibly miscategorizes Theranos' response to his questions. It is very misleading for the reporter to have written, as he did, that "Asked about the complaint, Theranos confirms that the Edison system produced results for several tests last year that differed from results obtained from traditional equipment." The reporter asked about a complaint, and separately we also talked to the reporter about the February 2014 experiment, but these two things are completely unrelated. To link them together is irresponsible journalism.
Here are the facts:
Proficiency testing at Theranos meets applicable regulatory requirements.
No regulator has ever cited Theranos for our use of alternative assessment procedures for our laboratory-developed tests.
Theranos first heard about this complaint from the reporter—not from any one of our regulators.
Providers: The reporter questioned the accuracy of Theranos' tests based on anecdotal accounts from a handful of providers. But during the reporting process, Theranos exposed the reporter for misrepresenting stories from four of the seven providers that he gave us. The other three refused to engage with us—and those are the only stories that the reporter decided to print. Here are the facts.
The real story: Theranos is committed to a dialogue with any providers who have issues or questions about our tests. Part of this commitment is quickly responding to any concerns. That is why, when the reporter made us aware of certain providers' questions, we followed up directly with them shortly thereafter.
The reporter sent Theranos seven provider anecdotes. Theranos reached out to all of the providers, and eventually met with four.
Each of those four provided additional facts and context, including about the health conditions of the patient in question, which showed that the reporter had told us an incomplete and misleading story. In addition:
Some said that the reporter had attributed things to them that they never said.
Some said that the reporter had left out of his allegations the positive experiences they had had with Theranos.
Some said that they did not know that they were on the record with a journalist when they spoke to him.
Two of the four providers who met with Theranos in fact asked Theranos if they could help further. The written statements—which the reporter incorrectly implies were prepared without their involvement—reflect only what they proactively told us that they wanted to say.
Those statements show that the reporter's story about them was untrue.
The other two providers who met with Theranos gave Theranos additional facts and circumstances that showed that the reporting of the anecdotes was misleading and imbalanced. Theranos gave the reporter those facts wherever it could do so consistent with our commitment to maintain patient privacy.
Theranos provided the reporter with no fewer than six on-the-record written statements about providers and patients—on July 1, July 3, July 26, August 8, October 5, and October 12, 2015—providing additional facts and context.
After meeting with Theranos, the majority of the providers expressed admiration for Theranos' services and said they would continue to use Theranos. In fact, Theranos was asked to be the preferred in-house laboratory at one of their practices.
The reporter printed only stories from three providers who refused to meet with Theranos to discuss any questions (or offered to do so on condition of a large, up-front payment in "guaranteed funds"). Those providers also demonstrated that they are hardly neutral or objective:
Nurse Carmen Washington
When Theranos attempted to call Ms. Washington to discuss her statements about Theranos, she demanded payment of $2,500 for a one-hour meeting. It is unknown whether she asked for the same payment from the reporter.
Some of the things Ms. Washington told the reporter are untrue, and Walgreens confirmed to the reporter that they were untrue.
Ms. Washington continued to write lab orders after the reported concerns were raised, for which Theranos ultimately performed testing services.
Naturopath Nicole Sundene
Although Theranos has had meetings with Ms. Sundene or her staff, she did not provide Theranos with necessary details during those meetings.
After the purported anecdote relayed in the article, Ms. Sundene referred many patients to Theranos for laboratory services.
Ms. Sundene has now refused to meet with Theranos since her interaction with the reporter, despite Theranos reaching out to her and requesting a meeting to discuss her purported concerns directly.
Dr. Gary Betz
Dr. Betz refused to speak with Theranos, although Theranos tried several times to reach out to him.
In Theranos' experience, providers generally call the laboratory and ask questions about patient results directly and contemporaneously, rather than relaying concerns more than a year later to a newspaper.
Dr. Betz continued to send patients to Theranos through November 2014, and nurses working in his office continued to send Theranos patients into early 2015—after which time he began advertising on his website that a phlebotomist from LabCorp, a competing laboratory, was working in his office. See http://www.thompsonpeakinternalmedicine.com/lab_services
We explained this to the reporter in several on-the-record statements, including on October 5 and October 12, 2015.
Moreover, the reporter did not procure HIPAA waivers that would allow Theranos to respond fulsomely to several anecdotes. Theranos could not provide any comment to the reporter consistent with our commitment to maintain patient privacy, which we told him.
We wish we could provide you with some of the additional facts that we found, but we hold our patients' private information in the strict confidence.
FDA: The reporter focused on Theranos' recent interactions with FDA. But the reporter's representation is inaccurate and misleading, and creates confusion about our testing methodologies. Here are the facts.
The truth is that we initiated our work to take our laboratory-developed tests (LDTs) through the FDA clearance process over two years ago—by choice, not necessity—because we are seeking to create a new model for laboratory testing standards, and have championed FDA oversight ever since. We engage with FDA regularly.
When FDA released its draft framework for the oversight of LDTs, we publicly supported it. You can read our public statement here: https://www.theranos.com/content/pdf/theranos_comment_ldt_guidance.pdf
In response to our outreach, FDA told us they wanted to regulate our Nanotainer™ tubes (the transportation vehicle for the few little drops of blood) and that they wanted to rigorously review our proprietary systems as part of clearing our first test.
FDA then rigorously reviewed our test systems and Nanotainer™ tubes as part of the HSV-1 clearance that we got this summer. (See FDA's Decision Summary describing how Theranos' clearance is for use for HSV-1 "on the Theranos System" and performed on "fingerstick ...obtained with the... Nanotainer™ Tubes, available here: http://www.accessdata.fda.gov/cdrh_docs/reviews/K143236.pdf.) As our press release describes (available here: https://www.theranos.com/news/posts/theranos-receives-fda-clearance-and-review-and-validation-of-revolutionary-finger-stick-technology-test-and-associated-test-system), our work in getting the FDA clearance involved many rigorous and detailed studies.
This past July, FDA cleared that test and the associated devices, software, and Nanotainer™ tubes. In a further statement of FDA's confidence in our technology, including the Nanotainer™ tube, FDA granted the HSV-1 test a "CLIA Waiver," permitting our test system (including the Nanotainer™ tube) to run HSV-1 tests in locations outside of traditional clinical laboratories. This was the first CLIA waiver granted for an HSV-1 test, and the first CLIA-waiver on Theranos' proprietary devices, which are different from any previously CLIA-waived devices. See https://www.theranos.com/news/posts/theranos-receives-clia-waiver-paving-the-way-for-greater-accessibility-of-health-information-at-the-time-and-place-it-matters
The next submissions Theranos and FDA have been focusing on completing were for general clearance for the Nanotainer™ tube across all tests, not just with our HSV-1 test and devices.
Theranos' first FDA inspection began on August 25, 2015. This inspection focused on Theranos' FDA quality systems compliance, on which we had been working previously, so as to transition from the CLIA laboratory quality systems and framework.
This is a time of evolving policy, and we have advocated for the transition from the LDT framework to the FDA quality systems framework. We are charting new pathways, and are working to create a model for the transition.
All of our Nanotainer™ tube submissions and the associated data have been submitted to FDA.
In our discussions with FDA, we determined that it was appropriate to temporarily pause use of the Nanotainer™ tubes for all tests except our cleared HSV-1 test as we now cut over to FDA quality systems, and wait for clearance. This transition is part of converting all our operations from the LDT framework to the FDA framework, because it is not possible to operate under both the CLIA lab and FDA quality framework at the same time. We believe this is the right action to take as we approach clearance of the Nanotainer™ tubes, for which we have submitted all our data to the agency, so that we incorporate them into other tests under the FDA quality systems framework upon clearance.
So while we wait for clearance, as of this exact moment, we are temporarily using a different tube—FDA-cleared tubes for venous blood that are ordinarily used in pediatric patients (very small tubes). Once we receive clearance, we now have the systems in place to reincorporate the Nanotainer™ tubes under the FDA quality systems framework.
During the time we are transitioning the Nanotainer™ tubes operations around FDA quality systems, we are still able to use all our proprietary methods and technology, including our devices, which were cleared for use this summer based on studies with more than 800 patient samples, including head-to-head comparisons to non-Theranos technology on venous samples.
FDA is reviewing the same tube that was cleared for the HSV-1 test. We've already met their rigorous standards in our first submission on our system, and we'll continue to do so.
Moreover:
The article confused the Nanotainer™ tubes, which is merely a substitute for a large vial used to carry the blood sample from the point of collection to the analyzing device, for the Theranos devices themselves. By obfuscating this issue, the reporter makes Theranos' recent decision to move our Nanotainer™ tube operations from the CLIA laboratory quality framework to the FDA quality (QSR) framework sound like something it is not.
The article implied that there is something suspect about the fact that we are the only lab to embrace, rather than fight, FDA regulation. We are the only lab with sufficient confidence in our tests to submit all of them for FDA review and clearance. Theranos is doing the right thing, which is also the hard thing, and we are charting a new path.
Additional background on the sources: The reporter purportedly spent five months working on a story about Theranos but his story did not reflect the views of the many eminent scientists, thought leaders, hospitals, and business partners who have worked and partnered with the company around our technology and found it to be innovative and revolutionary.
Instead, the reporter relied on:
Anonymous sources. This allowed the reporter to avoid having to reveal the biases and lack of knowledge of the sources.
Former employees. It is apparent that these former employees did not understand Theranos' technology; are unfamiliar with the processes currently employed by the company; and do not understand the regulatory framework that Theranos operates under. But by making them anonymous, readers do not know when they worked at Theranos, or for how long; whether they even worked in Theranos' laboratory, as opposed to some other division in the company; or why they left.
Lab experts. The reporter attributes certain statements to "lab experts." Most of these "experts" are anonymous.
There is no reason to keep these experts anonymous. All anonymity does is obfuscate their qualifications, and the potential that they have a vested interest in criticizing Theranos—such as being one of Theranos' competitors.
For example, the reporter relies on unidentified "lab experts" for the proposition that "finger-pricked blood samples can be less pure than those drawn from a vein because finger-pricked blood often mixes with fluids from tissue and cells that can interfere with tests." This is not dissimilar from a previous quote from Quest Diagnostics, who said nearly the exact same thing to The New Yorker in December 2014, when Nigel Clarke, one of Quest's senior scientific directors, argued "that finger-stick blood tests aren't reliable for clinical diagnostic tests; because the blood isn't drawn from a vein, the sample can be contaminated by lanced capillaries or damaged tissues." http://www.newyorker.com/magazine/2014/12/15/blood-simpler
Other sources. While ignoring Theranos' on the record statements, and our laboratory director and senior scientist, the reporter offered great latitude and seemingly abandoned his skepticism when speaking to Theranos critics. For example, the reporter relies on Rochelle Gibbons, but he failed to report that:
Rochelle Gibbons is a friend of an individual whom Theranos sued for patent invalidity and other claims. During litigation, last year, that individual repeatedly contacted Mrs. Gibbons via email to solicit her testimony against Theranos.
When that friend tried to use statements by Mrs. Gibbons against Theranos in litigation, she refused to make any statement under oath, and the court found that her statements should not be credited.
Mrs. Gibbons was married to a Theranos employee who passed away in May 2013. Mrs. Gibbons never worked for the company and had no input into Theranos' business, let alone its technology.
We told this to the reporter, including in writing, on July 1, 2015, and July 26, 2015.
From his very first interactions with Theranos, the reporter made abundantly clear that he considered Theranos to be a target to be taken down, and not simply the subject of an objective news story. The articles that appeared last week are the inevitable product of that approach.
As we provide these answers, it is important not to lose sight of a few simple truths:
Theranos' technology is reviewed by regulators, proven in the field, and praised by leaders in the industry and doctors and individuals that we serve.
We provide blood tests faster, requiring far less blood and patient discomfort, than for any test previously available—or available today from any other laboratory.
We have run more than 3.5 million tests, with patient satisfaction scores from tens of thousands of patients that consistently rate Theranos, on average, over 4.8 out of 5. To our knowledge, we are the first laboratory to publish our prices, lab proficiency-testing scores, customer satisfaction scores, guest visit times, and more on our website.
We are confident in the reliability of our tests, because we comprehensively validate the accuracy of every test we run. In addition, we are the only laboratory that has committed to submitting all of our laboratory-developed tests, including our technology, procedures, and methods, to FDA for review and clearance. FDA has already cleared one of those tests, including our underlying test systems and Nanotainer™ tubes, for use in detecting the herpes simplex virus (HSV-1). The decision summary, which describes the rigorous science behind the clearance, is available online.
The faster speed and lower cost of our tests mean that consumers, and their doctors, can monitor their health so that problems are identified early enough to be addressed by medical professionals. That lower prices can reduce the cost to insurers, Medicare, and Medicaid by billions of dollars, and can give individuals access to critical health data they could not otherwise afford.
Our proprietary devices are making it possible to run finger-stick samples for tests that could never be run on finger-stick before. We began using our proprietary devices in our lab at the launch of our retail operations. And we initiated filings with FDA two years ago—by choice, not necessity—because we are seeking to create a new model for laboratory testing standards, and have championed FDA oversight ever since. It is the right thing, which is also the hard thing.
We are the same company offering the same services as we were last week.
Now, let us tell you what the reporter got wrong and why.
What The Reporter Got Wrong—And Our Responses
The stories say or imply many things that are simply wrong. The following are examples of the reporter's incorrect points—and the truth.
Accuracy and Reliability: The stories suggest that Theranos' tests are not accurate. That is wrong: Theranos' tests are accurate and reliable. Here are the facts.
Satisfied customers: Theranos has provided accurate and reliable tests to tens of thousands of satisfied customers. You can see testimonials from some of those guests on our website: https://www.theranos.com/guest-stories
We gave a selection of testimonials in writing to the reporter on July 1, 2015.
FDA's clearance: Just this past July, FDA cleared Theranos' finger-stick test for HSV-1, which included a rigorous review of our underlying test systems and finger-stick technology.
In order to clear that technology, FDA rigorously reviewed our test systems and Nanotainer™ tubes. To view the studies that show the accuracy and reliability of Theranos' HSV-1 test, take a look at FDA's decision summary, available here: http://www.accessdata.fda.gov/cdrh_docs/reviews/K143236.pdf.
The FDA's decision provides independent validation of the ground breaking Theranos System upon which the HSV-1 test is run. FDA's decision summary describes how Theranos' clearance is for use for HSV-1 "on the Theranos System" and performed on "fingerstick ...obtained with the...Nanotainer™ Tubes." See http://www.accessdata.fda.gov/cdrh_docs/reviews/K143236.pdf, at page 2, under "Intended Use," et seq.
The studies provided to FDA were detailed and rigorous. Among other things:
Theranos provided comprehensive data to FDA on our test systems and methods, the chemistry, the hardware, and the software in the company's foundational system and Laboratory Developed Test (LDT) application.
Theranos provided study data from 818 subjects of varying age and ethnicity, demonstrating that our devices could be run accurately using only a finger-stick as well as a traditional venous draw across large numbers of Theranos devices, all compared against an FDA cleared, commercially available reference method.
Theranos' HSV-1 studies included matrix comparison studies across 69 Theranos devices. Use of such a large number of devices in the studies is unique, as such studies are often run on a single or small number of devices.
The larger number of devices allows for comprehensive characterization of variation across devices, which can be a source of variance in data across laboratories. Theranos' HSV-1 clinical studies also included sensitivity and specificity studies with hundreds of human samples – both capillary and venous, CDC panel testing – which demonstrated 100% agreement with the results provided by CDC, low-prevalence population tests, and other studies.
Theranos also completed full submissions for our HSV-1 IgG test on venous serum, venous plasma, capillary plasma and capillary whole blood, to demonstrate equivalence on our system across the different sample types, in contrast to conventional tests which have only been cleared for venous samples.
We told this information to the reporter on July 3, 2015.
The reporter did not ask a single question about Theranos' FDA clearance or CLIA waiver.
CMS and CLIA: To earn and maintain CLIA certification, Theranos must ensure the accuracy and reliability of our tests. CLIA's purpose is to ensure quality laboratory testing and ensure accurate and reliable test results.
Theranos is a full-service laboratory that is subject to regulation by the Centers for Medicare & Medicaid Services (CMS). CMS regulates all laboratory testing (except research) performed on humans in the U.S. through the Clinical Laboratory Improvement Amendments (CLIA).
Before we offered any test to anyone, anywhere, regardless of the machines or methods used, we had to scientifically verify the accuracy and reliability of that test. We have done that for each and every one of our tests.
Theranos publishes our proficiency-testing results and certain validation data on our website. To our knowledge, none of our competitors are willing to be so transparent. See https://www.theranos.com/our-lab.
We told this to the reporter on several occasions, including July 3, 2015.
People who know: In addition to our regulators, our business partners have spoken in support of Theranos' technology.
We gave quotes and sources to the reporter over several occasions, including on July 8 and July 17, 2015. He did not use any of those quotes or sources in his article.
Finger-stick tests in Theranos' clinical lab: The article implies that Theranos said that all our tests are performed with finger-sticks. That's not true: We have consistently said in public statements, on our website, and to our customers in Wellness Centers that some of our tests are performed on venous draws, in part because we have been becoming a full-service laboratory. In December 2014, Theranos offered more than 80 of the tests on our online test menu via finger-stick. Also, of the finger-stick tests that Theranos offered, all ran using proprietary technologies—technologies that are not commercially available—except a few routinely used point-of-care tests. Here are the facts.
Finger-stick tests: Because Theranos has been working to transition to an FDA framework, we have moved from use of our laboratory-developed tests (LDTs) to FDA-cleared and approved assays. At the same time, Theranos has been increasing the number of tests we offer, in order to become a full-service lab. Theranos offered tests on venous samples—run in our own laboratory—from the time we launched, and have expanded that capability to be able to offer rare and specialty tests at our unprecedented low prices, as those are the tests that often cost the most. Additionally, because of this offering, most patients would not have to go to multiple labs to fill lab orders.
As Theranos transitioned to the FDA framework, and added specialty and esoteric tests to our laboratory offerings, the percentages of guests getting finger-stick based tests evolved. For example, by the fourth quarter of 2014, 57% of guests got lab tests run on finger-stick samples. This transition was by choice—not by necessity. In December of 2014, more than 80 tests on Theranos' online test menu were offered via finger-stick and performed using proprietary technologies (other than a few routinely used point-of-care tests). Theranos has multiple proprietary technologies that run in our laboratory, none of which are commercially available.
Theranos' website: Since around the time of Theranos' launch, our website has said that Theranos processes both finger-sticks and venous draws.
Other statements: Additionally, on dozens of occasions, Theranos has explained that we are a full reference lab that can conduct a broad array of tests, and are using venipuncture. For example:
Forbes (October 2015): http://www.forbes.com/video/4543269142001/
Fortune (July 16, 2015): http://fortune.com/2015/07/16/fda-clears-theranos-to-do-test-outside-lab/
Fortune (May 7, 2015): http://fortune.com/2015/05/07/theranos-jump-starts-consumer-lab-testing/
The San Francisco Business Times (May 5, 2015): http://www.bizjournals.com/sanfrancisco/blog/biotech/2015/05/theranos-elizabeth-holmes-quest-labcorp-lh-dgx.html
Theranos told this in writing to the reporter three times: On June 30, July 26, and August 23, 2015.
What about the changes to Theranos' website? First of all, Heather King is misquoted as saying that the changes to Theranos' website were made for "marketing accuracy." Ms. King said no such thing. Instead, we explained to the reporter that the changes to the website were simply done to emphasize that, regardless of whether Theranos takes a venous sample or a finger-stick draw, our blood draws are smaller and less painful, improving the overall patient experience. Theranos updates our website regularly. As explained to the reporter, this update was not done at the request, complaint, or recommendation of FDA.
In addition to telling this to the reporter, we even put this in writing on October 11, 2015.
Capabilities of Theranos' devices: The article implies that Theranos' proprietary devices were only capable of running a limited number of tests. First, "Edison" is only one of many proprietary devices used as part of Theranos proprietary technologies. In total, Theranos research and development has developed hundreds of tests for finger-stick samples using our proprietary devices. Here are the facts.
Theranos' filings with FDA show the versatility of Theranos devices, and our confidence in the results of our tests: Theranos has publicly advocated for FDA regulation of laboratory-developed tests (LDTs), and over 120 of the tests developed for use on our devices used as part of Theranos proprietary technologies have been submitted in pre-submissions to FDA. These pre-submissions span a broad range of specialties, including hematology, microbiology, clinical chemistry, and molecular biology. And Theranos' proprietary systems, including the Nanotainer™ tube, have already received FDA clearance for the HSV-1 test. This voluntary engagement with FDA is not only a statement about our confidence in our technologies, but this summer's clearance and CLIA waiver would not be possible if our devices did not work with those tests.
We told this to the reporter on several occasions, including on July 3, 2015.
Even basic things are misrepresented: Theranos has invented much more than just the "Edison." In fact, "Edison" is an earlier version of just one of our proprietary devices used as part of Theranos proprietary technologies.
We told this to the reporter on October 11, 2015.
The Journal rejected our offer for a demonstration: On October 8, Theranos offered to bring one of our devices to The Journal for demonstration purposes, to show the capabilities of Theranos' technology so they could compare our results to any other lab of their choosing. We made this offer with no strings attached. The Journal declined.
Capillary versus finger-stick: The reporter writes that "Lab experts say finger-pricked blood samples can be less pure than those drawn from a vein because finger-pricked blood often mixes with fluids from tissue and cells that can interfere with tests." This is misleading: With each FDA filing, Theranos is showing that our finger-stick tests are just as accurate as venous draws, starting with our first FDA clearance this summer. Here are the facts.
Theranos' FDA submissions and FDA's clearance: As part of its test validation and before making submissions to FDA, Theranos generates validation data showing equivalence between results from capillary and venous blood samples run using our proprietary technologies. Indeed, FDA determined that Theranos' finger-stick test and associated device for HSV-1 performed just as accurately and reliably as a predicate venous test system. You can read that decision summary here: http://www.accessdata.fda.gov/cdrh_docs/reviews/K143236.pdf
We sent FDA's decision summary to the reporter on July 17, 2015.
This thesis comes from Theranos' competitors: This quote is nearly the same as the one that Quest Diagnostics, one of Theranos' biggest competitors, gave to The New Yorker in December 2014 for its profile of Theranos. Nigel Clarke, one of Quest's senior scientific directors, argued in that article "that finger-stick blood tests aren't reliable for clinical diagnostic tests; because the blood isn't drawn from a vein, the sample can be contaminated by lanced capillaries or damaged tissues." http://www.newyorker.com/magazine/2014/12/15/blood-simpler
February 2014 experiment: The reporter questions the accuracy of Theranos' tests based on a February 2014 experiment using leftover proficiency-testing samples. This is wrong and misleading: the reporter does not explain that the experiment was not designed to assess the accuracy of any Theranos tests, nor was it proficiency testing. Theranos has shared its proficiency testing methods with our regulators, and our proficiency testing methods meet the regulatory requirements. Here are the facts.
What is proficiency testing? Proficiency testing is used to monitor laboratories' continuing performance by comparing a laboratory's test results against other laboratories using similar methods, reagents, and/or instruments. Proficiency-testing samples are often modified by the proficiency-testing providers with processes like freeze-drying and adding preservatives, so that the samples will be stable when they're shipped all around the country. These samples can return different results when they are run on different machines, which is why lab proficiency test results are compared against other laboratories running the same methods and instruments—a peer group.
What the experiment was: In February 2014, some Theranos employees conducted an internal experiment using leftover samples from a previous proficiency-testing event.
According to guidance from industry standard-setting organizations, it is common practice for labs to use leftover proficiency-testing samples for experiments, after the results have been submitted.
In this case, the samples were run for proficiency testing in early February 2014, and the data from these samples was reported back shortly thereafter. Separately, the lab employees above conducted the experiment on the leftover samples.
That experiment was not "proficiency testing" because Theranos had already run proficiency testing on the proper machines and the results of this experiment were never intended to be, and never were, submitted to any of the accredited organizations that do proficiency-testing grading.
The experiment was intended to compare the results that would be produced when using those proficiency-testing samples on different devices.
Matrix effects: We explained to the reporter that proficiency-testing samples can behave differently on different platforms. This phenomenon is called "matrix effects," and it is well known in the industry. This is the reason why proficiency testing programs use "peer groups" for grading test results, namely comparing a lab's test results against other laboratories running the same methods and instruments.
Here are some of the sources that we gave to the reporter in writing on July 3, 2015 that explain matrix effects, and that the reporter to this day has never disclosed to the public:
Rex Astles, CLIA Proficiency Testing – Criteria for Acceptable Performance at 16–17 (Sept. 1, 2010), available at https://wwwn.cdc.gov/cliac/pdf/Addenda/cliac0910/Addendum%20K_Astles.pdf ("'Matrix effect,' refers to an analytical effect, inherent to the interaction between the PT material and the test system, which results in an analytical bias in PT results. Affected PT results tend to agree with PT results obtained using the same test system, but they will not agree with results from other, unaffected test systems. ... Biases are not necessarily predictable or correctable, and the PT test results cannot be compared with results from a reference method or all-methods mean.")
Daniel C. Edson et al., Proficiency Testing: A Guide to Maintaining Successful Performance, 38 LAB MEDICINE 184, 185 (Mar. 2007), available at http://labmed.ascpjournals.org/content/38/3/184.full.pdf ("Assignment to the wrong peer group can result in unsatisfactory performance because of matrix effects (bias introduced by artificial constituents added to a material). Because matrix effects impact all users of the same instrument/reagent group similarly, peer group grading mitigates this influence.").
Clinical & Laboratory Standards Institute, Using Proficiency Testing and Alternative Assessment to Improve Medical Laboratory Quality; Third Edition—Draft Guideline at 32 (Dec. 3, 2014) ("It is important to note that most PT samples are made with a different matrix than normal body fluids; hence, the results may not be commutable. This limits the ability to use PT material for assessing the trueness of a new analyzer and is the reason that most PT material are graded based on the peer group.").
Vitamin D: Vitamin D is an analyte for which the proficiency-testing providers have observed matrix effects, resulting in significant differences across different peer groups. The experiment in February 2014 in which Theranos employees ran experiments with leftover proficiency-testing samples included this very same analyte—Vitamin D.
There are data for Vitamin D proficiency testing available on API's website here: https://www.api-pt.com/pdsselect.aspx?te=2015CHEA. These data show differences between peer groups. In some cases, there is more than 75% difference—and in one case, it appears, more than 100% difference—in proficiency-testing values between peer groups, demonstrating matrix effects.
What the experiment showed: "Matrix effects" are why Theranos does alternative assessment procedures for proficiency testing on our systems, as described below.
What the experiment did not show: These results did not show that there was any issue with the accuracy of Theranos' technology.
We explained all of this—including the circumstances of the experiment and the existence of matrix effects—to the reporter at an in-person meeting on June 23, 2015; and in on-the-record statements of July 3, 2015, and August 23, 2015.
Proficiency testing: The reporter raises questions regarding whether Theranos' proficiency-testing protocols comply with regulations. But the approach that the reporter describes is unremarkable: It's just a description of Theranos' alternative assessment procedures. Theranos has explained our process to our regulators, and proficiency testing at Theranos meets the regulatory requirements. Here are the facts.
We described our proficiency testing and alternative assessment procedures to the reporter on at least four occasions—in a four-hour meeting with one of our laboratory directors on June 23, 2015; and in detailed, lengthy on-the-record statements of July 3, July 26, and August 23, 2015.
How does proficiency testing work? Proficiency testing is used to monitor laboratories' continuing performance by comparing a laboratory's test results against other laboratories using similar methods, reagents, and/or instruments. Proficiency-testing samples are often modified by the proficiency-testing providers with processes like freeze-drying and adding preservatives, so that the samples will be stable when they're shipped all around the country. These samples can return different results when they are run on different machines, which is why lab proficiency test results are compared against other laboratories running the same methods and instruments—a peer group.
Here are some of the sources that we gave to the reporter on July 3, 2015, that describe peer groups and why they are required:
Daniel C. Edson et al., Proficiency Testing: A Guide to Maintaining Successful Performance, 38 LAB MEDICINE 184, 185 (Mar. 2007), available at http://labmed.ascpjournals.org/content/38/3/184.full.pdf ("Assignment to the wrong peer group can result in unsatisfactory performance because of matrix effects (bias introduced by artificial constituents added to a material). Because matrix effects impact all users of the same instrument/reagent group similarly, peer group grading mitigates this influence.").
College of American Pathologists, Proficiency Testing Manual at 14 (2015), available at http://www.cap.org/apps/docs/proficiency_testing/surveys-excel-manual.pdf (emphasizing that "participants' results are combined into comparable method/instrument groups").
American Proficiency Institute, Frequently Asked Questions, available at https://www.api-pt.com/PersonalAssistant.aspx?r=Default.aspx ("We need a method for most tests that you are submitting results for. Choosing a method ensures that your results are compared to those of your individual peer group.").
What happens when there is no peer group? When no peer group is available—i.e., when no other lab is running exactly the same method and the same platform—the procedure for proficiency testing, as dictated by industry standards from the Clinical and Laboratory Standards Institute (CLSI) and others, may be different. It is called alternative assessment procedures, or AAP. This is provided for in CLIA regulations (42 C.F.R. § 493.1236(c)(2)), as we told the reporter, including in writing.
Here are some of the sources that we gave to the reporter on this procedure on July 3, 2015:
42 C.F.R. § 493.1236(c)(2) (providing that a laboratory must verify the accuracy of tests "for which compatible proficiency testing samples are not offered by a CMS-approved proficiency testing program" at least twice annually).
Centers for Medicare & Medicaid Services, State Operations Manual Appendix C – Survey Procedures and Interpretive Guidelines for Laboratories and Laboratory Services at D5219 (2015), available at https://www.cms.gov/Regulations-and-Guidance/Legislation/CLIA/Downloads/App-C_Survey-Procedures-IGs-for-Labs-Labs-Svcs-Final.pdf ("Laboratory tests or procedures that are not compatible [under 42 C.F.R. § 493.1236(c)(2)] may include new or emerging technologies for which PT is not yet available.") (emphasis added).
Clinical & Laboratory Standards Institute, Using Proficiency Testing and Alternative Assessment to Improve Medical Laboratory Quality; Third Edition—Draft Guideline at 27 ("In the absence of a comparable peer group to use for evaluation, the participant should consider the options listed for an AAP.").
When we told the reporter about CLSI—the industry standard-setting organization—he did not know what it was.
How does Theranos conduct proficiency testing? In accordance with industry standards, Theranos conducts standard proficiency testing when a peer group is available through organizations like CAP, API and the State of New York. This is done for our commercially available machines that other labs are also running, and that Theranos uses to process patient samples. For our proprietary technologies—for which there is no peer group—we verify the accuracy of our tests through alternative assessment procedures (AAP), in accordance with industry guidance, as follows:
Theranos frequently compares the performance of our own proprietary technology and the performance of commercially available analyzers we use for testing venous samples in our lab.
We do so using real clinical samples.
In other words, for each analyte, Theranos collects two paired samples from the same subject—one capillary and one venous. We run the capillary sample using our proprietary technologies. We run the venous sample using the commercially available device that Theranos runs for venous sample testing for that analyte—the same devices that have already gone through the ordinary proficiency-testing process (for that analyte). We repeat this test procedure for multiple subjects. Then, we compare all these data to assess the accuracy of our tests.
We keep the data from our AAP using Theranos proprietary technologies on file for regulators to see, as we are supposed to.
This process, including the fact that Theranos does AAP, has been disclosed to and discussed with regulators. Theranos' proficiency testing process meets the regulatory requirements.
Theranos explained this to the reporter in lengthy on-the-record statements, including with citations to hundreds of pages of industry sources, on June 23, 2015; July 3, 2015; July 26, 2015; and August 23, 2015.
What about the emails? First, the reporter cites claims of anonymous former employees that Sunny Balwani, our president and COO, ordered them to "report only the result from instruments bought from other companies." This is nothing more than a description of Theranos' proficiency-testing process and alternative assessment procedures. Among other things, what is missing from this explanation is that, in accordance with regulations, Theranos keeps AAP data on hand, so that CLIA surveyors can review it when they visit Theranos.
Second, the reporter omitted important content, and falsely characterized Mr. Balwani's email. Among other things, Mr. Balwani's email explicitly directed employees to follow lab standard operating procedures; discussed training of staff on Theranos' proficiency testing and alternative assessment procedures; and said that outside counsel and regulators are always available to answer any questions.
Third, the reporter selectively quotes from an email regarding an employee reading regulations. What the email actually says is that after "reading through the regulations more finely," the employee concluded that Theranos would have to conduct AAP under the regulations.
Please note: the reporter has taken this quote out of context.
The reporter interpreted this email, and published misleading excerpts from it, without even showing us a copy.
What about the complaint? The reporter referred to a purported complaint filed by a former employee with the New York State Department of Health in March 2014. This anecdote is irresponsibly reported.
Even the reader has no way of knowing the contents of the purported complaint, since the reporter then immediately cites only the response of Stephanie Schulman. This leaves the misleading impression that whatever this former employee "described" was what Theranos actually did. The reporter did not show Theranos a copy of this document.
The reporter also irresponsibly miscategorizes Theranos' response to his questions. It is very misleading for the reporter to have written, as he did, that "Asked about the complaint, Theranos confirms that the Edison system produced results for several tests last year that differed from results obtained from traditional equipment." The reporter asked about a complaint, and separately we also talked to the reporter about the February 2014 experiment, but these two things are completely unrelated. To link them together is irresponsible journalism.
Here are the facts:
Proficiency testing at Theranos meets applicable regulatory requirements.
No regulator has ever cited Theranos for our use of alternative assessment procedures for our laboratory-developed tests.
Theranos first heard about this complaint from the reporter—not from any one of our regulators.
Providers: The reporter questioned the accuracy of Theranos' tests based on anecdotal accounts from a handful of providers. But during the reporting process, Theranos exposed the reporter for misrepresenting stories from four of the seven providers that he gave us. The other three refused to engage with us—and those are the only stories that the reporter decided to print. Here are the facts.
The real story: Theranos is committed to a dialogue with any providers who have issues or questions about our tests. Part of this commitment is quickly responding to any concerns. That is why, when the reporter made us aware of certain providers' questions, we followed up directly with them shortly thereafter.
The reporter sent Theranos seven provider anecdotes. Theranos reached out to all of the providers, and eventually met with four.
Each of those four provided additional facts and context, including about the health conditions of the patient in question, which showed that the reporter had told us an incomplete and misleading story. In addition:
Some said that the reporter had attributed things to them that they never said.
Some said that the reporter had left out of his allegations the positive experiences they had had with Theranos.
Some said that they did not know that they were on the record with a journalist when they spoke to him.
Two of the four providers who met with Theranos in fact asked Theranos if they could help further. The written statements—which the reporter incorrectly implies were prepared without their involvement—reflect only what they proactively told us that they wanted to say.
Those statements show that the reporter's story about them was untrue.
The other two providers who met with Theranos gave Theranos additional facts and circumstances that showed that the reporting of the anecdotes was misleading and imbalanced. Theranos gave the reporter those facts wherever it could do so consistent with our commitment to maintain patient privacy.
Theranos provided the reporter with no fewer than six on-the-record written statements about providers and patients—on July 1, July 3, July 26, August 8, October 5, and October 12, 2015—providing additional facts and context.
After meeting with Theranos, the majority of the providers expressed admiration for Theranos' services and said they would continue to use Theranos. In fact, Theranos was asked to be the preferred in-house laboratory at one of their practices.
The reporter printed only stories from three providers who refused to meet with Theranos to discuss any questions (or offered to do so on condition of a large, up-front payment in "guaranteed funds"). Those providers also demonstrated that they are hardly neutral or objective:
Nurse Carmen Washington
When Theranos attempted to call Ms. Washington to discuss her statements about Theranos, she demanded payment of $2,500 for a one-hour meeting. It is unknown whether she asked for the same payment from the reporter.
Some of the things Ms. Washington told the reporter are untrue, and Walgreens confirmed to the reporter that they were untrue.
Ms. Washington continued to write lab orders after the reported concerns were raised, for which Theranos ultimately performed testing services.
Naturopath Nicole Sundene
Although Theranos has had meetings with Ms. Sundene or her staff, she did not provide Theranos with necessary details during those meetings.
After the purported anecdote relayed in the article, Ms. Sundene referred many patients to Theranos for laboratory services.
Ms. Sundene has now refused to meet with Theranos since her interaction with the reporter, despite Theranos reaching out to her and requesting a meeting to discuss her purported concerns directly.
Dr. Gary Betz
Dr. Betz refused to speak with Theranos, although Theranos tried several times to reach out to him.
In Theranos' experience, providers generally call the laboratory and ask questions about patient results directly and contemporaneously, rather than relaying concerns more than a year later to a newspaper.
Dr. Betz continued to send patients to Theranos through November 2014, and nurses working in his office continued to send Theranos patients into early 2015—after which time he began advertising on his website that a phlebotomist from LabCorp, a competing laboratory, was working in his office. See http://www.thompsonpeakinternalmedicine.com/lab_services
We explained this to the reporter in several on-the-record statements, including on October 5 and October 12, 2015.
Moreover, the reporter did not procure HIPAA waivers that would allow Theranos to respond fulsomely to several anecdotes. Theranos could not provide any comment to the reporter consistent with our commitment to maintain patient privacy, which we told him.
We wish we could provide you with some of the additional facts that we found, but we hold our patients' private information in the strict confidence.
FDA: The reporter focused on Theranos' recent interactions with FDA. But the reporter's representation is inaccurate and misleading, and creates confusion about our testing methodologies. Here are the facts.
The truth is that we initiated our work to take our laboratory-developed tests (LDTs) through the FDA clearance process over two years ago—by choice, not necessity—because we are seeking to create a new model for laboratory testing standards, and have championed FDA oversight ever since. We engage with FDA regularly.
When FDA released its draft framework for the oversight of LDTs, we publicly supported it. You can read our public statement here: https://www.theranos.com/content/pdf/theranos_comment_ldt_guidance.pdf
In response to our outreach, FDA told us they wanted to regulate our Nanotainer™ tubes (the transportation vehicle for the few little drops of blood) and that they wanted to rigorously review our proprietary systems as part of clearing our first test.
FDA then rigorously reviewed our test systems and Nanotainer™ tubes as part of the HSV-1 clearance that we got this summer. (See FDA's Decision Summary describing how Theranos' clearance is for use for HSV-1 "on the Theranos System" and performed on "fingerstick ...obtained with the... Nanotainer™ Tubes, available here: http://www.accessdata.fda.gov/cdrh_docs/reviews/K143236.pdf.) As our press release describes (available here: https://www.theranos.com/news/posts/theranos-receives-fda-clearance-and-review-and-validation-of-revolutionary-finger-stick-technology-test-and-associated-test-system), our work in getting the FDA clearance involved many rigorous and detailed studies.
This past July, FDA cleared that test and the associated devices, software, and Nanotainer™ tubes. In a further statement of FDA's confidence in our technology, including the Nanotainer™ tube, FDA granted the HSV-1 test a "CLIA Waiver," permitting our test system (including the Nanotainer™ tube) to run HSV-1 tests in locations outside of traditional clinical laboratories. This was the first CLIA waiver granted for an HSV-1 test, and the first CLIA-waiver on Theranos' proprietary devices, which are different from any previously CLIA-waived devices. See https://www.theranos.com/news/posts/theranos-receives-clia-waiver-paving-the-way-for-greater-accessibility-of-health-information-at-the-time-and-place-it-matters
The next submissions Theranos and FDA have been focusing on completing were for general clearance for the Nanotainer™ tube across all tests, not just with our HSV-1 test and devices.
Theranos' first FDA inspection began on August 25, 2015. This inspection focused on Theranos' FDA quality systems compliance, on which we had been working previously, so as to transition from the CLIA laboratory quality systems and framework.
This is a time of evolving policy, and we have advocated for the transition from the LDT framework to the FDA quality systems framework. We are charting new pathways, and are working to create a model for the transition.
All of our Nanotainer™ tube submissions and the associated data have been submitted to FDA.
In our discussions with FDA, we determined that it was appropriate to temporarily pause use of the Nanotainer™ tubes for all tests except our cleared HSV-1 test as we now cut over to FDA quality systems, and wait for clearance. This transition is part of converting all our operations from the LDT framework to the FDA framework, because it is not possible to operate under both the CLIA lab and FDA quality framework at the same time. We believe this is the right action to take as we approach clearance of the Nanotainer™ tubes, for which we have submitted all our data to the agency, so that we incorporate them into other tests under the FDA quality systems framework upon clearance.
So while we wait for clearance, as of this exact moment, we are temporarily using a different tube—FDA-cleared tubes for venous blood that are ordinarily used in pediatric patients (very small tubes). Once we receive clearance, we now have the systems in place to reincorporate the Nanotainer™ tubes under the FDA quality systems framework.
During the time we are transitioning the Nanotainer™ tubes operations around FDA quality systems, we are still able to use all our proprietary methods and technology, including our devices, which were cleared for use this summer based on studies with more than 800 patient samples, including head-to-head comparisons to non-Theranos technology on venous samples.
FDA is reviewing the same tube that was cleared for the HSV-1 test. We've already met their rigorous standards in our first submission on our system, and we'll continue to do so.
Moreover:
The article confused the Nanotainer™ tubes, which is merely a substitute for a large vial used to carry the blood sample from the point of collection to the analyzing device, for the Theranos devices themselves. By obfuscating this issue, the reporter makes Theranos' recent decision to move our Nanotainer™ tube operations from the CLIA laboratory quality framework to the FDA quality (QSR) framework sound like something it is not.
The article implied that there is something suspect about the fact that we are the only lab to embrace, rather than fight, FDA regulation. We are the only lab with sufficient confidence in our tests to submit all of them for FDA review and clearance. Theranos is doing the right thing, which is also the hard thing, and we are charting a new path.
Additional background on the sources: The reporter purportedly spent five months working on a story about Theranos but his story did not reflect the views of the many eminent scientists, thought leaders, hospitals, and business partners who have worked and partnered with the company around our technology and found it to be innovative and revolutionary.
Instead, the reporter relied on:
Anonymous sources. This allowed the reporter to avoid having to reveal the biases and lack of knowledge of the sources.
Former employees. It is apparent that these former employees did not understand Theranos' technology; are unfamiliar with the processes currently employed by the company; and do not understand the regulatory framework that Theranos operates under. But by making them anonymous, readers do not know when they worked at Theranos, or for how long; whether they even worked in Theranos' laboratory, as opposed to some other division in the company; or why they left.
Lab experts. The reporter attributes certain statements to "lab experts." Most of these "experts" are anonymous.
There is no reason to keep these experts anonymous. All anonymity does is obfuscate their qualifications, and the potential that they have a vested interest in criticizing Theranos—such as being one of Theranos' competitors.
For example, the reporter relies on unidentified "lab experts" for the proposition that "finger-pricked blood samples can be less pure than those drawn from a vein because finger-pricked blood often mixes with fluids from tissue and cells that can interfere with tests." This is not dissimilar from a previous quote from Quest Diagnostics, who said nearly the exact same thing to The New Yorker in December 2014, when Nigel Clarke, one of Quest's senior scientific directors, argued "that finger-stick blood tests aren't reliable for clinical diagnostic tests; because the blood isn't drawn from a vein, the sample can be contaminated by lanced capillaries or damaged tissues." http://www.newyorker.com/magazine/2014/12/15/blood-simpler
Other sources. While ignoring Theranos' on the record statements, and our laboratory director and senior scientist, the reporter offered great latitude and seemingly abandoned his skepticism when speaking to Theranos critics. For example, the reporter relies on Rochelle Gibbons, but he failed to report that:
Rochelle Gibbons is a friend of an individual whom Theranos sued for patent invalidity and other claims. During litigation, last year, that individual repeatedly contacted Mrs. Gibbons via email to solicit her testimony against Theranos.
When that friend tried to use statements by Mrs. Gibbons against Theranos in litigation, she refused to make any statement under oath, and the court found that her statements should not be credited.
Mrs. Gibbons was married to a Theranos employee who passed away in May 2013. Mrs. Gibbons never worked for the company and had no input into Theranos' business, let alone its technology.
We told this to the reporter, including in writing, on July 1, 2015, and July 26, 2015.
From his very first interactions with Theranos, the reporter made abundantly clear that he considered Theranos to be a target to be taken down, and not simply the subject of an objective news story. The articles that appeared last week are the inevitable product of that approach.