It was one of those amazing “we’re living in the future” moments. In an October 2013 press release, IBM declared that MD Anderson, the cancer center that is part of the University of Texas, “is using the IBM Watson cognitive computing system for its mission to eradicate cancer.”
Well, now that future is past. The partnership between IBM and one of the world’s top cancer research institutions is falling apart. The project is on hold, MD Anderson confirms, and has been since late last year. MD Anderson is actively requesting bids from other contractors who might replace IBM in future efforts. And a scathing report from auditors at the University of Texas says the project cost MD Anderson more than $62 million and yet did not meet its goals. The report, however, states: “Results stated herein should not be interpreted as an opinion on the scientific basis or functional capabilities of the system in its current state.”
“When it was appropriate to do so, the project was placed on hold,” an MD Anderson spokesperson says. “As a public institution, we decided to go out to the marketplace for competitive bids to see where the industry has progressed.”
The disclosure comes at an uncomfortable moment for IBM. Tomorrow, the company’s chief executive, Ginni Rometty, will make a presentation to a giant health information technology conference detailing the progress Watson has made in health care, and announcing the launch of new products for managing medical images and making sure hospitals deliver value for the money, as well as new partnerships with healthcare systems. The end of the MD Anderson collaboration looks bad.But IBM defended the MD Anderson product, known as the Oncology Expert Advisor or OEA. It says the OEA’s recommendations were accurate, agreeing with experts 90% of the time. “The OEA R&D project was a success, and likely could have been deployed had MD Anderson chosen to take it forward,” says an IBM spokesperson.
Watson, IBM’s language-based computing project, gripped the world’s imagination in 2011 when the supercomputer won an exhibition of the game show Jeopardy! against the show’s two highest rated players. In March 2012, IBM signed a deal with Memorial Sloan Kettering Cancer Center in New York to develop a commercial product that would use the same technology to analyze the medical literature and help doctors choose treatments for cancer patients.
MD Anderson, Memorial’s long-time rival, entered the fray after this agreement was already in place. Lynda Chin, the former chair of the MD Anderson Department of Genomic medicine and the wife of MD Anderson president Ronald DePinho, set up a collaboration with IBM to develop a separate project. Chin left MD Anderson for another job within the University of Texas system in 2015.
In a strange twist, MD Anderson would pay for the whole thing, eventually giving $39.2 million to IBM and $21.2 million to PricewaterhouseCoopers, which was hired to create a business plan around the product. According to the Washington Post, at least $50 million of the money came from Low Taek Jho, a flamboyant Malaysian financier whose business dealings are reportedly now under investigation by the U.S. Department of Justice.
Usually, companies pay research centers to do research on their products; in this case, MD Anderson paid for the privilege, although it would have apparently also owned the product. This was a “very unusual business arrangement,” says Vinay Prasad, an oncologist at Oregon Health & Science University.
According to the audit report, Chin went around normal procedures to pay for the expensive undertaking, even making sure individual payments to IBM were below a threshold that would have required her to get approval from MD Anderson’s board. She also didn’t get approval from the information technology department.
It seems “very strange” that the IT department was bypassed, and “very unusual” that payments were not based on measurable deliverables, says John Halamka, the chief information officer at Beth Israel Deaconness Medical Center in Boston. He also notes that payments seem to have been made from donations that had not yet been received.
Despite all this drama, initial reports on the MD-Anderson/Watson collaboration were positive. In 2015 The Washington Post said MD Anderson doctors-in-training were amazed by the machine’s recommendations. “I was surprised,” one told the newspaper. “Even if you work all night, it would be impossible to be able to put this much information together like that.”
But inside the University of Texas, the project was apparently seen as one that missed deadlines and didn’t deliver. The audit notes that the focus of the project was changed several times, first focusing on one type of leukemia, then another, then lung cancer. The initial plan was to test out the product out in pilots at two other hospitals. That never happened.
MD Anderson changed the software it uses for managing electronic medical records, switching to a system made by Epic Systems of Madison, Wis. It has blamed this new system for a $405 million drop in its net income. According to the audit report, the Watson product doesn’t work with the new Epic System, and must be revamped in order to be re-tested. The information in the MD-Anderson/Watson product is also now out of date.
In September, IBM stopped supporting the product, according the audit, which was produced last November. The Cancer Letter and The Houston Chronicle reported on the audit last week. Forbes obtained a copy of a request-for-proposals confirming that MD Anderson is actively looking for a company to take on IBM’s role. In a statement, MD Anderson said that it was not excluding companies that had previously worked with it from job, implying that it might choose to go with IBM to reboot the project.
Meanwhile, IBM now sells a product it developed with Memorial Sloan Kettering. The goal, as with the MD Anderson product, is to help doctors select treatments. Without a computer, this is done with a so-called “tumor board,” a group of experts who meet weekly. IBM points to a dozen studies presented at academic meetings showing that Watson’s recommendations agree with those of tumor boards.
When IBM CEO Rometty makes her announcements tomorrow at HIMSS, the health-tech conference, the question for doctors and investors will be this: are they more like the Memorial Sloan Kettering effort, which seems to have resulted in a real product? Or are they like the mess that seems to have happened at MD Anderson?