New AI tool improves cognitive testing
William Souillard-Mandar ’12, MEng ’15 (left); Professor Randall Davis; and their team have created a fast, inexpensive way to improve cognitive testing.
Digital pen plus speedy data analysis improves a classic test that helps detect cognitive impairment.
One good piece of news in recent years is that people around the world are living longer. The downside to this news is that older age exposes more of us to varieties of cognitive decline, including Alzheimer’s disease. According to the Alzheimer’s Association, the number of people with this diagnosis is likely to grow from around 5 million today to some 13.5 million in 2050. What could help stem this tsunami of pending illness? A team from MIT and Lahey Hospital and Medical Center has created a tool that will make it easier to detect cognitive impairment, enabling treatment to begin earlier and, potentially, improve outcomes.
Professor Randall Davis, an expert in knowledge-based systems and human-computer interaction at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL), and Dana Penney, the director of neuropsychology in neurology at Lahey Hospital and Medical Center, have led research that improves a classic tool for diagnosis, the Clock Drawing Test (CDT). For 50 years doctors have asked patients with suspected impairment to take the two-part test. In the first part, people are asked to draw a clock face that shows a specified time on a blank piece of paper. On the second part, they copy a clock face that shows the same time. As deceptively simple as this test seems to be, it is cleverly designed to examine different aspects of cognition. The first half places more emphasis on functions such as memory and language, while the second half focuses more on spatial reasoning and executive functions.
“Comparing performance across the two halves of the test enables us to use the person as their own control,” Penney says. “This, in turn, affords an opportunity to detect subtle problems in cognitive function before there are overt errors. One potential outcome of this would be the possibility of detecting Alzheimer’s sooner than it would be ordinarily, perhaps offering an opportunity for pharmaceutical companies to develop medications that target the earlier stages of the disease.”
Davis and Penney worked with a team, including then-graduate student William Souillard-Mandar ’12, MEng ’15, to improve the test by using a digitizing pen that captures time-stamped pen coordinates, allowing them to analyze both the product — the drawing — and the process used by the subject to draw it. Over a 10-year study, Penney and her colleagues administered this novel version of the test to some 4,000 subjects; they then used machine learning techniques to create prediction models that resulted in substantially more accurate detection of impairment than the methods routinely used by clinicians.
But classifiers produced by machine learning are often opaque, i.e., they are sufficiently complex that it is very difficult to understand why they produced the result they did. The team understood that a complex and opaque algorithm was unlikely to be accepted by either doctors or their patients. In response, they worked to make the analysis more transparent.
“The analysis can be split into two phases: understanding what the individual drew, for instance that a set of pen points represents a clock’s minute hand, and, based on that understanding, deriving clinician-interpretable metrics that assess cognitive function together with an overall test outcome,” Souillard-Mandar says. “It’s that second phase where we have worked to make our models interpretable, so a clinician can easily understand how the algorithm reached its decision.”
The new test saves time, boosts accuracy, and it is easy to use and inexpensive. “The earlier you can detect an illness, the earlier you can treat and make lifestyle changes that can help,” says Davis. “There are drugs that can slow down the progress of the disease. So the earlier you start the more effective the drugs might be.”
For the past two years, Davis and the team have been working to bring their research into the clinical world. Davis, Penney, and investor Philip Cooper SM '84 founded a new company, Digital Cognition Technologies (DCT). Souillard-Mandar is director of machine learning, working to transform the analytics into a commercial clinical tool while continuing to make sure that the technology remains transparent to practitioners.
DCT’s product, called DCTclock, includes a digital pen and a microcomputer that transmits the test data from the pen to DCT’s servers. The company’s test of cognitive abilities can be conducted as easily and quickly as taking blood pressure, and the results are available in under a minute. Pharmaceutical companies are using the product in clinical tests now and DCT expects to offer the service to clinicians in 2018.
Development of the DCTclock could be timely, since Medicare has recently decided to include cognitive testing in annual health assessments for Americans over 65, says Davis. There is also wide interest in the test because it is a fast and inexpensive compared to some of the traditional tools used to look for signs of declining cognitive health, such as PET scans. “PET scans measure biology,” says Davis. “We can measure behavior and that reveals cognition.”