Scientists have developed a new test that scans the shapes of tumor cells to pick out women with especially aggressive ovarian cancer, so treatment can be tailored to their needs.

A team at The Institute of Cancer Research, London, created an artificial intelligence (AI) tool that looks for clusters of cells within tumors with misshapen nuclei—the control centers within each cell.

Women identified with these clusters of shapeshifting cells had extremely aggressive disease—with only 15% surviving for 5 years or more, compared with 53% for other patients with the disease.

The researchers found that having misshapen nuclei was an indication that the DNA of cancer cells had become unstable—and believe it could in future help doctors to select the best treatment for each patient. Cancers with misshapen cell nuclei had hidden weaknesses in their ability to repair DNA, which could make them susceptible to drugs called PARP inhibitors or platinum chemotherapy.

The researchers also found that immune cells were not able to move into the clusters of cells with misshapen nuclei, which suggests that cancers with these clusters are better at evading the immune system. Understanding this immune escape mechanism could help develop new forms of immunotherapy to combat ovarian cancer.

Potential target for new immunotherapies

Scientists at The Institute of Cancer Research (ICR) applied their powerful new computer tool to automatically analyze tissue samples from 514 women with ovarian cancer—together looking at nearly 150 million cells.

The study, published in the journal Nature Communications and funded by the ICR itself, used AI to look at the shape and spatial distribution of ovarian cancer cells and their surroundings.

The researchers found that tumors containing clusters of cells whose nuclei varied highly in shape had lower levels of activity of key DNA repair genes, including BRCA1. The test could be used to pick out tumors with lower levels of activity of DNA repair genes, even in cases where the genetic code of the BRCA genes remains intact. These hidden DNA repair defects would be overlooked when only testing for faults in DNA repair genes. The presence of clusters was associated with even worse prognosis than mutations in the BRCA genes.

The team at the ICR—a research institute and a charity—also found that the clusters had higher levels of a protein called galectin-3, which is known to cause key immune cells to die. The researchers believe that galectin-3 could represent a brand new escape route from the immune system in ovarian cancer and a potential target for new immunotherapies—although further research is needed.

Detecting tumors with hidden weaknesses

Dr. Yinyin Yuan, Team Leader in Computational Pathology at the ICR said:

“We have developed a simple new computer test that can identify women with very aggressive ovarian cancer so treatment can be tailored for their needs. Using this new test gives us a way of detecting tumors with hidden weaknesses in their ability to repair DNA that wouldn’t be identified through genetic testing. It could be used alongside gene testing to identify women who could benefit from alternative treatment options that target DNA repair defects, such as PARP inhibitors. Our test also revealed that ovarian tumors with these clusters of misshapen nuclei have evolved a new way of evading the immune system, and it might be possible to target this mechanism with new forms of immunotherapy.”

Professor Paul Workman, Chief Executive of The Institute of Cancer Research, London, said:

“This extremely clever new study has shown that by using AI to analyze routinely taken biopsy samples, it is possible to uncover visual clues that reveal how aggressive an ovarian tumor is. What makes this test even more exciting is its ability to pick out in a new and different way those women whose tumors have weaknesses in DNA repair—who might therefore respond to treatments that target these weaknesses.”