Using Robotics and Artificial Intelligence To Mitigate the Reproducibility Crisis

Researchers led by the University of Cambridge have employed Eve, a “Robot Scientist,” to help address the reproducibility - or replicability - crisis which has long been an issue scientists and researchers have hoped to overcome.

Using Robotics and Artificial Intelligence To Mitigate the Reproducibility Crisis.

Image Credit: Shutterstock.com/ Andrey Suslov

Since the scientific revolution in the 17th century, reproducible results have been a scientific cornerstone that has ushered in wave after wave of progress.

However, in spite of the necessity and value of reproducible results, this is not tested on a large scale, and when reproducibility is tested, it can be extremely difficult to observe reproducibility.

This is what is known as the reproducibility crisis, which impedes the capability to reproduce experiments, yet this failure is a familiar aspect of research. Still, the ongoing issue of reproducibility is one of the largest crises modern science is facing.

The researchers developed a system that combines automated text analysis with Eve to aid the process of finding reproducible research results by analyzing more than 12,000 research papers that cover the biology of breast cancer cells. After managing to reduce the data set down to just 74 papers of significant scientific interest, just 22 of the initial 12,000 were reproducible.

One of the big advantages of using machines to do science is they’re more precise and record details more exactly than a human can.

Professor Ross King, Department of Chemical Engineering and Biotechnology, University of Cambridge

Of course, the seemingly obvious solution to combat the reproducibility crisis would be to recruit more scientists to try to reproduce the work of other scientists. However, there are a series of sociological and career concerns that inhibit a clear path to doing so: it is difficult to find funding, and many authors do not appreciate having their results doubted or challenged.

Attempts have been made to determine exactly what factors are crucial to the reproducibility of scientific results. However, due to the costs, the current funding model, and difficulties that arise when trying to confirm experimental results, it is improbable that human researchers will be able to conduct experiments that prove just a small portion of published results.

King and his team set about addressing these challenges by finding a feasible way to boost the amount of reproduced scientific results; one way is to bring automation to the table.

Thus, to ensure effective automation, the integration of text mining (extracting results from the literature) in combination with AI-based lab automation (to test the reproducibility of the results) could be a viable method.

Having received funding from DARPA, the research team developed an experiment with the application of AI and robotics in order to tackle the reproducibility crisis; by having AI systems read scientific papers and extract knowledge, the team used Eve to try to reproduce the results.

The cancer literature is enormous, but no one ever does the same thing twice, making reproducibility a huge issue.

Professor Ross King, Department of Chemical Engineering and Biotechnology, University of Cambridge

“Given the vast sums of money spent on cancer research, and the sheer number of people affected by cancer worldwide, it’s an area where we urgently need to improve reproducibility,” King continued.

From an initial data set of over 12,000 peer-reviewed papers, the researchers applied automated text mining to identify and pull statements where changes in gene expression as a response to drug treatment in breast cancer were clearly observed.

Subsequently, two human-led teams working with Eve and two breast cancer cell lines made an attempt to reproduce the results of the 74 papers the text mining was able to extract.

Evidence of repeatability with statistical significance was discovered in 43 of the papers, which indicates results are replicable under identical conditions, and there was strong evidence for reproducibility found in 22 papers, meaning the results could be replicated under similar lab conditions by different scientists.

While just under 1/3 of the papers demonstrated robust reproducibility in the automation-assisted experiment, the researchers made clear that this does not undermine the reproducibility or robustness of the remaining papers.

There are lots of reasons why a particular result may not be reproducible in another lab … Cell lines can sometimes change their behaviour in different labs under different conditions, for instance. The most important difference we found was that it matters who does the experiment, because every person is different.

Professor Ross King, Department of Chemical Engineering and Biotechnology, University of Cambridge

This work demonstrates that semi- and fully-automated laboratory techniques could be crucial to overcoming some of the challenges that the reproducibility crisis presents. Furthermore, the team believes that reproducibility should be a standard part of the whole scientific process.

It’s quite shocking how big of an issue reproducibility is in science, and it’s going to need a complete overhaul in the way that a lot of science is done…We think that machines have a key role to play in helping to fix it.

Professor Ross King, Department of Chemical Engineering and Biotechnology, University of Cambridge

References and Further Reading

Collins, S., (2022) ‘Robot scientist’ Eve finds that less than one third of scientific results are reproducible. [online] University of Cambridge. Available at: https://www.cam.ac.uk/research/news/robot-scientist-eve-finds-that-less-than-one-third-of-scientific-results-are-reproducible

Roper, K., et al., (2022) Testing the reproducibility and robustness of the cancer biology literature by robot. Journal of The Royal Society Interface, [online] 19(189). Available at: https://royalsocietypublishing.org/doi/10.1098/rsif.2021.0821

Disclaimer: The views expressed here are those of the author expressed in their private capacity and do not necessarily represent the views of AZoM.com Limited T/A AZoNetwork the owner and operator of this website. This disclaimer forms part of the Terms and conditions of use of this website.

David J. Cross

Written by

David J. Cross

David is an academic researcher and interdisciplinary artist. David's current research explores how science and technology, particularly the internet and artificial intelligence, can be put into practice to influence a new shift towards utopianism and the reemergent theory of the commons.

Citations

Please use one of the following formats to cite this article in your essay, paper or report:

  • APA

    Cross, David. (2022, September 06). Using Robotics and Artificial Intelligence To Mitigate the Reproducibility Crisis. AZoRobotics. Retrieved on September 25, 2022 from https://www.azorobotics.com/News.aspx?newsID=12878.

  • MLA

    Cross, David. "Using Robotics and Artificial Intelligence To Mitigate the Reproducibility Crisis". AZoRobotics. 25 September 2022. <https://www.azorobotics.com/News.aspx?newsID=12878>.

  • Chicago

    Cross, David. "Using Robotics and Artificial Intelligence To Mitigate the Reproducibility Crisis". AZoRobotics. https://www.azorobotics.com/News.aspx?newsID=12878. (accessed September 25, 2022).

  • Harvard

    Cross, David. 2022. Using Robotics and Artificial Intelligence To Mitigate the Reproducibility Crisis. AZoRobotics, viewed 25 September 2022, https://www.azorobotics.com/News.aspx?newsID=12878.

Tell Us What You Think

Do you have a review, update or anything you would like to add to this news story?

Leave your feedback
Your comment type
Submit