A fluid, game-changing combination of mathematical tools and Big Data seems ready to disrupt the field of radiology. However, it also promises to pave the way for what may turn out to be potentially-dramatic advances in healthcare.
There is some irony here. Data was once seen as a liability, to maintain and pay for. It is now being considered a potentially major asset. The key to this turnaround in perspectives lies in increasingly sophisticated, deep learning algorithms, advanced analytics and artificial intelligence which interpret the Big Data and make it usable.
Explosion in image numbers and volume
There is no hyperbole in the use of the term Big Data, as far as radiology is concerned. In recent years, there has been a veritable explosion in the stock of medical images. Emergency room radiologists often examine up to 200 cases a day, and each patient’s imaging studies can be around 250 GB of data. At the upper end, a ‘pan scan’ CT of a trauma patient can render 4,000 images. Currently, about 450-500 petabytes of medical imaging data are generated per year, but this is accelerating. Decisions are made on the basis of small parts of imaging data, the proverbial tip of the iceberg. Much of the information in this data has still to be deciphered and used.
Medical imaging and disease
Medical imaging provides important information on anatomy and organ function as well as detecting diseases states. Its analysis covers a gamut of areas from image acquisition and compression, to transmission, enhancement, segmentation, de-noising and reconstruction.
Technology has enabled often-dramatic leaps in image resolution, size and availability. Sophisticated picture archiving and communications systems (PACS) have allowed for the merger of patient images from different modalities and their integration with other patient data for analysis and use in a clinical setting.
Limits to vision – from digital to analogue
So far, radiology information to identify disease or other clinical conditions is presented in the form of images. Although scanners digitize data into pixels, this is reconstructed into shapes and shades or colours for display in a form that can be understood by the human brain.
This is where the ‘tip of the iceberg’ statement above comes into play. Medical scanners encode an image pixel in 56 bits, equivalent to 72,000 trillion ‘shades’. However, the scanner reduces the data amount to 16 bits, just 65,536 shades, for the human eye. As a result, 40 bits of information is lost, in just one pixel.
At some point in the future, it seems likely that radiologists use numbers rather than images to numerically define and detect patterns of diseases. The process may in fact have already begun.
Imaging analytics and deep learning
Such trends are being fuelled by rapid advances in imaging analytics. Smart, deep learning (DL) algorithms, which analyse pixels and other digital data bytes within an image, have the capacity to detect specific patterns associated with a pathology and provide conclusions in terms of a numerical metric.
One example of the use of numbers as a diagnostic definition concerns the use of algorithms in CT images to calculate bone density. The result is compared to a reference number, which au tomatically trigger alerts on low bone density. Avoiding the need for another dedicated examination, a physician can determine if a patient needs calcium supplements or another preventative measure.
Such algorithms also learn over time, and become better at what they do, resulting in even greater speed and more confidence in the future. Such a process has been driven by the steady acceleration, over the years, in computer processing speed. Indeed, while training an algorithm at the turn of the century took 2-3 months, the same results can now be achieved and iterated within minutes.
Neural systems and algorithms
Technically, deep learning produces a direct mapping from raw inputs to outputs such as image classes. Many DL algorithms are inspired by biologic neural systems. They are different from traditional machine learning, which requires manual feature extraction from inputs, and face limitations to use in the face of the large volumes of information associated with Big Data.
Big Data’s virtuous circle
Many DL algorithms directly seek to harness Big Data in radiology. Gigantic (and fast-growing) image libraries are being accessed for investigation to develop, test, validate and continuously refine algorithms, with the aim of covering a whole range of pathologies.
For radiologists, analytic results from an examination can be comprehensively evaluated against similar data obtained over a long period of time and evaluated to suggest appropriate diagnosis in current scenarios.
Such a virtuous cycle of algorithms and Big Data have become the focus for a host of major medical technology vendors as well as start-ups. However, the key enabling players are radiology departments, who own the data repositories and are uniquely placed to curate the data, in other words, organize it from fragments and make it available for running analytical algorithms.
The above process has, in some senses, been jump-started by previous efforts to data mine reports from radiology departments as they transitioned from PACS to enterprise imaging. The next step in this Big Data-driven opportunity will consist of linking information in radiology reports to the pixels of medical images.
The pixel goldmine
Few doubt any more that pixels are a goldmine, holding wholly new insights into a medical image and how best they could be utilized, not just by radiologists but other clinicians offering patient care. Alongside data mined from electronic medical records, quantitative pixel-based analysis algorithms are increasingly likely to be used to find patterns in images.
Big Data-based screening algorithms, for example, can be used to highlight subtle, multi-dimensional changes in a nodule or a lesion. This can be followed by applications such as curved planar or 3D multi-planar reconstructions, or dynamic contrast enhancement (DCE) texture analysis on highly targeted data subsets, instead of making the time-consuming effort of querying a complete imaging dataset.
Specific examples of such an approach might include diagnosis of lesions in the liver and identification of disease-free liver parenchyma. Another would be volume analysis of lung tumours and solitary pulmonary nodules to decide temporal evolution of lesion. Big data based pattern analysis modules can detect areas of opacities, honeycombing, reticular densities and fibrosis, and thereby provide a list of differentials, using computer aided diagnostic tools.
For tumours, in general, radiologists can run algorithms to check contrast enhancement characteristics, and such metrics can be compared to prior results as well as other pathology data to provide a specific differential list.
Decision support systems
One decision support system based on Big Data assists physicians in providing treatment planning for patients suffering from traumatic brain injury (TBI). The algorithm couples demographic data and medical records of the patient to specific features extracted from CT scans in order to predict intracranial pressure (ICP) levels.
Google’s entry into this field seeks to address real world limitations – not just in terms of human capacities but also trained medical personnel. Its first deep learning imaging algorithm sought to recognize diabetic retinopathy, the fastest growing cause of blindness in poor countries, where a shortage of specialists meant many patients lost their sight before diagnosis.
The promise of AI
Google’s algorithm is based on artificial intelligence (AI), seen as an especially promising catalyst for advances in such areas.
AI-based algorithms, for example, can calculate the volume of bleed on the basis of multiple brain CT slices in stroke patients, with the size of bleed volume indicating urgency as well as care pathway. Another recent algorithm assesses recent infarcts on CT, which can be missed if they are hyper-acute (less than 8-12 hours old), and is therefore relevant to all patients with sudden onset weakness. The University of California in San Francisco has been testing an algorithm to identify pneumothorax in chest radiographs of surgery patients, before they exit the OR (operating room). The aim is to not only avoid the huge costs of a collapsed lung but also ensure that the OR is freed from being used for an otherwise-avoidable procedure.
AI is also being considered for workflow management and triaging. In the near future, it is almost certain that images are screened as data is acquired by a scanner, to distinguish between ‘normal’ and ‘abnormal’ images, prioritize cases according to the likelihood of disease and alerting radiologists to conditions that require urgent attention. The results are tangible and impressive. One algorithm has helped physicians to shrink the time for cardiac diagnoses from 30 minutes to 15 seconds.
Certain vendors are leveraging AI to correlate findings on properties like morphology, cell density or physiological characteristics to expert radiologist’s reports, while taking additional clinical data such as biopsy results into account. Others use reasoning protocols as well as visual technologies such as virtual rendering to analyse medical images. This is then combined with data from a patient’s medical record to offer radiologists and clinicians decision-making support.
AI and the radiologist
So far, algorithms and emerging metrics are expected to be largely used as a complement to decisions made by radiologists.
However, at some point in the future, it seems plausible that radiologists no longer need to look at images at all. Instead, they would simply analyse outcomes of the algorithms.
Once again, AI is at play here. Apart from deep learning algorithms, radiology can claim to be witness to the first successes with the emerging science of ‘swarm’ AI, which helps form a diagnostic consensus by turning groups of human experts into super experts. Swarm AI is directly based on nature, which sees species accomplishing more by participating in a flock, school or colony (a ‘swarm’) than they can individually. One report, published in ‘Public Library of Science (PLOS)’, stated that swarm intelligence could improve other types of medical decision-making, ”including many areas of diagnostic imaging.”
In December 2015, a study in ‘IET Systems Biology’ reported about a swarm intelligence algorithm which assisted “in the identification of metastasis in bone scans and micro-calcifications on mammographs.” The authors, from universities in the UK and India, also reported about the use of the algorithm in assessing CT images of the aorta and in chest X-ray. They proposed a hybrid swarm intelligence approach to detect tumour regions in an abnormal MR brain image.
The future: human-machine symbiosis
AI is unlikely to become a replacement for radiologists, but a tool to help them. According to Curt Langlotz, MD, PhD, professor of radiology and biomedical informatics at Stanford, the “human-machine system always performs better than either alone.”
