Important announcement: Missinglink has shut down. Click here to learn more.

Deep Learning in Healthcare Cover

Deep Learning in Healthcare

Artificial Intelligence in Radiology

Radiology has been around for over a century and triggered a number of innovations, starting with the discovery of X-rays and followed by many others: Positron Emission Tomography (PET) in the 60s, Computed Tomography (CT) in the early 70s, and MRI in the late 70s.


We are on the verge of a new era of innovation in radiology: the era of artificial intelligence (AI). Many in the radiology field have high hopes for AI, as it can have major benefits in the analysis of medical imaging for patient diagnosis and research.


What are the Benefits of Artificial Intelligence in Radiology?

Radiologists cannot afford to make mistakes; imaging or image analysis errors can provide a false diagnosis or miss critical information that can impact patient health. For example, the failure of a radiologist to identify a fracture on an X-ray could lead to a false diagnosis, which can lead to claims for compensation.


Radiologists also need to interact with different medical professionals, including referring physicians, neurologists, urologists, orthopedic practitioners, and provide accurate and consistent information. AI can support radiologists in several ways:

Differentiated diagnosisMany AI solutions provide additional diagnostic information. AI can provide a physician with complementary information about an image. For example, AI can add normative values, so physicians can compare the results of their patient to an average based on a cross-section of the population.
Offer a second opinionAI software provides an easy way to obtain a second opinion. Physicians can compare their findings to the findings of the AI.
Automate routine tasksAI can automate simple and straightforward routine tasks such as measuring size changes in lesions over time with treatment.
Eliminate variability between reportsAt times, even the best trained most experienced radiologists might differ in their diagnosis. AI software can decrease or even eliminate variability between radiologist reports.
Increase diagnosis efficiency


AI can help speed up the diagnosis process by automating tasks that are time-consuming when performed manually, such as RECIST score measurement.


AI can help the radiologist prioritize urgent cases. AI can do the first assessment and move cases up the list if necessary.

Types of AI Analysis In Radiology

Some radiology tasks require a medical image as input and will base the analysis purely based on the pixels. Others will go one step further and will combine radiological images with information obtained from other sources.


The two major types of radiology AI analysis are:


  1. Relying solely on an image as input—AI that relies solely on a medical image as input will deliver similar results to the manual results provided by radiologists. Monotonous tasks like automatic segmentation of specific organs and quantification of specific distances (an automatic measurement of RECIST scores) are suitable for AI.
  2. Adding information from other patient exams—radiologists may have difficulty combining medical images with other information. For example, by linking image data to pathology lab results, it is possible to use an algorithm to derive pathology information from a medical image. A different type of analysis involves adding normative information. For example, you can compare patient organ volumes to the average population volumes.

The Current Challenges for AI in Radiology

AI in radiology has tremendous potential, but there are still some challenges before it can be widely applied and adopted into the radiology workflow.


The main challenges of applying AI to the radiology field are:

Quality labeled dataA typical medical imaging dataset contains approximately 1000 images. A non-medical dataset can contain up to millions of images.
3D imagingMost deep learning models are currently trained on simple 2D pictures. CT and MRI images are usually 3D, adding an extra dimension to the problem. Deep learning algorithms are also not adjusted to the projected character of 2D X-ray images.
User experienceRadiology software is generally very user-unfriendly. Creating user-friendly software is a must for AI companies that want their software to be used in clinics.
Image acquisition


Different scanner types and acquisition settings result in a non-standardized acquisition of medical images. Non-standardized and variable data requires large datasets to make the deep learning algorithm robust.
Performance metrics


AI radiology software has to be very clear on what it measures, how the measurement is done and why. However, metrics such as accuracy, precision, and recall are not always available in measurements.
Patient trustAI is often seen as a “black box”, as it is often unclear as to how it arrived at a certain conclusion. When it comes to medical image analysis, AI radiology companies need to strengthen the user’s trust in AI.
Financial valueAI companies need to have a clear view of how their software will financially benefit hospitals in the future.

Test Case: Implementation of Blood Cell Detection using Faster R-CNN

We will present a healthcare related dataset, with the goal of solving a Blood Cell Detection problem. Our task is to detect all the Red Blood Cells (RBCs), White Blood Cells (WBCs), and Platelets in each image taken via microscopic image readings.


Manually looking at the sample via a microscope is a tedious process. This is why artificial intelligence plays such a vital role. AI can classify and detect the blood cells from microscopic images with impressive precision.


The neural network used in this test case is Faster R-CNN, a convolutional neural network (CNN) architecture which provides a framework for object detection. Learn more about convolutional neural networks and using R-CNN with TensorFlow.


The tutorial below is based on the Github library by Shenggan.


1. Setting up the System

Ensure that the right libraries and frameworks have been installed. The following libraries are required to run this project: pandas, Matplotlib, TensorFlow, Keras, NumPy, OpenCV-Python, Scikit-learn, H5py.


2. Data Exploration

Create two folders to review the images you’ll be working with:

  • train_images: Contains images that we will be used to train the model.
  • test_images: Contains images that will be used to make predictions using the trained model.

Create a train.csv file with the name, class and bounding box coordinates for each image.


3. Read the training CSV file

import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
from matplotlib import patches

train = pd.read_csv(‘train.csv’)


4. Implementing Faster R-CNN

To implement the Faster R-CNN algorithm, first clone the repository (linked above). Open a new terminal window and type:

git clone

Move the train_images and test_images folder, as well as the train.csv file, to the cloned repository. In order to train the model on a new dataset, the format of the input should be:


The parameters are as follows:

  • filepath is the path of the training image
  • x1 is the xmin coordinate for bounding box
  • y1 is the ymin coordinate for bounding box
  • x2 is the xmax coordinate for bounding box
  • y2 is the ymax coordinate for bounding box
  • class_name is the name of the class in that bounding box


5. Train the model using the file

cd keras-frcnn
python -o simple -p annotate.txt

Manage Your Artificial Intelligence in Radiology Experiments With MissingLink

Running artificial intelligence in radiology experiments involves intensive tasks that require powerful hardware, and might prove challenging if you need to manage multiple experiments simultaneously.


MissingLink provides a platform that can easily manage deep learning experiments. With MissingLink you can schedule, automate, and record your experiments. This allows you to track all your experiments, code, machines and results.

missinglink screenshot

Train Deep Learning Models 20X Faster

Let us show you how you can:

  • Run experiments across hundreds of machines
  • Easily collaborate with your team on experiments
  • Reproduce experiments with one click
  • Save time and immediately understand what works and what doesn’t

MissingLink is the most comprehensive deep learning platform to manage experiments, data, and resources more frequently, at scale and with greater confidence.

Request your personal demo to start training models faster

    Thank you!
    We will be in touch with more information in one business day.
    In the meantime, why not check out how Nanit is using MissingLink to streamline deep learning training and accelerate time to Market.