Accelerating medical discovery using Deep Learning
Most discoveries in medicine are named after people who discovered it. They are called eponyms and they make learning medicine ridiculously harder. Just take a look at this list of all diseases named after people, and you’ll know that the struggle is real.
Look at the electron microscopic structure of this organelle for example.
If the same guy who had named Walky Talky had named it, he would have called it Jiggly Wiggly or something. But, it was discovered by Professor Camillo Golgi and he named it the Golgi Apparatus.
Not that I’m complaining.
But one fundamental thing that this trend conveys is that if you are smart enough to observe something that other people haven’t yet observed, your name shall be etched into an organelle, a syndrome or a disease.
When someone is diagnosed with cancer, a sample of the tumor is taken and histologists mostly rely on what they read from textbooks to detect certain indicators in the samples that they observe under the microscope to determine what kind of cancer it is. These indicators and features are usually discovered by “really observant” scientists.
We can do better than just waiting for a really observant scientist to report the next most important indicator to detect such diseases. Someone’s life depends on it.
Enter Deep Learning!
Deep Learning for Diagnosis
Both my parents are Ophthalmologists. I have seen many camps and campaigns for screening people with Diabetes for an eye condition called Diabetic Retinopathy.
In one of our research projects, we fed Deep Learning models with photos of the retina taken using a special camera and trained it to identify at what stage of Diabetic Retinopathy it was at.
It did a pretty neat job. Google had done some pioneering research along the same lines and had even obtained super-human accuracy of detection.
We trained a Convolutional Neural Networked with a labeled data-set obtained from a competition for Diabetic Retinopathy detection.
Our network architecture was nothing out of the ordinary. Just plain copy-pasting code from Google’s trained Inception Network and then retraining the final layer of the network on our data-set to fine tune it to detect stages of Diabetic Retinopathy instead of cats and dogs.
The network performed really well on the task of classification. But here is where things got interesting. We wanted to see what the network had actually learnt in order to this.
Deep Learning for Discovery
Deep Convolutional Neural Networks have always had this “black-box” like feeling attached to them. They do what they do really well, but no one really understands how they do it. It kind of like, we know that the humans can come up with new ideas, but don’t exactly understand the neural mechanisms of how.
In order to gain a perspective of what a Deep Learning algorithm has learnt, there have been various attempts previously. One of the popular ones which is also infamously creepy is the Deep Dream algorithm.
For example, take a network trained on pictures of Indian Cobras and Stethoscope. We can learn what the network thinks is a cobra or a stethoscope, by asking it to generate a picture of them from scratch. First, we start with randomly generated pixels, which looks like TV static. Then we change the pixels slowly until the neural network thinks its actually something.
We can see that it has learnt some features of an Indian Cobra, as our brains are tricked into thinking that there is a cobra in the above picture as well. In reality, there is not. It is a picture generated completely from scratch to make the activation of the neuron for Indian Cobra go nuts for a neural network.
With the same technique of pixel optimization if we feed an existing image instead of random pixels and ask the neural network to optimize the pixels for something it gives really weird results.
If you give a picture of a tree and ask the neural network to optimize it for buildings, it “sees” the “building in the tree” and enhances those features until it becomes a building.
We see that all the features that make a “building”, a building is present.
Now, let’s get back to our problem of Diabetic Retinopathy.
We gave a normal picture of the Retina and asked our neural network to optimize for Proliferative Diabetic Retinopathy (PDR), the worst form of the disease.
Let’s take a closer look. We can see that our visualization is rendering the textbook features of Diabetic Retinopathy like Exudates, Haemorrhages and Venous Beading very well.
Even some subtle features like microaneurysms and neovascularization are being generated.
This is a network that trained on just 1000 retinal images(something that could be generated in about a week of heavy ophthalmology practice or diabetes screening) on the different stages of the disease.
Compared to the 20 years it took for thewhole of human kind to come up all the indicators of Diabetic Retinopathy that we use now, and considering the fact that it just took about 1 hour to train this network on a GPU instance, it’s not that bad actually.
Conclusion
So, now coming back to our aunt with cancer. If the hospital had just collected and stored all of the microscopic images from patients just mentioning which patients ended up dying from cancer and which ones didn’t and we had run this deep learning algorithm to keep making inferences about the indicators, we wouldn’t need a “very observant” scientist. The visualization algorithm would scream in your face with all the features. If we find something that we don’t know about before, we would know something is up.
Think about all that data sleeping in data silos in the EMR department of hospitals! If we can just use Deep Learn…
Alright, Let’s focus on the negative aspects of this algorithm and future scope for improvement.
Flaw #1: Too many artifacts
We already know about all the features that we have “re-discovered” in this case. What if we had made sense of the visualizations only because we knew about it. All the other “new features” that the network was showing us could have been considered by us as artifacts.
Flaw #2: Bad color representation
Most of the visualization was rendered in colors that were not related to the original color. The shape is fine, but the color reproduction must to on point for it to be a tool that can be used with confidence to point out new features of diseases.
These are things we are working on improving. The future of Deep Learning in Medicine is bright. We have only scratched the surface.
If you liked the article be sure to check out our paper that mentions in more detail the technical architecture. Contact me for suggestions, contributions or criticism.
