Design Status

This is a prototype.

Purpose

This web app uses computer vision to detect viral and bacterial pneumonia on chest x-rays. It’s a triage tool to help doctors identify high risk children fast.

Image format

The app accepts images in jpg or png format.

Limitations

The Ai brain (model) that powers the app was trained using x-rays of children aged between 1 and 5 years old. There’s no guarantee that it will produce accurate results when given x-rays of adults.

Also the prediction accuracy could be affected by factors like x-ray machine type, varying image lighting conditions, damaged x-ray images (e.g. pen marks) and other factors.

Published Design

The design code and the step-by-step process used to train and test the model has been published on Kaggle. You can find the open source notebook here. The test results are also available in the notebook.

Dataset Licence

The model was trained using image data that was released under a CC BY 4.0 licence.

---

Will I need to wait for the model to download every time I use the app?

No. Once the model downloads it will be cached in memory. The next time you visit this site you won’t need to wait.

That said, I suggest that you always use this tool with your web browser set to incognito mode. The model won’t be saved, but this will ensure that each time you visit this site you will always use the most up to date version of the model and not the old one that's stored in your device's memory.

What about patient data privacy?

Your images are never sent to an external server. Any image you submit stays on your computer or mobile phone. This is because the model is running on your device. This is also why a batch of images can be analyzed in less than two seconds.

The tech that makes this possible is called Tensorflow.js. It was created by Google.

What do the decimal numbers mean?

They are probability scores between 0 and 1. They show how confident the model is that a particular condition is present on the image.
For example, Viral Pneumonia: 0.751 means that the model is 75% sure that viral pneumonia is present on the x-ray.

How do I use this app on a computer?

- Go to this site on your computer.
- Click the orange button.
- Select one image or multiple images and click ‘Open’.
- The app will print results for each image.

How do I use this app on a mobile phone?

You can take a new photo or you can submit photos that are stored on your phone. The steps below apply to Android. iOS may be similar but I'm not sure.

[ 1 ] Take a photo

- Go to this site on your phone.
- Tap the orange button.
- Select “Camera”
- Take a photo
- Tap the "tick"
- The photo will be sent to the app for analysis.

[ 2 ] Use images from your photo folder

- Go to this site on your phone.
- Tap the orange button.
- Tap "Files"
- Tap the image name
- To submit a batch of images, tap each image (tick) to select it and then tap "Open".

What would it take to make this tool more accurate?

Two things are needed - more high quality labeled training data and field testing.