Example of BlindAI deployment with COVID-Net¶
This example shows how you can deploy COVID-Net model to analyze X-Ray scans of chest X-rays to detect COVID.
By using BlindAI, people can send data for the AI to analyze their medical images without having to fear privacy leaks.
COVID-Net is a deep CNN to detect COVID from chest X-rays. You can learn more about it on the COVID-Net repository.
More information on this use case can be found on our blog post Confidential medical image analysis with COVID-Net and BlindAI.
Installing dependencies¶
Install the dependencies this example needs.
!pip install pillow numpy opencv-python onnxruntime matplotlib
Install the latest version of BlindAI.
!pip install blindai
Preparing the model¶
For this example, we will directly download a Covid-NET model that has already been trained. We have pre-exported the model from the COVID-Net repository in ONNX file so no need to export it.
Because the file is rather big, the download might take some time.
!wget --quiet --load-cookies /tmp/cookies.txt "https://docs.google.com/uc?export=download&confirm=$(wget --quiet --save-cookies /tmp/cookies.txt --keep-session-cookies --no-check-certificate 'https://docs.google.com/uc?export=download&id=1Rzl_XpV_kBw-lzu_5xYpc8briFd7fjvc' -O- | sed -rn 's/.*confirm=([0-9A-Za-z_]+).*/\1\n/p')&id=1Rzl_XpV_kBw-lzu_5xYpc8briFd7fjvc" -O COVID-Net-CXR-2.onnx && rm -rf /tmp/cookies.txt
Deployment on BlindAI¶
Now we can upload the model to BlindAI Cloud. To upload of the model, make sure you have an API key.
You can get one on the Mithril Cloud.
You might get an error if the name you want to use is already taken, as models are uniquely identified by their model_id. We will implement namespace soon to avoid that. Meanwhile, you will have to choose a unique ID. We provide an example below to upload your model with a unique name:
import blindai
import uuid
api_key = "YOUR_API_KEY" # Enter your API key here
model_id = "covidnet-" + str(uuid.uuid4())
# Upload the ONNX file to the remote enclave
with blindai.Connection(api_key=api_key) as client:
response = client.upload_model("COVID-Net-CXR-2.onnx", model_id=model_id)
Sending data for confidential prediction¶
Now it's time to check it's working live!
We will just prepare some input for the model inside the secure enclave of BlindAI to process it.
First, we need to fetch the CXR image to send to the model.
!wget --quiet https://raw.githubusercontent.com/lindawangg/COVID-Net/master/assets/ex-covid.jpeg
We will use the same preprocessing functions as in the COVID-Net repository.
# This uses OpenCV for image processing
import cv2
def crop_top(img, percent=0.15):
offset = int(img.shape[0] * percent)
return img[offset:]
def central_crop(img):
size = min(img.shape[0], img.shape[1])
offset_h = int((img.shape[0] - size) / 2)
offset_w = int((img.shape[1] - size) / 2)
return img[offset_h:offset_h + size, offset_w:offset_w + size]
def process_image_file(filepath, size, top_percent=0.08, crop=True):
img = cv2.imread(filepath)
img = crop_top(img, percent=top_percent)
if crop:
img = central_crop(img)
img = cv2.resize(img, (size, size))
return img
We can now load the image we have downloaded and preprocess it.
import numpy as np
img = process_image_file("./ex-covid.jpeg", size=480)
img = img.astype("float32") / 255.0
img = img[np.newaxis,:,:,:]
We can have a look at the model input.
import matplotlib.pyplot as plt
plt.imshow(img[0])
For data marshalling reason, we will flatten the image and convert it to a list.
We can then send the data to be processed by the BlindAI server!
with blindai.Connection(api_key=api_key) as client:
# Send data to the ResNet18 model
response = client.predict(model_id, img)
plt.imshow(img[0])
plt.title(f"Probability of COVID positivity: {response.output[0].as_flat()[1]}")
print(response.output)
Here we can compare the results against the original prediction.
import onnxruntime
ort_session = onnxruntime.InferenceSession("COVID-Net-CXR-2.onnx")
ort_inputs = {ort_session.get_inputs()[0].name: img}
onnx_outputs = ort_session.run(None, ort_inputs)
print(f"Probability of COVID positivity from original model: {onnx_outputs[0][0][1]}")