Example of BlindAI deployment with DistilBERT¶

This example shows how you can deploy DistilBERT model to analyze confidential text for sentiment analysis. The model will be left untrained for demonstration purposes. We could finetune it on positive/negative samples before deploying it.

By using BlindAI, people can send data for the AI to analyze sensitive text without having to fear privacy leaks.

DistilBERT is a state of the art Transformers model for NLP. You can learn more about it on this Hugging Face page.

More information on this use case can be found on our blog post Deploy Transformers models with confidentiality.

Installing dependencies¶

Install the dependencies this example needs.

In [ ]:

!pip install transformers[onnx] torch

!pip install transformers[onnx] torch

Install the latest version of BlindAI.

In [ ]:

!pip install blindai

!pip install blindai

Preparing the model¶

In this first step we will export a standard Hugging Face DistilBert model to an ONNX file, as BlindAI accepts only ONNX files.

In [ ]:

from transformers import DistilBertForSequenceClassification

# Load the model
model = DistilBertForSequenceClassification.from_pretrained("distilbert-base-uncased")

from transformers import DistilBertForSequenceClassification # Load the model model = DistilBertForSequenceClassification.from_pretrained("distilbert-base-uncased")

Then, we need to export this Pytorch model to ONNX. To do the export, we need an example for dynamic tracing to see how an input is processed by the graph. This may take a little while.

We have fixed a max length of 8 for simplicity. Having a fixed max size makes it easier for deployment. If the input is shorter, we can simply pad it to the max size.

In [ ]:

from transformers import DistilBertTokenizer
import torch

# Create dummy input for export
tokenizer = DistilBertTokenizer.from_pretrained("distilbert-base-uncased")
sentence = "I love AI and privacy!"
inputs = tokenizer(sentence, padding = "max_length", max_length = 8, return_tensors="pt")["input_ids"]

# Export the model
torch.onnx.export(
	model, inputs, "./distilbert-base-uncased.onnx",
	export_params=True, opset_version=11,
	input_names = ['input'], output_names = ['output'],
	dynamic_axes={'input' : {0 : 'batch_size'},
	'output' : {0 : 'batch_size'}})

from transformers import DistilBertTokenizer import torch # Create dummy input for export tokenizer = DistilBertTokenizer.from_pretrained("distilbert-base-uncased") sentence = "I love AI and privacy!" inputs = tokenizer(sentence, padding = "max_length", max_length = 8, return_tensors="pt")["input_ids"] # Export the model torch.onnx.export( model, inputs, "./distilbert-base-uncased.onnx", export_params=True, opset_version=11, input_names = ['input'], output_names = ['output'], dynamic_axes={'input' : {0 : 'batch_size'}, 'output' : {0 : 'batch_size'}})

Deployment on BlindAI¶

In [ ]:

import blindai
import uuid

api_key = "YOUR_API_KEY" # Enter your API key here
model_id = "distilbert-" + str(uuid.uuid4())

# Upload the ONNX file to the remote enclave
with blindai.Connection(api_key=api_key) as client:
    response = client.upload_model("distilbert.onnx", model_id=model_id)

import blindai import uuid api_key = "YOUR_API_KEY" # Enter your API key here model_id = "distilbert-" + str(uuid.uuid4()) # Upload the ONNX file to the remote enclave with blindai.Connection(api_key=api_key) as client: response = client.upload_model("distilbert.onnx", model_id=model_id)

This securely uploads the model to the Mithril Cloud. If you wish to run this example on premise, you should read the Deploy on Hardware documentation page.

Sending data for confidential prediction¶

Now it's time to check it's working live!

We will just prepare some input for the model inside the secure enclave of BlindAI to process it.

In [ ]:

from transformers import DistilBertTokenizer

# Prepare the inputs
tokenizer = DistilBertTokenizer.from_pretrained("distilbert-base-uncased")
sentence = "I love AI and privacy!"
inputs = tokenizer(sentence, padding = "max_length", max_length = 8, return_tensors="pt")["input_ids"]

from transformers import DistilBertTokenizer # Prepare the inputs tokenizer = DistilBertTokenizer.from_pretrained("distilbert-base-uncased") sentence = "I love AI and privacy!" inputs = tokenizer(sentence, padding = "max_length", max_length = 8, return_tensors="pt")["input_ids"]

Now we can get a prediction.

In [ ]:

with blindai.connect() as client:
  response = client.predict(model_id, inputs)
response.output[0].as_flat()

with blindai.connect() as client: response = client.predict(model_id, inputs) response.output[0].as_flat()

Here we can compare the results against the original prediction.

In [ ]:

model(inputs).logits.detach()

model(inputs).logits.detach()

Et voila! We have been able to apply a start of the art model of NLP, without ever having to show the data in clear to the people operating the service!

If you have liked this example, do not hesitate to drop a star on our GitHub and chat with us on our Discord!