MLOps: Deploy NLP model straightforward using mlflow & azure

Photo by C D-X on Unsplash

On the Leeway of MLOPs, you have plenty of open-source tools available to build your pipeline. I have picked up mlflow to track & maintain my NLP lifecycle while on the cloud counterpart, for model registry & deployment I have chosen azure ( because why not when you get most of the services in the free tier :P)

so without wasting any bit, let's jump on the bandwagon of MLOps.

Initial Prep

Listing down whatever we going to use in this experiment.

1. Spacy: To build an NLP model
2. Mlflow: Experiment with lifecycle management
3. Azure: pipeline orchestration & Deployment

and the sequence of action items would be as follows:

1. Training File Preparation
2. Setting up Azure
3. Setting up Mlflow
4. Pipeline Building
5. Deployment

Training file preparation

As our primary motto is to set up an MLOps pipeline not the modelling, so for that sake we’re going to train a basic Custom NER (Named Entity Recognition) model: Product name identification.

So the training file would take training_json_file (RX: refer to file), number_of_iteration, output_model_name, as input and the rest of the work spacy would do.

Training Sample (JSON file)

Setting up Azure

First, create an account on azure and you’d be eligible for the free tier ($200 credit). The best part is that you can access many services in this free tier, like Azure ML Studio and whatnot. Once you are done with account creation, you need to set up a workspace. So this workspace is an umbrella, under which you will get all resources for your machine learning activities.

If you’re stuck anywhere, please follow Microsoft’s official guide which explains this setup process.

So once the setup is done, you’d have a config file generated by azure for your workspace. It contains all the information which we need to connect/authenticate and access the services of azure. It looks like this

Dummy config file (replace it with your fields)

Setting up Mlflow

The most crucial component of our flow. So Mlflow is an open-source platform for managing the end-to-end machine learning lifecycle including tracking experiments, packaging code into reproducible runs, and sharing and deploying models.

In our case, we’re using Mlflow to track the experiments. Before we go into details let's get familiarised with Mlflow terminologies:

1. MLflow Project: It's a format for packaging data science code in a reusable and reproducible way.
2. Runs: Gist of Mlflow Tracking, runs are executions of some data science code. A run contains meta information such as code version, parameters, metrics, timestamp etc and artefacts: model files, data files, output files etc.
3. MLflow Models: It's a standard mlflow format to wrap the model which can easily be deployed for real-time serving or can be used for various downstream tasks. Mlflow provides a lot of built-in flavours such as model as python function, scikit learn model etc, these would make the model deployment seamless.
4. MLflow Model Registry: To maintain model lineage, model versioning, stage transitions and annotations.

Now we’re equipped with all our weaponry, so let's jump into the pipeline implementation arena.

Pipeline Building

NOTE: We’re running an experiment in which we’re building a NER model which is being tracked by MLflow on azure.

Step-1: Connect with Azure cloud so that whatever experiments we run record on your azure workspace.

#Setting up Azure

ws = Workspace.from_config()
mlflow.set_tracking_uri(ws.get_mlflow_tracking_uri())
experiment_name = 'ner_spacy_mlflow_azure_demo'
mlflow.set_experiment(experiment_name)

Step-2: Before we train our model, we need to make sure we’re running the model training phase under the hood of mlflow runs so that we can track the experiment. So for that, we start a mlflow run context manager and then we train the Custom NER model using spacy for a few iterations on the training JSON file which we’ve prepared.

Good thing here is you can log whatever you want mlflow to track across and in fact, they’ve given various flavours according to type such as log_param, log_metric, log_model etc. For example, I’ve tracked my #epoch and loss.

    with mlflow.start_run():

        mlflow.log_param('Totala epochs', iteration)

        with open(train_file, 'r') as f:
            TrainData = json.load(f)

        #YOUR_TRAINING_CODE GOES HERE

        mlflow.log_metric('loss', losses['ner'])

Step-3: Now here comes the most crucial thing, Loggin your model. As I mentioned above, it's better to wrap the model in the standard mlflow model format for seamless deployment. So I’m wrapping up my model as a Python function. This can easily be done by inheriting the mlflow.pyfunc.PythonModel and overriding the prediction method of the class.

class spacyNERModel(mlflow.pyfunc.PythonModel):

    def __init__(self, nlpModel):
        self.nlpModel = nlpModel

    def predict(self, context, model_input):
        # df = pd.read_json(model_input)
        res = model_input['text'].apply(lambda x: str(list(self.nlpModel(x).ents)))
        return res.values


nlp_pyfunc_model = spacyNERModel(nlp_model)

After that, you need to mention the Model Signature which is nothing but the schema of the model’s inputs and outputs. As this is a NER model, so input would be a string and the output would be a python list of entities.

input_schema = Schema([

            ColSpec("string", 'text')
        ])
output_schema = Schema([
            ColSpec("list", 'predictions')
        ])

model_signature = ModelSignature(inputs=input_schema, outputs=output_schema)

lastly, we need to provide python environment info (I’ll be using an anaconda environment thus creating a conda_env.yaml file) under which model inference would happen so that at the time of deployment mlflow can take care of that.

#conda_env.yaml
name: mlflow101_conda

dependencies:
  - python=3.6
  - pip
  - pip:
      - spacy==2.3.2
      - scikit-learn
      - mlflow
      - azureml-sdk
      - azureml-mlflow
      - pandas

Finally, with all things done, we can log our model under the same mlflow run context manager.

# Logging the model
mlflow_pyfunc.log_model(python_model=nlp_pyfunc_model, 
                        artifact_path='model', 
                        conda_env='envs/config/conda_env.yaml',
                        signature=model_signature

Step-4: Now we’re done with model creation, and model logging. we can then register our model on the azure cloud within this experiment. To register the model, you need to provide the model URI. URI can be created with mlflow_run_id and artifact we mentioned at the time of model logging.

model_uri = "runs:/{run_id}/{artifact_path}".format(
            run_id=mlflow.active_run().info.run_id,
            artifact_path="model")

mlflow.register_model(model_uri, 'spacyNERmodel')

That’s it, the pipeline is ready!

And here comes the beautiful part of Mlflow, No matter how many times you trigger the script, it will track it all, like if a registered model with the name doesn’t exist, the method registers a new model, creates Version 1, and returns a ModelVersion MLflow object. If a registered model with the name exists, the method creates a new model version and returns the version object.

And you’d be able to see it on MLflow UI but in our case, we’d see it on our azure workspace UI as we’re recording our experiment on azure only.

Experiments showing under azure workspace

Deployment

As we’ve already pushed our model to the model registry on azure. Now Azure gives us on-click deployment flexibility, that too on the free tier :P

So we go on the azure workspace UI, you’d see all your experiments there. and if you check under the Model Register tab, you’d see all your models with given meta information, annotations and logged parameters/metrics.

Now, to deploy the model, we gonna follow few steps:

Step-1: Go to Models tab, you’d see all the modes which been logged through experiments. Click on a model which you want to deploy.

Models listing

Step-2: Click on the Deploy button which is on the top (use Real-time endpoint(quick))

Step-3: Now fill up the configuration like virtual machine allocation, endpoint name, deployment name and all. Once done, click on deploy.

Step-4: That’s it. Now sit down and wait for azure to complete it all. Once done, you’d see provisioning state== succeed, the Swagger URI and REST endpoint created.

Deployment Succeeded

Now to test the Model API, azure has generated the code for you, so you can directly use that and test it or you can test it there already. Also, On azure, you’d have all the options like monitoring the model, checking deployment logs etc.

Endpoint Testing

NOTE: If some error comes up, you can always check on the deployment logs.

That’d be the wrap for this blog. Now you’ve got the pipeline ready to deploy. In MLOps, the next thing that comes is to Monitor the model and its performance. Therefore, In the upcoming blog of this series, I’m gonna cover the topic of how to set up that process in addition to it. If you want to check the codebase for this setup, please refer to the codebase:

impyadav - Overview

Thanks for the read. Anywho, if you have any feedback, I’m all ears.

Follow me: https://impyadav.medium.com

Connect with me: Twitter || LinkedIn