AI/ML: Go from Jupyter Notebook to Deployed Endpoints with SageMaker

I recently gave a presentation to my office on Machine Learning where I made a Machine Learning model from Kaggle’s Titanic Dataset. In my presentation, I showed attendees how you can take a dataset, analyze it, manipulate it, and ultimately train a model so that you can make future predictions.

After the presentation, I was chatting with a friend about the presentation, and they were wondering how you can take a notebook, and actually move that to a production environment.

In this tutorial, I will show you a simple approach to take your dataset, create your model, and then deploy your model so that your applications can start taking advantage of your ML predictions!

🤩 The Solution

I like to start from the end, so that you can see what exactly we are going to do in this before diving too deep into anything. So, here is the tech stack that we will be implementing:

• SageMaker Notebook using Python 3
• S3 bucket to store our data and our model artifacts
• SageMaker to create the endpoint configuration and to host the endpoint
• Lambda function (Node.js 10.x) to invoke our endpoint

Our lambda will take in a json object with the parameters for our model, and we will then invoke that model to get a prediction. The end result will look like this (Wow!).

SageMaker

When I first used SageMaker, I actually thought it was fairly daunting, but hopefully if you feel that way, we can break down some of those barriers. It isn’t too bad, especially if you are familiar with using Jupyter Notebooks. And if you aren’t too familiar with Jupyter Notebooks? Well, hopefully we can help you along in this journey to productionalize our model.

New to SageMaker? Then checkout the service page on AWS: SageMaker

Jupyter Notebooks + SageMaker

To start, log in to the AWS Console, go to SageMaker, and create a new notebook instance. When you get to the instance creation page, put in a name and create a new IAM role. You can leave the rest as defaults.

It will take a little bit for it to spin up, but once it has and is InService, then you can click on the Open Jupyter button that is associated with your new instance.

Next, download the Jupyter notebook from here: Jupyter Notebook Example. Upload the notebook to your SageMaker Jupyter instance and you are just about ready to go.

First time with Jupyter? Then welcome to Jupyter Notebooks! You might be interested in reading a little bit about the UI of Jupyter to familiarize yourself.

We are going to be analyzing the Titanic Dataset which you can find here: raw titanic dataset.

Before you run your Jupyter notebook, make sure that you do 2 things! Change the bucket variable to your own S3 bucket, then add the rawTitanic.csv data to the root of that bucket.

And of course, if you don’t already have a bucket, then take a quick break to go create your bucket in S3.

Feel free to step through each cell and read through my commentary of what is going on, or just run the entire notebook. I essentially go through 3 different areas in the ML process: Prepare, Train, and Deploy.

If you are already familiar with Jupyter and have a basic idea of Machine Learning, then most of this will seem familiar. The AWS & SageMaker piece kicks in when we want to start building our model.

Build and Train the Model

SageMaker has different model containers that it uses to train a model. I like to think of this like checking out a book at a library. It checks out the container, uses it to train the model, and in the end, it creates model artifacts, which are stored in S3. These model artifacts are how SageMaker is able to make inferences, or future predictions.

The fit() method is what actually trains the model. This can be a lengthy process (5-10+ minutes) as it checks out the container, trains the model, and then validates the model.

Success! Our model has been trained. The red can be rather distracting, and seem like an error, but this is actually exactly what we want to see. This shows our training error and validation error, split into 10 sections. So, our error rate is around 0.2. That is a decent error rate for our first pass at this!

Want to try and lower the error rate? An awesome resource from SageMaker is automatic hyperparameter tuning. This can help you optimize your hyperparameters to maximize performance!

🚀 Deploy

Now that you have run the fit() method, we can finish with running the deploy() method on that model. I am always so amazed with how easy AWS has made it to train and deploy.

With this one simple command, SageMaker does all of this:

• Creates the model in SageMaker from our S3 artifacts
• Creates the endpoint configuration
• Creates the endpoint and hosts our inference endpoint

With the endpoint created, we can now start to make inferences either through the URL endpoint, or through other AWS services. So for the next step, lets invoke our endpoint through a lambda!

Make Predictions with Lambda

Head over to lambda and create a new Node.js 10.x ensuring that you create or choose an IAM role. For this function to be able to access SageMaker, we will need to add a new execution role. I created a new role with the following policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "sagemaker:InvokeEndpoint",
            "Resource": "*"
        }
    ]
}

When you have your function created, then we can add our lambda code. Go ahead and paste in the following code into your lambda. Note that you may need to tweak the region to your specific region.

Don’t forget to add an environment variable as well! The Key should be ENDPOINT_NAME and the value should be the name of your endpoint from Sagemaker. My endpoint value is titanic-survived-predictor.

var AWS = require('aws-sdk');

exports.handler = (event) => {
    const sagemakerruntime = new AWS.SageMakerRuntime({region: 'us-west-2'});
    let csv = Object.values(event).join(",");

    const params = {
        Body: csv,
        EndpointName: process.env.ENDPOINT_NAME,
        ContentType: 'text/csv',
        Accept: 'text/csv'
    };

    return new Promise((resolve, reject) => {
        return sagemakerruntime.invokeEndpoint(params, (err, data) => {
            if (err) {
                let response = {
                    statusCode: 400,
                    body: JSON.stringify(err),
                };
                reject(response);
            }
            else {
                let willSurviveResponse = 'Yes, You will Survive the Titanic!';
                let survived = 'Yes'
                if (JSON.parse(Buffer.from(data.Body).toString('utf8')) === 0) {
                    survived = 'No'
                    willSurviveResponse = 'No... Perhaps you should avoid boats.';
                }

                let response = {
                    statusCode: 200,
                    survived: survived,
                    message: willSurviveResponse,
                };
                resolve(response);
            }
        });
    });
};

Setup Test to Run Lambda

Nice! Now we should have a working lambda, so open up the test dialog and let’s paste in some code to try it out! Here is some JSON for your test event.

{
  "Pclass": 1,
  "Sex": 1,
  "Fare": 5,
  "isMr": 0,
  "isMiss": 0,
  "isMrs": 1,
  "isRareTitle": 0
}

If you need a reminder of what the values are, here is a refresher: Pclass: Passenger Class (1st, 2nd, or 3rd) Sex: Male (0) or Female (1) Fare: How much they paid broken into a fare class (0, 1, 2, 3, 4, or 5) isMr: The passenger’s title. No (0) or Yes (1) isMiss: The passenger’s title. No (0) or Yes (1) isMrs: The passenger’s title. No (0) or Yes (1) isRareTitle: The passenger’s title. No (0) or Yes (1). Rare titles include the following: Lady, Countess, Capt, Col, Don, Dr., Major, Rev, Sir, Jonkheer, Master, etc.

Woohoo 🎉!

Conclusion

How did you fare!? Would you have survived the Titanic?

In this tutorial, we went through how to take a Jupyter Notebook, and run it through SageMaker to create a prediction engine. This can be done with different datasets, models, for tons of different outcomes. Hopefully you tried it out and learned something along the way. If you have any questions or concerns, then please reach out so that we can continue the conversation!

Wrapping Up / Shutting Services Down

So that you don’t incur any extra charges with AWS, let’s quickly go through the services that you need to stop.

S3

First, go to S3 and delete the bucket that you created for the experiment. If you wanted to keep your SageMaker endpoint running though, make sure that you keep the bucket as well, because it is housing your model artifacts.

Lambda

Next, delete your lambda function, and any associated IAM roles if you wish.

SageMaker

Finally, we will end where we started, with SageMaker. We need to close and delete 4 things in SageMaker.

1. Delete your model from the Inference section.
2. Delete your Endpoint Configuration
3. Delete your Endpoint
4. Shutdown and Delete your Notebook Instance from the notebook section

Thanks again! And good luck in your ML / AI journey ✌️!

Resources:

SageMaker Dev Guide: https://docs.aws.amazon.com/sagemaker/latest/dg/whatis.html

SageMaker Docs: https://sagemaker.readthedocs.io/en/stable/overview.html

Jupyter Docs: https://jupyter-notebook.readthedocs.io/en/stable/notebook.html