Pay $0.01 for Cloud Hosted MLflow

Let's say you're a ML practitioner working in start up, small company or any company that had limited budget and you would like to explore the use of MLflow to manage your team's machine learning life cycle, but you don't want to take the hassle of setup your own instance of MLflow. Then, the good new is "You can have your own MLflow tracking server and MLflow UI running on the cloud for $0.01 only", that's sound unbelievable, right?

I found the following great deal from AWS Marketplace:When I come across the offer above from InfinStor, I think it is too good to be true, there's must be a catch. I decided to take up the offer and try it out. First of all, I don't have any affiliation or association with InfinStor, I wrote this post is simply for sharing purpose.

Once you sign up and login to the InfinStor console, you will see the following screen which clearly tell you what is the possible "hidden" cost involved. 

In short, if you're NOT running the jupyter notebook and MLflow project using EC2 instance, the cost should be minimal. To setup the MLflow tracking server, please check the documentation under "Initial Configuration By Admin" section.

I tested the MLflow tracking server and MLflow UI by running a MLflow project locally. You can checkout the changes I made for helloworld example and go through the Using InfinStor Mlflow Projects to find out how to do that.

Hope this help. Happy weekend! :)

Four Key Stages of The Machine Learning Process

These four stages is defined by Emmanuel Ameisen in his recent book Building Machine Learning Powered Applications. As I think it is an important piece, I created a process diagram above and note down few important points for each stage. You can considered this is a study note.

1. Identifying the Right ML Approach

The best approach for a given problem (product goal) will depends on success criteria, data availability, task complexity and model choice.

2. Building an Initial Prototype

Start by building an end-to-end prototype before working on a model which aim to tackle the product goal with no ML involved and will allow you to determine how to best apply ML.

3. Iterating on Models

If you need ML, start gathering dataset, train a model and evaluate it shortcomings. The goal of this stage is to repeatedly alternate between error analysis and implementation.

4. Deployment & Monitoring

Once a model shows good performance, you should pick an adequate deployment option. Once deployed, models often fail in unexpected ways. The last two chapters of the book will cover methods to mitigate and monitor model errors.

Side Note

I added an arrow labelled "New product goal, Revised product goal" to the process diagram. Based on my experience, software products evolve, it will not end there after the ML solution went live. Hence, it is likely that the existing product goal will be revised or new product goal being inspired or discovered. Then, the cycle start from stage 1. Don't you think so?

Deploy Serverless Machine Learning Models to AWS Lambda

I just completed the Udemy course with the same title of the blog post.

The course cover three machine learning examples using different library:

1. scikit-learn: Using regression model from scikit-learn trained on California housing dataset for house price prediction.
2. spacy: Using a small English model from spaCy NLP framework for named entity recognition.
3. keras: Using ResNet50 computer vision deep learning model from Keras framework to create for image recognition.

Let's me tell you the following value-added changes of my code from the original source code from the course:

• The examples are using Python 3.8 instead of Python 3.6 in the course and latest version of libraries except the scikit-learn example using earlier version of scikit-learn to align with m2cgen which has the possibility to convert scikit-learn model to native code and C code running in the microcontroller with emlearn.
• To deploy all examples. you only need git, no other installations are required. All examples are setup with continuous deployment pipeline using Github Actions. The following diagram shows how it works from the high-level:

1. Developer update changes to repository by sending a git push command. 
2. The git push command will trigger the Github Actions workflow which run the serverless deploy command based on configurations in serverless.yml file. 
3. The CloudFormation stack consists of API Gateway, AWS Lambda, S3, ECR, etc. will be created on AWS.

After you forked my repository, you just need to add AWS_KEY and AWS_SECRET to Repository secrets of Settings/Secrets of your github repo like the following image:

Then, you can deploy all examples to AWS lambda with a git push command.

• All examples are setup with "warm start" and request throttling limit to 1 to protect your account and your wallet (You don't want to have surprises in your monthly credit card bill for demo projects, right?)
• All examples have a last step "Test Lambda functions" to verify the public API is working post-deployment.
• Spacy and keras example are deploy using Docker container image to AWS Lambda. Thanks to Jan Bauer published the blog post Using container images to run TensorFlow models in AWS Lambda, timing is just right on helping me to get the keras example working.
• For keras example, image uploaded directly to Lambda endpoint and store as temp file instead of upload to S3 bucket.
• Code of each example is located in its own branch.

Please take note of the storage cost of S3 and Elastic Container Registry (ECR), make sure you clean up unnecessary files and Docker image files there.

Lastly, if you look for someone on deploying your machine learning model to AWS lambda, I think I can help here. :)

Oh yeah, here is the github repo URL https://github.com/limcheekin/serverless-ml if you missed it.

I think the scope is broad enough for now, let's go deep.

Stay tuned to the next!