Deploy ML/DL models into a consolidated AI demo service stack

Keywords:  IRIS, IntegratedML, Flask, FastAPI, Tensorflow Serving, HAProxy, Docker, Covid-19


We touched on some quick demos of  deep learning and machine learning over the past few months, including a simple Covid-19 X-Ray image classifier and a Covid-19  lab result classifier for possible ICU admissions.  We also touched on an IntegratedML demo implementation of the ICU classifier.  While the "data science" hiking still goes on, it might also be a good time to try some AI service deployment from the "data engineering" perspective - could we wrap up everything we touched on so far into a set of service APIs?  What are the common tools, components, and infrastructure that we could leverage to achieve such a service stack in its simplest possible approach?


In scope:

As a jump start, we can simply use docker-compose to deploy the following dockerised components into an AWS Ubuntu server

Note:   Tensorflow Serving  with GPU is for demo purpose only - you can simply switch off the gpu related image (in a dockerfile) and the config (in the docker-compose.yml).

23 views0 comments