Cloud‑Based Machine Learning Model Deployment

Cloud Computing

Imagine we need electricity. We have two options:

1. Building our own power plant, which is expensive, requires a lot of space, needs experts to run it, and we have to maintain it 24/7.
2. On the other hand, use the Public Power Grid, where we just plug into the wall and pay for exactly the electricity we use. We don’t know where the power plant is, and we don’t care how it works.

Cloud computing is the “public power grid” for computing resources. Instead of owning our own computing infrastructure (data centers, servers, storage, etc.), we rent resources from providers and pay for what we use over the internet.

According to the U.S. National Institute of Standards and Technology (NIST), cloud computing is:

Cloud computing is a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction.

Service Models

• IaaS (Infrastructure as a Service)
  • provides us with raw computing resources, including virtual machines, storage, and networks.
  • We are responsible for installing Python and ML frameworks, as well as configuring the environment. This is flexible but requires the most setup.
  • Examples: AWS EC2, Azure Virtual Machines, Google Compute Engine.
  • Use for ML deployment: We can manually configure an EC2 VM to host a Flask/FastAPI ML API, but we must handle scaling and monitoring.
• PaaS (Platform as a Service):
  • The provider manages the runtime, scaling, and deployment pipeline.
  • We just push our code or container, and the platform makes it accessible online.
  • Examples: Heroku, Render, Railway, Google Cloud Run, Azure App Service, AWS Elastic Beanstalk.
  • Use for ML deployment: Ideal for students and small projects—upload a trained model + API, and the service handles scaling.
• SaaS (Software as a Service):
  
  • The provider delivers ready-to-use applications over the cloud.
  • We simply use the software via a web browser or API without worrying about infrastructure or deployment.
  • Examples: Google AutoML, Salesforce Einstein, Microsoft Power BI, Looker.
  • Use for ML deployment: Best for non-technical users to train or use pre-built ML models (e.g., fashion demand forecasting, customer insights) without writing code.
• FaaS (Function as a Service) / Serverless:
  • We write small pieces of code (functions) that are triggered by events (e.g., API call, file upload).
  • The platform automatically scales to zero when not in use, so we only pay when functions run.
  • Examples: AWS Lambda, Azure Functions, Google Cloud Functions.
  • Use for ML deployment: Suitable for lightweight models or preprocessing tasks, e.g., run an image‑classification model when a customer uploads a photo.
• Managed ML Platforms:
  • Full machine learning lifecycle support, including data prep, model training, deployment, monitoring, and versioning.
  • They integrate with other services for production MLOps.
  • Examples: AWS SageMaker, Google Vertex AI, Azure ML, Databricks Model Serving.
  • Use for ML deployment: Best for industry settings where deploying models with built‑in scaling, monitoring, and model registries.

Types of Deployment Models:

• Private Cloud
  • A dedicated cloud used only by one organization, offering full control and higher security.
  • Example: A fashion retailer running its own cloud for inventory analytics.
• Community Cloud
  • A shared cloud for a group of organizations with common needs like compliance or security.
  • Example: Multiple fashion brands sharing a cloud for sustainable supply chain tracking.
• Public Cloud
  • Cloud services are available to everyone, owned and operated by providers like AWS, GCP, Azure.
  • Example: Deploying a fashion recommendation API on AWS for global access.
• Hybrid Cloud
  • A mix of private, public, or community clouds working together, allowing data/app movement between them.
  • Example: Keeping customer data private while using public cloud for AI model training.

Cloud-Based Machine Learning Model Deployment

• The process of making a trained machine learning model available as a scalable, reliable, and secure web service on cloud infrastructure, so that other applications can use it to make predictions on new data.
• In simpler terms, it’s the act of taking a model from a experimental file on a data scientist’s computer and turning it into a live, functioning API on the internet that an app or website can call for real-time predictions (e.g., “is this transaction fraudulent?”) or to process batches of data (e.g., “generate product recommendations for all users overnight”).

Different cloud platforms

Student‑friendly (for demos, fashion retail projects)

• Hugging Face Spaces: share a fashion recommender (Streamlit/Gradio) with classmates.
• Streamlit Cloud: host an interactive sales forecasting dashboard.
• Render / Railway / Deta Space: quick deployment of a product‑recommendation API.

Industry platforms (for large‑scale retail)

• AWS: SageMaker endpoints for recommendation systems.
• Google Cloud: Vertex AI pipelines for demand forecasting.
• Azure: Azure ML for product‑classification models.

ML Deployment Workflow

Step 1: Train & Save the Model (Locally or in Cloud)

• Train model using Python (scikit-learn, TensorFlow, PyTorch, etc.).
• Save the model in a standard format:
  • model.pkl (sklearn/joblib)
  • model.h5 (Keras/TensorFlow)
  • model.pt (PyTorch)
• Example:

import joblib

joblib.dump(model, “model.pkl”)

Step 2: Create the App

• Build a lightweight API using Flask or FastAPI.
• Define an endpoint (e.g., /predict) that accepts input data (JSON) and returns model predictions.
• Load trained ML/DL model inside the app so it’s ready to serve requests.
• Test the app locally by running it with Flask or FastAPI’s server, to confirm we can send requests and get predictions.

Step 3: Containerize with Docker

• Write a Dockerfile that sets up a Python environment with our app and required dependencies.
• Use a base Python image, install dependencies (from requirements.txt), copy our code into the container, and set the startup command to run the app.
• Expose the correct port (e.g., 5000) so the app can be accessed from outside the container.
• Build the Docker image (docker build -t mymodel .) and run a container from it( docker run -p 5000:5000 mymodel).
• Test the containerized API in your browser or with tools like Postman or curl.

Step 4: Choose a Cloud Platform for Deployment

Once our model is containerized, the next step is deciding where to host it. We have a few options:

• AWS SageMaker
  • A fully managed service designed specifically for machine learning.
  • It simplifies deployment by handling scaling, monitoring, and endpoint management automatically.
  • Best if we want an ML-focused platform without managing servers.
• Azure Machine Learning
  • Provides an end-to-end environment for the entire ML lifecycle (training, tuning, deployment, monitoring).
  • Offers good integration with other Microsoft services (like Power BI and Azure Data Lake).
  • Suitable if we want a complete workflow from data prep to production.

• Google Vertex AI

  • A modern, unified platform for ML on Google Cloud.
  • Allows both training and deployment of models at scale.
  • Strong integration with Google’s data ecosystem (BigQuery, Dataflow, etc.).

• Generic Deployment with Docker + Kubernetes

  • Instead of using an ML-specific service, we can deploy our containerized app as a standard API.
  • Using Kubernetes (on AWS EKS, Azure AKS, or Google GKE) gives us full control over scaling, load balancing, and updates.
  • This is more flexible, but also requires more DevOps knowledge.

Step 5: Upload the Model to the Cloud

Before deployment, we need to make our trained model or container image available on the cloud. This usually involves storing it in cloud storage (for model files) or a container registry (for Docker images):

• AWS:
  • Store the model file in Amazon S3 (object storage).
  • Push your Docker image to Amazon ECR (Elastic Container Registry) for deployment.
• Azure:
  • Save the model in Azure Blob Storage.
  • Upload container images to the Azure Container Registry (ACR).
• Google Cloud (GCP):
  • Use Cloud Storage to store model artifacts.
  • Push container images to Artifact Registry (or older Container Registry).

In machine learning, the word artifacts usually means the files produced during the ML workflow that you need later for deployment, testing, or reuse.

Step 6: Deploy the Model as an Endpoint

After uploading our model or container image, the next step is to deploy it as a live API endpoint that clients can call for predictions. The process differs slightly by cloud provider:

• AWS SageMaker
  • Register the model in SageMaker.
  • Create an Endpoint Configuration.
  • Deploy the model to a managed SageMaker Endpoint, which automatically handles scaling and monitoring.
• Azure Machine Learning
  • Register the model in the Azure ML workspace.
  • Create an Inference Endpoint (real-time or batch).
  • Deploy the model so it can be accessed via a REST API.
• Google Cloud Vertex AI
  • Upload our model artifact to Vertex AI.
  • Deploy it to a managed Vertex Endpoint, which provides an API URL for prediction requests.
• Alternative: Containerized Deployment
  • Instead of using ML-specific services, we can deploy our Dockerized app as a standard API.
  • Options include:
    • AWS ECS or EKS
    • Azure AKS
    • Google Kubernetes Engine (GKE)
  • This provides more flexibility but requires self-management of scaling, networking, and monitoring.

Step 7: Expose the API to Users

Once the model is deployed, the cloud platform provides a REST API endpoint (URL) that external applications can use.

• The endpoint is typically a URL like:
  POST https://cloud-endpoint/predict
• How it works:
  • Our web app, mobile app, or BI tool sends a request to the endpoint.
  • The request body usually contains the input data in JSON format.
  • The cloud service runs the model and returns a prediction in the response.
• This makes our model accessible to any system that can call an API, allowing for seamless integration into dashboards, apps, or automated workflows.

Step 8: Monitor and Maintain (MLOps)

Deployment is not the end — keeping the model reliable in production requires continuous monitoring and maintenance. This is where MLOps practices come in:

• Track performance metrics → Monitor latency (response time), error rates, and infrastructure costs to ensure the service stays efficient and cost-effective.
• Detect data drift → Compare real-world input data with the training data. If distributions shift (e.g., customer behavior changes), the model’s accuracy may decline.
• Set up automation (CI/CD) → Build pipelines for automatic retraining, testing, and redeployment whenever new data or improved models are available.
• Logging & alerts → Collect logs, track usage, and set alerts for anomalies (like sudden spikes in errors or unexpected predictions).