Contents x
    Model Deployment
    • 03 Jun 2024
    • Print
    • Dark
      Light
    • PDF
    Contents

    Model Deployment

    • Updated On 03 Jun 2024
    • Print
    • Dark
      Light
    • PDF
    Article summary

    Overview

    A deployed model turns into a service in the Dataloop automation platform. Once deployed, the model Predict function can be used in automation pipelines, with triggers on items, or manually from UI-Slots. Service ops are monitored from service-analytics page. Adjust service settings such as instance-type and auto-scaling to make it production-ready.

    Deployed Models Tab

    Deployed models can be viewed in the Deployed tab. The table lists all related information:

    FieldsDescription
    NameThe name of the model. Click on it to open the model's detailed page.
    Play/PauseAn option to Pause/Start the deployed model.
    Created AtThe timestamp of the model's evaluation process creation.
    ExecutionsNumber of Predictions (executions) by the running model.
    InstancesCurrent instance and auto-scaling configuration.
    ServiceThe name of the deployment service. Click on it to open the service page.

    Search and Filter Deployed Models

    Dataloop platform allows you to search models that are deployed using Model Name and Model ID, providing users with the capability to refine and narrow down the displayed models.
    Use the Search field to search and filter the models.

    Deploy a Model

    To deploy a model:

    1. Open Models from the left-side menu.
    2. In the Versions tab, select the model that you intend to deploy. You can deploy models that are pre-trained, trained, or deployed.
    3. Click on the 3-dots action icon and select the Deploy option. The Model Version Deployment window is displayed.
    4. Review the following details and make changes, if needed:
      1. init inputs value (1): The initial input or configuration values provided to a function when it is first invoked or initialized. The values are displayed, if the init inputs are defined in the dpk/dpkConfig. Click on the Edit Init Input to add a new value.
      2. Secrets: The secret values are displayed, if the secret is defined in the dpk/dpkConfig. Click on the Edit Secret to select multiple secrets from your organization, or click Edit Secret -> Add New Secret to create a new secret for the model. Make sure you have enough permission to view or add a secret.
      3. Service Configuration: Click on the Edit Configuration to make changes to the following fields, if needed:
        1. Machine Types:
          1. Change the machine type from the list.
          2. Click on the Machine Settings to enable Preemptible Machine.
        2. Docker Image
        3. Concurrency
        4. Scaling Type:
          1. Autoscale: A component of the Dataloop FaaS system that automatically adjusts the number of serverless computing Resources allocated to the FaaS Service based on current demand. The purpose of the Autoscaler is to ensure that the system can handle peaks in demand without over provisioning Resources and incurring unnecessary costs.
            1. Min Instances: By default, 0. If you set the min instances to 0, the service will not run until it has something in the queue, it will be suspended.
            2. Max Instances: By Default, 1.
            3. Queue Size: By default, 10.
          2. Fixed size: By default, 1 instance is selected.
      4. Execution Configuration: Click on the Execution Configuration to make changes on the following fields:
        1. SDK Version:
        2. Execution Timeout: By default, 3600 is displayed.
        3. Drain Time: By default, 300 is displayed.
        4. On Reset Action: Select one of the following when you reset the execution.
          1. Fail Execution
          2. Rerun Execution
        5. Max Attempts: Maximum number of execution attempts allowed. By default, 3 is displayed.
        6. Rerun executions as process: Enable to make the execution as a process.
    5. Once you review the above parameters, click Deploy. A message will pop up when the model is deployed and ready as a service.

    Copy Deployed Model ID

    1. Go to the Models page from the left-side menu.
    2. Select the Deployed tab.
    3. Find the model to copy the ID.
    4. Click on the three-dots and select the Copy Model ID from the list.

    Copy Service ID of the Deployed Model

    1. Go to the Models page from the left-side menu.
    2. Select the Deployed tab.
    3. Find the model to copy the service ID.
    4. Click on the three-dots and select the Copy Service ID from the list.

    Unload the Model Deployment

    1. Go to the Models page from the left-side menu.
    2. Select the Deployed tab.
    3. Find the model to abort the deployment.
    4. Click on the three-dots and select the Unload Service from the list.
    Delete a model deployment

    When deleting a deployment of models, then the model status will be changed from Deployed to:

    • Pre-trained: If the model (entity) is not trained directly.
    • Trained: If the model (entity) is trained directly.

    Error and Warning Indications

    When a service encounters errors, such as Crashloop, ImagePullBackOff, OOM, etc., or when the service is paused, pending executions will get stuck in the queue (in "created" status). You can click or hover over the error or warning icon to view the details.

    • An error icon is displayed when the service status is failed at training, evaluation, prediction execution stages of the model. The service link is provided, and you click on it to view the respective executions tab of the service. The pending deployment execution status will be Created, In progress or Rerun.
    • A warning icon is displayed when a service is inactive. The service link provided, and you click on it to view the respective service page. The pending deployment execution status will be Created or Rerun.
    What's Next