Retrain triggers

If your MLOps pipelines are set up in the AWS ecosystem, then you probably need the Amazon EventBridge retrain trigger.

The trigger, when activated, will create an event in a Event Bus of your AWS account with custom metadata.

You need to create an Event Bus rule that recognizes the ML cube Platform event pattern and attach the target action you want. Examples of targets are:

• launching a Lambda function;
• launching a SageMaker pipeline;
• sending a message to a SQS Queue with a retraining request.

Event Bus Setup¶

1. In the AWS console, open the Amazon EventBridge service and select the Event buses option in the left-side menu

   

2. In the Custom event bus tab, create a new event bus with the default settings

3. Select the Rules section on the left-side menu and click the Create Rule button

   

4. Insert the rule name and the created Event Bus in the Event Bus section. Click Next

   1. Event source: select the voice AWS events or EventBridge partner events

      

   2. Sample event: copy and paste this:

      {
          "version": "0",
          "id": "fcdd87c7-f56e-c722-4f85-4cb6ba85a00a",
          "detail-type": "retrain_trigger",
          "source": "ml3_platform",
          "account": "123456789",
          "time": "2023-11-02T14:16:23Z",
          "region": "eu-west-3",
          "resources": [],
          "detail": {
              "first_param": "hi",
              "second_param": "bye"
          }
      }
      

      

   3. Creation method: select Custom pattern (JSON editor)

      

   4. Event pattern: copy and paste this:

      

      {
          "source": [{
              "prefix": "ml3_platform"
          }],
          "detail-type": [{
              "prefix": "retrain_trigger"
          }]
      }
      

   5. click the button Test pattern to check the match and then click next

5. Select the target that will handle the events. If you want to test the rule, you can add a CloudWatch target that stores the event to a new Log Group.

   

6. Create the rule

Retrain Trigger Setup

To integrate Amazon Event Bridge, you will need to create a set of AWS credentials, and add a policy that allows to put events in your event bus. Please refer to this page to know more.

Once the credentials and the policy have been created, you can set up the retrain trigger for your model through the SDK or the web application.

client.set_retrain_trigger(
    model_id='your_model_id',
    trigger=AWSEventBridgeRetrainTrigger(
        credentials_id='your_credentials_id',
        aws_region_name='e.g. eu-west-3',
        event_bus_name='your_event_bus',
    ),
)

If your MLOps pipelines are set up in the Google Cloud Platform ecosystem, then you probably need the GCP Pub/Sub retrain trigger.

The trigger, when activated, will create an event in a Pub/Sub topic of your GCP project with custom metadata.

Topic Setup¶

1. In the Google Cloud console, open the Pub/Sub service and select the Topics option in the left-side menu

   

2. Click on the Create topic button. Then give it a unique id and create it.

   

3. Configure your subscriptions as needed to configure the service that will handle the events.

The ML cube platform will send events with this format:

{
    "project_id": "your gcp project id",
    "topic_name": "your unique topic id",
    "source": "ml3_platform",
    "event_type": "retrain_trigger",
    "payload": {
        "model_id": "id of the model on ml cube platform"
    }
}

Retrain Trigger Setup

To integrate GCP Pub/Sub, you will need to create a set of GCP credentials, and add a policy that allows to put events in your Pub/Sub topic. Please refer to this page to know more.

Once the credentials and the policy have been created, you can set up the retrain trigger for your model through the SDK or the web application.

client.set_retrain_trigger(
    model_id='your_model_id',
    trigger=GCPPubSubRetrainTrigger(
        credentials_id='your_credentials_id',
        topic_name='your-topic'
    ),
)

If your MLOps pipelines are set up in the Microsoft Azure ecosystem, then you probably need the Azure Event Grid retrain trigger.

The trigger, when activated, will create an event in an Event Grid topic of your Azure resource group with custom metadata.

Topic Setup¶

1. In the Azure console, open the Event Grid service and select the Topics option in the left-side menu

   

2. Click on the Create button. Select an active subscription and the resource group, then give the topic a unique name, select your preferred region and create it.

   

3. Configure your subscriptions as needed to configure the service that will handle the events.

The ML cube platform will send events with this format:

{
    "subject": "ml3_platform",
    "event_type": "retrain_trigger",
    "data_version": "1.0",
    "data": {
        "model_id": "id of the model on ml cube platform"
    }
}

Retrain Trigger Setup

To integrate Azure Event Grid, you will need to create a set of Azure credentials, and add a role that allows to publish events in your Event Grid topic. Please refer to this page to know more.

Once the credentials and the policy have been created, you can set up the retrain trigger for your model through the SDK or the web application.

client.set_retrain_trigger(
    model_id='your_model_id',
    trigger=AzureEventGridRetrainTrigger(
        credentials_id='your_credentials_id',
        topic_endpoint='https://your-retrain-topic.italynorth-1.eventgrid.azure.net/api/events'
    ),
)