Data¶

To use ML cube Platform, you need to upload the data that your artificial intelligence algorithms deal with. They are represented by the Data logical entity, which is a complex object composed of many different elementary units.

According to the context, the Data object has some required entities to be considered valid: for instance, training data require both input and target, while production data are still valid with only inputs.

Each elementary unit in the Data object belongs to a Data Category, which represents an consistent group of data. Available categories are:

Data Category and Data Schema Column's Role¶

The Data Schema created for the Task contains a list of Column objects, each of which has a Role. Naturally, there is a relationship between the Column's Role and the Data Category. In fact, each Data Category comprises a set of Column objects with certain Roles. When you upload samples belonging to a Data Category, they must contain all the Columns objects declared on the Data Schema to be considered valid.

The following table shows these relationships:

sample_id,time_id,x_0,x_1,x_2
abc123,1234,3.5,100,23.0

Data Batch¶

ML cube Platform supports batch-based data processing, which means that data samples are uploaded together as a batch. Therefore, the Data object described above must contain, for each Data Category, the same set of samples to form a consistent batch. The only exception is for Target data that, given the fact that they may be obtained through labeling, can be present partially.

In any case, the presence of a Data Category in a Data Batch depends on the context in which it is uploaded. In the next section we explain what is meant by the Historical, Reference and Production data.

When uploading images on the platform, you also need to provide a mapping file that links the image filename to the sample ID and its timestamp. Likewise sample IDs, filenames must be unique across all the images uploaded for a task.

sample_id,time_id,file_name
abc123,1234,image_1.png

Data contexts¶

Data samples have different meanings depending on how you have used them to train, validate, or infer from your artificial intelligence model. Some data may be too old and, therefore, not used, others may belong to a training set, and, lastly, the newest one comes directly from production.

To handle all these scenarios, the ML cube Platform allows the upload of data specifying how they were used. Indeed, the operations of the ML cube Platform will be different to each context.

The main difference in the data usage is the split between the ones present before the model was in production and the others. For that reason, the ML cube Platform defines two data contexts: historical and production.

Historical¶

Historical data are all data samples that existed before the model was deployed in production. They include the training, validation and test sets that were used during model experimentation and training, or older data that were discarded.

Historical data can be uploaded in ML cube Platform anytime through different data batches.

Data categories that a batch of historical data contains depend on the Task Type, here is the summary:

• Supervised tasks like regression, classification and object detection requires Input and Target
• Retrieval Augmented Generation task requires Inputs and Predictions

The reason why it is not possible to upload the model's predictions in historical data of supervised tasks is that they may contain the training bias of the model. On the other hand, in RAG tasks there is no target or training dataset.

job_id = client.add_historical_data(
    task_id=task_id,
    inputs=inputs_data,
    target=target_data,
)

Reference¶

The Monitoring module requires the definition of a Reference dataset representing training, validation and test data. As mentioned before, historical data belong to both training and old data, but it is important distinguish them. Therefore, the reference data is initially loaded as historical data and then marked as reference data by providing the timestamp interval.

To provide sufficient statistical reliability, reference data must include at least 300 samples. This requirement also applies to the reference of each Segment, if the task involves any.

It is worth mentioning that also RAG Tasks can have reference data, even if they do not have a proper training dataset. Indeed, reference data are used by the Monitoring module to initialize drift detection algorithms, therefore, the reference data definition is a mandatory step to enable this feature. For RAG Tasks, reference data can be used to indicate the type of data expected in production.

job_id = client.set_model_reference(
    model_id=model_id,
    from_timestamp=from_timestamp,
    to_timestamp=to_timestamp,
)

Production¶

After the artificial intelligence model is deployed in a production environment, incoming data belongs to the Production context. They have different characteristics with respect to historical data because they are online.

While the assumption for historical data is that they belong to the past and are complete (with both input and target as an example), production data can be partial or different Data Categories are available at different times. For that reason, they can be uploaded asynchronously at different moments in time.

However, there are some constraints about the Production data upload. Since data samples are grouped in Data Batches they must be consistent in all the Data Categories. ML cube Platform allows to upload each category separately at different moments but the uploads must contains always the samples in the Data Batch.

Let's do an example with a Regression Task about forecasting the weekly produced power from a solar plant. In this case, inputs come from electrical and meteorological measurements from the past week and are immediately available; prediction is available as soon as the model makes an inference and is uploaded from the model inference service. Instead, target data are only available at the end of the week.

• Input samples are the first uploaded Data Category and they define the samples that belong to this Data Batch.
• Predictions are uploaded after the model inference is completed, in order to be valid they must match entirely the input samples already uploaded.
• Target data are uploaded at the end of the week and, as described before they can be a subset of already uploaded input samples.

job_id = client.add_production_data(
    task_id=task_id,
    inputs=inputs_data,
    target=target_data,
    predictions=[
        (
            model_id,
            prediction_data,
        )
    ],
)

job_id = client.add_production_data(
    task_id=task_id,
    inputs=inputs_data,
    target=None,
    predictions=None
)
# ...
job_id = client.add_production_data(
    task_id=task_id,
    inputs=None,
    target=target_data,
    predictions=[
        (
            model_id,
            prediction_data,
        )
    ],
)

Post labeling¶

In some use cases, target data are obtained only by human labeling, and since this is an expensive procedure, not all available samples are labelled but usually only a selected subset. Moreover, it is possible that labeling is done occasionally for samples that belong to different data batches uploaded in ML cube Platform.

Obviously, this scenario does not belong either to historical or production contexts and it has its own specificity. Hence, ML cube Platform provides another way to upload newly labelled samples in a single moment. In this case, there are no constraints about data batches or production boundary but you are allowed to upload all data freely. ML cube Platform will take care of assigning the new target to the right Data Batch.

job_id = client.add_target_data(
    task_id=task_id,
    target=target_data,
)

Data sources¶

The previous sections of this page described what a Data object is, how it is represented by Data Batch and its uploading contexts. To have more information about how to actually upload data we suggest to read the page Data Sources.