Skip to content

Injecting External Metrics

This topic shows you how to inject experiment external metrics.

Overview

The following steps are covered:

  • Get weights hash for a supported framework's models.
  • Set external metrics to the level you wish to set it to.

Note

The metrics that are being sent must be prefixed with 'ex_'.

Write code

  1. Run a MissingLink integrated experiment:

    In order to add external metrics to an experiment, the experiment that is run must be an experiment that has been integrated with MissingLink.

    Run a MissingLink integrated experiment from a supported framework:

  2. Get the experiment identifier:

    You can either get the experiment identifier by getting the experiment ID or by providing weights hash of the model. For more information, see Experiment Id and Model Weights Hash Retrieval.

    experiment_id = callback.experiment_id
    
    model_weights_hash = callback.calculate_weights_hash(model)
    
  3. Set external metrics either at the experiment level or on a specific epoch or iteration:

    At the experiment level

    update_metrics lets you inject metrics with a single value for the whole experiment.

    metrics = { 'ex_sample_size': 0.6 , 'ex_other_metric': 8}
    callback.update_metrics(metrics, experiment_id=experiment_id)
    

    or

    metrics = { 'ex_sample_size': 0.6 , 'ex_other_metric': 8}
    callback.update_metrics(metrics, model_weights_hash=model_weights_hash)
    

    On a specific epoch or iteration

    With update_metrics_per_itration you can inject metrics to specific points within the experiment. For that purpose, you need to calculate the weights hash for each iteration that you wish to inject the metric to.

    callback.update_metrics_per_iteration({ 'ex_val_1': 0.1 }, model_weights_hash=model_weights_hash_1)
    callback.update_metrics_per_iteration({ 'ex_val_1': 6 }, model_weights_hash=model_weights_hash_2)