Skip to content

Experiments Methods

experiments.logs

as_api().experiments.logs(project, experiment=None, disable_colors=False)

View the execution logs of an experiment that is running in a remote location.

Parameters

  • project: Integer. Project Id.
  • experiment: Optional. Integer. Experiment Id.
  • disable_colors: Optional. Boolean. Enable colors in the console? Default is False.

experiments.list

as_api().experiments.list(project)
List the experiments under the specified project.

Parameters

  • project: Integer. Project Id.

experiments.update_metrics

as_api().experiments.update_metrics(metrics, project=None, experiment=None, weights_hash=None)

Add metrics to an experiment that has already run.

Parameters

  • metrics: String. Metrics of the experiment as a jsonified string. The key should be the metric name with an ex prefix. For example, ex_cost. The value is the metric value in String, Float, Integer or Boolean format.
  • project: Optional. Integer. Project Id.
  • experiment: Optional. Integer. Experiment Id.
  • weights_hash: Optional. String. Hexadecimal sha1 hash of the model's weights.

experiments.update_chart

as_api().experiments.update_chart(chart_name, weights_hash=None, project=None, experiment=None, chart_x_name="X", chart_y_name="Y", chart_scope="test", chart_type="line" chart_x=None, chart_y=None)

Inject a custom chart into the given experiment.

The charts can be single line charts (having a single Y value per X point) or multipoint charts (having multiple Y values per X point). The experiment can be identified by the project and experiment Ids or by model weights hash.

Parameters

  • chart_name: String. The name of the chart. The name is used to identify the chart against different experiments and through the same experiment.
  • weights_hash: Optional. String. Hexadecimal sha1 hash of the model's weights. Not required if project and experiment are provided.
  • project: Optional. Integer. Project Id. Not required if weights_hash is provided.
  • experiment: Optional. Integer. Experiment Id. Not required if weights_hash is provided.
  • chart_x_name: Optional. String. Display name for chart's X axis. Default is X.
  • chart_y_name: Optional. String. Display name for chart's Y axis. Default is Y.
  • chart_scope: Optional. String. Scope type. Default is test. Other options are validation and train.
  • chart_type: Optional. String. Default is line.
  • chart_x: Optional. String.
  • chart_y: Optional. String.

experiments.update_metrics_per_iteration

as_api().experiments.update_metrics_per_iteration(weights_hash, metrics)

Send external metrics for a specific iteration.

The method lets you create a graph of the metrics for each iteration. The experiment and iterations are obtained from the weighted hash.

Each model's weights hash is attached to certain experiment epochs and thus can be used to send metrics that are relevant to those epochs. The model weights hash is a hexadecimal string. To calculate the weights hash of a model:

  • Calculate the sha1 strings of the weights for each layers

  • Calculate the sha1 string of the combine hashes.

    For example, the model has three layers with the layers' weight hashes: ['abc', '123', 'def'], the model weight hash is sha1('abc123def')

Warning

The same model weights hash can appear in different experiments or different epochs (for example, when running the same net twice). As such, this method will send the metrics to all the experiments/epochs that it can identify from the hash.

Parameters

  • weights_hash: String. Hexadecimal sha1 hash of the model's weights.
  • metrics: String. Metrics of the experiment as a jsonified string. The key should be the metric name with an ex prefix. For example, ex_cost. The value is the metric value in String, Float, Integer or Boolean format.