Experiments Methods
experiments.logs
as_api().experiments.logs(project, experiment=None, disable_colors=False)
View the execution logs of an experiment that is running in a remote location.
Parameters
- project: Integer. Project Id.
- experiment: Optional. Integer. Experiment Id.
- disable_colors: Optional. Boolean. Enable colors in the console? Default is
False
.
experiments.list
as_api().experiments.list(project)
Parameters
- project: Integer. Project Id.
experiments.update_metrics
as_api().experiments.update_metrics(metrics, project=None, experiment=None, weights_hash=None)
Add metrics to an experiment that has already run.
Parameters
- metrics: String. Metrics of the experiment as a jsonified string. The key should be the
metric name with an
ex
prefix. For example,ex_cost
. The value is the metric value in String, Float, Integer or Boolean format. - project: Optional. Integer. Project Id.
- experiment: Optional. Integer. Experiment Id.
- weights_hash: Optional. String. Hexadecimal sha1 hash of the model's weights.
experiments.update_chart
as_api().experiments.update_chart(chart_name, weights_hash=None, project=None, experiment=None, chart_x_name="X", chart_y_name="Y", chart_scope="test", chart_type="line" chart_x=None, chart_y=None)
Inject a custom chart into the given experiment.
The charts can be single line charts (having a single Y value per X point) or multipoint charts (having multiple Y values per X point). The experiment can be identified by the project and experiment Ids or by model weights hash.
Parameters
- chart_name: String. The name of the chart. The name is used to identify the chart against different experiments and through the same experiment.
- weights_hash: Optional. String. Hexadecimal sha1 hash of the model's weights. Not required if
project
andexperiment
are provided. - project: Optional. Integer. Project Id. Not required if
weights_hash
is provided. - experiment: Optional. Integer. Experiment Id. Not required if
weights_hash
is provided. - chart_x_name: Optional. String. Display name for chart's X axis. Default is
X
. - chart_y_name: Optional. String. Display name for chart's Y axis. Default is
Y
. - chart_scope: Optional. String. Scope type. Default is
test
. Other options arevalidation
andtrain
. - chart_type: Optional. String. Default is
line
. - chart_x: Optional. String.
- chart_y: Optional. String.
experiments.update_metrics_per_iteration
as_api().experiments.update_metrics_per_iteration(weights_hash, metrics)
Send external metrics for a specific iteration.
The method lets you create a graph of the metrics for each iteration. The experiment and iterations are obtained from the weighted hash.
Each model's weights hash is attached to certain experiment epochs and thus can be used to send metrics that are relevant to those epochs. The model weights hash is a hexadecimal string. To calculate the weights hash of a model:
-
Calculate the sha1 strings of the weights for each layers
-
Calculate the sha1 string of the combine hashes.
For example, the model has three layers with the layers' weight hashes: ['abc', '123', 'def'], the model weight hash is
sha1('abc123def')
Warning
The same model weights hash can appear in different experiments or different epochs (for example, when running the same net twice). As such, this method will send the metrics to all the experiments/epochs that it can identify from the hash.
Parameters
- weights_hash: String. Hexadecimal sha1 hash of the model's weights.
- metrics: String. Metrics of the experiment as a jsonified string. The key should be the metric name with an
ex
prefix. For example,ex_cost
. The value is the metric value in String, Float, Integer or Boolean format.