Experiment reproducibility and debriefing
To let you reproduce experiments easily, MissingLink captures all of the following experiment elements:
- Input data
- Runtime environment
When you integrate your experiments with other MissingLink features, MissingLink automatically links your experiment with the elements that are managed by those features:
- Integrate Experiment Management to let MissingLink document hyperparams, references to source code, and metrics.
- Integrate Data Management to let MissingLink document input data.
- Integrate Resource Management to let MissingLink document the runtime environment and exact resources you used to run the experiment.
Once all this is integrated, you can easily do the following:
- Reproduce an experiment with one click.
- Compare the data queries between experiments and analyze the impact of the different data sets on the experiment performance.
- Compare hyperparams and code changes between experiments.
Integration between Data Management and Experiment Management can be performed in two ways:
- Using Resource management. With this method, MissingLink Agent is responsible for getting the data to the training machine.
- Using Data Management iterators. Here, the data is fetched under the context of the training.
However, if you clone data by using the
ml data clone command, the data query is not automatically added into the experiment. In such a case, we recommend you document the query by other means.
All the elements that are associated with each experiment are shown in the tabs below it.