Experiment tracking and management

You can submit your experiments from your favorite interface - whether it is Jupyter Notebooks, a code editor or IDE like VS Code, the Python cli, the bash terminal, etc. You can also submit new experiments from Iterative Studio.

You can track live as well as completed experiments in Iterative Studio. First, click on Add a project to connect Iterative Studio to your ML project's Git repository. Then, follow the instructions presented below.

Track experiments in real-time

To quickly start tracking your experiments with Iterative Studio:

• In your model training environment, install DVCLive:

  pip install dvclive

• Copy your DVC Studio token and configure your model training environment to use the token:

  dvc config --global studio.token ***

• Use the DVCLive log_metric() method in your model training code:

  from dvclive import Live
  with Live(save_dvc_exp=True) as live:
    for epoch in range(epochs):
      live.log_metric("accuracy", accuracy)
      live.log_metric("loss", loss)
      live.next_step()

  DVCLive has implemented callbacks for several popular ML frameworks which simplify adding experiment tracking capabilities to your projects.

• Run the training job:

  python train.py

• The metrics and plots will be tracked live in the project in Iterative Studio.

Track reproducible pipelines

To set up, run and track reproducible pipelines:

• In your model training environment, install DVC:

  pip install dvc

• Initialize DVC in your Git repository:

  dvc init

  This will create a dvc.yaml file in the Git repository.

• Copy your DVC Studio token and configure your model training environment to use the token:

  dvc config --global studio.token ***

• Set up the DVC pipeline for model training.

• Run an experiment:

  dvc exp run

  Metrics, plots and other details of your experiment get saved in your project's dvc.yaml file.

• When the experiment completes, push the results:

  dvc exp push origin <exp_name>

• Iterative Studio reads the values of metrics, plots and other details from your project's dvc.yaml file, and displays them in an experiment row in the project table. The experiment row will appear nested within the parent Git commit row.

• From here, you can persist the experiments as Git commits and PRs as well as remove unnecessary experiments.

Visualize, compare and run experiments

Within a project, you can:

• Explore all the details of the experiments that you have pushed to your Git repository.
• Visualize the experiments using plots and trend charts.
• Compare experiments.
• Run new experiments, including on your own cloud instances, directly from Iterative Studio.
• Track live metrics and plots by sending them to Iterative Studio by using DVCLive.