Experiment tracking and management
You can submit your experiments from your favorite interface - whether it is Jupyter Notebooks, a code editor or IDE like VS Code, the Python cli, the bash terminal, etc. You can also submit new experiments from Iterative Studio.
You can track live as well as completed experiments in Iterative Studio. First,
click on Add a project
to connect Iterative Studio to your ML project's Git
repository. Then, follow the instructions presented below.
Track experiments in real-time
To quickly start tracking your experiments with Iterative Studio:
-
In your model training environment, install DVCLive:
pip install dvclive
-
Copy your DVC Studio token and configure your model training environment to use the token:
dvc config --global studio.token ***
-
Use the DVCLive
log_metric()
method in your model training code:from dvclive import Live with Live(save_dvc_exp=True) as live: for epoch in range(epochs): live.log_metric("accuracy", accuracy) live.log_metric("loss", loss) live.next_step()
DVCLive has implemented callbacks for several popular ML frameworks which simplify adding experiment tracking capabilities to your projects.
-
Run the training job:
python train.py
-
The metrics and plots will be tracked live in the project in Iterative Studio.
Track reproducible pipelines
To set up, run and track reproducible pipelines:
-
In your model training environment, install DVC:
pip install dvc
-
Initialize DVC in your Git repository:
dvc init
This will create a
dvc.yaml
file in the Git repository. -
Copy your DVC Studio token and configure your model training environment to use the token:
dvc config --global studio.token ***
-
Set up the DVC pipeline for model training.
-
dvc exp run
Metrics, plots and other details of your experiment get saved in your project's
dvc.yaml
file. -
When the experiment completes, push the results:
dvc exp push origin <exp_name>
-
Iterative Studio reads the values of metrics, plots and other details from your project's
dvc.yaml
file, and displays them in an experiment row in the project table. The experiment row will appear nested within the parent Git commit row. -
From here, you can persist the experiments as Git commits and PRs as well as remove unnecessary experiments.
Visualize, compare and run experiments
Within a project, you can:
- Explore all the details of the experiments that you have pushed to your Git repository.
- Visualize the experiments using plots and trend charts.
- Compare experiments.
- Run new experiments, including on your own cloud instances, directly from Iterative Studio.
- Track live metrics and plots by sending them to Iterative Studio by using DVCLive.