Edit on GitHub

Sharing Experiments

You can send live metrics and plots to DVC Studio, push entire completed experiments (including data, models, and code), and convert an experiment into a persistent branch or commit in your Git repo.

See the video below for how it all works using the DVC Extension for VS Code, or keep reading to go deeper.

By clicking play, you agree to YouTube's Privacy Policy and Terms of Service

Live metrics and plots

You can send live experiments to DVC Studio, which will show intermediate results for metrics and plots in any running experiments. To start sharing live metrics to DVC Studio, get your Studio token and save it in your dvc config or DVC_STUDIO_TOKEN environment variable. For example, to set it globally for all of a user's projects:

$ dvc config --global studio.token ***

While the experiment runs, you will see live updates like this in DVC Studio (and so will anyone else with access to the project):

Live metrics in DVC Studio

Live plots in DVC Studio

See DVC config for how to enable/disable live metrics and how to configure a different DVC Studio URL or Git repository, or see the DVC Studio guide on live experiments for more information on how to setup, view, and compare.

Push experiments

dvc exp push pushes experiment commits that Git can upload to remote servers like GitHub but don't show up in the UI (so they don't clutter your repo) and can be cleaned up without affecting the rest of your project.

To understand how dvc exp push works, let's compare to pushing a persistent commit. With a typical Git commit, you would use git push to upload it to your Git remote and dvc push to upload the corresponding data to your DVC remote.

 โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
 โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค  โ”‚   DVC remote   โ”‚ Remote locations
 โ”‚   Git remote   โ”‚  โ”‚    storage     โ”‚
 โ”‚                โ”‚  โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
 โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
         โ–ฒ                    โ–ฒ
         โ”‚                    โ”‚
      git push             dvc push
      git pull             dvc pull
         โ”‚                    โ”‚
         โ–ผ                    โ–ผ
 โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
 โ”‚    Code and    โ”‚  โ”‚      Data      โ”‚
 โ”‚    metafiles   โ”‚  โ”‚    (cached)    โ”‚ Local project
 โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜

dvc exp push and dvc exp pull take care of synchronizing to/from both Git and DVC remotes as needed:

 โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
 โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค  โ”‚   DVC remote   โ”‚ Remote locations
 โ”‚   Git remote   โ”‚  โ”‚    storage     โ”‚
 โ”‚                โ”‚  โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
 โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
         โ–ฒ                    โ–ฒ
         โ”‚   dvc exp push     โ”‚
         โ”‚   dvc exp pull     โ”‚
         โ–ผ                    โ–ผ
 โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
 โ”‚    Code and     โ”‚ โ”‚      Data      โ”‚
 โ”‚    metafiles    โ”‚ โ”‚    (cached)    โ”‚ Local project
 โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜

DVC experiments are tied to your Git repo, but they are ignored by normal Git operations like git push, git pull, and git clone. You can upload completed experiments using dvc exp push. This will push not only metrics and plots, but also the code and DVC-cached files (data and models) so you can pull an experiment, make it persistent, and reproduce it from your Git repo. For example, in the simplest case, push experiments to Git remote origin:

$ dvc exp push origin

If you don't know your Git remote, check with git remote -v or see troubleshooting for problems.

By default, DVC will also share cached data that is tracked by DVC, which requires remote storage (e.g. Amazon S3 or SSH). Add the --no-cache flag to exclude sharing cached data.

By default, dvc exp push origin will push all experiments derived from your current Git commit, but you may specify specific experiments as arguments or use the flags to select a different set of experiments to push.

Find pushed experiments

You can see pushed experiments in DVC Studio. From there, you can make an experiment persistent by creating a Git branch, or you can remove it from your Git remote:

DVC Studio Shared Experiments

From your workspace, you can see pushed experiments if you provide a Git remote name to dvc exp list.

$ dvc exp list origin
refs/tags/baseline-experiment:
    cnn-32
    cnn-64

dvc exp list origin lists remote experiments based on your current commit. You can use --all-commits (-A) to list all experiments, or add any other supported option.

Pull experiments

To download pushed experiments, use dvc exp pull (with the Git remote and experiment name).

$ dvc exp pull origin

This puts all the necessary files and data (from both Git and DVC remotes) in your project.

Add the --no-cache flag to exclude pulling from the DVC remote.

By default, dvc exp pull origin will pull all experiments derived from your current Git commit, but you may specify specific experiments as arguments or use the flags to select a different set of experiments to push.

Persist experiment

DVC experiments run outside of the regular Git workflow for faster iteration and to avoid polluting your repository's history, but you can easily bring back the most promising experiments into your regular Git workflow. You can convert any pushed experiment from DVC Studio into a persistent Git branch and create a pull request to merge it into your main repo branch:

DVC Studio Create a New Branch

Alternatively, from your workspace, to share an individual experiment the same way you share other Git commits, use dvc exp branch to create a Git branch from the experiment and share it like any Git branch.

$ dvc exp branch quare-zips
Git branch 'quare-zips-branch' has been created from experiment 'quare-zips'.

$ git checkout quare-zips-branch
Switched to branch 'quare-zips-branch'

$ git push origin quare-zips-branch

If you only need to share code and metadata (like parameters and metrics), then pushing to Git should be enough.

You may also have cached data, models, etc. tracked by DVC. To share these, dvc push them to remote storage (e.g. Google Drive or NAS).

$ dvc push

If you don't want to create a new Git branch and instead want to commit the experiment directly on top of your current Git branch, you can bring experiment results to your workspace.

Remove pushed experiments

As you share more experiments, DVC Studio and Git remotes may be become cluttered with experiment references.

You can remove experiments in DVC Studio:

DVC Studio Remove selected rows

To remove pushed experiments using the command line, use dvc exp remove -g:

$ dvc exp remove -g origin unwet-jinn
Removed experiments: unwet-jinn
Content

๐Ÿ› Found an issue? Let us know! Or fix it:

Edit on GitHub

โ“ Have a question? Join our chat, we will help you:

Discord Chat