In a regular Git workflow, DVC repository versions are typically synchronized among team members. And DVC Experiments are internally connected to this commit history. But to avoid cluttering everyone's copies of the repo, by default experiments will only exist in the local environment where they were created.
You must explicitly save or share experiments individually on other locations. This is done similarly to sharing regular project versions, by synchronizing with DVC and Git remotes. But DVC takes care of pushing and pulling to/from Git remotes in the case of experiments.
┌────────────────┐ ┌────────────────┐ ├────────────────┤ │ │ Remote locations │ DVC remote │ │ Git remote │ │ storage │ ├────────────────┤ └────────────────┘ └────────────────┘ ▲ ▲ │ dvc exp push │ │ dvc exp pull │ ▼ ▼ ┌─────────────────┐ ┌────────────────┐ │ │ │ Code and │ │ Cached data │ │ metafiles │ Local project └─────────────────┘ └────────────────┘
Specifically, data, models, etc. are tracked and cached by DVC and thus will be transferred to/from remote storage (e.g. Amazon S3 or Google Drive). Small files like DVC metafiles and code are tracked by Git, so DVC pushes and pulls them to/from your existing Git remotes.
Make sure that you have the necessary remotes setup. Let's confirm with
git remote -v and
dvc remote list:
$ git remote -v origin email@example.com:iterative/get-started-experiments.git (fetch) origin firstname.lastname@example.org:iterative/get-started-experiments.git (push) $ dvc remote list storage s3://mybucket/my-dvc-store
⚠️ Note that DVC can only authenticate with Git remotes using SSH URLs.
You can upload an experiment with all of its files and data using
dvc exp push, which takes a Git remote name and an experiment ID or name as
💡 You can use
dvc exp showto find experiment names.
$ dvc exp push origin exp-abc123
See also How to Share Many Experiments.
When you clone a DVC repository, it doesn't fetch any experiments by default. In
order to get them, use
dvc exp pull (with the Git remote and the experiment
name), for example:
$ dvc exp pull origin cnn-32
This pulls all the necessary files from both remotes. Again, you need to have both of these configured (see this earlier section).
If an experiment being pulled already exists in the local project, DVC won't
overwrite it unless you supply