Once you have saved several experiments, DVC provides commands to review, compare, and manage those experiments. Let's see how they can help you streamline the experimentation process.
Experimentation is about generating many possibilities before selecting a few of them. You can get a table of experiments, which displays all the metrics (yellow), parameters (blue) and dependencies (violet) in a nicely formatted table.
$ dvc exp show
──────────────────────────────────────────────────────────────────────────────────────────────────────────────── neutral:**Experiment** neutral:**Created** metric:**loss** metric:**acc** param:**train.epochs** param:**model.conv_units** dep:**src** dep:**data** ──────────────────────────────────────────────────────────────────────────────────────────────────────────────── workspace - 0.03332 0.9888 10 16 695e061 6875529 baseline-experiment Jan 14, 2022 0.03332 0.9888 10 16 695e061 6875529 ├── 38d6c53 [cnn-64] Jan 19, 2022 0.038246 0.988 10 64 c77a505 6875529 └── bc0faf5 [cnn-128] Jan 19, 2022 0.038325 0.989 10 128 bc75d6a 6875529 ────────────────────────────────── ──────────────────────────────────────────────────────────────────────────────
dvc exp show only tabulates experiments in the workspace and in
-A) flag to show all the experiments in the project
Show Experiments action in the DVC extension for VS Code.
The table output may become cluttered if you have a large number of metrics, parameters and dependencies. DVC provides several options to select the columns to be shown in the table.
dvc exp show --only-changed will drop any column with values that
do not change across experiments:
$ dvc exp show --only-changed
─────────────────────────────────────────────────────────────────────────────── neutral:**Experiment** neutral:**Created** metric:**loss** metric:**acc** param:**model.conv_units** ─────────────────────────────────────────────────────────────────────────────── workspace - 0.03332 0.9888 16 baseline-experiment Jan 14, 2022 0.03332 0.9888 16 ├── 38d6c53 [cnn-64] Jan 19, 2022 0.038246 0.988 64 └── bc0faf5 [cnn-128] Jan 19, 2022 0.038325 0.989 128 ───────────────────────────────────────────────────────────────────────────────
See more examples in the
dvc exp show reference for how to
filter columns, sort rows, and otherwise customize the table.
You can hide and move columns and filter rows in the
Show Experiments action
in the DVC extension for VS Code.
You can also sort both experiment and commit rows in the extension.
If viewing the table isn't enough and you need programmatic access to the experiments table, you can save it as a CSV or access it using the Python API.
The experiments table compares scalar parameters and metrics. You can also save plots data from each exepriment and overlay plots from different experiments to compare them. Plots can include metrics trends, non-scalar data like ROC curves or histograms, and static images to compare. Get more info on visualizing plots and see the examples below for how to compare plots between experiments.
You can use
dvc plots diff to compare plots across experiments. To get the
names of experiments to compare, use
dvc exp list --name-only, which will
return the names of all experiments in
HEAD. See the options for
dvc exp list to choose a different set of experiments.
$ dvc plots diff $(dvc exp list --name-only)
In the DVC extension for VS Code, select any experiments in the table to plot.
To inspect an experiment beyond what you can see in the table or plots (like inspecting code or data changes), you can restore a previous experiment's results to your workspace.
⚠️ Conflicting changes in the workspace are overwritten, but they can be recovered.
$ dvc exp apply ochre-dook Changes for experiment 'ochre-dook' have been applied...
You can apply the results of any experiment to the workspace in the DVC extension for VS Code.
You can use standard Git commands (e.g.
git add/commit/push) to persist this
experiment directly in the repository. DVC-tracked data and
artifacts are already in the DVC cache, and the rest (params, code and config
files, etc.) can be stored in Git.
Note that you need to
dvc pushin order to share or backup the DVC cache contents.
The methods above let you compare experiments from your local machine where you are working without any servers, logins, or infrastructure. However, sometimes you are away from your machine, or you want to share the results with others. DVC Studio provides a web collaboration hub for sharing experiments and visualizing and comparing experiments for your entire project history.
Although DVC uses minimal resources to keep track of the experiments, they may clutter tables and the workspace. When you want to discard experiments by their name, you can remove them.
dvc exp remove and supply the experiment name.
$ dvc exp list main: 2399f24 [cnn-128] 4e8a178 [cnn-32] f1edf21 [cnn-64] $ dvc exp remove cnn-32 cnn-64 Removed experiments: cnn-32,cnn-64
dvc exp remove options for how to choose sets of multiple experiments
You can remove any number of experiments in the DVC extension for VS Code.
Removing an experiment does not delete any objects in the DVC
cache. To remove the cache objects, e.g. model files, intermediate
artifacts, etc. related with the experiments, you can use