usage: dvc exp run [-h] [-q | -v] [-f] [<repro_options> ...] [--set-param [<filename>:]<params_list>] [-n <name>] [--queue] [--run-all] [-j <number>] [targets [targets ...]] positional arguments: targets Stages to reproduce. 'dvc.yaml' by default
Provides a way to execute and track
dvc experiments in your
project without polluting it with unnecessary commits, branches,
dvc exp runis equivalent to
dvc reprofor experiments. It has the same behavior when it comes to
targetsand stage execution (restores the dependency graph, etc.). See the command options for more on the differences.
Before running an experiment, you'll probably want to make modifications such as
data and code updates, or hyperparameter tuning. For the latter,
you can use the
-S) option of this command to change
dvc param values on-the fly.
dvc exp run creates and tracks a variation based on the latest project
version committed to Git (
HEAD). Experiments will have an auto-generated name
exp-bfe64 by default, which can be customized using the
The results of the last experiment can be seen in the workspace. To
display and compare your experiments, use
dvc exp show or
dvc exp diff. Use
dvc exp apply to restore the results of any other experiment instead.
Note that experiment data will remain in the cache until you use regular
dvc gcto clean it up.
To track successive steps in a longer experiment, you can register checkpoints with DVC during your code or script runtime (similar to a logger).
To do so, first mark stage
checkpoint: true in
least one checkpoint output is needed so that the experiment can
later continue from that output's last cached state.
Then, in your code either call the
(Python), or write a signal file (any programming language) following the same
make_checkpoint() — please refer to its reference for details.
You can now use
dvc exp run to begin the experiment. If the process gets
interrupted (e.g. with
[Ctrl] C or by an error), all the checkpoints so far
will be preserved. When a run finishes normally, a final checkpoint will be
added (if needed) to wrap up the experiment.
Following uses of
dvc exp run will continue from this point (using the latest
cached versions of all outputs). You can add a
--rev to continue from a
previous checkpoint instead (list them with
dvc exp show). Or use
start over (discards previous checkpoints and deletes
checkpoint outputs, like
dvc exp run) — useful for re-training ML models, for example.
--queue option lets you create an experiment as usual, except that nothing
is actually run. Instead, the experiment is put in a wait-list for later
dvc exp show will mark queued experiments with an asterisk
Note that queuing an experiment that uses checkpoints implies
--reset, unless a
--revis provided (refer to the previous section).
dvc exp run --run-all to process the queue. Adding
experiment queues can be run in parallel for better performance. This creates a
temporary workspace copy for each subprocess (in
.dvc/tmp/exps). See also the
--temp option. Queued (and temporary) experiment runs do not get applied to
the workspace like regular ones.
⚠️ Parallel runs are experimental and may be unstable at this time. ⚠️ Make sure you're using a number of jobs that your environment can handle (no more than the CPU cores).
Note that each job runs the entire pipeline (or
targets) serially — DVC makes no attempt to distribute stage commands among jobs.
--set-param [<filename>:]<params_list>- set the specified
dvc paramsfor this experiment.
filenamecan be any valid params file (
params_listaccepts a comma-separated list of key-value pairs in the form
key1=val1,key2=val2.... This will override the param values coming from the params file.
--name <name>- specify a name for this experiment. A default name will generated by default, such as
exp-f80g4(based on the experiment's hash).
--queue- place this experiment at the end of a line for future execution, but do not actually run it yet. Use
dvc exp run --run-allto process the queue. For checkpoint experiments, this implies
--run-all- run all queued experiments (see
-jto execute them in parallel.
--jobs <number>- run this
numberof queued experiments in parallel. Only applicable when used in conjunction with
--temp- run this experiment in a separate temporary directory (in
.dvc/tmp/exps) instead of your workspace.
--rev <commit>- continue an experiment from a specific checkpoint name or hash (
commit). This is needed for example to resume experiments from
checkpointoutputs before running this experiment (regardless of
dvc.lock). Useful for ML model re-training.
--force- reproduce pipelines even if no changes were found (same as
dvc repro -f).
--help- prints the usage/help message, and exit.
--quiet- do not write anything to standard output. Exit with 0 if all stages are up to date or if all stages are successfully executed, otherwise exit with 1. The command defined in the stage is free to write output regardless of this flag.
--verbose- displays detailed tracing information.
These examples are based on our Get Started, where you can find the actual source code.
Let's check the latest metrics of the project:
$ dvc metrics show Path avg_prec roc_auc scores.json 0.60405 0.9608
For this experiment, we want to see the results for a smaller dataset input, so
let's limit the data to 20 MB and reproduce the pipeline with
dvc exp run:
$ truncate --size=20M data/data.xml $ dvc exp run ... Reproduced experiment(s): exp-44136 Experiment results have been applied to your workspace. $ dvc metrics diff Path Metric Old New Change scores.json avg_prec 0.60405 0.56103 -0.04302 scores.json roc_auc 0.9608 0.94003 -0.02077
dvc metrics diff command shows the difference in performance for the
experiment we just ran (
You could modify a params file just like any other dependency and
run an experiment on that basis. Since this is a common need,
dvc exp run
comes with the
-S) option built-in. This saves you the need to
manually edit the params file:
$ dvc exp run -S prepare.split=0.25 -S featurize.max_features=2000 ... Reproduced experiment(s): exp-18bf6 Experiment results have been applied to your workspace.
To see the results, we can use
dvc exp diff which compares both params and
metrics to the previous project version:
$ dvc exp diff Path Metric Value Change scores.json avg_prec 0.58187 -0.022184 scores.json roc_auc 0.93634 -0.024464 Path Param Value Change params.yaml featurize.max_features 2000 -1000 params.yaml prepare.split 0.25 0.05
Notice that experiments run as a series don't build up on each other. They are all based on