exp run
Run or resume a DVC experiment based on a DVC pipeline.
When called with no arguments, this is equivalent to dvc repro
followed by
dvc exp save
.
Synopsis
usage: dvc exp run [-h] [-q | -v] [-f]
{ repro options ... } [-n <name>]
[-S [<filename>:]<override_pattern>]
[--queue] [--run-all] [-j <number>] [--temp]
[-r <experiment_rev>] [--reset] [-C <path>]
[--message <message>]
[targets [targets ...]]
positional arguments:
targets Stages to reproduce. 'dvc.yaml' by default
Description
Executes and tracks experiments in your repository without polluting it with unnecessary Git commits, branches, directories, etc.
Only files tracked by either Git or DVC are saved to the experiment. See
dvc exp save --include-untracked
for an alternative.
dvc exp run
has the same general behavior as dvc repro
when it comes to
targets
and stage execution (restores the dependency graph, etc.).
This includes committing any changed data dependencies to the DVC cache when preparing the experiment, which can take some time.
Use the --set-param
(-S
) option as a shortcut to change
parameter values on-the-fly before running the experiment.
It's possible to queue experiments for later execution with the --queue
flag. Queued experiments can be run with dvc queue start
and managed with
other dvc queue
commands.
It's also possible to run special checkpoint experiments that log the
execution progress (useful for deep learning ML). The --rev
and --reset
options have special uses for these.
See the Running Experiments guide for more details on these features and more.
Review your experiments with dvc exp show
. Successful ones can be made
persistent by restoring them via dvc exp branch
or dvc exp apply
and
committing them to the Git repo. Unnecessary ones can be cleared with
dvc exp remove
.
Options
In addition to the following,
dvc exp run
accepts the options indvc repro
except for--glob
,--no-commit
, and--no-run-cache
.
-
-S [<filename>:]<override_pattern>
,--set-param [<filename>:]<override_pattern>
- set the value ofdvc params
for this experiment. This will update the parameters file (params.yaml
by default) before running the experiment. Use the optional[<filename>:]
prefix to use a custom params file.Valid
<override_pattern>
values can be defined in Hydra's basic override syntax (see example). Hydra's choice and range sweep overrides are also supported, but these require the--queue
flag to be provided as well (see example). -
-n <name>
,--name <name>
- specify a unique name for this experiment. A default one will be generated otherwise, such aspuffy-daks
.The name of the experiment is exposed in env var
DVC_EXP_NAME
. -
--temp
- run this experiment outside your workspace (in.dvc/tmp/exps
). Useful to continue working (e.g. in another terminal) while a long experiment runs. -
--queue
- place this experiment at the end of a line for future execution, but don't run it yet. Usedvc queue start
to process the queue.For checkpoint experiments, this implies
--reset
unless a--rev
is provided. -
--run-all
- run all queued experiments (see--queue
) and outside your workspace (in.dvc/tmp/exps
). Use-j
to execute them in parallel.dvc exp run --run-all [--jobs]
is now a shortcut fordvc queue start [--jobs]
followed bydvc queue logs -f
. The--run-all
and--jobs
options will be deprecated in a future DVC release. -
-j <number>
,--jobs <number>
- run thisnumber
of queued experiments in parallel. Only has an effect along with--run-all
. Defaults to 1 (the queue is processed serially). -
-r <commit>
,--rev <commit>
- resume an experiment from a specific checkpoint name or hash (commit
) in--queue
or--temp
runs. -
--reset
- deletescheckpoint: true
outputs before running this experiment (regardless ofdvc.lock
). Useful for ML model re-training. -
-f
,--force
- reproduce pipelines even if no changes were found (same asdvc repro -f
). -
-C <path>
,--copy-paths <path>
- list of ignored or untracked paths to copy into the temp directory. Only used if--temp
or--queue
is specified. -
--message <message>
- custom message to use when saving the experiment. If not provided,dvc: commit experiment {hash}
will be used. -
-h
,--help
- prints the usage/help message, and exits. -
-q
,--quiet
- do not write anything to standard output. Exit with 0 if all stages are up to date or if all stages are successfully executed, otherwise exit with 1. The command defined in the stage is free to write output regardless of this flag. -
-v
,--verbose
- displays detailed tracing information.
Examples
This example is based on our Get Started, where you can find the actual source code.
Clone the DVC repo and download the data it depends on:
$ git clone git@github.com:iterative/example-get-started.git
$ cd example-get-started
$ dvc pull
Let's also install the Python requirements:
We strongly recommend creating a virtual environment first.
$ pip install -r src/requirements.txt
Let's check the latest metrics of the project:
$ dvc metrics show
Path avg_prec roc_auc
scores.json 0.60405 0.9608
For this experiment, we want to see the results for a smaller dataset input, so
let's limit the data to 20 MB and reproduce the pipeline with dvc exp run
:
$ truncate --size=20M data/data.xml
$ dvc exp run
...
Reproduced experiment(s): puffy-daks
Experiment results have been applied to your workspace.
$ dvc metrics diff
Path Metric HEAD workspace Change
scores.json avg_prec 0.60405 0.56103 -0.04302
scores.json roc_auc 0.9608 0.94003 -0.02077
The dvc metrics diff
command shows the difference in performance for the
experiment we just ran (puffy-daks
).
Example: Modify parameters on-the-fly
dvc exp run --set-param
(-S
) saves you the need to manually edit a params
file (see dvc params
) before running an experiment.
This option accepts Hydra's basic override syntax. For example, it can
override (train.epochs=10
), append (+train.weight_decay=0.01
), or remove
(~model.dropout
) parameters:
dvc exp run -S 'prepare.split=0.1' -S 'featurize.max_features=100'
...
Note that you can modify multiple parameters at once in the same command.
By default, -S
overwrites the values in params.yaml
. To use another params
file, add a <filename>:
prefix. For example, let's append a new parameter to
train_config.json
:
$ dvc exp run -S 'train_config.json:+train.weight_decay=0.001'
...
$ dvc params diff --targets train_config.json
Path Param HEAD workspace
train_config.json train.weight_decay - 0.001
Warnings
exp run --set-param
(-S
) doesn't update your dvc.yaml
to start or stop
tracking parameters. When appending or removing params, check if you need to
update the params
section accordingly.
Similarly, when using custom param files, check that these are defined in
dvc.yaml
.
Example: Run a grid search
Combining --set-param
and --queue
, we can perform a grid search for tuning
hyperparameters.
DVC supports Hydra's syntax for choice and range sweeps to add multiple experiments to the queue. These can be used for multiple parameters at the same time, adding all combinations to the queue:
$ dvc exp run -S 'train.min_split=8,64' -S 'train.n_est=range(100,500,100)' --queue
Queueing with overrides '{'params.yaml': ['train.min_split=8', 'train.n_est=100']}'.
Queued experiment 'azure-ices' for future execution.
Queueing with overrides '{'params.yaml': ['train.min_split=8', 'train.n_est=200']}'.
Queued experiment 'zingy-peri' for future execution.
Queueing with overrides '{'params.yaml': ['train.min_split=8', 'train.n_est=300']}'.
Queued experiment 'jammy-feds' for future execution.
Queueing with overrides '{'params.yaml': ['train.min_split=8', 'train.n_est=400']}'.
Queued experiment 'lowse-shay' for future execution.
Queueing with overrides '{'params.yaml': ['train.min_split=64', 'train.n_est=100']}'.
Queued experiment 'brown-hugs' for future execution.
Queueing with overrides '{'params.yaml': ['train.min_split=64', 'train.n_est=200']}'.
Queued experiment 'local-scud' for future execution.
Queueing with overrides '{'params.yaml': ['train.min_split=64', 'train.n_est=300']}'.
Queued experiment 'alpha-neck' for future execution.
Queueing with overrides '{'params.yaml': ['train.min_split=64', 'train.n_est=400']}'.
Queued experiment 'algal-hood' for future execution.
$ dvc queue start
...
Example: Include untracked or ignored paths
If your code relies on some paths that are intentionally untracked or ignored by
Git, you can use -C/--copy-paths
to ensure those files are accessible when you
use the --temp
or --queue
flags:
$ dvc exp run --temp -C secrets.txt -C symlinked-directory
The paths will be copied to the temporary directory but will not be tracked, to prevent unintentional leaks.