metrics diff

Show changes in metrics between commits in the DVC repository, or between a commit and the workspace.

Synopsis

usage: dvc metrics diff [-h] [-q | -v] [--targets [<paths> [<paths> ...]]]
                        [-R] [--all] [--show-json] [--show-md] [--no-path]
                        [--old] [a_rev] [b_rev]

positional arguments:
  a_rev                 Old Git commit to compare (defaults to HEAD)
  b_rev                 New Git commit to compare (defaults to the
                        current workspace)

Description

This command provides a quick way to compare metrics among experiments in the repository history. It requires that Git is being used to version the metrics.

This kind of metrics can be defined with the -m (--metrics) and -M (--metrics-no-cache) options of dvc run.

Run without arguments, this command compares metrics currently present in the workspace uncommitted changes) with the latest committed version.

The differences shown by this command include the new value, and numeric difference (delta) from the previous value of metrics (with 3-digit accuracy). They're calculated between two commits (hash, branch, tag, or any Git revision) for all metrics in the project, found by examining all of the DVC-files in both references.

Another way to display metrics is the dvc metrics show command, which just lists all the current metrics without comparisons.

Options

--targets <paths> - limit the comparison to these specific metric files.
-R, --recursive - determines the metric files to use by searching each target directory and its subdirectories for DVC-files to inspect. If there are no directories among the targets, this option is ignored.
--all - list all metrics, even those without changes.
--show-json - prints the command's output in easily parsable JSON format, instead of a human-readable table.
--show-md - prints the command's output in Markdown table format.
--old - show old metric value in addition to the new value.
--no-path - don't show metric path in the result table. This option is useful when only one metrics file is in use or there is no intersection between the metric names.
-h, --help - prints the usage/help message, and exit.
-q, --quiet - do not write anything to standard output. Exit with 0 if no problems arise, otherwise 1.
-v, --verbose - displays detailed tracing information.

Examples

Start by creating a metrics file and commit it (see the -M option of dvc run for more details):

$ dvc run -n eval -M metrics.json \
          'echo {"AUC": 0.9643, "TP": 527} > metrics.json'

$ cat metrics.json
{"AUC": 0.9643, "TP": 527}

$ git add dvc.* metrics.json
$ git commit -m "Add metrics file"

Now let's simulate a change in our AUC metric:

$ echo '{"AUC":0.9671, "TP":531}' > metrics.json

$ git diff
...
-{"AUC":0.9643, "TP":527}
+{"AUC":0.9671, "TP":531}

To see the change, let's run dvc metrics diff. This compares our current workspace (including uncommitted local changes) metrics to what we had in the previous commit:

$ dvc metrics diff
Path          Metric    Value    Change
metrics.json  AUC       0.9671   0.0028
metrics.json  TP        531      4

Example: compare metrics among specific versions

Metric files committed with Git can be compared by referencing the commits (any two revisions):