Generate live (real-time) metrics and plots for running experiments

In your model training script, you can use DVCLive to send live updates for metrics and plots without writing them to your Git repository, so that you can track your experiments in real-time from DVC Studio.

This requires a 2-step process:

1. Set up an access token
2. Send and view the updates

Set up an access token

DVC Studio uses access tokens to authorize DVC and DVCLive to send live experiment updates. The access token must be present in any request that sends data to the DVC Studio ingestion endpoint. Requests with missing or incorrect access tokens are rejected with an appropriate HTTP error code and error message. The access token is also used by DVC to notify DVC Studio when you push experiments using dvc exp push.

Once you create your DVC Studio client access token with Experiment operations scope, pass it to your experiment. If you are running the experiment locally, you can use dvc studio login to interactively set the token:

$ dvc studio login

If you are running the experiment as part of a CI job, a secure way to provide the access token is to create a GitHub secret containing the value of the token, and use the secret in your CI job using the DVC_STUDIO_TOKEN environment variable (see example below).

steps:
  - name: Train model
    env:
      DVC_STUDIO_TOKEN: ${{ secrets.DVC_STUDIO_TOKEN }}

$ export DVC_STUDIO_TOKEN="<token>"
$ export DVC_EXP_GIT_REMOTE="https://github.com/<org>/<repo>"

Send and view live metrics and plots

Send live updates using DVCLive

In the training job (which has been configured as detailed above), whenever you log your metrics or plots using DVCLive, they will be automatically sent to DVC Studio. See DVC config for how to enable/disable live experiment updates and how to configure a different DVC Studio URL or Git repository. Here is an example of how you can use DVCLive in your training code:

from dvclive import Live

with Live() as live:
  for i in range(params["epochs"]):
    ...
    live.log_metric("accuracy", accuracy)
    live.next_step()
  ...

Live experiments in DVC Studio

DVC Studio stores the live experiments data in its database. In the project table, the live experiments are displayed in experiment rows, which are nested under the parent Git commit. Updates to the live experiments are highlighted (in orange) in the project table and compare pane in real time.

The number of live experiments with recent updates are displayed in the Live icon, which can also be used to filter and show only live (running) experiments in the table.

Live plots are displayed in the plots pane. You can see them getting populated as Studio receives new updates.

Detached experiments

A live experiment for which the parent Git commit is missing in the Git repository is displayed in a separate section called Detached experiments at the top of the project table.

Some of the reasons for missing parent commits are:

• the parent commit exists in your local clone of the repository and is not pushed to the Git remote
• the parent commit got removed by some mutative Git action such as rebase, hard reset with a push, squash commit, etc.

Once you push the missing parent commit to the Git remote, the live experiment will get nested under the parent commit as expected.

You can also delete the detached experiments if they are no longer important.

Experiment status

An experiment can have one of the following statuses:

• Running - DVC Studio expects to receive live metrics and plots for these experiments.

  If the experiment stops due to any error, DVC Studio will not be aware of this and it will continue to wait for live updates. In this case, you can delete the row from the project table.

• Completed - DVC Studio does not expect to receive any more updates for these experiments. Once the experiment concludes, you can delete the row from the project table.

  DVC Studio does not automatically commit and push the final results of your experiment to Git. You can push the experiment using appropriate DVC and Git commands.