Edit on GitHub

Get Started

DVCLive is a Python library for logging machine learning parameters, metrics and other metadata in simple file formats, which is fully compatible with DVC, the VS Code Extension, and Iterative Studio.

Set up DVCLive

First of all, you need to add DVCLive to your Python code:

# train.py
from dvclive.keras import DVCLiveCallback

...

model.fit(
  train_dataset, validation_data=validation_dataset,
  callbacks=[DVCLiveCallback(save_dvc_exp=True)])
# train.py
from dvclive.huggingface import DVCLiveCallback

...

trainer.add_callback(DVCLiveCallback(save_dvc_exp=True))
trainer.train()
# train.py
from dvclive.lightning import DVCLiveLogger

...

trainer = Trainer(logger=DVCLiveLogger(save_dvc_exp=True))
trainer.fit(model)
# train.py
from dvclive import Live

with Live(save_dvc_exp=True) as live:
    live.log_param("epochs", NUM_EPOCHS)

    for epoch in range(NUM_EPOCHS):
        train_model(...)
        metrics = evaluate_model(...)
        for metric_name, value in metrics.items():
            live.log_metric(metric_name, value)
        live.next_step()

Including save_dvc_exp=True will automatically track the results using Git.

Check the ML Frameworks page for more details and other supported frameworks.

Run the code

Once you have added DVCLive to our python code, you can run the code as you would usually do:

$ python train.py

The logged data will be stored in plain text files. For this simple example, it would look as follows:

dvclive
├── metrics.json
├── params.yaml
├── plots
│   └── metrics
│       └── accuracy.tsv
└── report.html

You can find more details about the structure of files generated by DVCLive on the How it Works page.

Monitor the training

Depending on your setup, DVCLive enables different ways of monitoring the training progress:

At the end of each step, DVCLive will generate or update a report in Live.report_file displaying all the logged data.

HTML report

The DVC Extension for VS Code will display all the data logged by DVCLive.

VS Code monitoring

Compare results

If you track the results of each experiment, you can compare them in different ways:

You can use dvc exp show and dvc plots to compare and visualize metrics, parameters and plots across experiments.

$ dvc exp show
────────────────────────────────────────────────────────────────────────────────────
Experiment                 Created        train_loss   epoch   step   encoder_size
────────────────────────────────────────────────────────────────────────────────────
workspace                  -                0.020196       4    500   512
main                       Dec 06, 2022            -       -      -   -
├── c1759a5 [quare-foil]   08:55 PM         0.020196       4    500   512
├── affedee [bitty-tass]   08:55 PM          0.02038       4    500   256
├── a5bdc18 [murky-emeu]   08:55 PM         0.016396       4    500   128
├── 744f3b6 [sworn-wage]   08:54 PM          0.01972       4    500   64
└── 0c3ac81 [named-gaby]   08:54 PM         0.031206       4    500   32
────────────────────────────────────────────────────────────────────────────────────
$ dvc plots diff $(dvc exp list --name-only)

plots diff

You can compare and visualize results using the Experiments and Plots views.

VS Code monitoring

Share Results

After the experiment has finished and you have committed and pushed the results, Iterative Studio will automatically parse the outputs generated by DVCLive, allowing you to share your experiments online:

Studio view

Content

🐛 Found an issue? Let us know! Or fix it:

Edit on GitHub

Have a question? Join our chat, we will help you:

Discord Chat