Get Started

DVCLive is a Python library for logging machine learning parameters, metrics and other metadata in simple file formats, which is fully compatible with DVC, the VS Code Extension, and Iterative Studio.

Set up DVCLive

First of all, you need to add DVCLive to your Python code:

Keras

# train.py
from dvclive.keras import DVCLiveCallback

...

model.fit(
  train_dataset, validation_data=validation_dataset,
  callbacks=[DVCLiveCallback(save_dvc_exp=True)])

Hugging Face

# train.py
from dvclive.huggingface import DVCLiveCallback

...

trainer.add_callback(DVCLiveCallback(save_dvc_exp=True))
trainer.train()

Pytorch Lightning

# train.py
from dvclive.lightning import DVCLiveLogger

...

trainer = Trainer(logger=DVCLiveLogger(save_dvc_exp=True))
trainer.fit(model)

Python API

# train.py
from dvclive import Live

with Live(save_dvc_exp=True) as live:
    live.log_param("epochs", NUM_EPOCHS)

    for epoch in range(NUM_EPOCHS):
        train_model(...)
        metrics = evaluate_model(...)
        for metric_name, value in metrics.items():
            live.log_metric(metric_name, value)
        live.next_step()

Including save_dvc_exp=True will automatically track the results using Git.

Check the ML Frameworks page for more details and other supported frameworks.

Run the code

Once you have added DVCLive to our python code, you can run the code as you would usually do:

$ python train.py

The logged data will be stored in plain text files. For this simple example, it would look as follows:

dvclive
├── metrics.json
├── params.yaml
├── plots
│   └── metrics
│       └── accuracy.tsv
└── report.html

You can find more details about the structure of files generated by DVCLive on the How it Works page.

Monitor the training

Depending on your setup, DVCLive enables different ways of monitoring the training progress:

Standalone report

At the end of each step, DVCLive will generate or update a report in Live.report_file displaying all the logged data.

HTML report

DVC Extension for VS Code

The DVC Extension for VS Code will display all the data logged by DVCLive.

VS Code monitoring

Compare results

If you track the results of each experiment, you can compare them in different ways:

DVC CLI

You can use dvc exp show and dvc plots to compare and visualize metrics, parameters and plots across experiments.

$ dvc exp show
────────────────────────────────────────────────────────────────────────────────────
Experiment                 Created        train_loss   epoch   step   encoder_size
────────────────────────────────────────────────────────────────────────────────────
workspace                  -                0.020196       4    500   512
main                       Dec 06, 2022            -       -      -   -
├── c1759a5 [quare-foil]   08:55 PM         0.020196       4    500   512
├── affedee [bitty-tass]   08:55 PM          0.02038       4    500   256
├── a5bdc18 [murky-emeu]   08:55 PM         0.016396       4    500   128
├── 744f3b6 [sworn-wage]   08:54 PM          0.01972       4    500   64
└── 0c3ac81 [named-gaby]   08:54 PM         0.031206       4    500   32
────────────────────────────────────────────────────────────────────────────────────

$ dvc plots diff $(dvc exp list --name-only)

plots diff

DVC Extension for VS Code

You can compare and visualize results using the Experiments and Plots views.

After the experiment has finished and you have committed and pushed the results, Iterative Studio will automatically parse the outputs generated by DVCLive, allowing you to share your experiments online: