A useful plot to show the classification performance is the confusion matrix. In order to produce it, DVC expects a CSV plots file in the form:
actual,predicted 0,0 0,2 ...
We added a loop comparing the results to generate this file from the predictions.
Running the experiment with
dvc exp run will produce
You can run
dvc plots show on your terminal to generate and embed them in an
HTML file you can open in the browser (shown below). Or you can load your
project in VS Code and use the Plots Dashboard of the DVC Extension to
$ dvc plots show plots/confusion.csv --template confusion \ -x actual -y predicted file:///.../example-dvc-experiments/plots/confusion.json.html
Let's produce another plot to see misclassified examples from each class. This procedure generates the misclassification examples from the validation data and arranges them into a confusion table that shows the correct label, and misclassification sample. The code to generate an image from a set of training images is omitted here but you can find the code in the example project.
$ dvc plots show --open plots/misclassified.png
An important issue for deep learning projects is to observe in which epoch training and validation loss differ. DVC helps in that regard with its Python integrations to deep learning libraries via DVCLive.
The example project uses Keras to train a classifier, and we have a DVCLive callback that visualizes the training and validation loss for each epoch. We first import the callback from DVCLive.
from dvclive.keras import DVCLiveCallback
Then we add this callback to the
model.fit( ... callbacks=[DVCLiveCallback()], ...)
With these two changes, the model metrics are automatically logged to
dvclive/metrics.json and plotted in
DVCLive has other capabilities, like saving the model every epoch or modifying these default values.
In summary, DVC provides more than one option to use visualization in your workflow: