You can also run your ML experiments directly from DVC Studio. Your regular CI/CD set-up is used to run the experiments. For instance, CML Github Actions can be used to run ML experiments on each new commit.
Note that due to access restrictions, you cannot run experiments on the demo view (
example-get-started) that is provided to you by default. Once you create views for your ML project repositories, you can follow the instructions given below to run experiments directly from DVC Studio.
Watch this video for an overview of how you can run experiments from DVC Studio, or read below for details.
To run experiments from DVC Studio, select the commit that you want to use and
Run button. A form will let you specify all the changes that you
want to make to your experiment. On this form, there are 2 types of inputs that
you can change:
example-get-startedproject, you can change the
data.xmlfile. If you select the
Show all input parameters (including hidden)option, then you can also change the hidden files such as the
model.pklmodel file and the
scores.jsonmetrics file. You can also choose not to change any input data files if you only wish to change the values of one or more hyperparameters.
example-get-startedproject, you can change
max_features(the maximum number of features that the model uses),
ngrams, etc. You can also choose not to change any hyperparameters if you only wish to change one or more input data files.
The default values of the input data files and hyperparameters in this form are extracted from your selected commit.
Once you have made all the required changes, enter your Git commit message and
description. Then, select the branch to commit to. You can commit to either the
base branch or a new branch. If you commit to a new branch, a Git pull request
will automatically be created from the new branch to the base branch. Now, click
At this point, the new experiment appears in the view table. If you just committed to a new branch, then a new pull request will also have been created from the new branch to the base branch.
If your project is integrated with a CI/CD setup (e.g. GitHub Actions), the CI/CD setup will get invoked. If this setup includes a model training process, it will be triggered, which means that your ML experiment will run automatically. The model training can happen on any cloud or Kubernetes. For more details on how to set up CI/CD pipelines for your ML project, refer to CML.
Once the experiment completes, its metrics will be available in the view table. You can then generate plots and trend charts for it, or compare it with the other experiments.