Edit on GitHub

Run Experiments

You can also run your ML experiments directly from DVC Studio. Your regular CI/CD set-up is used to run the experiments. For instance, CML Github Actions can be used to run ML experiments on each new commit.

Note that due to access restrictions, you cannot run experiments on the demo view (example-get-started) that is provided to you by default. Once you create views for your ML project repositories, you can follow the instructions given below to run experiments directly from DVC Studio.

Watch this video for an overview of how you can run experiments from DVC Studio, or read below for details.

To run experiments from DVC Studio, select the commit that you want to use and click the Run button. A form will let you specify all the changes that you want to make to your experiment. On this form, there are 2 types of inputs that you can change:

  1. Input data files: You can change datasets as well as metrics and model files. The list of files that you can change will depend on your ML project. For instance, in the example-get-started project, you can change the data.xml file. If you select the Show all input parameters (including hidden) option, then you can also change the hidden files such as the model.pkl model file and the scores.json metrics file. You can also choose not to change any input data files if you only wish to change the values of one or more hyperparameters.
  2. Hyperparameters: You can change the values of the hyperparameters that are defined in your ML project. For instance, in the example-get-started project, you can change max_features (the maximum number of features that the model uses), ngrams, etc. You can also choose not to change any hyperparameters if you only wish to change one or more input data files.

The default values of the input data files and hyperparameters in this form are extracted from your selected commit.

Once you have made all the required changes, enter your Git commit message and description. Then, select the branch to commit to. You can commit to either the base branch or a new branch. If you commit to a new branch, a Git pull request will automatically be created from the new branch to the base branch. Now, click on Commit changes.

At this point, the new experiment appears in the view table. If you just committed to a new branch, then a new pull request will also have been created from the new branch to the base branch.

If your project is integrated with a CI/CD setup (e.g. GitHub Actions), the CI/CD setup will get invoked. If this setup includes a model training process, it will be triggered, which means that your ML experiment will run automatically. The model training can happen on any cloud or Kubernetes. For more details on how to set up CI/CD pipelines for your ML project, refer to CML.

Once the experiment completes, its metrics will be available in the view table. You can then generate plots and trend charts for it, or compare it with the other experiments.

๐Ÿ› Found an issue? Let us know! Or fix it:

Edit on GitHub

โ“ Have a question? Join our chat, we will help you:

Discord Chat