List project contents, including files, models, and directories tracked by DVC and by Git.
Useful to find data to
dvc import, or for
usage: dvc list [-h] [-q | -v] [-R] [--dvc-only] [--json] [--rev [<commit>]] url [path] positional arguments: url Location of DVC or Git repository to list from path Path to a file or directory in the repository
Produces a view of a DVC repository (usually online), listing data
files and directories tracked by DVC alongside the remaining Git repo contents.
This is useful because when you browse a hosted repository (e.g. on GitHub or
git ls-remote), you only see the
.dvc files with your
code (files tracked by Git).
This command's output is equivalent to cloning the repo and pulling the data (except that nothing is downloaded), like this:
$ git clone <url> example $ cd example $ dvc pull $ ls <path>
url argument specifies the address of the DVC or Git repository containing
the data source. Both HTTP and SSH protocols are supported (e.g.
url can also be a local file system path
(including the current project e.g.
.). Any path inside a DVC project will be
resolved to the project's root.
path argument specifies a file or directory to list (paths inside
tracked directories are supported). It should be relative to the root of the
repo (absolute paths are supported when
url is local). This is similar to
providing a path to listing commands such as
aws s3 ls.
Only the root directory is listed by default, but the
-R option can be used to
list files recursively.
dvc list doesn't check whether the listed data (tracked by DVC)
actually exists in remote storage, so it's not guaranteed whether it can be
dvc import, or
--recursive- recursively list files in all subdirectories.
--dvc-only- show only DVC-tracked files and directories (outputs).
--rev <commit>- commit hash, branch or tag name, etc. (any Git revision) of the repository to list content for. The latest commit (in the default branch) is used by default when this option is not specified.
--json- prints the command's output in easily parsable JSON format, instead of a human-readable table.
--help- prints the usage/help message, and exit.
--quiet- do not write anything to standard output. Exit with 0 if no problems arise, otherwise 1.
--verbose- displays detailed tracing information. when this option is not specified.
Example: Find files to download from a repository
We can use this command for getting information about a repository before using
other commands like
dvc get or
dvc import to reuse any file or directory
found in it. This includes files (or directories) tracked by DVC or by Git:
$ dvc list https://github.com/iterative/example-get-started .dvcignore .gitignore README.md data dvc.lock dvc.yaml model.pkl params.yaml prc.json scores.json src
If you open the
project's page, you will see a similar list but not the
model.pkl file. It's
tracked by DVC and not visible to Git. It's exported in the
dvc.yaml file as
an output of the
train stage (in the
We can now, for example, download the model file with:
$ dvc get https://github.com/iterative/example-get-started model.pkl
Example: List all files in a data registry
Let's imagine a DVC repo used as a
data registry, structured with
different datasets in separate directories. We can do this recursively, using
$ dvc list -R https://github.com/iterative/dataset-registry .gitignore README.md get-started/.gitignore get-started/data.xml get-started/data.xml.dvc images/.gitignore images/dvc-logo-outlines.png images/dvc-logo-outlines.png.dvc images/owl_sticker.png ...
Example: Create an archive of your DVC project
Just like you can use
git archive to make a quick bundle (ZIP) file of the
dvc list can be easily complemented with simple archive tools to
bundle the current data files in the project.
For example, here's a TAR archive of the entire workspace (Linux/GNU):
$ dvc list . -R | tar -cvf project.tar
Or separate ZIP archives of code and DVC-tracked data (POSIX terminal with
$ git archive -o code.zip HEAD $ dvc list . -R --dvc-only | zip -@ data.zip
ZIP alternative for POSIX on Windows (Python installed):
$ dvc list . -R --dvc-only | xargs python -m zipfile -c data.zip