List repository contents, including files, models, and directories tracked by DVC (as outputs) and by Git.
usage: dvc list [-h] [-q | -v] [-R] [--dvc-only] [--rev <commit>]
url [path]
positional arguments:
url Location of DVC or Git repository to list from
path Path to a file or directory within the repository
A side-effect of DVC is that it hides actual data paths, by effectively
replacing files and directories with DVC files. So you don't see
data files/dirs when you browse a DVC repository on Git hosting
(e.g. GitHub), you just see the dvc.yaml
and .dvc
files. This can make it
hard to navigate the project, for example to find files or directories for use
with dvc get
, dvc import
, or dvc.api
functions.
This command produces a view of a DVC repository, as if files and directories
tracked by DVC were found directly in the Git repo. Its output is equivalent to
cloning the repo and pulling the data (except
that nothing is downloaded by dvc list
), like this:
$ git clone <url> example
$ cd example
$ dvc pull
$ ls <path>
Only the root directory is listed by default, but the -R
option can be used to
list files recursively.
The url
argument specifies the address of the DVC or Git repository containing
the data source. Both HTTP and SSH protocols are supported (e.g.
[user@]server:project.git
). url
can also be a local file system path
(including the current project e.g. .
).
The optional path
argument is used to specify a directory to list within the
source repository at url
(including paths inside tracked directories). It's
similar to providing a path to list to commands such as ls
or aws s3 ls
.
Please note that dvc list
doesn't check whether the listed data (tracked by
DVC) actually exists in remote storage, so it's not guaranteed whether it can be
accessed with dvc get
, dvc import
, or dvc.api
.
-R
, --recursive
- recursively prints contents of all subdirectories.--dvc-only
- show only DVC-tracked files and directories
(outputs).--rev <commit>
- commit hash, branch or tag name, etc. (any
Git revision) of the repository to list
content for. The latest commit in master
(tip of the default branch) is used
by default when this option is not specified.-h
, --help
- prints the usage/help message, and exit.-q
, --quiet
- do not write anything to standard output. Exit with 0 if no
problems arise, otherwise 1.-v
, --verbose
- displays detailed tracing information. when this option is
not specified.We can use this command for getting information about a repository before using
other commands like dvc get
or dvc import
to reuse any file or directory
found in it. This includes files (or directories) tracked by DVC or by Git:
$ dvc list https://github.com/iterative/example-get-started
.dvcignore
.gitignore
README.md
data
dvc.lock
dvc.yaml
model.pkl
params.yaml
prc.json
scores.json
src
If you open the
example-get-started
project's page, you will see a similar list but the model.pkl
file. It's
tracked by DVC and not visible to Git. It's exported in the dvc.yaml
file as
an output of the train
stage (in the outs
field).
We can now, for example, download the model file with:
$ dvc get https://github.com/iterative/example-get-started model.pkl
Let's imagine a DVC repo used as a
data registry, structured
with different datasets in separate directories. We can do this recursively,
using -R
option:
$ dvc list -R https://github.com/iterative/dataset-registry
.gitignore
README.md
get-started/.gitignore
get-started/data.xml
get-started/data.xml.dvc
images/.gitignore
images/dvc-logo-outlines.png
images/dvc-logo-outlines.png.dvc
images/owl_sticker.png
...
Just like you can use git archive
to make a quick bundle (ZIP) file of the
current code, dvc list
can be easily complemented with simple archive tools to
bundle the current data files in the project.
For example, here's a TAR archive of the entire workspace (Linux/GNU):
$ dvc list . -R | tar -cvf project.tar
Or separate ZIP archives of code and DVC-tracked data (POSIX terminal with
zip
):
$ git archive -o code.zip HEAD
$ dvc list . -R --dvc-only | zip -@ data.zip
ZIP alternative for POSIX on Windows (Python installed):
$ dvc list . -R --dvc-only | xargs python -m zipfile -c data.zip