Skip to content
Edit on GitHub

list-url

Aliased to dvc ls-url

List contents from a supported URL (for example s3://, ssh://, and other protocols).

Useful to find data to dvc get-url or dvc import-url.

Synopsis

usage: dvc list-url [-h] [-q | -v] [-R] url

positional arguments:
  url              (See supported URLs in the description)

Description

Lists files and directories from an external location. dvc list-url provides a uniform interface to browse the contents of an external location using any protocol that is understood by dvc get-url or dvc import-url. For example, it is roughly equivalent to aws s3 ls when using the s3:// protocol, or ssh user@host ls -a when using ssh://.

The url argument specifies the location of the data to be listed. It supports several kinds of external data sources:

TypeDescriptionurl format example
s3Amazon S3s3://bucket/data
azureMicrosoft Azure Blob Storageazure://container/data
gsGoogle Cloud Storagegs://bucket/data
sshSSH serverssh://user@example.com/path/to/data
localLocal path/path/to/local/data

If you installed DVC via pip and plan to access cloud services as external data sources, you might need to install these optional dependencies: [s3], [azure], [gs], [oss], [ssh]. Alternatively, use [all] to include them all. The command should look like this: pip install "dvc[s3]". (This example installs boto3 library along with DVC to support S3 storage.)

Only the root directory is listed by default, but the -R option can be used to list files recursively.

Options

  • -R, --recursive - recursively list files in all subdirectories.

  • -h, --help - prints the usage/help message, and exit.

  • -q, --quiet - do not write anything to standard output. Exit with 0 if no problems arise, otherwise a non-zero value.

  • -v, --verbose - displays detailed tracing information.

Example: Amazon S3

This command will list objects and common prefixes under the specified path:

$ dvc list-url s3://bucket/path

DVC expects that AWS CLI is already configured. DVC will use the AWS credentials file to access S3.

Example: SSH

$ dvc list-url ssh://user@example.com/path/to/data

Using default SSH credentials, the above command lists files and directories inside data.

Example: local file system

$ dvc list-url /local/path/to/data

The above command will list the /local/path/to/data directory.

Content

🐛 Found an issue? Let us know! Or fix it:

Edit on GitHub

Have a question? Join our chat, we will help you:

Discord Chat