Edit on GitHub

Setup a Google Drive DVC Remote

Follow this guide to setup Google Drive as your DVC project's remote storage.

Configure a Google Cloud project

DVC uses the Google Drive API to connect to your Google Drive. This requires a Google Cloud project that allows Drive API connections, and its OAuth credentials (client ID and client secret).

  1. Sign into the Google API Console.

    Double check you're using the intended Google account (upper-right corner).

  2. Select or Create a project for DVC remote connections.

  3. Enable the Drive API from the APIs & Services Dashboard (left sidebar), click on + ENABLE APIS AND SERVICES. Find and select the "Google Drive API" in the API Library, and click on the ENABLE button.

  4. Go back to APIs & Services in the left sidebar, and select OAuth consent screen. Chose a User Type and click CREATE. On the next screen, enter an Application name e.g. "DVC remote storage", and click the Save (scroll to bottom).

  5. From the left sidebar, select Credentials, and click the Create credentials dropdown to select OAuth client ID. Chose Other and click Create to proceed with a default client name.

  6. The newly generated client ID and client secret should be shown to you now, and you can always come back to Credentials to fetch them.

⚠️ It should be safe to share client ID and client secret among your team. These credentials are only used to generate the authorization URL DVC will later prompt to visit in order to connect to the Google Drive.

Note that Google Drive API has usage limits/quotas per credentials in a project (which you can review in the OAuth consent screen). Please keep this in mind when sharing them, or you may exceed the limits.

URL format for Google Drive remotes

A GDrive DVC remote URLs is needed for the dvc remote add command. It can be constructed with a base, and an optional path to a folder i.e. gdrive://<base>/path/to/folder. The base can be one of:

  1. root - indicates your topmost Google Drive directory.
$ dvc remote add -d mygdroot gdrive://root
$ dvc remote add mygdfolder gdrive://root/path/to/folder
  1. Google Drive Folder ID

To obtain the unique Folder ID in question, navigate into that folder in your web browser, and find it in the address bar. For example, for https://drive.google.com/drive/folders/0AIac4JZqHhKmUk9PDA:

$ dvc remote add mygdfolder gdrive://0AIac4JZqHhKmUk9PDA
$ dvc remote add mygdsubfolder \
                 gdrive://0AIac4JZqHhKmUk9PDA/sub/path

Shared drives should be referenced by Folder ID in the same way.

  1. appDataFolder - special application-specific data folder only accessible by your application and hidden from the user.
$ dvc remote add mygdappata gdrive://appDataFolder

Configure the GDrive remote

Use the dvc remote modify command to set the credentials for each gdrive:// remote, for example:

$ dvc remote add -d mygdfolder gdrive://root/path/to/folder
$ dvc remote modify mygdfolder gdrive_client_id <client ID>
$ dvc remote modify mygdfolder gdrive_client_secret <client secret>

Authorization

On the first usage of a GDrive DVC remote, you'll be prompted to visit a web page to generate an access token, which will require you to sign into the corresponding Google account. The sign-in process will guide you through granting the necessary access permissions. On success, the token data will be cached in a Git-ignored directory located in .dvc/tmp/gdrive-user-credentials.json.

⚠️ In order to prevent unauthorized access to your Google Drive, do not share access token data with others. Each team member should generate their own tokens.

Please note our Privacy Policy (Google APIs).

Content


Configure a Google Cloud projectURL format for Google Drive remotesConfigure the GDrive remoteAuthorization

🐛 Found an issue? Let us know! Or fix it:

Edit on GitHub

Have a question? Join our chat, we will help you:

Discord Chat