How to Run DVC on Windows
Different issues can arise when running DVC on Microsoft Windows, mainly involving system performance. Some, for example, have to do with NTFS file system characteristics and Windows built-in security mechanisms. Below are some workarounds that can help avoid these potential problems:
Did you know that DVC is available for Microsoft Visual Studio Code? More details here!
POSIX-like command line experience
The regular Command Prompt (
cmd) in Windows will most likely not help you use
DVC effectively, nor help you follow the examples in our docs. Here are some
- WSL 2 with
Windows Terminal supports the
most CLI features (e.g.
\line continuation). It has good performance (and can even access GPUs).
- The full Cmder console emulator is another good option. It combines several useful tools like ConEmu terminal and Git for Windows (Git Bash), among other shells.
- Anaconda Prompt is also popular.
- Install an actual Linux distro (e.g. Ubuntu) on a virtual machine, or in an HD partition (dual boot).
Enable symbolic links
Symlinks are one of the possible file link types that DVC can use for
purposes. They're available on Windows, but the Create symbolic links user
privilege is needed. It's granted to the Administrators group by default, so
dvc in an admin terminal is a good option for occasional usage. For
regular users, it can be granted using the Local policy settings.
This is done automatically by DVC's Windows installer,
but you may want to
do it manually
after any other installation method (
Whitelist in Windows Security
Windows 10 includes the Windows Security antivirus. If user wants to avoid antivirus scans on specific folders or files to improve the performance, then whitelist them in Windows Security as per this guide. For example, we can whitelist DVC binary files on Windows to speed up the processes.
Enable long folder/file paths
dvc pull may fail in some cases when the folder path is longer than 260
characters. In the Windows API, the maximum allowed length for a path is 260
characters. If required, the user can explicitly enable long paths by following
Fix or disable Search Indexing
Search Indexing can also slow down file I/O operations on Windows. Try fixing or disabling this feature if you don't need it.
Disable short-file name generation
With NTFS, users may want to disable
8dot3 as per
to disable the short-file name generation. It is important to do so for better
performance when the user has over 300K files in a single directory.
Avoid directories with large number of files
The performance of NTFS degrades while handling large volumes of files in a directory, as explained in this issue.
Enabling paging with
By default, DVC tries to use Less
as pager for the output of
dvc dag. Windows doesn't have the
available however. Fortunately, there is a easy way of installing it via
Chocolatey. After installing Chocolatey, run:
$ choco install less
less can be installed in other ways, just make sure it's available in the
command line environment where you run
dvc. (This usually means adding the
less is installed to the
PATH environment variable.)