Edit on GitHub


Here we provide help for some of the problems that DVC user might stumble upon.

Too many open files error

A known problem some users run into with the dvc pull, dvc fetch and dvc push commands is [Errno 24] Too many open files (most common for S3 remotes on MacOS). The more --jobs specified, the more file descriptors need to be open on the host file system for each download thread, and the limit may be reached, causing this error.

To solve this, it's often possible to increase the open file descriptors limit, with ulimit on UNIX-like system (for example ulimit -n 1024), or increasing Handles limit on Windows. Otherwise, please try using a lower JOBS value.


๐Ÿ› Found an issue? Let us know! Or fix it:

Edit on GitHub

โ“ Have a question? Join our chat, we will help you:

Discord Chat