Here we provide help for some of the problems that DVC user might stumble upon.
A known problem some users run into with the
dvc fetch and
dvc push commands is
[Errno 24] Too many open files (most common for S3
remotes on MacOS). The more
--jobs specified, the more file descriptors need
to be open on the host file system for each download thread, and the limit may
be reached, causing this error.
To solve this, it's often possible to increase the open file descriptors limit,
ulimit on UNIX-like system (for example
ulimit -n 1024), or
increasing Handles limit
on Windows. Otherwise, please try using a lower