When I got to the last session on model textual, the model went through 5 epochs then threw this error:
IOError: [Errno 24] Too many open files
I went ahead trying to change $ulimit -n but realized the easiest way is to just change numb_workers to 4. It's arbitrary but someone suggested 4*num of GPU is a good approximation for num_workers.
It's not a specific issue per se but I think it may be beneficial for people to know this is one of the nuances in building ML pipeline which is not necessarily apparent.
When I got to the last session on model textual, the model went through 5 epochs then threw this error:
IOError: [Errno 24] Too many open files
I went ahead trying to change $ulimit -n but realized the easiest way is to just change numb_workers to 4. It's arbitrary but someone suggested 4*num of GPU is a good approximation for num_workers.
It's not a specific issue per se but I think it may be beneficial for people to know this is one of the nuances in building ML pipeline which is not necessarily apparent.