add thread queue for large sims#1600
Conversation
| # number of jobs exceeds available memory. | ||
| # If running jobs with a huge population, set the queue to a reasonable size | ||
| # like 1000000 to avoid memory issues. | ||
| generate.thread_queue_size = -1 |
There was a problem hiding this comment.
Looks like you are referring to maximum capacity(or max thread queue size) here? generate.thread_max_queue_size would be a better name to use here.
| // functionally this means that once the queue is full new tasks will | ||
| // be put into the sequence as fast as current tasks are completed | ||
| // by worker threads | ||
| executor.getQueue().put(rTask); |
There was a problem hiding this comment.
This might solve the memory issue, but might block the main thread indefinitely(a possible deadlock scenario).
Have you instead, considered using CallerRunsPolicy - https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ThreadPoolExecutor.CallerRunsPolicy.html
Another possible way would be to use offer() (with a timeout if you want)- Essentially a non-blocking operation instead of a direct put().
There was a problem hiding this comment.
I'm not too worried about a deadlock but a CallerRunsPolicy would be effective here, so I'll implement it.
dehall
left a comment
dehall
left a comment
There was a problem hiding this comment.
I tested this out and it seems to work well. But I actually don't see any obvious negative performance impact from setting it to a small number, even 1 seems to be just fine. So I'm good with this as-is but we could also consider a different default so that the default behavior is the bounded queue
3 participants
Fixes #1515 by adding a config option to limit the number of tasks allowed in the thread queue so that memory doesn't fill up with task objects waiting to be run by a thread. In large sims, this could cause the program to hang when the amount of tasks in the unbounded queue exceeded the available memory.
Setting
generate.thread_queue_sizeto any non-zero number will cap the queue at that number of tasks. The only difference between these two modes is that the unbounded queue might be slightly faster and have a little less overhead compared to the bounded one, but the bounded one won't freeze up. This would really only have a negative effect if the number of threads or available cpu fluctuates over the length of the sim. Otherwise these two modes should be the same.