Unsent WU
18 Apr 2016 13:29:42 UTC
Topic 198573
(moderation:
I wonder if anyone can tell me why a recent WU that I have processed has not been resent to another co worker? WU 243672480 was sent to me on the 8th April and to another on the 9th. On the 13th my co worker had an error whilst computing and since then the WU has not been resent although a new task number was generated.
Regards Peter
Unsent WU
)
This is a side effect of how the scheduler here at Einstein works.
For every Gravity Wave workunit you need a set of large data files, this set of files can be used by several hundred workunits and the scheduler tries to send workunits to hosts that already have the required data files or that only has to download a few files to complete the set required.
This might sometimes lead to workunits remaining unsent for several days as the scheduler waits for a suitable host to come as for work. If enough time passes the workunit will be sent out to the next host asking for this kind of work even though that host might not have any data files required and so will have to download the complete set.
So you just need to have a bit more patience and the workunit will be sent out again.
There was a problem with the
)
There was a problem with the locality scheduler that assigns work to hosts. Because of that the number of unsent tasks was increasing over the last 2 weeks. It took some time to understand what was happening but we finally fixed this last Friday. Since then the locality scheduler is assigning the already generated but still unsent tasks to requesting hosts instead of generating new work.
It took three days to go from 210k unsent to 150k now. So it will take until next week for the queue to empty.
Edit: What Holmis said is correct. But in this special case until last Friday the locality scheduler was not behaving like it should.
Thank you both for the
)
Thank you both for the explanation.
Pete