Why does a task show 99% progress although time keeps going?
9 Sep 2023 20:57:09 UTC
Topic 230064
(moderation:
Please forgive me if this has been asked before, but most of my searches return with "You don't have permission to access this resource." Thus, searching hasn't given me anything.
Regarding my question. A task can stop showing progress after 40 minutes and at 99%, and then go on and spend 20 additional minutes before it's finished. Can't time estimates be closer than that?
Thanks,
Bill
Hi Bill, Not sure exactly
)
Hi Bill,
Not sure exactly what tasks you are talking about, but I'll try to answer for you.
Einstein has tasks that require quite a bit of GPU & CPU time to complete. Even though the tasks in question are intended for GPUs, the CPU also has the need to do more double-precision calculations which is why your tasks stop showing progress. Leave them be, and they will eventually finish.
And no, the time estimates can't be closer than those already given, especially if you are being finicky with your projects because of this. Let them be, they will finish, and if left alone the time estimates may get closer to reality.
As for searching, if you are using the search function in Einstein you need to place the " + " sign (without the quotes) instead of a space between the words. A search cannot have any spaces within it.
Proud member of the Old Farts Association
wlathan wrote:... A task can
)
You have 8 computers listed on your account so I didn't try to find which particular one you were referring to.
My guess is you are describing the fairly new GW GPU search because (from memory) the last time I was running GW GPU tasks, the main calculations ended at 99% and a relatively short period of no apparent activity followed before suddenly jumping to 100%. That 1% allocation was reasonable for that former search and it's referred to as the 'followup' stage where the full list of candidate signals from the first 99% are trimmed to a shorter list (used to be the 10 best candidates) and re-evaluated in higher precision.
The current run looks for a lot more candidates in a lot more detail so this is probably why the followup stage is so much longer. I don't have GPUs with enough VRAM to run this search so I can't say if 20 mins for the followup stage is reasonable or not :-).
The latest GW GPU search has caused some problems but I haven't really been following it closely enough to know if the 40:20 time ratio you mention is to be expected. Have you perused the full announcement thread in Tech News? Maybe other users may have posted their experiences and maybe some actual times.
Do you run concurrent GPU tasks and do you run CPU tasks as well as GPU tasks. Maybe there is a bit of a 'fight' for CPU attention when the followup stage is underway. If you are, you could test for faster completion by running fewer CPU tasks, just as a trial. If you are getting variable times, that would be an indicator of the machine being overloaded with competing work.
Cheers,
Gary.
Many thanks for your
)
Many thanks for your responses. I will check out anything you suggested. And, thanks for the clue on the search.
Bill