Large number of WU stays in pending credits
12 Jul 2008 5:54:52 UTC
Topic 193759
(moderation:
The last hours I don't receive any new points. all result stay in pending. I have looked to some of the work units an I se that there are two result waiting for validation for this wu. Is ther something wrong with the validation.
Large number of WU stays in pending credits
)
If you check the server status page, you can see on the far right hand side that there seems to be an unusually large number of results that are waiting for validation. As it is flagged in red, I assume that it is unusual and that the Admins would be aware of this or at least some warning would have been issued to at least one of them - he might be still engaged in or sleeping off the effects of a wild Friday night party so maybe there won't be any immediate action :-).
The validator process itself is showing as "running" so obviously there is something unusual with the database that is preventing the validation queue from draining. The queue is currently 15,740 and the status page is updated every twenty minutes. It will be interesting to see what the number is after the next page update. The current page is timed at 6:20AM UTC.
Things like this can happen from time to time so nothing to worry about (yet) :-).
EDIT: The update came after 25mins at 6:45AM UTC and shows the queue is now at 16,264. Doesn't look like too much validation is going on at the moment.
Cheers,
Gary.
Apparently the validator took
)
Apparently the validator took itself a time-out but was sent back to work again, catching up on the backlog. All should be back to normal pretty soon.
CU
Bikeman
RE: Apparently the
)
I think the problem may be back. The number "waiting for validation" was declining for a while (from around 20,000 to around 16,000) - but its going back up again (18,079 right now).
The really odd part is that
)
The really odd part is that this acts like exactly the same problem that completely brought down Cosmology@Home and nearly ground SETI@Home to a halt as well within the last few days.
I've never been one to be a conspiracy nut, but it's sure something that makes you go "Hmmmmmm".
RE: The really odd part is
)
I had already been thinking the same thing with the issues with Cosmology and SETI, but now with it happening here too? Server-side code change that was installed recently???
Now we're up to: waiting for
)
Now we're up to: waiting for validation 32,581
The number is steadily growing and based on what I've seen from other projects, it's only a matter of time until the database is dealing with so many WU's that can't be purged out that it goes into a complete meltdown.
I sincerely hope I'm not just crying wolf here, but I hope there's some way for Bernd or someone else who can deal with the issue to be aware and try to correct it before it gets worse. :\\
RE: ... I hope there's some
)
David Hammer restarted the validator yesterday after being alerted by Bikeman. He has been alerted again that the problem has returned. I've also drawn the "conspiracy theory" to his attention so he can at least be warned that this might not be an isolated, random glitch.
Cheers,
Gary.
RE: RE: ... I hope
)
Thanks much Gary.
I know Bikeman reported that restarting the validator seemed to correct things here at E@H and I've not been able to follow things with my own WUs enough to know if any are being validated or if it's at a complete standstill. I know that the admin at Cosmology@Home reported (if I'm remembering correctly) a large amount of disk I/O that he had no explanation for before they went totally south. The folks at SETI have only described it as "database problems", but like E@H and Cosmo, the problem manifested as a large number of WUs waiting for validation. They seem to have rectified it, though I've seen no explanation of how as of yet. I don't believe it was as simple as just restarting the validator, so there's always a chance that these are just 3 unrelated problems that just happened to create similar symptoms.
Good luck E@H gurus! :)
Why are there so called
)
Why are there so called overreplications (Zuvielfachauslieferungen ^^)
> http://einsteinathome.org/workunit/41168115
> http://einsteinathome.org/workunit/41165172
> http://einsteinathome.org/workunit/41162167
> http://einsteinathome.org/workunit/41157579
Two results are reported in, one other was replicated without need.
One would not get any credit albeit the work is done and could be used.
Just curious...
Strange indeed. As to the
)
Strange indeed.
As to the vaidation problems mentioned above for other BOINC projects, are there summaries available on the Web that describe what went wrong?
TIA
Bikeman