Gary Roberts has offered the "it had more deadline pressure" explanation. I think there MAY be another explanation for why Gerry's task Z is crunching when his task A is not. The explanation is very simple... maybe it just looks like task A was overlooked.... perhaps A was crunching just a few minutes before Gerry took the screenshot.
That is possible but I would think less likely than the simpler explanation of BOINC doing the tasks it considers to be most at risk.
Since we don't have Gerry's logs to look at there's no way we can determine what actually happened.
Quote:
When I look at all the WCG tasks more closely, things are a bit more complicated than at first glance. Firstly, I now notice that there are a bunch of tasks with a deadline timestamp of 7:28:41AM which were all obviously downloaded in a single batch. However some are due on the 22nd and some are due exactly 1 day later on the 23rd. I know nothing about WCG but obviously they must have at least 2 different deadlines, one exactly 1 day longer than the other. I had initially overlooked this fact.
Secondly, task A has a deadline of 7:08:08PM. I had initially mistaken this for "AM" ie 20mins earlier than X but in fact it's half a day later than X or Y or Z so it's little surprise that these three are being done first since they really are under a lot more deadline pressure than A.
I think I made exactly the same mistake and I agree that things are not as simple as they might appear at first glance. Now I have to wonder why A started before X, Y and Z. I recall that WCG has several projects, each with a different application. Since the Application Name column is shrunk in Gerry's screeshot, it's impossible to tell but A, X, Y and Z may be different apps that start with the letter H. I believe they have/had apps named something like "Human_proteome...", "Help_fight_aids..." and "Healthy_rice...". We could be looking at more than 1 project under the WCG banner. Does the scheduler work on project URLs or app names? I think it's the latter but I'm not sure.
I think if we've learned anything from Gerry's screenshot, it's that it takes a lot more than just 1 screenshot to get an accurate picture of what's happening and why. Snippets from Gerry's logs would have helped though I'm not suggesting I have the time to pour through them.
When BOINC gets too much work for a given project, then does it not still calculate that it is indeed ovecommitted?
Of course it does but think about how inaccurate and unreliable that calculation can be. GIGO takes precedence over EVERYTHING and screws up the best laid plans and algorithms. Some projects feed BOINC garbage flops estimates. Yes, the DCF attempts to correct the garbage but without accurate duration estimates scheduling is bound to be faulty.
Also, it doesn't matter how perfect flops estimates and everything else is, scheduling and work fetch still depends on your usage pattern. You can get the situation where BOINC wasn't over committed until you started playing Crysis 10 hours a day instead of just 1 hour per day. In such a situation, BOINC will eventually adjust to the fact that it gets fewer CPU cycles but that takes time. Hmmm, maybe BOINC should be able to abort tasks it has not started when it dicovers it is over committed? That would be better than returning tasks late.
After a L-O-N-G read of this thread I'd like to make a few comments. I won't be quoting anyone, but these comments are relavent to other's comments.
Firstly, on the issue of task switching with only a little time remaining. It's my understanding that newer versions of Boinc will wait for a checkpoint before switching. If that is correct then at least no work is lost. I personally wish Boinc would finish such tasks rather than leave them preempted in memory. After all, the swapping to the swap file if needed will cause a small performance hit for my host while it happens.
I can only think of one reason for having the task switching. If a WU with a wrong duration estimate is started earlier by task switching than if it waited till another task finished, then Boinc will become aware of the problem earlier, and be able to give it more crunch time to finish by deadline.
As for starting a task for a project when one has already started, this was definitely seen in earlier versions of Boinc (not sure when it changed), but I haven't seen it for a long time now. The postings about deadline pressure would seem to explain Gerry's specific issue.
> have concluded (after several examples) that Einstein is periodically >consuming almost 90% of by CPU WHILE I"M WORKING - according to Task Manager. >This in spite of the fact that the total allocated for Einstein and SETI is >80%. I can't get any work done when this happens. I don't care if this is an >Einstein problem or a Windows Vista problem. I have decided to discontinue >working with Einstein for this reason.
>
>Please tell me how to dis-engage.
>
>Thanks - Don
----------------------------------------
It's not a problem with either, Don.
I think something that Don, and others, often miss is the word 'TIME'. That setting sets the amound of time the CPU works on the task. But, many interpret it as a simple throttle setting. They expect when they set it to 80%, that the cpu will run at only 80% of top speed all the time. That's not correct. What it will do is run the task at full speed 80% of the time, then stop altoghether the remaining 20%. If you watch in your task manager where it graphs the work the cpus are doing, you'll see this happening. There will be periods where the cpus are cranking, with regular periods of time where they are almost idle. So, yes, no matter how low you set that ratio, if you take a snapshot at the right instant, you could catch your cpu's running full load. It's doing exactly what it's programmed to.
Now, having said that, I too would rather have a way to program BOINC to use a more throttle-like setting as many expect. I think that would be better for heat, cpu life, etc to have it run at a percentage of it's capability at all times rather than simply bouncing between full tilt and idle constantly. But, having never tried to program an application that way, I'm not sure if that's even possible short of a crude way of crippling it.
Anyway, just thought that should be clarified that 'TIME' is the keyword when setting that preference. There's nothing wrong with any particular project nor the OS. BOINC is behaving exactly as it was programmed to. If someone doesn't want it to run the CPU's hard while he's working, then set the preferences to only run when idle.
RE: RE: Gary Roberts has
)
Since we don't have Gerry's logs to look at there's no way we can determine what actually happened.
I think I made exactly the same mistake and I agree that things are not as simple as they might appear at first glance. Now I have to wonder why A started before X, Y and Z. I recall that WCG has several projects, each with a different application. Since the Application Name column is shrunk in Gerry's screeshot, it's impossible to tell but A, X, Y and Z may be different apps that start with the letter H. I believe they have/had apps named something like "Human_proteome...", "Help_fight_aids..." and "Healthy_rice...". We could be looking at more than 1 project under the WCG banner. Does the scheduler work on project URLs or app names? I think it's the latter but I'm not sure.
I think if we've learned anything from Gerry's screenshot, it's that it takes a lot more than just 1 screenshot to get an accurate picture of what's happening and why. Snippets from Gerry's logs would have helped though I'm not suggesting I have the time to pour through them.
BOINC FAQ Service
Official BOINC wiki
Installing BOINC on Linux
RE: When BOINC gets too
)
Of course it does but think about how inaccurate and unreliable that calculation can be. GIGO takes precedence over EVERYTHING and screws up the best laid plans and algorithms. Some projects feed BOINC garbage flops estimates. Yes, the DCF attempts to correct the garbage but without accurate duration estimates scheduling is bound to be faulty.
Also, it doesn't matter how perfect flops estimates and everything else is, scheduling and work fetch still depends on your usage pattern. You can get the situation where BOINC wasn't over committed until you started playing Crysis 10 hours a day instead of just 1 hour per day. In such a situation, BOINC will eventually adjust to the fact that it gets fewer CPU cycles but that takes time. Hmmm, maybe BOINC should be able to abort tasks it has not started when it dicovers it is over committed? That would be better than returning tasks late.
BOINC FAQ Service
Official BOINC wiki
Installing BOINC on Linux
After a L-O-N-G read of this
)
After a L-O-N-G read of this thread I'd like to make a few comments. I won't be quoting anyone, but these comments are relavent to other's comments.
Firstly, on the issue of task switching with only a little time remaining. It's my understanding that newer versions of Boinc will wait for a checkpoint before switching. If that is correct then at least no work is lost. I personally wish Boinc would finish such tasks rather than leave them preempted in memory. After all, the swapping to the swap file if needed will cause a small performance hit for my host while it happens.
I can only think of one reason for having the task switching. If a WU with a wrong duration estimate is started earlier by task switching than if it waited till another task finished, then Boinc will become aware of the problem earlier, and be able to give it more crunch time to finish by deadline.
As for starting a task for a project when one has already started, this was definitely seen in earlier versions of Boinc (not sure when it changed), but I haven't seen it for a long time now. The postings about deadline pressure would seem to explain Gerry's specific issue.
I see JM7 has made a comment
)
I see JM7 has made a comment about the new scheduler and debt calculation in V6.6.x on Seti's number crunching board.
Post 866922
> have concluded (after
)
> have concluded (after several examples) that Einstein is periodically >consuming almost 90% of by CPU WHILE I"M WORKING - according to Task Manager. >This in spite of the fact that the total allocated for Einstein and SETI is >80%. I can't get any work done when this happens. I don't care if this is an >Einstein problem or a Windows Vista problem. I have decided to discontinue >working with Einstein for this reason.
>
>Please tell me how to dis-engage.
>
>Thanks - Don
----------------------------------------
It's not a problem with either, Don.
I think something that Don, and others, often miss is the word 'TIME'. That setting sets the amound of time the CPU works on the task. But, many interpret it as a simple throttle setting. They expect when they set it to 80%, that the cpu will run at only 80% of top speed all the time. That's not correct. What it will do is run the task at full speed 80% of the time, then stop altoghether the remaining 20%. If you watch in your task manager where it graphs the work the cpus are doing, you'll see this happening. There will be periods where the cpus are cranking, with regular periods of time where they are almost idle. So, yes, no matter how low you set that ratio, if you take a snapshot at the right instant, you could catch your cpu's running full load. It's doing exactly what it's programmed to.
Now, having said that, I too would rather have a way to program BOINC to use a more throttle-like setting as many expect. I think that would be better for heat, cpu life, etc to have it run at a percentage of it's capability at all times rather than simply bouncing between full tilt and idle constantly. But, having never tried to program an application that way, I'm not sure if that's even possible short of a crude way of crippling it.
Anyway, just thought that should be clarified that 'TIME' is the keyword when setting that preference. There's nothing wrong with any particular project nor the OS. BOINC is behaving exactly as it was programmed to. If someone doesn't want it to run the CPU's hard while he's working, then set the preferences to only run when idle.