Computer is connected to the Internet about every 0.5 days
Maintain enough work for an additional 2 days, (since turned that down to 0.5 also).
And what happened? More Malaria WU's came in - I give up...!!
Unfortunately Einstein and all the other are having to wait as all the Malaria ones have a deadline of yesterday if not before, so are running at high priority and when finished yet another takes over pushing anything beyond a 48 hour deadline back.
As I already have a stack of both Malaria and Einstein I have told those two not to accept new WU's until I tell them to.
Sorry, too late to edit the above...it seems to have calmed down now. I trust you meant adjust both those settings.
On the question of cache size in isolation, it is best to keep the CI (Connect to Internet) setting very low (if you do have a permanent connection) and to bump up the ED (Extra Days) setting to increase your cache size. This is primarily because BOINC (traditionally - don't know if this has changed in more recent versions) uses the setting for two purposes. Firstly it's to ensure enough work on hand to outlast the period of non-connection and secondly it's in case a finished result can't be returned until a connection is re-established. So BOINC has to plan for a possible delay of this amount between when work is completed and when the work can be returned. Read the hint on the website preference page carefully (leave at zero if always connected).
This could easily be the reason for the sudden download of extra Malaria tasks. If you previously had a CI of 2 days, BOINC would be limiting the work on hand so that any task could wait up to a further 2 days after completion before being returned and still not miss the deadline. If you use ED rather than CI to control your cache size, BOINC doesn't have to plan for any delay in returning work and so can keep some more work on hand.
The other bad thing about a large CI is that BOINC is more likely to find tasks that are possible deadline stress candidates and so will be forced into HP (High Priority) mode sooner and more often than it otherwise would. And of course if you're setting and unsetting NNT in the middle of all this, BOINC will have extra problems managing debt effectively.
You need to think carefully about a number of your preference settings if you wish to make it a bit easier for BOINC. Here are some notes that might assist.
* Think carefully about how many projects you really need to support. Ten is not really too many but the more you choose, the harder it will be to maintain any sort of equilibrium.
* When you have a lot of projects, keep the ED cache setting to a sensible value otherwise you may experience over-fetch problems with several of the projects. With 10 projects, you will get more predictable behaviour if CI+ED is less than 1 day. I've never tried that many projects but if I did I would use 0.01+0.25 to start with and watch the outcome for sufficient time (maybe several days) to see how BOINC handles things. If it isn't giving enough work, increase the ED in small steps (0.25->0.5, etc.) and wait to see what happens. Resist the temptation to fiddle until you are sure that the previous change is OK and you can go a bit more.
* When setting resource shares (RS), it will be easier for BOINC if each project has roughly the same share. Setting highly unequal shares is a recipe for disaster. The number set for any one project is relatively unimportant. What is important is that value in relation to the sum of ALL values. In your case you are fine. From the messages log snippet you posted, all values are either 100 or 200 with a total of 1500. That means than any project has either a 6.7% or 13.3% share of your system. It would be a problem if you set a project or two to a low RS like 10. That would mean that such a project would be entitled to less than 1% of your machine's time, approximately 1 hour per CPU core per week, if your machine was running 24/7. In your case, your preferences as shown in a previous message, limit your machine to crunching between 7.00am and 9.00pm - 14 hours per day if I'm reading things correctly. In that case, a RS of 10 would equate to only 0.5hrs per cpu core per week. If you can follow that, it's pretty easy to understand the commonly seen pattern - download some tasks, have them sit around for quite a while until they get all done in panic mode and then no more tasks for that project for a very long time. This isn't your problem because you have reasonable RS values. I'm taking advantage of this reply to talk to others who may not have set reasonable RS values.
* Don't be tempted to use a low RS if your object is to achieve a 'backup project'. You will not be able to get work from that project when you really need it. If you don't understand why, go back and think about the previous point. Use the zero resource share setting for a genuine backup project.
* When making changes to your preferences, don't make big changes and don't make multiple changes all at once. If making preference changes on the website, click on 'update' in BOINC Manager to apply the change immediately. Always give BOINC sufficient time to show the full effect of any change.
* If you can't be patient (that describes me) do some research on the structure of your state file (client_state.xml) and learn how to safely edit it. You can fix a lot of things very quickly (particularly to do with debt problems and incorrect DCF - Duration Correction Factor - once you know what you are doing) if you really understand what is in that file. If you don't fully understand that file, just learn to be patient :-).
* When digesting advice given in the forums, make sure you realise that we are all fallible and can come up with incorrect advice even though our intentions were good. The quality of the advice is often dependent on the quality of the problem description. The bits not described because they were thought to be unimportant, can often lead to incorrect advice being given. These comments are not directed at the OP because a lot of good info was contained in the previous posts - particularly message logs, preference settings and debt values.
* Finally, if you are lucky enough to have Richard Haselgrove posting about your problem, read what he says very carefully - it will be quality information. He mentioned the work fetch priority values (used to be called long term debt) and explained that these values would preclude the fetching of more E@H tasks until such time as the high negative value was worked off. The value he didn't discuss was the scheduling priority (used to be called short term debt). You also have a large negative value there which (I imagine) needs to be worked off as well. If you were prepared to edit 'delicate configuration files' (ie your state file) you could get back to a level playing field by zeroing these priority values - used to be called zeroing the debts. As Richard intimates, you can easily trash your entire cache of work if you don't take proper care. However, if you get it right, it's a pretty easy way of restoring order from chaos very quickly :-).
...If you were prepared to edit 'delicate configuration files' (ie your state file) you could get back to a level playing field by zeroing these priority values - used to be called zeroing the debts...
For the purpose of 'zeroing debts' there's the option in the Core Client configuration file (cc_config.xml), which spares you from having to edit client_state.xml (at the expense of introducing a new config file, at least temporarily ;-).
Gruß,
Gundolf
Computer sind nicht alles im Leben. (Kleiner Scherz)
Gary and Gundolf et alia, Thank you very much for your sage advice. I shall learn to do things in small steps.
Regarding the RS I used to have, a while back, all different values there - whatever had been set by default I guess as I kept adding projects over time - originally only had SETI. I noticed one or two projects were getting disproportionate attention so I made the ones that weren't 200, and the rest I evened out to 100.
So Gary you are saying that, as I am always connected (at least between those hours anyway), the CI should be left blank and I should experiment with the ED?
Also that the 200/100 RS is OK. If not let me know.
I must say I like to keep some work at hand for the rather frequent times that servers appear to crash on various projects.
I'd rather avoid messing around with the internal stuff, i.e. the XML files etc. But will gladly learn if necessary.
I can see that you guys have made this an art as well as a science.
Agreed, but the only other
)
Agreed, but the only other machines I have are a netbook - not suitable for such work and an iPad, equally useless.
By cache you mean:
Computer is connected to the Internet about every --- days
Maintain enough work for an additional 2 days
Reduce one of those to 0.5 days?
Wouldn't that cut out large and long lasting WU's such as CPDN ?
Peter
Toronto, Canada
Well I set it
)
Well I set it to...
Computer is connected to the Internet about every 0.5 days
Maintain enough work for an additional 2 days, (since turned that down to 0.5 also).
And what happened? More Malaria WU's came in - I give up...!!
Unfortunately Einstein and all the other are having to wait as all the Malaria ones have a deadline of yesterday if not before, so are running at high priority and when finished yet another takes over pushing anything beyond a 48 hour deadline back.
As I already have a stack of both Malaria and Einstein I have told those two not to accept new WU's until I tell them to.
Peter
Toronto, Canada
Sorry, too late to edit the
)
Sorry, too late to edit the above...it seems to have calmed down now. I trust you meant adjust both those settings.
Peter
Toronto, Canada
RE: Sorry, too late to edit
)
On the question of cache size in isolation, it is best to keep the CI (Connect to Internet) setting very low (if you do have a permanent connection) and to bump up the ED (Extra Days) setting to increase your cache size. This is primarily because BOINC (traditionally - don't know if this has changed in more recent versions) uses the setting for two purposes. Firstly it's to ensure enough work on hand to outlast the period of non-connection and secondly it's in case a finished result can't be returned until a connection is re-established. So BOINC has to plan for a possible delay of this amount between when work is completed and when the work can be returned. Read the hint on the website preference page carefully (leave at zero if always connected).
This could easily be the reason for the sudden download of extra Malaria tasks. If you previously had a CI of 2 days, BOINC would be limiting the work on hand so that any task could wait up to a further 2 days after completion before being returned and still not miss the deadline. If you use ED rather than CI to control your cache size, BOINC doesn't have to plan for any delay in returning work and so can keep some more work on hand.
The other bad thing about a large CI is that BOINC is more likely to find tasks that are possible deadline stress candidates and so will be forced into HP (High Priority) mode sooner and more often than it otherwise would. And of course if you're setting and unsetting NNT in the middle of all this, BOINC will have extra problems managing debt effectively.
You need to think carefully about a number of your preference settings if you wish to make it a bit easier for BOINC. Here are some notes that might assist.
* Think carefully about how many projects you really need to support. Ten is not really too many but the more you choose, the harder it will be to maintain any sort of equilibrium.
* When you have a lot of projects, keep the ED cache setting to a sensible value otherwise you may experience over-fetch problems with several of the projects. With 10 projects, you will get more predictable behaviour if CI+ED is less than 1 day. I've never tried that many projects but if I did I would use 0.01+0.25 to start with and watch the outcome for sufficient time (maybe several days) to see how BOINC handles things. If it isn't giving enough work, increase the ED in small steps (0.25->0.5, etc.) and wait to see what happens. Resist the temptation to fiddle until you are sure that the previous change is OK and you can go a bit more.
* When setting resource shares (RS), it will be easier for BOINC if each project has roughly the same share. Setting highly unequal shares is a recipe for disaster. The number set for any one project is relatively unimportant. What is important is that value in relation to the sum of ALL values. In your case you are fine. From the messages log snippet you posted, all values are either 100 or 200 with a total of 1500. That means than any project has either a 6.7% or 13.3% share of your system. It would be a problem if you set a project or two to a low RS like 10. That would mean that such a project would be entitled to less than 1% of your machine's time, approximately 1 hour per CPU core per week, if your machine was running 24/7. In your case, your preferences as shown in a previous message, limit your machine to crunching between 7.00am and 9.00pm - 14 hours per day if I'm reading things correctly. In that case, a RS of 10 would equate to only 0.5hrs per cpu core per week. If you can follow that, it's pretty easy to understand the commonly seen pattern - download some tasks, have them sit around for quite a while until they get all done in panic mode and then no more tasks for that project for a very long time. This isn't your problem because you have reasonable RS values. I'm taking advantage of this reply to talk to others who may not have set reasonable RS values.
* Don't be tempted to use a low RS if your object is to achieve a 'backup project'. You will not be able to get work from that project when you really need it. If you don't understand why, go back and think about the previous point. Use the zero resource share setting for a genuine backup project.
* When making changes to your preferences, don't make big changes and don't make multiple changes all at once. If making preference changes on the website, click on 'update' in BOINC Manager to apply the change immediately. Always give BOINC sufficient time to show the full effect of any change.
* If you can't be patient (that describes me) do some research on the structure of your state file (client_state.xml) and learn how to safely edit it. You can fix a lot of things very quickly (particularly to do with debt problems and incorrect DCF - Duration Correction Factor - once you know what you are doing) if you really understand what is in that file. If you don't fully understand that file, just learn to be patient :-).
* When digesting advice given in the forums, make sure you realise that we are all fallible and can come up with incorrect advice even though our intentions were good. The quality of the advice is often dependent on the quality of the problem description. The bits not described because they were thought to be unimportant, can often lead to incorrect advice being given. These comments are not directed at the OP because a lot of good info was contained in the previous posts - particularly message logs, preference settings and debt values.
* Finally, if you are lucky enough to have Richard Haselgrove posting about your problem, read what he says very carefully - it will be quality information. He mentioned the work fetch priority values (used to be called long term debt) and explained that these values would preclude the fetching of more E@H tasks until such time as the high negative value was worked off. The value he didn't discuss was the scheduling priority (used to be called short term debt). You also have a large negative value there which (I imagine) needs to be worked off as well. If you were prepared to edit 'delicate configuration files' (ie your state file) you could get back to a level playing field by zeroing these priority values - used to be called zeroing the debts. As Richard intimates, you can easily trash your entire cache of work if you don't take proper care. However, if you get it right, it's a pretty easy way of restoring order from chaos very quickly :-).
Cheers,
Gary.
RE: ...If you were prepared
)
For the purpose of 'zeroing debts' there's the option in the Core Client configuration file (cc_config.xml), which spares you from having to edit client_state.xml (at the expense of introducing a new config file, at least temporarily ;-).
Gruß,
Gundolf
Computer sind nicht alles im Leben. (Kleiner Scherz)
Gary and Gundolf et alia,
)
Gary and Gundolf et alia, Thank you very much for your sage advice. I shall learn to do things in small steps.
Regarding the RS I used to have, a while back, all different values there - whatever had been set by default I guess as I kept adding projects over time - originally only had SETI. I noticed one or two projects were getting disproportionate attention so I made the ones that weren't 200, and the rest I evened out to 100.
So Gary you are saying that, as I am always connected (at least between those hours anyway), the CI should be left blank and I should experiment with the ED?
Also that the 200/100 RS is OK. If not let me know.
I must say I like to keep some work at hand for the rather frequent times that servers appear to crash on various projects.
I'd rather avoid messing around with the internal stuff, i.e. the XML files etc. But will gladly learn if necessary.
I can see that you guys have made this an art as well as a science.
Thanks again.
Peter
Toronto, Canada