Smarter Scheduling
2 May 2014 2:23:03 UTC
Topic 197567
(moderation:
Right now, FGRP (gpu) uses a ton of gpu memory and little processing power on the gpu.
One task uses more than half my GPU memory though. BRP tasks on the other hand, are exactly the opposite.
I could run a BRP and a FGRP task at the same time, stay under 1GB of memory, and not waste a whole bunch of crunching power. But I can't have two FGRPs running at the same time.
Is there a way to do this?
Smarter Scheduling
)
I think so - I haven't actually tried this - as long as you were prepared to NOT run Einstein CPU tasks all the time or, alternatively, to run specific CPU tasks when that type was NOT simultaneously running on the GPU. You seem to be running a mix of most types of tasks, including S6CasA beta GPU tasks. I imagine there are a number of different ways to approach this so I'll just choose one as an example. Other people may have entirely different ideas.
For the example, let's say we want to run two tasks on the GPU but only ever 1 FGRP3 task. We don't mind if there are 2 of either BRP4G or BRP5. S6CasA GPU tasks are a bit of an unknown quantity. I have run three simultaneously on a 2GB GPU (very briefly - 3 seem to run fine) but because the Devs have limited participation to 1GB or above GPUs, I suspect there might be a problem running 2 on a 1GB GPU. So let's say we treat S6CasA the same way as FGRP3.
If we set up this particular app_config.xml :-
[pre]
einstein_S6CasA
1
0.5
1.0
hsgamma_FGRP3
1
0.5
1.0
einsteinbinary_BRP5
2
0.5
0.5
einsteinbinary_BRP4G
2
0.5
0.5
[/pre]
we can involve all the GPU apps but restrict FGRP3 and S6CasA as suggested above.
The file would support only 1 S6CasA or FGRP3 running at a given point. If one of those were running, there could also be a single BRP5 or BRP4G running concurrently. If neither of those (S6CasA or FRGP3) were running, there could be 2xBRP5 or 2xBRP4G or one of each. As the maximum cpu_usage would be 1.5 and the minimum would be 1.0 from any of those combinations, there would always be one CPU core tied up in GPU support duties. That could well be enough but if you wanted to try having more support (ie 2 free cores) when there was a S6CasA or FGRP3 task in the mix, you could set the cpu_usage figures to 1.5 for those two. Even if you started with the file as listed above, it would be interesting to see the changes with an extra free core. There would still be just one free core if there was a combination of BRP4G/BRP5 running. This should be more than enough.
Because of the max_concurrent limitations, S6CasA or FGRP3 CPU tasks could not run if the corresponding type of GPU task was running. One of each type could be running on CPUs if BRP4G/BRP5 were running on the GPU. If you think through all the possible scenarios, there will probably be combinations where not all available CPU cores would be allowed to be running tasks when that scenario was in play. If you designed to always have 2 free cores and two running cores, it should be possible for the other two to each be running a CPU task most of the time (I think). I don't know if you have other CPU projects that could 'take up the slack' if Einstein apps were prevented from running on all available cores at a particular point.
I'm not very good at thinking all this through so I would tend to do the experiment and see what happens and then refine my thinking about all the subtleties I forgot about initially :-). It would be a very interesting experiment though, so I hope you do it and then tell us all what actually happens :-).
As I warned at the beginning, I haven't actually tried anything like this before so I may be totally wrong in my thinking so ... caveat emptor!
Cheers,
Gary.
You're not restricted to
)
You're not restricted to round numbers for the GPU usage. If you put in something like 0.48 for BRP and 0.51 for FGRP, that would meet the objective of preventing two FGRP running together, but allowing either one of each or two BRP running together.
OK, so if you set 0.51 for
)
OK, so if you set 0.51 for both S6CasA and FGRP3 and also set max_concurrent for both these to 2, you would allow BOINC more choices for finding CPU tasks to run without risking two of these attempting to run on the GPU, or so it seems to me.
As I intimated, I often have problems seeing the bleeding obvious :-).
Cheers,
Gary.
RE: You're not restricted
)
This works right in the Einstein preferences online.
Not running the S6CasA beta right now, it did some bad things to my usability.
Semi related. I've had
)
Semi related.
I've had Boinc set to use not more than 50% of CPU cores (2 in this case).
Normally, it runs 2 cpu tasks and also gpu tasks. Technically, with BRP tasks that would be 2.4 cpu. FGRP tasks however, seem to count as a core to Boinc, so I get 1 CPU task, 1 FGRP and one BRP.
This makes sense, but I didn't think BOINC counted GPU use at all (such as allowing 4 cpu tasks + gpu tasks).
If I set it to run 3 cores, it downloads another CPDN file, and I'll have the same problem of running those tasks every time 2 BRP tasks run together. They can be so unstable there, I'd rather not be starting and stopping them.