Running multiple tasks concurrently per GPU - How to revert to just one task.

mikey
mikey
Joined: 22 Jan 05
Posts: 12676
Credit: 1839075724
RAC: 4001

KLiK wrote: Keith Myers

KLiK wrote:

Keith Myers wrote:

You need to try to stagger the task startups on the gpu so that they all don't converge at the 49% and 99% cpu offload points when running gpu task multiples.

Just suspend a task that starts anew on the gpu until the previous one had made progress halfway to one of its cpu compute intervals, then un-suspend the task.

Thank you.

Immediate result can be seen:

 

It sure looks like both tasks have started and running in sync instead of opposite to each other

Ian&Steve C.
Ian&Steve C.
Joined: 19 Jan 20
Posts: 3945
Credit: 46589512642
RAC: 64196783

mikey wrote: It sure looks

mikey wrote:

It sure looks like both tasks have started and running in sync instead of opposite to each other



you're focusing on the wrong part of the graph. the beginning is with the tasks not staggered, but the end shows the second task start right when the first task ends, leading to near constant GPU utilization. you can see a slight dip in temp, then it comes right back up instead of falling down like the previous two cycles.

_________________________________________________________________________

mikey
mikey
Joined: 22 Jan 05
Posts: 12676
Credit: 1839075724
RAC: 4001

Ian&Steve C. wrote: mikey

Ian&Steve C. wrote:

mikey wrote:

It sure looks like both tasks have started and running in sync instead of opposite to each other



you're focusing on the wrong part of the graph. the beginning is with the tasks not staggered, but the end shows the second task start right when the first task ends, leading to near constant GPU utilization. you can see a slight dip in temp, then it comes right back up instead of falling down like the previous two cycles. 

okay thanks

KLiK
KLiK
Joined: 1 Apr 14
Posts: 60
Credit: 426006605
RAC: 1374916

mikey wrote:Ian&Steve C.

mikey wrote:

Ian&Steve C. wrote:

mikey wrote:

It sure looks like both tasks have started and running in sync instead of opposite to each other



you're focusing on the wrong part of the graph. the beginning is with the tasks not staggered, but the end shows the second task start right when the first task ends, leading to near constant GPU utilization. you can see a slight dip in temp, then it comes right back up instead of falling down like the previous two cycles. 

okay thanks

Its like @mikey said. 

But also, there are 2 cards in the system. So both cards work, that is why there are 2 graphs!

(Though my M5000 & P4 was read with only single temp., so I changed from M5000 to P2000...waiting for Eintein@home tasks to start again, to show the real results.)

Main thing is: previous dips were after around 18~20min of GPU & lasted for about ~15min with CPU time. If I run 2x per GPU, then the 2nd task can almost be over by the time 1st one finished it CPU calc. ????

KLiK
KLiK
Joined: 1 Apr 14
Posts: 60
Credit: 426006605
RAC: 1374916

Keith Myers wrote: Yes, same

Keith Myers wrote:

Yes, same principle. But in my experience so far with MeerKAT tasks, they don't respond as well to 2X as the optimized app for GR#1.  You won't see as much benefit . . . .  if any.

They also use  more VRAM.  2X may be cutting it close with 6GB.

Are you sure about that? This is MeerKAT, with only 22~25% load & not much memory used.

Did anybody use 2x WU on stronger cards, like Teslas - which can crunch more data? What are the experiences? 

GWGeorge007
GWGeorge007
Joined: 8 Jan 18
Posts: 3060
Credit: 4961404353
RAC: 1396507

KLiK wrote: Keith Myers

KLiK wrote:

Keith Myers wrote:

Yes, same principle. But in my experience so far with MeerKAT tasks, they don't respond as well to 2X as the optimized app for GR#1.  You won't see as much benefit . . . .  if any.

They also use  more VRAM.  2X may be cutting it close with 6GB.

Are you sure about that? This is MeerKAT, with only 22~25% load & not much memory used.

Did anybody use 2x WU on stronger cards, like Teslas - which can crunch more data? What are the experiences? 

Your image doesn't show much of a load from a Tesla card, and your two Tesla K20Xm's have less than 6GB of VRAM in Windows 10 running an Intel i7-5820K CPU.  It may matter what your PC & BOINC specs are set to.

George

Proud member of the Old Farts Association

Marcelo
Marcelo
Joined: 14 Dec 11
Posts: 7
Credit: 175238086
RAC: 1120856

Hello friends of

Hello friends of science,

 

so far I have only been working with “simple” PC builds and found information about the settings relatively quickly in the forum. Now I wanted to integrate a second graphics card and run both in parallel. 

 

I activated multiGPU via cc_config as follows. 

 

 

  <cc_config>                                       
             <options>                                   
                                                                                                           
                      <use_all_gpus>1</use_all_gpus>           
                                                       
             </options>                                 
  </cc_config>

 

 

 

Because my 1st GPU (RTX 4080) has enough VRAM, I have set several WU per GPU via app_config.

 

 

<app_config>
    <app>
        <name>einstein_O3AS</name>
        <gpu_versions>
            <gpu_usage>0.25</gpu_usage>
            <cpu_usage>1</cpu_usage>
        </gpu_versions>
    </app>
</app_config>

 

 

 

My question:

 

I want the 1st GPU (RTX 4080) to calculate 4 WUs at the same time and the 2nd GPU (RTX 2070 Super) to calculate 3 WUs. Because the 2070 has too less VRAM and can only load 3 WUs.

 

How do I have to set this up?

 

Regards

 

Marcelo

Ian&Steve C.
Ian&Steve C.
Joined: 19 Jan 20
Posts: 3945
Credit: 46589512642
RAC: 64196783

Marcelo wrote: My

Marcelo wrote:

My question:

I want the 1st GPU (RTX 4080) to calculate 4 WUs at the same time and the 2nd GPU (RTX 2070 Super) to calculate 3 WUs. Because the 2070 has too less VRAM and can only load 3 WUs.

How do I have to set this up?

Regards

Marcelo


the short answer is you can't*

*the longer answer is you can, but only if you run two instances of BOINC with each GPU dedicated to a separate instance, or put the GPUs in separate computers

_________________________________________________________________________

Marcelo
Marcelo
Joined: 14 Dec 11
Posts: 7
Credit: 175238086
RAC: 1120856

Okay thanks for the quick

Okay thanks for the quick info. 

And is it possible that one GPU only processes O3ASHF1d WUs and the other GPU only executes Binary Radio Pulsar Search (MeerKAT) WUs?


I'm currently testing which configuration has the best efficiency (power consumption/crunch power/WUs etc :D)

 

 

 

 

San-Fernando-Valley
San-Fernando-Valley
Joined: 16 Mar 16
Posts: 397
Credit: 10108353455
RAC: 28559284

Check the exclude option

Check the exclude option (etc.) described in the User Manual under cc_config.

Might be of interest.

SFV

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.