Multiple GPU machine back up and running!
26 Sep 2015 22:18:54 UTC
Topic 198250
(moderation:
This purpose built machine simply stopped last year and finally I took it to the shop. It was working fine. Took it home and plugged it in. Working fine.
I built this machine to do a lot of work on very little power. My goal was to produce 100,000 stones per day on less then then 300 watts. As I recall it was doing something like that a year ago but have no remaining documentation.
Its only been up and running for two days so to early to see how its doing these days. Have not even measured its power draw!
I am running three cordless NVIDIA GTX 650s and one corded GTX 660. The 660 is less efficient then the 650s but the bus would not run four of the uncorded versions stable at one time. If I do not meet my goal I might just unplug the 660. Lower production but a bit more efficiency? Oh well.
Anyway, I am curious if anyone else has been working on this particular niche curiosity!
Arecibo 19 Oct 2012
Just Because The Space Alien Is Green
Does Not Mean You Should Go
Multiple GPU machine back up and running!
)
I just checked my power draw and my estimate was WAY off. Instead of less then 300 watts I am pulling 365. I do not believe this configuration can produce the number of cobblestones to get lower then 30 watts per 10,000 stones. And it may be worse then I thought.
Last year it seemed to me the single corded GTX 660 was drawing more then twice what each of the 650s were drawing but producing less then twice the work. My cursory review of times/credit these days seems to show the 660 DOES produce twice the work!
One variable is this. My machine was designed to support four 650s at about 55 watts each. Accordingly I sized my power supply at Platinum 400 watts. So by adding the 660 I am now very much streatching the capacity of the 400 watt power supply. I may have exceeded the efficiency of even a platinum supply.
Its all working fine, and probably the best approach is simply let this setup settle in to see what it produces. Then disconnect the 660 and see what the three individual 650s can do! Then run the 660 with TWO 650s and make a comparison.
Its a hobbie......
Arecibo 19 Oct 2012
Just Because The Space Alien Is Green
Does Not Mean You Should Go
I would think ditch the
)
I would think ditch the GTX660 and get a GTX750Ti instead. They are more energy efficient. You can even get versions without the PCIe power plug so they get their power directly from the motherboard, like you're going with the 650's.
BOINC blog
I just retired a GTX660
)
I just retired a GTX660 specifically for being the least power efficient GPU in my flotilla.
A GTX750, or 750Ti, or 970 (when suitably employing the memory overclock discussed in a sticky thread in these forums) will give a comfortably superior output/watt to a 660.
While no one seems to advocate the GTX750 here, preferring the 750Ti, if you get a good lower price on a 750, including the 1 GByte base memory size, you may see as I have by direct comparison that it comes very close to the 750Ti output. Reasons for preferring the 750Ti 2 Gbyte model may include future proofing, or performance on other applications, but specifically for Parkes PMPS here at Einstein, I think the GTX750 is an excellent economy choice, both in purchase price and in power efficiency.
The last couple of application version releases have left Einstein Parkes PMPS with a drastically lower "sharing penalty" for running multiple cards on the same host than I recall from BOINC applications in years past.
Host power is also an issue in getting really good productivity per watt. My most recent build was a dual-core Haswell, which is proving an entirely satisfactory host for a GTX970+GT750Ti. I suspect many people routinely selecting 4 physical core CPUs are harming their Einstein system level power productivity compared to a dual-core Haswell, which in the moderate speed grades is also quite inexpensive.
I am glad you guys do this
)
I am glad you guys do this just for the info but I just keep myself from checking how much power mine use so I never switch just to save a few dollars.
Especially since the newer cards cost so much.
I got my GeForce 660Ti SC when they first came out for $300 and just plan on running it until it dies.
Been running it 24/7 since day one and it still works great (BRP6 X2 in about 2hrs 45mins) that host also runs vLHC X2 and a CMS-dev on a not so new quad-core.
I have a couple of those 650Ti OC'd and they do the same in about 4hrs so they take a bit longer but they do run fuel efficient compared to the 660Ti
I also have a pair of 550Ti's and a 560Ti OC'd and I can imagine they use more power and the 550Ti's are a bit slower but the 560Ti is still a fast machine.
Only thing that slows down my 560Ti is it has a 3-core phenom which always runs the vLHC X2 but once in a while I trick it by running a CMS-dev on that 3rd core and watch the Boinc and OS Manager to see if it is still running fast or not since for some reason they run great and if I don't check it will slow down the GPU tasks to almost nothing (CPU and GPU are cooled)
But most of the time it will still run the BRP6 X2 in about 3hrs
On my wish list is just having a 8-core CPU that is closer to 4GHz and just add the Ram and best deal video card with the most Cuda cores.
GeForce is the only version I have had but here I have seen how good those Radeons are......the price is just what will stop me when I have to replace any of mine.
RE: I just checked my power
)
There are some changes you should know about if you want to maximise performance. For example, there is now a BRP6 app in beta test that uses a later version of the CUDA libs. You could perhaps get 15-20% performance improvement if you allowed beta apps (pref setting). These figures apply to single GPU doing BRP6 only on Kepler or later GPUs. There are some reports that mixed BRP4G/BRP6 might have performance problems. To see how much you can get out of your rig, it would be best just to run BRP6-beta.
I believe that running so many GPUs in the one rig will always cause a performance loss. Just as an example of what is possible on power hungry AMD GPUs, here is a link to a Pentium dual core (Haswell) of mine, with an AMD HD 7850 GPU. It pulls about 165W from the wall and has a RAC of 90K - less than 20W per '10,000 stones' :-).
I'm not at all suggesting that you go buy different GPUs. What I would suggest is you reduce from 4 to 3 in the one rig and buy a budget board and CPU to run the surplus GPU on its own using say a 300W efficient PSU. That way your 400W unit would be less stressed and the rig might actually produce nearly as much output. The second rig would be a nice bonus to experiment with.
Sure - definitely one for the 'shed'... Just don't tell the wife how expensive it is :-).
Cheers,
Gary.
RE: Its a
)
No, it is addiction. At least, for me.:)
Lots of good info from all
)
Lots of good info from all concerned. I am especially intrigued about reducing the load from four separate gpus down to a more reasonable number. And I do have a small second rig that has a couple of 650s in it but will not power up.
Earlier this year I took it into the shop and it ran fine. Home again and won't power up. Maybe I will use it for a test mule once I kick hard enough.
The current plan is to sit tight and see what THE STREASED OUT four gp machine will actually do. Then perhaps solicit some more specifics on tweaks and adjustments.
I am impressed with 20 watts or less per 10k stones!
Last summer I was no better then 27 and now more then thirty. But right now I just getting every thing pushing hard, not easy.
Later !!!
Arecibo 19 Oct 2012
Just Because The Space Alien Is Green
Does Not Mean You Should Go
At least, get the BRP6-beta
)
At least, get the BRP6-beta app going.
It's just a couple of clicks away ....
You know you want to do it :-).
Cheers,
Gary.
Hi
)
Hi
Mark
Archae
Majic
Gary
Sid
Its a very good thing to have people discussing topics I forgot last year! Before discussing new approaches here is an update on the only rig now working tasks. [The machine I am typing this on has fried PCIe slots - don't ask]
It mounts a 3.3 gig I3 processor, 3 x GTX650 + 1 GTX600 producing 120,000 stones @ 370 Watts all devices running two tasks each. That's 31.5 watts per 10,000 stones. Although that beats my target of 35 watts that a couple of years ago there are two problems.
First of all my power supply is fanless 400 watt Platinum unit. Does anyone else notice a potential problem? It has been happily doing this nearly impossible thing for many days now. I sort thought its why the machine went down a year ago in the fist place. Took it in to the shop last week and its just fine. Put in a new MB battery but do not figure how that would shut me down.
I have already orderred a used GTX750 ti to replace the hot-plate 660 that is drawing about 140 watts all by its lonesome. [did an A/B check on the watt meter]. I am now counting CUDA cores versus power draw. The GTX 650 has 398, the GTX660 has 960 and the GTX750 ti has 640. And instead of 140 watts its listed at 60!
Its an EBAY special for about $90 so the down side is ok. My problem putting the machine together in the first place was to find what would plug in to any one of the four slots and actually work together.
Arecibo 19 Oct 2012
Just Because The Space Alien Is Green
Does Not Mean You Should Go
Tell me more about this beta
)
Tell me more about this beta thing..... ;)
Arecibo 19 Oct 2012
Just Because The Space Alien Is Green
Does Not Mean You Should Go