Here's an interesting item
1 Nov 2005 3:38:40 UTC
Topic 190101
(moderation:
Hello all,
Hope everyone is crunching away...anyway. Here is something I found while fiddling with my machine.
This is the specs with Windows 98 installed:
Machine on Windows98
10/31/2005 06:09:49 PM||Running CPU benchmarks
10/31/2005 06:10:46 PM||Benchmark results:
10/31/2005 06:10:46 PM||Number of CPUs: 1
10/31/2005 06:10:46 PM||1379 double precision MIPS (Whetstone) per CPU
10/31/2005 06:10:46 PM||2242 integer MIPS (Dhrystone) per CPU
10/31/2005 06:10:46 PM||Finished CPU benchmarks
And, this is the results with Windows XP installed:
Machine on WindowsXP
10/30/2005 02:15:31 PM||Running CPU benchmarks
10/30/2005 02:16:30 PM||Benchmark results:
10/30/2005 02:16:30 PM||Number of CPUs: 2
10/30/2005 02:16:30 PM||1303 double precision MIPS (Whetstone) per CPU
10/30/2005 02:16:30 PM||1599 integer MIPS (Dhrystone) per CPU
10/30/2005 02:16:30 PM||Finished CPU benchmarks
One and the same machine just the OS is different.
Hypertheading was active in both configuations.
Greg
Here's an interesting item
)
How do the elapsed and CPU times compare?
Hyperthreading won't do you any good on Win98, it only supports one processor. So that one processor gets full use of the CPU.
Hyperthreading does mess up benchmarking - it thinks there are two full processors, but you really just have one. If you have a lot of spare time, you might try it with hypertheading turned off, and again with hyperthreading turned back on again but with your preferences set to use just one processor.
Walt
You will likely see a
)
You will likely see a "spread" of values ... I ran the benchmarks 10 times before I was confident I had a good average of the "actual" value.
RE: You will likely see a
)
How do you keep the average consistent? Like what happens five days later when BOINC decides the old numbers aren't good enough?
RE: RE: You will likely
)
Run it till you get numbers you like ... or edit the xml files ...
Fundamentally the use of Dhrystone and Whetstone are only the "best" of the worst choices. In other words, there is not much else, but, when you read things like:
"Q: Why does the rating vary between sequential runs?
A: On most systems, the value of the rating shouldn't change by more than about +/-5% from run to run. On systems with limited memory it may vary by +/-10% due to memory swapping. If you're seeing variations higher than this, some hardware or software is probably to blame. Do note that "limited memory" depends on operating system, installed drivers, running programs, etc. A badly configured system with 64MB may be worse memory-wise than an 16MB system..."
SANDRA Pro
(I was just looking at some other pages about Whetstone ...)
But it is not that I have ever been of any other opinion ... I never thought that the "synthetic" benchmarks were good for much other than to create a number. All in all, we are still avoiding the problem. You either have to directly count the actual ops, or you need to bench with the actual work. PERIOD (Paul's opinion)
I don't like counting the ops, because that does add overhead, and even 1% is a lot ... and to do accurate accounting, WHICH I ALSO THINK IS NECESSARY, the probability is that the overhead will be at least that much, if not more. The real problem is that virtually none of the applications we are running are deterministic.
That is why my proposal is as elaborate as it is. It is actually not that elaborate at all, ONCE YOU GET THE DRIFT OF IT. We run some work with instrumentation and "eat" the cost. But, the work is real work, not fake, and the participant gets full credit for all the time spent, including the accounting. But, then we don't put the accounting burden on most of the work performed.
That is it in a nutshell.
I do not believe that the latest change/refinement Dr. Anderson just added will not materially affect what is going on. All it will do is improve the "balance" of the FP/Integer ratios and it may reduce the wide variances we see in credit claims. But, I do not expect that ... in theory, we should be able to see an almost immediate change if he has already adjusted the SETI@Home numbers. Within a week or two we can start to "spot check" work to see if it has come down in "enthusiasm" ...
I have been looking at the code and I may take the 90,000 some odd results I have in my data base and play with his tool to see what it does. I have also been trying to think of a way that may allow us to use something of the sort to reduce the spread. But, till we have more data, and I "test drive" his tool ... no idea ...
Though, since I did stop doing SETI@Home work a week or so ago, I may also be a good test case. I can also update my data collection tools and start to do SETI work and also start to capture the new data ...
===== Edit
One of my biggest objections to the current system is that it can be "hacked" and since this was one of the issues in Classic, and there is no protection against subtle "hacks" to inflate a participant's score ... well ... sorry ...
Credit is a validation of what I have contributed, any allowance of compromise cheapens my contribution ...
Hay all, I'm going to shut
)
Hay all,
I'm going to shut down now and deactivate the Hypertheading...will post the results as soon as the system is up.
Greg
Well here is the results
)
Well here is the results without Hypertheading:
11/01/2005 07:59:36 PM||Running CPU benchmarks
11/01/2005 08:00:33 PM||Benchmark results:
11/01/2005 08:00:33 PM||Number of CPUs: 1
11/01/2005 08:00:33 PM||1478 double precision MIPS (Whetstone) per CPU
11/01/2005 08:00:33 PM||2294 integer MIPS (Dhrystone) per CPU
11/01/2005 08:00:33 PM||Finished CPU benchmarks
Still running Windows 98 OSR-2, here are the old numbers just for referance:
"Machine on Windows98
10/31/2005 06:09:49 PM||Running CPU benchmarks
10/31/2005 06:10:46 PM||Benchmark results:
10/31/2005 06:10:46 PM||Number of CPUs: 1
10/31/2005 06:10:46 PM||1379 double precision MIPS (Whetstone) per CPU
10/31/2005 06:10:46 PM||2242 integer MIPS (Dhrystone) per CPU
10/31/2005 06:10:46 PM||Finished CPU benchmarks
Machine on WindowsXP
10/30/2005 02:15:31 PM||Running CPU benchmarks
10/30/2005 02:16:30 PM||Benchmark results:
10/30/2005 02:16:30 PM||Number of CPUs: 2
10/30/2005 02:16:30 PM||1303 double precision MIPS (Whetstone) per CPU
10/30/2005 02:16:30 PM||1599 integer MIPS (Dhrystone) per CPU
10/30/2005 02:16:30 PM||Finished CPU benchmarks"
Greg
O.K. here is another item...I
)
O.K. here is another item...I just installed 512meg's of ram into this machine, upon boot up the bois reports 504megs of ram, (with the 256meg I had the bios would report 254megs of ram).
Without reguard of the above events I ran a benchmark off of Boinc to see if there would be any speed gained with the added ram. here is the results:
Machine on Windows98/Multithreading off, 512Meg Ram, Boinc 4.45
11/05/2005 06:09:29 PM||Running CPU benchmarks
11/05/2005 06:10:26 PM||Benchmark results:
11/05/2005 06:10:26 PM||Number of CPUs: 1
11/05/2005 06:10:26 PM||1468 double precision MIPS (Whetstone) per CPU
11/05/2005 06:10:26 PM||2190 integer MIPS (Dhrystone) per CPU
11/05/2005 06:10:26 PM||Finished CPU benchmarks
Here are the old values that started this threaded, just for background:
Machine on Windows98/Multithreading off, 256Meg Ram, Boinc 4.45
11/01/2005 07:59:36 PM||Running CPU benchmarks
11/01/2005 08:00:33 PM||Benchmark results:
11/01/2005 08:00:33 PM||Number of CPUs: 1
11/01/2005 08:00:33 PM||1478 double precision MIPS (Whetstone) per CPU
11/01/2005 08:00:33 PM||2294 integer MIPS (Dhrystone) per CPU
Machine on Windows98/Multithreading on, 256Meg Ram, Boinc 4.45
10/31/2005 06:09:49 PM||Running CPU benchmarks
10/31/2005 06:10:46 PM||Benchmark results:
10/31/2005 06:10:46 PM||Number of CPUs: 1
10/31/2005 06:10:46 PM||1379 double precision MIPS (Whetstone) per CPU
10/31/2005 06:10:46 PM||2242 integer MIPS (Dhrystone) per CPU
10/31/2005 06:10:46 PM||Finished CPU benchmarks
By the way the output below is the machine running the new Boinc 5.2.6
Machine on WindowsXP/Multithreading on, 256Meg Ram
10/30/2005 02:15:31 PM||Running CPU benchmarks
10/30/2005 02:16:30 PM||Benchmark results:
10/30/2005 02:16:30 PM||Number of CPUs: 2
10/30/2005 02:16:30 PM||1303 double precision MIPS (Whetstone) per CPU
10/30/2005 02:16:30 PM||1599 integer MIPS (Dhrystone) per CPU
10/30/2005 02:16:30 PM||Finished CPU benchmarks
The specific of the hardware of this machine is in my sig. Can anyone explain what is going on?
Greg
Yes ... :) The benchmarks
)
Yes ... :)
The benchmarks are unstable ...
If you run them 10 times in a row, there is a very good chance that you will see this sort of variability. This is probably the second most important indictment of the benchmarks. The first of course being that they do not characterize the systems very well ...
Hi Paul, Ahhh to bad, just
)
Hi Paul,
Ahhh to bad, just wanted to know if I was making some headway on inprovments to the system.
Greg
Greg, The "best" way is to
)
Greg,
The "best" way is to actually do a running average over some period of time using a number of actual work units.
In other words, take the completion time of, say, 10 work units; average that time. Make the change, collect time on another 10 work units, average the times, compare the averages ...
Though I keep saying it, there are plenty of people that still look at the benchmark numbers as if they have meaning ... basically we proved otherwise back in the Beta test. It is just that there has not been a "good" idea of what to do to replace the current system (well, I have one now, but it is not likely to see daylight anytime soon).