Pascal again available, Turing may be coming soon

I have two bits of news on the Turing highpay WU failure matter.  The first is that I today installed driver 316.81, released a couple of days ago, and the failure remained intact.  The second is some initial progress on work inspired by Vyper's observation of parameter dependence and Richard Haselgrove's suggestion that pursuing this point might be useful.

My initial measure was to pull into a comparison environment the parameter strings for a half dozen each of highpay and lowpay work units, selected in two batches a month apart.  As I hoped, the great majority of parameter values were identical.  Aside from the input files (data and template), I saw five candidate parameters, all of which appeared to have fixed values within WU type, but different between the two populations.

So far I have done half the intended work.  I built command lines for a Juha-type portable test case directory for lowpay (which untampered works on all drivers and all cards available to me) for which one single parameter was altered from the original value to that observed on highpay work.

Three of my five candidate parameters sailed through without observed effect--that is the core and memory clocks elevated promptly, and a bit later GPU usage went high and stayed high.  I terminated each after 90 seconds or so of uneventful running.  Somewhat to my surprise, switching either alpha or delta from the lowpay to the highpay value caused the (otherwise) unaltered lowpay test case to fail quickly in fashion observed on highpay work:  Clocks speed up at the usual time, GPU usage kicks up a bit later, but drops back within a very few seconds, and the process terminates in a bit over twenty seconds.

I neglected to check whether the failure signature in stderr resembled what I have seen on highpay failures.  I intend to reverse the transformation, trying command lines in which one of the five suspect parameters is changed from the highpay to the lowpay value, on an otherwise untampered highpay test directory.  I currently expect that all five of these will fail, but that a further test case in which I alter both alpha and delta to the lowpay values will cause the highpay test case, thus altered, to continue GPU processing without prompt termination (I don't think you can call this "working" and don't suppose the resulting output files would match a proper result).

I'll work more on this soon.  But, of course, a key question is how I might bring this result to the attention of anyone who might find it interesting or useful.

In case it may be interesting, the five non-file command line parameters I spotted as seeming to differ systematically between highpay and lowpay work were these:

Param       high-pay       low-pay
Alpha      4.4228137297    1.41058464281
Delta      0.0345036602638 0.444366280137
skyRadius  5.817760e-08    5.090540e-07
ldiBins    15              30
df1dot     2.71528666e-15  2.512676418e-15

 Obviously my conclusion to date is that the differences in skyRadius, IdiBins, and df1dot don't seem to trigger the problem of interest, while the differences in alpha and delta do.  By extension, it seems likely the problem of interest is not driven by the content of the data file or template file (as I formerly supposed) but rather by the parameters supplied to the application as being the necessary ones to process the data files.

[Edited the better part of a day later, to reflect additional results, which contradict my initial conclusions in the previous paragraph.

As planned I prepared a series of test cases in which I modified the 5 suspect numeric parameters singly on a highpay test case to assume their low-pay values.  I was not surprised that none made any difference (fast fail on the previous signature was uniformly observed).  Then I ran a test case in which I changed BOTH alpha and delta.  The surprising result was that also failed.  I then ran a test case in which I changed ALL FIVE of my suspect parameters simultaneously.  Still it failed.

Pending additional work, my revised opinion is that either alpha or delta can trigger the bad behavior on an otherwise safe lowpay test case, but that it seems likely that contrary to my conclusion yesterday, the content of the data file, the template file, or both may also be able to trigger the bad behavior.  An alternate possibility is that there is a triggering parameter value possible from a parameter not on my "suspected five" list.

I did check to see whether the stderr resulting when I converted a "good" lowpay test case to bad behavior just by changing the supplied value of alpha closely resembled my real-world failure cases.  It did.

As I've been visually monitoring MSIAfterburner graphs as part of assessing failure cases, let me be a bit more specific about the usual behavior in the failing cases.

Less than a second after I hit "enter" on the command line, both the core clock and memory clock go from low idle to high values.  GPU usage remains negligible until about 5 seconds elapsed time, when it quickly rises, and just as quickly falls, reaching negligible values before 10 seconds elapsed time.  The core and memory clocks remain far above idle levels, and finally at elapsed time 22 seconds the application terminates (which is visible in the command line window as advance to a fresh prompt).  The first observable difference between passing and failing cases occurs when the GPU usage begins to fall instead of staying high.  On my system this happens after about 8 seconds elapsed time, although the application does not actually terminate until 22 seconds elapsed time.

end of material added the next day]