Completed Result Refused: result already reported as success message
10 Jun 2016 3:53:30 UTC
Topic 198639
(moderation:
I noticed today that my latest Raspberry Pi 3 Node (https://einsteinathome.org/host/12282811 is showing that only one task is present. This was odd because all the other Pi 3 nodes I started last night have reported and received credit (rpi3-1 and 2). I looked in the log and saw for the first time ever several "Completed Result X Refused: result already reported as success message" messages in the log (copied below). This is all odd because I see that I still have all 4 cores processing work, and I haven't done anything to change the host name nor were work units duplicated amongst hosts because of anything I had done (the Raspbian image was setup w/ boing-client installed, but it was configured for E@H, I do that manually when bringing a node online). I have seen this error several times (this is the log for the last 24 hoursish, since the node came online) and I also see in the log that my machine got a new ID. Any thoughts on what might be happening and how I can fix it?
PS: I just looked at another node I put online last night and get the same thing (second code block below) - .
https://einsteinathome.org/host/12282800
Why are my there newest Pis getting the same work? (ETA third code block that shows same error for rpi3-1 - https://einsteinathome.org/host/12282545
pi3-3
08-Jun-2016 22:53:19 [Einstein@Home] Started download of einsteinbinary_BRP4_1.47_arm-unknown-linux-gnueabihf__NEON_Beta 08-Jun-2016 22:53:19 [Einstein@Home] Started download of p2030.20131013.G180.42-04.88.N.b3s0g0.00000_1905.bin4 08-Jun-2016 22:53:23 [Einstein@Home] Finished download of p2030.20131013.G180.42-04.88.N.b3s0g0.00000_1905.bin4 08-Jun-2016 22:53:23 [Einstein@Home] Started download of stochastic_full.bank 08-Jun-2016 22:53:24 [Einstein@Home] Finished download of einsteinbinary_BRP4_1.47_arm-unknown-linux-gnueabihf__NEON_Beta 08-Jun-2016 22:53:24 [Einstein@Home] Finished download of stochastic_full.bank 08-Jun-2016 22:53:24 [Einstein@Home] Started download of p2030.20131013.G180.42-04.88.N.b3s0g0.00000.zap 08-Jun-2016 22:53:24 [Einstein@Home] Started download of p2030.20131013.G180.42-04.88.N.b3s0g0.00000_1981.bin4 08-Jun-2016 22:53:26 [Einstein@Home] Finished download of p2030.20131013.G180.42-04.88.N.b3s0g0.00000.zap 08-Jun-2016 22:53:26 [Einstein@Home] Started download of p2030.20131013.G180.42-04.88.N.b3s0g0.00000_1986.bin4 08-Jun-2016 22:53:26 [Einstein@Home] Starting task p2030.20131013.G180.42-04.88.N.b3s0g0.00000_1905_0 08-Jun-2016 22:53:27 [Einstein@Home] Finished download of p2030.20131013.G180.42-04.88.N.b3s0g0.00000_1981.bin4 08-Jun-2016 22:53:27 [Einstein@Home] Started download of p2030.20131013.G180.42-04.88.N.b3s0g0.00000_1987.bin4 08-Jun-2016 22:53:27 [Einstein@Home] Starting task p2030.20131013.G180.42-04.88.N.b3s0g0.00000_1981_1 08-Jun-2016 22:53:28 [Einstein@Home] Finished download of p2030.20131013.G180.42-04.88.N.b3s0g0.00000_1986.bin4 08-Jun-2016 22:53:28 [Einstein@Home] Started download of p2030.20131013.G180.42-04.88.N.b2s0g0.00000_2032.bin4 08-Jun-2016 22:53:28 [Einstein@Home] Starting task p2030.20131013.G180.42-04.88.N.b3s0g0.00000_1986_1 08-Jun-2016 22:53:29 [Einstein@Home] Finished download of p2030.20131013.G180.42-04.88.N.b3s0g0.00000_1987.bin4 08-Jun-2016 22:53:29 [Einstein@Home] Finished download of p2030.20131013.G180.42-04.88.N.b2s0g0.00000_2032.bin4 08-Jun-2016 22:53:29 [Einstein@Home] Started download of p2030.20131013.G180.42-04.88.N.b2s0g0.00000.zap 08-Jun-2016 22:53:29 [Einstein@Home] Started download of p2030.20131013.G177.42+00.95.N.b2s0g0.00000_1576.bin4 08-Jun-2016 22:53:29 [Einstein@Home] Starting task p2030.20131013.G180.42-04.88.N.b3s0g0.00000_1987_1 08-Jun-2016 22:53:31 [Einstein@Home] Finished download of p2030.20131013.G180.42-04.88.N.b2s0g0.00000.zap 08-Jun-2016 22:53:31 [Einstein@Home] Started download of p2030.20131013.G177.42+00.95.N.b2s0g0.00000.zap 08-Jun-2016 22:53:32 [Einstein@Home] Finished download of p2030.20131013.G177.42+00.95.N.b2s0g0.00000_1576.bin4 08-Jun-2016 22:53:32 [Einstein@Home] Finished download of p2030.20131013.G177.42+00.95.N.b2s0g0.00000.zap 08-Jun-2016 22:53:32 [Einstein@Home] Started download of p2030.20131013.G177.42+00.95.N.b2s0g0.00000_1587.bin4 08-Jun-2016 22:53:32 [Einstein@Home] Started download of p2030.20131013.G177.42+00.95.N.b2s0g0.00000_1589.bin4 08-Jun-2016 22:53:34 [Einstein@Home] Finished download of p2030.20131013.G177.42+00.95.N.b2s0g0.00000_1587.bin4 08-Jun-2016 22:53:34 [Einstein@Home] Started download of p2030.20131013.G177.42+00.95.N.b2s0g0.00000_1590.bin4 08-Jun-2016 22:53:35 [Einstein@Home] Finished download of p2030.20131013.G177.42+00.95.N.b2s0g0.00000_1589.bin4 08-Jun-2016 22:53:35 [Einstein@Home] Finished download of p2030.20131013.G177.42+00.95.N.b2s0g0.00000_1590.bin4 08-Jun-2016 22:53:35 [Einstein@Home] Started download of p2030.20131013.G177.42+00.95.N.b2s0g0.00000_1591.bin4 08-Jun-2016 22:53:35 [Einstein@Home] Started download of p2030.20131013.G177.42+00.95.N.b2s0g0.00000_1592.bin4 08-Jun-2016 22:53:38 [Einstein@Home] Finished download of p2030.20131013.G177.42+00.95.N.b2s0g0.00000_1591.bin4 08-Jun-2016 22:53:38 [Einstein@Home] Finished download of p2030.20131013.G177.42+00.95.N.b2s0g0.00000_1592.bin4 08-Jun-2016 22:53:38 [Einstein@Home] Started download of p2030.20131013.G177.42+00.95.N.b2s0g0.00000_1721.bin4 08-Jun-2016 22:53:38 [Einstein@Home] Started download of einstein_icon.png 08-Jun-2016 22:53:40 [Einstein@Home] Finished download of p2030.20131013.G177.42+00.95.N.b2s0g0.00000_1721.bin4 09-Jun-2016 09:29:12 [Einstein@Home] Computation for task p2030.20131013.G180.42-04.88.N.b3s0g0.00000_1986_1 finished 09-Jun-2016 09:29:12 [Einstein@Home] Starting task p2030.20131013.G177.42+00.95.N.b2s0g0.00000_1576_0 09-Jun-2016 09:29:14 [Einstein@Home] Started upload of p2030.20131013.G180.42-04.88.N.b3s0g0.00000_1986_1_0 09-Jun-2016 09:29:17 [Einstein@Home] Finished upload of p2030.20131013.G180.42-04.88.N.b3s0g0.00000_1986_1_0 09-Jun-2016 09:33:56 [Einstein@Home] Computation for task p2030.20131013.G180.42-04.88.N.b3s0g0.00000_1981_1 finished 09-Jun-2016 09:33:56 [Einstein@Home] Starting task p2030.20131013.G180.42-04.88.N.b2s0g0.00000_2032_0 09-Jun-2016 09:33:58 [Einstein@Home] Started upload of p2030.20131013.G180.42-04.88.N.b3s0g0.00000_1981_1_0 09-Jun-2016 09:34:01 [Einstein@Home] Finished upload of p2030.20131013.G180.42-04.88.N.b3s0g0.00000_1981_1_0 09-Jun-2016 10:03:05 [Einstein@Home] Computation for task p2030.20131013.G180.42-04.88.N.b3s0g0.00000_1987_1 finished 09-Jun-2016 10:03:05 [Einstein@Home] Starting task p2030.20131013.G177.42+00.95.N.b2s0g0.00000_1591_0 09-Jun-2016 10:03:07 [Einstein@Home] Started upload of p2030.20131013.G180.42-04.88.N.b3s0g0.00000_1987_1_0 09-Jun-2016 10:03:10 [Einstein@Home] Finished upload of p2030.20131013.G180.42-04.88.N.b3s0g0.00000_1987_1_0 09-Jun-2016 10:12:03 [Einstein@Home] Computation for task p2030.20131013.G180.42-04.88.N.b3s0g0.00000_1905_0 finished 09-Jun-2016 10:12:03 [Einstein@Home] Starting task p2030.20131013.G177.42+00.95.N.b2s0g0.00000_1587_1 09-Jun-2016 10:12:05 [Einstein@Home] Started upload of p2030.20131013.G180.42-04.88.N.b3s0g0.00000_1905_0_0 09-Jun-2016 10:12:08 [Einstein@Home] Finished upload of p2030.20131013.G180.42-04.88.N.b3s0g0.00000_1905_0_0 09-Jun-2016 10:29:23 [Einstein@Home] Sending scheduler request: To report completed tasks. 09-Jun-2016 10:29:23 [Einstein@Home] Reporting 4 completed tasks 09-Jun-2016 10:29:23 [Einstein@Home] Not requesting tasks: don't need (job cache full) 09-Jun-2016 10:29:27 [Einstein@Home] Scheduler request completed 09-Jun-2016 10:29:27 [Einstein@Home] Completed result p2030.20131013.G180.42-04.88.N.b3s0g0.00000_1981_1 refused: result already reported as success 09-Jun-2016 10:29:27 [Einstein@Home] Completed result p2030.20131013.G180.42-04.88.N.b3s0g0.00000_1987_1 refused: result already reported as success 09-Jun-2016 10:29:27 [Einstein@Home] Completed result p2030.20131013.G180.42-04.88.N.b3s0g0.00000_1986_1 refused: result already reported as success 09-Jun-2016 10:29:27 [Einstein@Home] Completed result p2030.20131013.G180.42-04.88.N.b3s0g0.00000_1905_0 refused: result already reported as success 09-Jun-2016 10:29:27 [Einstein@Home] Generated new computer cross-project ID: 090e475dc4062ac9d5d81559b6629a8a 09-Jun-2016 21:19:46 [Einstein@Home] Computation for task p2030.20131013.G177.42+00.95.N.b2s0g0.00000_1576_0 finished 09-Jun-2016 21:19:46 [Einstein@Home] Starting task p2030.20131013.G177.42+00.95.N.b2s0g0.00000_1590_1 09-Jun-2016 21:19:49 [Einstein@Home] Started upload of p2030.20131013.G177.42+00.95.N.b2s0g0.00000_1576_0_0 09-Jun-2016 21:19:52 [Einstein@Home] Finished upload of p2030.20131013.G177.42+00.95.N.b2s0g0.00000_1576_0_0 09-Jun-2016 21:24:14 [Einstein@Home] Computation for task p2030.20131013.G180.42-04.88.N.b2s0g0.00000_2032_0 finished 09-Jun-2016 21:24:14 [Einstein@Home] Starting task p2030.20131013.G177.42+00.95.N.b2s0g0.00000_1592_0 09-Jun-2016 21:24:16 [Einstein@Home] Started upload of p2030.20131013.G180.42-04.88.N.b2s0g0.00000_2032_0_0 09-Jun-2016 21:24:19 [Einstein@Home] Finished upload of p2030.20131013.G180.42-04.88.N.b2s0g0.00000_2032_0_0 09-Jun-2016 21:54:07 [Einstein@Home] Computation for task p2030.20131013.G177.42+00.95.N.b2s0g0.00000_1591_0 finished 09-Jun-2016 21:54:07 [Einstein@Home] Starting task p2030.20131013.G177.42+00.95.N.b2s0g0.00000_1589_0 09-Jun-2016 21:54:10 [Einstein@Home] Started upload of p2030.20131013.G177.42+00.95.N.b2s0g0.00000_1591_0_0 09-Jun-2016 21:54:13 [Einstein@Home] Finished upload of p2030.20131013.G177.42+00.95.N.b2s0g0.00000_1591_0_0 09-Jun-2016 22:03:00 [Einstein@Home] Computation for task p2030.20131013.G177.42+00.95.N.b2s0g0.00000_1587_1 finished 09-Jun-2016 22:03:00 [Einstein@Home] Starting task p2030.20131013.G177.42+00.95.N.b2s0g0.00000_1721_0 09-Jun-2016 22:03:02 [Einstein@Home] Started upload of p2030.20131013.G177.42+00.95.N.b2s0g0.00000_1587_1_0 09-Jun-2016 22:03:05 [Einstein@Home] Finished upload of p2030.20131013.G177.42+00.95.N.b2s0g0.00000_1587_1_0 09-Jun-2016 22:20:22 [Einstein@Home] Sending scheduler request: To report completed tasks. 09-Jun-2016 22:20:22 [Einstein@Home] Reporting 4 completed tasks 09-Jun-2016 22:20:22 [Einstein@Home] Requesting new tasks for CPU 09-Jun-2016 22:20:28 [Einstein@Home] Scheduler request completed: got 1 new tasks 09-Jun-2016 22:20:28 [Einstein@Home] Completed result p2030.20131013.G177.42+00.95.N.b2s0g0.00000_1576_0 refused: result already reported as success 09-Jun-2016 22:20:28 [Einstein@Home] Completed result p2030.20131013.G180.42-04.88.N.b2s0g0.00000_2032_0 refused: result already reported as success 09-Jun-2016 22:20:28 [Einstein@Home] Completed result p2030.20131013.G177.42+00.95.N.b2s0g0.00000_1591_0 refused: result already reported as success 09-Jun-2016 22:20:28 [Einstein@Home] Completed result p2030.20131013.G177.42+00.95.N.b2s0g0.00000_1587_1 refused: result already reported as success 09-Jun-2016 22:20:30 [Einstein@Home] Started download of p2030.20131013.G177.28+00.72.N.b4s0g0.00000_1886.bin4 09-Jun-2016 22:20:30 [Einstein@Home] Started download of p2030.20131013.G177.28+00.72.N.b4s0g0.00000.zap 09-Jun-2016 22:20:33 [Einstein@Home] Finished download of p2030.20131013.G177.28+00.72.N.b4s0g0.00000.zap 09-Jun-2016 22:20:34 [Einstein@Home] Finished download of p2030.20131013.G177.28+00.72.N.b4s0g0.00000_1886.bin4
rpi3-2
09-Jun-2016 21:46:19 [Einstein@Home] Completed result p2030.20131013.G177.42+00.95.N.b2s0g0.00000_1576_0 refused: result already reported as success 09-Jun-2016 21:46:19 [Einstein@Home] Completed result p2030.20131013.G180.42-04.88.N.b2s0g0.00000_2032_0 refused: result already reported as success 09-Jun-2016 21:46:19 [Einstein@Home] Completed result p2030.20131013.G177.42+00.95.N.b2s0g0.00000_1591_0 refused: result already reported as success 09-Jun-2016 21:46:19 [Einstein@Home] Completed result p2030.20131013.G177.42+00.95.N.b2s0g0.00000_1587_1 refused: result already reported as success 09-Jun-2016 21:46:21 [Einstein@Home] Started download of p2030.20131013.G177.28+00.72.N.b4s0g0.00000_718.bin4 09-Jun-2016 21:46:21 [Einstein@Home] Started download of p2030.20131013.G177.28+00.72.N.b4s0g0.00000.zap 09-Jun-2016 21:46:24 [Einstein@Home] Finished download of p2030.20131013.G177.28+00.72.N.b4s0g0.00000.zap 09-Jun-2016 21:46:26 [Einstein@Home] Finished download of p2030.20131013.G177.28+00.72.N.b4s0g0.00000_718.bin4
rpi3-1
09-Jun-2016 20:44:28 [Einstein@Home] Completed result p2030.20131013.G180.42-04.88.N.b3s0g0.00000_1981_1 refused: result already reported as success 09-Jun-2016 20:44:30 [Einstein@Home] Started download of p2030.20131013.G177.28+00.72.N.b4s0g0.00000_260.bin4 09-Jun-2016 20:44:30 [Einstein@Home] Started download of p2030.20131013.G177.28+00.72.N.b4s0g0.00000.zap 09-Jun-2016 20:44:33 [Einstein@Home] Finished download of p2030.20131013.G177.28+00.72.N.b4s0g0.00000.zap 09-Jun-2016 20:44:34 [Einstein@Home] Finished download of p2030.20131013.G177.28+00.72.N.b4s0g0.00000_260.bin4 09-Jun-2016 20:45:57 [Einstein@Home] Computation for task p2030.20131013.G180.42-04.88.N.b3s0g0.00000_1987_1 finished 09-Jun-2016 20:45:57 [Einstein@Home] Starting task p2030.20131013.G177.42+00.95.N.b2s0g0.00000_1589_0 09-Jun-2016 20:45:59 [Einstein@Home] Started upload of p2030.20131013.G180.42-04.88.N.b3s0g0.00000_1987_1_0 09-Jun-2016 20:46:03 [Einstein@Home] Finished upload of p2030.20131013.G180.42-04.88.N.b3s0g0.00000_1987_1_0 09-Jun-2016 21:46:51 [Einstein@Home] Sending scheduler request: To report completed tasks. 09-Jun-2016 21:46:51 [Einstein@Home] Reporting 1 completed tasks 09-Jun-2016 21:46:51 [Einstein@Home] Requesting new tasks for CPU 09-Jun-2016 21:46:56 [Einstein@Home] Scheduler request completed: got 1 new tasks 09-Jun-2016 21:46:56 [Einstein@Home] Completed result p2030.20131013.G180.42-04.88.N.b3s0g0.00000_1987_1 refused: result already reported as success 09-Jun-2016 21:46:58 [Einstein@Home] Started download of p2030.20131013.G177.28+00.72.N.b4s0g0.00000_725.bin4 09-Jun-2016 21:47:02 [Einstein@Home] Finished download of p2030.20131013.G177.28+00.72.N.b4s0g0.00000_725.bin4
My YouTube Channel: https://www.youtube.com/user/KF7IJZ
Follow me on Twitter: https://twitter.com/KF7IJZ
Completed Result Refused: result already reported as success me
)
I'm not quite sure exactly what you mean by the highlighted words. It's quite OK to have a BOINC + Einstein template (all the 'static' files thereof) included in your image but you need to be very wary of any files whose content is transitory, like the state file, or contents of slot directories.
If the installed client included a state file (client_state.xml), that file could have contained details (workunits, results, data files, etc) of the other host the image was produced from. Even if such files didn't exist in the Einstein project directory, their entries in the state file could have caused fresh copies of those files to be downloaded when you first fired up BOINC. The client would have unknowingly crunched the tasks specified in the state file and returned them. The server would reject them, recognising them as duplicates originally assigned to a different host. Maybe part of the defense mechanism against this is to immediately force a change in hostID for the errant host trying to return the duplicate work.
The above is just a guess as to what might be happening. Can you spell out in more detail, what BOINC (and related) stuff is included in the image?
Cheers,
Gary.
The most likely cause of this
)
The most likely cause of this is indeed if you used one pi to clone the others. Copying the client_state.xml from one host to another is not possible. Your image should only contain a basic boinc-client installation and maybe an account_einstein.phys.uwm.edu.xml file described here: https://einstein.phys.uwm.edu/weak_auth.php This will automatically add the client to your einstein@home account when the boinc-client is started. Note: the authenticator should not be made public!
The image had never been
)
The image had never been associated with Einstein@Home. When I say the boinc-client installed, I literally mean 'apt-get install boinc-client'. Configuration of each node (boinccmd --project_attach URL KEY) was done manually on each machine, so nothing E@H related should have propagated. That is why this is so baffling.
For context, the latest Jessie Lite image was downloaded and written to a card. I did raspi-config to set the machine up how it needed to be setup, did apt-get update/upgrade, installed boinc-client and that was it-this is the image that was cloned for each subsequent Pi. For each node I bring online, I change the host name manually and attach boinc to E@H manually. This means that the Pi won't attempt to get work until AFTER the hostname is set to a unique hostname.
Am I getting the feeling that I should detach each machine and try to re-associate to the project? It's just odd that all three nodes that came online had duplicate work to some degree, but not ALL work was duplicated - rpi3-1 and 2 have both earned credit at about the same rate.
CB: Your warning about the authenticator - did I do this by publishing my log files? I didn't see anything in there that looked protected.
My YouTube Channel: https://www.youtube.com/user/KF7IJZ
Follow me on Twitter: https://twitter.com/KF7IJZ
My waring was just a
)
My waring was just a precaution. Logfiles do not include the authenticator. I have to do some more digging in logfiles next week to see what's going on. Can you please send me the hostids of the three pi's and the mac addresses of those via PM? BOINC determines if a host is new by assuming that MACs differ between different hosts. I don't think that's the problem because you would have network problems if that where the case.
RE: My waring was just a
)
Thanks for the offer of help. I will get that information to you this evening once I am home. I agree that the MAC address should not be a problem because I use rpi3-0 as a DHCP/NAT server for the rest of the cluster, which is physically mounted on an 8 port switch. I create a host entry in dhcpd.conf for each node to always be assigned the same address, and I know that the MACs were different.
Anything else I can send that would be helpful? Log files?
Based on what I've seen today, it looks like rpi3-3 is still getting a bunch of dupes because it should have reported something by now. I will probably reimage it this evening and have it try to get fresh work.
Also, as I finish my video this weekend, I will be bringing 4 more Pi3 nodes online and will send along any shenanigans if I run in to them.
My YouTube Channel: https://www.youtube.com/user/KF7IJZ
Follow me on Twitter: https://twitter.com/KF7IJZ
I think I have found the
)
I think I have found the issue, maybe.
I brought rpi3-4through7 online today. All picked up the same work units. Looking at the client_state.xml, the id for each client was the same despite different host names, etc. My guess is that this is a global boinc identifier that was calculated on installation, prior to joining E@H. The fix was to remove boinc from each of the machines, delete /var/lib/boinc-client/ and reinstall. All wound up with unique IDs, and I don't seem to have duplicate work now. Time will tell.
My YouTube Channel: https://www.youtube.com/user/KF7IJZ
Follow me on Twitter: https://twitter.com/KF7IJZ
The Client generates a host
)
The Client generates a host cross project ID on installation time. That's why the image didn't work. A solution might be to stop the BOINC Client and delete the client_state.xml right after installation. Then create the image and when you start it on a new PC it will create a new client_state.xml with a new host CPID using the MAC of the new Pi instead of the one where the image was created on.
Thanks for the feedback and
)
Thanks for the feedback and help everyone. I had (wrongly) assumed that the CLient ID was generated when attaching to a project. I now have installing boinc as a step on each individual Pi.
8 Pi 2s and 8 Pi 3s are successfully and happily crunching away!
My YouTube Channel: https://www.youtube.com/user/KF7IJZ
Follow me on Twitter: https://twitter.com/KF7IJZ