Global Correlation S5 search #1 1.05 gets stuck

Forums › Problems and Bug Reports

froehli
froehli
Joined: 21 Dec 06
Posts: 7
Credit: 1532293
RAC: 0
27 Oct 2010 18:24:16 UTC
Topic 195411
(moderation:
    )

    Hi,

    I just recently reanimated my BOINC accounts, and started crunching for Einstein@Home again on my AMD Athlon 64 3000+, running Ubuntu 10.10 64 Bit.
    I now noticed that the last two of the above mentioned tasks "got stuck" at varying percentages (the first at about 28%, the second just now at 10%); meaning that they don't seem to compute (CPU usage is at 0-20%), BOINC can't switch away from them, and the "Complete-Percentage" doesn't increase anymore; the currently affected task is http://einsteinathome.org/task/201163436

    Regarding technical issues in BOINC, I have to admit I'm rather a noob, so I'm hoping anybody in here might be able to help me: Is this a known problem, is there anything I can do about it?
    If you need any more information from my side, I'll be glad to provide it.

    > uname -a
    Linux Desktop 2.6.35-22-generic #35-Ubuntu SMP Sat Oct 16 20:45:36 UTC 2010 x86_64 GNU/Linux

    package versions:
    boinc 6.10.58+dfsg-3
    boinc-client 6.10.58+dfsg-3
    boinc-manager 6.10.58+dfsg-3

    Just as a side note: The first three tasks I completed after signing up again, finished with computation errors, and I don't know why (e.g. http://einsteinathome.org/task/201339110) - could that have to do anything with this? I'm not aware of any special conditions why the computation should fail (the CPU was definitely not too hot; however it already is quite old - > 3 years; could it be defective?)

    tullio
    tullio
    Joined: 22 Jan 05
    Posts: 2118
    Credit: 61407735
    RAC: 0

    Global Correlation S5 search #1 1.05 gets stuck

    27 Oct 2010 23:37:33 UTC
    Message 100285
    (moderation:
      )

      Are you sure you have the compatibility libraries installed? Einstein is a 32-bit application.
      Tullio

      froehli
      froehli
      Joined: 21 Dec 06
      Posts: 7
      Credit: 1532293
      RAC: 0

      I guess you're referring to

      28 Oct 2010 10:09:21 UTC
      Message 100286
      (moderation:
        )

        I guess you're referring to this? Yes, I do have the required packages installed; although I only yesterday installed the last two, libstdc++5 and freeglut3 (so they were not installed when I started computing for E@H); But I'd guess the first two are the more important ones, and shouldn't the task refuse to run (or fail with an error) if a required library isn't found?

        tullio
        tullio
        Joined: 22 Jan 05
        Posts: 2118
        Credit: 61407735
        RAC: 0

        RE: I guess you're

        28 Oct 2010 19:46:47 UTC
        Message 100287 in response to message 100286
        (moderation:
          )

          Quote:
          I guess you're referring to this? Yes, I do have the required packages installed; although I only yesterday installed the last two, libstdc++5 and freeglut3 (so they were not installed when I started computing for E@H); But I'd guess the first two are the more important ones, and shouldn't the task refuse to run (or fail with an error) if a required library isn't found?


          Yes, it exits with an error code.

          froehli
          froehli
          Joined: 21 Dec 06
          Posts: 7
          Credit: 1532293
          RAC: 0

          Hm, with me, it clearly

          4 Nov 2010 6:40:44 UTC
          Message 100288
          (moderation:
            )

            Hm, with me, it clearly doesn't abort right at the beginning, so I'd say this can't be it. Anybody else got any ideas? This is really a showstopper for me; since it's preventing any further BOINC tasks from running, I'm afraid I'll have to suspend E@H project for now until I know a solution.

            Gundolf Jahn
            Gundolf Jahn
            Joined: 1 Mar 05
            Posts: 1079
            Credit: 341280
            RAC: 0

            RE: Hm, with me, it clearly

            4 Nov 2010 8:40:29 UTC
            Message 100289 in response to message 100288
            (moderation:
              )

              Quote:
              Hm, with me, it clearly doesn't abort right at the beginning, so I'd say this can't be it...


              It might not abort, but the hang seems to happen right at the beginning, as all 5 tasks still listed show only the line in the protocol.

              Are you sure that the 28%/10% tasks did compute anything and not just counted up elapsed time?

              Does the computation continue after a reboot? If you look into the corresponding slot directory (beneath the BOINC data directory), there should be a stderr file with information on that in it.

              There have been threads on the forum about AMD processors having trouble getting past the initialisation phase.

              Gruß,
              Gundolf

              Computer sind nicht alles im Leben. (Kleiner Scherz)

              tullio
              tullio
              Joined: 22 Jan 05
              Posts: 2118
              Credit: 61407735
              RAC: 0

              My Opteron 1210 is crunching

              4 Nov 2010 10:14:34 UTC
              Message 100290 in response to message 100289
              (moderation:
                )

                My Opteron 1210 is crunching AQUA, which is multithreading, Einstein both versions and QMC. Other projects are not giving me work, but a CPDN unit errored only after 4600 hours. It may not be the fastest CPU but it is very reliable.
                Tullio

                froehli
                froehli
                Joined: 21 Dec 06
                Posts: 7
                Credit: 1532293
                RAC: 0

                Hi Gundolf, thanks for your

                4 Nov 2010 16:30:48 UTC
                Message 100291
                (moderation:
                  )

                  Hi Gundolf, thanks for your effort, here are my answers to your questions:

                  Quote:
                  It might not abort, but the hang seems to happen right at the beginning, as all 5 tasks still listed show only the line in the protocol.


                  You mean the 5 tasks from that same machine?
                  I couldn't check on the machine over the last few days, so I had to pause/abort them - as I said, BOINC gets stuck with these tasks (it can't switch away from them anymore; meaning: usually BOINC switches application every now and then (every 60 minutes for me), but it didn't seem to be able to do that with these buggers, they would run until I stopped BOINC).

                  Quote:
                  Are you sure that the 28%/10% tasks did compute anything and not just counted up elapsed time?


                  I guess that they did compute something at first, because the "complete percentage" increased in the beginning, and they did use the CPU; but from the point on when that percentage got stuck, only the elapsed time did increase, and as the CPU usage was very low from then on, I assume they didn't compute anymore.

                  Quote:
                  Does the computation continue after a reboot?


                  No. Once it had reached the point where the complete percentage wouldn't increase anymore, I could restart BOINC, and when it then switched to the E@H application, it was immediately stuck again.

                  Quote:
                  If you look into the corresponding slot directory (beneath the BOINC data directory), there should be a stderr file with information on that in it.


                  I'm sorry to say that I have now already aborted that task, so its slot directory seems not to be available anymore :-(. If I have time I'll try to run some E@H tasks again to reproduce the issue, and then I'll check for these files!

                  Quote:
                  There have been threads on the forum about AMD processors having trouble getting past the initialisation phase.


                  Hm, interesting to hear, is there any specific information available about this problem - is it a processor problem, or a problem the application has with specific processors?

                  Bernd Machenschalk
                  Bernd Machenschalk
                  Moderator
                  Administrator
                  Joined: 15 Oct 04
                  Posts: 4343
                  Credit: 252697567
                  RAC: 35958

                  I don't think that there is a

                  7 Nov 2010 9:09:35 UTC
                  Message 100292
                  (moderation:
                    )

                    I don't think that there is a problem specific to a particular CPU, at least there's nothing I can think about (yet).

                    What _might_ happen is that one some system the initialization takes longer than the task is set to run ("switch tasks every" or various suspend reasons) and the task is removed from memory when suspended, so at next startup it has to do the initialization again. The reason for a long initialization is a file system interface that is slow compared to the demand, i.e. either a slow harddisk or many tasks starting at once. I'm working on some code that reads the data more efficiently, but it will take a while until it will show up in an E@H App.

                    What confuses me in the case here is stderr of the aborted tasks: Actually it should look like in http://einsteinathome.org/task/201163436, but all following aborted tasks on that machine just show the client version and nothing else. Could be a problem with the Client or the file system on that machine.

                    BM

                    BM

                    froehli
                    froehli
                    Joined: 21 Dec 06
                    Posts: 7
                    Credit: 1532293
                    RAC: 0

                    RE: What confuses me in

                    7 Nov 2010 16:59:40 UTC
                    Message 100293
                    (moderation:
                      )

                      Quote:

                      What confuses me in the case here is stderr of the aborted tasks: Actually it should look like in http://einsteinathome.org/task/201163436, but all following aborted tasks on that machine just show the client version and nothing else. Could be a problem with the Client or the file system on that machine.


                      I aborted them before they even started to run.

                      I now started E@H again, and have run into the same issue with this task.

                      Here's the content of the stderr.txt from the slots directory for this task:

                      2010-11-05 09:14:11.4630 (8607) [normal]: This program is published under the GNU General Public License, version 2
2010-11-05 09:14:11.4633 (8607) [normal]: For details see http://einstein.phys.uwm.edu/license.php
2010-11-05 09:14:11.4633 (8607) [normal]: This Einstein@home App was built at: Jun 19 2010 09:56:10

                      2010-11-05 09:14:11.4633 (8607) [normal]: Start of BOINC application '../../projects/einstein.phys.uwm.edu/einstein_S5GC1_1.05_i686-pc-linux-gnu__S5GCESSE2'.
                      command line: ../../projects/einstein.phys.uwm.edu/einstein_S5GC1_1.05_i686-pc-linux-gnu__S5GCESSE2 --Freq=1091.8174671 --FreqBand=0.05 --dFreq=6.71056161393e-06 --f1dot=-2.64248266531e-09 --f1dotBand=2.90673093185e-09 --df1dot=5.77553186099e-10 --skyGridFile=../../projects/einstein.phys.uwm.edu/skygrid_1100Hz_S5GC1.dat --numSkyPartitions=812 --partitionIndex=114 --tStack=90000 --nStacksMax=205 --gammaRefine=1399 --ephemE=../../projects/einstein.phys.uwm.edu/earth_05_09 --ephemS=../../projects/einstein.phys.uwm.edu/sun_05_09 --nCand1=10000 -o ../../projects/einstein.phys.uwm.edu/h1_1091.55_S5R4__114_S5GC1a_2_0 --gridType=3 --printCand1 --semiCohToplist -d1 --Dterms=8 --DataFiles1=../../projects/einstein.phys.uwm.edu/h1_1091.55_S5R4;../../projects/einstein.phys.uwm.edu/h1_1091.55_S5R7;../../projects/einstein.phys.uwm.edu/l1_1091.55_S5R4;../../projects/einstein.phys.uwm.edu/l1_1091.55_S5R7;../../projects/einstein.phys.uwm.edu/h1_1091.60_S5R4;../../projects/einstein.phys.uwm.edu/h1_1091.60_S5R7;../../projects/einstein.phys.uwm.edu/l1_1091.60_S5R4;../../projects/einstein.phys.uwm.edu/l1_1091.60_S5R7;../../projects/einstein.phys.uwm.edu/h1_1091.65_S5R4;../../projects/einstein.phys.uwm.edu/h1_1091.65_S5R7;../../projects/einstein.phys.uwm.edu/l1_1091.65_S5R4;../../projects/einstein.phys.uwm.edu/l1_1091.65_S5R7;../../projects/einstein.phys.uwm.edu/h1_1091.70_S5R4;../../projects/einstein.phys.uwm.edu/h1_1091.70_S5R7;../../projects/einstein.phys.uwm.edu/l1_1091.70_S5R4;../../projects/einstein.phys.uwm.edu/l1_1091.70_S5R7;../../projects/einstein.phys.uwm.edu/h1_1091.75_S5R4;../../projects/einstein.phys.uwm.edu/h1_1091.75_S5R7;../../projects/einstein.phys.uwm.edu/l1_1091.75_S5R4;../../projects/einstein.phys.uwm.edu/l1_1091.75_S5R7;../../projects/einstein.phys.uwm.edu/h1_1091.80_S5R4;../../projects/einstein.phys.uwm.edu/h1_1091.80_S5R7;../../projects/einstein.phys.uwm.edu/l1_1091.80_S5R4;../../projects/einstein.phys.uwm.edu/l1_1091.80_S5R7;../../projects/einstein.phys.uwm.edu/h1_1091.85_S5R4;../../projects/einstein.phys.uwm.edu/h1_1091.85_S5R7;../../projects/einstein.phys.uwm.edu/l1_1091.85_S5R4;../../projects/einstein.phys.uwm.edu/l1_1091.85_S5R7;../../projects/einstein.phys.uwm.edu/h1_1091.90_S5R4;../../projects/einstein.phys.uwm.edu/h1_1091.90_S5R7;../../projects/einstein.phys.uwm.edu/l1_1091.90_S5R4;../../projects/einstein.phys.uwm.edu/l1_1091.90_S5R7;../../projects/einstein.phys.uwm.edu/h1_1091.95_S5R4;../../projects/einstein.phys.uwm.edu/h1_1091.95_S5R7;../../projects/einstein.phys.uwm.edu/l1_1091.95_S5R4;../../projects/einstein.phys.uwm.edu/l1_1091.95_S5R7;../../projects/einstein.phys.uwm.edu/h1_1092.00_S5R4;../../projects/einstein.phys.uwm.edu/h1_1092.00_S5R7;../../projects/einstein.phys.uwm.edu/l1_1092.00_S5R4;../../projects/einstein.phys.uwm.edu/l1_1092.00_S5R7;../../projects/einstein.phys.uwm.edu/h1_1092.05_S5R4;../../projects/einstein.phys.uwm.edu/h1_1092.05_S5R7;../../projects/einstein.phys.uwm.edu/l1_1092.05_S5R4;../../projects/einstein.phys.uwm.edu/l1_1092.05_S5R7
                      2010-11-05 09:14:11.4671 (8607) [debug]: Flags: LAL_NDEBUG, HS_OPTIMIZATION, i386, SSE, SSE2, GNUC
                      2010-11-05 09:14:11.4672 (8607) [debug]: glibc version/release: 2.12.1/stable
                      2010-11-05 09:14:11.4674 (8607) [debug]: Set up communication with graphics process.
                      Code-version: %% LAL: 6.4.1.2 (CLEAN d21b154897fb961919c51dff7be4f3fafe22e1e7)
                      %% LALPulsar: 1.0.0.2 (CLEAN d21b154897fb961919c51dff7be4f3fafe22e1e7)
                      %% LALApps: 6.4.0.2 (CLEAN d21b154897fb961919c51dff7be4f3fafe22e1e7)

                      2010-11-05 09:14:11.6563 (8607) [normal]: Reading input data ... done.
                      % --- GPS reference time = 847063082.5000 , GPS data mid time = 847063082.5000
                      % --- Setup, N = 205, T = 90000s, Tobs = 56435059s, gammaRefine = 1399.000000
                      2010-11-05 09:14:41.1962 (8607) [normal]: INFO: No checkpoint h1_1091.55_S5R4__114_S5GC1a_2_0.cpt found - starting from scratch
                      % --- Cpt:0, total:834, sky:1/139, f1dot:1/6
                      2010-11-05 09:14:41.2086 (8607) [normal]: 1/1
                      2010-11-05 09:14:41.2087 (8607) [debug]: CG:9881 FG:10423949
                      c
                      2010-11-05 09:15:24.3201 (8607) [normal]: 1/2
                      2010-11-05 09:16:07.4674 (8607) [normal]: 1/3
                      c
                      2010-11-05 09:16:50.4504 (8607) [normal]: 1/4
                      2010-11-05 09:17:33.4217 (8607) [normal]: 1/5
                      c
                      2010-11-05 09:18:16.5502 (8607) [normal]: 1/6
                      2010-11-05 09:19:00.3855 (8607) [normal]: 2/1
                      c
                      2010-11-05 09:19:43.4768 (8607) [normal]: 2/2
                      2010-11-05 09:20:28.3560 (8607) [normal]: 2/3
                      c
                      2010-11-05 09:21:11.4041 (8607) [normal]: 2/4
                      2010-11-05 09:21:54.3842 (8607) [normal]: 2/5
                      c
                      2010-11-05 09:22:37.5452 (8607) [normal]: 2/6
                      2010-11-05 09:23:20.5133 (8607) [normal]: 3/1
                      c
                      2010-11-05 09:24:03.6381 (8607) [normal]: 3/2
                      2010-11-05 09:24:46.6734 (8607) [normal]: 3/3
                      c
                      2010-11-05 09:25:29.7402 (8607) [normal]: 3/4
                      2010-11-05 09:26:12.9254 (8607) [normal]: 3/5
                      c
                      2010-11-05 09:26:55.9453 (8607) [normal]: 3/6
                      2010-11-05 09:27:38.9686 (8607) [normal]: 4/1
                      c
                      2010-11-05 09:28:22.1627 (8607) [normal]: 4/2
                      2010-11-05 09:29:05.1999 (8607) [normal]: 4/3
                      c
                      2010-11-05 09:29:48.2113 (8607) [normal]: 4/4
                      2010-11-05 09:30:32.2730 (8607) [normal]: 4/5
                      c
                      2010-11-05 09:31:15.3074 (8607) [normal]: 4/6
                      2010-11-05 09:31:58.3258 (8607) [normal]: 5/1
                      c
                      2010-11-05 09:32:41.5113 (8607) [normal]: 5/2
                      2010-11-05 09:33:24.9035 (8607) [normal]: 5/3
                      c
                      2010-11-05 09:34:08.1019 (8607) [normal]: 5/4
                      2010-11-05 09:34:51.1207 (8607) [normal]: 5/5
                      c
                      2010-11-05 09:35:34.4216 (8607) [normal]: 5/6
                      2010-11-05 09:36:17.6294 (8607) [normal]: 6/1
                      c
                      2010-11-05 09:37:00.5839 (8607) [normal]: 6/2
                      2010-11-05 09:37:43.5777 (8607) [normal]: 6/3
                      c
                      2010-11-05 09:38:26.7055 (8607) [normal]: 6/4
                      2010-11-05 09:39:09.7292 (8607) [normal]: 6/5
                      c
                      2010-11-05 09:39:52.7153 (8607) [normal]: 6/6
                      2010-11-05 09:40:36.7069 (8607) [normal]: 7/1
                      c
                      2010-11-05 09:41:19.6855 (8607) [normal]: 7/2
                      2010-11-05 09:42:02.7852 (8607) [normal]: 7/3
                      c
                      2010-11-05 09:42:45.8029 (8607) [normal]: 7/4
                      2010-11-05 09:43:28.7365 (8607) [normal]: 7/5
                      c
                      2010-11-05 09:44:11.8673 (8607) [normal]: 7/6
                      2010-11-05 09:44:54.8284 (8607) [normal]: 8/1
                      c
                      2010-11-05 09:45:37.8983 (8607) [normal]: 8/2
                      2010-11-05 09:46:21.1148 (8607) [normal]: 8/3
                      c
                      2010-11-05 09:47:04.1581 (8607) [normal]: 8/4
                      2010-11-05 09:47:47.1706 (8607) [normal]: 8/5
                      c
                      2010-11-05 09:48:30.3781 (8607) [normal]: 8/6
                      2010-11-05 09:49:13.4090 (8607) [normal]: 9/1
                      c
                      2010-11-05 09:49:56.4588 (8607) [normal]: 9/2
                      2010-11-05 09:50:40.5312 (8607) [normal]: 9/3
                      c
                      2010-11-05 09:51:23.6628 (8607) [normal]: 9/4
                      2010-11-05 09:52:06.8783 (8607) [normal]: 9/5
                      c
                      2010-11-05 09:52:49.9125 (8607) [normal]: 9/6
                      2010-11-05 09:53:33.2650 (8607) [normal]: 10/1
                      c
                      2010-11-05 09:54:16.4130 (8607) [normal]: 10/2
                      2010-11-05 09:54:59.3931 (8607) [normal]: 10/3
                      c
                      2010-11-05 09:55:42.3755 (8607) [normal]: 10/4
                      2010-11-05 09:56:25.5349 (8607) [normal]: 10/5
                      c
                      2010-11-05 09:57:08.5543 (8607) [normal]: 10/6
                      2010-11-05 09:57:51.5302 (8607) [normal]: 11/1
                      c
                      2010-11-05 09:58:34.6584 (8607) [normal]: 11/2
                      2010-11-05 09:59:17.6395 (8607) [normal]: 11/3
                      c
                      2010-11-05 10:00:00.6261 (8607) [normal]: 11/4
                      2010-11-05 10:00:44.6133 (8607) [normal]: 11/5
                      c
                      2010-11-05 10:01:27.6133 (8607) [normal]: 11/6
                      2010-11-05 10:02:10.7502 (8607) [normal]: 12/1
                      c
                      2010-11-05 10:02:53.7156 (8607) [normal]: 12/2
                      2010-11-05 10:03:36.7114 (8607) [normal]: 12/3
                      c
                      2010-11-05 10:04:19.8333 (8607) [normal]: 12/4
                      2010-11-05 10:05:02.8131 (8607) [normal]: 12/5
                      c
                      2010-11-05 10:05:45.7618 (8607) [normal]: 12/6
                      2010-11-05 10:06:28.9025 (8607) [normal]: 13/1
                      c
                      2010-11-05 10:07:11.8704 (8607) [normal]: 13/2
                      2010-11-05 10:07:55.0006 (8607) [normal]: 13/3
                      c
                      2010-11-05 10:08:38.1526 (8607) [normal]: 13/4
                      2010-11-05 10:09:21.8131 (8607) [normal]: 13/5
                      c
                      2010-11-05 10:10:05.9129 (8607) [normal]: 13/6
                      2010-11-05 10:10:49.0049 (8607) [normal]: 14/1
                      c
                      2010-11-05 10:11:32.0754 (8607) [normal]: 14/2
                      2010-11-05 10:12:15.3100 (8607) [normal]: 14/3
                      c
                      2010-11-05 10:12:58.3584 (8607) [normal]: 14/4
                      2010-11-05 10:13:41.6975 (8607) [normal]: 14/5
                      c
                      2010-11-05 10:14:24.8371 (8607) [normal]: 14/6
                      2010-11-05 10:15:07.8082 (8607) [normal]: 15/1
                      c
                      2010-11-05 10:15:50.7827 (8607) [normal]: 15/2
                      2010-11-05 10:16:33.9258 (8607) [normal]: 15/3
                      c
                      2010-11-05 10:17:16.9262 (8607) [normal]: 15/4
                      2010-11-05 10:17:59.8750 (8607) [normal]: 15/5
                      c
                      2010-11-05 10:18:43.0065 (8607) [normal]: 15/6
                      2010-11-05 10:19:26.0125 (8607) [normal]: 16/1
                      c
                      2010-11-05 10:20:10.9033 (8607) [normal]: 16/2
                      c
                      2010-11-05 14:21:33.5926 (8607) [normal]: 16/3
                      2010-11-05 14:22:16.8050 (8607) [normal]: 16/4
                      c
                      2010-11-05 14:22:59.8435 (8607) [normal]: 16/5
                      2010-11-05 14:23:43.0883 (8607) [normal]: 16/6
                      c
                      2010-11-05 14:24:26.2239 (8607) [normal]: 17/1
                      2010-11-05 14:25:09.2040 (8607) [normal]: 17/2
                      c
                      2010-11-05 14:25:52.2021 (8607) [normal]: 17/3
                      2010-11-05 14:26:35.3118 (8607) [normal]: 17/4
                      c
                      2010-11-05 14:27:18.2977 (8607) [normal]: 17/5
                      2010-11-05 14:28:01.2800 (8607) [normal]: 17/6
                      c
                      2010-11-05 14:28:44.4343 (8607) [normal]: 18/1
                      2010-11-05 14:29:27.3861 (8607) [normal]: 18/2
                      c
                      2010-11-05 14:30:11.4006 (8607) [normal]: 18/3
                      2010-11-05 14:30:54.3623 (8607) [normal]: 18/4
                      c
                      2010-11-05 14:31:37.3508 (8607) [normal]: 18/5
                      2010-11-05 14:32:20.4910 (8607) [normal]: 18/6
                      c
                      2010-11-05 14:33:03.6947 (8607) [normal]: 19/1
                      2010-11-05 14:33:46.7093 (8607) [normal]: 19/2
                      c
                      2010-11-05 14:34:29.9672 (8607) [normal]: 19/3
                      2010-11-05 14:35:12.9084 (8607) [normal]: 19/4
                      c
                      2010-11-05 14:35:55.9001 (8607) [normal]: 19/5
                      2010-11-05 14:36:39.0494 (8607) [normal]: 19/6
                      c
                      2010-11-05 14:37:22.0131 (8607) [normal]: 20/1
                      2010-11-05 14:38:05.1486 (8607) [normal]: 20/2
                      c
                      2010-11-05 14:38:48.0875 (8607) [normal]: 20/3
                      2010-11-05 14:39:31.1176 (8607) [normal]: 20/4
                      c
                      2010-11-05 14:40:15.1367 (8607) [normal]: 20/5
                      2010-11-05 14:40:58.1012 (8607) [normal]: 20/6
                      c
                      2010-11-05 14:41:41.0855 (8607) [normal]: 21/1
                      2010-11-05 14:42:24.2454 (8607) [normal]: 21/2
                      c
                      2010-11-05 14:43:07.2326 (8607) [normal]: 21/3
                      2010-11-05 14:43:50.1912 (8607) [normal]: 21/4
                      c
                      2010-11-05 14:44:33.3349 (8607) [normal]: 21/5
                      2010-11-05 14:45:16.3144 (8607) [normal]: 21/6
                      c
                      2010-11-05 14:45:59.4838 (8607) [normal]: 22/1
                      2010-11-05 14:46:42.6222 (8607) [normal]: 22/2
                      c
                      2010-11-05 14:47:25.5892 (8607) [normal]: 22/3
                      2010-11-05 14:48:08.7317 (8607) [normal]: 22/4
                      c
                      2010-11-05 14:48:51.7092 (8607) [normal]: 22/5
                      2010-11-05 14:49:34.6336 (8607) [normal]: 22/6
                      c
                      2010-11-05 14:50:18.6422 (8607) [normal]: 23/1
                      2010-11-05 14:51:01.7190 (8607) [normal]: 23/2
                      c
                      2010-11-05 14:51:45.1072 (8607) [normal]: 23/3
                      2010-11-05 14:52:28.3694 (8607) [normal]: 23/4
                      c
                      2010-11-05 14:53:11.7536 (8607) [normal]: 23/5
                      2010-11-05 14:53:54.7541 (8607) [normal]: 23/6
                      c
                      2010-11-05 14:54:37.9299 (8607) [normal]: 24/1
                      2010-11-05 14:55:20.8797 (8607) [normal]: 24/2
                      c
                      2010-11-05 14:56:04.0374 (8607) [normal]: 24/3
                      2010-11-05 14:56:46.9600 (8607) [normal]: 24/4
                      c
                      2010-11-05 14:57:30.4331 (8607) [normal]: 24/5
                      2010-11-05 14:58:13.6187 (8607) [normal]: 24/6
                      c
                      2010-11-05 14:58:56.6687 (8607) [normal]: 25/1
                      2010-11-05 14:59:39.7363 (8607) [normal]: 25/2
                      c
                      2010-11-05 15:00:23.8466 (8607) [normal]: 25/3
                      2010-11-05 15:01:06.8978 (8607) [normal]: 25/4
                      c
                      2010-11-05 15:01:49.9464 (8607) [normal]: 25/5
                      2010-11-05 15:02:33.1193 (8607) [normal]: 25/6
                      c
                      2010-11-05 15:03:16.4951 (8607) [normal]: 26/1
                      2010-11-05 15:03:59.5301 (8607) [normal]: 26/2
                      c
                      2010-11-05 15:04:42.6711 (8607) [normal]: 26/3
                      2010-11-05 15:05:25.6416 (8607) [normal]: 26/4
                      c
                      2010-11-05 15:06:08.7507 (8607) [normal]: 26/5
                      2010-11-05 15:06:51.7106 (8607) [normal]: 26/6
                      c
                      2010-11-05 15:07:34.6716 (8607) [normal]: 27/1
                      2010-11-05 15:08:17.8053 (8607) [normal]: 27/2
                      c
                      2010-11-05 15:09:00.8029 (8607) [normal]: 27/3
                      2010-11-05 15:09:43.9810 (8607) [normal]: 27/4
                      c
                      2010-11-05 15:10:28.0807 (8607) [normal]: 27/5
                      2010-11-05 15:11:11.1317 (8607) [normal]: 27/6
                      c
                      2010-11-05 15:11:54.1246 (8607) [normal]: 28/1
                      2010-11-05 15:12:37.2392 (8607) [normal]: 28/2
                      c
                      2010-11-05 15:13:20.2097 (8607) [normal]: 28/3
                      2010-11-05 15:14:03.3316 (8607) [normal]: 28/4
                      c
                      2010-11-05 15:14:46.3365 (8607) [normal]: 28/5
                      2010-11-05 15:15:29.4046 (8607) [normal]: 28/6
                      c
                      2010-11-05 15:16:12.5342 (8607) [normal]: 29/1
                      2010-11-05 15:16:55.4923 (8607) [normal]: 29/2
                      c
                      2010-11-05 15:17:38.4856 (8607) [normal]: 29/3
                      2010-11-05 15:18:21.6288 (8607) [normal]: 29/4
                      c
                      2010-11-05 15:19:04.6226 (8607) [normal]: 29/5
                      2010-11-05 15:19:47.5933 (8607) [normal]: 29/6
                      c
                      2010-11-05 15:20:32.4859 (8607) [normal]: 30/1
                      2010-11-05 15:21:15.7313 (8607) [normal]: 30/2
                      c
                      2010-11-05 15:21:58.7966 (8607) [normal]: 30/3
                      c
                      2010-11-05 17:26:06.5242 (8607) [normal]: 30/4
                      2010-11-05 17:26:49.5105 (8607) [normal]: 30/5
                      c
                      2010-11-05 17:27:32.9331 (8607) [normal]: 30/6
                      2010-11-05 17:28:15.9603 (8607) [normal]: 31/1
                      c
                      2010-11-05 17:28:59.0191 (8607) [normal]: 31/2
                      2010-11-05 17:29:42.0789 (8607) [normal]: 31/3
                      c
                      2010-11-05 17:30:26.1845 (8607) [normal]: 31/4
                      2010-11-05 17:31:09.2434 (8607) [normal]: 31/5
                      c
                      2010-11-05 17:31:52.3010 (8607) [normal]: 31/6
                      2010-11-05 17:32:35.3604 (8607) [normal]: 32/1
                      c
                      2010-11-05 17:33:18.7319 (8607) [normal]: 32/2
                      2010-11-05 17:34:01.7587 (8607) [normal]: 32/3
                      c
                      2010-11-05 17:34:44.7473 (8607) [normal]: 32/4
                      2010-11-05 17:35:27.9496 (8607) [normal]: 32/5
                      c
                      2010-11-05 17:36:10.9282 (8607) [normal]: 32/6
                      2010-11-05 17:36:53.8538 (8607) [normal]: 33/1
                      c
                      2010-11-05 17:37:36.8267 (8607) [normal]: 33/2
                      2010-11-05 17:38:19.8022 (8607) [normal]: 33/3
                      c
                      2010-11-05 17:39:02.8518 (8607) [normal]: 33/4
                      2010-11-05 17:39:45.9730 (8607) [normal]: 33/5
                      c
                      2010-11-05 17:40:30.0161 (8607) [normal]: 33/6
                      2010-11-05 17:41:13.0147 (8607) [normal]: 34/1
                      c
                      2010-11-05 17:41:55.9829 (8607) [normal]: 34/2
                      2010-11-05 17:42:38.9530 (8607) [normal]: 34/3
                      c
                      2010-11-05 17:43:21.9566 (8607) [normal]: 34/4
                      2010-11-05 17:44:04.9399 (8607) [normal]: 34/5
                      c
                      2010-11-05 17:44:47.9297 (8607) [normal]: 34/6
                      2010-11-05 17:45:31.0822 (8607) [normal]: 35/1
                      c
                      2010-11-05 17:46:14.0756 (8607) [normal]: 35/2
                      2010-11-05 17:46:57.0598 (8607) [normal]: 35/3
                      c
                      2010-11-05 17:47:40.0661 (8607) [normal]: 35/4
                      2010-11-05 17:48:23.2293 (8607) [normal]: 35/5
                      c
                      2010-11-05 17:49:06.2309 (8607) [normal]: 35/6
                      2010-11-05 17:49:49.1720 (8607) [normal]: 36/1
                      c
                      2010-11-05 17:50:33.1937 (8607) [normal]: 36/2
                      2010-11-05 17:51:16.2581 (8607) [normal]: 36/3
                      c
                      2010-11-05 17:51:59.2549 (8607) [normal]: 36/4
                      2010-11-05 17:52:42.2209 (8607) [normal]: 36/5
                      c
                      2010-11-05 17:53:25.2943 (8607) [normal]: 36/6
                      2010-11-05 17:54:08.2598 (8607) [normal]: 37/1
                      c
                      2010-11-05 17:54:51.2325 (8607) [normal]: 37/2
                      2010-11-05 17:55:34.4031 (8607) [normal]: 37/3
                      c
                      2010-11-05 17:56:17.3945 (8607) [normal]: 37/4
                      2010-11-05 17:57:00.3857 (8607) [normal]: 37/5
                      c
                      2010-11-05 17:57:43.3781 (8607) [normal]: 37/6
                      2010-11-05 17:58:26.6659 (8607) [normal]: 38/1
                      c
                      2010-11-05 17:59:09.8009 (8607) [normal]: 38/2
                      2010-11-05 17:59:52.8115 (8607) [normal]: 38/3
                      c
                      2010-11-05 18:00:36.9915 (8607) [normal]: 38/4
                      2010-11-05 18:01:20.0572 (8607) [normal]: 38/5
                      c
                      2010-11-05 18:02:03.1531 (8607) [normal]: 38/6
                      2010-11-05 18:02:46.1936 (8607) [normal]: 39/1
                      c
                      2010-11-05 18:03:29.5286 (8607) [normal]: 39/2
                      2010-11-05 18:04:12.5154 (8607) [normal]: 39/3
                      c
                      2010-11-05 18:04:55.5010 (8607) [normal]: 39/4
                      2010-11-05 18:05:38.9070 (8607) [normal]: 39/5
                      c
                      2010-11-05 18:06:21.9162 (8607) [normal]: 39/6
                      2010-11-05 18:07:04.9196 (8607) [normal]: 40/1
                      c
                      2010-11-05 18:07:47.9030 (8607) [normal]: 40/2
                      2010-11-05 18:08:30.9004 (8607) [normal]: 40/3
                      c
                      2010-11-05 18:09:13.9260 (8607) [normal]: 40/4
                      2010-11-05 18:09:56.9084 (8607) [normal]: 40/5
                      c
                      2010-11-05 18:10:41.1263 (8607) [normal]: 40/6
                      2010-11-05 18:11:24.4364 (8607) [normal]: 41/1
                      c
                      2010-11-05 18:12:07.5763 (8607) [normal]: 41/2
                      2010-11-05 18:12:50.6215 (8607) [normal]: 41/3
                      c
                      2010-11-05 18:13:34.3232 (8607) [normal]: 41/4
                      2010-11-05 18:14:17.3503 (8607) [normal]: 41/5
                      c
                      2010-11-05 18:15:00.3514 (8607) [normal]: 41/6
                      2010-11-05 18:15:43.5297 (8607) [normal]: 42/1
                      c
                      2010-11-05 18:16:26.5410 (8607) [normal]: 42/2
                      2010-11-05 18:17:09.5405 (8607) [normal]: 42/3
                      c
                      2010-11-05 18:17:52.7252 (8607) [normal]: 42/4
                      2010-11-05 18:18:35.7705 (8607) [normal]: 42/5
                      c
                      2010-11-05 18:19:18.7702 (8607) [normal]: 42/6
                      2010-11-05 18:20:01.7544 (8607) [normal]: 43/1
                      c
                      2010-11-05 18:20:46.6455 (8607) [normal]: 43/2
                      2010-11-05 18:21:29.7049 (8607) [normal]: 43/3
                      c
                      2010-11-05 18:22:12.6815 (8607) [normal]: 43/4
                      2010-11-05 18:22:55.7852 (8607) [normal]: 43/5
                      c
                      2010-11-05 18:23:39.2184 (8607) [normal]: 43/6
                      2010-11-05 18:24:22.2282 (8607) [normal]: 44/1
                      c
                      2010-11-05 18:25:05.4162 (8607) [normal]: 44/2
                      2010-11-05 18:25:48.4018 (8607) [normal]: 44/3
                      c
                      2010-11-05 18:26:31.4128 (8607) [normal]: 44/4
                      c
                      2010-11-05 20:55:54.6836 (8607) [normal]: 44/5
                      2010-11-05 20:56:37.6772 (8607) [normal]: 44/6
                      c
                      2010-11-05 20:57:20.6611 (8607) [normal]: 45/1
                      2010-11-05 20:58:03.6461 (8607) [normal]: 45/2
                      c
                      2010-11-05 20:58:46.6513 (8607) [normal]: 45/3
                      2010-11-05 20:59:29.6067 (8607) [normal]: 45/4
                      c
                      2010-11-05 21:00:13.8530 (8607) [normal]: 45/5
                      2010-11-05 21:00:56.8659 (8607) [normal]: 45/6
                      c
                      2010-11-05 21:01:39.8627 (8607) [normal]: 46/1
                      2010-11-05 21:02:23.2084 (8607) [normal]: 46/2
                      c
                      2010-11-05 21:03:06.1716 (8607) [normal]: 46/3
                      2010-11-05 21:03:49.1358 (8607) [normal]: 46/4
                      c
                      2010-11-05 21:04:32.1421 (8607) [normal]: 46/5
                      2010-11-05 21:05:15.3155 (8607) [normal]: 46/6
                      c
                      2010-11-05 21:05:58.3035 (8607) [normal]: 47/1
                      2010-11-05 21:06:41.2731 (8607) [normal]: 47/2
                      c
                      2010-11-05 21:07:24.2597 (8607) [normal]: 47/3
                      2010-11-05 21:08:07.2371 (8607) [normal]: 47/4
                      c
                      2010-11-05 21:08:50.2091 (8607) [normal]: 47/5
                      2010-11-05 21:09:33.4821 (8607) [normal]: 47/6
                      c
                      2010-11-05 21:10:17.5142 (8607) [normal]: 48/1
                      2010-11-05 21:11:00.4711 (8607) [normal]: 48/2
                      c
                      2010-11-05 21:11:43.4275 (8607) [normal]: 48/3
                      2010-11-05 21:12:26.3967 (8607) [normal]: 48/4
                      c
                      2010-11-05 21:13:09.3850 (8607) [normal]: 48/5
                      2010-11-05 21:13:52.3589 (8607) [normal]: 48/6
                      c
                      2010-11-05 21:14:35.7049 (8607) [normal]: 49/1
                      2010-11-05 21:15:19.1905 (8607) [normal]: 49/2
                      c
                      2010-11-05 21:16:02.2335 (8607) [normal]: 49/3
                      2010-11-05 21:16:45.2565 (8607) [normal]: 49/4
                      c
                      2010-11-05 21:17:28.3359 (8607) [normal]: 49/5
                      2010-11-05 21:18:11.4072 (8607) [normal]: 49/6
                      c
                      2010-11-05 21:18:54.4562 (8607) [normal]: 50/1
                      2010-11-05 21:19:37.5235 (8607) [normal]: 50/2
                      c
                      2010-11-05 21:20:22.5207 (8607) [normal]: 50/3
                      2010-11-05 21:21:05.6331 (8607) [normal]: 50/4
                      c
                      2010-11-05 21:21:48.6874 (8607) [normal]: 50/5
                      2010-11-05 21:22:31.7361 (8607) [normal]: 50/6
                      c
                      2010-11-05 21:23:15.0698 (8607) [normal]: 51/1
                      2010-11-05 21:23:58.0438 (8607) [normal]: 51/2
                      c
                      2010-11-05 21:24:41.0413 (8607) [normal]: 51/3
                      2010-11-05 21:25:24.2186 (8607) [normal]: 51/4
                      c
                      2010-11-05 21:26:07.1897 (8607) [normal]: 51/5
                      2010-11-05 21:26:50.1639 (8607) [normal]: 51/6
                      c
                      2010-11-05 21:27:33.1572 (8607) [normal]: 52/1
                      2010-11-05 21:28:16.1553 (8607) [normal]: 52/2
                      c
                      2010-11-05 21:28:59.1504 (8607) [normal]: 52/3
                      2010-11-05 21:29:42.1404 (8607) [normal]: 52/4
                      c
                      2010-11-05 21:30:26.1803 (8607) [normal]: 52/5
                      2010-11-05 21:31:09.1560 (8607) [normal]: 52/6
                      c
                      2010-11-05 21:31:52.3380 (8607) [normal]: 53/1
                      2010-11-05 21:32:35.3132 (8607) [normal]: 53/2
                      c
                      2010-11-05 21:33:18.3153 (8607) [normal]: 53/3
                      2010-11-05 21:34:01.3033 (8607) [normal]: 53/4
                      c
                      2010-11-05 21:34:44.3207 (8607) [normal]: 53/5
                      2010-11-05 21:35:27.5081 (8607) [normal]: 53/6
                      c
                      2010-11-05 21:36:10.4824 (8607) [normal]: 54/1
                      2010-11-05 21:36:53.4442 (8607) [normal]: 54/2
                      c
                      2010-11-05 21:37:36.6027 (8607) [normal]: 54/3
                      2010-11-05 21:38:19.6299 (8607) [normal]: 54/4
                      c
                      2010-11-05 21:39:02.7899 (8607) [normal]: 54/5
                      2010-11-05 21:39:45.7668 (8607) [normal]: 54/6
                      c
                      2010-11-05 21:40:29.8213 (8607) [normal]: 55/1
                      2010-11-05 21:41:12.8096 (8607) [normal]: 55/2
                      c
                      2010-11-05 21:41:55.7780 (8607) [normal]: 55/3
                      2010-11-05 21:42:38.7688 (8607) [normal]: 55/4
                      c
                      2010-11-05 21:43:21.7854 (8607) [normal]: 55/5
                      2010-11-05 21:44:04.9953 (8607) [normal]: 55/6
                      c
                      2010-11-05 21:44:48.2095 (8607) [normal]: 56/1
                      2010-11-05 21:45:31.4717 (8607) [normal]: 56/2
                      c
                      2010-11-05 21:46:14.5237 (8607) [normal]: 56/3
                      2010-11-05 21:46:57.5522 (8607) [normal]: 56/4
                      c
                      2010-11-05 21:47:40.6411 (8607) [normal]: 56/5
                      2010-11-05 21:48:23.7118 (8607) [normal]: 56/6
                      c
                      2010-11-05 21:49:06.8155 (8607) [normal]: 57/1
                      2010-11-05 21:49:49.8725 (8607) [normal]: 57/2
                      c
                      2010-11-05 21:50:33.9832 (8607) [normal]: 57/3
                      2010-11-05 21:51:17.1040 (8607) [normal]: 57/4
                      c
                      2010-11-05 21:52:00.1612 (8607) [normal]: 57/5
                      2010-11-05 21:52:43.1977 (8607) [normal]: 57/6
                      c
                      2010-11-05 21:53:26.6913 (8607) [normal]: 58/1
                      2010-11-05 21:54:09.6651 (8607) [normal]: 58/2
                      c
                      2010-11-05 21:54:52.6595 (8607) [normal]: 58/3
                      2010-11-05 21:55:35.8679 (8607) [normal]: 58/4
                      c
                      2010-11-05 21:56:18.8888 (8607) [normal]: 58/5
                      2010-11-06 17:58:26.3669 (4625) [normal]: This program is published under the GNU General Public License, version 2
                      2010-11-06 17:58:26.3670 (4625) [normal]: For details see http://einstein.phys.uwm.edu/license.php
                      2010-11-06 17:58:26.3671 (4625) [normal]: This Einstein@home App was built at: Jun 19 2010 09:56:10

                      2010-11-06 17:58:26.3671 (4625) [normal]: Start of BOINC application '../../projects/einstein.phys.uwm.edu/einstein_S5GC1_1.05_i686-pc-linux-gnu__S5GCESSE2'.
                      command line: ../../projects/einstein.phys.uwm.edu/einstein_S5GC1_1.05_i686-pc-linux-gnu__S5GCESSE2 --Freq=1091.8174671 --FreqBand=0.05 --dFreq=6.71056161393e-06 --f1dot=-2.64248266531e-09 --f1dotBand=2.90673093185e-09 --df1dot=5.77553186099e-10 --skyGridFile=../../projects/einstein.phys.uwm.edu/skygrid_1100Hz_S5GC1.dat --numSkyPartitions=812 --partitionIndex=114 --tStack=90000 --nStacksMax=205 --gammaRefine=1399 --ephemE=../../projects/einstein.phys.uwm.edu/earth_05_09 --ephemS=../../projects/einstein.phys.uwm.edu/sun_05_09 --nCand1=10000 -o ../../projects/einstein.phys.uwm.edu/h1_1091.55_S5R4__114_S5GC1a_2_0 --gridType=3 --printCand1 --semiCohToplist -d1 --Dterms=8 --DataFiles1=../../projects/einstein.phys.uwm.edu/h1_1091.55_S5R4;../../projects/einstein.phys.uwm.edu/h1_1091.55_S5R7;../../projects/einstein.phys.uwm.edu/l1_1091.55_S5R4;../../projects/einstein.phys.uwm.edu/l1_1091.55_S5R7;../../projects/einstein.phys.uwm.edu/h1_1091.60_S5R4;../../projects/einstein.phys.uwm.edu/h1_1091.60_S5R7;../../projects/einstein.phys.uwm.edu/l1_1091.60_S5R4;../../projects/einstein.phys.uwm.edu/l1_1091.60_S5R7;../../projects/einstein.phys.uwm.edu/h1_1091.65_S5R4;../../projects/einstein.phys.uwm.edu/h1_1091.65_S5R7;../../projects/einstein.phys.uwm.edu/l1_1091.65_S5R4;../../projects/einstein.phys.uwm.edu/l1_1091.65_S5R7;../../projects/einstein.phys.uwm.edu/h1_1091.70_S5R4;../../projects/einstein.phys.uwm.edu/h1_1091.70_S5R7;../../projects/einstein.phys.uwm.edu/l1_1091.70_S5R4;../../projects/einstein.phys.uwm.edu/l1_1091.70_S5R7;../../projects/einstein.phys.uwm.edu/h1_1091.75_S5R4;../../projects/einstein.phys.uwm.edu/h1_1091.75_S5R7;../../projects/einstein.phys.uwm.edu/l1_1091.75_S5R4;../../projects/einstein.phys.uwm.edu/l1_1091.75_S5R7;../../projects/einstein.phys.uwm.edu/h1_1091.80_S5R4;../../projects/einstein.phys.uwm.edu/h1_1091.80_S5R7;../../projects/einstein.phys.uwm.edu/l1_1091.80_S5R4;../../projects/einstein.phys.uwm.edu/l1_1091.80_S5R7;../../projects/einstein.phys.uwm.edu/h1_1091.85_S5R4;../../projects/einstein.phys.uwm.edu/h1_1091.85_S5R7;../../projects/einstein.phys.uwm.edu/l1_1091.85_S5R4;../../projects/einstein.phys.uwm.edu/l1_1091.85_S5R7;../../projects/einstein.phys.uwm.edu/h1_1091.90_S5R4;../../projects/einstein.phys.uwm.edu/h1_1091.90_S5R7;../../projects/einstein.phys.uwm.edu/l1_1091.90_S5R4;../../projects/einstein.phys.uwm.edu/l1_1091.90_S5R7;../../projects/einstein.phys.uwm.edu/h1_1091.95_S5R4;../../projects/einstein.phys.uwm.edu/h1_1091.95_S5R7;../../projects/einstein.phys.uwm.edu/l1_1091.95_S5R4;../../projects/einstein.phys.uwm.edu/l1_1091.95_S5R7;../../projects/einstein.phys.uwm.edu/h1_1092.00_S5R4;../../projects/einstein.phys.uwm.edu/h1_1092.00_S5R7;../../projects/einstein.phys.uwm.edu/l1_1092.00_S5R4;../../projects/einstein.phys.uwm.edu/l1_1092.00_S5R7;../../projects/einstein.phys.uwm.edu/h1_1092.05_S5R4;../../projects/einstein.phys.uwm.edu/h1_1092.05_S5R7;../../projects/einstein.phys.uwm.edu/l1_1092.05_S5R4;../../projects/einstein.phys.uwm.edu/l1_1092.05_S5R7
                      2010-11-06 17:58:26.4123 (4625) [debug]: Flags: LAL_NDEBUG, HS_OPTIMIZATION, i386, SSE, SSE2, GNUC
                      2010-11-06 17:58:26.4124 (4625) [debug]: glibc version/release: 2.12.1/stable
                      2010-11-06 17:58:26.4128 (4625) [debug]: Set up communication with graphics process.
                      Code-version: %% LAL: 6.4.1.2 (CLEAN d21b154897fb961919c51dff7be4f3fafe22e1e7)
                      %% LALPulsar: 1.0.0.2 (CLEAN d21b154897fb961919c51dff7be4f3fafe22e1e7)
                      %% LALApps: 6.4.0.2 (CLEAN d21b154897fb961919c51dff7be4f3fafe22e1e7)

                      2010-11-06 17:58:26.6107 (4625) [normal]: Reading input data ... done.
                      % --- GPS reference time = 847063082.5000 , GPS data mid time = 847063082.5000
                      % --- Setup, N = 205, T = 90000s, Tobs = 56435059s, gammaRefine = 1399.000000
                      2010-11-06 17:58:55.6764 (4625) [debug]: Successfully read checkpoint:346
                      % --- Cpt:346, total:834, sky:58/139, f1dot:5/6
                      2010-11-06 17:58:55.6839 (4625) [normal]: 58/5
                      2010-11-06 17:58:55.6840 (4625) [debug]: CG:9881 FG:10423949
                      c
                      2010-11-06 17:59:38.8523 (4625) [normal]: 58/6
                      2010-11-06 18:00:23.0560 (4625) [normal]: 59/1
                      c
                      2010-11-06 18:01:06.1876 (4625) [normal]: 59/2
                      2010-11-06 18:01:49.2929 (4625) [normal]: 59/3
                      c
                      2010-11-06 18:02:32.3927 (4625) [normal]: 59/4
                      2010-11-06 18:03:15.5059 (4625) [normal]: 59/5
                      c
                      2010-11-06 18:03:58.6043 (4625) [normal]: 59/6
                      2010-11-06 18:04:41.7183 (4625) [normal]: 60/1
                      c
                      2010-11-06 18:05:24.9914 (4625) [normal]: 60/2
                      2010-11-06 18:06:08.0959 (4625) [normal]: 60/3
                      c
                      2010-11-06 18:06:51.2030 (4625) [normal]: 60/4
                      2010-11-06 18:07:34.3075 (4625) [normal]: 60/5
                      c
                      2010-11-06 18:08:17.4081 (4625) [normal]: 60/6
                      2010-11-06 18:09:00.5371 (4625) [normal]: 61/1
                      c
                      2010-11-06 18:09:43.7032 (4625) [normal]: 61/2
                      2010-11-06 18:10:27.8522 (4625) [normal]: 61/3
                      c
                      2010-11-06 18:11:10.9707 (4625) [normal]: 61/4
                      2010-11-06 18:11:54.0730 (4625) [normal]: 61/5
                      c
                      2010-11-06 18:12:37.1834 (4625) [normal]: 61/6
                      2010-11-06 18:13:20.2854 (4625) [normal]: 62/1
                      c
                      2010-11-06 18:14:03.3922 (4625) [normal]: 62/2
                      2010-11-06 18:14:46.4931 (4625) [normal]: 62/3
                      c
                      2010-11-06 18:15:29.7778 (4625) [normal]: 62/4
                      2010-11-06 18:16:12.9063 (4625) [normal]: 62/5
                      c
                      2010-11-06 18:16:56.0035 (4625) [normal]: 62/6
                      2010-11-06 18:17:39.1180 (4625) [normal]: 63/1
                      c
                      2010-11-06 18:18:22.2348 (4625) [normal]: 63/2
                      2010-11-06 18:19:05.3369 (4625) [normal]: 63/3
                      c
                      2010-11-06 18:19:48.4443 (4625) [normal]: 63/4
                      2010-11-06 18:20:33.5192 (4625) [normal]: 63/5
                      c
                      2010-11-06 18:21:16.6373 (4625) [normal]: 63/6
                      2010-11-06 18:21:59.7431 (4625) [normal]: 64/1
                      c
                      2010-11-06 18:22:42.8666 (4625) [normal]: 64/2
                      2010-11-06 18:23:25.9707 (4625) [normal]: 64/3
                      c
                      2010-11-06 18:24:09.0887 (4625) [normal]: 64/4
                      2010-11-06 18:24:52.1742 (4625) [normal]: 64/5
                      c
                      2010-11-06 18:25:35.4833 (4625) [normal]: 64/6
                      2010-11-06 18:26:18.5957 (4625) [normal]: 65/1
                      c
                      2010-11-06 18:27:01.7144 (4625) [normal]: 65/2
                      2010-11-06 18:27:44.8268 (4625) [normal]: 65/3
                      c
                      2010-11-06 18:28:27.9323 (4625) [normal]: 65/4
                      2010-11-06 18:29:11.0299 (4625) [normal]: 65/5
                      c
                      2010-11-06 18:29:54.1301 (4625) [normal]: 65/6
                      2010-11-06 18:30:38.2668 (4625) [normal]: 66/1
                      c
                      2010-11-06 18:31:21.3868 (4625) [normal]: 66/2
                      2010-11-06 18:32:04.5245 (4625) [normal]: 66/3
                      c
                      2010-11-06 18:32:47.6384 (4625) [normal]: 66/4
                      2010-11-06 18:33:30.7415 (4625) [normal]: 66/5
                      c
                      2010-11-06 18:34:13.8505 (4625) [normal]: 66/6
                      2010-11-06 18:34:56.9443 (4625) [normal]: 67/1
                      c
                      2010-11-06 18:35:40.2157 (4625) [normal]: 67/2
                      2010-11-06 18:36:23.3114 (4625) [normal]: 67/3
                      c
                      2010-11-06 18:37:06.4250 (4625) [normal]: 67/4
                      2010-11-06 18:37:49.5209 (4625) [normal]: 67/5
                      c
                      2010-11-06 18:38:32.6187 (4625) [normal]: 67/6
                      2010-11-06 18:39:15.7958 (4625) [normal]: 68/1
                      c
                      2010-11-06 18:39:58.9228 (4625) [normal]: 68/2
                      2010-11-06 18:40:43.0529 (4625) [normal]: 68/3
                      c
                      2010-11-06 18:41:26.4980 (4625) [normal]: 68/4
                      2010-11-06 18:42:09.6601 (4625) [normal]: 68/5
                      c
                      2010-11-06 18:42:52.7719 (4625) [normal]: 68/6
                      2010-11-06 18:43:35.8671 (4625) [normal]: 69/1
                      c
                      2010-11-06 18:44:18.9912 (4625) [normal]: 69/2
                      2010-11-06 18:45:02.2292 (4625) [normal]: 69/3
                      c
                      2010-11-06 18:45:45.3721 (4625) [normal]: 69/4
                      2010-11-06 18:46:28.4665 (4625) [normal]: 69/5
                      c
                      2010-11-06 18:47:11.5777 (4625) [normal]: 69/6
                      2010-11-06 18:47:54.6544 (4625) [normal]: 70/1
                      c
                      2010-11-06 18:48:37.7601 (4625) [normal]: 70/2
                      2010-11-06 18:49:20.8428 (4625) [normal]: 70/3
                      c
                      2010-11-06 18:50:04.9588 (4625) [normal]: 70/4
                      2010-11-06 18:50:48.1617 (4625) [normal]: 70/5
                      c
                      2010-11-06 18:51:31.2566 (4625) [normal]: 70/6
                      2010-11-06 18:52:14.3787 (4625) [normal]: 71/1
                      c
                      2010-11-06 18:52:57.4748 (4625) [normal]: 71/2
                      2010-11-06 18:53:40.5602 (4625) [normal]: 71/3
                      c
                      2010-11-06 18:54:23.6610 (4625) [normal]: 71/4
                      2010-11-06 18:55:06.9025 (4625) [normal]: 71/5
                      c
                      2010-11-06 18:55:50.0322 (4625) [normal]: 71/6
                      2010-11-06 18:56:33.1279 (4625) [normal]: 72/1
                      c
                      2010-11-06 18:57:16.2296 (4625) [normal]: 72/2
                      2010-11-06 18:57:59.3269 (4625) [normal]: 72/3
                      c
                      2010-11-06 18:58:42.4268 (4625) [normal]: 72/4
                      20:10:42 (4625): No heartbeat from core client for 30 sec - exiting

                      froehli
                      froehli
                      Joined: 21 Dec 06
                      Posts: 7
                      Credit: 1532293
                      RAC: 0

                      The really strange thing is

                      11 Nov 2010 17:23:37 UTC
                      Message 100294
                      (moderation:
                        )

                        The really strange thing is now: The task won't run again! Since I restarted BOINC when the task was stuck, it hasn't started the E@H-task mentioned above again!
                        So it's just sitting there (now since 4 days already), same "complete percentage" as then (51,438)!

                        Comment viewing options

                        Select your preferred way to display the comments and click "Save settings" to activate your changes.

                        Forums › Problems and Bug Reports