ABP anti-center search
Bernd Machenschalk
Moderator
Administrator
Joined: 15 Oct 04
Posts: 4312
Credit: 250427373
RAC: 34918
29 Jul 2010 14:55:55 UTC
Topic 195214
(moderation:
We're currently transitioning to a new set of Arecibo data, known as the "anti-center" set, because the beams point away from the center of our galaxy.
This required to change some backend components (workunit generator, validator). Therefore currently we are not sending out ABP work.
We used this time of low load to upgrade the OS of einstein-dl.aei.uni-hannover.de, the machine that serves the ABP data. Everyhting went smoothly there and you shouldn't even have noticed (apart from possibly some transient upload rejects).
The update of the validator caused a little more trouble, though. When first run, some valid results got granted the wrong credit - even zero in some cases. The IDs of the 442 affected tasks have been recorded, I'll try to fix this manually in the next few days.
The tasks of the new anti-center search will (initially) run shorter than those of the previous center search, by about half the time. Initially we'll leave it that way, but if the short running tasks are stressing the DB too much, we'll increase the "bundle" size, meaning that there will be fewer longer running tasks. Credit and flops estimation will be adjusted.
BM
BM
ABP anti-center search
)
Do you have a rough estimate of when ABP work will be re-enabled?
Thanks.
Cheers,
Gary.
We generated and sent out the
)
We generated and sent out the first 314 tasks yesterday, and what we got back so far looks good at first glance. We'll probably enable automatic work generation later today.
BM
BM
OK, sounds good. Thanks for
)
OK, sounds good. Thanks for the response.
I have another question. You say that the tasks for the new 'anti-center' data will take about half the time that the old tasks took. Will the data for each new task still consist of 4x2MB files, ie, 8MB per task? If so we can expect to have to use twice as much bandwidth as previously for doing those tasks. In that case I will have to expand my use of bandwidth saving strategies on the GC1 tasks in order to compensate :-).
I've exacerbated my 'problem' by discovering how to make all my quad core machines speed up the crunching of the old ABP tasks by close to 25% anyway, so it will be really interesting to see what they will do with the new tasks.
I have around 30 quad core machines, a mixture of Q6600s, Q8400s and very recently some Phenom IIs (555BE x2s unlocked to x4s) which all were rather slow per core when compared with a dual core at around the same frequency. It puzzled me why they were so slow by comparison but I just put it down to 'memory contention' with 4 compute intensive tasks sharing the pool of available memory.
It bugged me enough to play around with adding extra memory and enabling dual channel mode. It doesn't make as much difference to a dual core machine (perhaps up to 10%) but it sure does to a quad - a 25% speedup. I'd hidden the solution from myself originally by deciding that each machine needed a max of 2GB and doing a bulk purchase of 2GB sticks at a good price that was quite a bit cheaper than buying twice as many 1GB sticks. I'd talked myself out of worrying about dual channel mode because of the very poor gains when I last tried it with the single core Athlon XP machines I started with a few years ago. I should have been smart enough to realise that it would be rather more important when using a fully loaded quad.
So a couple of weeks ago I started buying some 2x1GB kits on eBay and putting them in dual cores. The 2GB stick so retrieved has been going into a quad. I've still got a fair way to go. It's a bit dependent on how often the right sort of kit comes up at the right sort of price. I've been getting a bit impatient and started buying some new 2GB sticks as well :-).
It might sound like a pretty simple exercise to just add the extra memory and enjoy the speed increase. I've been taking the time to thoroughly clean the heat sink on each machine. On average most of them have been running around 6-9 months and the longer running ones had already previously had a good clean. I'm continuing to be disturbed at how easily the stock Intel heat sink gets clogged with a mat of fluff on the top surface. The computer repair business must be loving this as a source of ongoing revenue. Also it's often rather fiddly to find the right memory timings and frequency tweaks to get stable operation in dual channel mode. The machine has to be stress tested all over again and all up it can take many hours before being put back to work with a good guarantee of stability.
Cheers,
Gary.
About everything relevant is
)
About everything relevant is cut in half with the new data. The data files will be half the size of the previous ones, the Apps will need half the time (and a bit more than half the memory) to process them. We'll try to keep a bundle of 4 "micro-tasks" per task as long as the system can stand it, so the total runtime of a task should also be roughly half (bit more from general overhead). So the data volume per computing hour will stay about the same.
Edit: This might get "worse" for you if your machines do benefit more than proportional from the reduced memory usage.
BM
BM
Data files halving in size -
)
Data files halving in size - that's great! No problem then. Thanks very much!
To spread my consumption of bandwidth, I already have two accounts with my ISP. If all my quads end up consuming 25% more data, I could always get a third account. It costs only $35/month and I could get an extra 150GB allowance which should last me for a while :-).
Cheers,
Gary.
We're sending out ABP2
)
We're sending out ABP2 anti-center work. All workunit generator instances running.
BM
BM
Hi, the progress page has
)
Hi,
the progress page has been updated to show progress on the anti-center search as well. More details on some other small changes on the page in this forum post.
Cheers,
Ben
Einstein@Home Project
What is the reason reducing
)
What is the reason reducing the size of "anti-center" tasks in 2 times?
I guess that because of the reduction in the time of observation for one sky point / beam?
Observation time in seconds: 268.9792 (an example of "center" search WU)
Observation time in seconds: 134.4896 (an example of "anti-center" search WU)
But because of what it happened? So have decided on the Arecibo telescope to collect data with half-time of оbservation for each "anti-center" sky point? Or is this some kind of "optimization" performed on the E@H servers in the generation of jobs (part of the data is discarded)?
And another question - why it was decided to move to the "anti-center", if in the "central" part is still a lot of unprocessed sky points? It is planned to return to them later after all the "anti-center" the data (now being distributed only such a short "anti-center" jobs)?
Hi, the fact that the
)
Hi,
the fact that the anti-center pointings are shorter comes from the survey that was not designed by Einstein@Home, but the astronomer from the PALFA collaboration. Different models for the distribution of pulsars in our Galaxy were used and it turns out that most of those detectable are in the direction of the Galactic center and fewer in anti-center direction. So, to maximize the number of detections for a fixed amount of telescope time, it is best to spend more time in center directions. You can cut down the time in anti-center directions in two ways
a) do not cover the whole interesting part of the sky (bad… you might just miss a really bright pulsar because you didn't look at the right spot in the sky)
b) half the observation time per pointing (better… you still get the bright ones and might miss only a few weaker ones)
So, no data are discarded for the anti-center workunits.
The unfinished parts in the center search plots you see also are shorter observations from the early days of the survey, when PALFA did observe only 134 seconds in center direction as well. These empty spots on the sky will be filled as well, and those WUs will be generated at some point to the end of the anti-center search.
Cheers, Ben
Einstein@Home Project
Thanks for the explanation.
)
Thanks for the explanation.
But I can not understand this: "So, to maximize the number of detections for a fixed amount of telescope time, it is best to spend more time in center directions."
As far as I know the direction of the Arecibo telescope (to the center of the galaxy or the outer edge) is determined only by the earth's rotation around its axis. (As the telescope itself is stationary on the ground and can not move - only slightly shift the focus) so that to reduce the time of observation "anti-center" of course you can, but use this "saved time" for observing the center - physically impossible.