Volume of data.
12 Jan 2014 12:27:41 UTC
Topic 197337
(moderation:
Einstein@Home is occasionally sending me over 1GB of data in one go. Is it supposed to do this? (It's made up of dozens of files of 5.3 to 5.4MB each, with names like l1_0685.15_S6Directed)
Thanks,[/img]
Volume of data.
)
Are you sure that you get more than 1GB in one go?
You have a task which you received on 17 December and aborted on 7 January whose name indicated a frequency of 684.25Hz. For that task you would have needed a number of data files whose names (like the one you posted) would have indicated 'nearby' frequencies. I don't know exactly how many are needed but I doubt they would add up to anything like 1GB. These data files are retained since they can usually be used for quite a number of related tasks. The scheduler tries to send you tasks that depend on the same data if possible.
In your case (after 17th December) your next work request for CasA work was on 10th January, by which time there must have been no tasks remaining for the 685.xx frequency range so the scheduler sent a number of tasks for a higher frequency - 816.05Hz. So you would have received at that time, a whole batch of new data files. As you can see, you already have a number of tasks relevant to this data.
Over the coming days and weeks, further tasks you request should also be for the same frequency so there should be very few additional data downloads for a while. To see exactly how big the new data download really was, just check those data files whose name includes the frequency of 816.05Hz and above - by a reasonably small margin. I would be quite surprised if just these files added up to anything like 1GB.
You may very well have data files for other frequency ranges but they would have been downloaded at some other time and not as part of the most recent batch.
Cheers,
Gary.
Hi Gary, Thanks for the
)
Hi Gary,
Thanks for the quick reply. I had a problem with the 685Hz work units in that there was no progress showing on the task bars i.e still showing 0.00% after a week. I thought this might be down to the vast amount of data on these files (91 files of 5.3Mb each) so I aborted the project and deleted the files, restarting on the 10th Jan. Everything seems to be running smoothly now, I had a download of 28 files in the 816Hz region which fits in with what you're saying. However I also got all the 685Hz files back again. I suspect this represents a number of adjacent blocks of the spectrum and the way I got the project working again caused them to all be downloaded at once.
Thanks for your help, David.
So it seems the "one go"
)
So it seems the "one go" comment really applied to the resend of your deleted files.
I understand it to be normal BOINC behavior to resend files it "expects" to be present which turn up missing. You may find a project reset helpful if you wish to avoid this part of the behavior.
On the other hand, I also, have been pretty surprised at the quantity of S6extended files piling up on one of my machines. This machine had formally been running two GW tasks, and recently (when I got my power bill for December) cut back to one. So I was a bit surprised to find that there were about 8 Gigabytes of these files present. My personal response was to set a tiny work queue length (under .2 days), work down to almost nothing, then shut off task download, and work down to a single fractional task--then reset the project, resume fetch, and slowly bump up the queue size a little. So far, two and a half days later, I have just 86 S6Extended files, totalling just 461 Megabytes, all of which seem to be for a tight frequency range as they range from 0726.25 to 0728.35. This will be a bit covenient in the coming week when I plan to rebuild the machine with a new motherboard and new hard drive.
For me an extra problem is that these things are piling up on my backup disk. I need to add an exclusion for this file type. The Einstein project directory on my backup currently has a bit over 12 Gbytes, of which the great majority is S6extended files acquired in just the last couple of months. Thanks for making me think of this.
Hi archae86. "So it seems
)
Hi archae86.
"So it seems the "one go" comment really applied to the resend of your deleted files."
Not really the files I deleted were in excess of 0.5Gb and they all arrived at the same time. I did try the reset but nothing happened which is why I deleted them myself. Everything seems to be working normally now though. Now I have a better understanding of what these files are I can deal with volume. I've just had to set a download limit on the preferences so that I don't go over the limit set my providers. That's a good point about not needing to back these files up, I'll look into it.
David.