Chef recipe: donating AWS Amazon resources
20 Jun 2016 15:40:48 UTC
Topic 198657
(moderation:
Hello,
"Chef" is basically an automated instruction to install, configure and run arbitrary software. At
https://github.com/smoe/chefcookbook.git
I provide respective instructions to auto-configure Amazon instances for Einstein@Home. I thought that falling back to AWS would be rather nice for all those folks who reduced (or even stopped) contributing when they bought a laptop but still somehow feel connected.
It works, the just started instances are here
https://einsteinathome.org/host/12288841/tasks
which I will run for a while. Please fork the project and adjust the authorisation key setting in
https://github.com/smoe/chefcookbook/blob/master/boinc/recipes/default.rb#L25
prior to starting the instances - per default the credits to me, no offense, I just want to test it all. I carry the thought with me for a while already to offer a service to run Einstein@Home on AWS instances for as long as some donated amount of money carries it, with credits collected by whatever authorisation code is provided.
If there is some positive feedback towards employing AWS for our cause then I will add the package for GPU computing and perform a credits-per-dollar/Euro evaluation. I am certainly aiming at spot instances (an auction for compute time that is dramatically cheaper), too. In my perception it is more some extra research-directed tax that I like paying. And if some charity like rechenkraft.net would adopt any such service to which donations are tax-deducible, then this should be helpful, too.
Cheers,
Steffen
Chef recipe: donating AWS Amazon resources
)
Hi Steffen,
I think the potential is great but since the Spot instance can terminate at every moment this can also be very volatile. What happens with the data the instance used when it is terminated? Can this one be restarted and the checkpoint reused? Does the instance get a warning that the time is up and it has to quit? That would help to somehow save the current state to a more persistent memory and the next sport instance can continue where the other left of.
Nice idea about the crowd funding aspect.
Hi Christian, Spot
)
Hi Christian,
Spot instances are not persistent by nature and while stackoverflow has instructions to get around those limitations, I do not like this. My very current mood says that we should not care but rather aim at shorter jobs. If we have a week's average run time and a GPU machine that spits out results every 20-30 minutes then everything is just fine, I tend to think.
But you are perfectly right. This is also about crowd funding the compute. And whatever we could do with AWS our another cloud provider would need to be cross-evaluated against the option to further fill your compute facilities or fund an in situ upgrade to the NVidia Pascal technology. You can possibly even find a new good homes among your contributors for any card you remove from your systems.
I just started another instance, now with 8 cores and a bit faster
https://einsteinathome.org/host/12288984/tasks
The Chef instructions now also install the NVidia packages, so it should all run on GPUs and detect them, it just needs to be tested, still.
Steffen