News

We think we have found a simple fix for the database problems

April 22, 2009

We think we have found a simple fix for the database problems. The database has grown to 45 GB in size and has gotten too large for the physical memory of the machine that hosts it. However it turns out that due to some mistakes made in the project operations during the past weeks, about 80% of the work in the database is already completed. So we are running a db_purge task tonight that should remove this already-completed work from the database, leaving only the work-in-progress still in place in the database.

The daemons worked through the backlog last night, but turning on the scheduler this morning for a few hours brought the DB down again. Some of the problems have been identified and fixed or worked around, but we are still working on some issues

April 21, 2009

The daemons worked through the backlog last night, but turning on the scheduler this morning for a few hours brought the DB down again. Some of the problems have been identified and fixed or worked around, but we are still working on some issues. The project has been shut down again for some DB analyis runs, it will probably be up and down again a number of times today as we fix the remaining issues one by one.

We still have serious problems with our database and are working on it. For the moment we disabled the scheduler to let the rest of the daemons work through the backlogs, i.e

April 20, 2009

We still have serious problems with our database and are working on it. For the moment we disabled the scheduler to let the rest of the daemons work through the backlogs, i.e. no work is sent out or can be reported.