Mozdev sysadmin meeting minutes for 2007-11-20

Present: David Boswell, (davidwboswell), Doug Warner (silfreed), Gerry Murphy (gjm), Chris Neale (cdn-work), Brian King (kinger), Eric Jung (ericjung)

Discussion was held publically in #mozdev

Discussed developer priorities

- a little Drupal work (theming fixes, install profile fixes) - Thanksgiving theme - replaced mirror content - going to start putting news on front page from blog feed - would like to serve files directly from if the content on the mirrors is outdated - will be redirecting users to their closest mirror soon - gjm picked next Monday @ 10:00 UTC for the BZ 3.0 upgrade

Discussed sysadmin priorities

- mailman is working without problems now - one of the list admins set the bounce address to the list admin address, causing a very large bounce queue to grow - mailman devs created a patch for this and it was applied to our servers - the bounced mail problem has been going on since May 5, 2007 - outgoing queue in mailman was increased from 1 process to 4 processes; postfix outbound smtp clients has been increased 50% - held message nags are still causing most of the daily email lag to lists - still tweaking rewrite rules on

Mirror outage

- content was accidentally deleted when stage was synced with production - Doug needed to fix up the mirror sync script because it wasn't maintained in CVS - the CVS tree still needs synced from production to staging; gjm will take care of that downtime

- TWS rebooted the server without plugging a monitor in, so we don't know if anything was visible on the console - gjm was unable to find anything in logs - no ideas about what the problem was, unfortunately

MySQL Backups

- setting up a replication server on is one option; would increase traffic to, but would totally elimninate the locking issue - splitting out bugzilla tables into its own database will speed up the dump of the tigris database and clean up the database at the same time - gjm an silfreed will coordinate on splitting the tigris db and updating code