Speedy Server Setup

We tend to expand in bursts, so it’s helpful if I can be configuring multiple servers at once instead of dedicating an hour to one server, then another hour to another server. The most difficult part is removing all the unneeded packages from the boxes; installing the packages we want and configuring them is barely a quarter of the current setup script. Since we deal with so many hosts producing an image that we can have them create the server with isn’t exactly convenient; it’s been easier to take what they give us and then work from there.

The first thing we need to do is setup SSH key access to the new machine so that the rest of the install doesn’t need someone entering passwords. There might be a simpler way, but this is what we’ve got at the moment:

cat /home/lilypad/.ssh/id_rsa.pub | ssh root@$HOST "tee /dev/null > foo; mkdir .ssh 2> /dev/null; chmod 700 .ssh; chmod 600 foo; mv foo .ssh/authorized_keys"

So we pipe the SSH key over the SSH connection, write it to a file, make the .ssh directory and then move it to the correct location. At this point we now have easy SSH access to the machine, and we actually maintain active SSH master tunnels to all the machines on the network to reduce the connection lag when running scripts. More on how we do that in my next post.

Our first step on the new machine is to remove any software we explicitly know we don’t want and that will cause issues for our configuration. Things like Apache get nuked so that they don’t collide with the ports on which we run Squid. Then we update all the software on the box to the newest available versions in Debian 5 (a few of our boxes still start as Debian 4), then make the transition to Debian 6. At this point we still don’t have any of “our” packages installed so we start removing unneeded packages with a fairly simple set of rules:

1. If the package is on our whitelist of known needed packages, leave it.
2. If the package is on our blacklist of known unneeded packages, remove it.
3. If removing the package will only remove it and no other packages, remove it.
4. Ask!

Here’s the part of the script that handles those rules. The packages and packages-blacklist files are just lists of package names.

for i in `dpkg -l | sed -n s/"ii  \([^ ]*\).*"/"\\1"/p`
do
  grep "^$i$" setup/packages > /dev/null
  if [ $? -eq 0 ]; then
    echo KEEPING: $i
    continue
  fi

  grep "^$i$" setup/packages-blacklist > /dev/null
  if [ $? -eq 0 ]; then
    echo PURGING: $i
    apt-get -y purge $i
    continue
  fi

  echo $i | grep -v linux > /dev/null
  if [ $? -ne 0 ]; then
    echo ASKING: $i
    apt-get purge $i
    continue
  fi

  if [ `apt-get -s -qq remove $i | grep ^Remv | wc -l` -eq 1 ]; then
    echo PURGING: $i
    apt-get -y purge $i
    continue
  fi

  echo ASKING: $i
  apt-get purge $i
done

At this point it’s fairly rare that I get asked whether a package should be removed since I update the lists anytime a new package is encountered. Once that’s done we start copying our custom config files for each package and restart the program as needed. The install can run unattended and takes anywhere from 30-60 minutes depending on the speed of the downloads and the power of the machine, and I can be running multiple at once with little trouble.