Getting Casper Unstuck

First off let me say that I love Virtual Machines. Occasionally this bites me. Today is the story of that as well as how I got out of it.

Problem: Attempting to upgrade JAMF Casper from one version to another resulted in “Checking for tables to rename” at the final step of the upgrade. Restarting the server and rerunning the installer did not resolve this.

After doing some basic Google searching I determined that the database was likely corrupted. Here is how I fixed it and upgraded Casper.

  1. Created a checkpoint in Hyper-V. My Casper server is a Linux server inside a Hyper V container

  2. Stop Tomcat: /etc/init.d/jamf.tomcat7 stop There are other methods to stopping / starting / restarting Tomcat. This is the method that works for my install.

  3. Restart mySQL: /etc/init.d/mysqld restart

  4. Repair the mySQL database: mysqlcheck —auto-repair —check —optimize —all-databases This will first check every table and then repair tables that need it. This will likely take a few minutes, depending on how much data it needs to go through. For my install, it took about 15 minutes.

  5. Start Tomcat /etc/init.d/jamf.tomcat7 start

  6. Upgrade Casper. I was trying to go from v9.2 to v9.21

My server got into this state because the hard drive filled up and I didn’t check it (my bad). This happened in the past when I setup database backups and forgot automatically delete old backups. This time it happened because the backups were getting bigger than originally expected in addition to some very old backups that had no automatically been deleted from the server.

This caused the drive to go into a read only state, partially corrupting the database. Really, these backups are not much more than mistake recovery (for me) as there are other methods for server backups.

My next steps are to set this server up with PRTG monitoring on the network to setup alerting for hard drive space.

Alternately another fix for this server would be increase the hard drive space, but I’m intentionally trying to keep this server pretty light. A future change will be to move the backup files offsite (to another server).

Links (a combination of these and Google helped me out today):