Crash Recovery

If users are unable to log on to their network accounts, it is possible that the OSX server has crashed. More often than not, this is catastrophic (data-wise) and will require reinstallation of the OS and restoration from backups.

Initial troubleshooting steps

1. Try to connect
   1. Option A: Log in via Screen Sharing from another Mac
   2. Option B: Log in via SSH
      • The server can be via SSH reached at 216.128.150.40
2. Once logged in
   1. Check logs (mostly useless, require more information on which logs to check, etc.)
   2. Reboot the server by power cycling, or performing a restart while logged in.
      • This will most likely render the server completely unreachable, at which point you'll need to move to recovery.

Recovery steps

Initial recovery steps

1. Connect a monitor, keyboard, and mouse to the server
2. Reboot it, and hold down Command (⌘)-Option(alt)-R to boot into recovery mode.
   • If you're using a Windows keyboard, this is Win-Alt-R.
3. Select Disk Utility
4. Drives are listed in the following fashion:
   • Physical Drive
     • Logical Volume
   • Physical Drive
     • Logical Volume
   • Physical RAID Array
     • Logical RAID Volume
5. Select each drive in the list and run disk verification.
   • Disk verification is most likely to succeed on the individual physical disks and logical disks, and most likely to fail on the RAID array
6. If prompted, run disk repair
   • If disk repair fails, it will probably prompt you to reformat, reinstall the OS, and restore from backup.

Reformatting the hard drive

1. Select the Logical RAID Volume.
2. Select the Erase tab.
3. Format should be Mac OS Extended (Journaled)
4. Click the Erase button and confirm the prompt that pops up. Once it completes the reformat, you can move on to reinstalling the OS.

Reinstalling OS X

Hopefully you have a bootable thumb drive with an installation utility for the latest version of OSX, or the version of OSX you wish to be running. If not, this guide is not for you. Google that.

1. Plug the bootable thumb drive into the server
2. From within Recovery Mode, click the Apple icon in the top left corner and select Startup Disk
3. Select the thumb drive volume that appears in the Choose Startup Disk dialog.
4. Click Restart and confirm the prompt that pops up.
5. The server should now reboot into the installation utility.
6. Once the installation utility comes up, select Continue.
7. Select Continue on the EULA page and select Agree when the prompt pops up.
8. You will be presented with a choice of disks to install on. Select the RAID volume that was previously configured and click Continue
   • DO NOT SELECT THE THUMB DRIVE YOU ARE INSTALLING FROM.
9. Installation should begin, and once completed, the Mac should reboot.

Restoring from Backup

There are several options for restoring from backup. Not all of them may work for your situation, and you can expect to spend several hours waiting for a restoration to complete.

Restore during post-reinstallation configuration

This option may not be the best option, as it can potentially only present the most recent restore point which may be outside of the scope you are looking to restore from. That said, this is probably the best option of the three, and if the restore point is outside of the range you need, or is missing files, you will need to follow up with a Restore Using Time Machine.

1. Once installation is done, you will be presented with several configuration options. Configure them appropriately until you get to the Transfer Information to this Mac section.
2. Select From a Mac, Time Machine backup, or startup disk.
3. Select the Time Capsule server as the source.
4. When prompted for login credentials, if the username is incorrect, the login prompt will "bump" indicating a failure. If the password is incorrect, you will be presented with a warning that "There was a problem connecting to Time Capsule. Contact your system administrator."
5. Select the most recent backup available and click Continue to begin restoration

Restore using Time Machine

This option may not be the best option, as there may be critical files that you need that can only be restored while the OS is not using them, which only occurs when reinstalling the OS or when in Recovery Mode.

1. Open a Finder window
2. From the menu bar at the top of the screen, select Go and then select Computer.
   • Alternatively, the shortcut key is Shift-Command(⌘)-C.
3. Navigate to the primary OS drive (Mirror).
4. Launch Time Machine from Spotlight Search, Applications, or if present, click the icon at the top right hand side of the screen that looks like a clock with a counter-clockwise arrow and selecting Enter Time Machine.
5. The Finder window you opened should center on the screen and will be able to scroll back through backup repositories.
6. Find the files that you need to restore, select them, and press the Restore button.
   • NOTE: Files in use by the operating system, such as the Administrator user directory or any hidden system directories cannot be restored.
7. Once Restore is selected, Time Machine will copy the information from the backup to the correct location. Depending on the amount of data to transfer, this may take several hours.

Restore from Recovery Mode

This might appear to be an option, but it doesn't work. Don't do it. Ever.