I have a lot of multi-part posts going on over here, and who knows if I will ever finish all of them… but thats ok, I can write how I want to :-).
It’s been a rough week over here… Just before we decided to rebuild the Teacher server at MSA, it decided to beat us to the punch, and crash. And it crashed hard… In fact, crashed might not be the right word… Although I don’t know exactly *what* caused the problem, I do know that whatever it was caused a bunch of 0’s to be written to the drive… Not a good thing, as to my knowledge there is no “software” program that can recover a “zero’d” drive… And I’m not sure if anyone other than the Government and Colleges have the time or the means to do it regular hard drives.
That said, I’ve spent almost every night this week at MSA… Working at JR during the day for 8 to 10 hours, and then working at MSA for another 8 to 10 hours. It’s made for a really really long week of heartbreak and disappointment, as I kept running into “bad backups”. Needless to say though, we surrived (a little bit by luck). I’m going to document some of “best” practices that I’m learning/doing this time around as I rebuild the teacher server. Starting with this post, which is basically the setup of every feature and service we use here in order of operation (no really specific details):
- Install Any Bios Updates/Firmware Updates You Can Find
- Make sure the data on the drives is backed up before you proceed with format (unless it’s a new computer)
- Setup Raid – Ideally Raid 1, Raid 5 if you need a bit more performance. Raid 0+1 if you have the drives and drive space.
- Format the drives. Ideally into 2 partitions, a system and data drive. You may want 3 partitions if your going to put swap onto the third parition. Make your system drive at least twice what you think it’s going to need to be.
- Install Windows and Name Your Server. Pay Close attention to the name your server step… As after one of the steps, changing the name can result in additional hours of work.
- Install Latest Drivers
- Install Any Windows Updates
- Copy the files on the windows CD to a folder on the c: drive… this will save you some time in all future installs. Especially if your doing the installs via remote desktop in another room :-).
- Install DNS
- Install DHCP
- Install Routing and Remote Access (If you intend of having a VPN or routing your connections through this server).
- Install Active Directory (DCPromo)
- Install Exchange Server (Careful with this one… Follow all the steps to doing this properly. If it says reboot, REBOOT! A backup of AD might save you some time… Do a typical install. Don’t install extra stuff if you don’t need it. If you make a mistake in setup, be prepared to reinstall the entire exchange server.)
- Apply Exchange Server Service Packs
- Run Windows Update 1 more time (just in case).
- Install Printers and possibly Windows Print Server for some extra fun.
- Activate Volume Shadow Copy, particularly on the Data store (where the shares will be held). This acts sort of like a real time backup for those people who decide to delete files off their network share and then say “whoops”
- Install Windows File Service (If you want… we use it for NFS, you can use straight file sharing though)
- Install Windows Deployment Services (and setup your images if you have them ready to go… Image can wait though, this whole step can).
- Setup your File Shares
- Setup Backups…. This is the big one. SETUP BACKUPS… 3-2-1 Backups. 3 Copies. 2 Mediums. 1 Offsite. Here’s some more recommendations on this particular topic:
- Use good backup software such as Cobian Backup. Your backup sever should be able to rotate your backups. Create logs of what it did. As well as execute tasks after the backup has been performed.
- Break your backups up by size and usefulness. For example, I have all the backups broken into 5 users each, and different users are backed up on different days (Staff files are backed up 3 times a week). Important files are backed up daily. I determine how many “versions” of the file to keep backed up based on how important the data is as well. I do not recommend full system backups, except as a “4th” backup source… simply because depending on how your backup is stored, you could end up corrupting your entire backup. It’s also not easy to quickly restore the parts you need (again, depending on how the full backup is implemented). It’s almost more time consuming. It also does’t aid with the next thing.
- Offsite backups are a must… And thanks to the internet, there’s no reason not to do them with unlimited storage services like Amazon S3 and the Rackspace cloud. Data retention becomes a non-issue when storage is that cheap. However, keep the files your uploading small and easy to work with… that way your not bogging down your internet connection with backups all the time.
- Test all your backups! Regularly. If you can, have your backup software email you a report everytime it generates a backup. Read the report!
- Last but not least: Just remember there is no such thing as a perfect backup.
- Begin copying over data from your backup/restore that needs to be put into place.
- Configure Active Directory for the client machines and for the users (as well as exchange)… Basically all the user related stuff goes here.
That’s pretty much it, and what I’ve been doing this week. I’m still going in fact, completing some of the later steps relating to users. I’m also trying to get my backups to be as air tight as I think they need to be… In fact, I might post the backup policy I’ve put together just to emphasize how many backups MSA will now have to protect itself from what just happened this week.
Well back to work I go world. Thanks for the 1000 word break.