Spring 2009 - MP351 (2 Credits)
Roles
- ITL Director
- Network Administrator
- Member of CS Labs Admin. Board
- Server Setup / Services Admin.
Projects - End of Semester Summary
Network Status Server
As mentioned in my mid-semester summary, I rebuilt the Netstat VM to make the setup/upgrade process simpler. Currently, I'm in the process of converting Nagios to using SNMP to monitor everything instead of using NRPE and other add-ons. The benefit of this is that NRPE and the Nagios plugins won.t have to be custom compiled on each and every VM. All that will be required is installing SNMP (which every server supports) and setting up the SNMP config. Another major benefit of this is that setting up Cacti on Netstat will be extremely simple since Cacti uses SNMP in order to graph what you have it monitor. Of course, we are already monitoring the network usage of some of our servers and the labs in general through OIT's Cacti installation. So, my future plans for the network status server are to finish the Nagios conversion to SNMP, install Cacti, set up a page to display the status of all of our UPSs, and set up a page to show which VMs are running where.
Documentation Produced & Links
- New NetStat Setup Process
- Generic Nagios Install on CentOS 5
- Generic AWStats Install on CentOS 5
- CS Labs Network Usage
- CS Labs Individual Server Network Usage
Web Server & MySQL VMs
This semester I set up the Web1 and Mysql VMs, which are both used to provide web services in the labs. Web1 runs CentOS 5, Apache, and utilizes virtual hosts. It currently serves the COSI website, the CS Labs Website, Planet, the RRS website, the Xen website, and misc. project/management websites. The Mysql VM is used to provided MySQL databases in the labs. Currently, it doesn't host any databases at the moment but in the future it will host the database need for Laconica and other future projects.
Documentation Produced
Render Farm Project
This semester I continued to work with Roger Ignazio on the Render Farm for the Digital Arts Program. After some initial testing, we chose to go with Chris Hawkins' cluster/rendering suite because it made our setup quicker (since we were trying to have it ready in Feb.) and it made adding more notes easier since it uses PXE booting. Currently, we have roughly 20 nodes in the farm which is continuing to grow. Our future plans include finding an automated way of staring render jobs and a web submission page for jobs.
Mirror
This semester I've been working with Roger on the new Mirror server. Earlier in the semester, Brian and I worked on racking the system in the Unisys rack. We later wired the system and began trying to install CentOS on it; however, we ran into a few hardware issues with installing CentOS. Roger later decided to give Open Solaris a try so that we could utilize ZFS which looks quite promising for performance and redundancy. We eventually decided to go back to the original plan of CentOS since version 5.3 was about to release and we were concerned about potential problems that we were already having with Open Solaris and if we ran into problem in the future that we didn't know how to fix. So, I did some troubleshooting and discovered that we needed to adjust a few BIOS settings to fix the errors we were receiving. I also had to install a PCI card which added some IDE ports to the system that CentOS detected since the onboard IDE ports were not being seen. Finally, I was able to successfully install CentOS 5.3 on new mirror and I have begun resyncing all of our data. I hope for us to go live with new mirror by next week sometime.
Documentation Produced & Links
Server Management
In regards to server management, I've been working on pretty much everything. Aside from working on new mirror, I have done a CentOS 5.3 x64 build with Xen on two of the new Dom0 boxes (Xen1 & Xen2) which will hopefully be used to run all of our current VMs that currently run on the Blade Center. I have spent several hours figuring out the steps needed to migrate our VMs off of our current infrastructure which uses AoE and runs over the network, to the VMs running locally on each Dom0. I am now prepared to make the transition; however, I'm awaiting the outcome of a decision that might result in all of my efforts being in vain. I also rebuilt righteous and the three VMs (isengard, dns, & netstat) that run there. The main reason I rebuilt righteous (even though I just built it about a year ago and it had over 100 days of uptime), was due to some things that I've learned in terms of security and deploying a system. As for dns, I rebuilt/created this VM because we originally had three VMs for the same purpose but for each network. We had three VMs mainly because we were having network issues at the time they were created and I was hoping to be able to isolate things to each network. Therefore, there was no longer any need to have three separate VMs providing the service that one could handle. As for isengard, I rebuilt it so some extra protection could be put in place to make it more fault tolerant and because it needed to be upgraded anyways. I've also written some scripts that we can use for our daily backups in the future. I'm sure there are a few other things that I've done but I can't think of them at the momoent.
Documentation Produced
- Xen 1
- Xen 1 Setup Process
- Xen 2
- Xen 2 Setup Process
- New Righteous Setup Process
- New DNS Setup Process
- New Isengard Setup Process
Windows Lab Build
This semester I created the Windows build prior to the start of the semester. Afterwards, I handed it over to Jim Owens to take care of.
Presentations Given
- Sudo - Given on April 1st
- Clarkson's Network Monitoring Utilities - Given on April 1st
Future Projects
Spacewalk
I would like to set up Spacewalk to manage all of the CentOS systems that I maintain in the labs.
VPN Server
I still want to set up a VPN server in the labs.
HyperV & KVM
I still need try both of these out.
Windows Server 2008
I still need to try it out.
Open Solaris
Continue learning about it.
Projects - Mid-Semester Summary
Network Status Server
This semester I rebuilt the netstat VM again because of the latest Nagios and AWStats packages being available on the Dag Wieers repository as rpms. This made the setup much simpler and will make upgrading in the future extremely easy. My future plans for the server are to switch Nagios over to using SNMP for most its checks since I have plugins and configuration files that I can reference from OIT, to install Cacti to graph things for each server and the network, to get the server set up to monitor all of the UPSs, and to set up a way for the VM to determine which VMs are running where and to produce a web page showing such. I also recently added a link to OIT's Cacti installation that shows our network usage in and out of the labs as well as network usage for some specific hosts. I also wrote up two how tos for installing a basic Nagios installation and installing a basic AWStats installation.
New Web Server VM
This semester I finally got a chance to configure web1 to be set up to host many of the web sites in the labs. I still need to document everything on the wiki but as of right now it serves cosi.clarkson.edu, cslabs.clarkson.edu, planet.cosi.clarkson.edu, rrs.cslabs.clarkson.edu, xen.cslabs.clarkson.edu, and lab-build.cslabs.clarkson.edu.
Render Farm Project
Roger and I are still working on the render farm with the assistance of Ted Champagne from time to time. We made the decision at the beginning of the semester to go with a cluster/rendering suite that Chris Hawkins has provided to us for no charge. Presently, we have 14 rendering nodes and one master node. We will soon be gaining three other machines for the farm as well. For the future, we may move away from our current setup because of stability concerns and feature limitations. I am also looking at some network features of mental ray which would allow us to configure the CEC and potentially all PC labs on campus to act similar to the render farm.
Mirror
This semester I am working on the new mirror with Roger and Brian. We are currently trying to install CentOS 5 on the new server which has proved challenging because of the newer hardware which CentOS doesn't have the best support for yet.
Server Management
I continue to help maintain everything in the server room. I'm also working with everyone else to set up and configure the new servers. I'm also planning to rebuild Animal once current mirror is fully migrated to physical hardware. I'm also in the process of rebuilding some VMs including the three DHCP/DNS servers so that there is only one DHCP/DNS server (didn't do this originally because we were troubleshooting network issues on all three of our networks.
Windows Lab Build
This semester I created the Windows build like I always do; however, I am no longer the maintainer of the build. The Windows build team (Jim Owens, Justin Bennet, & Bob Appleton - But mostly just Jim) are the current maintainers of the build. I do have to say though that my life is a lot less stressful now that I no longer maintain the build especially since it took 40+ hours just to produce it.
Future Projects
VPN Server
Set up a VPN server in the labs.
ESXi, VMware Infrastructure, HyperV & KVM
Play with these.
Windows Server 2008
Play with this.
Open Solaris
Play with this as well