Christian Mesh
Back to Top


Serenity

What is Serenity?

A custom Linux distrobution that has ideological roots in Arch, Debian, and Gentoo. It has a hybrid source and binary package manager.

Why build Serenity?

As we hop between distros, many good ideas and pet peves start to emerge. For example, Gentoo has use flags for building packages from source which are usefull, however resource expensive. Debian is a quick and straitforward binary based system, however somewhat inflexible. All of these ideas result in Serenity.

Tools:

  • Forge - Package Creator

    Builds a package from a *.pie file. The pie files specify information about a package such as version, source, and build functions. Any ommited functions are replaced with default function that represent the majority of packages, IE: build() { make install }. The output is a spakg file which contains a pkginfo file (represents a built package), a md5sum list of files to be installed, and a tar file with all of the resultant files in it (fs.tar).

  • Wield - Package Installer

    Installs a package from a *.spakg file. It extracts the fs.tar to a temp directory, checks it against the md5sum file, and coppies all of the resultant files to the root directory.

  • Spack - Package Manager

    Coordinates the creation and installation of packages in a system. Manages source repositories using simple configuration files in /etc/spack/repos/. Currently spack supports http, ftp, rsync, and git repositories. Eventually it will manage binary repositories as well, for now it has a cache directory for packages it builds. Spack also handles dependency resolution for both build and install dependencies. Spack also handles removing packages and clearing them from the package cache.

What is working:

  • Dependency resolution (basic)
  • Successfully building a package
  • Successfully installing a package
  • Source repositories
  • pre and post install scripts

What is not working:

  • Dependencies (advanced)
  • Binary Repositores
  • Mark packages installed
  • System wide upgrades
  • Package search
  • Package info
  • Hooks
  • Flags

Stats:

  • 80+ commits
  • 2000+ lines added
  • 800+ lines removed
  • 9 contributors

VM Creation Script

What is it?

This script is designed to create, register and start a vm for the cslabs kvm infrastructure from a few simple question.

What is creates:

  • A debian vm image

    This is a full debian install, configured based on common projects in the labs, secured using hosts.allow and hosts.deny, with a list of packages installed. The image is sparse and has syslinux as a simple bootloader.

  • A libvirt configuration file

    This is a standard libvirt configuration file that represents the vm. It is added into the global vm list in libvirt and used to start the vm.

How it works:

  1. Creates a tmpfs to hold the image while it is created. A tmpfs is ideal if you have enough memory. It allows incredibly fast reads and writes for files in the directory. It is also more efficient space wise than using a sparse file from the get go, seeing as many files are created, modified and deleted.
  2. Bootstraps a debian install. Uses the debootstrap command to do a basic install to the tmpfs, pulling files from the clarkson mirror.
  3. Configures apt. Set all of the debian repositories commonly used in the labs and has them pointed at the Clarkson Mirror. It then sysnchronizes with the repositories and checks for updates.
  4. Installs basic packages such as vim and cron.
  5. Sets up hosts.allow and hosts.deny along with the networking interfaces
  6. Configures serial terminals, users, ntp and logwatch
  7. Creates a sparse disk file using truncate and losetup. It then coppies all of the files from the ramdisk to the sparse file.
  8. Unmounts the ramfs

Issues:

  • hosts.allow and hosts.deny should be moved to iptables config
  • losetup needs to be in the `latest` version for the -P (detect partitions) flag to be supported

Gitlab Setup

What is GitLab?

GitLab is a open source GitHub clone written in ruby on rails which depends on a mysql db and reddis server.

Why GitLab?

GitLab allows for unlimited public and private repositiories, along with keeping the controll of the repositories and users in the labs hands.

Setup:

Created a vm using the aforemetioned script and installed the latest version of GitLab. Followed the gitlab official instructionsinstructions. Installed a reddis and mysql server on the vm, which Corey Richardson later migrated to Jupiter.

Handoff:

The current maintainer is Corey Richardson. I handed off the server a few weeks after I set it up, as he expressed interest in running the server.

Zabbix Setup

I worked with Kyle Pederson on creating and setting up zabbix, the new monitoring system for the labs. As he took this for credit as well, I will leave the explanations to him.

Angular JS Workshop

What is AngularJS?

A client side MVC framwork written in JavaScript. It is an opensource project, backed by Google.

What was covered:

Scope Binding:

How to create and bind scope variable to html.

Advanced Binding:

How to use ng-repeat from lists created in the controller and put on the scope
How to bind controller functions on the scope to buttons

More Attributes:

Using ng-hide and ng-show to display elements based on scope variables
Using ng-even and ng-odd inside of a ng-repeat for staggered coloring

Routing and Dynamic Templates:

Using ng-tempate to encapsulate the display and formatting of objects in a seperate file
Using ng-template tied to a scope varaible to change how an object is displayed on a mouse click
Routing based on anchor url parameters to different sections of the website.

More Routing and Services

Routing to a 404 page when the url is not recognised.
Create a service that queries for weather information
Injecting that service into a controller

ITL Creation Script

Script to automate the creation of the base image for the ITL.

Usage:

./create_build.sh    (as root)
This generates a linux system at /itl-build.  The recommended way to use this system is rsync it to a server running nfs and configure a read only export.  Then boot the kernel with the standard nfsroot arguments (preferably over pxe).

Requirements:

  • debootstrap
  • chroot
  • systemd-nspawn
  • at least 6 GB at /itl-build (TODO location parameter)

How it works:

  1. Debootstrap

    Runs debootstrap targeting /itl-build with arguments specifying aditional packages to install.
    These packages are contained in packages_pre.sh.
    Several packages cannot be installed here and shall be installed later. This includes but is not limited to packages.
    requiring /proc to be mounted (debootstrap does not do this). Packages in repositories outside of main must be installed later.

  2. Additional Package Installation:

    Uses systemd-nspawn to install packages from packages.txt.
    Systemd-nspawn provides a better environment that chroot. It simulates a running system in more detail, including proc.
    It also will force all processes inside the environment to exit.

  3. Misc setup:

    Create users
    Configure packages
    Setup hosts.allow and hosts.deny
    etc

  4. Initramfs/AUFS: (the really cool part)

    We need a environment that has a read only root device that stores all modifications in ram. For this we use aufs and tmpfs. Aufs is a kernel module that can be used to union a ro and rw directory. Another thing to note is we need a more advanced mount utility than is provided by busybox.
    There are 2 scripts, one that is run when the initramfs is generated (hook), and one that is run toward the end of the
    initramfs scripts (just before the init system is started, systemd in this case) in the init-bottom directory.
    Hook:
    copies over aufs module , tmpfs module, rm, awk, free, grep, mount, and all of the libraries they depend upon.
    init-bottom:
    Moves ${rootmnt} to $ro_temp
    Mounts a tmpfs the size of the ram in the system at $rw_temp
    Creates the AUFS union from the two mount points.
    Moves $ro_temp and $rw_temp to /ro and /rw respectivley