Jim's Depository

Linux Containers are a mechanism by which you can run multiple virtual instances of linux inside a single machine.

• + more efficient than a virtual solution like Xen, VirtualBox, or KVM.
• - all instances share a single Linux kernel.
• - I think if you expect to have actively hostile users then you should not use containers. Just a gut feeling here. They are new-ish.

Following are my notes for using Linux containers with Debian Lenny. Read along on that IBM article I just linked. It is good stuff.

1. Following a catastrophic disk failure, I begin with a clean install on a new hard drive. (Bah on you Hitachi.)
2. Install lenny. Minimal install.
3. Add openssh and emacs for my sanity. Add build-essential for kernel building.
4. Try the 2.6.30 kernel from backports. Crash on ifconfig in an inner container. Try my own 2.6.30, same problem. Try 2.6.31-rc7. All is good. So checkout 2.6.31-rc7 (or probably a real 2.6.31 when it comes out) and build that kernel. I started with the .config file from the Debian 2.6.30 backports and answered the extra questions from there. My resulting .config file is attached to the end of this posting. I no longer use make-kpkg, it rotted away too far. I just install my kernel with “make install ; make modules_install” in the kernel build directory. You then have to “update-initramfs -k 2.6.31-rc7 -c” to get the init ram disk in place. Finally “update-grub”.
5. Fail to reboot. Ha ha! The linux kernel now names my hda drives as sda and fstab fails miserably. Use “ls -l /dev/disk/by-uuid” to find my UUIDs and use them in fstab.
6. The swap partition will still fail, but I can remake it with “mkswap -L SWAP /dev/hdaXXX” and then use LABEL=SWAP in fstab after the next boot.
7. Seek out the user space tools. Decide to try to 0.6.3-1 package from unstable. wget http://ftp.us.debian.org/debian/pool/main/l/lxc/lxc_0.6.3-1_i386.deb
8. Install manually with dpkg: dpkg -i lxc_0.6.3-1_i386.deb (note: As squeeze diverges, later versions of this may have linkage problems that won’t let them install on lenny)
9. Read the /usr/share/doc/lxc/README.Debian file. See some neat stuff.
10. Run lxc-checkconfig to verify the kernel has what it takes. Notice the cgroup memory controller is disabled, but don’t care.
11. Decision time: The cgroup filesystem has to get mounted somewhere. It doesn’t matter much where. I chose to put mine in /cgroup to avoid any mount order dependencies. Some people will have ontological problems with this. Imkdir /cgroup and add this to my /etc/fstab file: cgroup  /cgroup cgroup  defaults 0 0

12. I think I have all the pieces now. Pop over to LXC: Linux container tools and slide down to “Configure networking” and lets get going. I am going to have to change my ethernet to a bridge. Here is the before and after of my /etc/network/interfaces file

    # The primary network interface
    allow-hotplug eth1
    iface eth1 inet static
          address 172.18.36.4
          netmask 255.255.255.0
          gateway 172.18.36.1
    

    and the after

    # The primary network interface
    auto br0
    iface br0 inet static
          bridge_ports eth1
          bridge_maxwait 0
          address 172.18.36.4
          netmask 255.255.255.0
          gateway 172.18.36.1auto eth1
    iface eth1 inet manual
    

    Use ifdown -a and ifup -a to make the change, from the console. I failed here because I didn’t have bridge-utils installed. Make sure you do that first. If you are using dhcp just reboot and save yourself the heartache when your old dhcp trashes your address.

13. Try using lxc-debian to make a container. You can find this in /usr/share/doc/lxc/examples. Pull it out and uncompress it, then run it. You will need to install debootstrap first. It appears to be a trainwreck. I had to patch it a little to make it run. The patch is attached below.

14. Note: lxc-debian doesn’t allow you to specify your package source, it is hard coded to ftp.debian.org. If you have a local cache, or a faster source you will want to edit a line in there. debootstrap handles its own cache, so it only sucks once.

15. When you get the “Configuring Locales” dialog, take the time to scroll down to en_US.UTF-8 and turn it on, then select it on the next page to prevent a cloud of error lines. I suppose other ones work, but not generating and selecting makes noise.

16. The last line will tell you how to start your container, mine says: 

    You can run your container with the 'lxc-start -n test1'
    

17. You now have a container with a bunch of bad config files and no editor. Fortunately you can edit the afflicted files from the main linux.

18. Go look at your ROOT/etc/network/interfaces. Mine had a bad netmask.

19. Check your ROOT/etc/apt/sources.list, it is probably missing security and volatile.

20. Go ahead and start your container now and start working form the inside.

21. I am going to use aptitude, so “apt-get update ; apt-get install aptitude”

22. Let’s get the “normal” packages installed. “aptitude update” and then aptitude install "?priority(standard)" and aptitude install "?priority(important)"

23. There are still issues starting up as a daemon and some collisions with the system console, but I’ll address those in a later post.

This should be enough to get started.