Jim's Depository

this code is not yet written

Setting up nginx and lua fastcgi on Debian Squeeze

The available pages I can google about this seem overly complicated. This one documents setting up nginx to run lua scripts on Debian Squeeze. These instructions are derived from http://www.unreliablepollution.net/p/other/howto-nginx-and-lua and a dozen remedial googleings.

  1. aptitude install nginx
    The config in /etc/nginx is good in squeeze. We will need to add the fastcgi handler, but that is to be expected. (I edited it to pull the root up to /var/www instead of in an nginx-default subdirectory.)
  2. aptitude install liblua5.1-wsapi-fcgi-1
    There is also a ‘0’ version, but I took 1 on the theory that newer is better.
  3. aptitude install liblua5.1-coxpcall0
    liblua5.1-wsapi-fcgi-1 probably needs a dependency on this. Things will go poorly for you without it.
  4. aptitude install lua5.1
    Can’t forget that.
  5. aptitude install spawn-fcgi
    Older instructions talk about libfastcgi-dev, but it isn’t available and I think they just wanted spawn-fcgi out of it anyway.

  6. Put a Lua file in /var/www/hello.lua
    The WSAPI is a little strange. Apparently you don’t just print your output, but rather coroutine.yield() it. I used this example program from http://keplerproject.github.com/wsapi/manual.html …

    module(…, package.seeall)

    function run(wsapi_env) local headers = { [“Content-type”] = “text/html” }

    local function hello_text() coroutine.yield(”<html><body>”) coroutine.yield(”<p>Hello Wsapi!</p>”) coroutine.yield(”<p>PATH_INFO: “ .. wsapi_env.PATH_INFO .. “</p>”) coroutine.yield(”<p>SCRIPT_NAME: “ .. wsapi_env.SCRIPT_NAME .. “</p>”) coroutine.yield(”</body></html>”) end

    return 200, headers, coroutine.wrap(hello_text) end

  7. spawn-fcgi -a 127.0.0.1 -p 9100 -F 4 -- /usr/bin/wsapi.fcgi
    Now we have handlers on port 9100. This will need to be in /etc/init.d/blahblahblah but this will do for now.
    Note: spawn-fcgi silently fails if you get the command wrong. Do a “ps” and make sure they are there. You will have a section like this if it succeeds…
    18554 ?        Ss     0:00 lua5.1 /usr/bin/wsapi.fcgi
    18555 ?        Ss     0:00 lua5.1 /usr/bin/wsapi.fcgi
    18556 ?        Ss     0:00 lua5.1 /usr/bin/wsapi.fcgi
    18557 ?        Ss     0:00 lua5.1 /usr/bin/wsapi.fcgi

  8. Edit /etc/nginx/sites-available/default to know how we want .lua files handled. I put in this section
    location \~ \^(.+\.lua)(.*)\$ { root /var/www/; fastcgi_pass 127.0.0.1:9100; fastcgi_index index.lua; fastcgi_split_path_info \^(.+\.lua)(.*)\$; fastcgi_param PATH_INFO \$fastcgi_path_info; fastcgi_param SCRIPT_FILENAME \$document_root\$fastcgi_script_name; include fastcgi_params; } Be careful! Make ‘root’ match your root!

  9. Restart nginx. /etc/init.d/nginx restart

  10. Load http://YOURSERVER/hello.lua in your browser. You should get something along the lines of:

    Hello Wsapi!

    PATH_INFO: / SCRIPT_NAME: /hello.lua

  11. Celebrate!

Now my thoughts: This may be more than I wanted. I have to think about the implications of a scrutinizer layer above my Lua program. I really wanted to just blast my standard output back up the fastcgi socket. All this yielding seems strange.

Very strange indeed.  I still don't understand why every Lua implementation (except for luasp.org) seems to be done backwards. Why doesn't anyone try to take the php approach and allow web designers to work on designing instead of doing strange script-fu just to use Lua?
For anyone else who stumbles across this, I found a better solution: http://www.marmottus.net/blog/2012/03/25/nginx-and-lua/

Glad it's not my bug.

3:14am, spring daylight savings time day. The church bells across the street have been ringing for 14 minutes.

Declining to use Coda on Debian

After my dismal failure at Pohmelfs, I shopped for alternative distributed files systems. I can’t tell if CODA is abandoned or not, but I’ll give it a try.

  1. Build into kernel 2.6.33, no problems.
  2. Debian has let their user mode utilities rot away.
  3. CMU has prebuilt .debs, including for Squeeze. That looks good.
  4. Add the CMU apt sources, looks like this now… # cat /etc/apt/sources.list.d/coda.list deb http://www.coda.cs.cmu.edu/debian lenny/
  5. aptitude update ; aptitude install coda-server, scary text about unsigned packages follows, but it is OK.
  6. Configure server with vice-setup
  7. Similar for the client machines, except aptitude install coda-client… ends with 00:35:02 Venus starting… 00:35:02 /coda now mounted
  8. Test client with CMU’s server: # cfs lv /coda/testserver.coda.cs.cmu.edu Status of volume 7f000000 (2130706432) named “/” Volume type is ReadWrite Looks good. I can read the sample documents.
  9. Test client with my server: Much hanging and server not founding. Most likely related to my “hostname” being an internal address unreachable by the client. There is a spot in the /etc/coda/*.conf file and in one in the /vice directory, but it still attempts to use the unreachable address and no hints in vice-setup on how to force an address.
  10. I temporarily change my hostname to the external interface address. No luck. The client does a pile of DNS lookups for SRV records for my host with _codasrv._udp prepended, then does one for a regular A record. Gets my address, but never tries to contact.
  11. Reboot client. Success! Now we can connect. Something evil must have been cached.
  12. Many failures to authenticate with clog until I realize that the example I’m working from only has an account of admin because that is what they gave vice-setup. Use the account I gave with the password “changeme” and I’m in.
  13. Find cpasswd, change that password.
  14. Try to make myself a user account. au -h MYHOST nu, coda is a pretty hostile system. It echoes my new password in the clear even though it knows how not to, then fails with AuthNameToId() --> AUTH_FAILED
  15. Find pdbtool. Use it to make my account from the server side, but I can’t see how to set the password. Go back and try au from a client, and now I can set my password.
  16. Notice I have no group. Use pdbtool to make a group with my name, program explodes with an assert failure. That’s going to be a problem.
  17. Unpack a linux kernel on the client. It flew until arch/arm, not it is crawling along at about 1 file per second. No load on server, no load on client, no load on network, just glacial progress.
  18. Abandon Code!

Coda is badly documented, user hostile, and not robust.

I do not trust it.

Failing to use POHMELFS in Linux 2.6.33

I’ve enjoyed reading zbr’s blog postings about POHMELFS and the Elliptics network for years. With POHMELFS in the staging directories of Linux 2.6.33 and me needing to share some files across several machines, it is time to try it out.

  1. Build myself some fresh new 2.6.33 kernels with POHMELFS enabled.
  2. Look for a HOWTO of sorts, fail.
  3. Check the server out with git: git clone http://www.ioremap.net/git/pohmelfs-server.git
  4. Follow the included README
  5. Fail, install autoconf.
  6. Fail, finally discover that --with-kdir-path=<mylinuxsourc>/drivers/staging/pohmelfs/ is the right answer.
  7. Fail, install libssl-dev.
  8. make
  9. make install
  10. Copy /usr/local/bin/cfg over to the client machines.
  11. Make a new filesystem to export mounted on /jim.
  12. Start the server: fserver -r /jim
  13. Go to a client machine:  #cfg -A add -a 1.1.1.1 -p 1025 -i 1 # mount -t pohmel -o idx=1 none /jim Killed
  14. That isn’t good. dmesg reports there is a null pointer reference in the kernel.
  15. Find two patches that will be in 2.6.33.1 related to “bdi” and not having a real backing device. Add them and make a new kernel.
  16. Fail, badly. Please remember to change your version number in the kernel make file, lest you also forget the init ramdisk and make an unbootable system.
  17. Success! With these patches I can now mount my pohmelfs partition! Unpacking a linux source tree is pleasently quick.
  18. Failure! Chowning the source tree fills dmesg with things like…  pohmelfs_insert_hash: exist: parent: 856, ino: 879, hash: 9f4b42c5, len: 14, data: ‘README.syncppp’, new: ino: 37643, hash: 9f4b42c5, len: 14, data: ‘README.syncppp’. Chown silently and slowly fails.
  19. Unmount and remount. Now chown hangs after a small amount of progress. No messages in dmesg this time. No changes make their way to the server.
  20. I’m beginning to think Pohmelfs is not ready for me.

I will go shopping for other distributed file systems.

Attachments

patch-2.6.33-pohmelfs.diff 2274 bytes

Linux PPC with /boot on a non-root partion

Let’s just say you have taken your old PPC G4 mac Mini with the dead hard disk and fitted it with an 8GB compact flash card in a 2.5” IDE adapter. Installing Debian Lenny on there is easy, but you end up with an ext3 journaled filesystem that may be concentrating wear on the journal area.

NILFS2 is a spiffy new filesystem that probably doesn’t concentrate wear and is attractive when writing is slower than reading. Sounds great, but yaboot doesn’t know how to find the kernel and initrd.img on it, so you need to make an ext2 /boot partion and then tell yaboot to look there.

The secret is in the naming of the files in yaboot.conf. Here is mine:

boot=/dev/hda2
device=hd:
ofboot=hd:2
partition=4
root=/dev/hda4
timeout=50
install=/usr/lib/yaboot/yaboot
magicboot=/usr/lib/yaboot/ofboot
enablecdboot

image=hd:3,/vmlinux
    partition 3
    label=Linux
    read-only
    initrd=hd:3,/initrd.img

image=hd:3,/vmlinux.old
    partition 3
    label=old
    read-only
    initrd=hd:3,/initrd.img.old

As you can see, hda3 is my /boot, hda4 is my root.

Astute readers will also notice that you have to either save a partition or migrate from another drive to pull this off since you need to build yourself a shiny new 2.6.31 kernel with NILFS2 compiled in.

Note: My G4 Mini has developed an odd hanging problem when using X programs with the root on NILFS2. I’ll go back to ext2, 2.6.26 for now then try ext2 2.6.31 and see where the trouble lies.

A four day bug

I just finished finding a four day bug. It sounds simple in retrospect, but oh the wrong paths I have trod and wild codes I have explored.

If, like me, you run Debian on tiny routers, like WRTSL54GS or other Broadcom units from Linksys, you probably use an OpenWRT kernel and a Debian root. Somewhere between 2.6.25 and 2.6.28 the OpenWRT people chopped the FPU emulator out of their Linux tree. Sadly for Debian people this causes any program you try to run, like init, to hang in an infinite loop.

The solution is now enshrined over at https://dev.openwrt.org/ticket/5774 where it talks about the invisible kernel config option: MIPS_FPU_EMU.

Addendum: Oh yeah, you are going to want to go into drivers/usb/host/ehci-ssb.c and pull the static off of ssb_ehci_attach and ssb_ehci_detach if you want USB 2.0 built into the kernel.

The OpenWRT folks "resolved" this by adding a printk that mentions the FPU emulation is disabled somewhere early in the boot, but not making the kernel option visible or preventing the infinite emulator trap.

At best this will let your kernel tell you "I told you so." after you spend four days debugging.

Letting your LXC containers use virtual consoles.

If you are going to run virtual machines in LXC containers it is nice to let each one have a console on your physical box. You can do this thusly…

Note: we are going under the knickers of lxc. It may not appreciate having its config files changed, but I didn’t find an exposed command to accomplish this.

  1. Take some virtual consoles away from the main container. Edit your /etc/inittab and comment out the four lines for tty3 to tty6. We will save these for the containers.
  2. kill -HUP 1
  3. Now we will give tty3 to my container named rattus. You will replace rattus with your machine name.
  4. Go to /var/lib/lxc/rattus.
  5. Edit cgroup. The devices like 4:N are the virtual console terminals. We want to have a line 4:3 and no other 4:N where N is an integer.
  6. Do the same for config.
  7. Cut rattus down to two virtual consoles, the kind that can be used with lxc-console. This will be tty1 and tty2 leaving tty3 to be the real one. Edit /var/lib/lxc/rattus/ttys to be a 2 instead of a 4. Beware: There is a NUL after the digit. It may be important. Emacs will preserve it.
  8. Edit the ROOT/etc/inittab file and comment out the getty line for tty4 since we don’t get to use it and it won’t have a virtual console on it. While you are in here, take out the console line as well. I think it is responsible for the spurious login prompts while you are working in the other consoles.

Using Linux Containers with Debian Lenny.

Linux Containers are a mechanism by which you can run multiple virtual instances of linux inside a single machine.

  • + more efficient than a virtual solution like Xen, VirtualBox, or KVM.
  • - all instances share a single Linux kernel.
  • - I think if you expect to have actively hostile users then you should not use containers. Just a gut feeling here. They are new-ish.

Following are my notes for using Linux containers with Debian Lenny. Read along on that IBM article I just linked. It is good stuff.

  1. Following a catastrophic disk failure, I begin with a clean install on a new hard drive. (Bah on you Hitachi.)
  2. Install lenny. Minimal install.
  3. Add openssh and emacs for my sanity. Add build-essential for kernel building.
  4. Try the 2.6.30 kernel from backports. Crash on ifconfig in an inner container. Try my own 2.6.30, same problem. Try 2.6.31-rc7. All is good. So checkout 2.6.31-rc7 (or probably a real 2.6.31 when it comes out) and build that kernel. I started with the .config file from the Debian 2.6.30 backports and answered the extra questions from there. My resulting .config file is attached to the end of this posting. I no longer use make-kpkg, it rotted away too far. I just install my kernel with “make install ; make modules_install” in the kernel build directory. You then have to “update-initramfs -k 2.6.31-rc7 -c” to get the init ram disk in place. Finally “update-grub”.
  5. Fail to reboot. Ha ha! The linux kernel now names my hda drives as sda and fstab fails miserably. Use “ls -l /dev/disk/by-uuid” to find my UUIDs and use them in fstab.
  6. The swap partition will still fail, but I can remake it with “mkswap -L SWAP /dev/hdaXXX” and then use LABEL=SWAP in fstab after the next boot.
  7. Seek out the user space tools. Decide to try to 0.6.3-1 package from unstable. wget http://ftp.us.debian.org/debian/pool/main/l/lxc/lxc_0.6.3-1_i386.deb
  8. Install manually with dpkg: dpkg -i lxc_0.6.3-1_i386.deb (note: As squeeze diverges, later versions of this may have linkage problems that won’t let them install on lenny)
  9. Read the /usr/share/doc/lxc/README.Debian file. See some neat stuff.
  10. Run lxc-checkconfig to verify the kernel has what it takes. Notice the cgroup memory controller is disabled, but don’t care.
  11. Decision time: The cgroup filesystem has to get mounted somewhere. It doesn’t matter much where. I chose to put mine in /cgroup to avoid any mount order dependencies. Some people will have ontological problems with this. Imkdir /cgroup and add this to my /etc/fstab file: cgroup  /cgroup cgroup  defaults 0 0

  12. I think I have all the pieces now. Pop over to LXC: Linux container tools and slide down to “Configure networking” and lets get going. I am going to have to change my ethernet to a bridge. Here is the before and after of my /etc/network/interfaces file

    # The primary network interface
allow-hotplug eth1
iface eth1 inet static
      address 172.18.36.4
      netmask 255.255.255.0
      gateway 172.18.36.1

    and the after

    # The primary network interface
auto br0
iface br0 inet static
      bridge_ports eth1
      bridge_maxwait 0
      address 172.18.36.4
      netmask 255.255.255.0
      gateway 172.18.36.1auto eth1
iface eth1 inet manual

    Use ifdown -a and ifup -a to make the change, from the console. I failed here because I didn’t have bridge-utils installed. Make sure you do that first. If you are using dhcp just reboot and save yourself the heartache when your old dhcp trashes your address.

  13. Try using lxc-debian to make a container. You can find this in /usr/share/doc/lxc/examples. Pull it out and uncompress it, then run it. You will need to install debootstrap first. It appears to be a trainwreck. I had to patch it a little to make it run. The patch is attached below.

  14. Note: lxc-debian doesn’t allow you to specify your package source, it is hard coded to ftp.debian.org. If you have a local cache, or a faster source you will want to edit a line in there. debootstrap handles its own cache, so it only sucks once.

  15. When you get the “Configuring Locales” dialog, take the time to scroll down to en_US.UTF-8 and turn it on, then select it on the next page to prevent a cloud of error lines. I suppose other ones work, but not generating and selecting makes noise.

  16. The last line will tell you how to start your container, mine says: 

    You can run your container with the 'lxc-start -n test1'

  17. You now have a container with a bunch of bad config files and no editor. Fortunately you can edit the afflicted files from the main linux.

  18. Go look at your ROOT/etc/network/interfaces. Mine had a bad netmask.

  19. Check your ROOT/etc/apt/sources.list, it is probably missing security and volatile.

  20. Go ahead and start your container now and start working form the inside.

  21. I am going to use aptitude, so “apt-get update ; apt-get install aptitude”

  22. Let’s get the “normal” packages installed. “aptitude update” and then aptitude install "?priority(standard)" and aptitude install "?priority(important)"

  23. There are still issues starting up as a daemon and some collisions with the system console, but I’ll address those in a later post.

This should be enough to get started.

Attachments

lxc-debian.diff 303 bytes
container.config 98265 bytes
more articles