help

Jim's Depository

this code is not yet written

Browsers

googlebot25.0%
Firefox19.3%
yahoobot14.9%
msnbot11.5%
IE 610.6%
hiding5.7%
Chrome4.8%
Safari3.1%
IE 71.8%
IE 81.8%
Opera0.8%
Konqueror0.4%
jeevesbot0.1%
IE 50.1%
Wordpress0.0%
unknown0.0%

4k sector disks on Debian Squeeze

My new Western Digital Green 2TB (WDC WD20 EARS-00MVWB0) drives have 4k sectors, but they report as 512 byte for compatibility.  Linux 2.6.32 and the fdisk on squeeze will do the wrong thing by default and place partitions on bad boundaries.

fdisk -b 4096 -u /dev/XXXX

… is the right command to force the proper sector size. It will then die with a floating point exception unless you type a 'c' to disable MS-DOS compatibility mode once you are in fdisk.

(If you are just going to use them with LVM, maybe skip the silly label anyway and use the whole disk for LVM. Be careful though, currently on squeeze the lvm scan does not happen after the USB scan, so they won't be found at reboot unless you do another scan, like in rc.local.)
comment

An external USB drive enclosure that supports S.M.A.R.T.

Today, my gift to you: After extensive research and testing, I have found a 3.5" SATA⇒USB enclosure that will work with smartmontools to allow you to monitor the S.M.A.R.T. data of your drive. This one may be unique in that I have an actual link you can click and buy the enclosure! 


Ugly Caveat: There are at least two different designs sold under the same model number. Some, possibly older ones though 2 of the 3 I got from Amazon today are this variant, have silver screws and absolutely awful power adapters with white labels. The part where the 120v cord enters is badly made and the cord will not make contact or make intermittent contact unless you cram it in very hard. The variant with black screws has a different model of power supply and seems to be ok. The box art work matches the unit inside. Both variants have the JMicron JM20337 chipset so all is good.
I've got a Western Digital Green 1TB (older model, 512 byte sectors) running in there and smartctl -d usbjmicron -a /dev/sdg is working perfectly on my Linux box.

(smartmontools 5.38 from Debian Lenny was not new enough. I think 5.39 would be, but I jumped straight to the dev edge.)

I couldn't get S.M.A.R.T. to work with this drive on Mac OS X, same version of smartmontools, but it doesn't want to talk to the drive.
comment

Web Storage and privacy.

I was experimenting with HTML-5 Web Storage when I realized Safari wasn't showing me my storage use under the Safari⇒Preferences…⇒Security⇒Show Databases panel. A little find action shows that they live in databases named things like ~/Library/Safari/LocalStorage/http_SITENAME_0.localstorage, and good heavens there are a lot of them!

Most of them are for some sort personal information harvesting operation named loomia. They seem to be about personalized recommendations, but I don't remember authorizing them to collect information on me, and I sure didn't ask them to tag me with a tracking code tucked away in a location I can't see that will survive cookie clearing.

Most of the rest are of the form IXAIInvited{number} = true. I have no idea what that is about, but the number isn't long enough to be a unique identifier for me.

So Safari guys: I know you've been busy, but how about letting me see and manage my localStorage from the user interface?

(And while you're at it, either make localStorage.setItem() work for objects, or update the documentation to mention that it is strings only. And give me my hour of debugging time back.)
comment
comment by jim, 7 weeks ago
If you are of a mood to see your own localStorage situation, you could:

for v in ~/Library/Safari/LocalStorage/* ;do 
echo "#### $v ####"
echo "select * from ItemTable;" | sqlite3 $v 
done

New Toy: Kodak Pulse, 7" wifi picture frame

I ordered a pair of Kodak Pulse 7" wifi picture frames from Provantage about 90 days ago which was long enough to forget, so it was like a little surprise gift to myself.

I intend to rip them open and see if I can replace their firmware with something I can control, but I fired one up to see if it could be made to work as a remote webcam viewer with its intended firmware.

I have to say I am impressed. The picture quality is quite nice. The web site for controlling the content is very nice (except that their applet signature does not have a valid signature). You can feed it from a web page, email, facebook, or some Kodak online gallery.

But here is the thing that makes me feel best about it:

On the support system they have an actual, honest, system status display. Very classy.

The 30 minute hacking report is:
  • I don't think I can make it display a once/{timeunit} update of a webcam using the intended APIs. I could mail a new image to it regularly, but it doesn't display the latest one by preference. There is not a way to email a "delete".
  • It contacts Kodak home base (running on Amazon's EC2, S3, and CloudFront) once per minute where it does:
    1. Some secret SSL shrouded query that they don't let me see.
    2. An NTP lookup from a random server on the planet.
    3. An HTTP query to see if it needs an update, nice XML response.
    4. If needed, the files are then requested by a UUID and come down by HTTP.
  • Blocking the SSL shrouded query stops the others, but I'm pretty sure I could make a proxy to replace the content with my own. This however only works for picture frames in my domains where I control the routers and can intercept.
  • I captured its firmware upgrade. The file will be useful.
Looking forward:
  • I need to look at Amazon data rates and see if it would be abusive to use their browser API to upload and delete pictures, say once a minute all day. I'm sure that's more traffic than they expect, but perhaps the total cost is small enough to be ethical.
  • I'm itching to pop apart the 2nd unit, but I will wait until I have time to photodocument to see what is on the inside of these things.

Attachments

kodak.gif 35361 bytes
comment

Some code optimization

I posted my code optimization with explanations regarding assembler etc at http://geekofpassage.blogspot.com/2010/05/win2k3r2ee-virtual-appliance-running-in.html

The unified diffs are available there too.

Cheers!
comment

Setting up nginx and lua fastcgi on Debian Squeeze

The available pages I can google about this seem overly complicated. This one documents setting up nginx to run lua scripts on Debian Squeeze. These instructions are derived from http://www.unreliablepollution.net/p/other/howto-nginx-and-lua and a dozen remedial googleings.
  1. aptitude install nginx
    The config in /etc/nginx is good in squeeze. We will need to add the fastcgi handler, but that is to be expected. (I edited it to pull the root up to /var/www instead of in an nginx-default subdirectory.)
  2. aptitude install liblua5.1-wsapi-fcgi-1
    There is also a '0' version, but I took 1 on the theory that newer is better.
  3. aptitude install liblua5.1-coxpcall0
    liblua5.1-wsapi-fcgi-1 probably needs a dependency on this. Things will go poorly for you without it.
  4. aptitude install lua5.1
    Can't forget that.
  5. aptitude install spawn-fcgi
    Older instructions talk about libfastcgi-dev, but it isn't available and I think they just wanted spawn-fcgi out of it anyway.
  6. Put a Lua file in /var/www/hello.lua
    The WSAPI is a little strange. Apparently you don't just print your output, but rather coroutine.yield() it. I used this example program from http://keplerproject.github.com/wsapi/manual.html …
    module(..., package.seeall)

    function run(wsapi_env) local headers = { ["Content-type"] = "text/html" }

    local function hello_text() coroutine.yield("<html><body>") coroutine.yield("<p>Hello Wsapi!</p>") coroutine.yield("<p>PATH_INFO: " .. wsapi_env.PATH_INFO .. "</p>") coroutine.yield("<p>SCRIPT_NAME: " .. wsapi_env.SCRIPT_NAME .. "</p>") coroutine.yield("</body></html>") end

    return 200, headers, coroutine.wrap(hello_text) end

  7. spawn-fcgi -a 127.0.0.1 -p 9100 -F 4 -- /usr/bin/wsapi.fcgi
    Now we have handlers on port 9100. This will need to be in /etc/init.d/blahblahblah but this will do for now.
    Note: spawn-fcgi silently fails if you get the command wrong. Do a "ps" and make sure they are there. You will have a section like this if it succeeds…
    18554 ?        Ss     0:00 lua5.1 /usr/bin/wsapi.fcgi
    18555 ?        Ss     0:00 lua5.1 /usr/bin/wsapi.fcgi
    18556 ?        Ss     0:00 lua5.1 /usr/bin/wsapi.fcgi
    18557 ?        Ss     0:00 lua5.1 /usr/bin/wsapi.fcgi
  8. Edit /etc/nginx/sites-available/default to know how we want .lua files handled. I put in this section
    location ~ ^(.+\.lua)(.*)$ {
    root /var/www/; fastcgi_pass 127.0.0.1:9100; fastcgi_index index.lua; fastcgi_split_path_info ^(.+\.lua)(.*)$; fastcgi_param PATH_INFO $fastcgi_path_info; fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; include fastcgi_params; }
    Be careful! Make 'root' match your root!
  9. Restart nginx. /etc/init.d/nginx restart
  10. Load http://YOURSERVER/hello.lua in your browser. You should get something along the lines of:
    Hello Wsapi!

    PATH_INFO: / SCRIPT_NAME: /hello.lua

  11. Celebrate!
Now my thoughts: This may be more than I wanted. I have to think about the implications of a scrutinizer layer above my Lua program. I really wanted to just blast my standard output back up the fastcgi socket. All this yielding seems strange.
comment

Glad it's not my bug.

3:14am, spring daylight savings time day. The church bells across the street have been ringing for 14 minutes.
comment

Declining to use Coda on Debian

After my dismal failure at Pohmelfs, I shopped for alternative distributed files systems. I can't tell if CODA is abandoned or not, but I'll give it a try.

  1. Build into kernel 2.6.33, no problems.
  2. Debian has let their user mode utilities rot away.
  3. CMU has prebuilt .debs, including for Squeeze. That looks good.
  4. Add the CMU apt sources, looks like this now…
    # cat /etc/apt/sources.list.d/coda.list
    deb http://www.coda.cs.cmu.edu/debian lenny/
  5. aptitude update ; aptitude install coda-server, scary text about unsigned packages follows, but it is OK.
  6. Configure server with vice-setup
  7. Similar for the client machines, except aptitude install coda-client… ends with
    00:35:02 Venus starting... 00:35:02 /coda now mounted
  8. Test client with CMU's server:
    # cfs lv /coda/testserver.coda.cs.cmu.edu
    Status of volume 7f000000 (2130706432) named "/" Volume type is ReadWrite
    Looks good. I can read the sample documents.
  9. Test client with my server: Much hanging and server not founding. Most likely related to my "hostname" being an internal address unreachable by the client. There is a spot in the /etc/coda/*.conf file and in one in the /vice directory, but it still attempts to use the unreachable address and no hints in vice-setup on how to force an address.
  10. I temporarily change my hostname to the external interface address. No luck. The client does a pile of DNS lookups for SRV records for my host with _codasrv._udp prepended, then does one for a regular A record. Gets my address, but never tries to contact.
  11. Reboot client. Success! Now we can connect. Something evil must have been cached.
  12. Many failures to authenticate with clog until I realize that the example I'm working from only has an account of admin because that is what they gave vice-setup. Use the account I gave with the password "changeme" and I'm in.
  13. Find cpasswd, change that password.
  14. Try to make myself a user account. au -h MYHOST nu, coda is a pretty hostile system. It echoes my new password in the clear even though it knows how not to, then fails with AuthNameToId() --> AUTH_FAILED
  15. Find pdbtool. Use it to make my account from the server side, but I can't see how to set the password. Go back and try au from a client, and now I can set my password.
  16. Notice I have no group. Use pdbtool to make a group with my name, program explodes with an assert failure. That's going to be a problem.
  17. Unpack a linux kernel on the client. It flew until arch/arm, not it is crawling along at about 1 file per second. No load on server, no load on client, no load on network, just glacial progress.
  18. Abandon Code!
Coda is badly documented, user hostile, and not robust.
I do not trust it.
comment

Failing to use POHMELFS in Linux 2.6.33

I've enjoyed reading zbr's blog postings about POHMELFS and the Elliptics network for years. With POHMELFS in the staging directories of Linux 2.6.33 and me needing to share some files across several machines, it is time to try it out.

  1. Build myself some fresh new 2.6.33 kernels with POHMELFS enabled.
  2. Look for a HOWTO of sorts, fail.
  3. Check the server out with git:
    git clone http://www.ioremap.net/git/pohmelfs-server.git
  4. Follow the included README
  5. Fail, install autoconf.
  6. Fail, finally discover that --with-kdir-path=<mylinuxsourc>/drivers/staging/pohmelfs/ is the right answer.
  7. Fail, install libssl-dev.
  8. make
  9. make install
  10. Copy /usr/local/bin/cfg over to the client machines.
  11. Make a new filesystem to export mounted on /jim.
  12. Start the server: fserver -r /jim
  13. Go to a client machine: 
    #cfg -A add -a 1.1.1.1 -p 1025 -i 1
    # mount -t pohmel -o idx=1 none /jim
    Killed
  14. That isn't good. dmesg reports there is a null pointer reference in the kernel.
  15. Find two patches that will be in 2.6.33.1 related to "bdi" and not having a real backing device. Add them and make a new kernel.
  16. Fail, badly. Please remember to change your version number in the kernel make file, lest you also forget the init ramdisk and make an unbootable system.
  17. Success! With these patches I can now mount my pohmelfs partition! Unpacking a linux source tree is pleasently quick.
  18. Failure! Chowning the source tree fills dmesg with things like… 
    pohmelfs_insert_hash: exist: parent: 856, ino: 879, hash: 9f4b42c5, len: 14, data: 'README.syncppp', new: ino: 37643, hash: 9f4b42c5, len: 14, data: 'README.syncppp'.
    Chown silently and slowly fails.
  19. Unmount and remount. Now chown hangs after a small amount of progress. No messages in dmesg this time. No changes make their way to the server.
  20. I'm beginning to think Pohmelfs is not ready for me.
I will go shopping for other distributed file systems.

Attachments

patch-2.6.33-pohmelfs.diff 2274 bytes
comment

Linux PPC with /boot on a non-root partion

Let's just say you have taken your old PPC G4 mac Mini with the dead hard disk and fitted it with an 8GB compact flash card in a 2.5" IDE adapter. Installing Debian Lenny on there is easy, but you end up with an ext3 journaled filesystem that may be concentrating wear on the journal area.

NILFS2 is a spiffy new filesystem that probably doesn't concentrate wear and is attractive when writing is slower than reading. Sounds great, but yaboot doesn't know how to find the kernel and initrd.img on it, so you need to make an ext2 /boot partion and then tell yaboot to look there.

The secret is in the naming of the files in yaboot.conf. Here is mine:

boot=/dev/hda2
device=hd:
ofboot=hd:2
partition=4
root=/dev/hda4
timeout=50
install=/usr/lib/yaboot/yaboot
magicboot=/usr/lib/yaboot/ofboot
enablecdboot

image=hd:3,/vmlinux
    partition 3
    label=Linux
    read-only
    initrd=hd:3,/initrd.img

image=hd:3,/vmlinux.old
    partition 3
    label=old
    read-only
    initrd=hd:3,/initrd.img.old


As you can see, hda3 is my /boot, hda4 is my root.

Astute readers will also notice that you have to either save a partition or migrate from another drive to pull this off since you need to build yourself a shiny new 2.6.31 kernel with NILFS2 compiled in.

Note: My G4 Mini has developed an odd hanging problem when using X programs with the root on NILFS2. I'll go back to ext2, 2.6.26 for now then try ext2 2.6.31 and see where the trouble lies.

comment
older articles