Jim's Depositorythis code is not yet written |
help |
Little known and not terribly documented, but true:
You can tell your Debian machine to do an apt update and and download any files that you might need by adding two lines to your apt config (which I'll bet you didn't know you had.)
cat > /etc/apt/apt.conf.d/50autoupdate
APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Download-Upgradeable-Packages "1";
Then the script /etc/cron.daily/apt will keep you all up to date savings you seconds to minutes every time you decide to upgrade.
Cacheing common reports from a database inside the database itself is surprisingly easy and makes a huge difference for Ajax style web pages.
Step 1: Get the browser cache working for me.
If I break the request in to midnight aligned 24 hour periods, then I can cache the result for any completed day. This way I only need to pull new days of data.
This helps, but it turns out I don't revisit days often during a session.
Step 2: Tune the database indices.
Fail miserably. It turns out SQLite does my query faster without indicies, so I took them off completely. (Sequential read on my virtual server is much faster than random.)
Step 3: Server side cacheing. In the database.
Now we get to the meat. I can cache pre-compressed reports for each of my daily periods. There are a couple of wrinkles though. I need some way to invalidate a report when the underlying data changes. (Sometimes some observations can be delayed and trickle in later.) I can't think of a good way to have the database delete a cache file, so instead I store the cached copies in the database.
This turns out to be surprisingly clean to code, and a simple set of triggers on the underlying data can remove any affected report.
It would be more efficient to keep the cached reports as files, or better let in a httpd cache, but then they could not be invalidated by the database.
Step 4: Clean up the little PHP wrapper and stick it here.
I'll get to that.
Certain corrupt JPEG files will explode Quicktime Player if it encounters them in an "Import Image Sequence…" operation. This bug has been present for many years and shows no sign of going away.
If you are archiving images from cheap webcams you will encounter these.
I am not aware of anything included with Mac OS X Leopard that validates a JPEG file, so I built jpeginfo for Leopard.
With that, you can do things like…
for v in *.jpg ;do jpeginfo -c $v || rm $v ;done
… to delete all the corrupted JPEGs.
You can find the sources at http://www.iki.fi/tjko/projects.html
You will also need to build libjpeg, from ftp://ftp.uu.net/graphics/jpeg/jpegsrc.v6b.tar.gz
Build libjpeg first (./configure ; make ), then build jpeginfo (./configure --with-libjpeg=THE-RIGHT_DIRECTORY ; make )
But you don't need to do that... I've attached the copy I built.
comment by jim, 3 weeks ago
I might mention that there are still some jpegs that will crash the quicktime encoder that are valid according to jpeginfo, but it gets most of them.
Safari is odd with contentEditable divs. It doesn't assume you will have a pre-formatted div as a container so it puts each line into its own div… sometimes marked with code class, sometimes not.
That makes a mess but is tolerable, until my HTML sanitizer burps out the clean version with newlines between the divs and causes accidental double spacing.
I could rewrite the ->saveHTML() method of the PHP DOMDocument to not put newlines in between divs, but that would give awful looking HTML.
For now I added a function to the sanitizer to remove extra divs from inside code formated divs. It tries to be smart about inserting newline characters, but it may not be smart enough.
Someone should revisit the whole contentEditable thing and specify precisely what is to meant by all of the operations.
If you run your own bind server for your domain you can easily support dynamic updates from your machines that have transient IP addresses. bind is capable of many things, but I'll just show the bits you need…
First, use bind 9.3 or better.
On the server, go edit the file that contains your zone... perhaps it looks like this...
zone "studt.net" {
type master;
file "/etc/bind/studt.net";
};
… you are going to need to generate a key for each host that has a transient address, add it to this file, and then tell the zone that machines are allowed to update their own addresses.
dnssec-keygen -a HMAC-MD5 -b 512 -n HOST jimshouse.studt.net
… will generate two files with long names. Keep them, you will need them on the client machines. But look inside the K*.private file and copy out the value of the "Key: " line. You are about to paste it into your zone file.
Add a key for each host that will update its name, and an update-policy to the zone, you may end up looking like this…
key jimshouse.studt.net {
algorithm HMAC-MD5;
secret "eQGH--lots-of-more-key-i-left-out==";
};
zone "studt.net" { type master; notify yes; forwarders { }; file "/etc/bind/studt.net"; update-policy { grant * self * A TXT; }; }; Restart your bind server and your server is ready to go. (Yes, you could reload but there was a bug in 1999 or so and I have never gotten over it.)
Possible Bug: Your bind process has to be able to write ".jnl" files. Debian etch is configured to put them in /etc/bind but the bind user can't write there. I chmodded /etc/bind to 775 to deal with that. You'll know you have this problem when your client update fails with a SERVFAIL and you tail your syslog on the server and read the error messages.
Dangerous Note: Before you edit the zone file you have to first stop the dynamic updates so the .jnl file gets merged with the zone file...
rndc freeze studt.net
edit the studt.net zone file
rndc unfreeze studt.net
Now for the client side. You could set dhclient.conf to do this automatically, but for primitive cave programmers like me you can just execute a command. First... copy your K*.private key file for the client to the client machine, you're going to need it. Second… use the nsupdate command to set the name's value. I do it in a script sort of like this…
#!/bin/sh
TTL=600
SERVER=ns.studt.net. ZONE=studt.net HOSTNAME=jimshouse.studt.net. KEYFILE=/path/to/where/you/keep/Kjimshouse.studt.net.+157+26806.private IP=99.153.198.165 nsupdate -v -k $KEYFILE > /dev/null << EOF server $SERVER zone $ZONE update delete $HOSTNAME A update add $HOSTNAME $TTL A $IP send EOF See that TTL? That says 10 minutes, so a computer on the internet might keep using your old address for up to 10 minutes after your address changes. You can adjust that number for your situation. Clients won't like you if you get too short.
I'd just like to mention that I would feel better if Tyrannosaurus Rex had lived in Europe. 1,000 miles and 65 million years is too close. I want there to be an ocean between them and me as well.
I generally forget some nice packages when I toss up a new debian machine and then spend too much time trying to remember which ones they are. Now I keep them listed here. Maybe you will like them too.
That attachment is an etch build of dfu-programmer, I can't test it, but it probably works on Etch.
comment by jim, 6 weeks ago
I just built dfu-programmer for etch with these steps, and attached it to the parent article (I guess I should implement attachments for comments.)
I probably should have modified the version number to mark it as an Etch build, but this works for private use.
macvlan is used to give a second MAC address to a network adapter and see it as a new device at the higher levels. It is useful to pretend you are multiple machines, as the container people do, or in my case, to implement your own TCP stack without interference from the kernel IP code.
My ugly scars thus far:
Not macvlan's fault, but if you are working on a user space TCP stack and you are wondering why it seems to be sending RST packets... make sure you aren't accidentally sharing the link with the kernel.
The Road to Bankruptcy is a short blog of Darren's house rebuilding woes. I highly recommend it for anyone that has every owned a house.
Background: Family with four nine-year old children in a small house with a fatally flawed foundation decides that somehow tearing down the house and constructing a new one is a good idea.
I've been working on a musical instrument tuner for the iPhone. Sadly, I was not one of the lucky 500 first round of developers accepted by Apple, and have been excluded from the market.
At some point when Apple decides my $100 bill is good enough to let me into the market my application would be starting far back in the pack, behind whichever one of these first, blessed three gains the title as "the tuner you use on an iPhone".
Given that handicap, I'll probably not release the application, but it was interesting writing it. Among the things I discovered:
The femtoblogger software is being written by Jim Studt. The content of this page is provided by anonymous individuals. If you believe something on this page is innapropriate contact Jim Studt. |
Contributeloginlogout post create account (12 seconds) recent comments FilterSearchBrowsers
Archives
|