Jim's Depository

ifstat - A little like a vmstat for network interfaces. This is pretty handy for a quick look at network saturation without have to break out some massive GUIfied tool with a hideous interface.

asbestos:\~# ifstat -b -i eth0,eth1,eth2,eth3
       eth0                eth1                eth2                eth3      
 Kbps in  Kbps out   Kbps in  Kbps out   Kbps in  Kbps out   Kbps in  Kbps out
   13.44     19.06    914.15    953.60      0.00      0.00      0.00      0.00
    9.73     10.71    632.22    630.49    300.84    286.86      5.27      4.72
   21.01     23.73    211.98    226.27      0.00      0.00      0.00      0.00
    2.23      3.33    260.78    281.29    293.32    315.72      5.80      4.72
    3.47      6.03    180.47    227.07      0.00      0.00      0.00      0.00

There is a debian package. And note that “-b” option to report in bits/second instead of bytes. What kind of person measures network use in bytes per second?

And in the same vein: iostat is hiding over in the systat package on debian. Use it too.

In this category I make notes of useful linux tools and utilities that are perhaps newer than 1997. If, like me, you tend to use your existing knowledge to get the job done, this list might be useful to see some of the things developed in the past 10 years.

Femtoblogger now shows on IE7. Not too bad. It only took 2 hours to work out the layout problems.

The most severe is that max-width: 600px in the CSS file brings up a bug when used inside a table cell. The cell gets expanded as if there was no max-width so the table ends up expanding to fill the screen. All the articles were cowering against the lefthand border of the screen with the nav stuff still on the right. (Firefox does this too but only if you are clicking a link in a table cell. There went 120 minutes of my life for that one.)

My ultimate solution was to make IE use a width: 600px instead. There is now an IE only CSS file that is loaded after the real one and has a rule to patch that up. Thankfully the IE blokes must have known they were being a menace to society and they built a special comment hack into their HTML parser to let people cope.

<!--[if IE]> <link rel=stylesheet type=text/css href=femtoblogger-ie.css> <![endif]-->

… it would have been nice to have something similar in CSS, that would save a file and a download.

You can read more about this hack at CSS Hacks- The good, the bad, and the ugly- Conditional comments

I have abandoned CSS columns. They are an ongoing time sink. If you are willing to specify hard physical pixel widths for the columns they almost work like I want, but not quite. Their background colors end early, and when you get a wider than planned element inside one it spills out on to neighboring content instead of resizing.

Back to the grand one row table for me. The CSS zealots may scoff, but my columns work right all the time.

On the plus side, I may now be web 2.0. I added a couple rounded corners and a drop shadow. Unless you are using IE.

I went ahead and registered femtoblogger.com which reminds me how much easier it is for me to write code than prose. Filling a page is just painful. I’ll have to make filling in that outline be a long term project.

For now it serves as a placeholder for people that might guess at the URL and will send them back to this blog for femtoblogger news. I suppose I need to rename this blog now which means more code. Blast.

This is half baked, but I’m going to roll it out and try to live with it. Articles and comments will now use a WYSIWYG interface for styling. I hope to remove the How do I insert-a-link/bold/blockquote? type of questions that interrupt people’s thought process when posting in a new venue.

The downsides are many:

• This uses the designMode and the execCommand() interface. execCommand() was probably designed by children and is functionally incomplete and inadequately specified. It promises to be a festering source of browser quirks and user bafflement for years to come.
• The HTML sanitizer will need to normalize as well to unquirk the postings.
• I need to also support a raw HTML mode for people with odd browsers.
• I will need to make decisions about copying in images and possibly resizing.
• I will probably need onscreen hinting ofsome sort to let people know how design mode works, e.g. They can drag in links and images.
• I will probably screw up and butcher comments at least several times.

But… what would life be without a little excitement, so here we go.

I noticed when abandoning the sexy 1 byte unicode dingbats for 160 byte PNGs in femtoblogger (to support the wretched Windows users of this world) that my images were being transfered on each page load. Bad for load time.

I had wrongly assumed that the Apache expiration module would give them a short expiration time by default. No problem, I added the expiration specification to my .htaccess file and managed to break my files completely. The system-wide Override setting prohibit that.

I could make server-wide changes, but with an eye toward users that can’t I came up with a new, better solution.

I serve the images through PHP. That is about half the speed of native, but I can put on a proper Expires header. (And PHP doesn’t put on its don’t cache me ever headers if you have a image/png content type.)

That gets more portable and safer, but how about better? I set my images so they all expire out of cache at the same time. Now I won’t have odd windows where some images have changed and others haven’t. When one icon expires, they all expire. It goes like this…

<php?
$icon = $HTTP_GET_VARS['icon'];$icons = array( 'edit' => 'images/edit.png',
        'delete' => 'images/delete.png');Header('Content-type: image/png');
Header('Expires: ' . gmdate('r', strtotime('next Sunday')));fpassthru( fopen( $icons[$icon], 'r'));
?>

Then you can load icon.php?edit and get the PNG for the edit icon with these headers…

HTTP/1.1 200 OK
Date: Wed, 12 Sep 2007 07:23:51 GMT
Server: Apache/2.2.3 (Debian) …
Expires: Sun, 16 Sep 2007 05:00:00 +0000
Content-Length: 164
Content-Type: image/png

I decided the femtoblogger article posting and comment writing needed a WYSIWYG editor. TinyMCE seems to be the solution of choice, but I don’t think I can tolerate 143k for the compacted Javascript plus a large flurry of images. That is about 4 times the size of femtoblogger at the moment.

I’ve embarked on writing my own small content editor. There were some large unknowns in the beginning, but I think I know know what the unknowns are.

• Focus handling is going to be odd. You can’t make an arbitrary element take focus so I have to make my own fake notion of focus and integrated it with the normal one.
• The DOM is silent on finding exactly where on which element a user clicks. Fortunately there is a window.getSelection() and document.selection depending which way your browser swings that handles that. (Phew, I thought I was going to end up with each character in its own little SPAN for a while there.)
• Spaces are going to be rough. I will have to either prevent the user from entering adjacent spaces, or convert multiple spaces to non-breaking spaces.
• up and down arrows promise to be mysterious to code as well. I may have to figure out how to fake a mouse click, I’m not sure that is even possible.

Back to the code mines. More here as lessons are learned.

When I added HTML encoded articles and comments to femtoblogger I had to ensure the user generated content was free of cross site scripting attacks, malicious code, and horrible html. It looks like in 2007 the king of this field is HTML Purifier, but it is a bit heavy for femtoblogger. (This one function is 30 times as large as all of femtoblogger.)

After reading the excellent ‘comparison’ on the HTML Purifier site I decided:

• I need a solid sanitizer.
• It needs to be small.
• I don’t care about ensuring the HTML complies with any standard, just that it is safe.
• I can tolerate weak error reporting for seriously malformed HTML.

To that end I have built a minimal solution from PHP-5 parts. The process goes like this:

1. Parse HTML document into a DOM representation (1 function call)
2. Walk the DOM and get rid of anything I don’t wish to allow (100 lines of code).
3. Turn the DOM representation into HTML. (1 function call)

It is a little more complicated since I’m working with HTML fragments. I have to paste on an HTML-BODY-DIV at the front end of the process and cut them back off at the end. 5 more lines of code.

The walker goes just like you would expect. Things to remember to do:

• If you don’t recognize an element, kill it.
• If you don’t recognize an attribute of an element, kill it.
• If one of your legal attributes can be a URL, parse it and if you don’t like the scheme, then kill it.

And don’t kill things while you foreach() over their container. It screws up. Do your kills after the foreach() finishes.

You can now put html into femtoblogger comments.

Ornamental Tags : b i em strong tt quote Structure Tags : br p pre blockquote div Linking : img a (but no funny URLs).

The downside is that I had to take out the “single newline to <br>“ translator and some bare punctuation is now illegal. I’m not sure this is a win, but I wanted to put in a hyperlink.

I think this paves the way for an even better solution which involves a Javascript WYSIWYG editor for posting. Something like TinyMCE but at less than 140k of Javascript and a 10 second startup time.