Jim's Depository

The robots are reading at least. The rate is low enough I'll just manually delete them for now.

I used to use the google recaptcha, but I know I'm annoyed at finding stop lights and street signs and in a more tracking sensitive era I don't feel like feeding people's reading habits to third parties.

That probably means I'll eventually have to make my own human detector system and lose a week to that.

This blog started 13 years ago musing about a language. I'm back to the language.

The goal is a language to use for the user mode code of a concurrent communications based operating system. I need it to catchiest mistakes at compile time, be agile enough to try language experiments, and have reasonable performance which is maybe a small predictable integer off of top notch C.

The State of the Language

• I chose a name! Yes, this usually happens late for me, but it is atuin. After the turtle.

• The compiler written in Swift, which is by far the most productive language I've used for writing a compiler. It emits LLVM IR (in text), which clang then compiles into an executable. I've gone to some length to keep my LLVM IR legible, it really helps diagnosing problems.

• It looks a lot like Swift, but without the hard parts. Call it a 50% of a 90% Swift, without the other 90%.

• The parser is built to be extended at runtime. Building new features into a language by forcing them into existing syntax seems to just make a mess of your source code. If I want to have something like async/await, I'll extend it into the language instead of making a dance of calls and wrapper types. Source code should be clean.

• I'm skipping a lot of the complications for now. There is no FFI (I don't need one) and no separate compilation (I don't need to). The parser is slow, it is a combinator based recursive descent which is supposed to be a Packrat Parser, but until I notice it taking up time the rats can wait. I build an AST which directly matches the parse, then walk it and create a whole new AST to generate the code (with a slightly different shape). Coming from single pass compilers with no AST back during the day, this seems colossally wasteful, but after buying big enough hardware to build Swift over and over during my porting attempt, I have cpu cycles.

• Using the parser is gorgeous! Swift operators and Unicode characters in the source file make it a bit like reading BNF. After each rule there is a binding from its products to variables which are used to build an AST node as a product. Very clean.

Status?

• integers, characters, strings, bools, arrays

• structs, classes, enums, functions, methods, closures

• all the usual flow of control, including defer (there went a week of re-engineering)

• next up? reference counting (class instances are currently immortal).

• and then start writing some real programs instead of test programs and see what's needed.

While pondering what progress could be made in programming languages, I delved into how one might write single process, concurrent code to build systems with many simultaneous clients. It seemed that a stumbling block to feeding such a system was the weird mismatch with Unix and its mostly blocking API.

So, I did the only reasonable thing and started to build my own operating system. It started simply enough with a visit to OSDev.org and wondering if I could just make something say Hello World., and then maybe some keyboard input… and a VT console emulator (multiple virtual), well then I needed serial ports to send out diagnostics, and then a TCP/IP over Ethernet. All that meant I needed a filesystem like thing to store my executables and loaders for the executables and some interpreters to run… Lua, then Wren…

Mini-Moral: Don't visit OSDev.org unless you have a few free years.

Stubborn Guiding Principles of My OS

• Force concurrent programming to be the best way. This is to force me to encounter the warts and problems and resolve them or document them and live with them.

• Don't have a full C Runtime Library and POSIX. This will keep me from falling back into the easy, decades old solutions.

• I can make C do concurrent programming, but it really hurts me. For the low level code user mode code I wrote a concurrent programming library for C, but it is really a pain to use it. This is not the answer, but it let me try out some program architectures by implementing the various OS bits and bobs in different styles, if somewhat clumsily.

• I want the user mode programs to be in a safe, high level language. There just isn't much point in programming in a language that finds bugs at runtime when you build cycle is "build new OS image, reboot computer, test". There isn't much point in a language that finds its bugs at run time anyway, but certainly not here.

• Communication between user mode programs must be fast and clean. Ideally there will be zero OS intervention in the communication. (except some scheduling when a process blocks on all of its IO options). I don't want IPC costs to impact the architecture. If lots of heterogeneous processes is the answer, then that's the way I'll go.

Questions You Might Have

What is it called?

For a long time it was named os. But I recognized that was going to be a ungoogleable. Eventually I spent a day and named it "osy", because that was short, googleable, a nod to various simple OS names over the years, and clearly one better than OS X. Apple changed their OS name and ruined my joke. I'm keeping the name.

What kind of OS is it?

I don't want to call it a micro kernel, but the kernel does processes, memory management, scheduling, some primitive bootstrap facilities that are done better in user mode once we get up, and a couple of devices that kind of have to be done in the kernel (interrupt controllers and timers).

Have you ever heard of L4?

Yes. I wasn't aiming that way, but I did land pretty close to L4.

What is the kernel written in?

The kernel is ~6600 lines of C and 400 lines of assembler. That for ia32 and x86_64 architectures, though I'll throw out the ia32 next time I'm in there.

I use a number of Clang C extensions to great effect. Between cleanup and overloadable I end up with much more readable code, and cleanup in particular eliminates a whole class of errors.

You seem to have left out a lot of stuff, like devices.

Not a question, but yes. That is all done out in user mode processes. Not untrusted, if you are going to give a process enough access to enumerate your PCI address space, then you are trusting it.

The goal is to make use of the large numbers of cores available in modern systems by splitting tasks among them vertically as well as horizontally. IPC is cheap by design, and we'll see where that leads us in the solution space.

For now Ethernet, IP, ARP, UDP, and TCP are each a separate process communicating as needed. Surely that isn't the ultimate answer, but it lets me stress the communication. Oddly, it might be the answer for the general case, but acknowledge that UDP and TCP arrive as if by magic through off chip accelerators on high performance network cards.

There still seems to be a lot missing.

Most of the work ended up being in user mode. There I struggle with different models of concurrent programming and try to decide what I want.

I've been through C, Lua, and Wren for my user mode code. Device drivers are all in C, but are pretty simple in architecture so that works out. Lua was quick to port, but ultimately it isn't a sturdy enough language to make me happy. Wren was trivial to port and I like it, but I keep having odd performance issues I can't pin down and I've decided I need a real compiled language if I'm to make any judgements about performance.

I spent a couple weeks trying to port Swift to the OS, but was ultimately defeated by the thousands and thousands of lines of CMake with endless build variants and strange special case rules for OSes. So I went back to the "language" that started this blog, started over using Swift to write the compiler, and will use that to make my user mode programs. I just need to understand what the language is generating, and be able to alter the language to handle the concurrent communication without descending into a see of language warts.

The Swift server packages are documented with the inline comment documentation system. I went ahead and used it for my recent test mule project (femtoblogger, you're soaking in it).

It's easy enough to add documentation, Xcode has a CMD-click Add Documentation menu that will jam in a template for your function with spaces for all the parameters and return types. You just type markdown after that. If you add an argument, click it again. Xcode will add more template.

Now people can Option-click on your function name (or whatever) and get a lovely Quick Help pop up.

I appreciate this greatly for libraries.

But…

There doesn't seem to be any pathway from this snippets to a real document where the big picture is explained, and all the functions listed in well organized categories. You know,… like the good old Apple documentation.

"Good enough" is the enemy of "Good".

This isn't a special flaw for the Swift comment based documentation. Essentially all in code systems I've ever used have suffered from this. (Halfway excepting one I wrote, but you can't use.)

The Moral

• Do your "in code" comments.

• Update your "in code" comments when you change the code.

• Write book like documentation somewhere! and put the links to it in HUGE type in the middle of your project page.

Well that was a fair bit of gap in between posts.

Femtoblogger is now on its 4th implementation. I feel the need to try out Swift on the server, so here we are. 100% pure Swift Femtoblogger.

The previous incarnation was Go. Before that was PHP. There was probably a never deployed Python version in there too.

Thoughts on Swift and the design of the server this time around…

• I very much like the type safety of Swift. When I code in Swift there are no runtime explosions, and very few bugs discovered in testing.

• The Swift code is more information dense than the Go code was. The Go code felt like a great sea of text with a little bit of content. The Swift code packs a lot of content into a screen of code.

• Performance appears to be a non-issue. SQLite backed pages are serving in 3-12ms. For comparison I added a caching layer for most popular queries and dropped that into the 80µs range for the query handling. That's not real, I handle them by doing a X-Accel-Redirect header back to Nginx and letting it serve a file out of a RAM disk directly. But it means my process is not going to be buried.

• I haven't bothered with any compiler optimization. I want the checks to stay in place while I see if I've screwed up somewhere.

• It's a small thing, but Swift Argument Parser really got things right. Very easy and natural to use.

The Network End of Things

• The Swift server network library, NIO, really wants to push you into a futures and promises style of programming. I resisted because it is so clumsy at the source code level. Your code begins to vanish under a sea of ritual. That said, I probably would have been better off to embrace it. A request needs to get out of the network queues to free them up while it gets in a non-concurrent model queue to work with the data, then out of there as soon as possible and into a render queue to do template expansion and back into the network queues to send the data out. It's a lot to coordinate.

• I suppose I'll continue avoiding futures and promises in femtoblogger until Swift 6 and the beginning of its new concurrency model make it into NIO.

• I didn't use Vapor or similar web frameworks, I build my server directly on NIOHTTP1 so I'd understand the architecture. The first 90% was simple! Then I needed to parse some form data and couldn't find the calls in the API. Eventually I found an email in a filing cabinet behind a sign that said "Beware of the Leopard" where the NIO team proclaimed that "out of scope". Maybe I'll make a post on parsing "multipart/form-data" (which can't be a String because it might not be valid) and parsing HTTP header values.

Using Other People's Packages

• I did reach out for 'random dude' packages for a higher level SQLite API, Markdown to HTML conversion, and template expansion. It's a bit of a mixed bag there. I'm glad I did it, but there were issues. As always it seems to take a lot of research time to figure out which solution is well supported, not horrible, and will likely last a while.

• I used SQLite.swift for accessing SQLite. The tutorial writing crowd on the internet likes it. It gives you a type safe way of interacting with SQLite without too much ritual. The package is a few years old and has been dormant for most of a year. I found a design flaw where it would try to read a mismatched datatype and explode your process. (Just because you called a SQLite column TEXT doesn't mean it isn't going to return a BLOB for one of the rows.) It looked to be an involved fix to use the receiver's datatype instead of the provider's, and in the absence of an active maintainer I didn't want to invest in making the changes. It was easier to rewrite all the rows to have their TEXT really be TEXT.

• I used Ink for Markdown to HTML conversion. It is simple to use. Out of the box it choked on CRLF line endings. The patch ended up being a , "\r\n" in just the right place, plus about 16 lines of test cases. Hint: "\r\n" is a single Character in Swift… grapheme clusters and all. The maintainer is active so I sent that one in.

• I used Stencil for template expansion. I've used it before and the rough edges I used to have to hammer off are now addressed. Note: Stencil could really use a SafeExpansion mode which escapes HTML for when you have untrusted users. It has user created filters, I ended up making one instead of forking and adding a safe expansion feature.

When your friend's laptop is at full disk, stop them from trying to make space by going through their tiny documents and deleting things that will be missed or finding that pesky Library folder that contains a bunch of gunk they never wanted and deleting that.

Go to the Mac App Store and download Free Disk Space. It will show what is really taking up all the space and should be dealt with.

Personally, I've used Space Gremlin for years, but Free Disk Space is free and does the job.

Update: Apparently Free Disk Space decided money is good. They are $10 now,

I’ve been using USB cameras on tiny repurposed routers, beagle bones, and raspberry pis for some time. I’d like to deploy more cameras, but I am on a tight power budget where every watt matters and my DC power controller that allocates the power is out of ports and I don’t really want to build another one.

Enter “power over ethernet” (PoE) cameras. These hang on an ethernet cable and receive their power from that cable so I only have to run out a cat5 and I’m set. I can buy ethernet switches with PoE and control which devices are powered at any given time.

Danger: most lower cost “smart” switches allow you to control the port power, but only through their silly web interfaces, they don’t allow it over SNMP until you pay for an enterprise device. It appears SMC has a model that will allow SNMP control.

In the made up land of retail pricing, PoE cameras are a little pricey compared to their less capable friends. Fortunately China is a really big place and lots of things are made and sold there. I found 720P PoE cameras for $25 each delivered half a planet away.

Good: They have pretty good picture quality, aluminum “weather proof” enclosures, IR lights for night time, and you can order with four different focal lengths. H.264 over rstp.

Bad: They really sad documentation, on a mini-CD, like I own a tray loading CD drive. Are hardwired to boot on 192.168.0.103 despite documentation that gives a different address, have an open telnet port with a root account backdoor with fixed password, respond with a web interface only in Chinese that doesn’t let you configure the camera in Chrome and Safari. Chrome will offer to guess at translations and turn the text back near English.

That “Bad” entry took me about 3 hours to work out. I do not have the root password, but apparently some tech support responses give it to you.

Proceeding onward, I got Microsoft to give me a demo copy of Windows 8.1 and spun that up in a VirtualBox. If you hit the camera with Internet Explorer, enable Active-X downloads from unknown and untrusted developers and basically give total control of your machine over to unabashedly anonymous people, then you get a web interface that can be used in English and can tell the camera to use DHCP or some static address of your choosing. (I use DHCP for everything and then tell my DHCP server to give a static address to certain MACs.)

The rstp streams can be found at rtsp://camera.example.com:554/user=ACCOUNT&password=PASSWORD&channel=1&stream=0.sdp?real_stream--rtp-caching=100 for the high res stream, and stream=1 for the low res.

Unresolved: I’d like to find a URL to pull a single frame. Perhaps if I figure out how to query the ONVIF interface enough it will tell me.

Further thoughts: It is hard to fathom how someone can sell me a computer, with a camera for $25, in a nice aluminum case and ship it half a planet on demand. Because I need individual control of the power on the switch ports, I’m spending almost that much on the hole in the switch!

Update:

These were garbage. They weren’t weatherproof in any meaningful way and rapidly succumbed. They also have hard coded root backdoors in them.

The following modules need to be added to the Debian init ramdisk to support root on a 9p disk:

./kernel/fs/9p/9p.ko
./kernel/net/9p/9pnet.ko
./kernel/net/9p/9pnet_rdma.ko
./kernel/net/9p/9pnet_virtio.ko

To add these modules, you will extract an init ramdisk (which is just a compressed cpio archive), add the files, and run a depmod -a in it. Then re-archive it.

The commands I used to make mine are:

#!/bin/sh

set -e

mkdir /tmp/9p-ramdisk
cd /tmp/9p-ramdisk
zcat /boot/initrd.img-3.16.0-4-amd64 | cpio -id

mkdir lib/modules/3.16.0-4-amd64/kernel/fs/9p
mkdir lib/modules/3.16.0-4-amd64/kernel/net/9p

cp /lib/modules/3.16.0-4-amd64/kernel/fs/9p/9p.ko lib/modules/3.16.0-4-amd64/kernel/fs/9p/9p.ko 
cp /lib/modules/3.16.0-4-amd64/kernel/net/9p/9pnet.ko lib/modules/3.16.0-4-amd64/kernel/net/9p/9pnet.ko 
cp /lib/modules/3.16.0-4-amd64/kernel/net/9p/9pnet_rdma.ko lib/modules/3.16.0-4-amd64/kernel/net/9p/9pnet_rdma.ko 
cp /lib/modules/3.16.0-4-amd64/kernel/net/9p/9pnet_virtio.ko lib/modules/3.16.0-4-amd64/kernel/net/9p/9pnet_virtio.ko 

depmod -b /tmp/9p-ramdisk

find . | cpio -o -H newc | gzip > /boot/initrd.img-3.16.0-4-amd64-9p

Update: This works, but seems to be a bad idea. When I try to do aptitude update it fails with lines like this…

E: Unable to determine file size for fd 7 - fstat (2: No such file or directory)
E: Problem opening /var/lib/apt/lists/ftp.debian.org_debian_dists_jessie_contrib_binary-amd64_Packages
E: The package lists or status file could not be parsed or opened.
E: Unable to determine file size for fd 8 - fstat (2: No such file or directory)
E: Problem opening /var/lib/apt/lists/ftp.debian.org_debian_dists_jessie_contrib_binary-amd64_Packages
E: The package lists or status file could not be parsed or opened.

… which seems to say that 9p has a problem with some operation aptitude needs.

Back to disk images for me.

If you have an old HP plotter, you will find that the Mac OS X drivers available from HP’s support site can not be installed because they are in an old VISE .hqx format installer.

If you go and get a generic PPD for the printer, and stick it gzipped in /Library/Printers/PPDs/Contents/Resources then you will be able to select it as a printer type when you add the printer.

Attached is the one I found and used. It needs more work, I am getting postscript errors at the end of prints which can use a lot of paper. But it is working.

Update: It looks like you can disable postscript error printing with…

defaults write com.apple.print DoPostScriptErrorHandler NO

…and re-enable it with…

defaults delete com.apple.print DoPostScriptErrorHandler

Update2: The defaults thing does not appear to work.

Update3: Using HP JetDirect as the protocol instead of LPR seems to have fixed the PostScript error problem.

I’d just like to mention that when you use QEMU from a Macintosh running OS X, using the regular Terminal.app terminal, the terminal type will be listed as xterm-256. This will make your monitor console be black on black no mater what you do, except that making the background transparent will make the text somewhat visible.

The “solution” is to change your terminal type to regular old xterm. I’m doing something likes this on my QEMU commands…

$ TERM=xterm qemu-system-i386 ...the-rest-of-my-arguments…