Jim's Depository

I brought up email on a domain which hasn't had it for 15 years or so. I am instantly being barraged with spammers who actually remember the names of the people on that domain!

But I have to wonder, are these real messages and just a very persistent delivery agent? 15 years of trying to deliver the message? I'm tempted to add the users just to see.

If you want to look up countries for an IP address, because say all your comment spam in your blog comes from the same country, then the nice folks at MaxMind will hook you up with their GeoLite2 databases for no cost, though there is a license to pay attention to.

These files are in MMDB format which excels at encoding partitions of natural numbers. The format is documented and there are a number of libraries to access the files, but I wanted native Swift to keep my cross platform building nightmares in check.

You can check out swift-mmdb which I keep on github. I haven't made a change in 4 years, but that's just because it keeps working.

The performance cost is about 9MB for the data and 150µS per lookup.

I've finished rewriting femtoblogger again. Now it is in Swift 6 concurrency using the Hummingbird 2 web framework.

Thoughts:

• Swift 6 is surprisingly good at identifying concurrency foot guns. Sometimes it is obtuse about corrective action, but it caught a number of cases where I would have risked uncontrolled concurrent access.
• Hummingbird is ok, but I find it missing some features which should really be included. There are places to hook them in, and they generally write what you need in the documentation, but it would be nicer if they were included. (e.g.: access to the remote IP in the request context, or parsing multipart/form-data.
• For building on Debian, the binaries are tied to the Swift Stdlib which might not be synced on your deploy machine, so I've resorted to swift build --static-swift-stdlib in my Makefile and keeping the Swift Stdlib embedded.
• For simple packages, you can organize all the parts in a directoy at build time and use dpkg-deb --build --root-owner-group to make a Debian package with minimal fuss. If you need to screw with file owners other than root then you'll have to get into a fakeroot or something. The dpkg-deb man page alludes to them adding a owner/mode manifest, so I'd hold out for that.
• brew can install dpkg on your Mac so you can test building the package there. You'll get your macos binary, so it isn't usable. If you used the Swift Static SDK you could build your Debian packages on the mac, but I didn't want to mess with that.
• I used Lighter to access the database, but I'm not sure I like it. The API has like three ways of doing everything and doesn't really talk about them, so it's kind of random exploration to get something to work. It makes a set of struct to map to your SQLite tables which is kind of its thing, but I kept finding myself working with partial data so bypassing that layer on insert, update, or delete. For reading I ended up making my own structs anyway for reasons on about half the tables. Still, I like the strongly typed access.

So, maybe I'll write more here now that the software works again.

If you have had to do something heinous in your code to avoid triggering a Swift compiler bug, you might consider…

#if swift(>=7)
#warning("Check if Swift bug is fixed and we can remove the '@unchecked Sendable'.")
#endif

In my case, certain presentations of indirect enums which reference each other (and that's totally legal!) trip a circular reference error in the analysis. The work around is to jiggle the order of your declarations, or dodge the analysis by marking it as an @unchecked Sendable.

The bug has been there since sometime in version 5, we are up in 6 now, so safe to say it isn't a priority. Setting a reminder that someday when you hit Swift 7 it is worth a review let's you forget about it now.

I know my daily readers have been wondering what's up these past 700 or so days…

• There will be no more OS work. I rolled a critical fumble on COVID and can no longer work with complex abstractions in my head. I was kind of hoping for a recovery, but it appears there will not be one.

• An OS which makes limited use of system calls and primarily operates by streaming requests from user space to the OS and responses make to the user space asynchronously was looking extremely promising and anyone looking to build one sadly only gets that anecdote from my work.

• Most of the entities which the user space communicates with are ordinairy, though trusted, processes in their own right, the kernel is quite small and really about connecting things. Breaking traditional OS subsystems off into their own processes has a nice synergy with high core count processors by putting more L1/L2 cache to work.

• Popular programming languages can be made to work in this environment, but there is a lot of friction. A language with builtin light weight concurrency is more ideal. I was in the middle of a sort of Swift subset which fulfilled that and looked very promising.

Not so OS related…

• I'm ok. I've been doing a lot of physical manufacturing skill work and exploring what I can create. I think I can do small software bits, but I have to finish them in one push, if I have to pick them up later it makes no sense to me.

• So I guess I'll take a little pivot here and start documenting small things and maybe just showing stuff I make.

It is worth knowing that libicu likes to append version strings to its API symbols behind your back. If your header files do not match your libraries, like say you have the Swift compiler installed on Debian and it has replaced /usr/include/unicode/ with its own nailed down version, then you try to link symbols like utext_close_swift_65 or similar and you will be unhappy.

I removed the swift compiler and reinstalled the libicu-dev package and all is good now.

Sometimes I want to work on my experimental operating system when I'm away from my Linux monsters. To that end I keep the ability to build it on macOS. Broadly speaking that means a qemu built to support SDL because the Cocoa one in brew hasn't worked in years.

But the wrinkle… you can't use the qemu -kernel xxx flags with 64 bit multiboot kernels. It appears to be a "won't fix" from qemu, so I need to make ISO images, but making a working, modern grub on macOS cross compiling from ARM is not anything the grub guys envisioned.

Note: You will find people using legacy grub with just an eltorito.bin. This won't work well for 64 bit x86, you have to relink to elf32 then your symbol entries are all in the wrong format and insanity ensues.

So this is how I tackle it:

• Use mkisofs from cdrtools from brew to build the ISO image.

• Rip the guts out of an ISO I made on a Linux machine using grub and stuff those into my ISO.

If you mount an ISO you have made, you will find a boot/grub directory. Take that and stick it into your ISO, add your own boot/grub/grub.cfg, pack it all up and you are ready to go.

The relevant part of my Makefile looks a bit like this…

iso/boot/grub/grub.cfg : $(srcdir)/scripts/grub.cfg | iso/boot/grub
        cp $< $@

iso/boot/grub/i386-pc/eltorito.img : $(srcdir)/scripts/grub.tar.gz | iso/boot/grub
        tar -C iso/boot -xf $<

os.iso: iso/boot/kernel.elf iso/boot/grub/grub.cfg iso/boot/grub/i386-pc/eltorito.img
        mkisofs -R -b boot/grub/i386-pc/eltorito.img -no-emul-boot -boot-load-size 4 -boot-info-table -o $@ iso

You've noticed I grabbed a grub.tar.gz from someplace you can't see. I've added it as an attachment to this post. I'm using a single file (eltorito.img) as a proxy for the whole thing being unpacked. Don't fiddle with the pieces and expect make to notice.

The pinouts diagram for the Raspberry Pi Pico is miserable when printed on a monochrome laser printer. Fortunately is was in SVG so I took a copy, drug it into Graphic (the tragic rename by Autocad) and made it boring but legible. (Assuming you are on a white background. It is displayed here on the view default, and in dark mode looks like garbage).

I don't print enough color to keep the ink live in my inkjet printers and HP abandoned my networked color laser printer (then a mouse ate part of it).

If you find yourself in the same position, then enjoy! Here it its PDF and SVG glory.

The PDF is down in the attachments.

The Google scanned indicator has been updated to this decade's aesthetic. Also, it and the verified user indicator have been brought into the CSS to prevent loading image resources.

I'm not totally happy about that. They are spans with images embedded, I'd rather have them be images with shared content for the sake of alt tags. But oh well.

Beginning now, well, slightly ago… all post authors get a blue checkmark by their name.