Jim's Depository

this code is not yet written
 

If you are thinking things changed, you are probably getting the dark mode support. It's your browser's choice. I send the same stuff to you, but the CSS is now littered with @media (prefers-color-scheme: dark) { sections to change things up for dark mode.

I wish Safari didn't put all the editable text, radio buttons, and buttons in blinding white. I wonder if I gorked something up there.

Update: Yeah. It was me. Somewhere in the process of switching to use CSS variables for my colors the problem went away. I probably had a background forced to something unfortunate.

I've switched from the Ink markdown processor to Down. I liked that Ink was pure swift, but I kept spending time enhancing it to meet my needs. Down uses libcmark, which I'm not wild about having inside my server, but hopefully I won't spend any more time thinking about it.

It's a testament to both that swapping took just a couple minutes. Half of it finding the typo in my Packages.swift file.

print("Hello World! I'm beautiful!")

As long as I was fiddling with femtoblogger I touched up the CSS so code will stand out a little better, and then splurged and added Prism to highlight source code for you. I feel a little bad about the 12k of javascript, but you only load it once. We will survive this.

So far I recommend Prism. It was a CSS add in <HEAD> and a <SCRIPT> add by </BODY> and everything worked.

The biggest add to femtoblogger is an out of band management interface, but you don't get to see that.

I was typing up my thoughts on using SwiftUI and broke femtoblogger! My screed gets truncated about half way through.

I'm not sure if that means I should:

  • settle down?
  • fix femtoblogger?
  • tighten up my writing?

Who am I kidding… where did I leave that source code…

Update: I try to be a good modern programmer and haul in functionality from strangers on GitHub instead of spending weeks writing it myself… and then… BAM. The Markdown to HTML translator I'm using would allow raw HTML through, which in a public environment is irresponsible, so I kill any HTML I find. Sadly, if you include <table> in your comment, the markdown engine thinks everything from the opening tag to the end is the HTML and I kill it.

I guess I'll have to add a real "safe mode" to the markdown translator. The author hasn't integrated a pull request in months, that is kind of deterrent to making nice additions.

Update: Three hours to study the markdown processor enough to figure out how to cleanly add a safe mode and tests. I'd have been done faster until I realized it was written by John Sundell. Google frequently brings me his articles for solving Swift conundrums. He deserves an extra tidy patch.

The robots are reading at least. The rate is low enough I'll just manually delete them for now.

I used to use the google recaptcha, but I know I'm annoyed at finding stop lights and street signs and in a more tracking sensitive era I don't feel like feeding people's reading habits to third parties.

That probably means I'll eventually have to make my own human detector system and lose a week to that.

Well that was a fair bit of gap in between posts.

Femtoblogger is now on its 4th implementation. I feel the need to try out Swift on the server, so here we are. 100% pure Swift Femtoblogger.

The previous incarnation was Go. Before that was PHP. There was probably a never deployed Python version in there too.

Thoughts on Swift and the design of the server this time around…

  • I very much like the type safety of Swift. When I code in Swift there are no runtime explosions, and very few bugs discovered in testing.
  • The Swift code is more information dense than the Go code was. The Go code felt like a great sea of text with a little bit of content. The Swift code packs a lot of content into a screen of code.
  • Performance appears to be a non-issue. SQLite backed pages are serving in 3-12ms. For comparison I added a caching layer for most popular queries and dropped that into the 80µs range for the query handling. That's not real, I handle them by doing a X-Accel-Redirect header back to Nginx and letting it serve a file out of a RAM disk directly. But it means my process is not going to be buried.
  • I haven't bothered with any compiler optimization. I want the checks to stay in place while I see if I've screwed up somewhere.

The Network End of Things

  • The Swift server network library, NIO, really wants to push you into a futures and promises style of programming. I resisted because it is so clumsy at the source code level. Your code begins to vanish under a sea of ritual. That said, I probably would have been better off to embrace it. A request needs to get out of the network queues to free them up while it gets in a non-concurrent model queue to work with the data, then out of there as soon as possible and into a render queue to do template expansion and back into the network queues to send the data out. It's a lot to coordinate.
  • I suppose I'll continue avoiding futures and promises in femtoblogger until Swift 6 and the beginning of its new concurrency model make it into NIO.
  • I didn't use Vapor or similar web frameworks, I build my server directly on NIOHTTP1 so I'd understand the architecture. The first 90% was simple! Then I needed to parse some form data and couldn't find the calls in the API. Eventually I found an email in a filing cabinet behind a sign that said "Beware of the Leopard" where the NIO team proclaimed that "out of scope". Maybe I'll make a post on parsing "multipart/form-data" (which can't be a String because it might not be valid) and parsing HTTP header values.

Using Other People's Packages

  • I did reach out for 'random dude' packages for a higher level SQLite API, Markdown to HTML conversion, and template expansion. It's a bit of a mixed bag there. I'm glad I did it, but there were issues. As always it seems to take a lot of research time to figure out which solution is well supported, not horrible, and will likely last a while.
  • I used SQLite.swift for accessing SQLite. The tutorial writing crowd on the internet likes it. It gives you a type safe way of interacting with SQLite without too much ritual. The package is a few years old and has been dormant for most of a year. I found a design flaw where it would try to read a mismatched datatype and explode your process. (Just because you called a SQLite column TEXT doesn't mean it isn't going to return a BLOB for one of the rows.) It looked to be an involved fix to use the receiver's datatype instead of the provider's, and in the absence of an active maintainer I didn't want to invest in making the changes. It was easier to rewrite all the rows to have their TEXT really be TEXT.
  • I used Ink for Markdown to HTML conversion. It is simple to use. Out of the box it choked on CRLF line endings. The patch ended up being a , "\r\n" in just the right place, plus about 16 lines of test cases. Hint: "\r\n" is a single Character in Swift… grapheme clusters and all. The maintainer is active so I sent that one in.
  • I used Stencil for template expansion. I've used it before and the rough edges I used to have to hammer off are now addressed. Note: Stencil could really use a SafeExpansion mode which escapes HTML for when you have untrusted users. It has user created filters, I ended up making one instead of forking and adding a safe expansion feature.

I made a video of my cat chasing the shadow of her tail. It got posted to a social media site. I have contributed to videos of kittens.

After too many years of femtoblogger succumbing to PHP language drift, I’ve been idly reimplementing it in Go.

I’ve been through several iterations wherein I tried to force sophisticated abstractions onto Go, with rather poor success. I finally realized that Go deliberately eschews abstraction. I gave up abstracting and just wrote out the code in as many minor variations as required to do the job, and it was a much more pleasant experience.

In the end I’m looking at about 2,000 lines of Go (not counting imports I didn’t write) which has most of the functionality of the 3,000 lines of PHP. So, within a power of two, they appear to be a wash.

femtoblogger has gained some important ground:

  1. It is statically type checked. Typos and errors won’t be lurking around to explode at run time.

  2. I have an executable which will continue to run until I break it. No more will the web site disintegrate when I update PHP on the server because the language drifted or a Debian packager changed a setting.

  3. I feel better about its security. PHP always made me slightly nervous.

  4. I’ve ditched WYSIWYG editing for markdown. The HTML folk have had plenty of time to make editing work, it’s not my problem if they can’t get their act together without thousands of lines of Javascript repair code.

So here’s to another 7 years of femtoblogger. Who knows what I’ll rewrite it in when 2021 comes around.

Safari is odd with contentEditable divs. It doesn’t assume you will have a pre-formatted div as a container so it puts each line into its own div… sometimes marked with code class, sometimes not.

That makes a mess but is tolerable, until my HTML sanitizer burps out the clean version with newlines between the divs and causes accidental double spacing.

I could rewrite the ->saveHTML() method of the PHP DOMDocument to not put newlines in between divs, but that would give awful looking HTML.

For now I added a function to the sanitizer to remove extra divs from inside code formated divs. It tries to be smart about inserting newline characters, but it may not be smart enough.

Someone should revisit the whole contentEditable thing and specify precisely what is to meant by all of the operations.

femtoblogger has reached that odd state for software. It works well enough that I am happy using it. There are rough edges, but not rough enough that I will fix them.

The only thing I have changed recently is to add a meta robots tag to suggest the aggregate pages, like the front page and archive months, not be indexed. That should help keep the clicks on target. I already had robot tags to deter indexing of all the non-content pages.

There remain two rough points:

  1. The WYSIWYG editing: This is still a bit awkward. Sometimes I get stuck in bold and have to pop into HTML mode to get out. Pasting in code ends up double spaced. I could make lots of elaborate workarounds, but I consider these to be browser bugs and hope they will shake themselves out over time. I could also switch to one of the giant WYSIWYG javascript editors, but then I wouldn’t be very femto, would I.
  2. I keep having a nagging desire to have images. I could do it now by attaching the image and making my own IMG tag, but I’m too lazy for that. I’ve been holding off coding proper image support (with resizing for display and full resolution click through) on the grounds that if I’m too lazy to type my own image tag then surely I shouldn’t spend a couple hours making full image support.

I suppose since femtoblogger has become stable it is time to move it into a public subversion repository.

I’ve added a few features today.

  • There is now a ‘recent comments’ link under Contribute. It also has its own RSS feed, so I can watch for comments should the comment spammers get through.
  • Articles now have URLs constructed from their titles. I feel I could do better at this. I need to read some papers on automated abbreviators.
  • I fixed a nasty little problem where a sqlite3 “insert or replace” was whacking my password. Turns out that deletes the whole row on a replace, so you better be specifying all the field values.

And now 4 days later.

  • I now log much of the POST methods into my database. There is a distributed botnet trying to leave a comment on one article in this blog. I’m curious what they wish to say. I’d hate to miss out on a first contact situation.
  • Found a couple URLs that were missing their “.php”.  Apache was ok with this, lighttpd is not.
I have made contact with the robots. We should all be afraid. Thus far the robots have attempted to add these comments:
  • SEX
  • SEX
  • SEX SEX SEX LOVE
  • zubav1na-ps1h1chesk1e-bolezn1  except the digits 1 are supposed to be the letter 'i', I just didn't want to get indexed by it.
I suppose some filtering software will now block my site because it talks about sex.
More robot chatter:
  • fandango
  • tatuazh
So if a tattooed robot offers to dance the fandango with you, you should know it only wants sex.
more articles