Marzipan and Electron

Chris is missing the point of both technologies. And I'm sure not a brat Millennial.

Marzipan (candy frosting) is a legacy porting technology: Existing iOS apps can cost more to port to AppKit than they're worth, but may be worth something as a cheap Marzipan port. Nobody ports their iOS apps to tvOS or watchOS because it's not profitable, and everyone (in the first world with money) has an iPhone already.

I loved my UNIX® workstation Macs after suffering with Linux for a decade+, but Timmy Cook's Apple abandoned the Mac after Steve's death and Scott Forstall's firing. Anyone making new native Mac apps is in an abusive relationship: Apple does not love you, and does not care about the Mac.

I'd rather eat broken glass than run Linux again, and I have never and will never be a Windows weenie, but I'm not relying on Apple to support desktop developers ever again.

Apple's Mac apps have generally been shit for years now, because they won't spend the resources to develop & support their own stuff. iTunes is a bloated pile of crap, half-broken because it has to run on Windows, too; you don't like hybrid web apps? Everything except your library is web or XML rendering. Pages and Numbers were fast, minimally useful apps that got rewritten based on the iOS versions, and are just about useful for a memo or a chart, but not real work. Mail's a clusterfuck and not half as useful as when it supported more scripting and addons.

The new Mac commercials, first in years, show the broken-keyboard laptops and models they no longer make, nobody coding Mac apps, no desktop Macs. Where's that shiny "new" iMac Pro from last winter? Isn't that what a real musician would use? A near-blind photographer squints at a tiny Mac laptop instead of a giant 27" retina display?

This is how technical, developer-oriented Apple ads were in 2002:
apple_unix_ad-s

Electron and Node are the future (along with Elixir, Go, Rust, maybe others?). It's 100x faster and more fun (attach the FunMeter™ somewhere fun) to code in than Swift, you can use any good editor instead of fucking Xcode, do layout with HTML/CSS instead of the rotting corpse of Interface Builder trapped inside Xcode. And it's cross-platform; 95% of the users (even in the first world with money) don't run Mac, because Apple never updates the Macs, they failed utterly to follow-through with Macs behind iPhones. There's 20x bigger market potential.

The future is certainly not banging your head against the Swift and Xcode walls, just to make a pure Mac app nobody will see. You can't make fun of Electron's runtime, which needs Node and Chromium, if you use Swift, which has a giant runtime turd because their amateur hour C++ compiler nerds can't make a stable ABI. The Mac's only future source of native apps is Marzipan ports.

There can be performance problems in Electron, but Slack's an outlier at 196MB binary and devouring 1.2GB RAM(!!!); it's the bloated WalMart-shopping fat-ass of Electron apps. Discord is also Electron, it has a 136MB binary and uses 360MB RAM, and does more, faster and better than Slack. Atom is the original Electron, has a 541MB binary, and uses 600MB RAM, for an entire editor/IDE.

My game currently has a 139MB binary, and uses 200-300MB RAM when running. Comparing to a random casual game from my Steam library, Chainsaw Warrior (well, "casual"; I've only beaten it once on Easy). It's based on Unity (another VM!), has a 249MB binary, and uses 200MB RAM when running, plus Steam itself uses 130MB RAM (I may yet integrate Steam into mine, so that may even out). It doesn't seem excessive.

I can't compare my Swift game prototype from 3 years ago, because it was written in a version of Swift that doesn't compile in any Xcode that runs on current hardware & OS, and Xcode "helpfully" deleted the built binary; who needs working binaries, right? I might have an old Xcode on my old laptop? Maybe I could waste a couple days fixing the code by hand in current Xcode, if I hated myself or loved Timmy Cook's Apple that much?

New languages evolve fast, but I can run 20-year-old Javascript and it'll run thousands of times faster than it did in the '90s, because the language was improved with forwards-compatibility in mind, hardware caught up, and the newer VMs compile & run it faster. I can compile 30-year-old Objective-C, and it'll run.

We had something similar to web tech 20 years ago with desktop Java, but the convicted criminal organization Microsoft sabotaged it and made a shitty single-platform ripoff called C#. Viruses became a problem for applets, which had nothing to do with desktop Java, but killed Java deployment even before Oracle bought & ruined SUN. Android runs on another Java ripoff, but their dev tools and APIs are even shittier than Xcode or C#, and the users are poor, so why make anything for them? Server-side development in Java, Clojure, or Scala, running on the Java VM, is hidden away in a back room, and made as boring as possible.

So now we have to reinvent the runtime, this time with Node & Chromium. OK with me.

Debugging a Flicker

Working on the game, and every time I mouse-click to move (finally adding that, it's been all keyboard up to now), the screen was flickering! This had never happened before. Literally 3 hours of debugging, adding voodoo CSS incantations like:

-webkit-backface-visibility: hidden;
-webkit-tap-highlight-color: rgba(0,0,0,0);

which did nothing…

Finally made a new test project with just the canvas code and a mouse handler. Still not flickering. Then I add random lines on the screen to see some content, need to find the canvas size to make those lines, and that flushes out the offending line:

1: canvasSize() {
2:  const canvas = $("canvas");
3:  const size = [canvas.offsetWidth, canvas.offsetHeight];
4:  canvas.width = size[0]; canvas.height = size[1];
5:  return size;
6: }

See it? Line 4 is reassigning the width & height of the element, based on the display width/height. Which has to be done during setup and on window resize, but blanks out the entire canvas.

SIGH. Split that out into setup, and the flicker goes away.

It's so impossible to know what's my fault and what's HTML/CSS/JS being weird.

At least I'm only fighting a single rendering engine here. I miss doing this stuff in UIKit/SpriteKit or OpenGL ("I made a triangle!"), but the former's sinking in the Swamp of Swift, and the latter's RIP deprecated, and I'm hardly going to chain myself further to Apple with Metal.

Learn2JS

I've updated the Learn2JS shell, which makes it easy to get started writing client-side Javascript.

  • Added eslint tags and an initial config, so you can check for errors while editing.
  • Switched from a Python web server to light-server on Node; even though the shell doesn't use Node directly, it seems more consistent.
  • Added Canvas class to simplify setting up and drawing on HTML canvas.

Any suggestions or feedback? I just use this myself to script things quickly, but I'm happy to enhance it into a better learning tool.

Learn 2 JS

Almost the simplest possible tool for coding in Javascript, slightly above Hello, World!, I packaged up how I did my Advent of Code 2017 entries (up to the point I quit for lack of time/sleep). I start with something like this in every JS project, and now you can have it.

  • Setup

Download learn2js

Unzip it to Documents (or wherever you like).

Open Terminal on a Mac; or xterm on Linux; or I have no idea what the terminal/shell situation is on Windows, feel free to enlighten me.

% cd Documents/learn2js
% ls
run.sh  src
% ./run.sh
Serving HTTP on 127.0.0.1 port 8000 (http://127.0.0.1:8000/) ...

This will require you have Python3 on your system to use as a simple web server, since a browser can't open multiple local files without a web server, but there's no other Python in this.

Right-click that URL (the part in parentheses!) and open it in your browser, or copy-paste it, and you should see a simple page with an Input: box. Anything you type is uppercased, that means it's working!

  • Coding

Now edit src/main.js with your editor of choice, like BBEdit on a Mac; or Atom anywhere. Please don't use Notepad on Windows, even if it handles correct line endings now.

Should be pretty self-explanatory, the TODOs tell you where to start.

Anytime you make a change to the source, just reload the page and hit Run again. Just like hacking on BASIC code back in 1980, dudes & dudettes!

  • Exercises for the Student:
    • I didn't make a favicon for this, and you should, which will introduce you to the horrible world of W3C/WhatWG specifications.
    • You might want to move inputTable below the output div in index.html, but then it scrolls down all the time, so now you need to anchor it at the bottom and make output fixed in place and scrollable, as I do in Mainmenu… I may add that option later, but it complicates a very simple page.
    • I didn't set up eslint for this, configuring it is somewhat annoying; probably will next time I update it.
    • Setting up Node is even more of an advanced topic, and 90% of the interesting work is in front end JS like this.
  • Troubleshooting: If a change isn't appearing, you may need to empty your web cache: In Safari, turn on Develop menu from Preferences>Advanced, then Develop>Empty Caches; in Chrome, Chrome>Clear Browsing Data and then only select "Cached images and files"; I have no idea what you do in Edge on Windows. I can't easily fix that, and users would never see it, but you will.
  • License: I put all the code under BSD license, the documents under Creative Commons Share-Alike. You can keep your code more or less private (except in practice, you have to let everyone see your page to let it load!), but if you make changes, share them and link back to me. Don't be a Stallman. Note in particular, you cannot relicense this under a restrictive license like GPL, and don't just put it on github unmodified. Sorry to have to preach about this, but some people need to be told where the lines are.
  • Shipping: To let others see your masterpiece, just change DEBUG to false, upload everything in src somewhere, your own site or something like Neocities, and you're done! You only need run.sh & Python on your local system.

Eloquent Javascript

A free, up to date, possibly good book on JS programming? Flipping thru, a few things pop out at me.

This is a petty pet peeve, but I greatly dislike that he writes arrow functions without parens:

n => { return n * n; } instead of (n)=>{ return n * n; }

When they are required for multiple arguments: (x, y)=>{ return x * y; }

On first appearance, he dismisses arrow functions as just being shorter than function expressions, which is incorrect (arrows fix the 'this' reference which is never correct in function expressions). But then he consistently uses arrow functions (in his ugly parens-elided style), so crisis averted?

"Every now and then, usually between eight and ten in the evening, Jacques finds himself transforming into a small furry rodent with a bushy tail."

Which example then leads into a statistical analysis story, and the kind of data hackery that JS (and Python) are very good for.

The robot delivery example is another fairly detailed story with pathfinding, tho his algorithm is defective (it fails and/or consumes all memory forever on more complex graphs than the very simple one given).

I'll have a look at the rest of the book later.

None of the examples thus far actually build and run in a web page, or any sort of UI, except in the online document. You can copy-paste these examples into Safari's console and run them. I really don't think it's useful to learn a language outside the context of a running environment, so next post I'll give you one.

Tower of Babel

Ugly shit from my package.json:

 "scripts": {
    "start": "electron .",
    "build": "rm -rf build && babel app -d build/app && cp -R assets package.json build && cp app/*.html build/app && cd build && npm install --only=prod && cd ..",
    "dist-mac": "electron-packager build --out=dist --asar --overwrite --icon './assets/i/appicon.icns' --platform=darwin",
    "dist-linux": "electron-packager build --out=dist --asar --overwrite --icon './assets/i/appicon-256.png' --platform=linux",
    "dist-windows": "electron-packager build --out=dist --asar --overwrite --icon './assets/i/appicon.ico' --platform=win32",
    "dist-all": "npm run dist-mac && npm run dist-linux && npm run dist-windows",
    "dist-help": "electron-packager --help 2>&1 |less"
  },

So I wanted to filter my source through babel-minify.

Build out to a "build" dir… and everything breaks. Copy in assets, package, HTML files… "Missing module". Turns out known issue, closed but I don't see an actual solution: Nuking & reinstalling my node_modules did nothing, and I have latest stable versions (maybe "stable" is the problem?) of everything.

As a brute-force Hulk-smash solution, shoving that "npm install" in the build script works, and is easier than figuring out what I should change.

Javascript NaN

I just spent 30 minutes reading log files to find an actual bug that would've been prevented by a strong type system: Multiplying a coordinate by an array instead of an element of that array; Javascript helpfully gave me the result NaN and carried on instead of throwing an exception, because Javascript. I may put a few defensive asserts of Number.isInteger(n), Number.isFinite(x) in functions that process numbers.

This is the first such bug I've had in so long I can't think of the last. Many years not spent fighting BDSM type systems and slow-ass compilers, so I'm still happy with this choice.

Javascript Use Strict

Every Javascript file should be in Strict Mode, and if you run eslint — which you absolutely should — you should specify an eslint-env:

/* file header */
/* eslint-env node */

"use strict";

So just as a paranoid check, I wrote this script which I'll call from my build script (in the src folder, not project base where node_modules lives!):

jsStrictCheck.zsh

#!/bin/zsh
err=0
for f in **/*.js; do
    grep "\/\* eslint-env" "$f" >/dev/null
    if [[ $? != 0 ]]; then
        echo "$f: Missing /* eslint-env node */"
        err=1
    fi
    grep "use strict" "$f" >/dev/null
    if [[ $? != 0 ]]; then
        echo "$f: Missing \"use strict\";"
        err=1
    fi
done
exit $err

Already caught one file without strict, one without eslint-env, so my paranoia is justified.

New Electron Dance

No, wait, that's the Neutron Dance, I get those confused.

Electron

Since abandoning any hope of the iOS App Store paying my bills, I've had to look back at the web or desktop. My current available time-at-computer and energy these days isn't sufficient for a day job or even contracts, much to everyone's dismay. So time for another hard look at the situation.

I like working on my Mac, but Mac isn't that big a market. I also want to ship on Windows (and Linux, I suppose). Objective-C is one of my favorite languages ever, Cocoa & UIKit (on iOS) were great APIs, AppKit on the Mac much less so, but since Apple's killed Obj-C and it's not portable, my happy years of typing [ ] are over.

WHAT HAVE YOU DONE?!

Swift might be the worst mental disorder to strike programmers in decades. Swift is orders of magnitude slower than Objective-C, crashes constantly, the moving-target "spec" creates incompatible changes every year, and because they're too stupid to standardize a binary interface, every program has a 20MB+ blob of Swift runtime. For a single-platform joke language perpetrated by a C++ bozo who fucked off after a year to play with cars. So I'm all too happy to say good riddance to that bullshit. I mean exactly this: If you're using Swift, you either don't know better (it's OK to say you don't know!), or are defrauding your employer for hours, or have something wrong inside.

13 years ago, Project Builder/Interface Builder was a pretty good dev toolkit since I could use a real editor (BBEdit) with it, but Xcode locked that out, and then as Apple sucked in more tools over time, it sucked harder and harder; I can't stand the rickety deathtrap these days. I was getting by in JetBrains' AppCode, but still had to use Xcode for Interface Builder (RIP) and to get builds onto a device half the time. Xcode is a crashy, substandard pile of shit with maybe the worst editor in any IDE in history. Syntax highlighting stops working at random, for most of a decade it has code-completed "nss" as "NSStreamDelegate" rather than the slightly more useful "NSString" (before that it couldn't code-complete at all!), I could go on for hours or days about how Xcode kicks you in the input/output ports every time.

And the worst part is you can't fix the fucking thing, no user-serviceable parts inside, Radar is a black hole, no scripting or plugins. Just bend over and take what Apple Developer gives you good and hard. It's kind of a relief that current Xcode doesn't run on the last stable MacOS version (Sierra).

I'll stick with BBEdit for text and Atom for code, thanks. If I'm angry at Atom I can fix it myself or file a publicly-trackable ticket; I'm rarely angry at BBEdit but I can ask Rich to fix it.

So I'm writing web-type software in Javascript, with Node or Electron behind it. Javascript aka ECMAScript has become a good language in the last 5-10 years, and the V8 runtime in Node/Electron runs close enough to native now for most needs. I love that I can just write UI in HTML again. No fucking around with Apple's bullshit of deprecating APIs out from under me (I "get" to rewrite alert/menu code again?!), or promising to support SpriteKit/SceneKit across iOS & Mac and then doing fuck-all on either. WebGL (or Three.js, anyway) isn't fast enough for complex scene-graphs, but 2D work in Canvas is mostly fine (and it gets better every year, instead of bit-rotting like unused S*Kit APIs). localstorage in a web page isn't enough for any real program, thus Node is needed to reach the filesystem.

I slander Swift for leaving a giant runtime turd in every program, but Electron's the same way: It has to contain a browser, Node, and system APIs. But I'm not at the mercy of Apple's marketing-driven dev tools.

Certain Mac nerds obsess about Purity of Essence, insisting that everyone should love Xcode, Swift, and AppKit, and that use of any other technology is an abomination to the end-users, whom they clearly love more than me. Can you hear that slurping sound? That's someone fellating Apple marketing. Roughly 4 billion more people are familiar with web pages and will find a web-like UI more comfortable.

I intend to keep up my experiments in Scheme and Pascal when I have time, I'd far rather have small, fast, native binaries on every platform, but shipping beats purity.

Progress is being made:

tile-20180426-map

tile-20180426-view

(the + road texture there will get replaced soonish)

Advent of Code 2017: Week 1

After 7 days, let's see how my Advent of Code 2017 is going.

  • Web framework: I made a standard web console, which I then copy forward to the next day. I could just as easily have put all the buttons on a single page, but it'd get too long by day 31.

  • Unit testing: As seen in stdlib.js, my test framework is very simple: On page load, setup, run some asserts, finish to get stats and redbar/greenbar the test console. This has been a great win, even though several of the days had only one or two examples.

  • 01:

    "The captcha requires you to review a sequence of digits (your puzzle input) and find the sum of all digits that match the next digit in the list. The list is circular, so the digit after the last digit is the first digit in the list."

    • So this is nice and easy, linear problem for JS string processing. On the second variant I turned the inner function to pick the next char into a function, which I pass in. No external state.
  • 02:

    "The spreadsheet consists of rows of apparently-random numbers. To make sure the recovery process is on the right track, they need you to calculate the spreadsheet's checksum. For each row, determine the difference between the largest value and the smallest value; the checksum is the sum of all of these differences."

    • Needed to write more utils, this time to process strings into a table of numbers. My first solution kept it as a table of strings, and then I had problems with JS type coercion. Same strategy of finding a pure functional inner loop and extracting that as a function. Actually, subtotalFunc1 mutates cols, but you'll never use it again so it doesn't matter.
  • 03:

    "Each square on the grid is allocated in a spiral pattern starting at a location marked 1 and then counting up while spiraling outward. For example, the first few squares are allocated like this:

    17 16 15 14 13
    18 5 4 3 12
    19 6 1 2 11
    20 7 8 9 10
    21 22 23---> ...

    While this is very space-efficient (no squares are skipped), requested data must be carried back to square 1 (the location of the only access port for this memory system) by programs that can only move up, down, left, or right. They always take the shortest path: the Manhattan Distance between the location of the data and square 1."

    • This was a little insane. There's apparently a pure math solution, but I am not a mathematician, I am a turtle; also song; also Lewis Carroll; also Gödel Escher Bach. So I solved it by moving a turtle around the spiral, turning left whenever there's an open space, and then I could just examine the spiral for values.
    • Yes, I store points as a vector-2 of numbers, rather than making a "Point" class with x,y. In C using a struct makes more sense (since it only takes 8 bytes intead of the 40+ in any object system!), but in anything else, the vector requires the least memory, least work, and is easiest to serialize, use as a hashtable key, and so on.
    • I just copy-pasted and modified for the second task, instead of making a nice inner function to pass in. It's not incredibly hard to parameterize, but I would need to break out of that inner function to return early, so probably using an exception for flow control?
    • Javascript objects are super-useful, like Python dicts but even easier to use. Stuff a bunch of points for the grid, and properties for the minPt, maxPt (bounds) and lastPt, one structure gets to hold everything about the spiral. Writing this in a type-safe or pure functional language would be annoying.
  • 04:

    "A passphrase consists of a series of words (lowercase letters) separated by spaces.
    To ensure security, a valid passphrase must contain no duplicate words."

    • Super simple, a histogram that only counts to 1. The second part only requires sorting the chars in each word, so you can check they aren't anagrams. I was worried that day 3 was the start of an exponential curve in difficulty, so by the end it'd take to the end of the Earth to finish.
  • 05:

    "The message includes a list of the offsets for each jump. Jumps are relative: -1 moves to the previous instruction, and 2 skips the next one. Start at the first instruction in the list. The goal is to follow the jumps until one leads outside the list.
    In addition, these instructions are a little strange; after each jump, the offset of that instruction increases by 1. So, if you come across an offset of 3, you would move three instructions forward, but change it to a 4 for the next time it is encountered.
    How many steps does it take to reach the exit?"

    • I returned the program counter at first, and that passes the sample data test, so I "guessed" wrong the first time, but then reread and found my bug. Having only one example is a problem. Both parts are as usual solved by passing in a function to determine the next state of the instruction. Trivial one aside from my stupid bug.
    • I'm building up a good library of tools by now.
  • 06:

    "In each cycle, it finds the memory bank with the most blocks (ties won by the lowest-numbered memory bank) and redistributes those blocks among the banks. To do this, it removes all of the blocks from the selected bank, then moves to the next (by index) memory bank and inserts one of the blocks. It continues doing this until it runs out of blocks; if it reaches the last memory bank, it wraps around to the first one.
    The debugger would like to know how many redistributions can be done before a blocks-in-banks configuration is produced that has been seen before."

    • Making the balancer was straightforward, I stored each state as a string in a history array, and a parallel histogram to catch the duplicate; I could get rid of hist but it turned out to make the second task easier.
  • 07: (Don't read this or the code until tomorrow if you don't want a spoiler)

    "You offer to help, but first you need to understand the structure of these towers. You ask each program to yell out their name, their weight, and (if they're holding a disc) the names of the programs immediately above them balancing on that disc. You write this information down (your puzzle input). Unfortunately, in their panic, they don't do this in an orderly fashion; by the time you're done, you're not sure which program gave which information.
    Before you're ready to help them, you need to make sure your information is correct. What is the name of the bottom program?"

    • Building the tree was a good puzzle, but not hard: Parse the text into nodes, keep them in a dictionary, then build the structure, and return the only parentless node (the root). Making a recursive toString so I could debug it was important…
    • Second task looked to be tedious (depth-first search and pass the result all the way up), but then I just looked at my output and saw the unbalanced numbers, so entered it by hand. I am a computer, too.

I've got both gold stars each day. I'm completely incapable of reliably checking in at exactly 21:00 PST, and I'm too fussy about my code to ever be "the fastest", so my rankings are awful; maybe it ought to count from when you read the problem set, but then everyone would cheat on at least the first task.