Python 3.7

  • Python 3.7 released: Standard checklist:
    • Run installer
    • Delete the old 3.6 folder from /Applications
    • Run the certificate command in the new 3.7 folder (the other shits a PATH into my shell profile, don't need it)
    • Run IDLE and verify it's 3.7.0. Happily, no longer have to fight with updating Tcl/Tk.
    • Run "python3" from Terminal and verify it's 3.7.0
    • Run a random Python script to make sure nothing's broken.

Nanosecond-accurate time functions and switching more ASCII/C Locale into UTF-8 are nice improvements, but those are more patching up legacy annoyances than "must have".

I'm mostly interested in dataclasses, which makes it much easier to build little struct-type objects instead of random dicts or lists which have all sorts of problems (no equality, hashing, typo-safety).

I greatly dislike the addition of BDSM typing, but it's mostly optional, EXCEPT you have to use them in dataclasses:

from dataclasses import dataclass
@dataclass
class Point:
    x : float = 0.0
    y : float = 0.0

>>> p = Point()
>>> p
Point(x=0.0, y=0.0)
>>> q = Point(1.1, 2.2)
>>> q
Point(x=1.1, y=2.2)

If I define Point without the type annoytations[my new favorite typo!], only the default constructor works, and it doesn't print the fields as a string.

@dataclass
class Pointless:
    x = 0.0
    y = 0.0

>>> f = Pointless()
>>> f
Pointless()
>>> f.x
0.0
>>> f.y
0.0

Real examples might be a lot more complex than a point, and by then the cost of building a proper class with __init__ and everything yourself isn't such a big deal, so I can see dataclasses mostly being used for very simple struct-like containers.

A Little Scheme

I go through phases of playing with Scheme for utility code, maybe even portable dev; while FreePascal is a better language for this, I'm frustrated by the lack of library support and the useless iOS situation.

Scheme's always been an emergency backup language; it was fun to learn back in the '80s and early '90s, and both SICP and TSPL are good books, but nobody wanted to pay for Scheme dev, and anyway the language is very annoying to write. I often treat it as a logic puzzle to get anything done, not a useful tool. But it does have good library support, and it can compile to very fast binaries, despite having GC pauses and consuming 2x as much memory as a C program. Maybe I can get better at solving problems in it, build up some libraries, and make it useful?

So the current landscape is:

  • Chez Scheme:
    • Pros:
      • Very fast to compile and at runtime, competitive with C compilers.
      • Great interactive REPL, not just a half-broken readline history like pretty much every other Scheme.
      • Debugger is reasonably good, and integrated in the REPL. All I really use myself is (trace FOO) and (inspect BAR), but non-caveman coders will make better use of it.
      • Current R6RS implementation plus extensive chezscheme library.
      • By R. Kent Dybvig, author of TSPL, and Cisco currently employs him to maintain Chez Scheme.
      • REPL environment is called a café, which I find charming. Yes, I also liked all the coffee puns and iconography from early Java programming.
    • Cons:
      • Not as widely supported by tools & documentation as Racket.
  • Racket:
    • Pros:
      • Very nice GUI.
      • Current R6RS implementation plus extensive racket library.
      • Built around making multiple languages; I don't really care about this. I loathe "Typed Racket", one of the worst combinations of ideas in history.
      • Tons of documentation.
    • Cons:
      • Mediocre performance. There's a project to rehost Racket on Chez Scheme, which would fix this, but then why use Racket?
      • Doing anything in the GUI destroys your environment, all the objects you've made, unlike any LISP or Scheme ever. So it's utterly fucking useless as an interactive REPL. I can't say enough bad things about this. ★☆☆☆☆ Kill On Sight.
  • Chicken:
    • Pros:
      • Compiles to C and thence to native binaries, with nice FFI to C libraries.
    • Cons:
      • Mostly old R5RS, with a few extension libraries.
      • Terrible REPL, only really usable as a compiled language.
  • Scheme R7RS benchmarks

Chez Scheme is the clear winner for me; if I was a novice, I might choose Racket and not realize that the REPL is a broken abomination for a while. If I was only doing C interop, Chicken would be better.

Editing in BBEdit works OK, but it doesn't know how to find function definitions. I guess Vim has current syntax, but I'm kinda over that habit unless I have to sysadmin. I have never been emacsulated and never will.

Atom's symbols list doesn't do any better. But if you do want to use it, install package language-racket (all other language-schemes are R5RS at best), and then add some file types to config.cson:

"*":
  core:
    customFileTypes:
      "source.racket": [
        "scm"
        "ss"
      ]

In any editor, any language, I use hard tabs (1 char = 1 logical indentation level, obviously), and normally tabstop at 8 chars which discourages very long nesting and encourages me to extract functions. Scheme is indentation hell, so set the tabstop to 4 spaces. (The code blocks below won't show that.)

Do not criticize my C-like paren/brace placement; I prefer clear readability of code structure to some obsolete Emacs dogma.

So, let's see it work, with hello.ss:

#!/usr/bin/env scheme-script
(import (chezscheme))
(format #t "Cheers 🍻 , ~a!~%" (car (command-line-arguments)))
(exit)
% chmod 755 hello.ss
% ./hello.ss Mark
Cheers 🍻 , Mark!

Now for something more serious:

stdlib.ss:

;; stdlib.ss
;; Copyright © 2015,2018 by Mark Damon Hughes. All Rights Reserved.
(library (stdlib)
    (export inc! dec! currentTimeMillis randomize input atoi)
    (import (chezscheme))

;; Variables

(define-syntax inc!
    (syntax-rules ()
        ((_ x)      (begin (set! x (+ x 1)) x))
        ((_ x n)    (begin (set! x (+ x n)) x))
    )
)

(define-syntax dec!
    (syntax-rules ()
        ((_ x)      (inc! x -1))
        ((_ x n)    (inc! x (- n)))
    )
)

;; Date-Time

(define (currentTimeMillis)
    (let [(now (current-time))]
        (+ (* (time-second now) 1000)
            (div0 (time-nanosecond now) 1000000))
    )
)

;; Random Numbers
;; "Anyone who attempts to generate random numbers by deterministic means is, of course, living in a state of sin." —John von Neumann

(define (randomize)
    (random-seed (bitwise-and (currentTimeMillis) #xffffffff) )
)

;; Input/Output

;; Reads a line from stdin, ends program on EOF
(define (input)
    (let [(s (get-line (current-input-port)) )]
        (if (eof-object? s)
            [begin (display "Bye!\n")
                (exit)
            ]
            s
        )
    )
)

;; Strings

;; Converts a string to an integer, 0 if invalid
(define (atoi s)
    (let [(n (string->number s))]
        (if (eqv? n #f)
            0
            (inexact->exact (truncate n))
        )
    )
)

)

guess.ss:

#!/usr/bin/env scheme-script
;; guess.ss
;; Copyright © 2015,2018 by Mark Damon Hughes. All Rights Reserved.

(import (chezscheme))
(import (stdlib))

(define (guess)
    (display "I'm thinking of a number from 1 to 100, try to guess it!\n")
    (let [(theNumber (+ (random 100) 1))]
        (define guesses 1)
        (do [(break #f)] (break)
            (format #t "Guess #~a? " guesses)
            (let [(g (atoi (input)))]
                (cond
                    [(or (<= g 0) (>= g 100))
                        (display "Try a number from 1 to 100.\n")
                    ]
                    [(< g theNumber)
                        (display "Too low!\n")
                        (inc! guesses)
                    ]
                    [(> g theNumber)
                        (display "Too high!\n")
                        (inc! guesses)
                    ]
                    [else
                        (display "You got it!\n")
                        (set! break #t)
                    ]
                )
            )
        )
    )
    (display "***GAME OVER***\n")
)

(randomize)
(guess)
(exit)

chez-compile.zsh, with my thanks to Graham Watt for explaining wpo and libraries:

#!/bin/zsh
if [ $# -ne 1 ]; then
    echo "Usage: chez-compile.zsh MAINNAME"
    exit 1
fi
rm -f *.so
rm -f *.wpo
mkdir -p bin

cat <<ENDTEXT |scheme -q --optimize-level 3
(compile-imported-libraries #t)
(generate-wpo-files #t)
(compile-program "$1.ss")
(compile-whole-program "$1.wpo" "bin/$1")
ENDTEXT

rm -f *.so
rm -f *.wpo
if [ -f "bin/$1" ]; then
    chmod 755 "bin/$1"
fi

Now I just:

% chez-compile.zsh guess
compiling guess.ss with output to guess.so
compiling stdlib.ss with output to stdlib.so
((stdlib))
()
% bin/guess
I'm thinking of a number from 1 to 100, try to guess it!
Guess #1? 50
Too low!
Guess #2? ^DBye!

Well, that was an adventure to get the equivalent of my first BASIC program from 1980, which can be run in Chipmunk BASIC if you don't happen to have a TRS-80 Model I handy:

1 REM GUESS. COPYRIGHT (C) 1980,2018 BY MARK DAMON HUGHES. ALL RIGHTS RESERVED.
5 RANDOMIZE INT(TIMER()):FOR I=1 TO 10:A=RND(1):NEXT I:REM CHIPMUNK'S RANDOMIZE SUCKS
10 N=INT(100*RND(1))+1:T=1
20 PRINT "I'M THINKING OF A NUMBER FROM 1 TO 100, TRY TO GUESS IT!"
100 PRINT "GUESS #";T;"? ";:INPUT "",G
110 IF G<=0 OR G>=100 OR G<>INT(G) THEN 200
120 IF G<N THEN 210
130 IF G>N THEN 220
140 GOTO 230
200 PRINT "TRY A NUMBER FROM 1 TO 100.":GOTO 100
210 PRINT "TOO LOW!":T=T+1:GOTO 100
220 PRINT "TOO HIGH!":T=T+1:GOTO 100
230 PRINT "YOU GOT IT!":PRINT "*** GAME OVER ***"
240 EXIT:REM CHIPMUNK

But now I can think about more complex problems in Chez Scheme!

Here's the tiniest piece of what I've been thinking about next:

Island in emoji

CS Larva

CS Larva
My first infinite loop was summer, age 10, only because I hadn't seen a computer until the year before. My pre-college computing skills were 90% self-taught; LOGO class at ~12 and a year of "comp sci" at 14 were minimal. ⌨️🖥

Do Something Weird

"This is the kind of possibility that the pointy-haired boss doesn't even want to think about. And so most of them don't. Because, you know, when it comes down to it, the pointy-haired boss doesn't mind if his company gets their ass kicked, so long as no one can prove it's his fault. The safest plan for him personally is to stick close to the center of the herd.

Within large organizations, the phrase used to describe this approach is "industry best practice." Its purpose is to shield the pointy-haired boss from responsibility: if he chooses something that is "industry best practice," and the company loses, he can't be blamed. He didn't choose, the industry did.

I believe this term was originally used to describe accounting methods and so on. What it means, roughly, is don't do anything weird. And in accounting that's probably a good idea. The terms "cutting-edge" and "accounting" do not sound good together. But when you import this criterion into decisions about technology, you start to get the wrong answers.

Technology often should be cutting-edge. In programming languages, as Erann Gat has pointed out, what "industry best practice" actually gets you is not the best, but merely the average. When a decision causes you to develop software at a fraction of the rate of more aggressive competitors, "best practice" is a misnomer."
—Paul Graham, Revenge of the Nerds

Advent of Code 2017: Week 1

After 7 days, let's see how my Advent of Code 2017 is going.

  • Web framework: I made a standard web console, which I then copy forward to the next day. I could just as easily have put all the buttons on a single page, but it'd get too long by day 31.

  • Unit testing: As seen in stdlib.js, my test framework is very simple: On page load, setup, run some asserts, finish to get stats and redbar/greenbar the test console. This has been a great win, even though several of the days had only one or two examples.

  • 01:

    "The captcha requires you to review a sequence of digits (your puzzle input) and find the sum of all digits that match the next digit in the list. The list is circular, so the digit after the last digit is the first digit in the list."

    • So this is nice and easy, linear problem for JS string processing. On the second variant I turned the inner function to pick the next char into a function, which I pass in. No external state.
  • 02:

    "The spreadsheet consists of rows of apparently-random numbers. To make sure the recovery process is on the right track, they need you to calculate the spreadsheet's checksum. For each row, determine the difference between the largest value and the smallest value; the checksum is the sum of all of these differences."

    • Needed to write more utils, this time to process strings into a table of numbers. My first solution kept it as a table of strings, and then I had problems with JS type coercion. Same strategy of finding a pure functional inner loop and extracting that as a function. Actually, subtotalFunc1 mutates cols, but you'll never use it again so it doesn't matter.
  • 03:

    "Each square on the grid is allocated in a spiral pattern starting at a location marked 1 and then counting up while spiraling outward. For example, the first few squares are allocated like this:

    17 16 15 14 13
    18 5 4 3 12
    19 6 1 2 11
    20 7 8 9 10
    21 22 23---> ...

    While this is very space-efficient (no squares are skipped), requested data must be carried back to square 1 (the location of the only access port for this memory system) by programs that can only move up, down, left, or right. They always take the shortest path: the Manhattan Distance between the location of the data and square 1."

    • This was a little insane. There's apparently a pure math solution, but I am not a mathematician, I am a turtle; also song; also Lewis Carroll; also Gödel Escher Bach. So I solved it by moving a turtle around the spiral, turning left whenever there's an open space, and then I could just examine the spiral for values.
    • Yes, I store points as a vector-2 of numbers, rather than making a "Point" class with x,y. In C using a struct makes more sense (since it only takes 8 bytes intead of the 40+ in any object system!), but in anything else, the vector requires the least memory, least work, and is easiest to serialize, use as a hashtable key, and so on.
    • I just copy-pasted and modified for the second task, instead of making a nice inner function to pass in. It's not incredibly hard to parameterize, but I would need to break out of that inner function to return early, so probably using an exception for flow control?
    • Javascript objects are super-useful, like Python dicts but even easier to use. Stuff a bunch of points for the grid, and properties for the minPt, maxPt (bounds) and lastPt, one structure gets to hold everything about the spiral. Writing this in a type-safe or pure functional language would be annoying.
  • 04:

    "A passphrase consists of a series of words (lowercase letters) separated by spaces.
    To ensure security, a valid passphrase must contain no duplicate words."

    • Super simple, a histogram that only counts to 1. The second part only requires sorting the chars in each word, so you can check they aren't anagrams. I was worried that day 3 was the start of an exponential curve in difficulty, so by the end it'd take to the end of the Earth to finish.
  • 05:

    "The message includes a list of the offsets for each jump. Jumps are relative: -1 moves to the previous instruction, and 2 skips the next one. Start at the first instruction in the list. The goal is to follow the jumps until one leads outside the list.
    In addition, these instructions are a little strange; after each jump, the offset of that instruction increases by 1. So, if you come across an offset of 3, you would move three instructions forward, but change it to a 4 for the next time it is encountered.
    How many steps does it take to reach the exit?"

    • I returned the program counter at first, and that passes the sample data test, so I "guessed" wrong the first time, but then reread and found my bug. Having only one example is a problem. Both parts are as usual solved by passing in a function to determine the next state of the instruction. Trivial one aside from my stupid bug.
    • I'm building up a good library of tools by now.
  • 06:

    "In each cycle, it finds the memory bank with the most blocks (ties won by the lowest-numbered memory bank) and redistributes those blocks among the banks. To do this, it removes all of the blocks from the selected bank, then moves to the next (by index) memory bank and inserts one of the blocks. It continues doing this until it runs out of blocks; if it reaches the last memory bank, it wraps around to the first one.
    The debugger would like to know how many redistributions can be done before a blocks-in-banks configuration is produced that has been seen before."

    • Making the balancer was straightforward, I stored each state as a string in a history array, and a parallel histogram to catch the duplicate; I could get rid of hist but it turned out to make the second task easier.
  • 07: (Don't read this or the code until tomorrow if you don't want a spoiler)

    "You offer to help, but first you need to understand the structure of these towers. You ask each program to yell out their name, their weight, and (if they're holding a disc) the names of the programs immediately above them balancing on that disc. You write this information down (your puzzle input). Unfortunately, in their panic, they don't do this in an orderly fashion; by the time you're done, you're not sure which program gave which information.
    Before you're ready to help them, you need to make sure your information is correct. What is the name of the bottom program?"

    • Building the tree was a good puzzle, but not hard: Parse the text into nodes, keep them in a dictionary, then build the structure, and return the only parentless node (the root). Making a recursive toString so I could debug it was important…
    • Second task looked to be tedious (depth-first search and pass the result all the way up), but then I just looked at my output and saw the unbalanced numbers, so entered it by hand. I am a computer, too.

I've got both gold stars each day. I'm completely incapable of reliably checking in at exactly 21:00 PST, and I'm too fussy about my code to ever be "the fastest", so my rankings are awful; maybe it ought to count from when you read the problem set, but then everyone would cheat on at least the first task.

Advent of Code 2017

I've joined the Advent of Code, and I'll be doing it in JS. Got my 2 stars for the day.

For good discipline (or as a handicap), I'm building a halfway-decent set of pages, unit testing framework, and sort of doing things right (good ES6 practices) instead of easy (hack some inline JS in compatibility mode). I'll link it in the sidebar tomorrow, when challenge 1 expires: My Advent of Code

Go on and do it yourself! I say "easy mode is for babies" and make it hard on myself, but really you can do this in anything. There's a perfectly nice Chipmunk BASIC or FreePascal if you're old-school.

Note: The leaderboard reset time is ridiculous, and I don't care about speed-coding or leaderboards. Don't stress about competing for first 100 completions, just do the thing.

Swiftian Satire, or Tragedy?

I honestly cannot tell if Swift developers are seriously eating Irish babies, or taking the Mickey.

From bad implementations of Equatable and Hashable, the wag leaps to:

extension GridPoint : HashVisitable {
    func hash<H: Hasher>(_ hasher: inout H) {
        self.x.hash(&hasher)
        self.y.hash(&hasher)
    }
}

Bravo! That's easily the funniest punchline to a programming joke since "where do you think the chaos came from?"

Starts with the most bizarre strawman Objective-C Sieve of Erathosthenes I've ever seen. A real implementation would be in C, because Obj-C is C with objects, and it'd be massively faster:

#include <stdlib.h>
#include <stdio.h>

typedef char BOOL; // or link in Foundation
#define YES 1
#define NO 0

int main(int argc, char **argv) {
    if (argc != 2) {
        printf("Usage: primes COUNT\n");
        exit(1);
    }
    long n = atoi(argv[1]);
    BOOL p[n];
    p[0] = NO;
    p[1] = NO;
    for (long i = 2; i < n; ++i) {
        p[i] = YES;
    }
    for (long i = 2; i < n; ++i) {
        for (long j = i*i; j < n; j += i) {
            p[j] = NO;
        }
    }
    for (long i = 1; i < n; ++i) {
        if (p[i]) {
            printf("%ld ", i);
        }
    }
    puts("");
    return 0;
}

This does use more lines of code, but they're short, low-density, and it's instantly obvious what it's doing (my predilection for 1-char var names aside). Can you actually decode his filter-based version?

func sieve(_ sorted: [Int]) -> [Int] {
    guard !sorted.isEmpty else { return [] }
    let (head, tail) = (sorted[0], sorted[1..<sorted.count])
    return [head] + sieve(tail.filter { $0 % head > 0 })
}

let numbers = Array(2...1000000)
let primes = sieve(numbers)
print(primes)

And the runtime experiment:

mdh@Aegura:~/Code/CodeC% time clang -O3 -o primes primes.c                
clang -O3 -o primes primes.c  0.04s user 0.41s system 83% cpu 0.534 total
mdh@Aegura:~/Code/CodeC% time ./primes 1000000 >~/Desktop/primes.txt      
./primes 1000000 > ~/Desktop/primes.txt  0.02s user 0.00s system 89% cpu 0.025 total

mdh@Aegura:~/Code/CodeSwift% time swiftc -O -o swiftPrimes swiftPrimes.swift
swiftc -O -o swiftPrimes swiftPrimes.swift  0.57s user 0.64s system 69% cpu 1.754 total
mdh@Aegura:~/Code/CodeSwift% time ./swiftPrimes >~/Desktop/swiftPrimes.txt  
./swiftPrimes > ~/Desktop/swiftPrimes.txt  51.79s user 26.49s system 99% cpu 1:18.78 total

So the naïve C implementation is about 3,151x faster. I can't measure it precisely because a limit measurable in C, would take Swift until the heat death of the Universe.

So here's my question: Is Vincent aware of this, and his theme of "diabetes", "sugar", "saccharine", etc. pointing at how fat, bloated, slow, and deadly Swift is? He never lets on if this is a joke, he keeps tossing more syntax layers on top of Swift.

Not So Easy to Get a Program Right

"By June 1949 people had begun to realize that it was not so easy to get a program right as had at one time appeared. I well remember when this realization first came on me with full force. The EDSAC was on the top floor of the building and the tape-punching and editing equipment one floor below on a gallery that ran around the room in which the differential analyser was installed. I was trying to get working my first non-trivial program, which was one for the numerical integration of Airy's differential equation. It was on one of my journeys between the EDSAC room and the punching equipment that hesitating at the angles of stairs the realization came over me with full force that a good part of the remainder of my life was going to be spent in finding errors in my own programs."

-Maurice Wilkes, Memoirs