Tower of Babble

Programmers almost compulsively make new languages; within just a few years of there being computers, multiple competing languages appeared:

It proliferated from there into millions; probably half of all programmers with 10+ years of experience have written one or more.

I've written several, as scripting systems or toys. I really liked my Minimal script in Hephaestus 1.0, which was like BASIC+LISP, but implemented as it was in Java the performance was shitty and I had better options to replace it. My XML game schemas in GameScroll and Aiee! were half programmer humor, but very usable if you had a good XML editor. Multiple apps have shipped with my tiny lisp interpreter Aspic, despite the fruit company's ban on such things at the time. A Brainfuck/FORTH-like Stream, working-but-incomplete tbasic, and a couple PILOT variants (I think PILOT is hilariously on the border of "almost useful").

Almost every new language is invented as marketing bullshit based on a few Ur-languages:

  • C++: Swift
  • Java: Javascript (sorta), C#, Go
  • Awk: Perl, Python, PHP, Julia
  • C: Rust
  • Smalltalk: Objective-C
  • Prolog: Erlang, Elixir
  • ALGOL: C, Pascal, PL/1, Simula, Smalltalk, Java
  • LISP: Scheme, ML, Haskell, Clojure, Racket
  • BASIC: None, other than more dialects of BASIC.
  • FORTRAN: None in decades, but is the direct ancestor of ALGOL & BASIC.
  • COBOL: None in decades.

A few of these improve on their ancestors in some useful way, often performance is better, but most do nothing new; it's plausible that ALGOL 68 is a better language than any of its descendants, it just has mediocre compiler support these days.

Certainly I've made it clear I think Swift is a major regression, less capable, stable, fast, or even readable than C++, a feat I would've called impossible except as a practical joke a decade ago. When Marzipan comes out, I'll be able to rebuild all my 15 years of Objective-C code and it'll work on 2 platforms. The Swift 1.0 app I wrote and painfully ported to 2.0 is dead as a doornail, and current Swift apps will be uncompilable in 1-2 years; and be lost when Apple abandons Swift.

When I want to move my Scheme code to a new version or any other Scheme, it's pretty simple, I made only a handful of changes other than library importing from MIT Scheme to Chez to Chicken 4 to Chicken 5. When I tested it in Racket (which I won't be using) I had to make a handful of aliases. Probably even CLISP (which is the Swift of LISPs, except it fossilized in 1994) would be 20 or 30 aliases; their broken do iterator would be hard but the rest is just naming.

Javascript is a pernicious Herpes-virus-like infection of browsers and desktops, and nothing can ever kill it, so where it fits the problem, there's no reason not to use it. But there's a lot it doesn't do well.

I was leery of using FreePascal because it has a single implementation (technically Delphi still exists, but it's $X,000 per seat on Windows) and minimal libraries, and in fact when it broke on OS X Mojave, I was disappointed but I-told-you-so.

I'm not saying we should quit making new Brainfuck and LOLCODE things, I don't think it's possible for programmers to stop without radical brain surgery. But when you're evaluating a language for a real-world problem, try moving backwards until you find the oldest and most stable thing that works and will continue to work, not piling more crap into a rickety new framework.

The Biblical reference in the title amuses me, because we know now that it requires no malevolent genocidal war deity scared of us invading Heaven to magically confuse our languages and make us work at cross purposes; anyone who can write and think splinters their thought into a unique language and then argues about it.

Eldritch World

I wrote a text adventure:

It's like 15% complete, you can reach the first of four "recursions"/other worlds but then you're stuck. And today it only has a Mac console binary (you can run it from source on other platforms, with a little effort), I'll get on the cross-platform compiling and Terminal wrappers tomorrow, but this is a playable thing under a deadline!

How Fast is My Scheme

I ran a dumb Sieve of Eratosthenes benchmark on a few Schemes. I have previously found idiomatic Swift 3,151x slower than C so surely a GC'd LISP can't do that great, right?

Chicken (compiled) and Chez do great; I would expect Chez to do better on a less numeric example, this kind of array-humping is what Chicken's perfect for, since it just compiles to C. Chicken's interpreter is ridiculous, in dev it's fine for seeing if something works, but you'd never ship that. Racket's sad, and it gets worse if you try to do anything productive. Biwascheme was impossible to even test properly, but at 10% completion it wasn't going to do anything useful.

Also, I want to complain about every Scheme having a different way to call the interpreter as a script, and invoke main with command line arguments. NOT A ONE of these are consistent, and only Chicken does what's reasonable.

# C - 100% C
mdh@Aegura:~/Code/CodeC% time ./primes 1000000 >~/tmp/primes-c.txt
./primes 1000000 > ~/tmp/primes-c.txt  0.05s user 0.00s system 92% cpu 0.054 total

# Chicken interpreter - 1.4% C
mdh@Aegura:~/Code/CodeScheme% time src/primes.scm 1000000 >~/tmp/primes-chicken.txt
src/primes.scm 1000000 > ~/tmp/primes-chicken.txt  3.74s user 0.12s system 99% cpu 3.872 total

# Chicken compiled - 26.2% C
mdh@Aegura:~/Code/CodeScheme% ./build.zsh primes
Compiling primes.scm
./build.zsh:40: no matches found: *.import.scm
Built bin/primes
mdh@Aegura:~/Code/CodeScheme% time bin/primes 1000000 >~/tmp/primes-chicken.txt
bin/primes 1000000 > ~/tmp/primes-chicken.txt  0.19s user 0.01s system 98% cpu 0.206 total

# Chez interpreted - 23.8% C
mdh@Aegura:~/Code/CodeSchemeOld/chez% time ./ 1000000 >~/tmp/primes-chez.txt
./ 1000000 > ~/tmp/primes-chez.txt  0.20s user 0.03s system 98% cpu 0.227 total

# Chez compiled - 23.9% C
mdh@Aegura:~/Code/CodeSchemeOld/chez% chez-compile.zsh primes
compiling with output to
mdh@Aegura:~/Code/CodeSchemeOld/chez% time bin/primes 1000000 >~/tmp/primes-chez.txt
bin/primes 1000000 > ~/tmp/primes-chez.txt  0.20s user 0.02s system 97% cpu 0.226 total

# Racket interpreter - 9.6% C
mdh@Aegura:~/Code/CodeRacket% time ./primes.rkt 1000000 >~/tmp/primes-racket.txt
./primes.rkt 1000000 > ~/tmp/primes-racket.txt  0.46s user 0.09s system 99% cpu 0.560 total

# Racket compiled - 12% C
mdh@Aegura:~/Code/CodeRacket% raco exe primes.rkt
mdh@Aegura:~/Code/CodeRacket% time ./primes 1000000 >~/tmp/primes-racket.txt
./primes 1000000 > ~/tmp/primes-racket.txt  0.37s user 0.07s system 99% cpu 0.443 total

# Biwascheme - 0.03% C
# Used stopwatch, took 13.89s for 100,000, couldn't get a result for 1,000,000

Scheme Record to RecordType

So, in "classic" Scheme (up to R5RS), there were no structs/records/classes. You could fake them with a Vector and writing all the accessor methods by hand, but it sucked.

SRFI-9 added a very minimalist record type with a lot of repetition, and no inheritance, though at least SRFI-17 fixed setters.

R6RS (as implemented in Chez Scheme) added a much more powerful system with inheritance and constructor methods, but half the Scheme community hates nice things and voted it down. There's a half-assed reimplementation of R6RS in SRFI-99 and sequels, but it still doesn't have read/write mechanisms. R7RS still only ships with SRFI-9 built in. Unbelievable mess.

Chicken has a convenient define-record macro, and read/write methods, but by default uses SRFI-9, and hides SRFI-99 in an egg; so chicken-install -s srfi-99 and then (import srfi-99) everywhere, and then write a ton of boilerplate for every type. So I just automated it with Python (doing string parsing in Scheme is more annoying):

Documentation is in the module help (or just read the source, Luke). I use it by writing the Chicken macro (define-record Point x y), then at shell:

% pbpaste||pbcopy

And paste back:

;; (define-record Point x y)
(define-record-type Point  
(define-reader-ctor 'Point make-Point)
(define-record-printer (Point p out)
    (format out "#,(Point ~S ~S)"
        (Point-x p) (Point-y p) 
; module exports:
; make-Point Point? Point-x Point-y

Note that none of this really gets me an "object-oriented" system, but records are sufficient for most programs, and inheritance works with define-record-type. There are OOP systems but I don't especially like any of them so I'm passing on them for now.

Functional Thinking Books

Advent of Code 2018

I don't know how much I'll do this year, but I'll do it in Chicken Scheme as a way of improving the text-processing and math functions in my library, and publish it. Source is now on gitlab, above.

The competitive part is still bullshit, the single starting time of midnight EST is utterly useless to most people who could participate; even in Pacific time, that's late at night. On my quiet night schedule, that's way too early to wake up; in Europe, that's 04:00 or so, long before coffee could percolate. Mid-workday for Korea & Japan. So, only for Finns and Russians?

Inline Documentation, or Lack Thereof in Scheme

I'm a big fan of inline documentation and "light" versions of Literate Programming, because docs that are more than one screen away from code are always wrong. That this is ever a revelation to anyone suggests to me that they've never written code or read API docs.

When I write Python, I write:

>>> import math
>>> def foo(x):
    "Square root of `x`"
    return math.sqrt(x)

>>> foo(5)
>>> help(foo)
Help on function foo in module __main__:
    Square root of `x`

Similarly in Java with Javadoc, I write:

/** Square root of {@code x} */
double foo(double x) {
    return Math.sqrt(x);

Javadoc isn't usable on live code or in a REPL (Java doesn't really have one), but you get nice HTML docs out of it. Javascript & Node don't have an official tool, but most code is marked up with Javadoc.

Common LISP, archaic pain in the ass though it is, has:

(defun foo (x)
    "Square root of `x`"
    (sqrt x))
> (documentation 'foo 'function)
"Square root of `x`"

Sadly and typically, the Scheme situation is much less organized.

There's a Chicken 4 egg hahn which is ugly, @() special forms and all the nested structures instead of just a string, but it's workable. Otherwise, everything seems to be external docs.

Racket has Scribble with a teeny-tiny side-note that you can put your docs in code, but no examples. There is a literate programming tool as well, but that's not quite what I'm after.

Chez Scheme has no solution, which is a little surprising given the "batteries included" philosophy.

Well, maybe I can get away with doing CLISP-type docstrings and worry about making a tool later? Does this extra junk hurt performance?

(import (chicken time))
(define (sqrt-without-docs x) (sqrt x))
(define (sqrt-with-docs x) "docs" (sqrt x))
(display "without docs\n")
(time {do [(i 0 (add1 i))] [(>= i 1000000)] (sqrt-without-docs i)})
(display "with docs\n")
(time {do [(i 0 (add1 i))] [(>= i 1000000)] (sqrt-with-docs i)})

In the interpreter, there's a 10-50% speed penalty (csi -s or in the REPL) for having that extra string created & GC'd, but compiled (csc), there's no noticeable difference. So I guess that's my solution for now.

Don't Increment Your Chickens Before They're Hatched

It hasn't shown up in MacPorts yet, so I pulled down the tarball and did:

% make PLATFORM=macosx PREFIX=/usr/local/chicken5
% sudo make PLATFORM=macosx PREFIX=/usr/local/chicken5 install

Then added to .profile, before the MacPorts parts of my paths:

export CHICKEN5_HOME="/usr/local/chicken5"
export MANPATH="$MANPATH:$CHICKEN5_HOME/share/man"
export PATH="$PATH:$CHICKEN5_HOME/bin"

When I want to switch back to Chicken 4, or when MacPorts gets Chicken 5, I can just comment those lines out and start a new shell.

I run csi, and it errors out because readline isn't installed. And doesn't exist as an egg anymore. Instead, I did

% chicken-install -sudo breadline

And changed my ~/.csirc based on example

    (chicken-4 (begin
        (import miscmacros)
        (use readline)
        (install-history-file  "/.csi_history")
        (current-input-port (make-readline-port))
        (printf "Chicken 4 READY~%")
    (chicken-5 (begin
        (import (chicken format))
        (import (chicken process-context))
        (import miscmacros)
        (import (prefix breadline "rl-"))
        (import (prefix breadline-scheme-completion "rl-"))
        (rl-history-file (format "~A/.csi_history" (get-environment-variable "HOME")))
        (rl-completer-word-break-characters-set! "\t\n\"\'`;|()[]{}")
        (rl-completer-set! rl-scheme-completer)
        (rl-basic-quote-characters-set! "\"|")
        (rl-variable-bind! "blink-matching-paren" "on")
        (rl-paren-blink-timeout-set! 200000)
        (current-input-port (rl-make-readline-port))
        (printf "Chicken 5 READY~%")

UGH, semi-random API changes, but it does a little more than raw arrow-key editing now.

sdl2 isn't ported yet, so I can't really progress with that, but I can do other things, and most of my code only needs a cond-expand changing use to import and renaming a bunch of eggs.

Chicken Soup

(a bunch of stuff in a pot)


The Chicken csi REPL is appalling after using some nice REPLs, it doesn't even have history by default. I couldn't reliably get non-GNU readline-likes to work, so:

% chicken-install -sudo readline
% cat >~/.csirc
(use readline)
(current-input-port (make-readline-port))
(install-history-file  "/.csi_history")

So at least now it has the usual up/down/emacs-like keys.

Long fucking ways from the old Symbolics LISP Machines. Why don't we have environments like that anymore? Why is everyone content to just use fucking emacs (I've never been emacsulated) or other editor, and a boring REPL? DrRacket is just a REPL that destroys its memory every time you edit code, and it's the most graphically advanced LISP-type environment. And this is why I still just use Atom with Symbols Tree View (even though it thinks variable definitions are functions), and copy-paste into iTerm if I want to test something.

Value Unpacking

Not having nice R6RS macros for this, and unwilling to fight with classic macros, I've been using values to unpack lists into variables, and because I can never remember the exact syntax, I made this cheat-sheet:

(define a '(1 2 3))
(define b '(4 5 6))
;; then one of these:
(define-values (x y z) (apply values a)) (printf "~s,~s,~s\n" x y z)
(set!-values (x y z) (apply values b)) (printf "~s,~s,~s\n" x y z)
(let-values [[(x y z) (apply values a)] [(q r s) (apply values b)]] (printf "qrs:~s,~s,~s xyz:~s,~s,~s\n" q r s x y z))

Probably not efficient, but better than car, cadr, caddr, etc. Maybe I should move all my list-structures into vectors, but then I'd still have to convert them to lists half the time. Here's where Python is the programmer's best friend, even if it is 10,000x slower:

a = (1, 2, 3)
x, y, z = a

Why Did LISP Fail?

How did a more advanced language with better tools just die off commercially, and now if you want to work in it, you have to cobble together a bunch of half-broken shit?

I think there's 3 reasons:

  1. It's hard and ugly. It may be logically compelling, but when you see a page of parens your brain panics and looks for a place to hide.
  2. Companies value the fake productivity of thousands of lines of C, Java, or Swift (aka C++2020) code more than having safety, security, and correct reasoning. Who cares if millions of people will suffer and possibly die from your code, as long as you can ship TODAY?
  3. A lot of LISP "hackers" are insufferable douchebags, both old beardy fuckers who've been doing it for 50 years and mewling children who learned it last week. Every new variant makes the older contingent more angry at even seeing a mention of it, and the sneering fetuses think whatever variant they learned is Divine Wisdom, rather than just an engineering tool that may need to be improved.

Building a Binary with Chicken Scheme

So that was fucking fun. Seems Chicken's docs aren't correct on how to build with modules, because those were added after the dark-ages R5RS it was modelled on.

There is an "egg" system which is used for building the libraries, but it's difficult to use for making binaries in your own destination dir, and fills your work dir with spam temp files. Unfortunately basically useless for work.

Finally got this working:


(declare (unit somelib))
(module somelib

(import scheme)

(define (hello) (display "Hello!\n"))


(import scheme
    (chicken load)

    (compiling (declare (uses somelib)))
    (else (load-relative "somelib.scm")))
(import somelib)


Don't you just love that muffin-man-muffin-man repetitious bullshit? declare is for the compiler, load is for the interpreter, then import for both. You import builtins like chicken, but use libraries (aforementioned sdl) without an import. Bizarre and contradictory.

build.zsh: [script updated 2019-06-08]


# EXE is binary filename
# MAIN is main script
# LIBS is space-delimited, must not include main script

function usage {
    echo "Usage: build.zsh PROJECT || MAIN.scm [LIBS] || ls || -?"
    exit 1

if [[ $# -eq 0 || "$1" == "-?" || "$1" == "--help" ]]; then

case $1 in
        echo "Known projects:"
        # Add known projects here
        echo "eldritch test"
        exit 0
        LIBS="marklib.scm marklib-geometry.scm marklib-ansi.scm"
        LIBS="marklib.scm marklib-geometry.scm"
        # command-line project: MAIN=$1, LIBS=$2...
        EXE=`basename $1 .scm`

mkdir -p bin

cd src
for SF in ${=LIBS}; do
    SNAME=`basename $SF .scm`
    echo "Compiling $SF"
    csc -c -j $SNAME $SF -o $SNAME.o || exit 1
echo "Compiling $MAIN"
csc -c $MAIN -o `basename $MAIN .scm`.o || exit 1
echo "Linking..."
csc *.o -o ../bin/$EXE || exit 1
rm -f *.o
rm -f *.import.scm
cd ..

echo "Built bin/$EXE"
% ./build.zsh
Built bin/something
% ./bin/something

Hello, working native binary! There's a bunch more stuff about making a deployable app, but I'll take this for now. My actual program can show a blank green graphics window!