Category: Scheme

Eldritch World

I wrote a text adventure:

It's like 15% complete, you can reach the first of four "recursions"/other worlds but then you're stuck. And today it only has a Mac console binary (you can run it from source on other platforms, with a little effort), I'll get on the cross-platform compiling and Terminal wrappers tomorrow, but this is a playable thing under a deadline!

How Fast is My Scheme

I ran a dumb Sieve of Eratosthenes benchmark on a few Schemes. I have previously found idiomatic Swift 3,151x slower than C so surely a GC'd LISP can't do that great, right?

Chicken (compiled) and Chez do great; I would expect Chez to do better on a less numeric example, this kind of array-humping is what Chicken's perfect for, since it just compiles to C. Chicken's interpreter is ridiculous, in dev it's fine for seeing if something works, but you'd never ship that. Racket's sad, and it gets worse if you try to do anything productive. Biwascheme was impossible to even test properly, but at 10% completion it wasn't going to do anything useful.

Also, I want to complain about every Scheme having a different way to call the interpreter as a script, and invoke main with command line arguments. NOT A ONE of these are consistent, and only Chicken does what's reasonable.

# C - 100% C
mdh@Aegura:~/Code/CodeC% time ./primes 1000000 >~/tmp/primes-c.txt
./primes 1000000 > ~/tmp/primes-c.txt  0.05s user 0.00s system 92% cpu 0.054 total

# Chicken interpreter - 1.4% C
mdh@Aegura:~/Code/CodeScheme% time src/primes.scm 1000000 >~/tmp/primes-chicken.txt
src/primes.scm 1000000 > ~/tmp/primes-chicken.txt  3.74s user 0.12s system 99% cpu 3.872 total

# Chicken compiled - 26.2% C
mdh@Aegura:~/Code/CodeScheme% ./build.zsh primes
Compiling primes.scm
Linking...
./build.zsh:40: no matches found: *.import.scm
Built bin/primes
mdh@Aegura:~/Code/CodeScheme% time bin/primes 1000000 >~/tmp/primes-chicken.txt
bin/primes 1000000 > ~/tmp/primes-chicken.txt  0.19s user 0.01s system 98% cpu 0.206 total

# Chez interpreted - 23.8% C
mdh@Aegura:~/Code/CodeSchemeOld/chez% time ./primes.ss 1000000 >~/tmp/primes-chez.txt
./primes.ss 1000000 > ~/tmp/primes-chez.txt  0.20s user 0.03s system 98% cpu 0.227 total

# Chez compiled - 23.9% C
mdh@Aegura:~/Code/CodeSchemeOld/chez% chez-compile.zsh primes
compiling primes.ss with output to primes.so
()
()
mdh@Aegura:~/Code/CodeSchemeOld/chez% time bin/primes 1000000 >~/tmp/primes-chez.txt
bin/primes 1000000 > ~/tmp/primes-chez.txt  0.20s user 0.02s system 97% cpu 0.226 total

# Racket interpreter - 9.6% C
mdh@Aegura:~/Code/CodeRacket% time ./primes.rkt 1000000 >~/tmp/primes-racket.txt
./primes.rkt 1000000 > ~/tmp/primes-racket.txt  0.46s user 0.09s system 99% cpu 0.560 total

# Racket compiled - 12% C
mdh@Aegura:~/Code/CodeRacket% raco exe primes.rkt
mdh@Aegura:~/Code/CodeRacket% time ./primes 1000000 >~/tmp/primes-racket.txt
./primes 1000000 > ~/tmp/primes-racket.txt  0.37s user 0.07s system 99% cpu 0.443 total

# Biwascheme - 0.03% C
# Used stopwatch, took 13.89s for 100,000, couldn't get a result for 1,000,000

Scheme Record to RecordType

So, in "classic" Scheme (up to R5RS), there were no structs/records/classes. You could fake them with a Vector and writing all the accessor methods by hand, but it sucked.

SRFI-9 added a very minimalist record type with a lot of repetition, and no inheritance, though at least SRFI-17 fixed setters.

R6RS (as implemented in Chez Scheme) added a much more powerful system with inheritance and constructor methods, but half the Scheme community hates nice things and voted it down. There's a half-assed reimplementation of R6RS in SRFI-99 and sequels, but it still doesn't have read/write mechanisms. R7RS still only ships with SRFI-9 built in. Unbelievable mess.

Chicken has a convenient define-record macro, and read/write methods, but by default uses SRFI-9, and hides SRFI-99 in an egg; so chicken-install -s srfi-99 and then (import srfi-99) everywhere, and then write a ton of boilerplate for every type. So I just automated it with Python (doing string parsing in Scheme is more annoying):

Documentation is in the module help (or just read the source, Luke). I use it by writing the Chicken macro (define-record Point x y), then at shell:

% pbpaste|schemeRecordToRecordType.py|pbcopy

And paste back:

;; (define-record Point x y)
(define-record-type Point #t #t
    (x)
    (y)
)
(define-reader-ctor 'Point make-Point)
(define-record-printer (Point p out)
    (format out "#,(Point ~S ~S)"
        (Point-x p) (Point-y p) 
))
; module exports:
; make-Point Point? Point-x Point-y

Note that none of this really gets me an "object-oriented" system, but records are sufficient for most programs, and inheritance works with define-record-type. There are OOP systems but I don't especially like any of them so I'm passing on them for now.

Functional Thinking Books

Advent of Code 2018

I don't know how much I'll do this year, but I'll do it in Chicken Scheme as a way of improving the text-processing and math functions in my library, and publish it. Source is now on gitlab, above.

The competitive part is still bullshit, the single starting time of midnight EST is utterly useless to most people who could participate; even in Pacific time, that's late at night. On my quiet night schedule, that's way too early to wake up; in Europe, that's 04:00 or so, long before coffee could percolate. Mid-workday for Korea & Japan. So, only for Finns and Russians?

Inline Documentation, or Lack Thereof in Scheme

I'm a big fan of inline documentation and "light" versions of Literate Programming, because docs that are more than one screen away from code are always wrong. That this is ever a revelation to anyone suggests to me that they've never written code or read API docs.

When I write Python, I write:

>>> import math
>>> def foo(x):
    "Square root of `x`"
    return math.sqrt(x)

>>> foo(5)
2.23606797749979
>>> help(foo)
Help on function foo in module __main__:
foo(x)
    Square root of `x`

Similarly in Java with Javadoc, I write:

/** Square root of {@code x} */
double foo(double x) {
    return Math.sqrt(x);
}

Javadoc isn't usable on live code or in a REPL (Java doesn't really have one), but you get nice HTML docs out of it. Javascript & Node don't have an official tool, but most code is marked up with Javadoc.

Common LISP, archaic pain in the ass though it is, has:

(defun foo (x)
    "Square root of `x`"
    (sqrt x))
> (documentation 'foo 'function)
"Square root of `x`"

Sadly and typically, the Scheme situation is much less organized.

There's a Chicken 4 egg hahn which is ugly, @() special forms and all the nested structures instead of just a string, but it's workable. Otherwise, everything seems to be external docs.

Racket has Scribble with a teeny-tiny side-note that you can put your docs in code, but no examples. There is a literate programming tool as well, but that's not quite what I'm after.

Chez Scheme has no solution, which is a little surprising given the "batteries included" philosophy.

Well, maybe I can get away with doing CLISP-type docstrings and worry about making a tool later? Does this extra junk hurt performance?

(import (chicken time))
(define (sqrt-without-docs x) (sqrt x))
(define (sqrt-with-docs x) "docs" (sqrt x))
(display "without docs\n")
(time {do [(i 0 (add1 i))] [(>= i 1000000)] (sqrt-without-docs i)})
(display "with docs\n")
(time {do [(i 0 (add1 i))] [(>= i 1000000)] (sqrt-with-docs i)})

In the interpreter, there's a 10-50% speed penalty (csi -s or in the REPL) for having that extra string created & GC'd, but compiled (csc), there's no noticeable difference. So I guess that's my solution for now.

Don't Increment Your Chickens Before They're Hatched

It hasn't shown up in MacPorts yet, so I pulled down the tarball and did:

% make PLATFORM=macosx PREFIX=/usr/local/chicken5
…
% sudo make PLATFORM=macosx PREFIX=/usr/local/chicken5 install

Then added to .profile, before the MacPorts parts of my paths:

export CHICKEN5_HOME="/usr/local/chicken5"
export MANPATH="$MANPATH:$CHICKEN5_HOME/share/man"
export PATH="$PATH:$CHICKEN5_HOME/bin"

When I want to switch back to Chicken 4, or when MacPorts gets Chicken 5, I can just comment those lines out and start a new shell.

I run csi, and it errors out because readline isn't installed. And doesn't exist as an egg anymore. Instead, I did

% chicken-install -sudo breadline

And changed my ~/.csirc based on example

(cond-expand
    (chicken-4 (begin
        (import miscmacros)
        (use readline)
        (install-history-file #f "/.csi_history")
        (current-input-port (make-readline-port))
        (printf "Chicken 4 READY~%")
    ))
    (chicken-5 (begin
        (import (chicken format))
        (import (chicken process-context))
        (import miscmacros)
        (import (prefix breadline "rl-"))
        (import (prefix breadline-scheme-completion "rl-"))
        (rl-history-file (format "~A/.csi_history" (get-environment-variable "HOME")))
        (rl-completer-word-break-characters-set! "\t\n\"\'`;|()[]{}")
        (rl-completer-set! rl-scheme-completer)
        (rl-basic-quote-characters-set! "\"|")
        (rl-variable-bind! "blink-matching-paren" "on")
        (rl-paren-blink-timeout-set! 200000)
        (current-input-port (rl-make-readline-port))
        (printf "Chicken 5 READY~%")
    ))
)

UGH, semi-random API changes, but it does a little more than raw arrow-key editing now.

sdl2 isn't ported yet, so I can't really progress with that, but I can do other things, and most of my code only needs a cond-expand changing use to import and renaming a bunch of eggs.

Chicken Soup

(a bunch of stuff in a pot)

REPL

The Chicken csi REPL is appalling after using some nice REPLs, it doesn't even have history by default. I couldn't reliably get non-GNU readline-likes to work, so:

% chicken-install -sudo readline
% cat >~/.csirc
(use readline)
(current-input-port (make-readline-port))
(install-history-file #f "/.csi_history")
^D

So at least now it has the usual up/down/emacs-like keys.

Long fucking ways from the old Symbolics LISP Machines. Why don't we have environments like that anymore? Why is everyone content to just use fucking emacs (I've never been emacsulated) or other editor, and a boring REPL? DrRacket is just a REPL that destroys its memory every time you edit code, and it's the most graphically advanced LISP-type environment. And this is why I still just use Atom with Symbols Tree View (even though it thinks variable definitions are functions), and copy-paste into iTerm if I want to test something.

Value Unpacking

Not having nice R6RS macros for this, and unwilling to fight with classic macros, I've been using values to unpack lists into variables, and because I can never remember the exact syntax, I made this cheat-sheet:

(define a '(1 2 3))
(define b '(4 5 6))
;; then one of these:
(define-values (x y z) (apply values a)) (printf "~s,~s,~s\n" x y z)
(set!-values (x y z) (apply values b)) (printf "~s,~s,~s\n" x y z)
(let-values [[(x y z) (apply values a)] [(q r s) (apply values b)]] (printf "qrs:~s,~s,~s xyz:~s,~s,~s\n" q r s x y z))

Probably not efficient, but better than car, cadr, caddr, etc. Maybe I should move all my list-structures into vectors, but then I'd still have to convert them to lists half the time. Here's where Python is the programmer's best friend, even if it is 10,000x slower:

a = (1, 2, 3)
x, y, z = a
print(f"{x},{y},{z}")

Why Did LISP Fail?

How did a more advanced language with better tools just die off commercially, and now if you want to work in it, you have to cobble together a bunch of half-broken shit?

I think there's 3 reasons:

  1. It's hard and ugly. It may be logically compelling, but when you see a page of parens your brain panics and looks for a place to hide.
  2. Companies value the fake productivity of thousands of lines of C, Java, or Swift (aka C++2020) code more than having safety, security, and correct reasoning. Who cares if millions of people will suffer and possibly die from your code, as long as you can ship TODAY?
  3. A lot of LISP "hackers" are insufferable douchebags, both old beardy fuckers who've been doing it for 50 years and mewling children who learned it last week. Every new variant makes the older contingent more angry at even seeing a mention of it, and the sneering fetuses think whatever variant they learned is Divine Wisdom, rather than just an engineering tool that may need to be improved.

Building a Binary with Chicken Scheme

So that was fucking fun. Seems Chicken's docs aren't correct on how to build with modules, because those were added after the dark-ages R5RS it was modelled on.

There is an "egg" system which is used for building the libraries, but it's difficult to use for making binaries in your own destination dir, and fills your work dir with spam temp files. Unfortunately basically useless for work.

Finally got this working:

src/somelib.scm:

(declare (unit somelib))
(module somelib
    (hello)

(import scheme)

(define (hello) (display "Hello!\n"))
)

src/somemain.scm:

(import scheme
    (chicken load)
)

(cond-expand
    (compiling (declare (uses somelib)))
    (else (load-relative "somelib.scm")))
(import somelib)

(hello)

Don't you just love that muffin-man-muffin-man repetitious bullshit? declare is for the compiler, load is for the interpreter, then import for both. You import builtins like chicken, but use libraries (aforementioned sdl) without an import. Bizarre and contradictory.

build.zsh: [script updated 2019-06-08]

#!/bin/zsh

# EXE is binary filename
# MAIN is main script
# LIBS is space-delimited, must not include main script

function usage {
    echo "Usage: build.zsh PROJECT || MAIN.scm [LIBS] || ls || -?"
    exit 1
}

if [[ $# -eq 0 || "$1" == "-?" || "$1" == "--help" ]]; then
    usage
fi

case $1 in
    ls)
        echo "Known projects:"
        # Add known projects here
        echo "eldritch test"
        exit 0
    ;;
    eldritch)
        EXE=eldritch
        MAIN=eldritch.scm
        LIBS="marklib.scm marklib-geometry.scm marklib-ansi.scm"
    ;;
    test)
        EXE=marklib-test
        MAIN=marklib-test.scm
        LIBS="marklib.scm marklib-geometry.scm"
    ;;
    *)
        # command-line project: MAIN=$1, LIBS=$2...
        EXE=`basename $1 .scm`
        MAIN=$1
        shift
        LIBS="$*"
    ;;
esac

mkdir -p bin

cd src
for SF in ${=LIBS}; do
    SNAME=`basename $SF .scm`
    echo "Compiling $SF"
    csc -c -j $SNAME $SF -o $SNAME.o || exit 1
done
echo "Compiling $MAIN"
csc -c $MAIN -o `basename $MAIN .scm`.o || exit 1
echo "Linking..."
csc *.o -o ../bin/$EXE || exit 1
rm -f *.o
rm -f *.import.scm
cd ..

echo "Built bin/$EXE"

% ./build.zsh
Built bin/something
% ./bin/something
Hello!

Hello, working native binary! There's a bunch more stuff about making a deployable app, but I'll take this for now. My actual program can show a blank green graphics window!

More Fun and Swearing with Scheme

I have ideas for some little games, which aren't suitable as giant Electron-powered applications. My preference in language for this would be Objective-C, Scheme, Pascal, C if I absolutely had to. Obj-C's lack of portability rules it out for now, but if GNUstep gets their crap together it may go back in the running. Scheme looked promising, I've liked using Chez Scheme for some data processing.

So after 2 days of experimentation and abuse, and none of my tools working right in the process, I was unable to get thunderchez/sdl2 to link. I can explicitly load the dylib, and it doesn't find SDL_Init. I wrote a little C program to prove that SDL is reachable, and it is. Chez just won't load it.

Frustrated, I grabbed Chicken Scheme again, did a chicken-install -sudo sdl2, ran the sample code(!), and bam, it just worked. 15 minutes of effort at most, and I'm up and running.

Down side, Chicken compiles slow, and the interpreter is PAINFULLY slow; Chez "interprets" by compiling fast with low optimizations. And Chicken defaults to R5RS which is… 1990 called and wants me to watch MIT SICP lectures on VHS. It has some R7RS support, but I prefer the guarantees of portability, safety, and specified behavior in R6RS. Have to go looking thru SRFI docs to find any feature, it's not batteries-included. Oh well, I'll probably live just fine without ideological purity.