Script the Scheme REPL with Expect

Routinely I want to open the Chez Scheme REPL in my code dir and load my standard libraries; I've been copy-pasting from a Stickies to get my setup each time, because you can't easily set a common prelude from command line. Finally solved that.

In ~/bin/scheme-repl, I put:

#!/usr/bin/env expect -f
log_user 0
cd "$env(HOME)/Code/CodeChez"
spawn scheme
expect "> "
send -- "(import (chezscheme) (marklib) (marklib-os)"
send -- "  (only (srfi s1 lists) delete drop-right last)"
send -- "  (only (srfi s13 strings) string-delete string-index string-index-right string-join string-tokenize) )\n"
log_user 1
interact

(make sure to change the path to wherever you keep your Scheme scripts, and whatever imports you like)

Added alias s=scheme-repl to my .zshrc

Now I can just hit one letter for shortcut:

% s
(import (chezscheme) (marklib) (marklib-os)  (only (srfi s1 lists) delete drop
-right last)  (only (srfi s13 strings) string-delete string-index string-index-r
ight string-join string-tokenize) )
> (os-path 'pwd)
"/Users/mdh/Code/CodeChez"
>

You can sort of do the same thing with a Chez boot file, but I wasn't able to get it to load libraries from thunderchez, even with --libdirs flag, so screw it.

I'd forgotten everything I ever knew about expect, and the only resources online are exact copies of the same "log into ssh with expect!" (which you should never do! Set up SSH keys for Cthulhu's sake!) tutorial over and over again, so had to read the man page to make even this trivial thing work.

Death to Freenode, Long Live the New Flesh^W^W Libera Chat

So, on Tuesday, Freenode IRC blew up: FAQ
This is where a lot of software development chat happens. So for a rich, Trumpian, bitcoiner, Korean royalty, asshole to take it over by treachery is just unacceptable.

And so the staff and users have moved over to Libera Chat — as of today, it has more online users than Freenode.

Most of the channels I'm in have moved quickly, so I shut off Freenode this morning. If you need me, I'm in and others, as mdhughes.

A few usage & system bot notes:

  • Staff info channel:
  • NickServ: /msg nickserv help
  • ChanServ: /msg chanserv help
  • Channel List Service: /msg alis help list
  • Cloak: join -cloak and /topic

Basic Games in Scheme

The first project I write in any new language is usually a guess-the-number game, a die roller, or an RPN calculator, then I start collecting those and other toys and utilities into a single "main menu" program, and use that to drive me to develop my libraries, play with different algorithms. Occasionally it's useful, mostly it's just a pile of stuff.

The Scheme version has a couple useful things. I was mostly thinking about old BASIC games, so it's "BasicSS" (SS being the Chez Scheme file extension, not anything more nautical or sinister).

I wrote a fairly malevolent wordsearch generator in the process of testing some file parsing, so here's one for 20 programming languages. I can tell you that B, C, C#, and D are not in my list. I'm doubtful that anyone can find all of them, or even half.

Hangman depends on /usr/share/dict/words, 235,886 lines on my system, which is very unfair:

 
 #     |
 #    ---
 # \ (o o) /
 #  \ --- /
 #   \ X /
 #    \X/
 #     X
 #     X
 #    / \
 #   /   \
 #
Word: TE---EN--
Guesses: E, T, A, O, I, N, B, R, S
YOU LOSE! You have been hung.
The word was TEMULENCY.

Seabattle ("you sunk my…") sucks, it just picks targets at random; teaching it some AI would help.

Hurkle, like all the early-'70s "find a monster on a grid" games, is awful, but the map display makes it a little easier to track your shots. "The Hurkle is a Happy Beast" by Theodore Sturgeon is one of his 10% good stories, but it provides only a little context.

Some of this I can release source for, some I probably shouldn't, so it's just a binary for now.

Reinventing the Wheel

Sure, there's existing code. Somebody Else's Code. It works fine, maybe not as fast as you'd like, or the interface isn't quite right. That's how it often is with me and SRFI-13. Olin Shivers is a skilled Schemer, back when that wasn't cool (OK, it's still not cool), but some of his APIs enshrined in early SRFIs drive me a little nuts, and the implementation is slow because it's so generalized.

So after a few false starts and failed tests, I now have these pretty things: (updated 2020-11-10, enforced hstart, hend boundaries)

;; Returns index of `needle` in `haystack`, or  if not found.
;; `cmp`: Comparator. Default `char=?`, `char-ci=?` is the most useful alternate comparator.
;; `hstart`: Starting index, default 0.
;; `hend`: Ending index, default (- haystack-length needle-length)
(define string-find (case-lambda
    [(haystack needle)  (string-find haystack needle char=? 0 )]
    [(haystack needle cmp)  (string-find haystack needle cmp 0 )]
    [(haystack needle cmp hstart)  (string-find haystack needle cmp hstart ) ]
    [(haystack needle cmp hstart hend)
        (let* [ (hlen (string-length haystack))  (nlen (string-length needle)) ]
            (set! hstart (max 0 (min hstart (sub1 hlen))))
            (unless hend (set! hend (fx- hlen nlen)))
            (set! hend (max 0 (min hend hlen)) )
            (if (or (fxzero? hlen) (fxzero? nlen))
                
                (let loop [ (hi hstart)  (ni 0) ]
                    ;; assume (< ni nlen)
                    ;(errprintln "hi=" hi ", ni=" ni ", hsub=" (substr haystack hi hlen) ", bsub=" (substr needle ni nlen))
                    (cond
                        [(cmp (string-ref haystack (fx+ hi ni)) (string-ref needle ni))  (set! ni (fx+ ni 1))
                            ;; end of needle?
                            (if (fx>=? ni nlen)  hi  (loop hi ni) )
                        ]
                        [else  (set! hi (fx+ hi 1))
                            ;; end of haystack?
                            (if (fx>? hi hend)    (loop hi 0) )
                        ]
        ))))
    ]
))

;; Test whether 'haystack' starts with 'needle'.
(define (string-has-prefix? haystack needle)
    (let [ (i (string-find haystack needle char=? 0 0)) ]
        (and i (fxzero? i))
))

;; Test whether 'haystack' ends with 'needle'.
(define (string-has-suffix? haystack needle)
    (let* [ (hlen (string-length haystack))  (nlen (string-length needle))
            (i (string-find haystack needle char=? (fx- hlen nlen)))
        ]
        (and i (fx=? i (fx- hlen nlen)))
))

Written for Chez Scheme, caveat implementor. BSD license, do what thou wilt. If you find a bug, send me a failing test case.

I don't normally bother with fx (fixnum) operations, but in tight loops it makes a difference over generic numeric tower +, etc.

Racket CS 7.8

Big plus: ARM64 support!

Big minus: Neither DRRacket nor any built .app launches on the Mac, you have to call them from command line. Gross incompetence & zero testing.

My basic gridtest.rkt demo still takes 100ms to render a frame, needs to be <16ms. But that's interpreting from the shell, and I may look and see if they have a more efficient graphics API.

The bigger problem is that compiling doesn't work with packages, or at least I don't have the right magic invocations:

% cat build.zsh
#!/bin/zsh

rm -rf build
mkdir build
# Windows: --ico foo.ico
# Mac: --icns foo.icns
time raco exe --cs --gui ++lib memoize ++lib rsound ++lib portaudio -o build/gridtest gridtest.rkt

% ./build.zsh && ./build/gridtest.app/Contents/MacOS/gridtest
raco exe --cs --gui ++lib memoize ++lib rsound ++lib portaudio -o    39.14s user 2.40s system 87% cpu 47.390 total
ffi-lib: could not load foreign library
  path: libportaudio.2.dylib
  system error: dlopen(libportaudio.2.dylib, 6): image not found
  context...:
   raise-dll-error
   .../ffi/unsafe.rkt:131:0: get-ffi-lib
   call-in-empty-metacontinuation-frame
   proc
   call-in-empty-metacontinuation-frame
   body of '#%embedded:portaudio/portaudio:
   temp35_0
   run-module-instance!
   [repeats 6 more times]
   perform-require!
   call-in-empty-metacontinuation-frame
   eval-one-top
   eval-compiled-parts
   embedded-load
   proc
   call-in-empty-metacontinuation-frame

Well, that's fucked. 40 seconds and a fuck you because it doesn't bring in the dylib it needs.

Obviously I could just say "install Racket, run this script", but that's lame and distributes source, which in a real game I don't want to do.

LISP Machines

I really wish we had a front-end to software even half as useful as this in the 21st C, but technology has regressed massively since the '80s and '90s. Some of the old IDEs, before they became bloated "enterprise" software (because giant mega-corporations paid for them, not individual programmers, so the IDE makers serve their paymasters), started to slouch towards this kind of usefulness but fast and small. CodeWarrior, Project Builder/Interface Builder, Borland's Turbo Whatever.

emacs isn't the answer, it's an abandonment of the question; "modern" emacs turned away from the LM zmacs model, it's now just a terrible LISP interpreter with a bad text-only-editor front end; you can make tools in it, but nobody sane will want to use them. I'm perfectly comfortable with the emacs editing keys (well, obviously, Macs use emacs keys for all text areas), but the machinery in it is just broken.

I get excited about Chez Scheme having a nice REPL, with history and can edit multi-line blocks in the REPL, unlike the crappy readline almost every other Scheme uses. But it doesn't do hypertext, it doesn't do graphics, it barely has tab completion (procedure names only), it doesn't have any inline documentation & source inspector; all of those were in the LISP Machine.

When I'm working, I have my text editor (BBEdit or Atom mostly), a terminal with the REPL running and I copy-paste to it, 3-4 PDFs open (R6RS, R6RS-lib, CSUG, sometimes TSPL), and a web browser pointed at the SRFIs. If the Internet connection went down, I'd have to search the SRFI sources to figure out what's in there. I really need a better tool for this.

DrRacket can do some graphics inline, and the tab completion shows documentation hints in the top right corner, but as I note every time, it doesn't really have a REPL because it destroys the interactive environment every time you edit code; utterly useless for code exploration. And in practice, Racket is really horrifyingly slow; it does fine in focused benchmarks but real-world use it just falls over drooling.

The graphics part's sort of irrelevant, and sort of not; the way the LISP Machine worked was getting a "presentation" form for an object, which would render as text or drawings or images, and interacting with it sent messages back to that object. That's probably out of scope for anything except a complete new OS and terminal. A simplistic number-tagged hypertext would be good enough and orders of magnitude easier.

So. I'm not sure what to do here, I don't want to just complain about tools and not do anything about them. I could try to extract the docs to make a hypertext doc system; a lot of text processing on TeX source sounds painful, and a one-off job, I want a more universal solution. It may be possible to hook into Chez's completion to call a help system. Or it could be a standalone program that you feed several doc sources into, and it lets you search against them. Dash does that, but it's Mac and C/Objective-C primarily, and does poorly at other docsets.

Scheme-Test-Unit

Moving my library from Chicken to Chez (when I moved it from Chez to Chicken originally, it was much smaller and I only used a few asserts to test), I discovered that:

  1. Chicken's test egg is pretty nice but non-standard.
  2. SRFI-64 (from Thunderchez ) is OK as far as it goes, but has an inadequate test runner (the default just lists PASS/FAIL with no explanation for each test, and has one total). Ridiculous when you have dozens or hundreds of tests.
  3. There's no good alternative. There's a SchemeUnit which is for PLT Scheme née Racket, and a couple others which aren't SRFI-64 and aren't for Chez.

So I ended up writing my own:

Here's how it works:

#!/usr/bin/env scheme-script
;; example-test.ss

(import (chezscheme)
    (srfi s64 testing) ;; thunderchez
    (scheme-test-unit)
)

(define (all-tests)
(test-group "All Tests"

(test-group "Math"
    (test-equal "add" 4 (+ 2 3))
)

(test-group "Strings"
    (test-equal "append" "foobar" (string-append "foo" "bar"))
)

) ;; test group "All Tests"
) ;; all-tests

(define (main argv)
    (scheme-test-configure argv)
    (all-tests)
)

(main (command-line-arguments))

----
% chmod 755 example-test.ss
% ./example-test.ss --help
Usage: scheme-test-configure [-v|--verbose|-q|--quiet|-o FILENAME|--output FILENAME]
% ./example-test.ss
*** All Tests START
*** Math START
- add [(+ 2 3)] expected <<<5>>> but got <<<4>>>
*** Math END, PASS: 0 / FAIL: 1
*** Strings START
+ append [(string-append foo bar)]
*** Strings END, PASS: 1 / FAIL: 0
*** All Tests END, PASS: 0 / FAIL: 0
FINAL PASS: 1 / FAIL: 1

My own test cases come out:

% ./marklib-test.ss -q
*** Control END, PASS: 4 / FAIL: 0
*** Logic END, PASS: 12 / FAIL: 0
*** Math END, PASS: 18 / FAIL: 0
*** Strings END, PASS: 29 / FAIL: 0
*** Any END, PASS: 31 / FAIL: 0
*** Hashtable END, PASS: 9 / FAIL: 0
*** List END, PASS: 6 / FAIL: 0
*** Maybe END, PASS: 6 / FAIL: 0
*** Stack END, PASS: 10 / FAIL: 0
*** Vector END, PASS: 8 / FAIL: 0
*** Data Structures END, PASS: 0 / FAIL: 0
*** Dice END, PASS: 7 / FAIL: 0
*** All Tests END, PASS: 0 / FAIL: 0
FINAL PASS: 140 / FAIL: 0

Green Window Means Thunderchez is Working

This weekend, I wanted to have dynamic access to SDL for some interactive graphing, and again ran into the problem from More Fun and Swearing with Scheme. With more experienced eyes, I read the Chez Scheme User's Guide and discovered everything I'd done wrong a year+ ago.

First, install Thunderchez somewhere, I just put it in ~/Code/CodeChezScheme

In particular, read ffi-utils.sls which provides somewhat easier wrapping of binary flags; thunderchez is a very low-level port.

Second, install SDL2, I used port install libsdl2 and that puts it in /opt/local/lib, but in production I'll use a local dir. Read the SDL2 API for reference.

Took me a few hours to translate the basic functions I needed, and now I have:

sdltest.ss:

#!/usr/bin/env scheme-script
;; sdltest.ss
;; Copyright © 2020 by Mark Damon Hughes. All Rights Reserved.

(import (chezscheme)
    (sdl2)  ;; thunderchez
)

;; Set this to your location. I used `port install libsdl2`, YMMV.
;; In production, I'll use a local dir with libs.
(load-shared-object "/opt/local/lib/libSDL2.dylib")

(define kWindowWidth 320)
(define kWindowHeight 240)

(define gWindow )
(define gRender )
(define gState 'quit)

(define (gr-init title x y w h)
    (sdl-set-main-ready)
    (sdl-init sdl-initialization-everything)
    (set! gWindow (sdl-create-window title x y w h (sdl-window-flags 'shown 'allow-highdpi)) )
    (set! gRender (sdl-create-renderer gWindow -1 (sdl-renderer-flags 'accelerated)) )
    (sdl-show-window gWindow)
    (sdl-raise-window gWindow)
)

(define (gr-event)
    (make-ftype-pointer sdl-event-t (foreign-alloc (ftype-sizeof sdl-event-t)))
)

(define (gr-rect x y w h)
    (let [ (rect (make-ftype-pointer sdl-rect-t (foreign-alloc (ftype-sizeof sdl-rect-t)))) ]
        (ftype-set! sdl-rect-t (x) rect x)
        (ftype-set! sdl-rect-t (y) rect y)
        (ftype-set! sdl-rect-t (w) rect w)
        (ftype-set! sdl-rect-t (h) rect h)
        rect
))

(define (gr-mainloop)
    (let [ (event (gr-event)) ]
        (let mainloop []
            ;; event
            (sdl-poll-event event)
            (let [ (evt-type (ftype-ref sdl-event-t (type) event)) ]
                (cond
                    [(eqv? evt-type (sdl-event-type 'quit))  (set! gState 'quit)]
                    [(eq? gState 'main)  (main-event evt-type event) ]
                ))
            ;; TODO: loop polling until no event left
            ;; update
            (case gState
                [(main)  (main-update) ]
            )
            ;; render
            (sdl-render-clear gRender)
            (case gState
                [(main)  (main-render) ]
            )
            (sdl-render-present gRender)
            ;; loop
            ;; TODO: wait timer, currently burns CPU
            (unless (eq? gState 'quit) (mainloop))
    ))
    (gr-shutdown)
)

(define (gr-shutdown)
    (sdl-quit)
)

(define (main-event evt-type event)
     ;; no event handling yet
)

(define (main-update)
     ;; no updates yet
)

(define (main-render)
    (sdl-set-render-draw-color gRender 0 255 0 255) ;; green
    (sdl-render-fill-rect gRender (gr-rect 0 0 kWindowWidth kWindowHeight))
)

(define (main argv)
    (gr-init "Hello, Thunderchez!" 100 100 kWindowWidth kWindowHeight)
    (set! gState 'main)
    (gr-mainloop)
)

(main (command-line-arguments))

To start it I just need:

% CHEZSCHEMELIBDIRS=thunderchez-trunk: ./sdltest.ss

Green window means it's working! And it can be closed with the window close button or Cmd-Q, but menu or other events don't work yet.

So… to port my graphics library from Chicken to Chez is probably a week's work, maybe less—I only fill rects, blit images, play sounds, and read a few events. My sound support on Chicken is minimal because the maintainer didn't bother to port the audio API, I had to do that myself.

Rehosting my game on Chez is an unknowably difficult debugging problem, but I code in a way that's usually portable. Large chunks of my large marklib library exist only to work around the non-portable parts and missing features of R5RS.

But the advantages are:

  1. Chez's R6RS is a better language than Chicken's R5RS++/R7RS-not-really.
  2. Chez is much faster, sometimes just 10-25%, sometimes hundreds of times, even compiled.
  3. Chez's JIT compiler is fast and invisible. Working on Chez is instant gratification. Chicken has a 30-second compile before I see anything, which has been impossibly frustrating.

At the very least, if I get my graphics library ported I can do the graphing I wanted last weekend.

Perilar: Dark Weaver

Perilar was my favorite of my games, an original iPhone RPG (loosely inspired by Rogue, Ultima, & Zelda). Alas, the App Store is a pain in the ass to stay current in, and Perilar needed updates I wasn't willing to jump thru Apple's hoops to deliver, so it's been gone from the store for a few years. Does anyone even remember me?

Then I wrote & released Brigand, a stripped-down realtime roguelike, which apparently everyone got mad at either because it was too hard (it was, but I liked that), or cost $9.99 which is SO MUCH MONEY on the goddamned App Store. And again the App Store made it obsolete and it wasn't reasonable to update it.

So I got back to a new desktop version of Perilar. I wrote a JS prototype (Fallen Kingdom) that wasn't fast enough to be usable, but let me rapidly test a bunch of new things. Now in Scheme, after a very long time, I have a nice, fast, nearing-complete sequel to Perilar: Dark Weaver.

Where I am right now:

  • World has both hand-designed and random sectors, tho the latter need some smoothing out to look like real terrain. I can walk across the world, at least until I hit impassable sectors. Porting my complex map and dungeon generators in is the next task. I have spent the last 6 months building features in the Town of Amity, and I'm ready to move on!
  • Most of the tile art is from the DawnLike set, with a lot of customization and new art where needed, and I've stuck to the DawnLike palette, it really has a nice old-timey look, a little less garish than the Atari 800, brighter than C64.
  • Player art isn't going to be these sprites, but the paperdolls I have are 2-facing (East/West), and I'd prefer 4- or 8-facing (you can move diagonally!); so I still need to find or draw (oh no) those.
  • NPCs have dialogue trees, stores, and special abilities (like the Innkeeper healing you; they're not super-powered).
  • Combat, with multiple attack/defense options, works in my test area. I haven't spread monsters around the sectors yet, but they've been developed and tested in the JS prototype.
  • Loot is extensive, magical weapons and armor have all the complex options you'd expect. I'm being a hardass on encumbrance in this one, because you can drop loot anywhere and come back for it. (Not quite the hardest possible ass; gold doesn't count towards weight, which it does in tabletop RPGs!)
  • Spells beyond Magic Missile are not implemented at all yet; will probably ship with only the dozen basic spells from the original release, and advanced spells added in an update. You won't find anyone to teach those for a long time anyway. Despite that, Wizards are still useful with magic wands.
  • New bosses, boss arenas, deeper dungeons, main quest, and sidequests.
  • At least one sector will be user-modifiable, tho I don't know if it'll be in the first release. You can buy furniture and walls, and fix up your own town. There's useful things you'll be able to get from that. (The building mechanic half works now; gathering doesn't).
  • Currently tested on Mac, should be buildable with no or very few changes to Windows, Linux, BSD, etc., but I need to get proper test environments for all of those.
  • Will be for sale on itch.io sometime this year. Price TBD.

I feel super awkward about self-promotion, but I do have a Patreon, and for Gold level you'll get betas; I haven't explained this, but at any level, when you've paid up whatever the cover price of the game ends up being, you'll get a full release license for it, too.

darkweaver-2020-01-08-2
darkweaver-2020-01-08-3