Retrospective: PortalWorlds

What worked, what sucked, lessons learned, The More You Know, and knowing is half the battle, go Cobra, etc.

JS: Test Your Libraries -1

Especially the hard-to-test parts. I had one "obviously correct" array shuffle function I've been using for maybe 10 years:

WRONG RIGHT
function arrayShuffle(arr) {
    for (let i = arr.length-1; i >>= 1; --i) {
        const j = Math.floor(Math.random() * i); // WRONG. NO.
        const tmp = arr[i];
        arr[i] = arr[j];
        arr[j] = tmp;
    }
    return arr;
}
/** Fisher-Yates */
function arrayShuffle(arr) {
    for (let i = arr.length-1; i >= 1; --i) {
        const j = Math.floor(Math.random() * (i+1)); // RIGHT. YES.
        const tmp = arr[i];
        arr[i] = arr[j];
        arr[j] = tmp;
    }
    return arr;
}

And because of that, I got a distorted shuffle. I only noticed because no mob using my dumb-as-rocks not-even-AI would ever choose to move north… everything ended up piled up at the bottom of the map.

It's hard to test randomness, but you can check that values are in a sane range. For a decade I've had wrong code just copy-pasted into new projects because I never checked my assumptions.

JS: UI Libraries are Great +1

I mean, I knew that. But how easy it is to throw something up is a bit of a surprise every time I do it. It's a fast, dynamic, functional/OOP language that also has good graphics and sound libraries (I didn't do sound in PortalWorlds, but I will before 1.0), and tolerable event handling these days.

I'm not releasing the full source, but in a week or two I'll add a few more things and release my common library under BSD license. This is mostly stuff from my Learn2JS project, and some from TTMS-76, but those have a giant framework for running scripts, this is just a few functions for a pure JS application, and greatly condensed.

Pulling in common code makes this stuff so much easier.

JS: Application Structure -1

There's no proper "run loop" in PortalWorlds, and late in the process that bit me in the ass.

What you should do is, collect events, put them in a queue. Every X milliseconds (I usually use 30 FPS, so 1000/30=33.333ms) have setInterval do an event-update-redraw loop. If something takes a long time or requires animation updates, it must be done in chunks or in a background thread (using Web Workers ). This is true of any application, not just games.

What I did wrong is having events immediately perform actions, and the setInterval just does update-redraw. This is much easier, but it's wrong. I have no control currently over blocking user actions, and animation has to all happen at once.

The trick I used is to keep a "frame" counter, and that chooses animation frames, moves floating text and the tracers of missiles and fireballs. Next turn invoked by user event just wipes those out. An amusing side-effect is missiles vaporize corpses as they fly past.

Switching to a correct run loop isn't super hard, but does require changes to all my timing and animation hacks, so at this point it's not worth it.

JS: Fonts -1

I used Oryx's "simplex" font, which looks fine and bitmappy in web pages, but in Canvas it gets antialiased, and I wasn't able to make it stop. So I have to make all fonts slightly larger than I'd like, and they're still kind of blurry.

The smarter way would be to use a bitmap font, and a very simple bitmap text renderer. That's what I do in Perilar: Dark Weaver, with an ATASCII-inspired fantasy font. But in 7 days I didn't have time to write and test that.

That's on a "maybe later" list for 1.0. Alternately, I could figure out how to get them antialiased in Canvas?

JS: Minimizer -1

I used to use yuicompressor, which renames variables and aggressively minimizes your code into unreadable but very compact line noise. But with the long-drawn-out death of Yahoo!, that hasn't been updated in a decade, and it doesn't handle ES2020.

I could probably have run everything thru Babel and then yuicompressor, but it's time to move on? So I just used Crockford's jsmin which only removes whitespace. It might be a good time to look into compiling JS to WASM binaries.

My build.zsh script:

#!/bin/zsh
rm -rf portalworlds*.zip
rm -rf build
mkdir -p build
mkdir -p build/js

# *.js -> .min.js
for f in js/*; do
    n=`basename $f .js`
    perl -ne 's/const DEBUG = true;/const DEBUG = false;/; s/^\s*DLOG.*$//; print;' $f >build/$f
    jsmin-crockford <build/$f >build/js/$n.min.js "Copyright (c) 2020 by Mark Damon Hughes. All Rights Reserved."
    rm build/$f
done

# index.html
perl -ne "s/\.js'/.min.js'/; print;" index.html >build/index.html

cp -R favicon.* style.css i ttf build
cd build
zname=portalworlds-$(date "+%Y%m%d_%H%M%S").zip
zip -r9Xq ../$zname * -x "*.DS_Store"
cd ..
echo $zname

Game: Combat Design +1

So I knew I wanted combat to be very swingy (allowing anything from instant death to instant kill), but not have levels or many stats; I didn't have time or inclination to make another detailed RPG!

Instead, combat works by adding your current strength and enemy's current strength, rolling 2 dice in half that range, so central results are more likely, but it's easy to get results at either far end. Then apply the difference from your strength as damage to you or the monster, depending on which side of the line it's on:

const roll = Math.floor(dice(2, this.strength + mob.strength)/2);
if (roll <= this.strength) {
    const dmg = Math.floor( (this.strength - roll) * (100 + this.getDamage()) / 100 );
    mob.takeDamage(dmg, true);
} else {
    const dmg = Math.floor( (roll - this.strength) * (100 - this.getDefense()) / 100 );
    message(mob.toString()+" hits you for "+dmg+" damage");
    this.takeDamage(dmg);
}

Kind of ridiculously simple, but it makes for symmetric combat, so player-attacks-monster and monster-attacks-player have the same results. It's not practical for a tabletop game, but this is a really fun mechanic in a computer game.

Damage and Defense from gear just modify the result by percentiles, they don't affect attack roll at all. If your strength + defense % is less than the monster's strength, you can be one-hit-killed, and vice versa.

Experience adds to current strength, and increases base strength if you're near max; you need to finish fights against slightly superior foes unharmed to grind up strength.

Game: Spells +1/-1

I knew I didn't have time for a lot of magic, and I never feel like I'm finished with magic systems anyway. So here I just picked 4 types: AOE damage, AOE control, escape, and heal; or as the game knows them, Fireball, Sleep, Invisible, and Heal. Then rather than have MP and worry about regeneration, I just give you "base" spells in each based on class, and you can pick up scrolls to add new points to current spell total. When you heal with mana potions or going thru a portal, you get back your base spells.

Last three were trivial: Sleep just searches a radius and has a die(100) > strength chance to give humanoid or animal targets a sleep condition for 2d4 turns; if they're sleeping, they skip the turn. Invis just sets an invis condition for 2d4 turns; mobs ignore you if you have that condition. Couple places like melee and taking damage I manually clear the Invis and Sleep conditions. Heal just heals 50% of base strength, woo.

Even with just 4, doing Fireball turned out to be quite challenging (the Application Structure problem); how do I show it go across the board, maybe have a turn or two delay, then explode? Well, it does it by moving 4 steps every turn, and only having the fake animation for a tracer of where it's been.

Game: Permadeath +0

Permadeath happened more by lack of save state than any intention. I much prefer to have save slots, and then you can choose to save your game and reload last or earlier, or you can play hardcore and never reload. It puts the moral burden of choosing permadeath on the player, not the developer.

But JS localStorage doesn't really have room to stash a huge amount of data. In Reaper's Crypt I had a very aggressive compression for the map (which looks like a giant grid of tiles but it's really about 64 rooms per level), and it's still problematic, sometimes maps just can't be saved.

I could save the character as of last portal you entered, and regenerate maps. But that encourages "stair-scumming", which I do have a problem with. So instead this is very arcade-like, you just play until you die, then insert another quarter.

I do plan to add a scoreboard, both local and maybe server-based for a copy hosted on mysticdungeon.club.

Personal: Shipping is Awesome +1

Actually finishing something and making it public without a decade-long process is amazing. It's not perfect, and I know nothing is perfect, I should just ship things, but I can't normally do that. Having a hard deadline and meeting it was the best feeling.

Personal: Drone-Slack Balance -1

I was desperately exhausted afterwards. Bone tired all day Sunday, and I'm still feeling it on Monday. 7 days in a row, even with a couple shorter days, is about 3 or 4 days too many.

I'm nocturnal. I like to wake up ideally just before midnight (but sometimes backslide to evening if Real Life interferes), eat and coffee, then work until dawn, walk the dog at sunrise, maybe get my own walk in (tiny dog cannot keep up for a mile+), then finish up and goof off until early afternoon when I can sleep. The schedule weirds out some daywalkers, but it's quieter and more compatible with "morning people" (ugh) than waking in afternoon and sleeping at dawn, which I did for decades.

But normally that work is 3 days of code, 1-2 days of writing or art, 2-3 days of just playing videogames or going Outside, doing Real Life AFK stuff.

Drones who can work all the time frighten me, they're basically Terminators. People who Slack all the time frighten me, they're a waste of precious oxygen & water, and I may foolishly try to rely on these people and get nothing. You need to be in the middle area.

Scheme-Test-Unit

Moving my library from Chicken to Chez (when I moved it from Chez to Chicken originally, it was much smaller and I only used a few asserts to test), I discovered that:

  1. Chicken's test egg is pretty nice but non-standard.
  2. SRFI-64 (from Thunderchez ) is OK as far as it goes, but has an inadequate test runner (the default just lists PASS/FAIL with no explanation for each test, and has one total). Ridiculous when you have dozens or hundreds of tests.
  3. There's no good alternative. There's a SchemeUnit which is for PLT Scheme née Racket, and a couple others which aren't SRFI-64 and aren't for Chez.

So I ended up writing my own:

Here's how it works:

#!/usr/bin/env scheme-script
;; example-test.ss

(import (chezscheme)
    (srfi s64 testing) ;; thunderchez
    (scheme-test-unit)
)

(define (all-tests)
(test-group "All Tests"

(test-group "Math"
    (test-equal "add" 4 (+ 2 3))
)

(test-group "Strings"
    (test-equal "append" "foobar" (string-append "foo" "bar"))
)

) ;; test group "All Tests"
) ;; all-tests

(define (main argv)
    (scheme-test-configure argv)
    (all-tests)
)

(main (command-line-arguments))

----
% chmod 755 example-test.ss
% ./example-test.ss --help
Usage: scheme-test-configure [-v|--verbose|-q|--quiet|-o FILENAME|--output FILENAME]
% ./example-test.ss
*** All Tests START
*** Math START
- add [(+ 2 3)] expected <<<5>>> but got <<<4>>>
*** Math END, PASS: 0 / FAIL: 1
*** Strings START
+ append [(string-append foo bar)]
*** Strings END, PASS: 1 / FAIL: 0
*** All Tests END, PASS: 0 / FAIL: 0
FINAL PASS: 1 / FAIL: 1

My own test cases come out:

% ./marklib-test.ss -q
*** Control END, PASS: 4 / FAIL: 0
*** Logic END, PASS: 12 / FAIL: 0
*** Math END, PASS: 18 / FAIL: 0
*** Strings END, PASS: 29 / FAIL: 0
*** Any END, PASS: 31 / FAIL: 0
*** Hashtable END, PASS: 9 / FAIL: 0
*** List END, PASS: 6 / FAIL: 0
*** Maybe END, PASS: 6 / FAIL: 0
*** Stack END, PASS: 10 / FAIL: 0
*** Vector END, PASS: 8 / FAIL: 0
*** Data Structures END, PASS: 0 / FAIL: 0
*** Dice END, PASS: 7 / FAIL: 0
*** All Tests END, PASS: 0 / FAIL: 0
FINAL PASS: 140 / FAIL: 0

Green Window Means Thunderchez is Working

This weekend, I wanted to have dynamic access to SDL for some interactive graphing, and again ran into the problem from More Fun and Swearing with Scheme. With more experienced eyes, I read the Chez Scheme User's Guide and discovered everything I'd done wrong a year+ ago.

First, install Thunderchez somewhere, I just put it in ~/Code/CodeChezScheme

In particular, read ffi-utils.sls which provides somewhat easier wrapping of binary flags; thunderchez is a very low-level port.

Second, install SDL2, I used port install libsdl2 and that puts it in /opt/local/lib, but in production I'll use a local dir. Read the SDL2 API for reference.

Took me a few hours to translate the basic functions I needed, and now I have:

sdltest.ss:

#!/usr/bin/env scheme-script
;; sdltest.ss
;; Copyright © 2020 by Mark Damon Hughes. All Rights Reserved.

(import (chezscheme)
    (sdl2)  ;; thunderchez
)

;; Set this to your location. I used `port install libsdl2`, YMMV.
;; In production, I'll use a local dir with libs.
(load-shared-object "/opt/local/lib/libSDL2.dylib")

(define kWindowWidth 320)
(define kWindowHeight 240)

(define gWindow #f)
(define gRender #f)
(define gState 'quit)

(define (gr-init title x y w h)
    (sdl-set-main-ready)
    (sdl-init sdl-initialization-everything)
    (set! gWindow (sdl-create-window title x y w h (sdl-window-flags 'shown 'allow-highdpi)) )
    (set! gRender (sdl-create-renderer gWindow -1 (sdl-renderer-flags 'accelerated)) )
    (sdl-show-window gWindow)
    (sdl-raise-window gWindow)
)

(define (gr-event)
    (make-ftype-pointer sdl-event-t (foreign-alloc (ftype-sizeof sdl-event-t)))
)

(define (gr-rect x y w h)
    (let [ (rect (make-ftype-pointer sdl-rect-t (foreign-alloc (ftype-sizeof sdl-rect-t)))) ]
        (ftype-set! sdl-rect-t (x) rect x)
        (ftype-set! sdl-rect-t (y) rect y)
        (ftype-set! sdl-rect-t (w) rect w)
        (ftype-set! sdl-rect-t (h) rect h)
        rect
))

(define (gr-mainloop)
    (let [ (event (gr-event)) ]
        (let mainloop []
            ;; event
            (sdl-poll-event event)
            (let [ (evt-type (ftype-ref sdl-event-t (type) event)) ]
                (cond
                    [(eqv? evt-type (sdl-event-type 'quit))  (set! gState 'quit)]
                    [(eq? gState 'main)  (main-event evt-type event) ]
                ))
            ;; TODO: loop polling until no event left
            ;; update
            (case gState
                [(main)  (main-update) ]
            )
            ;; render
            (sdl-render-clear gRender)
            (case gState
                [(main)  (main-render) ]
            )
            (sdl-render-present gRender)
            ;; loop
            ;; TODO: wait timer, currently burns CPU
            (unless (eq? gState 'quit) (mainloop))
    ))
    (gr-shutdown)
)

(define (gr-shutdown)
    (sdl-quit)
)

(define (main-event evt-type event)
    #f ;; no event handling yet
)

(define (main-update)
    #f ;; no updates yet
)

(define (main-render)
    (sdl-set-render-draw-color gRender 0 255 0 255) ;; green
    (sdl-render-fill-rect gRender (gr-rect 0 0 kWindowWidth kWindowHeight))
)

(define (main argv)
    (gr-init "Hello, Thunderchez!" 100 100 kWindowWidth kWindowHeight)
    (set! gState 'main)
    (gr-mainloop)
)

(main (command-line-arguments))

To start it I just need:

% CHEZSCHEMELIBDIRS=thunderchez-trunk: ./sdltest.ss

Green window means it's working! And it can be closed with the window close button or Cmd-Q, but menu or other events don't work yet.

So… to port my graphics library from Chicken to Chez is probably a week's work, maybe less—I only fill rects, blit images, play sounds, and read a few events. My sound support on Chicken is minimal because the maintainer didn't bother to port the audio API, I had to do that myself.

Rehosting my game on Chez is an unknowably difficult debugging problem, but I code in a way that's usually portable. Large chunks of my large marklib library exist only to work around the non-portable parts and missing features of R5RS.

But the advantages are:

  1. Chez's R6RS is a better language than Chicken's R5RS++/R7RS-not-really.
  2. Chez is much faster, sometimes just 10-25%, sometimes hundreds of times, even compiled.
  3. Chez's JIT compiler is fast and invisible. Working on Chez is instant gratification. Chicken has a 30-second compile before I see anything, which has been impossibly frustrating.

At the very least, if I get my graphics library ported I can do the graphing I wanted last weekend.

Adult Engineer Over-Optimization as the Motie Problem

Looking at my Scheme code and the way I customize it, I'm starting to see the real reason evil megacorps (and wannabe evil startups) won't hire even middle-aged programmers or use your favorite weirdo language, they just want young idiots who code Java or Go.

If you think about a standard software career, there's maybe 10 years of a submissive fool badly coding crap languages ^1 like Java, Go ^3, PHP, JavaScript ^4. They just got out of college or self-trained, and can barely copy existing algorithms, let alone think of one for themselves. This is why FizzBuzzTest ^5 is such a good novice coder test: It requires following directions exactly, and slightly competent logic skills, but not much more.

Then maybe 10 years of them being project managers and "architects", running waterfall and GANTT charts; they'll say they're "agile" but then have a giant JIRA repo of "backlog" features which have to be implemented before shipping, weekly 4-hour planning "backlog grooming" meetings, and unrealistic estimates. This is sufficient to build all kinds of horrible vertical prisons of the mind like Azkaban Facebook.

Then they either retire, or are "downsized", and now what? So they work on their own code, do maintenance on old systems, or leave the industry entirely.

If they work on their own, freed of evil megacorp constraints, they're going to end up in something idiosyncratic and expressive, like Scheme, LISP, Forth, or a custom language. Make their own weirdo environment that's perfectly fit to themself, and unusable/unreadable by anyone else.

Case in point, I needed an object model. There's one I like in Gerbil, and Gerbil's blazing fast, but I can't make a full SDL2 library for it yet (Gambit's FFI is hard, I've hit some bugs, and there's a LOT of library to interface to), and I'm using a bunch of other Chickenisms anyway, so I can't really move to it yet. Instead I just made my own simple object libary, with a couple macros to hide the ugly reality behind it:

(test-group "Object"
    (test "Object" 'Object (class-name Object))
    (let [ (obj (@new Object))  (bug #f)  (cow #f)  (duck #f) ]
        (test "Object-to-string" "[Object]" (@call obj 'to-string))

        (define-class Animal Object)
        (define-field Animal 'legs 0)
        (define-field Animal 'color #f)
        (define-method Animal 'init (self legs color)
            (set! (@field self 'legs) legs)
            (set! (@field self 'color) color) )
        (define-method Animal 'speak (self)
            (sprintf "The ~A ~A with ~A legs says " (@field self 'color) (class-name (@class self)) (@field self 'legs)) )

        (set! bug (@new Animal 6 "green"))
        (test "bug-legs" 6 (@field bug 'legs))
        (test "bug-color" "green" (@field bug 'color))
        (test "Bug speak" "The green Animal with 6 legs says " (@call bug 'speak))

        (define-class Cow Animal)
        (define-method Cow 'init (self color)
            (@super self 'init 4 color) )
        (define-method Cow 'speak (self)
            (string-append (@super self 'speak) "MOO!") )
        (set! cow (@new Cow "brown"))

        ;; second class to make sure classes don't corrupt shared superclass
        (define-class Duck Animal)
        (define-method Duck 'init (self color)
            (@super self 'init 2 color) )
        (define-method Duck 'speak (self)
            (string-append (@super self 'speak) "QUACK!") )
        (set! duck (@new Duck "black"))

        (test "Cow speak" "The brown Cow with 4 legs says MOO!" (@call cow 'speak))
        (test "Cow to string" "[Cow color:brown;legs:4]" (@call cow 'to-string))
        (test "Duck speak" "The black Duck with 2 legs says QUACK!" (@call duck 'speak))
        (test "Duck to string" "[Duck color:black;legs:2]" (@call duck 'to-string))

        (test "instance-of?" #t (instance-of? cow Cow))
        (test "instance-of? parent" #t (instance-of? cow Animal))
        (test "instance-of? grandparent" #t (instance-of? cow Object))
        (test "instance-of? cousin-false" #f (instance-of? cow Duck))
        (test "instance-of? not an obj-false" #f (instance-of? "wtf" Cow))
    )
)

The implementation code's not much longer than the tests, but it's not quite done for me to show off; I need to switch my macros into non-hygeinic forms so I can get rid of the (self) in define-method, and introduce an Objective-C-like _cmd field for self-reflection, and message-not-understood handling. There's always more tinkering to do.

Which is great for me, but makes my code an undocumented (mostly) new language, unusable by anyone normal. A giant pile of crap Java program, no matter how old, can be "worked on" (more crap piled on top) by any teenage Bro Coder.

All of which brought to mind The Mote in God's Eye, where the Motie Engineers over-optimize everything into a tangled mess, and the Watchmaker vermin are even worse, wiring up everything to everything to make new devices. The threat posed by and solution to Scheme programmers, in your usual authoritarian megacorp scenario, is similar to Watchmakers.


^1 Swift is intended to fit this niche much more than weirdo expressive Smalltalk+C Objective-C was, BDSM ^2 to prevent one from writing "bad" code, but it's not there yet; the reality of low-level software dev can't be simplified as much as Apple wants, and their C++ developers weren't up to the task anyway.

^2 Bondage-Domination-Sado-Masochism; aka strict type systems and code flow analysis, that prevent one from writing "bad" code at the cost of annotating everything with types instead of doing useful work. I'm not kink-shaming people who do that for sex, only those who do it to their own software.

^3 Rob Pike has openly said they can't give a powerful language to newbie Googlers, they mostly just know Java, C, C++, which is why Go is so limited and generic.

^4 Oddly, JS is basically a LISP with really shitty syntax. It's easy to make trivial, broken junk in it, but it's also powerful and expressive if you're an old maniac who understands the Self-based object system.

^5 Oh, fine, but only so I can demonstrate something:

(define (fizzbuzz-test i n s)  (if (zero? (modulo i n))  (begin (display s) #t)  #f) )
(define (fizzbuzz i)
    (unless (any identity (list (fizzbuzz-test i 3 'Fizz) (fizzbuzz-test i 5 'Buzz)))  (display i))
    (newline) )
(for (i 1 100) (fizzbuzz i))

Totally different structure from the usual loop-if-else repetition and hardcoding of everything, because Scheme encourages coding in small pieces. Of course I wrote my own for macro which expands to a named let loop; there's many like it but this one is mine. More Motie engineering.

Programming the Atari 8-bit

My programming started in 1979 with the TRS-80 Model I, but in late 1981? early 1982?, I got my Atari 800, and later a 1200XL, then Atari ST. Those are what I consider "my computers".

Last few weeks for hobby time, I've taken up playing with an Atari 8-bit emulator, and may soon buy an old machine (130XE? I guess?) and a modern SD-card reader, and HDMI adapter unless I want to set up my old CRT… Yes, this is "pointless", but it's the most emotionally rewarding programming I've done in some time.

Had to do a lot of setup to get to this point, though. Follows are my excessive notes, which will hopefully be useful to others.

The keyboard mapping in AtariMacX is weird, I finally figured out:

Mac Key Atari Key
` Break
F2 Option
F3 Select
F4 Start
F5 Reset
Sh-F5 Cold Boot
Opt-F5 Insert Char (be REAL DAMN CAREFUL not to miss the Opt key!)
Sh-Opt-F5 Insert Line (same, DANGER WILL ROBINSON DANGER)
Home / Opt-F7 Clear
End Atari/Inverse
PgDn / Opt-F10 Help (XL/XE)
Opt-F1 F1 (1200XL)
Opt-F2 F2 (1200XL)
Opt-F3 F3 (1200XL)
Opt-F4 F4 (1200XL)
Capslock Cycle caps, may take several tries of caps A backspace repeat until you get lowercase, not graphics or uppercase.
Sh-Capslock Uppercase, almost always works

Typing on a real Atari keyboard is probably the #1 reason to get real hardware instead of emulation.

Immediately it comes rushing back, how much I didn't like the default environment of blue screen, clicky keyboard, inset margins. Easy to fix with a few pokes, but I don't want to do that every time I reboot, so I need a startup program.

  • First, configure Atari800MacX with the subdirectories next to it. It comes with all these folders in user space, but it's actually mapped to somewhere in /var, which is awful.
  • Make a boot disk. Media -> Disk Image Conversions -> XFD to ATR, pick the DOS25.XFD image in OSRoms, and call that boot.atr, store it in Disks, Load it in D1 Cmd-1 and pick boot.atr.
  • Reboot into DOS, by Control -> Disable BASIC. Bask in the glory of Atari DOS 2.5.
  • Make a data disk, Media -> New Floppy Image, I went with Medium Density (130K) since almost everything can read that, assign to Drive 2, and call that disk2.atr or whatever.
    • From Atari DOS, Format: I <return> D2: <return> Y <return>
    • Preferences -> Boot Media -> Set to Current Media, Save Configuration
  • My Atari BASIC project on Gitlab
    • Based on what I remember of my old main menu, I had a ton more stuff but I'm slowly adding routines as I need them. This can also be a shell for new programs, delete 11-9998 and use the subroutines. I wrote Draw to test joystick & function key scanning, not to be a good paint program, typed in a Music demo to make sure I had sound working.
      • Digression: This is not an efficient structure, because high line numbers take longer to find; an optimizing Poindexter would put the subroutines tightly packed at 1-999 and the program at 1000+, but it's massively easier to read & work with this way. I won't be in BASIC that much anyway, it's just for utility work.
    • Download AUTO.LST, convert Unix newlines (char 10) to the ATASCII newline (char 155 õ), and drop it in the HardDrive1 folder.
    • % LANG=C tr '\233' '\n' <AUTO.LST.TXT >AUTO.LST
    • Or you can just Media -> Edit an .ATR disk image, import file, and that has a newline conversion.
    • From BASIC, E."H1:AUTO.LST" <return> RUN <return>, pick Y. (Script AUTORUN.SYS), and enter:
      • ?"MAINMENU"
      • E."H1:AUTO.LST"
      • RUN
      • .
    • Change H1 to D1 if you saved it in your boot.atr.
    • Now it'll do that on every bootup from that floppy. Reboot to be sure it works.
    • If you make changes to your main menu, remember to LIST "H1:AUTO.LST". I use LIST/ENTER (text LST format) instead of SAVE/LOAD (tokenized BAS format) so I can read it from the Mac; BAS is slightly smaller and much faster to load/save, but it doesn't matter with emulation or an SD-card.

    • Atari-autorunsys

  • BASIC set up and tested, and it's a convenient place for little utilities, but now for real programming.

  • Atari Macro Assembler and Program Text Editor

    • Download this, read the fine manuals; more for MEDIT than the assembler unless you're really hardcore. I will probably do little or no assembly, even tho back in the '80s I could hand-assemble short programs directly into ATASCII codes to run from BASIC; bug-eating freak that I was.
    • Read the MEDIT manual. It's quite a respectable full-screen editor with command mode for search/replace, block editing, etc.
    • Open the Atari Macro Assembler and Program Text Editor.atx (ATX is write-protected or encrypted or something; you can't use them directly, and have to disable the SIO speedup hack in emulator) disk in drive 2 of your Atari (Cmd-2), Control -> Disable BASIC (which will reboot to DOS). So you want the program files off that:
      • DOS: C <return> D2:MEDIT,D1:MEDIT <return>
      • DOS: C <return> D2:MEDITCM.BAS,D1:MEDITCM.BAS <return>
      • DOS: C <return> D2:AMAC,D1:AMAC <return> (skip if you'll never write ASM)
      • DOS: C <return> D2:SYSTEXT,D1:SYSTEXT <return> (I think only needed for AMAC?)
      • Eject: Ctrl-Cmd-2
      • Reload your data disk, Cmd-1, disk2.atr.
    • Control -> Enable BASIC, LOAD "D1:MEDITCM.BAS" <return> RUN <return> and configure MEDIT however you like.
      • Language: PAS
      • Tabstops: Set at 5 and +4 after the existing ones, because 8-wide tabs are crazy in a 40-column screen. Yes, I'm a tabs not spaces guy, OBVIOUSLY.
      • Margins: 1,40
      • Colors: 12,4,14 (sadly can't be 0 or 2 background luminance, because the cursor is black)
      • Flags: Tabs: Expand, Shift-Lock: No (starts in lowercase).
      • Save & Return to DOS.
      • You can just copy the MEDITPAS.ECF to MEDITTXT.ECF, etc., you don't need to run the tool for each language, but it doesn't have a default mode. Note you also have to copy these to each disk you're editing on, or it switches back to the stupid defaults:
      • DOS: C <return> D1:*.ECF,D2: <return>
      • DOS: L <return> MEDIT <return>, filename D2:HELLO.PAS, and enter:
        program hello;
        var c: char;
        begin
          writeln('Hello, Atari!');
          read(c);
        end.
        
      • <option> exit <start> to save & exit. Note return doesn't execute commands in MEDIT, start does. Kids Today™ have some meme about how hard it is to exit vi? Ha ha, they have no idea. RTFM.

  • Finally ready to program in Action! or Pascal, which is what I mainly did back in the day.

    • Deep Blue C: Tragically underpowered version of Small-C. I loved it as an intro to C, but didn't use C for real until the Atari ST. It did produce standalone binaries and the compiler was easy to use, IIRC.
      UNSUPPORTED FEATURES
        Features in C not supported in DEEP BLUE C are:
        1) structures, unions
        2) multidimension arrays
        3) Floating point numbers
        4) Functions returning anything but int
        5) Unary operators: sizeof
        6) Binary operators: typecasting
      DIFFERENCES FROM STANDARD C
        THE DEEP BLUE C language has the  following nonstandard features:
        1. The last clause of a "switch" statement, either "case" or "default", must
      be terminated with a "break", a "continue" or a "return" statement.
        2. The ancient =<op> construct has been removed. Use <op>= instead.
        3. Characters are unsigned. Chars range in value from 0 to 255.
        4. Strings can not be continued on the next logical line.
        5. C source code lines can be a maximum of 79 characters long.
        6. Functions can have a maximum of 126 arguments.
      SPECIAL SYNTAX
        C uses several ASCII characters not available on the ATARI computer's
      keyboard. In particular the braces have been replaced by to two-letter
      combinations $( and $), and the tilde has been replaced by $-.  The $ character
      is not used in C, so your editor's find and replace command can be used to
      convert standard c programs into a format acceptable to DEEP BLUE C.
      
    • Action!: Custom language on cart for Atari, fantastic built-in editor (later the basis for the Paperclip word processor!), had a disk runtime system so you could distribute programs (also on AtariMania). But it came out a little later than my Pascal adventures, and it's a weird super-low-level language, and I think I'm in no mood to relearn it right now. Super goddamned fast, tho. May get into this if I'm frustrated later.

    • APX Pascal: Excessively complex process with a disk swap for every compile, compiles & links into PCode, no explanation of how to boot it. This is a very user-hostile compiler.
    • Kyan Pascal: Maze of command line tools. Doesn't work, at least for me, on emulation. It cycles through the tools, but never actually builds anything, eventually crashes and corrupts video. Makes a big deal of being usable from RAMDisk, but that doesn't matter on modern hardware.
    • Draper Pascal: Which I used in the '80s. Hilariously bad editor (but I can use MEDIT, so fuck that), compiler just fucking works, but only produces PCode (.PCD), so has to start from bootdisk or run Draper's menu then your program, ick. But this was no trouble to get running, so it wins.
      • Insert drpascal.atr in drive 1, reboot, boots into a menu.
      • 3 Compile program: D2:HELLO.PAS
      • 1 Run program: D2:HELLO
      • Total success! \o/ Hit any key to exit the program.
      • Drive 1, boot.atr, Drive 2, drpascal.atr, reboot
      • DOS: C <return> D2:AUTORUN.SYS,D1:PASCAL.COM <return>
      • DOS: C <return> D2:INIT.PCD,D1:INIT.PCD <return>
      • Cmd-2, disk2.atr
      • So now I can: DOS: L <return> PASCAL.COM <return>
      • And run Pascal programs. I could make a more focused runtime menu for it, maybe dir & list all the PCD files, the INIT.PAS source is included. If I ask it to compile, it prompts to insert drpascal.atr, and then I can switch back, which is reasonable.
      • Standard library is small but effective, seems like it has all the BASIC equivalent commands, and enough POKE/PEEK/ADDR stuff to let me do everything, including Player-Missile Graphics.
      • I can presumably now move all my source and disk2.atr contents to H1, so they can be managed & edited on the Mac, but I just wanted to get things running first.
      • Probably make another gitlab project (and actually sync it from git) when I get somewhere with that.

This took quite a lot of my hobby time doing something harder than actual work, to be honest. But I'm in a good place with it now.

Building Command Lines in zsh

I love zsh, but sometimes it's a tough love. Building command lines in strings is very hard because zsh doesn't want you hurting yourself on argument splitting like bash, so I just banged my head on that for like 15 minutes before I quit being dumb. Building them in arrays is super trivial:

% a=(args foo bar)
% b=$(ls)
% a=($a $b) # expands $a, adds $b to the end as a single argument
% $a
1=foo
2=bar
3=file1
file2

Note that arg 3 is multiple words/lines, but one arg. That's a nightmare to express in bash, but makes perfect sense at the exec level used by "real" (non-shell) programming languages.

args is a very useful debugging script I've had in some form since the '80s:

#!/bin/zsh
i=1
while [[ $# -gt 0 ]]; do
    echo "$i=$1"
    let i=i+1
    shift
done

A Scheme of Gerbils Runnin' in a Wheel

Chicken's mediocre performance is causing me problems, so I'm taking yet another language tour/spike.

Looking at Gerbil Scheme, as a faster Scheme and as a bonus it's closer to R7RS. It has far less library support than Chicken, but does have a FFI, and compiles to a native binary through C, so I really don't need too many libraries.

Instead of the #!key #!optional #!rest keywords used in Chicken and some others, Gerbil uses a literal template style for functions:

;; Chicken: (define (foo a #!key (b 'bar) #!optional (c "see?") #!rest rest)
;; Gerbil:
(def (foo a b: (b 'bar) (c "see?") . rest)
    (print "a=" a " b=" b " c=" c " rest=" rest "\n")
)
(foo "Mark" 1 b: "bees!" 'what) ;; note out-of-order optional & keyword
;; a=Mark b=bees! c=1 rest=what

I like this style better, but I dislike the "def" keyword (which uses the enhanced lambda that does this) instead of "define" which uses "lambda%" (traditional lambda).

Gerbil uses [] for list construction, and {} for method dispatch, so goodbye to the nicely distinguishable braces I was using in Chicken. The spec says implementations can do what they want with those symbols, but I wish they wouldn't. Ah, well. I'll add a shitload more whitespace and some comments and it'll be fine.

The struct/object system in Gerbil is pretty nice, they're slightly upgraded records, but there's an (@ obj field) syntax instead of (MyClass-field obj), and (defmethod {mymethod MyClass} ...) creates a polymorphic {mymethod obj args} which finds the right method for any object, which is especially useful for inheritance.

I tried doing some VT100 graphics just to proof-of-concept, but it's doing something to the terminal, such that the same escape codes which work in Python don't clear the screen fully in Gerbil. After a short losing battle with stty and termcap, I give up on that and I'll jump right to writing a C FFI to SDL, because in 2019 that's easier than writing to a console.

Daily reminder that everything we have made since 1984 is overcomplicated junk. On an Atari 800, this took a few seconds to type, and you could start coding a nice UI instantly:

atari-is-awesome-graphics1

Alas, we live in a fallen world, so this is going to be trouble. Here's my Gerbil FFI template so far:

package: myffi

(import
    :std/foreign
)

(export #t)

(begin-ffi
    ;; names of all Scheme wrappers to expose
    (chello)

(c-declare #<<CDECLEND

#include <stdio.h>

int chello(void) {
    printf("Hello this is C!\n");
    return 1;
}

CDECLEND
)

    (define-c-lambda chello () int "chello")

) ;; begin-ffi


; TODO for constants: (define FOO ((c-lambda () int "___result = FOO;")))

(define (main . args)
    (chello)
)

Gerbil has a Scheme-based build system, but I'm a caveman so I make another build.zsh:

#!/bin/zsh

function usage {
    echo "Usage: build.zsh MAIN.scm [LIBS] || -?"
    exit 1
}

if [[ $# -eq 0 || "$1" == "-?" || "$1" == "--help" ]]; then
    usage
fi

mkdir -p bin

main=`basename $1 .scm`

gxc -exe -static -O -o bin/$main "$@" || exit 1
echo "Built bin/$main"

Now:

% ./build.zsh myffi.scm
Built bin/myffi
% bin/myffi
Hello this is C!

Hooray! Unconditional success! Only took all afternoon and a half-pot of coffee!

Now I "merely" have to wrap all of SDL (well, just the parts I need) and get linking working and oh sweet merciless Cthulhu. And I still won't know how much this'll help my performance until I'm days into it. But, the first step is the hardest.

REXX Primes

Just a quick sanity check on performance.

/** primes.rexx */
/* Created 2019-09-26 */
/* Copyright © 2019 by Mark Damon Hughes. All Rights Reserved. */

PARSE ARG primeCount
IF \ DATATYPE(primeCount, "W") THEN DO
    SAY "Usage: primes.rexx N"
    EXIT 1
END

CALL clearPrimes
CALL sievePrimes
CALL printPrimes
EXIT 0

clearPrimes: PROCEDURE EXPOSE primes. primeCount
    primes. = 1
    primes.0 = primeCount
    primes.1 = 0
RETURN

sievePrimes: PROCEDURE EXPOSE primes. primeCount
    DO i = 2 TO primeCount
        DO j = (i * i) TO primeCount BY i
            primes.j = 0
        END
    END
RETURN

printPrimes: PROCEDURE EXPOSE primes. primeCount
    DO i = 1 TO primeCount
        IF primes.i THEN CALL CHAROUT , i || " "
    END
RETURN
# REXX: 0.8% C
% time rexx primes.rexx 1000000 >~/tmp/primes-rexx.txt
rexx primes.rexx 1000000 > ~/tmp/primes-rexx.txt  6.43s user 0.36s system 99% cpu 6.831 total

# Regina: 1.53% C
% time regina primes.rexx 1000000 >~/tmp/primes-regina.txt
regina primes.rexx 1000000 > ~/tmp/primes-regina.txt  3.25s user 0.26s system 99% cpu 3.521 total

# Python: 4.8% C
% time ./primes.py 1000000 >~/tmp/primes-python.txt
./primes.py 1000000 > ~/tmp/primes-python.txt  0.75s user 0.02s system 68% cpu 1.123 total

# Julia: 1.4% C
% time ./primes.jl 1000000 >~/tmp/primes-julia.txt
./primes.jl 1000000 > ~/tmp/primes-julia.txt  0.45s user 0.35s system 21% cpu 3.797 total

Most of REXX's bad time can be attributed to using stem variables in a tight loop, effectively string-keyed hashtables, so I'm sure an ooRexx Array implementation would be significantly faster. But stems are what you'd use in "real code", so caveat coder. In a long real-world program I don't think it's as big an issue, but it's definitely not received the kind of optimization love that newer languages have. Aesthetically, the REXX source is a little wordier and more explicit about globals access, but not hard to write or read.

I went ahead and grabbed Regina, and it doubles the speed of ooRexx. I'm still wary of it, but that's a big win.

Python's not terrible at anything; it's within a stone's throw of a compiled language at this point. Competent mediocrity has really made Python the new dynamic Java of our time. But you still can't do multithreading in it.

Julia's slow because the startup time is just atrocious; if I timed it internally after startup it'd be as fast as a compiled native program, but as a scripting language Julia's bad news.

(I do have real work to do, but I'll keep playing with REXX more over the next few days)

REXX Monarchial Madness is Hereditary

So, I was trying to figure out how to manage global vs. local state, and made the dumbest program possible:

/** array.rexx */

a. = 0
a.0 = 10
DO i = 1 TO a.0; a.i = i; END

SAY array_string("a.") -- prints 1 to 10
SAY alen -- prints "ALEN" because it doesn't exist in global scope

EXIT 0

/* Returns string representation of an array, stem. passed as first arg. */
array_string:
    PARSE ARG _arrayvar .
RETURN _array_string()

_array_string: PROCEDURE EXPOSE (_arrayvar)
    alen = VALUE(_arrayvar || 0)
    str = ""
    DO i = 1 TO alen
        IF i > 1 THEN str = str || ", "
        str = str || VALUE(_arrayvar || i)
    END
RETURN str

So… I've made a global variable "a." full of numbers. PROCEDURE makes a local variable context until you hit RETURN; you should never work at global scope if you can avoid it. So to pass a. in, I have to bounce through a global function array_string(), set a global var _arrayvar to whatever arg was given, which calls a local function _array_string and the paren in EXPOSE (_arrayvar) expands that into "a.", and then use VALUE() to dynamically read the variable.

Well, it works. I remember now how crazy complex REXX programs got, with giant global databases being exposed in every function. I bet this is the origin of my "World object" pattern in so many later languages.

This and another dumb example or two ate a couple hours of my brain, but it kinda makes sense again, you know? Interesting for a language I haven't written in anger in 20 years. I went fully back to SHOUTING KEYWORDS, it just makes sense.

The Python version of this is:

#!/usr/bin/env python3

def array_string(arrayvar):
    return ", ".join(map(str, arrayvar))

a = []
for i in range(1, 11): a.append(i)
print(array_string(a))

Which I'd argue is just as stupid, because Python thinks ranges end 1 before the last number, doesn't know how to join numbers with strings, on a large array that map will consume all memory, and is a single-pass interpreter so I have to put the function above the script code; but it is much, much shorter.

REXX is Still the King

Random REXX mention on Fediverse reminded me of good times with it. REXX was originally a scripting language for the IBM /360 series, later OS/2 where I encountered it. There's three main branches: REXX (original, simple procedural language), ooRexx (strict superset adding OOP tools to REXX), and NetRexx (almost-REXX implemented in Java, so it can interact with the entire Java API).

The things it does well are: Simple notation, like BASIC, but it's structured and dynamic: Variables have their NAME as default value, and it's trivial to evaluate strings as code. Variables can also be "stems", either indexed or associative arrays depending on key. a.=0; a.0=10 and you have an 10-element array of 0's (using .0 as length is common). Any command it doesn't recognize is passed to the outside environment, which might be a command-line shell, another program you're scripting, whatever, no special syntax needed. REXX is preposterously fast for a scripting language, and has perfect decimal math (Cowlishaw's main academic area of interest), though it's still not compiled language fast.

OS/2 made great use of it. The E and E/PM editors written in REXX are all about scripting; like Emacs but far more so, there's almost no program except the E script.

Back in the '90s, I wrote a ton of programs for clients (all dreadfully dull line-of-business stuff) which were thousands of lines of REXX with VT100 UI or a little teeny GUI shell in C it controlled and got messages from; my turnaround on a business change could be hours instead of days or weeks for a C-only program. There's somewhere on FTP sites or in my old code archive my QUEST game, which was just a random D&D wilderness crawl hack-and-slash, written the same way, and a huge script for writing a nice config file to generate the impossibly fussy bit-twiddling world files for DikuMUD and CircleMUD.

Down side, by the late '90s OS/2 was shut down, Mike Cowlishaw started doing everything in NetRexx, so the core language languished, and there weren't good ports to Linux or Mac (either System 9 or early OS X). The open-source "Regina" interpreter was almost but not quite compatible, and was very frustrating to debug around. I completely gave up on it around 1999, when Python got usable.

Seems in the last decade they've somewhat recovered, REXX Language Association, aka ooRexx.org, and NetRexx all have maintained sites, the old RexxInfo site is as hideous as it was in the '90s, and unmaintained now, but does have a free copy of the REXX Programmer's Reference book which is nicely complete.

The RexxLA symposium is going on as we speak. No livestreams, sadly, but last year's presentation decks are up. You might want to start with Rexx Tutorial for Beginners part 1, part 2, and ooRexx Tutorial.

I'd really like to hear "Rexx from OS/2 to macOS - a travel in time & space"!

% sudo port install oorexx
…
% cat hello.rexx
/** hello.rexx */

name = input("What is your name? ")
say "Hello," reverse(name) || "!"
exit

input: procedure
    parse arg prompt
    call charout ,prompt
    return linein()

% rexx hello.rexx
What is your name? Mark
Hello, kraM!

Works fine. The default I/O commands are a little weird so I rewrote that input function almost from muscle memory. PULL var and PUSH x operate on a stack and get uppercased, PARSE PULL var is very powerful and lets you interpolate strings for parsing input (you can also use it on function args, as seen here), but here all I want is a complete line as a function. SAY is a cutesy form of CALL LINEOUT ,whatever, (the missing first arg to CHAROUT or LINEOUT is the output file name/handle). Note the two ways to call a function: y=f(x) or CALL f x depending on if it has a return value. I write everything lowercase, modern REXX is case-insensitive, but historically it was an uppercase-only language, so I always SHOUT THE KEYWORDS when talking about them.

I didn't get far into ooRexx back in the day; I don't think most of its "enhancements" are needed to a nice simple language, but there's places where stems aren't a sufficient abstract data structure and the like.

This looks like fun to play with again, though I wouldn't expect to ship anything useful in it. The one problem is there's no REPL. You can actually make a simple REPL with just DO FOREVER; PARSE PULL line; INTERPRET line; END but there's a lot of error-handling and structured stuff that won't work in that. I think I have a real REPL in my code? I'll go looking in a bit.