Building Command Lines in zsh

I love zsh, but sometimes it's a tough love. Building command lines in strings is very hard because zsh doesn't want you hurting yourself on argument splitting like bash, so I just banged my head on that for like 15 minutes before I quit being dumb. Building them in arrays is super trivial:

% a=(args foo bar)
% b=$(ls)
% a=($a $b) # expands $a, adds $b to the end as a single argument
% $a
1=foo
2=bar
3=file1
file2

Note that arg 3 is multiple words/lines, but one arg. That's a nightmare to express in bash, but makes perfect sense at the exec level used by "real" (non-shell) programming languages.

args is a very useful debugging script I've had in some form since the '80s:

#!/bin/zsh
i=1
while [[ $# -gt 0 ]]; do
    echo "$i=$1"
    let i=i+1
    shift
done

A Scheme of Gerbils Runnin' in a Wheel

Chicken's mediocre performance is causing me problems, so I'm taking yet another language tour/spike.

Looking at Gerbil Scheme, as a faster Scheme and as a bonus it's closer to R7RS. It has far less library support than Chicken, but does have a FFI, and compiles to a native binary through C, so I really don't need too many libraries.

Instead of the #!key #!optional #!rest keywords used in Chicken and some others, Gerbil uses a literal template style for functions:

;; Chicken: (define (foo a #!key (b 'bar) #!optional (c "see?") #!rest rest)
;; Gerbil:
(def (foo a b: (b 'bar) (c "see?") . rest)
    (print "a=" a " b=" b " c=" c " rest=" rest "\n")
)
(foo "Mark" 1 b: "bees!" 'what) ;; note out-of-order optional & keyword
;; a=Mark b=bees! c=1 rest=what

I like this style better, but I dislike the "def" keyword (which uses the enhanced lambda that does this) instead of "define" which uses "lambda%" (traditional lambda).

Gerbil uses [] for list construction, and {} for method dispatch, so goodbye to the nicely distinguishable braces I was using in Chicken. The spec says implementations can do what they want with those symbols, but I wish they wouldn't. Ah, well. I'll add a shitload more whitespace and some comments and it'll be fine.

The struct/object system in Gerbil is pretty nice, they're slightly upgraded records, but there's an (@ obj field) syntax instead of (MyClass-field obj), and (defmethod {mymethod MyClass} ...) creates a polymorphic {mymethod obj args} which finds the right method for any object, which is especially useful for inheritance.

I tried doing some VT100 graphics just to proof-of-concept, but it's doing something to the terminal, such that the same escape codes which work in Python don't clear the screen fully in Gerbil. After a short losing battle with stty and termcap, I give up on that and I'll jump right to writing a C FFI to SDL, because in 2019 that's easier than writing to a console.

Daily reminder that everything we have made since 1984 is overcomplicated junk. On an Atari 800, this took a few seconds to type, and you could start coding a nice UI instantly:

atari-is-awesome-graphics1

Alas, we live in a fallen world, so this is going to be trouble. Here's my Gerbil FFI template so far:

package: myffi

(import
    :std/foreign
)

(export )

(begin-ffi
    ;; names of all Scheme wrappers to expose
    (chello)

(c-declare #<<CDECLEND

 <stdio.h>

int chello(void) {
    printf("Hello this is C!\n");
    return 1;
}

CDECLEND
)

    (define-c-lambda chello () int "chello")

) ;; begin-ffi


; TODO for constants: (define FOO ((c-lambda () int "___result = FOO;")))

(define (main . args)
    (chello)
)

Gerbil has a Scheme-based build system, but I'm a caveman so I make another build.zsh:

#!/bin/zsh

function usage {
    echo "Usage: build.zsh MAIN.scm [LIBS] || -?"
    exit 1
}

if [[ $# -eq 0 || "$1" == "-?" || "$1" == "--help" ]]; then
    usage
fi

mkdir -p bin

main=`basename $1 .scm`

gxc -exe -static -O -o bin/$main "$@" || exit 1
echo "Built bin/$main"

Now:

% ./build.zsh myffi.scm
Built bin/myffi
% bin/myffi
Hello this is C!

Hooray! Unconditional success! Only took all afternoon and a half-pot of coffee!

Now I "merely" have to wrap all of SDL (well, just the parts I need) and get linking working and oh sweet merciless Cthulhu. And I still won't know how much this'll help my performance until I'm days into it. But, the first step is the hardest.

REXX Monarchial Madness is Hereditary

So, I was trying to figure out how to manage global vs. local state, and made the dumbest program possible:

/** array.rexx */

a. = 0
a.0 = 10
DO i = 1 TO a.0; a.i = i; END

SAY array_string("a.") -- prints 1 to 10
SAY alen -- prints "ALEN" because it doesn't exist in global scope

EXIT 0

/* Returns string representation of an array, stem. passed as first arg. */
array_string:
    PARSE ARG _arrayvar .
RETURN _array_string()

_array_string: PROCEDURE EXPOSE (_arrayvar)
    alen = VALUE(_arrayvar || 0)
    str = ""
    DO i = 1 TO alen
        IF i > 1 THEN str = str || ", "
        str = str || VALUE(_arrayvar || i)
    END
RETURN str

So… I've made a global variable "a." full of numbers. PROCEDURE makes a local variable context until you hit RETURN; you should never work at global scope if you can avoid it. So to pass a. in, I have to bounce through a global function array_string(), set a global var _arrayvar to whatever arg was given, which calls a local function _array_string and the paren in EXPOSE (_arrayvar) expands that into "a.", and then use VALUE() to dynamically read the variable.

Well, it works. I remember now how crazy complex REXX programs got, with giant global databases being exposed in every function. I bet this is the origin of my "World object" pattern in so many later languages.

This and another dumb example or two ate a couple hours of my brain, but it kinda makes sense again, you know? Interesting for a language I haven't written in anger in 20 years. I went fully back to SHOUTING KEYWORDS, it just makes sense.

The Python version of this is:

#!/usr/bin/env python3

def array_string(arrayvar):
    return ", ".join(map(str, arrayvar))

a = []
for i in range(1, 11): a.append(i)
print(array_string(a))

Which I'd argue is just as stupid, because Python thinks ranges end 1 before the last number, doesn't know how to join numbers with strings, on a large array that map will consume all memory, and is a single-pass interpreter so I have to put the function above the script code; but it is much, much shorter.

REXX is Still the King

Random REXX mention on Fediverse reminded me of good times with it. REXX was originally a scripting language for the IBM /360 series, later OS/2 where I encountered it. There's three main branches: REXX (original, simple procedural language), ooRexx (strict superset adding OOP tools to REXX), and NetRexx (almost-REXX implemented in Java, so it can interact with the entire Java API).

The things it does well are: Simple notation, like BASIC, but it's structured and dynamic: Variables have their NAME as default value, and it's trivial to evaluate strings as code. Variables can also be "stems", either indexed or associative arrays depending on key. a.=0; a.0=10 and you have an 10-element array of 0's (using .0 as length is common). Any command it doesn't recognize is passed to the outside environment, which might be a command-line shell, another program you're scripting, whatever, no special syntax needed. REXX is preposterously fast for a scripting language, and has perfect decimal math (Cowlishaw's main academic area of interest), though it's still not compiled language fast.

OS/2 made great use of it. The E and E/PM editors written in REXX are all about scripting; like Emacs but far more so, there's almost no program except the E script.

Back in the '90s, I wrote a ton of programs for clients (all dreadfully dull line-of-business stuff) which were thousands of lines of REXX with VT100 UI or a little teeny GUI shell in C it controlled and got messages from; my turnaround on a business change could be hours instead of days or weeks for a C-only program. There's somewhere on FTP sites or in my old code archive my QUEST game, which was just a random D&D wilderness crawl hack-and-slash, written the same way, and a huge script for writing a nice config file to generate the impossibly fussy bit-twiddling world files for DikuMUD and CircleMUD.

Down side, by the late '90s OS/2 was shut down, Mike Cowlishaw started doing everything in NetRexx, so the core language languished, and there weren't good ports to Linux or Mac (either System 9 or early OS X). The open-source "Regina" interpreter was almost but not quite compatible, and was very frustrating to debug around. I completely gave up on it around 1999, when Python got usable.

Seems in the last decade they've somewhat recovered, REXX Language Association, aka ooRexx.org, and NetRexx all have maintained sites, the old RexxInfo site is as hideous as it was in the '90s, and unmaintained now, but does have a free copy of the REXX Programmer's Reference book which is nicely complete.

The RexxLA symposium is going on as we speak. No livestreams, sadly, but last year's presentation decks are up. You might want to start with Rexx Tutorial for Beginners part 1, part 2, and ooRexx Tutorial.

I'd really like to hear "Rexx from OS/2 to macOS - a travel in time & space"!

% sudo port install oorexx
…
% cat hello.rexx
/** hello.rexx */

name = input("What is your name? ")
say "Hello," reverse(name) || "!"
exit

input: procedure
    parse arg prompt
    call charout ,prompt
    return linein()

% rexx hello.rexx
What is your name? Mark
Hello, kraM!

Works fine. The default I/O commands are a little weird so I rewrote that input function almost from muscle memory. PULL var and PUSH x operate on a stack and get uppercased, PARSE PULL var is very powerful and lets you interpolate strings for parsing input (you can also use it on function args, as seen here), but here all I want is a complete line as a function. SAY is a cutesy form of CALL LINEOUT ,whatever, (the missing first arg to CHAROUT or LINEOUT is the output file name/handle). Note the two ways to call a function: y=f(x) or CALL f x depending on if it has a return value. I write everything lowercase, modern REXX is case-insensitive, but historically it was an uppercase-only language, so I always SHOUT THE KEYWORDS when talking about them.

I didn't get far into ooRexx back in the day; I don't think most of its "enhancements" are needed to a nice simple language, but there's places where stems aren't a sufficient abstract data structure and the like.

This looks like fun to play with again, though I wouldn't expect to ship anything useful in it. The one problem is there's no REPL. You can actually make a simple REPL with just DO FOREVER; PARSE PULL line; INTERPRET line; END but there's a lot of error-handling and structured stuff that won't work in that. I think I have a real REPL in my code? I'll go looking in a bit.

Modern C 2.0

Fantastic book about C as actually used in modern code-bases. Interesting structure of levels 0-3, each of which covers a cross-section of topics at an understanding level. If you know everything in the summary already, you can just skip ahead to the parts you do want detail on.

If the Benefits of Software are So Great…

"When we start cataloging the gains in tools sitting on a computer, the benefits of software are amazing. But, if the benefits of software are so great, why do we worry about making it easier—don’t the ends pay for the means? We worry because making such software is extraordinarily hard and almost no one can do it—the detail is exhausting, the creativity required is extreme, the hours of failure upon failure requiring patience and persistence would tax anyone claiming to be sane. Yet we require that people with such characteristics be found and employed and employed cheaply."
—Richard P. Gabriel, Patterns of Software

Design Patterns

It is sometimes suggested by well-meaning language enthusiasts that "My language is complete and powerful, so design patterns don't apply here!" Sadly, they are incorrect.

Design patterns happen in every language. The "Gang of Four" Design Patterns book just collected the ones observed in Smalltalk, and ported them to C++, later rewrites to Java, etc. These are not recipes to blindly follow, but examples meant to show you how to find and regularize the ones in your code.

It's somewhat difficult to see them unless you've read Christopher Alexander's books, and written a lot of programs in some language, and specifically looked for the places where you repeat a structure for livability's sake. Just as it's hard for an architect to make a path where people will want it, unless they first observe how people live and get around that space, and then convert the ad-hoc trails people follow into paths.

Smalltalk is an extremely expressive language (it failed in the market because every ST program is IDE-specific), it has closures, allows you to very trivially make new control structures; it doesn't need a hack like macros because the entire language is that freeform. And this is where the GoF authors observed these paths being made by themselves and other developers, not just in limited BDSM languages like Java.

So, a little light reading:

Tower of Babble

Programmers almost compulsively make new languages; within just a few years of there being computers, multiple competing languages appeared:

It proliferated from there into millions; probably half of all programmers with 10+ years of experience have written one or more.

I've written several, as scripting systems or toys. I really liked my Minimal script in Hephaestus 1.0, which was like BASIC+LISP, but implemented as it was in Java the performance was shitty and I had better options to replace it. My XML game schemas in GameScroll and Aiee! were half programmer humor, but very usable if you had a good XML editor. Multiple apps have shipped with my tiny lisp interpreter Aspic, despite the fruit company's ban on such things at the time. A Brainfuck/FORTH-like Stream, working-but-incomplete tbasic, and a couple PILOT variants (I think PILOT is hilariously on the border of "almost useful").

Almost every new language is invented as marketing bullshit based on a few Ur-languages:

  • C++: Swift
  • Java: Javascript (sorta), C#, Go
  • Awk: Perl, Python, PHP, Julia
  • C: Rust
  • Smalltalk: Objective-C
  • Prolog: Erlang, Elixir
  • ALGOL: C, Pascal, PL/1, Simula, Smalltalk, Java
  • LISP: Scheme, ML, Haskell, Clojure, Racket
  • BASIC: None, other than more dialects of BASIC.
  • FORTRAN: None in decades, but is the direct ancestor of ALGOL & BASIC.
  • COBOL: None in decades.

A few of these improve on their ancestors in some useful way, often performance is better, but most do nothing new; it's plausible that ALGOL 68 is a better language than any of its descendants, it just has mediocre compiler support these days.

Certainly I've made it clear I think Swift is a major regression, less capable, stable, fast, or even readable than C++, a feat I would've called impossible except as a practical joke a decade ago. When Marzipan comes out, I'll be able to rebuild all my 15 years of Objective-C code and it'll work on 2 platforms. The Swift 1.0 app I wrote and painfully ported to 2.0 is dead as a doornail, and current Swift apps will be uncompilable in 1-2 years; and be lost when Apple abandons Swift.

When I want to move my Scheme code to a new version or any other Scheme, it's pretty simple, I made only a handful of changes other than library importing from MIT Scheme to Chez to Chicken 4 to Chicken 5. When I tested it in Racket (which I won't be using) I had to make a handful of aliases. Probably even CLISP (which is the Swift of LISPs, except it fossilized in 1994) would be 20 or 30 aliases; their broken do iterator would be hard but the rest is just naming.

Javascript is a pernicious Herpes-virus-like infection of browsers and desktops, and nothing can ever kill it, so where it fits the problem, there's no reason not to use it. But there's a lot it doesn't do well.

I was leery of using FreePascal because it has a single implementation (technically Delphi still exists, but it's $X,000 per seat on Windows) and minimal libraries, and in fact when it broke on OS X Mojave, I was disappointed but I-told-you-so.

I'm not saying we should quit making new Brainfuck and LOLCODE things, I don't think it's possible for programmers to stop without radical brain surgery. But when you're evaluating a language for a real-world problem, try moving backwards until you find the oldest and most stable thing that works and will continue to work, not piling more crap into a rickety new framework.

The Biblical reference in the title amuses me, because we know now that it requires no malevolent genocidal war deity scared of us invading Heaven to magically confuse our languages and make us work at cross purposes; anyone who can write and think splinters their thought into a unique language and then argues about it.

Lost Treasure

In 1979, I learned to program in BASIC on a TRS-80 Model I. Sometime in the next year, I read one of my first programming books:

I played Monster Chase and Lost Treasure, modified them extensively, and combined them, so the cave on the island had a monster chase to reach the exit. I recall having problems getting Starship Alpha and Devil's Dungeon to work, but they joined my software library eventually.

One of my earliest and happiest programming memories was sitting at the dining room table, reading Monster Chase, and writing out a smarter movement system and obstacles in a notebook; at the time the only computers were at school, so I wrote code on paper and typed them in later.

So when I found the book again on archive.org last night, I was very excited, and had to reimplement it. I actually typed this into Pythonista on my phone with the PDF open on an iPad, only moved it to the computer to do some final cleanup and upload it.

The book suggests some modifications, and I did some minor ones: Lowered the movement error to 10%, and risk of shark attack to 10%, rising by 1.5x rather than a flat +50% each time; being anywhere near the island edge killed you too often in the original. I also don't move you out of the water automatically, that should cost a turn.

I realized in converting it that I hate, hate, hate Row,Column coordinates instead of Cartesian X,Y; tons of mainframe-era computing resources used Row,Column, and you can still see it in some APIs like Curses. Note that the original program is 74 lines, mine's 214; BASIC is a terrible language, but it's terse.

I could adapt this into another doorgame for my Mystic Dungeon BBS, but I'm not sure what the multiplayer aspect would be, and it has limited replayability without doing some randomization.

Return of the Objective-C Jedi

[[[ These ]]] are your father's square brackets, the weapons of a Jedi Knight.
Not as clumsy or random as C++.
Elegant weapons for a more civilized age.

What's Different in Mulle-ObjC

This is like Objective-C circa 2010(?), good but not fully baked. Far better than circa 1986-2009, when it was a very thin translation layer over C.

  • No ARC (Automatic Reference Counting). This is just invisible sugar to hide retain/release/autorelease, and while ARC's convenient, it's trivial if you actually know how reference counting works. Don't really miss it.
  • No dot property syntax. [[myObj name] length] instead of myObj.name.length, and [myObj setName:newName] instead of myObj.name = newName. I can live with it, but I really did like dot syntax, even if it does "overload" the . operator and hide the distinction between methods and variables.
    • When dot syntax came out, Objective-C nerds came close to fistfights over this. You would not believe the venom some people had for it. Most of those nerds died or quit or got old & tired before fucking Swift came around, I guess.
  • No array syntax. [myList objectAtIndex:i] instead of myList[i]. This is a pain in the ass, I'll have to write some shorthand macros (or rather, go dig them out of my very oldest code).
  • No blocks. This one hurts, but it's a reasonable pick-your-battles decision. Classic: Write a method, dispatch to it, and call back success somehow. Blocks: create a weakSelf reference, enclose it, search-replace self in your block, pick one of a half-dozen complex GCD methods, get a memory leak because you retained something across the block boundary. This is annoying but logically simpler:
    [self performSelectorInBackground:@selector(computeData) withObject:inputData];
    
    - (void)computeData:(id)inputData {
        // create outputData
        [self setOutputData:outputData];
        [[NSNotificationCenter defaultCenter] postNotification:NOTI_DataComputed];
    }
    
  • Has object literals: @42 and @(var) create an NSNumber, @[] creates an NSArray, @{} creates an NSDictionary; dicts use key:value order, not the reverse order used in -[NSDictionary dictionaryWithObjectsAndKeys:], and array and dicts don't need a trailing nil, which was a constant source of mystifying bugs back in the day. Big win!
    • Hmn, crashes if you do something janky like [@[] mutableCopy]: mulle_objc_universe 0x1006adef0 fatal: unknown method 5e1b0403 "-getObjects:range:" in class 7aa0d636 "_MulleObjCEmptyArray"
  • Has for (id x in container) loops, using NSFastEnumeration. The 1.0 process of looping enumerations was awful, so this is very nice.
  • Huh, does have @autoreleasepool, so maybe I should use that instead of NSAutoreleasePool like a caveman? It compiles and seems to work.
  • Properties have properties assign/retain nonatomic/atomic nonnullable readonly, default is assign nonatomic, no "nullable" or "readwrite" flags needed. As it should be.
  • Weird isa define instead of pointer: blog post

TODO

  • I haven't set up an NSRunLoop or the equivalent of NSApplication (which is in AppKit, not Foundation), need to do that and then I'll have a working app template.