Interactive Python IDLE

When using Python from a shell, the REPL is fairly awful and doesn't let you copy-paste or save, except through the shell itself. I especially find that copy-pasting functions in is error-prone. There's a nice interactive environment for Python called IDLE (after Eric ): It's probably an application in your /Applications/Python 3.7 folder, or whatever other OS's do, or can be run with IDLE3 from the shell. Other than using ^N/^P for next/previous history line, it works pretty much exactly as you'd expect a GUI REPL to work, and lets you save your session as a text file, easy to extract some functions from later or use as a doctest.

Trouble is, IDLE doesn't automatically pick up the ~/.pystartup script; I had to remember to call it with IDLE3 -s, and there's no easy way to do that from the desktop, where I'm often lazily clicking. This has been frustrating me very slightly for years.

So: open Automator, new Document, add Utilities/Run Shell Script, and paste in:

IDLE3 -s

Save as an Application, name it IdleStart. The icon's ugly and generic, so I made a half-assed icon, copy it from an image editor or Preview, and paste into the icon in Get Info on the application.

python-foot

Now I have a nice stupid foot to click on, and get a proper REPL. The running program has a hideous 16x16 or 32x32 icon scaled up, which I don't think I can easily solve; I looked at the idlelib source and while I could patch it, it's not easily configured. Maybe later. There's also no way to specify the window location, which I'd like to have mid-left of my screen, but again, maybe later.

While I'm at it, the themes are garish, and the customizer is unpleasant. So I just edited this in ~/.idlerc/config-highlight.cfg, then picked Mark's Dark theme:

[Mark's Dark]
normal-foreground = 
normal-background = #111111
keyword-foreground = 
keyword-background = #111111
builtin-foreground = 
builtin-background = #111111
comment-foreground = 
comment-background = #111111
string-foreground = 
string-background = #111111
definition-foreground = 
definition-background = #111111
hilite-foreground = 
hilite-background = #808080
break-foreground = 
break-background = #808000
hit-foreground = #002040
hit-background = 
error-foreground = 
error-background = 
cursor-foreground = 
stdout-foreground = 
stdout-background = #111111
stderr-foreground = 
stderr-background = #111111
console-foreground = 
console-background = #111111
context-foreground = 
context-background = #444444

My current ~/.pystartup is short (in python2 it was much longer, since nothing worked right), just some common imports and functions I use enough to miss:

import os, sys
import math
import random as R
import re
import time
import turtle as T

def dice(n, s):
    t = 0
    for i in range(n):
        t += R.randint(1, s)
    return t

def roundup(n):
    return math.floor(n+0.5)

def sign(i):
    if i > 0: return 1
    elif i == 0: return 0
    return -1

print("READY")

Now if I click the foot and see "READY" above >>>, it's all working.

Writing a Doorgame

So, how did I write this? Mystic BBS has a Pascal-based scripting, but I didn't really like using that; it's very powerful, but a pain in the ass to code/compile/test on, and it's just different enough from normal FreePascal to annoy me. And then tried to use Mystic's "integrated Python", which just doesn't work; I tried everything to make it find Python. And it's 2.7, so bleh. BUT: Adding a doorgame with the menu system lets you specify any executable, including a Python 3.7 script, and pass in a few command-line arguments about user and directories, so that's what I did (though the server I'm using only has Python 3.6, which hasn't been a problem yet, but be aware)…

The BBS handles all the redirection from network to stdin/stdout, otherwise I could use inetd for the same thing, but then I'd have to deal with all the login and networking state. Doing this as a door simplifies programming back to 1970s level!

I'll eventually open source it, but while I'm iterating and early on, I want to keep some of my secrets.

ANSI Color

There's ANSI color everywhere, of course. curses is fucking awful and archaic, and built to handle many incompatible terminals, which isn't a problem anymore. Mystic uses the same trick I'm using, and many others back in the day: Strings have escape codes |XX in them. Mystic uses numbers for colors, which is awful, and has hundreds of BBS-specific strings, which I have no need of. So all I did is this:

# set to 1200, 9600, etc. to simulate modem transmission rate
kModemSpeed = 0
kLinesPerScreen = 24

kAnsiEsc = "\x1b["

#      0   1   2   3   4   5   6   7   8   9   10
kAnsiBoxLine =  ["─",   "│",    "┌",    "┬",    "┐",    "├",    "┼",    "┤",    "└",    "┴",    "┘",]
kAnsiBoxDouble =["═",   "║",    "╔",    "╦",    "╗",    "╠",    "╬",    "╣",    "╚",    "╩",    "╝",]
kAnsiBoxHeavy = ["━",   "┃",    "┏",    "┳",    "┓",    "┣",    "╋",    "┫",    "┗",    "┻",    "┛",]

# ansiFilter, called by write/writeln, replaces |XX with escape codes:
kAnsiEscapeCode = {
    "CD": kAnsiEsc+"J",         # Clear Down
    "CL": kAnsiEsc+"2K",            # Clear Line
    "CS": kAnsiEsc+"2J"+kAnsiEsc+"H",   # Clear Screen
    "DI": kAnsiEsc+"2m",            # Dim
    "ES": "\x1b",               # Escape ASCII 27
    "GG": "\x07",               # Bell ASCII 7
    "HR": kAnsiBoxDouble[0] * 79,       # Horizontal Ruler
    "LI": kAnsiEsc+"1m",            # Light/Bright
    "PI": "|",              # Pipe char |
    "RE": kAnsiEsc+"0m",            # Reset Attributes
    "UN": kAnsiEsc+"4m",            # Underscore

    "FK": kAnsiEsc+"30m",           # Foreground Black
    "FR": kAnsiEsc+"31m",           # Foreground Red
    "FG": kAnsiEsc+"32m",           # Foreground Green
    "FY": kAnsiEsc+"33m",           # Foreground Yellow
    "FB": kAnsiEsc+"34m",           # Foreground Blue
    "FM": kAnsiEsc+"35m",           # Foreground Magenta
    "FC": kAnsiEsc+"36m",           # Foreground Cyan
    "FW": kAnsiEsc+"37m",           # Foreground White

    "LK": kAnsiEsc+"1;30m",         # Foreground Light Black
    "LR": kAnsiEsc+"1;31m",         # Foreground Light Red
    "LG": kAnsiEsc+"1;32m",         # Foreground Light Green
    "LY": kAnsiEsc+"1;33m",         # Foreground Light Yellow
    "LB": kAnsiEsc+"1;34m",         # Foreground Light Blue
    "LM": kAnsiEsc+"1;35m",         # Foreground Light Magenta
    "LC": kAnsiEsc+"1;36m",         # Foreground Light Cyan
    "LW": kAnsiEsc+"1;37m",         # Foreground Light White

    "BK": kAnsiEsc+"40m",           # Background Black
    "BR": kAnsiEsc+"41m",           # Background Red
    "BG": kAnsiEsc+"42m",           # Background Green
    "BY": kAnsiEsc+"43m",           # Background Yellow
    "BB": kAnsiEsc+"44m",           # Background Blue
    "BM": kAnsiEsc+"45m",           # Background Magenta
    "BC": kAnsiEsc+"46m",           # Background Cyan
    "BW": kAnsiEsc+"47m",           # Background White
}

def ansiFilter(s):
    """Replaces |XX codes with values defined in `kAnsiEscapeCode`."""
    out = ""
    slen = len(s)
    i = 0
    while i < slen:
        c = s[i]
        i += 1
        if c == "|":
            esc = s[i:i+2]
            i += 2
            out += kAnsiEscapeCode.get(esc, esc)
        else:
            out += c
    return out

def ansiGoto(pt):
    return "%s%s;%sH" % (kAnsiEsc, pt[1], pt[0])

def readln(prompt):
    """Calls input after filtering prompt."""
    return input(ansiFilter(prompt))

def write(s, end=None):
    """Filters `s` and writes to stdout, writes `end` if needed, flushes stdout."""
    if type(s) != str:
        s = str(s)
    lines = 0
    out = ansiFilter(s)
    for c in out:
        sys.stdout.write(c)
        if kModemSpeed:
            time.sleep(1.0/kModemSpeed)
        if c == '\n':
            lines += 1
        if lines >= kLinesPerScreen:
            readln("|RE|FC[MORE]|RE")
            lines = 0
    if end:
        sys.stdout.write(end)
    sys.stdout.flush()

def writeln(s):
    """Calls write with `end='\n'`."""
    write(s, end='\n')

Now if I want someone to talk in bold cyan (which I chose as my standard NPC color), I just
writeln("|RE|LC%s says, \"Mellow greetings, sir!\"|RE" % (npc["name"],))
RE before and after text is important, resetting colors from previous strings.

The one problem is I can't use | in ASCII art, so to generate my big character logos, I write a script mysticFiglet.zsh, making : the vertical char:

figlet -f small "$*"|sed "-es/|/:/g"

Flow Control

The great part about a doorgame is, no event loop. Oh, there's a REPL of sorts:

    def run(self):
        self.running = True
        try:
            : Python 3.8: while line := self.prompt():
            while self.running:
                line = self.prompt()
                if line == None:
                    break
                words = line.split()
                if len(words) == 0:
                    continue
                self.parseWords(words)
            self.save()
            self.doHelp("?", "_outro")
            return 0
        except (EOFError, KeyboardInterrupt):
            self.save()
            return 0
        except Exception:
            writeln("|GG|RE|BR|FWERROR! Something went wrong.|RE")
            logging.error("", exc_info=True)
            self.save()
            return 1

It just blocks at input (called by readln, called by prompt), and continues on until self.running = False, or an exception is thrown. Whee! The usual game loop makes it impossible to get stateful input without leaving giant memos of current state, so the event loop can keep cycling at 60fps.

Data Files

One solution would be to just make everything in classes, and use Python's pickle to archive everything. But the problem is that's pure binary, you can't easily hack on it or see what's happened, and on load you can't change any values. So instead I only use primitive types and archive everything to and from JSON. Floor maps and Players have a toData() method which produces a JSON-compatible data structure, and their initializers take a data structure and pull out or translate the values they need. It's a little more work, but means I can debug faster and add new fields faster.

The floors use a super-simple data file format, with an ASCII roguelike map and a series of command lines to create NPCs, traps, text, etc. The weapons and armor are listed in an even simpler format, with space-delimited fields.

Both the command lines and item formats are just run thru a tokenizer I wrote, that reads words, "strings", numbers, and [lists]:

type    name        cost    stat    bonus   damage  inflicts    twoh
wpn "Club"      5   Str 0   3   [blunt]     0

becomes:

{'type': 'wpn', 'name': 'Club', 'cost': 5, 'bonus': 0, 'damage': 3, 'inflicts': ['blunt'], 'twoh': 0}

Python 3.7

  • Python 3.7 released: Standard checklist:
    • Run installer
    • Delete the old 3.6 folder from /Applications
    • Run the certificate command in the new 3.7 folder (the other shits a PATH into my shell profile, don't need it)
    • Run IDLE and verify it's 3.7.0. Happily, no longer have to fight with updating Tcl/Tk.
    • Run "python3" from Terminal and verify it's 3.7.0
    • Run a random Python script to make sure nothing's broken.

Nanosecond-accurate time functions and switching more ASCII/C Locale into UTF-8 are nice improvements, but those are more patching up legacy annoyances than "must have".

I'm mostly interested in dataclasses, which makes it much easier to build little struct-type objects instead of random dicts or lists which have all sorts of problems (no equality, hashing, typo-safety).

I greatly dislike the addition of BDSM typing, but it's mostly optional, EXCEPT you have to use them in dataclasses:

from dataclasses import dataclass
@dataclass
class Point:
    x : float = 0.0
    y : float = 0.0

>>> p = Point()
>>> p
Point(x=0.0, y=0.0)
>>> q = Point(1.1, 2.2)
>>> q
Point(x=1.1, y=2.2)

If I define Point without the type annoytations[my new favorite typo!], only the default constructor works, and it doesn't print the fields as a string.

@dataclass
class Pointless:
    x = 0.0
    y = 0.0

>>> f = Pointless()
>>> f
Pointless()
>>> f.x
0.0
>>> f.y
0.0

Real examples might be a lot more complex than a point, and by then the cost of building a proper class with __init__ and everything yourself isn't such a big deal, so I can see dataclasses mostly being used for very simple struct-like containers.