Julia String Concatenation

Last time, I was uncertain about string concatenation, so I did a test:

#!/usr/bin/env julia

const kTestText = "abcdefghijklmnopqrstuvwxyz0123456789\n"
const kLoops = 10000

function stringString()
    s = ""
    for i in 1:kLoops
        s = "$s$kTestText"
    return s

function bufferString()
    sb = IOBuffer()
    for i in 1:kLoops
        print(sb, kTestText)
    return String(take!(sb))

function vectorString()
    sb = Vector()
    for i in 1:kLoops
        push!(sb, kTestText)
    return join(sb, "")

function typedVectorString()
    sb = Vector{AbstractString}()
    for i in 1:kLoops
        push!(sb, kTestText)
    return join(sb, "")

println("*** stringString")
@timev stringString()

println("\n*** bufferString")
@timev bufferString()

println("\n*** vectorString")
@timev vectorString()

println("\n*** typedVectorString")
@timev typedVectorString()

*** stringString
  1.100197 seconds (21.24 k allocations: 1.725 GiB, 15.94% gc time)
elapsed time (ns): 1100197167
gc time (ns):      175349222
bytes allocated:   1851904041
pool allocs:       11292
non-pool GC allocs:9950
GC pauses:         79

*** bufferString
  0.006864 seconds (11.80 k allocations: 1.134 MiB)
elapsed time (ns): 6864042
bytes allocated:   1189493
pool allocs:       11794
non-pool GC allocs:3
realloc() calls:   8

*** vectorString
  0.017380 seconds (26.68 k allocations: 2.191 MiB)
elapsed time (ns): 17380237
bytes allocated:   2297091
pool allocs:       26659
non-pool GC allocs:9
realloc() calls:   8

*** typedVectorString
  0.031384 seconds (44.75 k allocations: 2.999 MiB, 10.00% gc time)
elapsed time (ns): 31384383
gc time (ns):      3137654
bytes allocated:   3144221
pool allocs:       44730
non-pool GC allocs:8
realloc() calls:   8
GC pauses:         1

Well, there's me told off. I expected #1 typedVector, #2 vector, #3 buffer, then stringString way at the bottom. Instead the first 3 are reversed.

IOBuffer, as ugly as it is, is the clear winner. Vector did OK, but twice as much CPU & RAM loses. Amusing that typedVector is twice as slow and memory-heavy as the untyped (explained ). On larger loops, buffer gets slower, but vector remains a memory pig, and in GC that's unacceptable. Of course stringString is terrible, and it's almost exactly the same for string(s, kTestText).

Time to rewrite some text processing.

More Julia

Decided to take another day on Julia, write something more serious and see how that goes.

There's an uber-juno "IDE" plugin for Atom, which at least turns on syntax highlighting and puts an interactive console in the editor. Yay. It's not capable of linting yet, though it says it is.

So I'm rewriting a simple 1970s-style dungeon crawl game (Chorus:"As if you could make any other kind of game, Mark!" Mark:"I assure you I could, I just choose not to.") as a test of data structures and application programming in Julia. It's a little challenging, but not impossible.

Using Modules

The current directory is not included in the default LOAD_PATH, so you can't import local modules right off. One solution is to put it in your startup:

% mkdir -p ~/.julia/config
% echo '@everywhere push!(LOAD_PATH, pwd())' >>~/.julia/config/startup.jl

All names are in the same namespace. This means your module and a struct or method in it can't have the same names… My current solution has been to pluralize the module, so GridMaps contains a struct GridMap.

The export rules are a little annoying. If you use the @enum macro to make a ton of constants, they aren't exported when you export the enum type; you can either manually export each constant name, or just use ModuleName.ConstantName in other modules. Bleh.


Bug #1: The terminator problem is pretty bad in Julia:

% cat Foo.jl
module Foo
for i=1:10
#missing end
end #module

% julia Foo.jl
ERROR: LoadError: syntax: incomplete: "module" at /Users/mdh/Code/CodeJulia/Foo.jl:1 requires end
 [1] include at ./boot.jl:317 [inlined]
 [2] include_relative(::Module, ::String) at ./loading.jl:1038
 [3] include(::Module, ::String) at ./sysimg.jl:29
 [4] exec_options(::Base.JLOptions) at ./client.jl:229
 [5] _start() at ./client.jl:421
in expression starting at /Users/mdh/Code/CodeJulia/Foo.jl:1

Good luck finding that missing end if you have a 1000-line module. C used to be just as bad about semicolons and braces, but modern compilers are pretty good at guessing where you fucked up. Python's whitespace-as-control is brilliant, because you can't ever do that. A passable solution would be each control keyword having its own unique end keyword, but it's too late for that. In the actual bug, I had to comment out half the code, run the module, uncomment and comment the other half, repeat until I isolated it.

Bug #2: Type annotations need to be very generic or left off entirely. As code-as-documentation, I declare a function as parseLine(line::String), and it gives me:

MethodError(Main.Foo.parseLine, ("a",), 0x00000000000061bb)

Well, thanks. Turns out I need to use AbstractString, because a prior function returns an AbstractString and not String. Or I can just leave the typing off, as was my first instinct.

String Concatenation

This is super ugly. Currently I've fallen back on:

sb = Vector()
push!(sb, "part of ")
push!(sb, "a string")
return join(sb, "")

Strings aren't mutable, and there's no StringBuffer/NSMutableString equivalent. The other option is to use an IOBuffer. I haven't done timings yet to see which is faster/uses less memory, I just find pushing to a vector simpler.


There's no switch statement. I could put functions in a dictionary and dispatch on that, which is not ideal for looping over a bunch of simple values, and requires passing around control flags instead of a simple break or return. Caveman solution is a chain of if/elseif, but I don't like it. Possibly a macro could be written?


Turns out you can make binaries: Julia apps on the App Store: Building and distributing an application written in Julia

It's an ugly process, that Nathan's working around, but it's a start. This should 100% be in the core libraries, and should have a cross-compiler.

Getting this working is tomorrow's problem, I think.


Interesting language, originally a math/statistics package but now as general-purpose as any lang. More or less Pythonic, though it has some type-annotation stuff, and heavily-optimized Julia looks like a mess of annotations with your code buried somewhere inside.

The Mac version comes as a dmg with an app (which I'd prefer for easy install/uninstall), or brew (which I prefer not to use). The app just launches a single command in a new Terminal window; add that path/bin to the PATH in your .profile, e.g.:

export JULIA_HOME="$HOME/Applications/Julia-1.0.app/Contents/Resources/julia"
export MANPATH="$MANPATH:$JULIA_HOME/share/man"
export PATH="$PATH:$JULIA_HOME/bin"

And now in Terminal:

% echo 'println("Hello, world!")' >hello.jl
% julia hello.jl
Hello, world!

The only way I can see to make it compile to a binary is embedding, and I'm not clear on how you package that with a full Julia distribution yet. That's unfortunate. I like REPL workflows as much as anyone, but binaries are what "normal" people run.

Getting Started

<voice tone="excessively chipper"> Let's read the manual! </voice>

Syntax is nicer than usual: function/end, if/elseif/else/end, for/end, while/end, begin/end, let/end, which beats the hell out of Python's def, if/elif/else; to say nothing of abominations like Swift's "func". No do/while loop, which is annoying especially for file processing, but I suspect that can be fixed with macros.

There's a lot of ways to write functions, which is nice but allows some ugly choices. Anonymous functions are x->x^2 or function(x) x^2 end; named functions can just be assigned f(x)=x^2 or written in full:

function f(x)
    return x^2

Whitespace is not significant, and indentation is not enforced, which is a major bummer for style-enforcing-structure, but I'm sure sloppy jerks will love that.

You can use tuples for multiple returns, or as ad-hoc structures:

> point(x, y) = (x=x, y=y)
point (generic function with 1 method)
> p = point(13, 2)
(x = 13, y = 2)
> a, b = p
(x = 13, y = 2)
> a

It's pass-by-reference, not copying, so be careful with mutable data.

My only real kvetch so far is that arrays are 1-indexed and column-major, like FORTRAN, not 0-indexed and row-major, like C. For a numeric package, that makes sense, but for other programming tasks it's frustrating and error-prone, see EWD 831.

This is a functional language, and there are no classes/inheritance/methods, however "methods" are functions which are overloaded based on types, and can be used like class methods:

> quack(x::Int64) = "int $x"
> quack(x::Float64) = "float $x"
> quack(1)
"int 1"
> quack(6.66)
"float 6.66"

As well, you can use closures to make pseudo-classes, the same way you do in Scheme:

let state = 0
    global counter() = (state += 1)
    global counterReset() = (state = 0)

struct (immutable) and mutable struct make "Composite types", and can make objects the usual way:

struct Point
pointDist(p::Point) = sqrt(p.x^2 + p.y^2)
Base.show(io::IO, p::Point) = print(io, "{$(p.x),$(p.y)}")

> p = Point(6, 6)
> pointDist(p)

The default constructor can be overridden at the end of the field list, it's defined as Point(x,y) = new(x,y). The "toString" equivalent there is ugly as hell, but there's a ton of options for overloading it by type of output.

There's a lot of fucking around with generics and strong typing (for weak minds), but ignore all that crap.

Interfaces are a somewhat messy use of several methods to create pseudo-types; define the basic interface methods for your type, and most things calling those interface methods will work. So, a couple iter() functions and you have an iterable, and so on. This would work much better if Julia had an actual OOP class system and real interfaces, but Python half-asses interfaces the same way and aside from being 1000x slower than you'd like, it gets by.

Quickly skimming modules, seems pretty standard import mechanism, but I don't see any way to make something private. OK, I'm bored of reading docs. Let's do something. Something semi-practical here, my standard RPN calculator, one command per line.

Docs/libraries are kind of a mess, Vector is discussed in Base.Arrays, push!/pop! methods are discussed in Collections (an interface). parse is under Numbers, not Strings, as one might expect.

eof() does the extremely unfortunate thing of blocking for input, so it's utterly useless in a main interactive loop.

… About 30 minutes later, I have a working, final version. Well, that was pretty easy, and it's a clean implementation, other than the interactive loop.

Next time I open this, I'll put it in a module, and tokenize the line instead of requiring just one token per line, and have some command-line argument to suppress help and prompts.

I should also investigate IJulia which is a Jupyter notebook, which seems like the "expected" way to make it interactive and handle graphics or media.


#!/usr/bin/env julia
# RPNCalc.jl
# Copyright ©2018 by Mark Damon Hughes. All Rights Reserved.

stack = Vector()

function checkStack(n)
    if length(stack) < n
        error("Stack underflow: Needs $n values")

function parseLine(s)
    s = strip(s)
    if s == "+"
        b = pop!(stack); a = pop!(stack)
        push!(stack, a + b)
    elseif s == "-"
        b = pop!(stack); a = pop!(stack)
        push!(stack, a - b)
    elseif s == "*"
        b = pop!(stack); a = pop!(stack)
        push!(stack, a * b)
    elseif s == "/"
        b = pop!(stack); a = pop!(stack)
        push!(stack, a / b)
    elseif s == "="
        push!(stack, parse(Float64, s) )

function main()
    println("RPN Calc: Type numbers or operators (+, -, *, /) one at a time, = to show top of the stack, ^D to end.")
    while true
        print("> "); flush(stdout)
        s = readline(stdin, keep=true)
        if s == ""
        catch e


100 Days of Programmer Burnout

This is a fantastic way to make someone hate computers and programming, and take up a life of farming and/or sex work so they never have to see a computer again. (Which, sometimes doesn't seem such a bad idea, anyone want an aging gigolo in between potato plantings?)

When I was trying to do the Advent of Code, I couldn't do half the days because life got in the way. 100 days would be homicide-inducing, not habit-forming.

Here's my challenge: Program when you feel like it, when you have some problem amenable to computational solution. Take the weekend off; or if it's not your day job, only program on the weekends. Spend as many minutes or hours as it takes. For the love of fuck, do not commit to a Microsoft® Github® log, what is wrong with these people?!

Bring Out the Type System

By the way, about void-safety: for a decade now, Eiffel has been void-safe, meaning a compile-time guarantee of no run-time null pointer dereferencing. It is beyond my understanding how the rest of the world can still live with programs that run under myriad swords of Damocles: x.op (…) calls that might any minute, without any warning or precedent, hit a null x and crash.
—Bertrand Meyer, Why not program right?

I knew this would be exasperating, but really now. At this point, my eyes rolled completely out of my head and I no longer have eyes. 🙄

References don't just randomly become null without warning. You chose to call a function that might return null, and didn't bother to put in an if or assert when that's a possibility. Typically the exception system catches it if you do miss it.

The Objective-C model of nil messaging just returning nil or 0 was theoretically dangerous, but in practice incredibly useful. Crashing out in Javascript means I have to wrap everything with (x ? x.op() : null) to get the same effect, which might require a lot of temp vars.

Do type devotees actually believe in randomly-appearing errors, or that dynamic programmers just flail our limbs on a keyboard until something manages to pass tests, or do they just exaggerate a rare edge case they saw once, or are they completely fabricating this stuff to justify their waste of time/perversion?

Type systems are self-inflicted BDSM, and it is not self-evident that everyone wants to wear a gimp suit.

Dr Dobb's Journal of Computer Calisthenics & Orthodontia

DDJ, especially the run 1984 to 2000-ish, is how I learned C and assembly, and much of my attitude towards software.

These days I read PragPub ed. by Michael Swaine of DDJ infamy, though it's more architect/software manager-oriented than in-the-trenches bit-eating, but it still has some real working code.

Also, I really miss Jolt Cola, like a detoxing junkie misses a needle.

Python 3.7

  • Python 3.7 released: Standard checklist:
    • Run installer
    • Delete the old 3.6 folder from /Applications
    • Run the certificate command in the new 3.7 folder (the other shits a PATH into my shell profile, don't need it)
    • Run IDLE and verify it's 3.7.0. Happily, no longer have to fight with updating Tcl/Tk.
    • Run "python3" from Terminal and verify it's 3.7.0
    • Run a random Python script to make sure nothing's broken.

Nanosecond-accurate time functions and switching more ASCII/C Locale into UTF-8 are nice improvements, but those are more patching up legacy annoyances than "must have".

I'm mostly interested in dataclasses, which makes it much easier to build little struct-type objects instead of random dicts or lists which have all sorts of problems (no equality, hashing, typo-safety).

I greatly dislike the addition of BDSM typing, but it's mostly optional, EXCEPT you have to use them in dataclasses:

from dataclasses import dataclass
class Point:
    x : float = 0.0
    y : float = 0.0

>>> p = Point()
>>> p
Point(x=0.0, y=0.0)
>>> q = Point(1.1, 2.2)
>>> q
Point(x=1.1, y=2.2)

If I define Point without the type annoytations[my new favorite typo!], only the default constructor works, and it doesn't print the fields as a string.

class Pointless:
    x = 0.0
    y = 0.0

>>> f = Pointless()
>>> f
>>> f.x
>>> f.y

Real examples might be a lot more complex than a point, and by then the cost of building a proper class with __init__ and everything yourself isn't such a big deal, so I can see dataclasses mostly being used for very simple struct-like containers.

A Little Scheme

I go through phases of playing with Scheme for utility code, maybe even portable dev; while FreePascal is a better language for this, I'm frustrated by the lack of library support and the useless iOS situation.

Scheme's always been an emergency backup language; it was fun to learn back in the '80s and early '90s, and both SICP and TSPL are good books, but nobody wanted to pay for Scheme dev, and anyway the language is very annoying to write. I often treat it as a logic puzzle to get anything done, not a useful tool. But it does have good library support, and it can compile to very fast binaries, despite having GC pauses and consuming 2x as much memory as a C program. Maybe I can get better at solving problems in it, build up some libraries, and make it useful?

So the current landscape is:

  • Chez Scheme:
    • Pros:
      • Very fast to compile and at runtime, competitive with C compilers.
      • Great interactive REPL, not just a half-broken readline history like pretty much every other Scheme.
      • Debugger is reasonably good, and integrated in the REPL. All I really use myself is (trace FOO) and (inspect BAR), but non-caveman coders will make better use of it.
      • Current R6RS implementation plus extensive chezscheme library.
      • By R. Kent Dybvig, author of TSPL, and Cisco currently employs him to maintain Chez Scheme.
      • REPL environment is called a café, which I find charming. Yes, I also liked all the coffee puns and iconography from early Java programming.
    • Cons:
      • Not as widely supported by tools & documentation as Racket.
  • Racket:
    • Pros:
      • Very nice GUI.
      • Current R6RS implementation plus extensive racket library.
      • Built around making multiple languages; I don't really care about this. I loathe "Typed Racket", one of the worst combinations of ideas in history.
      • Tons of documentation.
    • Cons:
      • Mediocre performance. There's a project to rehost Racket on Chez Scheme, which would fix this, but then why use Racket?
      • Doing anything in the GUI destroys your environment, all the objects you've made, unlike any LISP or Scheme ever. So it's utterly fucking useless as an interactive REPL. I can't say enough bad things about this. ★☆☆☆☆ Kill On Sight.
  • Chicken:
    • Pros:
      • Compiles to C and thence to native binaries, with nice FFI to C libraries.
    • Cons:
      • Mostly old R5RS, with a few extension libraries.
      • Terrible REPL, only really usable as a compiled language.
  • Scheme R7RS benchmarks

Chez Scheme is the clear winner for me; if I was a novice, I might choose Racket and not realize that the REPL is a broken abomination for a while. If I was only doing C interop, Chicken would be better.

Editing in BBEdit works OK, but it doesn't know how to find function definitions. I guess Vim has current syntax, but I'm kinda over that habit unless I have to sysadmin. I have never been emacsulated and never will.

Atom's symbols list doesn't do any better. But if you do want to use it, install package language-racket (all other language-schemes are R5RS at best), and then add some file types to config.cson:

      "source.racket": [

In any editor, any language, I use hard tabs (1 char = 1 logical indentation level, obviously), and normally tabstop at 8 chars which discourages very long nesting and encourages me to extract functions. Scheme is indentation hell, so set the tabstop to 4 spaces. (The code blocks below won't show that.)

Do not criticize my C-like paren/brace placement; I prefer clear readability of code structure to some obsolete Emacs dogma.

So, let's see it work, with hello.ss:

#!/usr/bin/env scheme-script
(import (chezscheme))
(format #t "Cheers 🍻 , ~a!~%" (car (command-line-arguments)))
% chmod 755 hello.ss
% ./hello.ss Mark
Cheers 🍻 , Mark!

Now for something more serious:


;; stdlib.ss
;; Copyright © 2015,2018 by Mark Damon Hughes. All Rights Reserved.
(library (stdlib)
    (export inc! dec! currentTimeMillis randomize input atoi)
    (import (chezscheme))

;; Variables

(define-syntax inc!
    (syntax-rules ()
        ((_ x)      (begin (set! x (+ x 1)) x))
        ((_ x n)    (begin (set! x (+ x n)) x))

(define-syntax dec!
    (syntax-rules ()
        ((_ x)      (inc! x -1))
        ((_ x n)    (inc! x (- n)))

;; Date-Time

(define (currentTimeMillis)
    (let [(now (current-time))]
        (+ (* (time-second now) 1000)
            (div0 (time-nanosecond now) 1000000))

;; Random Numbers
;; "Anyone who attempts to generate random numbers by deterministic means is, of course, living in a state of sin." —John von Neumann

(define (randomize)
    (random-seed (bitwise-and (currentTimeMillis) #xffffffff) )

;; Input/Output

;; Reads a line from stdin, ends program on EOF
(define (input)
    (let [(s (get-line (current-input-port)) )]
        (if (eof-object? s)
            [begin (display "Bye!\n")

;; Strings

;; Converts a string to an integer, 0 if invalid
(define (atoi s)
    (let [(n (string->number s))]
        (if (eqv? n #f)
            (inexact->exact (truncate n))



#!/usr/bin/env scheme-script
;; guess.ss
;; Copyright © 2015,2018 by Mark Damon Hughes. All Rights Reserved.

(import (chezscheme))
(import (stdlib))

(define (guess)
    (display "I'm thinking of a number from 1 to 100, try to guess it!\n")
    (let [(theNumber (+ (random 100) 1))]
        (define guesses 1)
        (do [(break #f)] (break)
            (format #t "Guess #~a? " guesses)
            (let [(g (atoi (input)))]
                    [(or (<= g 0) (>= g 100))
                        (display "Try a number from 1 to 100.\n")
                    [(< g theNumber)
                        (display "Too low!\n")
                        (inc! guesses)
                    [(> g theNumber)
                        (display "Too high!\n")
                        (inc! guesses)
                        (display "You got it!\n")
                        (set! break #t)
    (display "***GAME OVER***\n")


chez-compile.zsh, with my thanks to Graham Watt for explaining wpo and libraries:

if [ $# -ne 1 ]; then
    echo "Usage: chez-compile.zsh MAINNAME"
    exit 1
rm -f *.so
rm -f *.wpo
mkdir -p bin

cat <<ENDTEXT |scheme -q --optimize-level 3
(compile-imported-libraries #t)
(generate-wpo-files #t)
(compile-program "$1.ss")
(compile-whole-program "$1.wpo" "bin/$1")

rm -f *.so
rm -f *.wpo
if [ -f "bin/$1" ]; then
    chmod 755 "bin/$1"

Now I just:

% chez-compile.zsh guess
compiling guess.ss with output to guess.so
compiling stdlib.ss with output to stdlib.so
% bin/guess
I'm thinking of a number from 1 to 100, try to guess it!
Guess #1? 50
Too low!
Guess #2? ^DBye!

Well, that was an adventure to get the equivalent of my first BASIC program from 1980, which can be run in Chipmunk BASIC if you don't happen to have a TRS-80 Model I handy:

10 N=INT(100*RND(1))+1:T=1
100 PRINT "GUESS #";T;"? ";:INPUT "",G
110 IF G<=0 OR G>=100 OR G<>INT(G) THEN 200
120 IF G<N THEN 210
130 IF G>N THEN 220
140 GOTO 230
210 PRINT "TOO LOW!":T=T+1:GOTO 100
220 PRINT "TOO HIGH!":T=T+1:GOTO 100

But now I can think about more complex problems in Chez Scheme!

Here's the tiniest piece of what I've been thinking about next:

Island in emoji

CS Larva

CS Larva
My first infinite loop was summer, age 10, only because I hadn't seen a computer until the year before. My pre-college computing skills were 90% self-taught; LOGO class at ~12 and a year of "comp sci" at 14 were minimal. ⌨️🖥