Some of My History of Hypertext

So in 1979, young Digital Mark is shown a computer, a TRS-80 Model I, by 4th-grade teacher. Other kids play Snake, I hit break and type LIST, seeing BASIC code, because I've been reading the instruction card. "I can read and learn this!" Nothing else now matters to Mark.

That summer, I take an adult college summer course on programming. Over the next few years before I get my Atari 800, I use school computers, mostly TRS-80 but some Apple ][, to program obsessively. Libraries and book stores are scoured for information.

The main sources of knowledge for me then were:

  • Stimulating Simulations, by C. William Engel (1977)
  • Creative Computing magazine, ed. David H. Ahl (1974-85)
  • Basic Computer Games, Microcomputer Edition, by David H. Ahl (1978)
  • some TRS-80 Level II BASIC book
  • My Computer Likes Me When I Speak BASIC, by Bob Albrecht, People's Computer Company (1972)
  • Computer Lib/Dream Machines, by Ted Nelson (1974-5)

The first three I could buy, the last two I only ever found in the library. They are now all on Internet Archive. As noted here a few years ago.

Computer Lib is bizarre rantings mixed with really visionary stuff. It's a two-sided flippy book, pages numbered CL 1-68 one side, DM 1-60 the other). Computer Lib is about how computers work "now" (1965-1974), teletypes talking to mainframes or minicomputers. This is important! There's a range of computing levels.

Ted talks about BASIC and how to use it seriously, using arrays as data structures, in a few pages. Some weird stuff about other languages I haven't heard of. More about data structures, links between data in different places. I'M LIKE TEN YEARS OLD, man, you can't dump that on me! I didn't fully understand this stuff until Sedgewick's Algorithms years later.

Political ranting, some of it now unacceptably phrased, but CYBERCRUD is a good primer on how computers aren't the problem, people with computers are the problem. This echoes the People's Computer Company slogans

BASIC is the People's Language!

Use computers FOR people, not against them!

Dream Machines has a 1975 supplement about the Altair, bitmap graphics, and cheap microcomputers, which largely lines up with Computer Lib's earlier predictions. It goes in depth into how you can use graphics, image and audio processing.

If Computers Are The Wave Of The Future, Displays Are The Surfboard

Branching Presentational Systems - Hypermedia

And here he develops a theory of Hypertext, "by which I mean non-sequential writing". Everything you now know about the World Wide Web and GUIs, you pretty much got by way of Ted Nelson, Doug Englebart of NLS, JCR Licklider of Project MAC (multi-user time-sharing), and Ivan Sutherland of Sketch. I read this wide-eyed and believing everything.

The giant over-arching project for this is XANADU (earworm Olivia Newton-John song here). Xanadu is a very well-specified, complex, solves everything system. Ted's argument is that long-run everyone will use his system for every computation. This starts at DM 44, read that entire section if you read nothing else.

Returning CL/DM must've been the hardest thing ever. I read it again a few times over the years. (I have a stack of notes about it from more recent reading, but I'm here staying brief and focused.)

Over the next decade, we got and then slowly lost things like Hypercard. HC's an amazing technology, it is The Future We Didn't Get, because it was Mac-only, monochrome-only, Hypertalk is a somewhat annoying hybrid of BASIC, COBOL, and C, limited to either tiny windows smaller than the small Mac desktop, or full-screen Mac 512x344. Later that got better, and Myst was written in HC + many addons, but it remained a weird silo.

Repeatedly Ted Nelson says Xanadu is coming soon, shares new technical details which conflict with previous statements, various people on his team promote it, then unpromote it. The exchange in Dr Dobb's Journal 1983 disillusions me, convinces me Xanadu will not be coming soon.

The Internet became available to me in 1989, and Gopher came out in 1991, providing a real hypertext solution. It didn't have inline links, "transclusion" (include a chunk from another document), or bidirectional links, but it did let you make complex menus with text, links, images and other media, and interaction with forms/search fields. Gopher became the best way to hyperlink and index everything.

The WWW was not an attractive system until Mosaic came out in 1993, mid-year it got inline image tags and imagemaps as a dumb hack, and Netscape Navigator commercialized it in 1994. Gopher might've survived this except the UMinn administration tried to extort payments for servers in 1993. I moved everything into my web site, as did everyone else.

The thing is, these are not Xanadu. Ted got increasingly strident that the WWW is not Hypertext because it's not bidirectional etc. But of course this is impossible: A local database can be forced to be consistent, but a network of unreliable computers cannot. One-way works with stupid complexity and GeoCities, and two-way does not. Imagine entering a web page and seeing 500 "sites who called here" links, some of them private. Each of them has to validate that it's paid the micropayment for copyright access to the site. It's not "web scale".

In the end, post 2000, there's a demo "release" of Xanadu, with a hand-hacked single-site database, no editor, no way to link your own stuff with it. It's not anything.

More notes to come, we'll be talking about this more on the Lispy Gopher Climate Show every Tuesday midnight UTC (like 17:00 PST). hashtag

The Mother of All Demos

December 9, 1968, Douglas Engelbart's presentation of NLS and teleconferencing:

  • Youtube: This is at 360p, most other uploads are at 240p fuzzy mud, I'd love to have a good HD one where I can read the text. Alas.
  • TheDemo@50

"If in your office, you as an intellectual worker were supplied with a computer display, backed up by a computer that was alive for you all day, and was instantly responsible—responsive—to every action you had, how much value would you derive from that?"

Of course, in reality what we mostly do with that is look at social media, hardly any better than watching TV. But we could do more.

It's been years since I've watched this, and some things jump out at me as I rewatch:

The keyboard beep is infuriating, it's what I consider an error sound. And Doug's fumbling a few times, which suggests the keybindings aren't visible, well-organized, or practiced yet. We see later that they're just code mapped to keys in a resource list.

The NLS word demo is somewhat like a modern programmer's editor with code folding; but notably I don't ever use folding, it's slow (even on 1 million times faster machines!) and error-prone, sucking up far more text than expected. It's also a lot like outliners like OmniOutliner; but while I do sometimes use OO to organize thoughts, I would never keep permanent data in it, because I can't get it into anything else I use. Dumb text is still easier and more reliable; I put my lists in Markdown lists:

- Produce
    + Banana
        * Skinless

Maybe the answer is we should have better tools and APIs for managing outlines? Right now I can manage dumb text from the shell, or any scripting language, or with a variety of GUI tools. OmniOutliner's "file format" is a bundle folder with some preview images and a hideous XML file with lines like:

    <item id="kILRUkulXwk" expanded="yes">
      <values>
        <text>
          <p>
            <run>
              <lit>Stuff</lit>
            </run>
          </p>
        </text>
      </values>
      <children>

Nothing sane can read that; even if I use an xml-tree library, it's still item.values[0].text.p.run.lit to get a single value out!

If I export it to OPML, it loses all formatting and everything else nice, but I get a more acceptable:

    <outline text="Stuff">

Back to the demo.

The drawing/map editor's interesting. This is pretty much what Hypercard was about, and why it's so frustrating that nobody can make a good modern Hypercard.

Basically every document seems to be a single page, fixed on screen. If a list gets too long, what happens? It doesn't scroll, just fully page forward/back.

Changing the view parameters is basically CSS; CSS for the editor! Which is what makes Atom so powerful, but it's not easy to switch between them, probably have to make your own theme plugins, or just a script to alter the config file and then reload the editor view.

Inline links to other documents in your editor, is interesting. Obviously we can write HTML links, but they have to be rendered out and no editor can figure out where a reference goes and let you click on it. Actually, this does work in Vim's help system, but nowhere else.

The mouse changed three ways since then: The tail moved to the top, the wheels became a ball which drives two roller-potentiometers inside, and then was replaced with a laser watching movement under a window. Don't look into the butthole of your mouse with remaining eye. But the basic principle of relative movement of a device moving the pointer, rather than a direct touch like Don Sutherland's Sketch light pen, or modern touch screens, or a Bluetooth stylus, remains unchanged and still the fastest way to point at a thing on a screen. Oh, and the pointer was called a "bug" and pointed straight up, Xerox copied this directly in their Star project, while everyone since Apple has used an angled arrow pointer.

The chording keyboard never took off, and I've used a few, and see why: It's incredibly hand-cramping. While a two-handed keyboard is awkward with a mouse, you have room to spread your fingers out, and only half the load of typing is borne by each hand. On a chord, each finger is doing heavy work every character.

The remote screen/teleconferencing setup is hilarious: a CRT being watched by a TV camera, which runs to a microwave transmitter; they couldn't send it over phone lines, acoustic coupler modems were only 300 baud (bits per second, roughly) at the time.

As with Skype today, every chat starts with "I can't hear you, can you hear me? Fucking (voice chat system)." Later, audio drops out, and all Doug can do is wave his mouse at the other presenter. I've joked before that the most implausible thing in Star Trek isn't FTL, even though that's physically impossible; it's not aliens indistinguishable from humans with pointy ears, half black/white makeup, or bumpy foreheads; it's that you meet an alien starship and can instantly set up two-way video conferencing.

They seem to have a mess of languages; MOL (Machine Oriented Language) is a macro assembler in modern terms. All the languages have to be adapted to NLS, they couldn't just use LISP or FORTRAN. Since changes are recorded by userid, they had git blame!

Split screen! That's a thing I love, and few editors do. You can drag a bar down from the top in BBEdit, and Atom has "Split up/down/left/right" for panes, but then you have to re-open the document in each and it's a pain.

Messaging is a public board (or rather, an outline with each statement as a message), with for addressing, like @USERNAME in the Twitters and such. Like those, there's too much data to process for live updating, everything runs as a batch job that can crash the database. War Computing never changes.

Cold & hot retrieval are just file search; on the Mac we have Spotlight, and can search by keywords or filename. Though I have some problems with the cmd-space search these days, and mostly open Finder and search from there to get a list of files matching various requirements, or sometimes use mdfind whatever|less from shell, then winnow down "whatever" until I have only a few results. On Windows or Linux, you're fucked; get used to very long slow full-text searches.


What NLS Did, and How We Can Do That

  1. Mouse, Keyboard, bitmapped displays: We have that.
  2. Teleconferencing: Still sucks.
  3. System-Wide Search: Mac users have that, everyone else is boned.
    • It's faster on Linux or Windows to search Google for another copy of existing data than to search the local machine.
  4. Outlining to enter hierarchical data: Nope.
    • All data goes into outlines contained in files.
    • Code as data: Some data is program instructions, in a variety of languages, which can operate on outlines.
    • To enter this outline, I had to keep adjusting my numbers, because I'm writing it in markdown text.

As mentioned above, OmniOutliner is logically very similar, but it's a silo, a trap for your data. The pro version (WHY not every version?!) lets you use Omni Automation, which is basically AppleScript using JavaScript syntax; the problem is waiting for an app to launch, then figuring out where your data is hidden inside some giant structure like app.documents[0].canvases[0].graphics[2] (example from omni docs ), just so you can extract it for your script.

Brent Simmons is working on Rainier/Ballard, which is a reimagining of Dave Winer's Frontier. I think building a new siloed language and system doesn't solve the real problem, but maybe it'll get taken up by others.

I have for some time been toying with enhancing my Learn2JS shell into an Electron application that would let you write, load, save, and run scripts in a common framework, without any of the boilerplate it needs now. A pure JS shell is just too limited around file and network access, and node by itself is too low-level to get any useful work done. I'm not sure how that works with everything else in your system. While browser localStorage of 2MB or so is sufficient for many purposes, you really want to save local files. While this doesn't force data into outlines, it makes code-as-data easy, and JavaScript Object Notation (JSON) encourages storing everything as big trees of simple objects, which your functions operate on.

(I'm having fun with Scheme as a logic puzzle; but it's not anything I'd inflict on a "normal" person trying to work on data that matters).

If you want to talk about doing more with this, reach me @mdhughes on Fediverse.

Beyond Cyberpunk Web Design

What I want to note here is the UI in the original BCP and Billy's app. Borders filled with wiring and lights. Knobs and switches. Big chunky click areas. Punk rock, graffiti art. When you click things, audio and animations tell you something happened. Not so much the "Jacking into the Matrix. Into the FUTURE!" clip.

It's much easier to find and read information in the web version, but it's not fun. It's ugly and boring. Like almost everything on the web and apps these days, from Jony IVE-1138's sterile white room prisons where you're tortured for daring to have a personality, to all these endless linkblogs.

There are places with personality, but not many. The web looks like shit. Update: Brutalist Websites has some GeoCities-like aesthetics in a few. Others are sterile voids.

And that's bothering me about this blog. It looks OK, the stolen Midgar art and my '80s neon colors set some kind of tone, but it can be so much more. So in the weeks and months to come, I'm gonna be doing some redesign, make this into something weirder, if not full-on GeoCities. The RSS feed should be uninterrupted, but I'm going to put a lot more resources on the front page.

Hypercard! The Software Tool of Tomorrow!

Encouraging people to write & submit new Hypercard stacks. Which, as a retro-tech challenge, I think is great. But. How about first having a decent modern Hypercard environment?

Why not? There's the classic John Gruber hit piece Why Hypercard Failed:

"Apple PR says it's a dead product so it doesn't matter if you like it! I like the Yankees who are also a bullshit PR project!"
—semantic analysis of all Gruber's posts produced this summary.

Stanislav (not a pleasant or generally useful person to me, but perhaps correct for once), had a different read of Why Hypercard Had to Die:

The reason for this is that HyperCard is an echo of a different world.  One where the distinction between the “use” and “programming” of a computer has been weakened and awaits near-total erasure.  A world where the personal computer is a mind-amplifier, and not merely an expensive video telephone.  A world in which Apple’s walled garden aesthetic has no place.

Apple did have a near-Hypercard tool, Dashcode, which was slightly more technical but not much; it auto-generated placeholder functions and you'd fill them in with JS and use local storage as your database. They never fully supported it, killed it, and pushed Xcode instead, which is like giving kids a backfiring nailgun with no safety instead of a plastic hammer. Now they're ludicrously trying to teach kids BDSM Swift with the lldb debugger repackaged as "Playgrounds". I feel so sad for a kid whose first experience of programming is 100s of "unable to satisfy template constraint" errors; that's some hard unyielding playground equipment there.

There's a few modern variants, but nothing I know of that works:

  • Uli Kusterer's Stacksmith is unfinished, has no binary download or web site, and the build instructions are very pro-dev. Last time I tried it I couldn't get it to build, so…
  • SuperCard is $180/$280. Ha ha… uh, no.
  • HyperNext Studio is based on RealBASIC, and is free, but rarely updated. Does not run classic Hypercard stacks.

So everyone just gives up and uses emulation, because making a new Hypercard is impossible. If you're going to do that, do it the easy way: