Command line `--help` to HTML pages

Project URL: snail — Snail

View generator source: Glitch :・゚✧

There’s this library that does command line arguments parsing, commander.js. You can set up a bunch of subcommands each options and stuff. That builds up this big structure in memory, and there’s a built-in “help” system that can print it out on the terminal:

And this help system is customizable in code, so you can write some functions in JS that control how it shows different parts, like the “usage” line, description, options, subcommands, etc.

In this project, I wrote up some a set of such custom functions and some other helpers to make it generate HTML web pages instead of terminal output. And now I have those being served from a generated-static site on Glitch:

image

It turned out to be hacky. Maybe too hacky. But what am I gonna do, manually make web pages for each command? (According to a printout that this program generates, this program I’m using it on has 53 commands :scream: Did I really write all those?)

I’m gonna discuss some of the weird hacks in later posts.

4 Likes

Alright so here were the objectives:

  1. Make it so you can view the help on the web.
  2. Try not to have to rewrite the code that generates the terminal help output.
  3. Have links to the subcommands.

Here’s what we have to work with:

  • There’s this configureHelp thing that lets you pass some custom functions that the built in help generator will call.
  • Example here commander.js/configure-help.js at 43f4743864e2f670db5eebcf88c92aa4612c54f1 · tj/commander.js (github.com), showing a customized subcommandTerm.
  • A method like subcommandTerm affects the “term” part of an entry in the subcommands section of a given parent command. These lists are two-column things with a “term” on the left and a “description” on the right.
  • Various other functions you can provide allow you to customize the terms and descriptions of the sections that contain listings, and still more functions let you customize the other parts such as the usage line and the command description.
  • In these functions, you get some command object in (or similar), and you return a string that the help generator system will put into the generated help printout.

I got to work writing these little functions that would (1) call the default implementation and (2) wrap them in some HTML tags that I could then style with CSS. For subcommandTerm in particular, I would add link tags as well.

And at that point, it all seemed pretty darned doable.

2 Likes

By the time I finished up the code to wrap various parts in HTML tags, I was already aware that it wouldn’t just work. A help output looks like this:

Usage: snail asset push [options] <src>

upload an assset

Options:
  -p, --project <domain>       specify which project (taken from remote if not set)
  -n, --name <name>            destination filename (taken from src if not set)
  -t, --type <type>            asset MIME type (default: "application/octet-stream")
  -a, --max-age <age_seconds>  max-age for Cache-Control (default: 31536000)
  -h, --help                   display help for command

Implementation problems:
Does not maintain .glitch-assets.

It has plenty of those < and > signs, which I would have to escape for HTML. I slipped in a function to do that too. And I was ready to behold version 2.0 of my generated help files (I forgot to mention, version 1 had been more or less a copy and paste of the terminal output into a <pre> tag, which lacked links for the subcommands). This newly HTML-ed up version looked like this:

(Not really, I hadn’t put in the stylesheets and stuff. This is a dramatization where I intentionally broke some things from the latest version. Anyway.) It was a mess, visually. What the heck?

Of course, there was a good reason why it looked that way. The page looked that way because the source code looked like this:

The HTML code was beautiful. Everything lined up, just as the authors of the help system intended. There was a tidy column for the “terms,” which looked like <span class="subcommand-term"><a href="rsync.html">rsync &lt;args...&gt;</a></span>, and a tidy column for the “descriptions,” which looked like <span class="subcommand-description">launch rsync with snail pipe as the transport</span>.

That is, the help system did its best to line things up, but those very things were full of tags and character entities that mess with everything’s width once it was viewed as a webpage. And different things would have different lengths due to the non-constant link URL and the varying number of escaped < and >.

That day (so to speak, I worked on this across multiple days), I really had to think about objective number 2.

  1. Try not to have to rewrite the code that generates the terminal help output.
2 Likes

Here’s one idea to fix this: do the columns in CSS. That’s the normal thing to do in HTML, after all. You’d normally make the whitespaces insignificant and use some other system to decide how things are laid out.

I messed around with some more HTML tags so that it would make a tree like this

pre
| ...
+- "Commands:\n"
+- span.item
|  +- span.term
|  |  +- a[href=...]
|  |     +- "auth"
|  +- span.description
|     +- "sign in"
+- span.item
   ...

That is, you could have it put these additional “item,” “term,” and “description” spans around stuff. If we could just put those related span.items into container element, we could use table layout to make a column for the span.terms and a column for the span.descriptions.

But then I got stuck. You can’t put one big container around all the subcommands. The code from the automated help system builds it like this:

    // Commands
    const commandList = helper.visibleCommands(cmd).map((cmd) => {
      return formatItem(helper.subcommandTerm(cmd), helper.subcommandDescription(cmd));
    });
    if (commandList.length > 0) {
      output = output.concat(['Commands:', formatList(commandList), '']);
    }

    return output.join('\n');

There’s even a formatList function there which would have been perfect, but it’s not part of the helper object, so you can’t override it.

Without being able to put these span.items into a parent element, I couldn’t/still can’t think of a way to make the browser automatically figure out how wide the term column should be. We could at least use a fixed width for the terms though.

Different commands have different options and subcommands, and there’s a dedicated part in the automated help system to figure out how wide the widest thing is. Using a fixed width would still meet all the objectives, but it seemed somehow uncool to leave this piece of functionality out.

So I set this idea aside and thought about other ways get the layout working.

1 Like

It’s not so much that that there was HTML code that led to the wrong layout.

The last version looked all messy because different items were off by different amounts, so the descriptions on the right didn’t line up. If they would just be off by the same amount, things would still line up, and they’d probably look fine. As long as they didn’t need to wrap.

But the different widths seem so intrinsic to HTML. Links that go to different places have different lengths because the href is inline. Some characters, like s, take up one character in code, while some characters like < and > take multiple, as &lt; and &gt;.

The second idea is to generate the help in a format other than HTML—something that we can convert into HTML long after the automated help system does the layout for us. It can’t be as simple as plain text, as we need links. But it should be something that doesn’t have so much variance in how wide the features beyond plain text are.

It should look something like this:

Commands:
  (A)auth(B)                             (C)sign in(D)
...
  (E)setenv [options] XnameY XvalueY(F)  (G)set an environment variable(H)

(A) through (H) are something fixed-width, but unique enough that we can eventually put a link to the auth command’s page in (A) and a link to setenv in (E). X and Y are something that are a single character wide that will represent < and >. And all of this stuff, as well as everything else that would show up in the help text, should be distinct.

There we have it, the next thing we’re going to do is design a markup language of our own!

(Yeah, I also wondered if this was going too far off course :sweat:)

1 Like

I love that you’re sharing the discovery steps. One of the things causing issues is (maybe) the end goal is too broad.

If you look at the example below it displays the options, the option, an alias, parameters, a description and default information. That is impossible to do on a single line if aligning vertically is added as well.

I tend to like a design that displays what I asked for and doesn’t try to guess what I might have wanted. So in the example below the -p option would be listed but not the alias and parameters. If I want to know more about -p I get help on it and it would explain there is an alias and parameters but it would do it without my having to see all the aliases and parameters for all the other options (I’m not currently interested in).

Ultimately the individual “help items” can be placed into a much larger formatted page which a person can opt to read on the terminal or pipe to a file and download if they want to review the docs in one go.

Options:
-p, --project specify which project (taken from remote if not set)

3 Likes

good foreshadowing there, in particular regarding this remark in the previous post

they’d probably look fine. As long as they didn’t need to wrap.

because if it’s not being displayed on a big widescreen monitor, it’s probably going to have to wrap

that’s a cool idea, especially for a web version, where you’re only a click away from more information!

1 Like

We’re making our own language to represent this kind of help document, but which beyond plaintext, supports hyperlinks. And it’s free to be as far from HTML as needed, just as long as we can convert it to HTML later. It just needs to add (or even subtract :woman_shrugging:) the same width to each item. Here’s another copy of the thing we wanted to encode, for your convenience:

Commands:
  (A)auth(B)                             (C)sign in(D)
...
  (E)setenv [options] XnameY XvalueY(F)  (G)set an environment variable(H)

So here’s what I came up with:

  • X is encoded as plain old <.
  • Y is encoded as plain old >.
  • We’re not gonna use HTML, so let’s go ahead and make these not special characters in the first place.
  • (A) is encoded as U+E000, a Unicode “private-use” character.
  • (B) is encoded as U+E001, the next private-use character.
  • And so on.

It’s kind of a silly language, but it actually works:

  • Everything that normally shows up, the various []<>{}() have nothing special about them, and they can stay as they are and not change width.
  • Trivially all of (A), (B), … are the same width.
  • I pinky promise you that I will never write a real U+E000 etc. in the documentation, so there will be no ambiguity that they stand for an (A) or whatever.

And we can write out some pseudocode on how to convert a document in this language to HTML:

  1. Replace < with &lt;.
  2. Replace > with &gt;.
  3. Replace U+E000 with <a href="auth.html">.
  4. Replace U+E001 with </a>.
  5. (and so on)

We don’t even have to come up with a whole codebook full of <a href="..."> <-> U+Exxx entries in advance. We can come up with the “language” and the eventual convert-to-HTML in the middle of processing the automated help. The pseudocode looks like this:

let codebook be a string to string mapping
whenever we would have wanted to insert some HTML code C:
  let H be the next unused private use character
  add H -> C to codebook
  insert H instead
use the automated help system lay out the help text, resulting in S
escape HTML characters in S
replace private use characters in S based on codebook

And this custom language gets us to a working page with subcommand hyperlinks and columns that lines up.

But things still go wrong when a long description has to wrap. It would only be off by two columns (the single-character (A) and (B) around the term part), but I worked out a way to make it behave. It uses some non-ancient CSS, so I guess I’ll write a little about that too. Next time though.

1 Like

I designed and wrote a hyper-text help system for a DOS-based business app in 1996. This won’t actually help you in any way but I thought I would mention it :slight_smile:

If your help system was an app, i.e. help is running until the user exits, rather than simply a page of output you may have considerably more options, or is it?

2 Likes

When the description of one of these term–description items is long, the automated help system wraps it into multiple lines. It’s supposed to look like this:

  term  long description lorem ipsum dolor
        sit amet, consectetur adipiscing
        elit.

The automated help system does this in a function—one you can override—called wrap. You get in a string like this

 .--------------------------------------- two column spacer, added outside of wrap
 |         .----------------------------- term, padded to column width
 |         | .--------------------------- two column spacer
 |         | |                         .- description
<><--------><><------------------------>
  term        description, possibly long
  <------------------------------------>
  <---------->                         '- you get this string
             '--------------------------- and you get this length

You wrap it to a given width and you indent wrapped lines to a given “indent” column, then the automated help system indents it a little more, because whatever.

I wrote up a wrap function that (1) ignores the desired width and (2) adds some HTML tags to create elements arranged like this:

<>ABC         <> D
  [[[term]      ][the description, possibly long]]

Where:

  • <> are the various spacers, which this wrap doesn’t touch
  • A is an element with display: inline-flex
  • B is an element that combines the term and the padding into a single flex child
  • C is a link that we added earlier as part of the term, which this wrap doesn’t touch
  • D is an element with flex: 1; white-space: pre-wrap

And that overall works alright. When the viewport gets too narrow, the D element gets narrower and the description inside it wraps.

image


I feel like if we would switch over to an interactive help app, we wouldn’t be using this built in automated help system which is specialized for that one-shot, page-of-output design. And then we wouldn’t have given ourselves that “try not to rewrite the code …” rule. In which case yeah we would have considerably more options :laughing:

2 Likes