A bot for Glitch's editor

[eris]

go ahead. I’m going to try to squash it and will edit if I get it down

EDIT: squashed, my code now has a fix
the offset is unsubmittedTx2.tx[0][0] - end + whitespaceLength + 5

subtract the offset from relative offset
add the offset to splice length

(Glitch, what the FU-)

[wh0]

okay after tons of printf debugging, here’s how things were going haywire. don’t blame Glitch, it was a bug in my code.

1. the Glitch editor, when you press enter on an indented line, as was the case in your HTML use case, creates a new line and indents it appropriately.
2. when it tells the Glitch OT server about the new indented line, it doesn’t insert it as “newline, space, space, …”. it first inserts just the newline, then in a second operation, it inserts the indentation spaces.
3. my code would handle an incoming list of operations (an “oplist”) by increasing the version number and then examining each operation in sequence.
4. when my code examines an operation that finishes one or more lines, it invokes the callback for handling a new line, synchronously. but in this case, it hasn’t even finished processing the oplist.
5. the synchronous part of the line handler starts sending out a new transaction, mistakenly thinking it is already operating on the new version, as the version number has already been increased.
6. then my code finally gets to the second part of the incoming oplist, and it incorrectly sees this a something that needs to fix up the outstanding transaction.

so as a result,

• an oplist goes out to the OT server that tries to operate on the wrong starting state.
• the client’s view of the outstanding transaction incorrectly gets updated, which later causes the successor transaction to be created with the wrong information too
• basically everything thereafter goes wrong because of the above

I’ve made the following fixes in the original maple-separated-seal:

1. Move the version increment to the end of the oplist apply-er:

    function opListApply(opList) {
   -  version = opList.version + 1;
      for (const op of opList.ops) {
        docsVisitOp(op);
        txsVisitOp(op);
        readerVisitOp(op);
      }
   +  version = opList.version + 1;
    }
   

   this is a safeguard to make any transactions sent while in the middle of processing an incoming oplist fail on the server. okay and I just realized that I had not settled whether oplist is one word or two words together, as opList .

2. wait for a microtask at the top of readerVisitLine

    function readerVisitLine(docId, unsubmittedTx) {
   +  (async () => {
   +    try {
   +      // wait for a microtask, after we finish processing the whole oplist that
   +      // this line appeared in.
   +      await Promise.resolve();
   +
      if (!unsubmittedTx.tx) throw unsubmittedTx.reason;
      ...
      const submitTxPromise = submitUnsubmittedTx(unsubmittedTx);
   -  (async () => {
   -    try {
          const unsubmittedTx2 = await submitTxPromise;
   

   this prevents us from submitting a transaction until after the whole incoming oplist is processed. note that the first if (!unsubmittedTx) throw is now required, as an oplist may create a line and then disturb it in the same version, before we finish awaiting that Promise.resolve().

[eris]

This is only a partial fix sadly

Magic spaces were part of the problem. Using your new code:

• Magic spacing corresponding to the indent appears! That was part of the problem if you had no indent (Or a lot of unneeded indent), as that spacing can possibly cause issues.

I’ll update my bot to the new code but now I need to add even more code to handle the dumb spaces.

EDIT: New code makes it even harder to work around the dumb space issue >:/
I’m going to stay on the old codebase in order to keep the workaround for now

[wh0]

oh is this for when you don’t have a newline at the end of the string that the bot adds?

[eris]

No the newline just moves the spaces AFTER the newline and makes mitigation worse

[wh0]

but isn’t it nice to have the extra newline and indent?

(indent)      // > (your message)(your newline)
(bot indent)  // < (bot message)(bot newline)
(added indent)(cursor)

that way you can start typing another // > ... right away

vs without the extra newline:

(indent)      // > (your message)(your newline)
(bot indent)  // < (bot message)(added indent)(cursor)

[eris]

It would be nice if the indent worked right, currently it only spawns the indent the editor expects (Which often doesn’t work with extra / removed indents)

[wh0]

do you have an example of that, when it’s wrong?

[eris]

• make a .html file
• add a script element
• remove the indent (bad idea but oh well)
• use the bot
• indent comes back >:/
• add a bunch of tabs as unnecesary indent
• use the bot
• unneccesary tabs are not applied

edge case but still a thing

[wh0]

it’s inserting responses at the same indent level as the request

the glitch editor has its opinion on how much to indent a new line, and it does so unrelated to the bot

[eris]

there’s the problem. I want it to be on the same line as the first request, and ignore glitch’s annoying indent. I’m assuming this is impossible

also if you remove the newline the spaces just appear after the response anyway

[wh0]

I suppose you could scan forward in the document and detect spaces and delete them from the transaction

[eris]

here’s an idea on how I could do this

• grab the next ~200 characters from our // < ... starting point

• make a regex statement matching that starting part and then all spaces until a non-space is hit

• collect those spaces into a regex group and do some stuff with them- namely, removing or adding more spaces to compensate for the extra ones

I have no idea how to do that but it would work

[wh0]

@eris

I want it to be on the same line as the first request

wait did I read this right? did you want it like this?

(indent)      // > (your message) < (bot message)(your newline)
(added indent)(cursor)

[eris]

yes. the negative indent too

[wh0]

there’s no negative indent there.

but there’s a part where it sets the relative offset to end, meaning to insert the text right after the newline. you would instead use end - 1, so that it gets inserted before the newline. and with that, don’t insert any newlines, because that would make it look like the same line has been created again