A prototype bot relay for Discord

Introducing daffy-circular-chartreuse (DCC), a prototype relay system for making Discord bots viable on Glitch and other serverless platforms

Try interacting with a bot built with DCC (wake C1 https://daffy-circular-chartreuse.glitch.me/ if bot is offline): (updated 2020/07/08 to non-expiring invitation)

View system source code:

Clerical note, 2020/09/19: I’ll be posting updates in the Discord server above, in accordance with the new rules for Gallery threads.

Background

What makes it bad to host a Discord bot on Glitch? A few things, but in the last week, this support forum was made very aware of one of them: the Discord API is designe for bot implementations that remain online all the time.

I looked into the details of this. Here’s how the programming model for the Discord API works. There’s this server on Discord’s side called a “gateway.” You write a program that connects to the gateway with a web socket. This is an outgoing connection: your program is the client, and the gateway is the server. Now, as long as you hold this connection open, the gateway will send “dispatch” messages about things happening around your bot, such as people messaging it, people messaging each other, etc.

This design is at odds with how Glitch, as well as other free hosts’ and “serverless” platforms, manage resources on demand. Here, an incoming request causes the platform to allocate some temporary resources for your program. Your program runs. Then, the platform can take away those resources.

Currently, there’s no way to configure Discord to make a request to a program of yours.

Approach

In the absence of an official solution from Discord to better support serverless platforms, the DCC system introduces a component C1 (the architectural name of which is to be decided) that goes between a bot’s business logic and the Discord gateway. C1 connects to the gateway on the bot’s behalf and implements a vision of what a “Discord sends me requests for dispatches” kind of API might be like. The bot registers itself with C1, and C1 delivers dispatch messages to the bot.

The prototype C1 in daffy-circular-chartreuse https://glitch.com/edit/#!/daffy-circular-chartreuse?path=server%2Fserver.js is implemented to be able to handle the gateway connections for multiple bots.

For the rest of the communications with Discord, i.e. retrieving information on-demand and taking actions, the bot connects directly to Discord.

Could we really trust a man-in-the-middle?

This I’m not sure about. But there are some signs that we could. If you’ve run a bot on Glitch before, you’ve trusted Glitch with your bot token and the communications that your bot sent and received. I think if we’re adamant enough about having bots hosted for free, we will be able to find an entity we can trust to host C1.

Key distinctions from pinging

I want to bring up two important ways C1’s communication with the bot differs from pinging.

  1. Communications occur when the bot has activity to respond to, whereas pinging goes on “all the time.” (Side note: you may be aware that the gateway delivers many events that a bot normally doesn’t have to respond to. There’s a filtering system in the prototype C1 that I can describe in a later post.) This allows the bot to sleep when it has nothing to do.
  2. Communications carry information that the bot uses, whereas pinging does not deliver any information.

How is the prototype hosted?

Currently, the C1 prototype is not running on a suitable host. It’s deployed as yet another free Glitch project, which means it won’t stay online, and thus it won’t be able to serve the functionality needed of a C1 to wake up a bot. I’m looking into ways to host this prototype C1.

Other problems with hosting a bot on Glitch

Of course, even with DCC, there are still other problems with hosting a bot on Glitch:

  • Discord rate limits shared with other untrusted projects. DCC actually makes this worse, with the bot no longer keeping a cache of everything ever mentioned over the gateway connection. This would take more requests to load information separately.
  • Continuing platform reliability issues. There are reports that projects aren’t starting correctly and that code is being rolled back. As far as we know, these are unrelated to whether one makes a bot go online by boosting or by using DCC.
  • Project wake up latency. If a bot’s project is sleeping, it will have to wake up before it can process a message delivered by C1. This will result in a delayed response for users when they interact with a bot after a long period of no interaction.

Roadmap

In any order, this project needs:

  • a host for the prototype C1
  • an entity we can trust to operate the prototype C1
  • load testing to see how much traffic a C1 instance can handle
  • technical writeups about a few special things in this prototype
  • packaging up the client library for redistribution
  • further study of how to program normal bot things without connecting to the gateway

And ultimately, for Discord to support a serverless-compatible API of its own

9 Likes

Since your projects already on public, could you not host on repl.it?
Great idea btw :slight_smile:

Thanks for the suggestion. I’ve found the following:

https://docs.repl.it/repls/http-servers

The server will stay awake and active until an hour after its last request, after which it will enter a sleeping stage.

It sounds like repl.it is providing the same kind of serverless platform that makes it unsuitable for hosting C1. Additionally, their terms of service are similar to Glitch’s, with clauses against pinging:

Terms of Service - Replit

You agree not to interfere with or take action that results in any interference with or distribution of the Service or networks connected to the Service.

Compare with Glitch’s:

Glitch: The friendly community where everyone builds the web

We reserve the right to delete, suspend, or terminate your access to, or ability to use, any and all Services that we determine to be placing undue strain on our infrastructure.

These are both broad enough that they could argue keeping a project awake is placing undue strain or interfering with the service.

There’s also this:

Terms of Service - Replit

You agree not use or launch any automated system, including without limitation, “robots”, “spiders”, or “offline readers” that access the Service in a manner that sends more request messages to the Neoreason servers in a given period of time than a human can reasonably produce in the same period by using a conventional online web browser.

Humans might have an edge being able to mash F5 faster than once every hour, but a pinger would far surpass a human in terms of making requests all day long without sleep.

2 Likes

Chapter 2. Going into technical stuff today.

Prototype C1

There’s four main parts to the current prototype implementation for C1: (1) how it connects to Discord, (2) how it filters dispatches, (3) how it connects to a bot’s logic component, and (4) how it handles registration.

Connecting to Discord

The implementation uses the Eris library for managing a connection with Discord’s gateway. Although I want to have Eris handle the connection logic including the handshake protocol, heartbeats, rate limiting, and session resuming, the public API has that stuff coupled with some dispatch handling logic. Namely, it maintains a cache of objects that the gateway mentions, storing them in memory when a dispatch tells us about a new object, deleting them when a dispatch informs us of the object being deleted from Discord. As a side note, I looked into discord.js a little too, and it seems to have this kind of coupling as well.

This dispatch handling logic is undesirable. It has a non-zero memory cost, and it would be of no value to a dumb relay. I found that Eris passes all dispatch events through a method named wsEvent. With the looseness of the JavaScript runtime environment being a plus here, we use a nasty technique called monkey patching to replace the wsEventmethod from outside of Eris. That happens in this part of the server code: https://github.com/wh0/dc-chartreuse/blob/c8bfcb7c050044d570ba5d0dedb950807556f23a/server/server.js#L95.

With this hack in place, I hope to be able to relay potentially long gateway connections about many objects without using up proportionally as much memory. This should help the C1 be able to handle connections for multiple bots.

Filtering dispatches

As I’ve said, the Discord gateway is broad in the data that it gives out. They’ve recently introduced a feature called “gateway intents” that lets you filter the data, but still at a coarse level. If you want to watch for commands, you can for example request “give me messages but don’t bother telling me when people react to a message.” Of course, it’s very common for bots to need to listen for messages directed at them, and it’s very common for servers to have at least a trickle of chatter throughout the day. Gateway intents alone wouldn’t go very far in terms of letting a bot sleep when it has nothing to do.

The approach being explored in this prototype adds a further layer of filtering based on the content of the dispatches, and it’s a filter that the DCC system allows the bot to specify for itself.

Naturally, one would be concerned about the computational resources that might be needed to perform the filtering. So I’ve tried to integrate a restricted filtering language that doesn’t allow general computation. In the prototype, I’m using a language inspired by MongoDB’s query language and implemented in an npm package called ‘sift,’ with a couple of minor further restrictions. First, arbitrary JavaScript in the “$where” selector is disallowed. Second, patterns for the “$regex” selector are limited to a reduced feature set, to prevent a class of DoS vulnerabilities with catastrophically expensive regular expressions. And this all operates statelessly, so memory usage is supposed to stay level even when filtering many dispatches.

There’s good information that comes in; for example, take a look at the fields available in a message https://discord.com/developers/docs/resources/channel#message-object, which is part of the dispatch for a message being posted. Author info, mentions, the text itself, it’s all there. Even with this highly restricted filtering language, there’s already enough to implement common bot logic, such as looking for a prefix or messages from a specific sender. See the sample bot https://github.com/wh0/dc-chartreuse/blob/c8bfcb7c050044d570ba5d0dedb950807556f23a/sample-cookout/app.js#L20 for how I’ve implemented these rules.

One notable thing a developer can check for with this filtering system is whether a message came from a user or a bot. We do this in our sample bot https://github.com/wh0/dc-chartreuse/blob/c8bfcb7c050044d570ba5d0dedb950807556f23a/sample-cookout/app.js#L18. Often bots are configured not to respond to other bots. And configuring this within C1 does something qualitatively important. It bounds the communications from C1 to the bot to the speed of human activity. It will require a human to send a message or click a reaction etc. to have C1 deliver a dispatch to the bot. This quality has, in other places, been an important consideration in judging whether a program is abusive or not. Blizzard Entertainment’s community has summarized the company’s history of decisions as “one action per keypress” for the line between permitted accessibility tools and offending macros.

More on the other parts tomorrow.

1 Like

Chapter 3. Continuing on about C1.

Prototype C1 (continued)

Connecting to the bot

First I want to talk about Discord, and the way things work in the traditional design where your bot would connect to Discord’s gateway. Discord’s gateway has some sophisticated mechanisms to help you avoid missing dispatches. It’s almost like a TCP on top of TCP. There’s an application layer session ID (compare: source address/port in TCP) sequence number with each dispatch message (compare: TCP sequence numbers). If your web socket connection to the gateway dies, you can open a new connection and send a so-called “resume” message with the session ID and last sequence number you received (compare: TCP negative acknowledgements). There’s also a “heartbeat” mechanism to prevent a connection from being idle for too long (compare: TCP keepalive).

These features would be nice to have for the connection between C1 and the bot’s logic component, but I haven’t taken the time to implement them. Currently C1 opens a web socket connection to the bot if it needs to send message, and closes it if there’s 60 seconds of inactivity. This inactivity timer is shorter than Glitch’s inactivity time for putting projects to sleep, so it should not delay project sleep by much.

When C1 connects to the bot, it presents a secret token so that the bot can authenticate C1.

Bot registration

We’ve designed bot registration to be simple. The bot makes a REST call to C1 to state the following:

  • a unique, stable alias to identify itself
  • its Discord token, for C1 to use in connecting to the Discord gateway
  • the desired gateway intents bitfield
  • the dispatch filtering criteria
  • a URL where it wants to receive the dispatches
  • the secret token that the bot and C1 shall use to authenticate one another

This call is idempotent, and it’s fine to call it every time the bot starts up. We do this in the sample bot https://github.com/wh0/dc-chartreuse/blob/c8bfcb7c050044d570ba5d0dedb950807556f23a/sample-cookout/app.js#L244.

The bot is free to change the information other than its alias whenever it wants, using the same call. C1 determines what it needs to do based on what changes: opening a new connection to the Discord gateway, recompiling its dispatch filter, and/or reconnecting to the bot. Or if the bot has just started up with the exact same settings as before, none of those things.

A client library https://github.com/wh0/dc-chartreuse/blob/c8bfcb7c050044d570ba5d0dedb950807556f23a/client/index.js provides some helpers for sending out the registration and setting up a web socket handler to receive dispatches.

1 Like

Im not that technical of which hosting service can be used for this project but would vercel.com work?

Thanks for the suggestion. You’re talking about hosting C1, right? I’m looking at Vercel now, and it seems “Serverless Function Duration” is limited to 10 seconds? Limits | Vercel Docs

But the other way around might be a good fit. You could put up a bot’s logic on Vercel and have C1 elsewhere. Although it would take some modification, because I’m also now seeing that Vercel doesn’t support web sockets. Limits | Vercel Docs

… and the farther I look into this… if I click through to see what the so called “SWR” workaround is:

:grimacing: what in the world

1 Like

Chapter 4. Finally, a tutorial.

Writing a new bot on DCC

Suppose we’ll build a bot that tracks volunteers who will bring items to a cookout. As you read, you can follow along in the completed code: Glitch :・゚✧

  1. Get a token and user ID for your bot from Discord. This tutorial doesn’t cover the specifics of this, but it’s in the Discord Developer Portal Discord Developer Portal. For identification, we’ll need the bot’s user ID, which is labeled client ID on the application page (as far as I know, it’s all the same ID). We’ll use this to help figure out which messages are directed at the bot. For credentials, we’ll need the bot’s token. We’ll use this to connect to Discord’s API.

    We need a suitable place to store these. On Glitch, the .env file is the place to put secret information, such as the bot token. We’ll also put the client ID there because, while it’s not secret, it’s just nicer not to hard code it. With these, we’ll have this in .env:

    # from https://discord.com/developers/applications
    DISCORD_TOKEN=xxxxxxxxxxxxxxxxxxxxxxxx.xxxxxx.xxxxxxxxxxxxxxxxxxxxxxxxxxx
    # from https://discord.com/developers/applications
    BOT_USER_ID=111111111111111111
    

    We’ll load the token into our program under a config object:

    const config = {
      token: process.env.DISCORD_TOKEN,
    };
    
    
  2. Write up some non-Discord-specific logic. We have a data model and a web service.

    The data model we need for this bot is a small list of items we’re tracking and the user ID of who’s bringing each. The whole list is of a small bounded size, so we’ll keep the whole list in memory. When we run a bot on a platform like Glitch with DCC, our process will be started and stopped as needed, so having a copy in memory alone is not sufficient. For persistence across stops and starts, we’ll save changes to disk as they’re made and load them at startup.

    const fs = require('fs');
    
    //
    // model
    //
    
    const assignments = new Map();
    
    function saveItem(name) {
      const volunteer = assignments.get(name);
      if (!volunteer) {
        try {
          fs.unlinkSync('.data/' + name);
        } catch (e) {
          if (e.code === 'ENOENT') {
            // no one was bringing it. leave it that way
          } else {
            throw e;
          }
        }
      } else {
        fs.writeFileSync('.data/' + name, volunteer);
      }
    }
    
    function loadItem(name) {
      let volunteer = null;
      try {
        volunteer = fs.readFileSync('.data/' + name, {encoding: 'utf8'});
      } catch (e) {
        if (e.code === 'ENOENT') {
          // no one's bringing it. this is a valid state
        } else {
          throw e;
        }
      }
      assignments.set(name, volunteer);
    }
    
    loadItem('plates');
    loadItem('hamburgers');
    loadItem('hot dogs');
    loadItem('buns');
    loadItem('drinks');
    

    The web service lets a visitor see what items already have volunteers. We’re writing this part in Express. We’ll later need access to an HTTP server that we can hook up to receive dispatch messages, so we won’t use app.listen(...). We’ll use http.createServer(app) and hold on to the result so we can add more handlers to it later.

    const http = require('http');
    
    const express = require('express');
    
    //
    // web service
    //
    
    const app = express();
    app.get('/', (req, res) => {
      let message = `We have the following covered:
    
    `;
      for (const [item, volunteer] of assignments) {
        const check = volunteer ? 'x' : ' ';
        message += `[${check}] ${item}
    `;
      }
      res.end(message);
    });
    const server = http.createServer(app);
    
  3. Initialize a bot library. Usually a bot would add a library to communicate with Discord, and that library would help with three things: (a) receiving dispatch messages by connecting to the Discord gateway, (b) caching everything it sees from the gateway for rapid access, and (c) actually doing things on Discord, such as sending messages. When we build a bot with DCC, DCC will take care of (a), and we simply don’t have (b) because we’re trying to avoid having our bot stay awake all the time to watch everything. So we’d still like to have a library take care of (c).

    In this tutorial, we’re using Eris. There may be a similar way to set up discord.js or another library. To set up Eris, we’ll do three things out of the ordinary. First, we’ll pass restMode: true as one of the options in the constructor. This configures Eris to enable routines that fetch data from Discord on-demand (whereas otherwise you would look at data in the cache). Second, in conjunction with the restMode option, we’ll prefix our token with the word Bot (Discord accepts two different kinds of tokens, and normally Eris would figure out which one it is by seeing how the gateway reacts to it). And third, we won’t call .connect() on the bot.

    const eris = require('eris');
    
    //
    // bot
    //
    
    const bot = new eris.Client('Bot ' + config.token, {restMode: true});
    bot.on('debug', (message, id) => {
      console.log('bot debug', message, id);
    });
    bot.on('warn', (message, id) => {
      console.warn('bot warn', message, id);
    });
    bot.on('error', (err, id) => {
      console.error('bot error', err, id);
    });
    
  4. Come up with an alias and client secret for your bot. These are a little like a username and password, used just between your bot and DCC.

    For the alias, you might as well use (possibly a simplified version of) your bot’s name. It can contain numbers, letters, and underscores. Or set the alias to something else, if you’d rather not have it publicly identified. The C1 publicly displays the aliases of what bots are using it.

    For the client secret, generate a random string. You can use this snippet of shell script to generate one:

    echo "admin:$(head -c16 /dev/urandom | base64)"
    

    We’ll add the client secret to .env, which will now have this:

    # from https://discord.com/developers/applications
    DISCORD_TOKEN=xxxxxxxxxxxxxxxxxxxxxxxx.xxxxxx.xxxxxxxxxxxxxxxxxxxxxxxxxxx
    # from https://discord.com/developers/applications
    BOT_USER_ID=111111111111111111
    # echo "admin:$(head -c16 /dev/urandom | base64)"
    DCC_SECRET=admin:xxxxxxxxxxxxxxxxxxxxxx==
    

    The alias is meant to stay the same, so we’re putting it directly in the code. We expand the config object to this:

    const config = {
      alias: 'sample_cookout',
      token: process.env.DISCORD_TOKEN,
      clientSecret: process.env.DCC_SECRET,
    };
    
  5. Create a DCC client to receive dispatch messages. The client library is now available on NPM https://www.npmjs.com/package/dcc-client. This object sets up a web socket handler to receive dispatch messages from C1 and emits them as dispatch events. It needs the bot’s alias and client secret (actually the protocol doesn’t currently use the alias, but the parameter is there anyway), as well as a web socket server options object. Here’s a formula that works if you have an HTTP server object and no other web socket handlers on your app: pass that server as the server field and set path field to some route for DCC to use for this bot. Or see the ws package’s documentation if you aren’t adding a web frontend ws - npm or if you need multiple web socket handlers ws - npm.

    const dcc = require('dcc-client');
    
    const client = new dcc.Client(config.alias, config.clientSecret, {
      path: '/dcc/v1/sample_cookout',
      server,
    });
    client.on('dispatch', (packet) => {
      console.log('received packet', JSON.stringify(packet));
    });
    

    This is it for setting up listeners on the HTTP server, so now we start listening.

    //
    // start
    //
    
    server.listen(process.env.PORT, () => {
      console.log('listening', process.env.PORT);
    });
    
  6. Register the client with a C1 deployment. At this point, even though your app is listening, DCC doesn’t know how to talk to it. Now we’ll specify that in the form of a web socket URL. Here we’re using our Glitch project domain along with the path we configured the client to look out for, and all over TLS. We’ll add that URL to the config object:

    const config = {
      alias: 'sample_cookout',
      token: process.env.DISCORD_TOKEN,
      dst: 'wss://' + process.env.PROJECT_DOMAIN + '.glitch.me/dcc/v1/sample_cookout',
      clientSecret: process.env.DCC_SECRET,
    };
    

    Now we use the registermethod from the client library to send our information to C1. We’ll schedule this to be done once our HTTP server has started listening. Surprise! We’ve been setting up this config object to match exactly the options that this method takes. Okay, your program probably doesn’t revolve around DCC, so if you don’t have an object structured like that, just be sure to pass alias, token, dst, and clientSecret.

    server.listen(process.env.PORT, () => {
      console.log('listening', process.env.PORT);
      dcc.register(config).then(() => {
        console.log('register ok');
      }).catch((e) => {
        console.error('register failed', e);
      });
    });
    
  7. Check DCC status online. Go to https://daffy-circular-chartreuse.glitch.me/ and click “list relays.” Find your bot in the list by its alias. Here’s ours:

    "sample_cookout": {
      "enabled": true,
      "lastDisableReason": "token and/or intents changed",
      "connectRunning": false,
      "numQueuedEvents": 0,
      "lastWSError": "(not collected)",
      "wsReadyState": "(no ws)",
      "botShardStatus": "ready"
    }
    

    There are a few different things to keep an eye on here. First, make sure your bot has enabled true. It gets disabled if C1 encounters an unrecoverable error. See the lastDisableReason for why it’s disabled (the reason will be retained for your reference even after the bot is reenabled, so having a message there doesn’t mean it’s currently experiencing that problem). Re-register your bot to reenable it.

    Second, make sure the botShardStatus reaches ready. If it doesn’t reach ready and it doesn’t have connectRunning true, then C1 might be having trouble logging to the Discord gateway with the token you provided. However, this would be hard to debug, because for privacy purposes, we don’t display Discord-related errors here. You might try connecting to the Discord gateway locally.

    Third, make sure numQueuedEvents doesn’t start growing. If it does, check lastWSError for why C1 can’t connect to your bot. You may have the dst or clientSecret out of sync between your registration and client. Re-register your bot to correct this. If this number reaches 1,000, C1 will throw away all of your queued messages and call your parents at work.

  8. Draw the rest of the owl. We have a callback that will log the dispatch messages we get. We have a data model for what we’re tracking. What’s left is to pound out about a dozen if statements to hook them up. The code for the cookout bot is long, so I won’t dump it here. Go over to the finished code to look at it.

    After writing this bot logic, we have a better idea of what dispatch messages we really care about. We only want to see messages being created, and among messages, we only want to see direct messages to our bot, messages that mention our bot, or messages with a certain prefix. We control this with two fields in the registration options: intents, which cuts broad categories of data before they’re even sent to C1, and criteria, which filters dispatch messages on a finer level using cold, hard CPU power on C1.

    Intents are sent as a number representing a bit field Discord Developer Portal.

    Criteria are defined by a language MongoDB’s query selctors https://docs.mongodb.com/manual/reference/operator/query/, as if selecting from a collection of the dispatch messages.

    In our bot, we pass these from the config object, which we expand to this:

    const config = {
      alias: 'sample_cookout',
      token: process.env.DISCORD_TOKEN,
      intents: eris.Constants.Intents.guildMessages | eris.Constants.Intents.directMessages,
      criteria: {
        // corresponds to what we declared in the gateway, but further filters out messages like
        // READY, CHANNEL_CREATE, and MESSAGE_UPDATE
        t: 'MESSAGE_CREATE',
        // ignore messages from self and other bots
        $not: {'d.author.bot': true},
        $or: [
          // DMs
          {'d.guild_id': {$exists: false}},
          // mentions
          {'d.mentions': {$elemMatch: {id: process.env.BOT_USER_ID}}},
          // prefix
          {'d.content': {$regex: '^cookout\\b'}},
        ],
      },
      dst: 'wss://' + process.env.PROJECT_DOMAIN + '.glitch.me/dcc/v1/sample_cookout',
      clientSecret: process.env.DCC_SECRET,
    };
    
  9. Choose a C1 operator. Note: this step is not covered in the completed sample code. Up until now, we’ve been using a development C1 deployment hosted on Glitch at https://daffy-circular-chartreuse.glitch.me. It comes on line when your bot registers, but it doesn’t stay up all day. You should switch to an always-on C1 deployment to keep your bot online.

    To do this, first we’ll stop our program so that it doesn’t re-register while we’re changing this. On Glitch, we’ll comment out the part that registers.

    server.listen(process.env.PORT, () => {
      console.log('listening', process.env.PORT);
      // dcc.register(config).then(() => {
      //   console.log('register ok');
      // }).catch((e) => {
      //   console.error('register failed', e);
      // });
    });
    

    We have to deregister with the previous C1 so that it stops sending us dispatch messages. If we didn’t do this, we would get duplicate dispatch messages. We send a deregistration call to the previous C1:

    curl -f -X DELETE -u "$DCC_SECRET" https://daffy-circular-chartreuse.glitch.me/relays/sample_cookout
    

    When we switch to a different C1 deployment, it’s safest to generate a new client secret and bot token as well:

    # from https://discord.com/developers/applications
    DISCORD_TOKEN=yyyyyyyyyyyyyyyyyyyyyyyy.yyyyyy.yyyyyyyyyyyyyyyyyyyyyyyyyyy
    # from https://discord.com/developers/applications
    BOT_USER_ID=111111111111111111
    # echo "admin:$(head -c16 /dev/urandom | base64)"
    DCC_SECRET=admin:yyyyyyyyyyyyyyyyyyyyyy==
    

    We add an endpoint field to the options passed to register, which for us goes in the config object:

    const config = {
      alias: 'sample_cookout',
      token: process.env.DISCORD_TOKEN,
      intents: eris.Constants.Intents.guildMessages | eris.Constants.Intents.directMessages,
      criteria: {
        // corresponds to what we declared in the gateway, but further filters out messages like
        // READY, CHANNEL_CREATE, and MESSAGE_UPDATE
        t: 'MESSAGE_CREATE',
        // ignore messages from self and other bots
        $not: {'d.author.bot': true},
        $or: [
          // DMs
          {'d.guild_id': {$exists: false}},
          // mentions
          {'d.mentions': {$elemMatch: {id: process.env.BOT_USER_ID}}},
          // prefix
          {'d.content': {$regex: '^cookout\\b'}},
        ],
      },
      dst: 'wss://' + process.env.PROJECT_DOMAIN + '.glitch.me/dcc/v1/sample_cookout',
      clientSecret: process.env.DCC_SECRET,
      // don't literally use the dcc.example.com address below, of course.
      // scroll down for actual providers' endpoints
      endpoint: 'https://dcc.example.com',
    };
    

    Then we reenable registration.

    server.listen(process.env.PORT, () => {
      console.log('listening', process.env.PORT);
      dcc.register(config).then(() => {
        console.log('register ok');
      }).catch((e) => {
        console.error('register failed', e);
      });
    });
    

New always-on C1 deployments

Today I’m posting two C1 deployments:

| Administrator | Hosting provider | Endpoint |
|-|-|-|-|
| @wh0 | Google Cloud https://cloud.google.com/ | https://dcc.wh00.ml |
| @wh0 | Library of Code Library of Code™ | https://dcc.cloud.libraryofcode.org |

My statement as administrator of these deployments:

I won’t look at your private stuff, such as your bot’s Discord token or the dispatch messages. The hosting providers have their own privacy policies. These deployments are “always on” in that the hosting providers have not indicated that they will systematically take these deployments offline. There is no guarantee of actual uptime.

Run your own C1 deployment

I’ve added licenses to the components of this project, so you now formally have permission to use them in your projects.

This also means you can run your own C1 deployment. Get the software here https://github.com/wh0/dc-chartreuse/tree/master/server. If you have any questions about how, I’ll see if I can answer them.

3 Likes

Dude, this is so awesome, how much time did you spend on this?

2 Likes

Probably about an hour on that tutorial post :joy:
Looks really cool @wh0!

1 Like

Thanks. It took like two weeks. I had to learn a lot about Discord and Discord bots.

Thanks. It took like 6 hours to write that tutorial :sob:. But that’s what happens when you don’t leave a lot of comments and then you have to explain your code again.

3 Likes

:open_mouth: :open_mouth: :open_mouth:
:joy:

1 Like

Is it possible to make a bot like this using Discord.js or any of the other discord bot languages or is it using Eris-specific features?

Beautifully written!

A counter reason is the usual use case of remixing to require a different alias.

1 Like

If you need a reliable host lmk, I can let you into my paid hosting plan.

I’ll see what it’s like to use this with Discord.js. The samples I have right now use a so-called “restMode” option on Eris which makes it work without connecting to the gateway. We would need to get a feel for the equivalent in Discord.js

Thanks. I’ve since altered a starter project Glitch :・゚✧ to make it set the alias from the project domain, after all.

Yeah, I think it’ll be good for people to have more choices. I’ll DM you.

2 Likes

Chapter 5. Deploying C1.

New always-on C1 deployment

Today I’m posting one C1 deployment:

Administrator Hosting provider Endpoint
@Anish, @wh0 Sponsored by DiscordLabs.org https://dcc.dlabs.cc

Experiences deploying C1 outside of Glitch

In setting up the above deployment of C1, we’re starting to come up with a set of instructions for it. This has really put the non-vendor-lock-in qualities of Glitch to the test, and it’s pretty good.

  1. Get the code here https://github.com/wh0/dc-chartreuse/tree/master/server

  2. Use npm to install the dependencies.

  3. Run this to set up the .data dir and initialize the database

    mkdir .data && node migrate.js
    
  4. Have a reverse proxy terminate HTTPS and forward to port 3000 (or other, with corresponding adjustment to later steps as necessary)

  5. Have a service manager run PORT=3000 server.js.

Some specific learnings about specific steps:

  • In Phussion Passenger, they have a quickstart guide describing how port binding is all very different, but never explains how to develop an app to work under this very different system. But many links deeper they reveal that the supervisor actually runs your app in an instrumented manner that alters how Node.js’s http.Server.listen behaves. Ultimately, you don’t even set a PORT environment variable, and you can shoot the language designers a “you win this round” glare for making it non-fatal to access a nonexistent environment variable.
  • On some hosts, using SQLite with a file on disk is disjoint from their supported best practices. And migrations? Don’t even get me started.
  • I’m using the following nginx block for reverse proxy on dcc.wh00.ml:
    server {
            listen 443 ssl;
            server_name dcc.wh00.ml;
            ssl_certificate /etc/letsencrypt/live/dcc.wh00.ml/fullchain.pem;
            ssl_certificate_key /etc/letsencrypt/live/dcc.wh00.ml/privkey.pem;
    
            location / {
                    proxy_pass http://localhost:3000;
            }
    }
    server {
            server_name dcc.wh00.ml;
            return 301 https://$host$request_uri;
    }
    
  • Certbot’s nginx installer is just barbaric! I’m running it with webroot.
  • I’m using systemd to supervise on dcc.wh00.ml, with the following service unit file:
    [Install]
    WantedBy=default.target
    
    [Service]
    ExecStart=node server.js
    Restart=always
    
    WorkingDirectory=/opt/dc-chartreuse-server
    User=www-data
    Environment=PORT=3000
    
  • I think it would be nice to check these in to the repo, but there’s way too much specific info like paths in there.

Overall, the non-vendor-lock-in-ness of Glitch is pretty good. You’re more likely to have trouble moving into a more opinionated host.

C1 server v0.1.1

This release corrects a range of bad behaviors when multiple clients register new aliases at the same time.

All deployments that I know about are updated.

4 Likes

This looks awesome! I’m really looking forward to seeing what we can do with discord.js!

Hey, long time no update! The rules of this forum have been updated to prohibit use of Gallery threads as update logs, so I’ll be doing that on Discord instead. For now, I’ll be using the same old test server for updates as well. Here’s the invite to that server again, copied from the first post:

Mixing ostensibly important updates with testing spam sounds like a terrible idea, you say? It is. Sorry about this. I can look into creating separate channels if this actually becomes a problem.

Also, this change is only about not posting “ongoing updates and support.” As I understand it, we can continue discussing the project in general right here.

2 Likes