[TUTORIAL] How to auto-update your project with GitHub

Glitch Tutorials
1

Hi! Recently I asked on the forum whether it is possible to automatically update my project with a pull-request on GitHub? I was answered by cori, by giving me a great idea how you can implement integration with GitHub yourself.

Here is the plan:
Git repo (using webhooks) makes a POST-request to the address {project}.glitch.me/git, and the project executes the .sh script, and then reloads the project with the command refresh.

NOTE: you will need to create remote branch using git remote add origin {url}

To get started, create a project or go to an already created one. You will need the pre-installed express package. We will need to create a path for request - for example, /git.

app.post('/git', (req, res) => {
  // If event is "push"
  if (req.headers['x-github-event'] == "push") { 
    /* Here will be our updating code */
  }

  return res.sendStatus(200); // Send back OK status
});

NOTE: req.headers['x-github-event'] checks if it is a GitHub event and checks if it is “push”

Next, we want to install node-cmd package to run bash commands. Include it somewhere above.

const cmd = require("node-cmd");

Then, let’s update or code for updating block!

if (req.headers['x-github-event'] == "push") {
  cmd.run('chmod 777 git.sh'); /* :/ Fix no perms after updating */
  cmd.get('./git.sh', (err, data) => {  // Run our script
    if (data) console.log(data);
    if (err) console.log(err);
  });
  cmd.run('refresh');  // Refresh project

  console.log("> [GIT] Updated with origin/master");
}

Fill git.sh with this code:

#/bin/sh

# Fetch the newest code
git fetch origin master

# Hard reset
git reset --hard origin/master

# Force pull
git pull origin master --force

That’s all! Next: create new webhook in your repo’s settings.

image
image946×242 34.6 KB

image
image680×638 59.8 KB

Time to test it out! Good luck!

UPD: If you don’t want to let stranger make POST request to refresh your project and, maybe, break something, add some secrets!

image

image

New field appeared!

Then, update code as following:

const crypto = require("crypto"); // pre-installed node package
...
  let hmac = crypto.createHmac("sha1", process.env.SECRET);
  let sig  = "sha1=" + hmac.update(JSON.stringify(req.body)).digest("hex");
  
  if (req.headers['x-github-event'] == "push" && 
      sig == req.headers['x-hub-signature']) {
    ...
  }
}

BONUS
image

let commits = req.body.head_commit.message.split("\n").length == 1 ?
              req.body.head_commit.message :
              req.body.head_commit.message.split("\n").map((el, i) => i !== 0 ? "                       " + el : el).join("\n");
console.log(`> [GIT] Updated with origin/master\n` + 
            `        Latest commit: ${commits}`);
16 Likes
2

Hey @jarvis394,

Thanks for the explanation! :tada:
How could you possibly do the reciprocal with Glitch?
For instance, fetch the GitHub repository and update the code of the project, however, without doing it manually.
If you could possibly create a guide for…
Thanks, anyway.

1 Like
3

Very nice work, @jarvis394, thanks for putting it together!

It might also be worth adding the process for setting up the information to handle the X-Hub-Signature that goes along with the secret on the webhook - this is a way to enable folks implementing this to verify that the request is genuinely coming from GitHub, and isn’t just some random internet person hitting your `/git endpoint.

3 Likes
4

If I understand you correctly, this tutorial has been already made for Glitch. Visit my project (from line 92) to see, how it works.
(i’m just simply using Glitch for hosting, not code-editing, so i’ve made that to automate my work).

1 Like
5

Hello, cori,
Updated, thanks!

1 Like
6

Nice work, though I’ve seen this somewhere idk
You can look into not using node-cmd and using node’s child process to reduce dependencies

2 Likes
7

Nice, tutorial @jarvis394!

I found myself needing to have GitHub and glitch in sync, so I expanded your explanation in this GitHub repo and added a step-by-step guide.

5 Likes
8

Very cool. Will test it out. Will it delete my gitignored files (like .env) or no

2 Likes
9

@WilsontheWolf, no, git pull only updates files, that were in git repo

1 Like
10

this is awesome ! following the steps will make my comments later

2 Likes
11

Dude this is exactly what I needed, thanks so much for writing it!

1 Like
12

Is there an tutorial for auto-updating projects with GitLab or both GitHub and GitLab?

13

Does GitLab repos have a webhooks feature? You can do the same with it then

1 Like
14

If you’re using GitLab, our help article about Repo mirroring might be of interest.

1 Like
15

What if GitHub received the 404 response? I configured it like this one on my git-deploys.js on my project. Any tips regarding the 404 error bug?

// In this code uses Express and Node-CMD packages to auto-update your Glitch project
// with either GitHub, GitLab or both of them.
// See https://support.glitch.com/t/tutorial-how-to-auto-update-your-project-with-github/8124 for
// detailed help.

const cmd = require("node-cmd");
const express =  require("express");
const crypto = require("crypto"); // NPM Package "crypto" is pre-installed, so forget about digging search results again.
const app = express()

app.post('/git', (req, res) => {
  let hmac = crypto.createHmac("sha1", process.env.GitHub_webhookSecret);
  let sig  = "sha1=" + hmac.update(JSON.stringify(req.body)).digest("hex");
  
  if (req.headers['x-github-event'] == "push") {
      cmd.run('chmod 777 github.sh'); /* :/ Fix no perms after updating */
      cmd.get('./deploy/github.sh', (err, data) => {  // Run our script
        if (data) console.log(data);
        if (err) console.log(err);
    });
  cmd.run('refresh');  // Refresh project
  let commits = req.body.head_commit.message.split("\n").length == 1 ?
              req.body.head_commit.message :
              req.body.head_commit.message.split("\n").map((el, i) => i !== 0 ? "                       " + el : el).join("\n");
  console.log(`> [GIT] Source code updated with github:MadeByThePinsHub/RecapTime-ProbotApp/master\n` + 
            `        Latest commit: ${commits}`);
  }
  return res.sendStatus(200).json({ status: 200, description: 'GitHub webhook message received, waiting for Glitch to update source code...'}); // Send back OK status
})

app.get('/git', (req, res) => {
  cmd.run('chmod 777 github.sh'); /* :/ Fix no perms after updating */
  cmd.get('./deploy/github.sh', (err, data) => {  // Run our script
    if (data) console.log(data);
    if (err) console.log(err);
    });
  cmd.run('refresh');  // Refresh project
  let commits = req.body.head_commit.message.split("\n").length == 1 ?
              req.body.head_commit.message :
              req.body.head_commit.message.split("\n").map((el, i) => i !== 0 ? "                       " + el : el).join("\n");
  console.log(`> [GIT] Source code updated with github:MadeByThePinsHub/RecapTime-ProbotApp/master\n` + 
            `        Latest commit: ${commits}`);
  return res.sendStatus(200).json({ status: 200, description: 'We received your GET request, waiting for Glitch to update source code...'}); // Send back OK status
})
16

Hey @ajhalili2006 where are you receiving the 404 message, and what does the GitHub address or message that receives the 404 response look like? If you execute the underlying pieces of code do you get the same response (i.e. if you run ./deploy/github.sh in the console do you see a different message than when your Express code runs the command? If so, what’s the difference in how they’re run?)

17

Something like this on the body…

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Error</title>
</head>
<body>
<pre>Cannot POST /webhooks/github/MaglubayTest123DHusdg</pre>
</body>
</html>

…and on the headers (not required, but I added here for details)…

Connection: keep-alive
Content-Length: 177
content-security-policy: default-src &#39;none&#39;
Content-Type: text/html; charset=utf-8
Date: Sat, 21 Sep 2019 07:34:54 GMT
x-content-type-options: nosniff
x-powered-by: Express
x-request-id: ef4447bbe81a3324

Speaking of the request headers and body…

Request URL: https://recaptime-probotapp.glitch.me/webhooks/github/MaglubayTest123DHusdg
Request method: POST
content-type: application/json
Expect: 
User-Agent: GitHub-Hookshot/acf94e7
X-GitHub-Delivery: 4d9481a6-dc42-11e9-9b41-89fe5d2bacac
X-GitHub-Event: push

{
  "ref": "refs/heads/master",
  "before": "ec8345e5cecd9dfc3fed54dcf47a229a933e409a",
  "after": "0ba72bbae46e66dda695f4fc65ff9b5bee7b5fc2",
  "repository": {
    "id": 208974529,
    "node_id": "MDEwOlJlcG9zaXRvcnkyMDg5NzQ1Mjk=",
    "name": "RecapTime-ProbotApp",
    "full_name": "MadeByThePinsHub/RecapTime-ProbotApp",
    "private": false,
    "owner": {
      "name": "MadeByThePinsHub",
      "email": "support@supportcentral-madebythepins.freshdesk.com",
      "login": "MadeByThePinsHub",
      "id": 34394576,
      "node_id": "MDEyOk9yZ2FuaXphdGlvbjM0Mzk0NTc2",
      "avatar_url": "https://avatars3.githubusercontent.com/u/34394576?v=4",
      "gravatar_id": "",
      "url": "https://api.github.com/users/MadeByThePinsHub",
      "html_url": "https://github.com/MadeByThePinsHub",
      "followers_url": "https://api.github.com/users/MadeByThePinsHub/followers",
      "following_url": "https://api.github.com/users/MadeByThePinsHub/following{/other_user}",
      "gists_url": "https://api.github.com/users/MadeByThePinsHub/gists{/gist_id}",
      "starred_url": "https://api.github.com/users/MadeByThePinsHub/starred{/owner}{/repo}",
      "subscriptions_url": "https://api.github.com/users/MadeByThePinsHub/subscriptions",
      "organizations_url": "https://api.github.com/users/MadeByThePinsHub/orgs",
      "repos_url": "https://api.github.com/users/MadeByThePinsHub/repos",
      "events_url": "https://api.github.com/users/MadeByThePinsHub/events{/privacy}",
      "received_events_url": "https://api.github.com/users/MadeByThePinsHub/received_events",
      "type": "Organization",
      "site_admin": false
    },
    "html_url": "https://github.com/MadeByThePinsHub/RecapTime-ProbotApp",
    "description": "The official GitHub mirror of Recap Time Telegram bot repository from Glitch and deployed thru GitLab.",
    "fork": false,
    "url": "https://github.com/MadeByThePinsHub/RecapTime-ProbotApp",
    "forks_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/forks",
    "keys_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/keys{/key_id}",
    "collaborators_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/collaborators{/collaborator}",
    "teams_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/teams",
    "hooks_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/hooks",
    "issue_events_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/issues/events{/number}",
    "events_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/events",
    "assignees_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/assignees{/user}",
    "branches_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/branches{/branch}",
    "tags_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/tags",
    "blobs_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/git/blobs{/sha}",
    "git_tags_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/git/tags{/sha}",
    "git_refs_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/git/refs{/sha}",
    "trees_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/git/trees{/sha}",
    "statuses_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/statuses/{sha}",
    "languages_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/languages",
    "stargazers_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/stargazers",
    "contributors_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/contributors",
    "subscribers_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/subscribers",
    "subscription_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/subscription",
    "commits_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/commits{/sha}",
    "git_commits_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/git/commits{/sha}",
    "comments_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/comments{/number}",
    "issue_comment_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/issues/comments{/number}",
    "contents_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/contents/{+path}",
    "compare_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/compare/{base}...{head}",
    "merges_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/merges",
    "archive_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/{archive_format}{/ref}",
    "downloads_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/downloads",
    "issues_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/issues{/number}",
    "pulls_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/pulls{/number}",
    "milestones_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/milestones{/number}",
    "notifications_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/notifications{?since,all,participating}",
    "labels_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/labels{/name}",
    "releases_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/releases{/id}",
    "deployments_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/deployments",
    "created_at": 1568701244,
    "updated_at": "2019-09-21T07:20:21Z",
    "pushed_at": 1569051293,
    "git_url": "git://github.com/MadeByThePinsHub/RecapTime-ProbotApp.git",
    "ssh_url": "git@github.com:MadeByThePinsHub/RecapTime-ProbotApp.git",
    "clone_url": "https://github.com/MadeByThePinsHub/RecapTime-ProbotApp.git",
    "svn_url": "https://github.com/MadeByThePinsHub/RecapTime-ProbotApp",
    "homepage": "",
    "size": 56,
    "stargazers_count": 0,
    "watchers_count": 0,
    "language": "JavaScript",
    "has_issues": true,
    "has_projects": true,
    "has_downloads": true,
    "has_wiki": true,
    "has_pages": false,
    "forks_count": 0,
    "mirror_url": null,
    "archived": false,
    "disabled": false,
    "open_issues_count": 0,
    "license": {
      "key": "mit",
      "name": "MIT License",
      "spdx_id": "MIT",
      "url": "https://api.github.com/licenses/mit",
      "node_id": "MDc6TGljZW5zZTEz"
    },
    "forks": 0,
    "open_issues": 0,
    "watchers": 0,
    "default_branch": "master",
    "stargazers": 0,
    "master_branch": "master",
    "organization": "MadeByThePinsHub"
  },
  "pusher": {
    "name": "AndreiJirohHaliliDev2006",
    "email": "andreijiroheugeniohalili24680@gmail.com"
  },
  "organization": {
    "login": "MadeByThePinsHub",
    "id": 34394576,
    "node_id": "MDEyOk9yZ2FuaXphdGlvbjM0Mzk0NTc2",
    "url": "https://api.github.com/orgs/MadeByThePinsHub",
    "repos_url": "https://api.github.com/orgs/MadeByThePinsHub/repos",
    "events_url": "https://api.github.com/orgs/MadeByThePinsHub/events",
    "hooks_url": "https://api.github.com/orgs/MadeByThePinsHub/hooks",
    "issues_url": "https://api.github.com/orgs/MadeByThePinsHub/issues",
    "members_url": "https://api.github.com/orgs/MadeByThePinsHub/members{/member}",
    "public_members_url": "https://api.github.com/orgs/MadeByThePinsHub/public_members{/member}",
    "avatar_url": "https://avatars3.githubusercontent.com/u/34394576?v=4",
    "description": "The official org of Malas Pins on GitHub made by @AndreiJirohHaliliDev2006. Our dev website and email address will be available soon in the near future."
  },
  "sender": {
    "login": "AndreiJirohHaliliDev2006",
    "id": 34998342,
    "node_id": "MDQ6VXNlcjM0OTk4MzQy",
    "avatar_url": "https://avatars1.githubusercontent.com/u/34998342?v=4",
    "gravatar_id": "",
    "url": "https://api.github.com/users/AndreiJirohHaliliDev2006",
    "html_url": "https://github.com/AndreiJirohHaliliDev2006",
    "followers_url": "https://api.github.com/users/AndreiJirohHaliliDev2006/followers",
    "following_url": "https://api.github.com/users/AndreiJirohHaliliDev2006/following{/other_user}",
    "gists_url": "https://api.github.com/users/AndreiJirohHaliliDev2006/gists{/gist_id}",
    "starred_url": "https://api.github.com/users/AndreiJirohHaliliDev2006/starred{/owner}{/repo}",
    "subscriptions_url": "https://api.github.com/users/AndreiJirohHaliliDev2006/subscriptions",
    "organizations_url": "https://api.github.com/users/AndreiJirohHaliliDev2006/orgs",
    "repos_url": "https://api.github.com/users/AndreiJirohHaliliDev2006/repos",
    "events_url": "https://api.github.com/users/AndreiJirohHaliliDev2006/events{/privacy}",
    "received_events_url": "https://api.github.com/users/AndreiJirohHaliliDev2006/received_events",
    "type": "User",
    "site_admin": false
  },
  "created": false,
  "deleted": false,
  "forced": false,
  "base_ref": null,
  "compare": "https://github.com/MadeByThePinsHub/RecapTime-ProbotApp/compare/ec8345e5cecd...0ba72bbae46e",
  "commits": [
    {
      "id": "0ba72bbae46e66dda695f4fc65ff9b5bee7b5fc2",
      "tree_id": "238d2615fc50858cd9ca75e922b15f2a144bc1b0",
      "distinct": true,
      "message": "☕️👬 Checkpoint\n./website.js:4790460/24",
      "timestamp": "2019-09-21T07:31:56Z",
      "url": "https://github.com/MadeByThePinsHub/RecapTime-ProbotApp/commit/0ba72bbae46e66dda695f4fc65ff9b5bee7b5fc2",
      "author": {
        "name": "Glitch (recaptime-probotapp)",
        "email": "none"
      },
      "committer": {
        "name": "Glitch (recaptime-probotapp)",
        "email": "none"
      },
      "added": [

      ],
      "removed": [

      ],
      "modified": [
        "website.js"
      ]
    }
  ],
  "head_commit": {
    "id": "0ba72bbae46e66dda695f4fc65ff9b5bee7b5fc2",
    "tree_id": "238d2615fc50858cd9ca75e922b15f2a144bc1b0",
    "distinct": true,
    "message": "☕️👬 Checkpoint\n./website.js:4790460/24",
    "timestamp": "2019-09-21T07:31:56Z",
    "url": "https://github.com/MadeByThePinsHub/RecapTime-ProbotApp/commit/0ba72bbae46e66dda695f4fc65ff9b5bee7b5fc2",
    "author": {
      "name": "Glitch (recaptime-probotapp)",
      "email": "none"
    },
    "committer": {
      "name": "Glitch (recaptime-probotapp)",
      "email": "none"
    },
    "added": [

    ],
    "removed": [

    ],
    "modified": [
      "website.js"
    ]
  }
}
18

GitLab is doing request on /webhooks/github..., not /git. Change the address in your repository settings!

1 Like
19

i just need this in reverse direction now xD

20

You can make up cron service executing git add . | git commit -m "Checkpoint" | git push origin glitch