[TUTORIAL] How to auto-update your project with GitHub

jarvis394 · February 3, 2019, 2:21pm

Hi! Recently I asked on the forum whether it is possible to automatically update my project with a pull-request on GitHub? I was answered by cori, by giving me a great idea how you can implement integration with GitHub yourself.

Here is the plan:
Git repo (using webhooks) makes a POST-request to the address {project}.glitch.me/git, and the project executes the .sh script, and then reloads the project with the command refresh.

NOTE: you will need to create remote branch using git remote add origin {url}

To get started, create a project or go to an already created one. You will need the pre-installed express package. We will need to create a path for request - for example, /git.

app.post('/git', (req, res) => {
  // If event is "push"
  if (req.headers['x-github-event'] == "push") { 
    /* Here will be our updating code */
  }

  return res.sendStatus(200); // Send back OK status
});

NOTE: req.headers['x-github-event'] checks if it is a GitHub event and checks if it is “push”

Next, we want to install node-cmd package to run bash commands. Include it somewhere above.

const cmd = require("node-cmd");

Then, let’s update or code for updating block!

if (req.headers['x-github-event'] == "push") {
  cmd.run('chmod 777 git.sh'); /* :/ Fix no perms after updating */
  cmd.get('./git.sh', (err, data) => {  // Run our script
    if (data) console.log(data);
    if (err) console.log(err);
  });
  cmd.run('refresh');  // Refresh project

  console.log("> [GIT] Updated with origin/master");
}

Fill git.sh with this code:

#/bin/sh

# Fetch the newest code
git fetch origin master

# Hard reset
git reset --hard origin/master

# Force pull
git pull origin master --force

That’s all! Next: create new webhook in your repo’s settings.

Time to test it out! Good luck!

UPD: If you don’t want to let stranger make POST request to refresh your project and, maybe, break something, add some secrets!

New field appeared!

Then, update code as following:

const crypto = require("crypto"); // pre-installed node package
...
  let hmac = crypto.createHmac("sha1", process.env.SECRET);
  let sig  = "sha1=" + hmac.update(JSON.stringify(req.body)).digest("hex");
  
  if (req.headers['x-github-event'] == "push" && 
      sig == req.headers['x-hub-signature']) {
    ...
  }
}

BONUS

let commits = req.body.head_commit.message.split("\n").length == 1 ?
              req.body.head_commit.message :
              req.body.head_commit.message.split("\n").map((el, i) => i !== 0 ? "                       " + el : el).join("\n");
console.log(`> [GIT] Updated with origin/master\n` + 
            `        Latest commit: ${commits}`);

chroventer · February 3, 2019, 3:22pm

Hey @jarvis394,

Thanks for the explanation!
How could you possibly do the reciprocal with Glitch?
For instance, fetch the GitHub repository and update the code of the project, however, without doing it manually.
If you could possibly create a guide for…
Thanks, anyway.

cori · February 3, 2019, 3:36pm

Very nice work, @jarvis394, thanks for putting it together!

It might also be worth adding the process for setting up the information to handle the X-Hub-Signature that goes along with the secret on the webhook - this is a way to enable folks implementing this to verify that the request is genuinely coming from GitHub, and isn’t just some random internet person hitting your `/git endpoint.

jarvis394 · February 3, 2019, 4:06pm

If I understand you correctly, this tutorial has been already made for Glitch. Visit my project (from line 92) to see, how it works.
(i’m just simply using Glitch for hosting, not code-editing, so i’ve made that to automate my work).

jarvis394 · February 3, 2019, 4:15pm

Hello, cori,
Updated, thanks!

Excigma · February 6, 2019, 10:58am

Nice work, though I’ve seen this somewhere idk
You can look into not using node-cmd and using node’s child process to reduce dependencies

javier · March 19, 2019, 9:42am

Nice, tutorial @jarvis394!

I found myself needing to have GitHub and glitch in sync, so I expanded your explanation in this GitHub repo and added a step-by-step guide.

WilsontheWolf · April 6, 2019, 9:53pm

Very cool. Will test it out. Will it delete my gitignored files (like .env) or no

jarvis394 · April 7, 2019, 7:57am

@WilsontheWolf, no, git pull only updates files, that were in git repo

lucasverra · April 7, 2019, 4:33pm

this is awesome ! following the steps will make my comments later

shadowdevelopment · August 10, 2019, 6:54pm

Dude this is exactly what I needed, thanks so much for writing it!

AJHaliliDev2006PH · September 18, 2019, 6:05am

Is there an tutorial for auto-updating projects with GitLab or both GitHub and GitLab?

jarvis394 · September 18, 2019, 6:58am

Does GitLab repos have a webhooks feature? You can do the same with it then

cori · September 18, 2019, 2:15pm

If you’re using GitLab, our help article about Repo mirroring might be of interest.

AJHaliliDev2006PH · September 20, 2019, 10:38am

What if GitHub received the 404 response? I configured it like this one on my git-deploys.js on my project. Any tips regarding the 404 error bug?

// In this code uses Express and Node-CMD packages to auto-update your Glitch project
// with either GitHub, GitLab or both of them.
// See https://support.glitch.com/t/tutorial-how-to-auto-update-your-project-with-github/8124 for
// detailed help.

const cmd = require("node-cmd");
const express =  require("express");
const crypto = require("crypto"); // NPM Package "crypto" is pre-installed, so forget about digging search results again.
const app = express()

app.post('/git', (req, res) => {
  let hmac = crypto.createHmac("sha1", process.env.GitHub_webhookSecret);
  let sig  = "sha1=" + hmac.update(JSON.stringify(req.body)).digest("hex");
  
  if (req.headers['x-github-event'] == "push") {
      cmd.run('chmod 777 github.sh'); /* :/ Fix no perms after updating */
      cmd.get('./deploy/github.sh', (err, data) => {  // Run our script
        if (data) console.log(data);
        if (err) console.log(err);
    });
  cmd.run('refresh');  // Refresh project
  let commits = req.body.head_commit.message.split("\n").length == 1 ?
              req.body.head_commit.message :
              req.body.head_commit.message.split("\n").map((el, i) => i !== 0 ? "                       " + el : el).join("\n");
  console.log(`> [GIT] Source code updated with github:MadeByThePinsHub/RecapTime-ProbotApp/master\n` + 
            `        Latest commit: ${commits}`);
  }
  return res.sendStatus(200).json({ status: 200, description: 'GitHub webhook message received, waiting for Glitch to update source code...'}); // Send back OK status
})

app.get('/git', (req, res) => {
  cmd.run('chmod 777 github.sh'); /* :/ Fix no perms after updating */
  cmd.get('./deploy/github.sh', (err, data) => {  // Run our script
    if (data) console.log(data);
    if (err) console.log(err);
    });
  cmd.run('refresh');  // Refresh project
  let commits = req.body.head_commit.message.split("\n").length == 1 ?
              req.body.head_commit.message :
              req.body.head_commit.message.split("\n").map((el, i) => i !== 0 ? "                       " + el : el).join("\n");
  console.log(`> [GIT] Source code updated with github:MadeByThePinsHub/RecapTime-ProbotApp/master\n` + 
            `        Latest commit: ${commits}`);
  return res.sendStatus(200).json({ status: 200, description: 'We received your GET request, waiting for Glitch to update source code...'}); // Send back OK status
})

cori · September 20, 2019, 1:45pm

Hey @AJHaliliDev2006PH where are you receiving the 404 message, and what does the GitHub address or message that receives the 404 response look like? If you execute the underlying pieces of code do you get the same response (i.e. if you run ./deploy/github.sh in the console do you see a different message than when your Express code runs the command? If so, what’s the difference in how they’re run?)

AJHaliliDev2006PH · September 21, 2019, 7:40am

Something like this on the body…

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Error</title>
</head>
<body>
<pre>Cannot POST /webhooks/github/MaglubayTest123DHusdg</pre>
</body>
</html>

…and on the headers (not required, but I added here for details)…

Connection: keep-alive
Content-Length: 177
content-security-policy: default-src &#39;none&#39;
Content-Type: text/html; charset=utf-8
Date: Sat, 21 Sep 2019 07:34:54 GMT
x-content-type-options: nosniff
x-powered-by: Express
x-request-id: ef4447bbe81a3324

Speaking of the request headers and body…

Request URL: https://recaptime-probotapp.glitch.me/webhooks/github/MaglubayTest123DHusdg
Request method: POST
content-type: application/json
Expect: 
User-Agent: GitHub-Hookshot/acf94e7
X-GitHub-Delivery: 4d9481a6-dc42-11e9-9b41-89fe5d2bacac
X-GitHub-Event: push

{
  "ref": "refs/heads/master",
  "before": "ec8345e5cecd9dfc3fed54dcf47a229a933e409a",
  "after": "0ba72bbae46e66dda695f4fc65ff9b5bee7b5fc2",
  "repository": {
    "id": 208974529,
    "node_id": "MDEwOlJlcG9zaXRvcnkyMDg5NzQ1Mjk=",
    "name": "RecapTime-ProbotApp",
    "full_name": "MadeByThePinsHub/RecapTime-ProbotApp",
    "private": false,
    "owner": {
      "name": "MadeByThePinsHub",
      "email": "support@supportcentral-madebythepins.freshdesk.com",
      "login": "MadeByThePinsHub",
      "id": 34394576,
      "node_id": "MDEyOk9yZ2FuaXphdGlvbjM0Mzk0NTc2",
      "avatar_url": "https://avatars3.githubusercontent.com/u/34394576?v=4",
      "gravatar_id": "",
      "url": "https://api.github.com/users/MadeByThePinsHub",
      "html_url": "https://github.com/MadeByThePinsHub",
      "followers_url": "https://api.github.com/users/MadeByThePinsHub/followers",
      "following_url": "https://api.github.com/users/MadeByThePinsHub/following{/other_user}",
      "gists_url": "https://api.github.com/users/MadeByThePinsHub/gists{/gist_id}",
      "starred_url": "https://api.github.com/users/MadeByThePinsHub/starred{/owner}{/repo}",
      "subscriptions_url": "https://api.github.com/users/MadeByThePinsHub/subscriptions",
      "organizations_url": "https://api.github.com/users/MadeByThePinsHub/orgs",
      "repos_url": "https://api.github.com/users/MadeByThePinsHub/repos",
      "events_url": "https://api.github.com/users/MadeByThePinsHub/events{/privacy}",
      "received_events_url": "https://api.github.com/users/MadeByThePinsHub/received_events",
      "type": "Organization",
      "site_admin": false
    },
    "html_url": "https://github.com/MadeByThePinsHub/RecapTime-ProbotApp",
    "description": "The official GitHub mirror of Recap Time Telegram bot repository from Glitch and deployed thru GitLab.",
    "fork": false,
    "url": "https://github.com/MadeByThePinsHub/RecapTime-ProbotApp",
    "forks_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/forks",
    "keys_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/keys{/key_id}",
    "collaborators_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/collaborators{/collaborator}",
    "teams_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/teams",
    "hooks_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/hooks",
    "issue_events_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/issues/events{/number}",
    "events_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/events",
    "assignees_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/assignees{/user}",
    "branches_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/branches{/branch}",
    "tags_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/tags",
    "blobs_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/git/blobs{/sha}",
    "git_tags_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/git/tags{/sha}",
    "git_refs_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/git/refs{/sha}",
    "trees_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/git/trees{/sha}",
    "statuses_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/statuses/{sha}",
    "languages_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/languages",
    "stargazers_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/stargazers",
    "contributors_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/contributors",
    "subscribers_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/subscribers",
    "subscription_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/subscription",
    "commits_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/commits{/sha}",
    "git_commits_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/git/commits{/sha}",
    "comments_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/comments{/number}",
    "issue_comment_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/issues/comments{/number}",
    "contents_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/contents/{+path}",
    "compare_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/compare/{base}...{head}",
    "merges_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/merges",
    "archive_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/{archive_format}{/ref}",
    "downloads_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/downloads",
    "issues_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/issues{/number}",
    "pulls_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/pulls{/number}",
    "milestones_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/milestones{/number}",
    "notifications_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/notifications{?since,all,participating}",
    "labels_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/labels{/name}",
    "releases_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/releases{/id}",
    "deployments_url": "https://api.github.com/repos/MadeByThePinsHub/RecapTime-ProbotApp/deployments",
    "created_at": 1568701244,
    "updated_at": "2019-09-21T07:20:21Z",
    "pushed_at": 1569051293,
    "git_url": "git://github.com/MadeByThePinsHub/RecapTime-ProbotApp.git",
    "ssh_url": "git@github.com:MadeByThePinsHub/RecapTime-ProbotApp.git",
    "clone_url": "https://github.com/MadeByThePinsHub/RecapTime-ProbotApp.git",
    "svn_url": "https://github.com/MadeByThePinsHub/RecapTime-ProbotApp",
    "homepage": "",
    "size": 56,
    "stargazers_count": 0,
    "watchers_count": 0,
    "language": "JavaScript",
    "has_issues": true,
    "has_projects": true,
    "has_downloads": true,
    "has_wiki": true,
    "has_pages": false,
    "forks_count": 0,
    "mirror_url": null,
    "archived": false,
    "disabled": false,
    "open_issues_count": 0,
    "license": {
      "key": "mit",
      "name": "MIT License",
      "spdx_id": "MIT",
      "url": "https://api.github.com/licenses/mit",
      "node_id": "MDc6TGljZW5zZTEz"
    },
    "forks": 0,
    "open_issues": 0,
    "watchers": 0,
    "default_branch": "master",
    "stargazers": 0,
    "master_branch": "master",
    "organization": "MadeByThePinsHub"
  },
  "pusher": {
    "name": "AndreiJirohHaliliDev2006",
    "email": "andreijiroheugeniohalili24680@gmail.com"
  },
  "organization": {
    "login": "MadeByThePinsHub",
    "id": 34394576,
    "node_id": "MDEyOk9yZ2FuaXphdGlvbjM0Mzk0NTc2",
    "url": "https://api.github.com/orgs/MadeByThePinsHub",
    "repos_url": "https://api.github.com/orgs/MadeByThePinsHub/repos",
    "events_url": "https://api.github.com/orgs/MadeByThePinsHub/events",
    "hooks_url": "https://api.github.com/orgs/MadeByThePinsHub/hooks",
    "issues_url": "https://api.github.com/orgs/MadeByThePinsHub/issues",
    "members_url": "https://api.github.com/orgs/MadeByThePinsHub/members{/member}",
    "public_members_url": "https://api.github.com/orgs/MadeByThePinsHub/public_members{/member}",
    "avatar_url": "https://avatars3.githubusercontent.com/u/34394576?v=4",
    "description": "The official org of Malas Pins on GitHub made by @AndreiJirohHaliliDev2006. Our dev website and email address will be available soon in the near future."
  },
  "sender": {
    "login": "AndreiJirohHaliliDev2006",
    "id": 34998342,
    "node_id": "MDQ6VXNlcjM0OTk4MzQy",
    "avatar_url": "https://avatars1.githubusercontent.com/u/34998342?v=4",
    "gravatar_id": "",
    "url": "https://api.github.com/users/AndreiJirohHaliliDev2006",
    "html_url": "https://github.com/AndreiJirohHaliliDev2006",
    "followers_url": "https://api.github.com/users/AndreiJirohHaliliDev2006/followers",
    "following_url": "https://api.github.com/users/AndreiJirohHaliliDev2006/following{/other_user}",
    "gists_url": "https://api.github.com/users/AndreiJirohHaliliDev2006/gists{/gist_id}",
    "starred_url": "https://api.github.com/users/AndreiJirohHaliliDev2006/starred{/owner}{/repo}",
    "subscriptions_url": "https://api.github.com/users/AndreiJirohHaliliDev2006/subscriptions",
    "organizations_url": "https://api.github.com/users/AndreiJirohHaliliDev2006/orgs",
    "repos_url": "https://api.github.com/users/AndreiJirohHaliliDev2006/repos",
    "events_url": "https://api.github.com/users/AndreiJirohHaliliDev2006/events{/privacy}",
    "received_events_url": "https://api.github.com/users/AndreiJirohHaliliDev2006/received_events",
    "type": "User",
    "site_admin": false
  },
  "created": false,
  "deleted": false,
  "forced": false,
  "base_ref": null,
  "compare": "https://github.com/MadeByThePinsHub/RecapTime-ProbotApp/compare/ec8345e5cecd...0ba72bbae46e",
  "commits": [
    {
      "id": "0ba72bbae46e66dda695f4fc65ff9b5bee7b5fc2",
      "tree_id": "238d2615fc50858cd9ca75e922b15f2a144bc1b0",
      "distinct": true,
      "message": "☕️👬 Checkpoint\n./website.js:4790460/24",
      "timestamp": "2019-09-21T07:31:56Z",
      "url": "https://github.com/MadeByThePinsHub/RecapTime-ProbotApp/commit/0ba72bbae46e66dda695f4fc65ff9b5bee7b5fc2",
      "author": {
        "name": "Glitch (recaptime-probotapp)",
        "email": "none"
      },
      "committer": {
        "name": "Glitch (recaptime-probotapp)",
        "email": "none"
      },
      "added": [

      ],
      "removed": [

      ],
      "modified": [
        "website.js"
      ]
    }
  ],
  "head_commit": {
    "id": "0ba72bbae46e66dda695f4fc65ff9b5bee7b5fc2",
    "tree_id": "238d2615fc50858cd9ca75e922b15f2a144bc1b0",
    "distinct": true,
    "message": "☕️👬 Checkpoint\n./website.js:4790460/24",
    "timestamp": "2019-09-21T07:31:56Z",
    "url": "https://github.com/MadeByThePinsHub/RecapTime-ProbotApp/commit/0ba72bbae46e66dda695f4fc65ff9b5bee7b5fc2",
    "author": {
      "name": "Glitch (recaptime-probotapp)",
      "email": "none"
    },
    "committer": {
      "name": "Glitch (recaptime-probotapp)",
      "email": "none"
    },
    "added": [

    ],
    "removed": [

    ],
    "modified": [
      "website.js"
    ]
  }
}

jarvis394 · September 21, 2019, 12:30pm

GitLab is doing request on /webhooks/github..., not /git. Change the address in your repository settings!

blubbll · September 26, 2019, 6:19am

i just need this in reverse direction now xD

jarvis394 · September 26, 2019, 7:20pm

You can make up cron service executing git add . | git commit -m "Checkpoint" | git push origin glitch

Topic		Replies	Views
Update changes on glitch Glitch Help	2	621	October 1, 2021
Update glitch project in VSCode Glitch Help	1	873	June 2, 2019
Will glitch deploy master branch automatically when master branch is updated? Glitch Help javascript , nodejs , express	3	718	April 23, 2021
Automated Github Deploys Glitch Feature Ideas	6	1948	April 5, 2019
Automatically update project from GitHub Glitch Feature Ideas	3	895	April 8, 2017

[TUTORIAL] How to auto-update your project with GitHub

Related Topics