https should work fine without any further changes - fly.io and Custom Domains supports it out of the box.
Unfortunately there’s no good way to force https using Glitch Custom Domains; if you want to do that you’d have to set up a fly.io app manually, as outlined in custom domain on fly.io.