How do I open an FTP directory so that I can use it the same way I could a normal directory?

I am trying to make an application that downloads something and puts it on an FTP server. There is not enough space to, even temporarily, put those files that are being downloaded on my computer, so I want them to download into the FTP server.

Specifically, I am using pytube to download a video using:

The .download() can hold a path variable, though I do not know how to use it to send the FTP server this way. I am looking for a way to open an FTP directory, so that I can use the path to fill in there.

Any help would be greatly appreciated.

Does this help?

In glitch you can download them to the /tmp folder which has a higher storage limit than the /app directory. What you want to do is download a chunk of video from youtube, then append to the file on the remote ftp server.
Here’s a stackoverflow post that describes in much more detail how to upload in chunks

I don’t know much about pytube but you can also get the raw url of a youtube video through libraries like pafy or youtube_dl. However if you want to stick with pytube I’ll give you this link to the documentation

It looks like you can override the on progress function to instead upload to the ftp server that way, or append to a queue of chunks to be consumed by the ftp server
1 Like

Also another intresting solution but utilizing curl, if you can output the video source into stdout, you can pipe it into curl which can upload it into the FTP server.

Hey @SinglePaper! @javaarchive is right overriding on_progress and pushing onto a queue to send to the FTP server is probably the best option, sounded fun so I gave it a shot, below seems to work ok :).

import io
from threading import Thread
from pytube import YouTube
from ftplib import FTP, error_temp
from queue import Queue

class BytePipe:

    def __init__(self, size: int):
        self.q = Queue()
        self.size = size
        self.bytes_read = 0

    def read(self, blocksize: int):
        # if we have piped the whole file return False to ftplib
        if self.bytes_read == self.size:
            return False

        chunk = self.q.get(block=True)
        self.bytes_read += len(chunk)
        return chunk

    def write(self, chunk: bytes):
        self.q.put(chunk, block=True)

if __name__ == "__main__":
    ftp = FTP("")

    gangnam = YouTube('')
    stream = gangnam.streams.first()

    pipe = BytePipe(stream.filesize)

    # warning, hack begins here :)
    # override the on_progress of Stream object and pipe chunks to ftp
    stream.on_progress = lambda chunk, fh, br: pipe.write(chunk)

    with io.BytesIO() as buffer:  # we don't really need this buffer - we're piping straight to ftp - but create one anyway because pytube expects one
        t = Thread(target=stream.stream_to_buffer, args=(buffer,))

        # retry on temporary errors
        while True:
                # set the same block size that pytube defaults to
                ftp.storbinary("STOR file.txt", pipe, blocksize=9437184)
            except error_temp as e:
                print("temp error, retrying...")

1 Like