Upload files to S3 with Python (keeping the original folder structure )

This is a sample script for uploading multiple files to S3 keeping the original folder structure. Doing this manually can be a bit tedious, specially if there are many files to upload located in different folders. This code will do the hard work for you, just call the function upload_files('/path/to/my/folder'). The param of the function must be the path of the folder containing the files in your local machine.

Install Boto3

You will need to install Boto3 first:

pip install boto3


import boto3
import os

def upload_files(path):
    session = boto3.Session(
    s3 = session.resource('s3')
    bucket = s3.Bucket('YOUR_BUCKET_NAME')

    for subdir, dirs, files in os.walk(path):
        for file in files:
            full_path = os.path.join(subdir, file)
            with open(full_path, 'rb') as data:
                bucket.put_object(Key=full_path[len(path)+1:], Body=data)

if __name__ == "__main__":

The script will ignore the local path when creating the resources on S3, for example if we execute upload_files('/my_data') having the following structure:


The resulting structure on S3 will be:


Leave a Reply

Your email address will not be published. Required fields are marked *