Downloading files with Python 3

This is an easy script for downloading files from certain URLs using Python 3. You just need to define the url and the output file, urllib and shutil will do the hard work for you.

import urllib
import shutil

url = 'http://www.somewebsite.com/my-image.jpg'
file = '/my-images/my-image.jpg'
with urllib.request.urlopen(url) as response, open(file, 'wb') as out_file:
    shutil.copyfileobj(response, out_file)

Getting a 403 Forbidden error

Some pages have spider/both protection, which means that the remote server is reading the User Agent header from our request and perhaps not authorizing the access. We can easily fix the problem replacing the default user Agent by one used by regular browsers:

    request = urllib.request.Request(url, headers={'User-Agent': 'Mozilla/5.0'})
    with urllib.request.urlopen(request) as response, open(file, 'wb') as out_file:
        shutil.copyfileobj(response, out_file)

Leave a Reply

Your email address will not be published. Required fields are marked *