In web development, moving from staging to production servers, it’s useful to check that staging is pointing to production after the DNS changes. Simple to check all urls if the site you’re working on only has few pages. A simple curl -I http://your.url.here.com will work just fine.
But what if you have a hundred or more pages to check? If I was paid by the hour, yeah I would copy/paste each url into a terminal and check, but I’m salary. So here’s a quick python script to check them for me.
You need to provide it a text file with all the urls on their own line. Easy if the urls are in a spread sheet, simple copy/paste into the text file and off to the races. This code will account for white-space and empty lines, however, it will not account for missing http/https prepends. So far testing shows compatible with Python2 and Python3. Requires the requests library.
# requires requests lib
# - pip install requests
import requests
import sys
def redirectTest(url):
with open("no_redirects.txt", "a") as no_redirect:
try:
r = requests.head(url, allow_redirects=False)
if (r.status_code == 301):
print("+ %s :: %d" % (url, r.status_code))
else:
print("- WARNING: %s :: %d" % (url, r.status_code))
no_redirect.write("%s :: %d\n" % (url, r.status_code))
except requests.exceptions.RequestException as e:
print("! Error with request: %s :: %s" % (url, e))
def load_urls(urlfile):
# clean urls of white space and remove empty lines
# this does not prepend http/https to urls missing them
clean_urls = []
with open(urlfile) as f:
urllist = f.readlines()
for i in urllist:
i = i.strip()
if i:
clean_urls.append(i)
return (clean_urls)
args = sys.argv
if len(args) < 2:
print("! Error: Missing arguments.\n\nUsage: python check301.py urls.txt")
else:
try:
l = load_urls(args[1])
for i in l:
redirectTest(i)
except Exception as e:
print("Something went wrong: %s" % e)
Feel free to change, modify, use, extend, and improve this code. You can find it on GitHub here.