Learn How to Create a Website Status Checker in Python
If you often find yourself fetching data from websites, you should probably consider automating the process. Sometimes referred to as “web scraping”, the process is a common one for sites that do not provide a formal API or feed. Of course, you won’t get anywhere if the site you’re trying to fetch is unavailable.
If you run your own site, you’ve probably had to deal with downtime before. It can be frustrating, causing you to lose visitors and interrupting whatever activity your site may be responsible for. In such circumstances, it pays to be able to easily check your website’s availability.
Python is a great language for scripting, and its concise yet readable syntax makes implementing a site checker a simple task.
Creating Your Personalized Website Checker
The website checker is tailor-made to accommodate multiple websites at once. This allows you to easily switch out sites you no longer care about, or start checking sites that you launch in the future. The checker is an ideal “skeleton app” on which you could build further, but it demonstrates a basic approach to fetching web data.
Import Libraries in Python
To kick off the project, you must import therequestslibrary in Python with theimportfunction.
The Requests library is useful for communicating with websites. You can use it to send HTTP requests and receive response data.
Store the Website URLs in a List
Once you import the library, you should define and store the website URLs in a list. This step allows you to retain multiple URLs, which it’s possible to check with the website checker.
The variablewebsite_urlstores the list of URLs. Inside the list, define each URL you want to check as an individual string. You can use the example URLs in the code for testing or you can replace them to start checking your own sites right away.
Next, store the messages for commonHTTP response codes. You can keep these in a dictionary, and index each message by its corresponding status code. Your program can then use these messages instead of status codes for better readability.
Creating a Loop to Check Website Status
To check each URL in turn, you’ll want toloop through the listof websites. Inside the loop, check the status of each site by sending a request via the requests library.
The Entire Code Snippet
If you prefer to review the entire code in one go, here’s a full code listing for reference.
And here’s an example run of the code:
Python’s Coding Capabilities in Web Scraping
Python’s third-party libraries are ideal for tasks like web scraping and fetching data via HTTP.
You can send automated requests to websites to perform various types of tasks. These might include reading news headlines, downloading images, and sending emails automatically.
Wouldn’t it be easier if your computer read you the latest news headlines without you needing to do much? Good news: you can achieve exactly that.
I found my TV was always listening—so I shut it down.
These plugins will make you wonder why you used Photoshop in the first place.
Anyone with more than a passing interest in motorsports must see these films.
Lose your laptop without this feature, and you’ll wish you had turned it on.
Your phone is a better editor than you give it credit for.
