DIY Command Line Tool in Python
Josh Shields
- Audience level:
- Novice
- Category:
- Python Core (language, stdlib, etc.)
Description
Have you ever wanted your own console commands, but found that a BASH script wasn't practical? Python can do it! Python's standard library has great tools for controlling your development workspace. As a programmer, it is second nature to automate tasks. This project, a sitemap file generator called py_sitemapper, showcases an example of a Command Line Interface written in Python.
Abstract
The current progress of py\_sitemapper can be seen at:
[https://github.com/jshields/py_sitemapper](https://github.com/jshields/py_sitemapper)
This tool is meant to take a URL as input and output an XML sitemap. py\_sitemapper was written with a focus on leveraging the standard library in order to minimize development time and maximize usefulness. It's an example to show how command line tools can be written in Python. One of the goals of py\_sitemapper is to use established patterns, following best practice, in a way that can be repeated to write other CLI tools.
Standard library modules used in this project:
- HTMLParser
- argparse
- logging
- re
- requests
- sys
Modules written for this project:
- cli: Command Line Interface wrapper to drive functionality based on user input.
- parse: Parser for web pages. Manage a session and comb HTML for links, in the case of this project.
- sitemap: Contains a Sitemap class that has XML content and can be exported.
Discussion points:
- What tools have you written?
- How does writing BASH scripts and programs compare to Python?
- The example tool here, py\_sitemapper, provides information about hyperlinks from the hypertext on a website, but how can useful information be harvested by a web crawler from sites that rely on frameworks rather than plain HTML?
PyCon 2016 in Portland, OR is a production of the 