[A Config Nightmare] Argparse - A Shallowish Dive (Part 2)

Last week, we talked a bit about what's wrong with configuration management,  and we said that we were going to take a deep dive into the configuration management tools out there, and take a look at what's up with each of them, and see how they can be improved. We're going to start with the OG of all CLI configuration tools - argparse.

What is Argparse?

According to the argparse module:

The argparse module makes it easy to write user-friendly command-line interfaces. The program defines what arguments it requires, and argparse will figure out how to parse those out of sys.argv. The argparse module also automatically generates help and usage messages and issues errors when users give the program invalid arguments.

Basically, argparse is a tool for adding command line options to your program. Nothing more, nothing less. In fact, for what it does, Argparse has a lot of really impressive features:

  • Automated argument parsing and help message generation
  • Complex argument parsing
  • Default arguments
  • Basic argument validation
  • Subparsers (we'll talk about this in detail in a bit)
  • Argument abbreviation
  • Usage message generation

All of these are essential features for a good command line parser. In the next section, I'm going to talk about some things that I like about Argparse, and some things that make it great.

How Argparse Works

Basic Usage

Argparse is built around the concept of an ArgumentParser, an object which takes sys.argv[1:] and turns it into a Namespace, which is essentially a glorified dictionary. Imagine we want to add a basic integer argument to our program, we can do so by writing only the following code:

> parser = argparse.ArgumentParser()
> parser.add_argument('--count', type=int, default=7, help='The number to count to')
> args = parser.parse_args()

I can then very easily access the argument later using args.count.  This is something that I like about argparse a lot: It's easy to configure and use right out of the box, and there are a number of really powerful tools that you can set up. Notice here how I've added basic type-checking, a default argument, and a help message all with a single line of code. Not only this, but argparse builds a nice help message when using --help at the command line based on these functions.

Imagine that I want to add an "enable" style flag, that is, I can type --enable_feature at the command line, and that feature will be turned on. This is easy to do with argparse as well using the store_true action:

> parser = argparse.ArgumentParser()
> parser.add_argument('--enable_feature', action='store_true')
> args = parser.parse_args()

I can then check if the feature is enabled with: if args.enable_feature: ... Argparse can also handle parsing out comma separated lists, and multiple repeated arguments putting everything in a nice list under the args object.

Subparsers

One feature of argparse which is often overlooked is its ability to handle subparsers. Often, a program can perform a number of complex functions, so you need independent actions, or help messages. Think git with it's multiple different options such as git clone, git commit, git push etc. Each of these functions is separate, and requires different arguments. Argparse handles this gracefully by allowing us to add subparsers. For example, we can create a simple program with two parsers:

> # create the top-level parser
> parser = argparse.ArgumentParser(prog='PROG')
> parser.add_argument('--foo', action='store_true', help='foo help')
> subparsers = parser.add_subparsers(help='sub-command help')
>
> # create the parser for the "a" command
> parser_a = subparsers.add_parser('a', help='a help')
> parser_a.add_argument('bar', type=int, help='bar help')
>
> # create the parser for the "b" command
> parser_b = subparsers.add_parser('b', help='b help')
> parser_b.add_argument('--baz', help='baz help')

We cam then use things like PROG a --bar=7 or PROG b --baz=XYZ. This has the advantage of being able to wrap two different functionalities of a program very nicely.

Configuration Files

Argparse is a CLI configuration manager, however it was never really designed to manage configurations through configuration files. It does, however, have an out for people who want to save configuration files to a file for use later. We can store a a set of arguments in a file that looks like the following:

# args.txt
-f 17

We can then create a parser using the following options:

> parser = argparse.ArgumentParser(fromfile_prefix_chars='@')
> parser.add_argument('-f')

This allows us to type: prog @args.txt and load the arguments from the file.

The benefits to using Argparse

  1. It handles command line arguments very well

If you need to handle command line configuration, it works a treat. There is a lot of good support for configuring different types of arguments, different actions when the arguments are executed, and defaults/abbreviations and help messages that we'd like in a CLI

2. It's easy to use

Not only is argparse packaged in every python distribution by default, it's very easy to configure, and has a simple readable syntax for people who aren't familiar with it.

Why not use Argparse?

Let's look back at our set of design goals for a good configuration manager, and evaluate where argparse falls on each of these:

  • Be flexible

Argparse is a rather inflexible tool. There's only one way to pass arguments to the program (on the command line), and this means that the tool itself doesn't really allow for flexible configuration. In addition to this, it's hard to add new options to the code. Argparse requires that arguments are parsed by a parser object, in the main method, which means that if I want to run a small part of the code as a standalone CLI, I have to write my own custom parser for that particular part of the code (I can't just wrap the function and have the CLI library sort it out).

  • Seamlessly scale from small projects to large codebases

Argparse scales, but it does so at a cost. I have written huge files which are solely composed of argparse lines, and not much else. In addition to this, the automated validation options are not great - basic casting and conversion is supported, but you have to roll your own implementation of everything else. Wrapping up multiple programs in a single CLI is also a hassle, but not one that I want to focus on too much.

  • Leave footprints

This is where argparse truly fails. There is no method built into argparse for saving configurations. There's no logging. None at all. That's really all I have to say about this one.

  • Be "magical" to use, but safe enough to bet the farm on

Argparse is safe, useful, and powerful, but it comes at the cost of having to write most of your own implementation details. For somebody who wants something out of the box, it's not an excellent tool - but it's a good tool to build your on implementations on, and is an extremely flexible command line parser.

Conclusion

Argparse is an amazing tool, with a single function. While hampered by it's inflexibility and lack of features, it is amazing at what it does, and it provides an un-opinionated tool which can be used for parsing command line arguments. If you want to spend a lot of time building your own configuration management around argparse, it's the perfect tool for the job - but as a "magical experience" that "just works", it leaves something to be desired.

Next time, we'll take a look at a tool from the opposite end of the spectrum Sacred, which takes a very different approach to configuration management, and one that is more like what we would like to emulate in our final product.