CLI subcommands with lazy imports

In case you didn't hear, PEP 810 got accepted which means Python 3.15 is going to support lazy imports! One of the selling points of lazy imports is with code that has a CLI so that you only import code as necessary, making the app a bit more snappy at startup. A common example given is when you run --help you probably don't need all modules imported to make that work. But another use case for CLIs and lazy imports is subcommands where each subcommand very likely only needs a subset of modules to work.

How to make subcommands work with argparse

There's two ways to typically do subcommands in argparse. The old-fashioned way is with a dict that dispatches to a function based on the subcommand name that was specified on the terminal. An example of that which I wrote can be found in the stdlib.

cpython/Platforms/WASI/__main__.py at 2be2dd5fc219a5c252d72f351c85db14314bfca5 · python/cpython
The Python programming language. Contribute to python/cpython development by creating an account on GitHub.

The other approach is covered in the argparse docs for subcommands which involves setting a default value for a subparser for the subcommand.

Regardless of which approach you use, the key detail is that something stores the callable you want to use based on the chosen subcommand.

Why these approaches don't work with lazy imports

Subcommands for a CLI is a great use of lazy imports. By making the imports that are not universally needed lazy, you can avoid paying any extra cost for importts you don't care about for any specific subcommand. This is even easier to do when you put a subcommand's code in a separate module as you can just lazy import the main function to call in your __main__.py, e.g. lazy from spam import main as spam_main and then use spam_main as the function to call when the spam subcommand is called.

The problem is that reification (the act of making the lazy import do the actual import and become the object it's meant to be) is triggered by assignment. That means using the lazy import object as a value in a dict or passing it as an argument to anything in argparse triggers the import. And since you need to do all your wiring upfront for argparse to do its thing, all of your lazy imports will get triggered that you assign using either of the approaches I listed above for making subcommands work. But I think lazy imports for subcommands is worth the hassle of trying to find a solution.

Some solutions to this problem

To make this all play nicely with lazy imports, you need to either avoid touching the lazy import objects if you don't need to use them or you need a level of indirection. To avoid touching the objects, you can do something like turning the dict approach into using a match statement.

match context.subcommand:
    case "spam":
        spam_main(context)

Another way to do it is to wrap the lazy import object in a lambda so there's a layer of indirection between the object and the assigment.

parser_foo.set_defaults(func=lambda args: foo(args))

Both approaches work and keep the lazy import object from being acidentally reified.