Unravelling t-strings

PEP 750 introduced t-strings for Python 3.14. In fact, they are so new that as of Python 3.14.0b1 there still isn't any documentation yet for t-strings. 😅 As such, this blog post will hopefully help explain what exactly t-strings are and what you might use them for by unravelling the syntax and briefly talking about potential uses for t-strings.

What are they?

I like to think of t-strings as a syntactic way to expose the parser used for f-strings. I'll explain later what that might be useful for, but for now let's see exactly what t-strings unravel into.

Let's start with an example by trying to use t-strings to mostly replicate f-strings. We will define a function named f_yeah() which takes a t-string and returns what it would be formatted had it been an f-string (e.g. f"{42}" == f_yeah(t"{42}")). Here is the example we will be working with and slowly refining:

def f_yeah(t_string):
    """Convert a t-string into what an f-string would have provided."""
    return t_string


if __name__ == "__main__":
    name = "world"
    expected = f"Hello, {name}! Conversions like {name!r} and format specs like {name:<6} work!"
    actual = f_yeah(expected)

    assert actual == expected

As of right now, f_yeah() is just the identity function which takes the actual result of an f-string, which is pretty boring and useless. So let's parse what the t-string would be into its constituent parts:

def f_yeah(t_string):
    """Convert a t-string into what an f-string would have provided."""
    return "".join(t_string)


if __name__ == "__main__":
    name = "world"
    expected = f"Hello, {name}! Conversions like {name!r} and format specs like {name:<6} work!"
    parsed = [
        "Hello, ",
        "world",
        "! Conversions like ",
        "'world'",
        " and format specs like ",
        "world ",
        " work!",
    ]
    actual = f_yeah(parsed)

    assert actual == expected

Here we have split the f-string output into a list of the string parts that make it up, joining it all together with "".join(). This is actually what the bytecode for f-strings does once it has converted everything in the replacement fields – i.e. what's in the curly braces – into strings.

But this is still not that interesting. We can definitely parse out more information.

def f_yeah(t_string):
    """Convert a t-string into what an f-string would have provided."""
    return "".join(t_string)


if __name__ == "__main__":
    name = "world"
    expected = f"Hello, {name}! Conversions like {name!r} and format specs like {name:<6} work!"
    parsed = [
        "Hello, ",
        name,
        "! Conversions like ",
        repr(name),
        " and format specs like ",
        format(name, "<6"),
        " work!",
    ]
    actual = f_yeah(parsed)

    assert actual == expected

Now we have substituted the string literals we had for the replacement fields with what Python does behind the scenes with conversions like !r and format specs like :<6. As you can see, there are effectively three parts to handling a replacement field:

  1. Evaluating the Python expression
  2. Applying any specified conversion (let's say the default is None)
  3. Applying any format spec (let's say the default is "")

So let's get our "parser" to separate all of that out for us into a tuple of 3 items: value, conversion, and format spec. That way we can have our f_yeah() function handle the actual formatting of the replacement fields.

def f_yeah(t_string):
    """Convert a t-string into what an f-string would have provided."""
    converters = {func.__name__[0]: func for func in (str, repr, ascii)}
    converters[None] = str

    parts = []
    for part in t_string:
        match part:
            case (value, conversion, format_spec):
                parts.append(format(converters[conversion](value), format_spec))
            case str():
                parts.append(part)

    return "".join(parts)


if __name__ == "__main__":
    name = "world"
    expected = f"Hello, {name}! Conversions like {name!r} and format specs like {name:<6} work!"
    parsed = [
        "Hello, ",
        (name, None, ""),
        "! Conversions like ",
        (name, "r", ""),
        " and format specs like ",
        (name, None, "<6"),
        " work!",
    ]
    actual = f_yeah(parsed)

    assert actual == expected

Now we have f_yeah() taking the value from the expression of the replacement field, applying the appropriate conversion, and then passing that on to format(). This gives us a more useful parsed representation! Since we have the string representation of the expression, we might as well just keep that around even if we don't use it in our example (parsers typically don't like to throw information away).

def f_yeah(t_string):
    """Convert a t-string into what an f-string would have provided."""
    converters = {func.__name__[0]: func for func in (str, repr, ascii)}
    converters[None] = str

    parts = []
    for part in t_string:
        match part:
            case (value, _, conversion, format_spec):
                parts.append(format(converters[conversion](value), format_spec))
            case str():
                parts.append(part)

    return "".join(parts)


if __name__ == "__main__":
    name = "world"
    expected = f"Hello, {name}! Conversions like {name!r} and format specs like {name:<6} work!"
    parsed = [
        "Hello, ",
        (name, "name", None, ""),
        "! Conversions like ",
        (name, "name", "r", ""),
        " and format specs like ",
        (name, "name", None, "<6"),
        " work!",
    ]
    actual = f_yeah(parsed)

    assert actual == expected

The next thing we want our parsed output to be is be a bit easier to work with. A 4-item tuple is a bit unwieldy, so let's define a class named Interpolation that will hold all the relevant details of the replacement field.

class Interpolation:
    __match_args__ = ("value", "expression", "conversion", "format_spec")

    def __init__(
        self,
        value,
        expression,
        conversion=None,
        format_spec="",
    ):
        self.value = value
        self.expression = expression
        self.conversion = conversion
        self.format_spec = format_spec


def f_yeah(t_string):
    """Convert a t-string into what an f-string would have provided."""
    converters = {func.__name__[0]: func for func in (str, repr, ascii)}
    converters[None] = str

    parts = []
    for part in t_string:
        match part:
            case Interpolation(value, _, conversion, format_spec):
                parts.append(format(converters[conversion](value), format_spec))
            case str():
                parts.append(part)

    return "".join(parts)


if __name__ == "__main__":
    name = "world"
    expected = f"Hello, {name}! Conversions like {name!r} and format specs like {name:<6} work!"
    parsed = [
        "Hello, ",
        Interpolation(name, "name"),
        "! Conversions like ",
        Interpolation(name, "name", "r"),
        " and format specs like ",
        Interpolation(name, "name", format_spec="<6"),
        " work!",
    ]
    actual = f_yeah(parsed)

    assert actual == expected

That's better! Now we have an object-oriented structure to our parsed replacement field, which is easier to work with than the 4-item tuple we had before. We can also extend this object-oriented organization to the list we have been using to hold all the parsed data.

class Interpolation:
    __match_args__ = ("value", "expression", "conversion", "format_spec")

    def __init__(
        self,
        value,
        expression,
        conversion=None,
        format_spec="",
    ):
        self.value = value
        self.expression = expression
        self.conversion = conversion
        self.format_spec = format_spec


class Template:
    def __init__(self, *args):
        # There will always be N+1 strings for N interpolations;
        # that may mean inserting an empty string at the start or end.
        strings = []
        interpolations = []
        if args and isinstance(args[0], Interpolation):
            strings.append("")
        for arg in args:
            match arg:
                case str():
                    strings.append(arg)
                case Interpolation():
                    interpolations.append(arg)
        if args and isinstance(args[-1], Interpolation):
            strings.append("")

        self._iter = args
        self.strings = tuple(strings)
        self.interpolations = tuple(interpolations)

    @property
    def values(self):
        return tuple(interpolation.value for interpolation in self.interpolations)

    def __iter__(self):
        return iter(self._iter)


def f_yeah(t_string):
    """Convert a t-string into what an f-string would have provided."""
    converters = {func.__name__[0]: func for func in (str, repr, ascii)}
    converters[None] = str

    parts = []
    for part in t_string:
        match part:
            case Interpolation(value, _, conversion, format_spec):
                parts.append(format(converters[conversion](value), format_spec))
            case str():
                parts.append(part)

    return "".join(parts)


if __name__ == "__main__":
    name = "world"
    expected = f"Hello, {name}! Conversions like {name!r} and format specs like {name:<6} work!"
    parsed = Template(
        "Hello, ",
        Interpolation(name, "name"),
        "! Conversions like ",
        Interpolation(name, "name", "r"),
        " and format specs like ",
        Interpolation(name, "name", format_spec="<6"),
        " work!",
    )
    actual = f_yeah(parsed)

    assert actual == expected

And that's t-strings! We parsed f"Hello, {name}! Conversions like {name!r} and format specs like {name:<6} work!" into Template("Hello, ", Interpolation(name, "name"), "! Conversions like ", Interpolation(name, "name", "r"), " and format specs like ", Interpolation(name, "name", format_spec="<6")," work!"). We were then able to use our f_yeah() function to convert the t-string into what an equivalent f-string would have looked like. The actual code to use to test this in Python 3.14 with an actual t-string is the following (PEP 750 has its own version of converting a t-string to an f-string which greatly inspired my example):

from string import templatelib


def f_yeah(t_string):
    """Convert a t-string into what an f-string would have provided."""
    converters = {func.__name__[0]: func for func in (str, repr, ascii)}
    converters[None] = str

    parts = []
    for part in t_string:
        match part:
            case templatelib.Interpolation(value, _, conversion, format_spec):
                parts.append(format(converters[conversion](value), format_spec))
            case str():
                parts.append(part)

    return "".join(parts)


if __name__ == "__main__":
    name = "world"
    expected = f"Hello, {name}! Conversions like {name!r} and format specs like {name:<6} work!"
    parsed = t"Hello, {name}! Conversions like {name!r} and format specs like {name:<6} work!"
    actual = f_yeah(parsed)

    assert actual == expected

What are t-strings good for?

As I mentioned earlier, I view t-strings as a syntactic way to get access to the f-string parser. So, what do you usually use a parser with? The stereotypical thing is compiling something. Since we are dealing with strings here, what are some common strings you "compile"? The most common answer are things like SQL statements and HTML: things that require some processing of what you pass into a template to make sure something isn't going to go awry. That suggests that you could have a sql() function that takes a t-string and compiles a SQL statement that avoids SQL injection attacks. Same goes for HTML and JavaScript injection attacks.

Add in logging and you get the common examples. But I suspect that the community is going to come up with some interesting uses of t-strings and their parsed data (e.g. PEP 787 and using t-strings to create the arguments to subprocess.run())!