Writing Python Code Like a Pro: A Deep Dive into Type Checking #4

In our previous post (Part 3), we discussed the basics of typing and introduced eight different types. In this post, we will explore more advanced typing concepts and how to use them effectively.

1. Specify only return type with typing.Callable[..., ReturnType]

In our previous post, we briefly discussed the typing.Callable type and how to use it for functions that take arguments. However, it can also be useful to specify only the return type of a callback function, without having to worry about its arguments. In this case, you can use the ... (Ellipsis) in place of the arguments for the Callable type.

For example, consider the following code:

from typing import Callable

def calculate(fn: Callable[..., float], *args: float) -> float:
    return fn(args)

def multiply(*args: float) -> float:
    v = 1
    for arg in args:
        v *= arg
    return v

def sum(*args: float) -> float:
    v = 0
    for arg in args:
        v += arg
    return v

calculate(multiply, 1, 2, 3)

2. typing.TypeVar can handle multiple types in dependency relationship

typing.TypeVar can be used to represent a generic type. This is useful when you need to write a function that can handle multiple types. When you use TypeVar, you can specify any type you want, such as Sequence[int], Sequence[str], or Sequence[Person].

from typing import Sequence, TypeVar, Iterable

T = TypeVar("T")

def batch_iter(data: Sequence[T], size: int) -> Iterable[Sequence[T]]:
    for i in range(0, len(data), size):
        yield data[i:i + size]

Additionally, you can limit the type by specifying a bound when declaring TypeVar. In the following example, the T type is restricted to int, str, bytes, or a type that inherits from them.

from typing import Sequence, TypeVar, Iterable, Union

T = TypeVar("T", bound=Union[int, str, bytes])

def batch_iter(data: Sequence[T], size: int) -> Iterable[Sequence[T]]:
    for i in range(0, len(data), size):
        yield data[i:i + size]

Using TypeVar with a bound can help ensure that only the appropriate types are passed to the function.

It is important to note that the type's dependency relationship is significant. In the example above, the return type Iterable[Sequence[T]] and the input type Sequence[T] are dependent. In other words, the input type determines the return type. For example, Iterable[Sequence[int]] will return Sequence[int], while Iterable[Sequence[str]] will return Sequence[str].

3. Use generic type with typing.Generic

TypeVar allows us to express dependencies between types within functions, but it falls short when it comes to declaring generic types whose type parameters are resolved at class definition time. In this case, we can use typing.Generic and typing.TypeVar together.

# https://docs.python.org/3/library/typing.html#user-defined-generic-types

from typing import TypeVar, Generic
from logging import Logger

T = TypeVar('T')


class LoggedVar(Generic[T]):
    def __init__(self, value: T, name: str, logger: Logger) -> None:
        self.name = name
        self.logger = logger
        self.value = value

    def set(self, new: T) -> None:
        self.log('Set ' + repr(self.value))
        self.value = new

    def get(self) -> T:
        self.log('Get ' + repr(self.value))
        return self.value

    def log(self, message: str) -> None:
        self.logger.info('%s: %s', self.name, message)

In the above example, the type of value in the __init__ method determines the type of the new parameter in the set method, and the return type of the get method. That is, the set and get methods depend on the type of value. In such cases, we can define a generic class using typing.Generic.

# https://docs.python.org/3/library/typing.html#user-defined-generic-types

from collections.abc import Iterable

def zero_all_vars(vars: Iterable[LoggedVar[int]]) -> None:
    for var in vars:
        var.set(0)

By specifying [<type>] after the generic class, we can indicate a specific type of generic class. The above code only accepts Iterable[LoggedVar[int]] as an argument, and Iterable[LoggedVar[str]] or any other type will fail the type check.

Note

• For those who want to know more about generic classes, be sure to check out the official Python documentation: User-defined generic types .

4. Flexible Function Signatures with typing.ParamSpec

In cases where the callback functions of the Callable type take arguments with types that the calling function depends on, typing.ParamSpec can be used. This is often seen with decorators, for example. In this case, typing.ParamSpec can be used to represent the generic type.

from typing import TypeVar, Callable, ParamSpec
import logging


T = TypeVar('T')
P = ParamSpec('P')


def add_logging(f: Callable[P, T]) -> Callable[P, T]:
    '''A type-safe decorator to add logging to a function.'''
    def inner(*args: P.args, **kwargs: P.kwargs) -> T:
        logging.info(f'{f.__name__} was called')
        return f(*args, **kwargs)
    return inner


@add_logging
def add_two(x: float, y: float) -> float:
    '''Add two numbers together.'''
    return x + y


@add_logging
def send_msg(msg: str) -> None:
    print(f"I Sent {msg}")

The above is a slightly modified example from the Python official documentation. The add_logging decorator adds the ability to log to the function being called. As is typical with decorators, the inner function (in the above code, inner) takes arguments and return values that are similar to those of the callback function. In this case, P.args and P.kwargs defined above can be used to represent them.

Note

• This type can be used starting from Python 3.10. To use it with earlier versions of Python, you need to install the type-extensions library.
• If you are not familiar with decorators, please study the Decorator section of Python Coding Dojang first.

5. typing.Protocol

Up until now, we have seen types that define types based on static properties such as the type of a given argument, whether an object is an instance of a certain class, and so on. However, we can also define types based on behaviors and responsibilities. Since behavior is usually represented by functions (methods) in programming, we can define a type as one that can perform a certain set of behaviors (responsibilities) represented by functions (methods). In Python, this type of type is called a protocol , which is similar to an interface in other programming languages such as Go, TypeScript, and Java.

Let's take a look at an example:

from typing import Protocol


class Flyable(Protocol):
    def fly(self): ...


class Bird:
    def fly(self):
        print("Bird is flying")


class Plane:
    def fly(self):
        print("Plane is flying")


class Dog:
    def walk(self):
        print("Dog is walking")


def take_off(flyable: Flyable):
    flyable.fly()

In the above code, we define a Flyable protocol (interface). If a certain object has a fly method, it satisfies the protocol and can be used as an argument for the take_off function. If it does not have a fly method, type checking will fail.

As a result, the Bird and Plane classes satisfy the protocol, but the Dog class does not.

Note

• Protocols were introduced in Python 3.8, so they may not be available in earlier versions of Python.
• For more information on protocols, see the Python documentation on typing.Protocol .