by Mai Norapong

Contents:

Motivations for using type hints

  • Improve readability for humans and computers
    • Better code completion and refactoring in IDEs
  • Acts as live documentation
    • Solves the problem of docstrings not being maintained
    • Docstrings don’t allow complex types
  • Reduce errors
    • Static analysis detects more errors when type hints are present
    • IDE can better detect errors as you type
    • Helps a lot in large projects

While Python is known and loved for dynamic or “duck” typing, many (including the Python creator Guido van Rossum) agree that static type checking is welcome in the form of “gradual type hinting”.

Ways to type hint

Function annotations (PEP 3107)

Function annotations were introduced in PEP 3107 and are available from Python 3.0.

def foo(a: expression, b: expression = default_value):
    ...
def bar(*args: expression, **kwargs: expression):
    ...
def bazz() -> expression:
    ...

These annotations result in a dict named __annotations__. This dict is a mapping from parameter names to a Python expression; the return value has key 'return'. The expression is evaluated during function definition.

PEP 3107 allows for any valid python expression to be there including any str, int, or whatever, but in current practice, we put classes in there.

def catch_all(*args, **kwargs) -> None:
    return

def double_string(s: str, sep: str = '') -> str:
    return f'{s}{sep}{s}'

def my_abs(x: int) -> int:
    if x < 0:
        x = -x
    return x

>>> catch_all.__annotations__
{'return': None}
>>> double_string.__annotations__
{'string': <class 'str'>, 'return': <class 'str'>}
>>> my_abs.__annotations__
{'x': <class 'int'>, 'return': <class 'int'>}

Compare the above with the untyped code below. In the code below, you’re not really sure what the function does.

def catch_all(*args, **kwargs):
    return

def double_string(s, sep):
    return f'{s}{sep}{s}'

def my_abs(x):
    if x < 0:
        x = -x
    return x

The typing module (PEP 484)

The typing module introduced by PEP 484 (added in Python 3.5) provides “hints” for more complex types.

Let’s see it in action.

from typing import Any, Union

Number = Union[int, float, complex]

def catch_all(*args: Any, **kwargs: Any) -> None: ...

def double_string(string: str, sep: str = '') -> str: ...

def my_abs(x: Number) -> Number: ...

With Union, you can now call my_abs() with any numbers and the IDE or mypy won’t yell at you.

Types for collections such as List, Set, Dict:

from typing import List

def get_even(list_: List[int]) -> List[int]:
    ...

this means that get_even() accepts a list of int values and returns such a list, too. of int or a None (probably if there aren’t any even numbers).

A dictionary where keys are strings, and values can be any type:

from typing import Dict, Any

def parse_request_data(data: Dict[str, Any]) -> None: ...

A function that may return an int or may not return anything!

from typing import Optional, List

def index_of(value: str, lst: List[str]) -> Optional[int]:
    ...

You can use application classes as type hints, too:

class Question:
    ...
    def get_voted_choices(self) -> models.QuerySet:
        """
        Returns an iterable that iterates over all choices of this
        question that has been voted.
        """

Some other useful ones I’ve used:

  • All the standard type extensions: List, Tuple, Set, etc.
  • Iterable when I only require a function / method’s input to be an iterable (that is, I can use a for loop with it at least once)
  • Callable when you’re making higher level functions or using a function somewhere
  • …

See the typing module for more details.

Variable annotations (PEP 526)

PEP 526 introduces a new syntax for defining variables available from Python 3.6 onwards.

annotated_assignment_stmt ::=  augtarget ":" expression ["=" expression]

Don’t worry if you don’t understand what this means. This is in an Extended Backus-Naur Form, a modified version of BNF used by Python.

What that means is basically that you can now do this

x: int = 5

or just this

x: int

It might not look very useful cause many of the time that’s pretty obvious and static type checkers and IDE can figure these things out pretty easily. But in some cases:

rating_str: str = soup.find(**{'class': 'wpb_wrapper'}).find(string='Rating').parent.find('span').string

The IDE already lost track of what the types are after the first few calls. (And probably also whoever is reading your code.) By adding type hints to the variable IDE regains knowledge of what type each variable in the call chain is, so it can now do code completion for you again, and whoever reads your code can now be a little bit happier.

Type hints for Classes

You can use type hints to convey the meaning “this class provides a behavior”. In Java or C-sharp this would be “implements an interface”.

  • A class named Scoreboard has an __iter__ method that can be used to create an iterator, or as the data source in a for x in data statement.
from typing import Iterable, Iterator

class Scoreboard(Iterable):
   ...

   def __iter__(self) -> Iterator:
      """This method should return an Iterator.
         Code that wants an iterator will call iter(obj)
         to create one.
      """
      pass
  • If Scoreboard returns an Iterator whose values are always int, you can improve type checking by specifying this using the notation Iteratable[type]:
from typing import Iterable, Iterator, List

class Scoreboard(Iterable[int]):
   def __init__(self):
       self.data: List[int] = []

   def __iter__(self) -> Iterator[int]:
      """This method should return an Iterator.
         Code that wants an iterator will call iter(obj)
         to create one.
      """
      # create an iterator from a range
      return iter(self.data)

In Java, we would do this by writing:

class Scoreboard implement Iterator<Integer>
  • A class has a __len__ method that returns the “length” or “size” of the object. In Python code, this means you can write len(obj). The typing package defines a Sized type for this:
from typing import Sized

class CourseList(Sized):
    """List of courses taken by a student"""

    def __init__(self, student_id):
        self.student_id = student_id
        self.courses = []

    def add_course(self, course: Course):
        self.courses.append(course)

    def __len__(self):
        return len(self.courses)

Self-Referencing Type Hints

If a type hint refers to a class currently being defined, you will get an error:

class Node:
    """A node in a graph has a parent node and child nodes."""

    def __init__(self, parent: Node):
        self.parent = parent
        self.children: Set[Node] = set()

when you run this code, Python reports:

NameError: name 'Node' is not defined

There are 2 ways to correct this:

  1. In the type hints, quote the class name: 'Node' 
    class Node:
     def __init__(self, parent: 'Node'):
         self.children: Set['Node'] = set()
  2. Add from __future__ import annotations 
    from __future__ import annotations

class Node:
     def __init__(self, parent: Node):
         self.children: Set[Node] = set()

Static Analysis Tools

These tools examine code to look for errors. They make use of type hints to do better analysis:

mypy the most popular static type analyzer for Python. PyPi calls it “a Python linter on steroids”.

pylint and flake8 also uses type hints to find syntax and semantic errors.

VS Code’s Pylance and Pyright (older) perform static analysis and also use type hints.

Summary

You can annotate variables, parameters, and the return value of functions. You can annotate classes to indicate behavior they provide (like Java interfaces). You can also define your own generic classes with the aid of the typing module, for annotating more complex types.

All of these require Python 3. Some type hinting is possible in Python 2 (now obsolete) using type comments and stub files (the .pyi files you sometimes see). Type hinting for C libraries is also done using stub files.

Note: some corporations (like Dropbox) uses Python 3.5 so variable annotations would not be available in those cases (But in the specific case of Dropbox, they don’t really use the distributed version of Python so I’m not 100% sure if it’s available or not—though they do like mypy, but just aware of compatibility issues if that’s a concern.)

References

A good place to start is the first reference.

Python typing Package - the type hints you can use to designate Python types
Python Collection Base Classes in the package collections.abc.

PEP 484 the Typing module
PEP 526 annotations for variables
PEP 3107