Blog

Integrating mypy into a Django project
May 31, 2019

Python is one of my favorite languages but over time I have come to find its dynamically typed syntax to have its pitfalls. This is doubly so in bigger and more complex projects. Using Mypy, a great static type checker for python, and the typing module has been a boon to the quality of my python projects.

Recently I started a new Django project and wanted to seamlessly integrate mypy into my development workflow. The steps I followed were:

  • Install mypy
  • Install django-stubs (type hints for django)
  • Configure django-stubs mypy plugin
  • Add a custom check to django so runserver, migrate, and check commands will run mypy

Installing mypy and django-stubs

Both of these projects are available on PyPi and can easily be installed with pip. However, at the time of this article django-stubs 0.12.1 didn't work correctly with my installation and I had to use the master branch from github.

pip install mypy
pip instal -e git+https://github.com/mkurnikov/django-stubs#egg=django-stubs

However in my case I used pipenv and only want these dependencies installed in a dev environment so my install looked like so:

pipenv install mypy --dev
pipenv install -e git+https://github.com/mkurnikov/django-stubs#egg=django-stubs --dev

Configure django-stubs mypy plugin

The django-stubs project exposes an optional mypy plugin that allows for some more specific configuration. To enable the plugin I created a default mypy.ini configuration file at the top-level of my project added the plugin to the config.

mypy.ini

[mypy]
plugins = mypy_django_plugin.main

When I ran mypy against my project it all worked great except that it was now throwing errors because I did not specify attributes on the model fields. I didn't want to do this as it felt redundant since the model field was quite specific already, e.g. name = models.CharField(). The django-stubs mypy plugin allows us to disable this check if we add it to mypy_django.ini, e.g.

mypy_django.ini

[mypy_django_plugin]
ignore_missing_model_attributes = True

I didn't like having two separate ini files for configuring mypy and ideally wanted all the settings in mypy.ini. Thankfully, the plugin allows you to override the environment variable MYPY_DJANGO_CONFIG to set the config file. Since I am using pipenv this was quite easy, I created a .env file to set this variable and then updated mypy.ini` with the django-stubs mypy settings.

.env

MYPY_DJANGO_CONFIG=./mypy.ini

mypy.ini

[mypy]
plugins = mypy_django_plugin.main

[mypy_django_plugin]
ignore_missing_model_attributes = True

Finally, I was getting errors on migrations so I wanted to ignore errors on the migrations. To do so I added another section to my config to ignore those files. The final config looks like this:

mypy.ini

[mypy]
plugins = mypy_django_plugin.main

[mypy_django_plugin]
ignore_missing_model_attributes = True

[mypy-*.migrations.*]
ignore_errors = True

For more mypy and django-stubs mypy plugin configurations, please check the respective documentation.

Setup a Django mypy check

Finally, I didn't want to have to constantly run mypy <myproject> everytime I made changes to my and wanted this to happen automatically. Rather than use a thirdparty watcher program, I wanted this to be integrated into django somehow, ideally into the runserver command as it was already part of my development workflow and already has auto-reloading functionality.

Thankfully, this is possible with the django check framework. I created a new checks.py to contain my new mypy check. The check has to be registered early in the project so I import it into my project's __init__.py file. Also, since I'm using pipenv and only install mypy in my dev environment, I want to make sure my check only gets imported if mypy is available. The result looks like this:

init.py

import importlib.util

mypy_package = importlib.util.find_spec("mypy")
if mypy_package:
    from .checks import mypy

For the check, I use the mypy api to programatically run the check and then parse the results. The mypy results are in a string format so I parse them with a simple regex and then build a django CheckMessage object for each error. I then pass the results to the check framework for django to manage.

import re
from typing import List
from django.core.checks import register
from django.core.checks.messages import CheckMessage, DEBUG, INFO, WARNING, ERROR
from django.conf import settings

from mypy import api


# The check framework is used for multiple different kinds of checks. As such, errors
# and warnings can originate from models or other django objects. The `CheckMessage`
# requires an object as the source of the message and so we create a temporary object
# that simply displays the file and line number from mypy (i.e. "location")
class MyPyErrorLocation:
    def __init__(self, location):
        self.location = location

    def __str__(self):
        return self.location


@register()
def mypy(app_configs, **kwargs) -> List:
    print("Performing mypy checks...\n")

    # By default run mypy against the whole database everytime checks are performed.
    # If performance is an issue then `app_configs` can be inspected and the scope 
    # of the mypy check can be restricted
    mypy_args = [settings.BASE_DIR]
    results = api.run([settings.BASE_DIR])
    error_messages = results[0]

    if not error_messages:
        return []

    # Example: myproject/checks.py:17: error: Need type annotation for 'errors'
    pattern = re.compile("^(.+\d+): (\w+): (.+)")

    errors = []
    for message in error_messages.rstrip().split("\n"):
        parsed = re.match(pattern, message)
        if not parsed:
            continue

        location = parsed.group(1)
        mypy_level = parsed.group(2)
        message = parsed.group(3)

        level = DEBUG
        if mypy_level == "note":
            level = INFO
        elif mypy_level == "warning":
            level = WARNING
        elif mypy_level == "error":
            level = ERROR
        else:
            print(f"Unrecognized mypy level: {mypy_level}")

        errors.append(CheckMessage(level, message, obj=MyPyErrorLocation(location)))

    return errors

Finally, all I need to do is run python manage.py runserver as usual and I have mypy seamlessly integrated. This is what it looks like in action:

Watching for file changes with StatReloader
Performing system checks...

Performing mypy checks...

Exception in thread django-main-thread:
Traceback (most recent call last):
  File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "/home/ralph/.local/share/virtualenvs/myproject--HcI8Ois/lib/python3.6/site-packages/django/utils/autoreload.py", line 54, in wrapper
    fn(*args, **kwargs)
  File "/home/ralph/.local/share/virtualenvs/myproject--HcI8Ois/lib/python3.6/site-packages/django/core/management/commands/runserver.py", line 117, in inner_run
    self.check(display_num_errors=True)
  File "/home/ralph/.local/share/virtualenvs/myproject--HcI8Ois/lib/python3.6/site-packages/django/core/management/base.py", line 436, in check
    raise SystemCheckError(msg)
django.core.management.base.SystemCheckError: SystemCheckError: System check identified some issues:

ERRORS:
myproject/checks.py:29: Incompatible types in assignment (expression has type "int", variable has type "str")