Testing Django Performance

Performance Testing

Code Profiling: what do we try next?
Relative Performance: is it helping?
Absolute Performance: can we survive success?

Test Platform

Don't even bother testing on your laptop
I ❤ AWS EC2
Try to get a whole machine: eg:
- m1.xlarge
- m3.2xlarge
- cc2.8xlarge

Test Platform Config — 1

Install Packages:


$ sudo apt-get update
$ sudo apt-get install nginx postgresql postgresql-server-dev-all python-dev python-pip
$ sudo pip install django mezzanine psycopg2 uwsgi loremipsum inflect

Make Django Project:


$ mkdir testproject
$ cd testproject
$ mezzanine-project

Configure Postgres User:


create user mezzanine with login password 'mezzanine';
create database mezzanine;
grant all on database mezzanine to mezzanine;

Test Platform Config — 2

local_settings.py


DEBUG = True

DATABASES = {
    "default": {
        "ENGINE": "django.db.backends.postgresql_psycopg2",
        "NAME": "mezzanine",
        "USER": "mezzanine",
        "PASSWORD": "mezzanine",
        "HOST": "localhost",
        "PORT": "",
    }
}

ALLOWED_HOSTS=['*']

Test Platform Config — 3

NGINX config:


server {
    listen *:80;

    location /static {
        root /usr/local/lib/python2.7/dist-packages/mezzanine/core;
    }

    location / {
        include uwsgi_params;
        uwsgi_pass localhost:9000;
    }
}

UWSGI config:


[uwsgi]
chdir=/home/ubuntu
module=testproject.wsgi
processes=15
socket=localhost:9000
env=DJANGO_SETTINGS_MODULE=testproject.settings

Synthetic Test Data

Empty databases go fast!
Your database won't stay empty
Model for success
Generate data through Django

Synthetic Test Data


from django.contrib.auth.models import User
from mezzanine.blog.models import BlogPost






users = User.objects.all()

for n in range(1,10001):
    BlogPost(
        user =
        title =
        content =
    ).save()

Synthetic Test Data





import random
import loremipsum
import inflect






                  random.choice(users)
                  inflect_engine.ordinal(n)
                  loremipsum.get_paragraphs(2)

Synthetic Test Data


from django.contrib.auth.models import User
from mezzanine.blog.models import BlogPost

import random
import loremipsum
import inflect
inflect_engine = inflect.engine()

users = User.objects.all()

for n in range(1,10001):
    BlogPost(
        user =    random.choice(users),
        title =   inflect_engine.ordinal(n),
        content = loremipsum.get_paragraphs(2)
    ).save()

Simulating Load — Siege

Concurrency
URLs
Cacheability

Antipatterns: N+1


for foo in Foo.objects.filter(...):
    print foo.bar().name


for foo in Foo.objects.all():
    for bar in foo.bar_set().all():
        for baz in bar.baz_set().all():
            for qux in baz.qux_set().all():
                print qux.quux


foo.select_related('bar__baz__qux')

May still generate horrible SQL

Query Logging

Log All Queries
Log Slow Queries
(mysql) Log Lousy Queries
EXPLAIN and EXPLAIN ANALYSE

Other Bottlenecks

Disk IOPS
Network
Entropy