Fika time. 広っ！ First coffee in a cafe since February-ish.

Espresso burger

Social distancing at the beach.

Django has a nice security feature that verifies the request HOST header against the ALLOWED_HOSTS whitelist and will return errors if the requesting host is not in the list. Often you’ll see this when first setting up an app where you only expect requests to app.example.com but some bot makes a request to <server ip address>.

While it’s not strictly harmful to add your server ip to your ALLOWED_HOSTS, in theory, it does allow bots to easily reach and fire requests to your Django app, which will needlessly consume resources on your app server. It’s better to filter out the requests before they get to your app server.

For HTTP requests, you can block requests by adding default_server that acts as a catchall. Your app server proxy then set its server_name to the a domain in your ALLOWED_HOSTS. This simple configuration will prevent http://<server ip address> requests from ever reaching your app server.

// default.conf
server {
  listen 80 default_server;
  return 444;
}

// app.conf
upstream app_server {
server 127.0.0.1:8000 fail_timeout=0;
}

server {

listen 80; server_name {{ WEB_SERVER_NAME }};
access_log /var/log/nginx/access.log access_json;
error_log /var/log/nginx/error.log warn;

location /static/ {
alias /var/app/static/;
}

location / {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Request-Id $request_id;
proxy_redirect off; proxy_pass http://app_server;
}
}

However, once you enable SSL with Let’s Encrypt, despite the fact that they matching by host, as there is only one SSL server configuration by default, it routes all https traffic to the same host. What this means is that while requests made to http://<server ip address> will continue to be blocked, requests to https://<server ip address> will begin to be forwarded to your django app server, resulting in errors. Yikes!

The solution is to add a default SSL enabled server, much like your http configuration. Thee only tricky bit is that all ssl configurations must have a valid ssl certificate configuration as well.  Rather than making a self-signed certificate I reused my let’s encrypt ssl configuration.

// default.conf
server {
  listen 80 default_server; return 444;
}

server {
listen 443 ssl default_server;
ssl_certificate /etc/letsencrypt/live/{{ WEB_SERVER_NAME }}/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/{{ WEB_SERVER_NAME }}/privkey.pem;
include /etc/letsencrypt/options-ssl-nginx.conf;
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;

if ($host != {{ WEB_SERVER_NAME }}) {
return 444;
}
}

By adding a default SSL server to your nginx config your server_name settings will be respected and requests that do not match your host name will no longer be forwarded to your app server.

Recently at work I’ve been working quite a bit with Django and GraphQL. There doesn’t seem to be much written about best practices for organizing your Graphene-Django projects, so I’ve decided to document what’s working for me. In this example I have 3 django apps: common, foo, and hoge.

There’s two main goals for this architecture:

1. Minimize importing from “outside” apps.


2. Keep testing simple.

Queries and Mutations Package

Anything beyond simple queries (i.e. a query that just returns all records of a given model) are implemented in their own file in the queries or mutations sub-package. Each file is as self-contained as possible and contains any type definitions specific to that query, forms for validation, and an object that can be imported by the app's schema.py.

Input Validation

All input validation is performed by a classic Django form instance. For ease of use django form input does not necessarily match the GraphQL input. Consider a mutation that sends a list of dictionaries with an object id.

{
  "foos": [
    {
        "id": 1,
        "name": "Bumble"
    },
    {
        "id": 2,
        "name": "Bee"
  ]
}

Before processing the request, you want to validate that the ids passed actually exist and or reference-able by the user making the request. Writing a django form field to handle input would be time consuming and potentially error prone. Instead each form has a class method called convert_graphql_input_to_form_input which takes the mutation input object and returns a dictionary that can be passed the form to clean and validate it.

from django import forms
from foo import models

class UpdateFooForm(forms.Form):
foos = forms.ModelMultipleChoiceField(queryset=models.Foo.objects)

@classmethod
def convert_graphql_input_to_form_input(cls, graphql_input: UpdateFooInput):
return { "foos": [foo["id"] for foo in graphql_input.foos]] }

Extra Processing

Extra processing before save is handled by the form in a prepare_data method. The role this method plays is to prepare any data prior to / without saving. Usually I'd prepare model instances, set values on existing instances and so forth. This allows the save() method to use bulk_create() and bulk_update() easily to keeps save doing just that - saving.

Objects/List of objects that are going to be saved / bulk_created / updated in save are stored on the form. The list is defined / set in init with full typehints. Example:

from typing import List, Optional

class UpdateFooForm(forms.Form):
foos = forms.ModelMultipleChoiceField(queryset=models.Foo.objects)

def __init__(*args, **kwargs)
super().__init__(*args, **kwargs)
self.foo_bars: List[FooBar] = []
self.bar: Optional[Bar] = None

Type Definition Graduation

Types are defined in each query / mutation where possible. As schema grows and multiple queries/mutations or other app's queries/mutations reference the same type, the location where the type is defined changes. This is partially for a cleaner architecture, but also to avoid import errors.

└── apps
├── common
│   ├── schema.py
│   └── types.py  # global types used by multiple apps are defined here
└── hoge
├── mutations
│   ├── create_hoge.py  # types only used by create_hoge are in here
│   └── update_hoge.py
├── queries
│   └── complex_query.py
├── schema.py
└── types.py  # types used by either create/update_hoge and or complex_query are defined here

Example Mutation

The logic kept inside a query/mutation is as minimal as possible. This is as it's difficult to test logic inside the mutation without writing a full-blown end-to-end test.

from graphene_django.types import ErrorType

class UpdateHogeReturnType(graphene.Union):
class Meta:
types = (HogeType, ErrorType)

class UpdateHogeMutationType(graphene.Mutation):

class Meta:
output = graphene.NonNull(UpdateHogeReturnType)

class Arguments:
update_hoge_input = UpdateHogeInputType()

@staticmethod
def mutate(root, info, update_hoge_input: UpdateHogeInputType) -> str:
data = UpdateHogeForm.convert_mutation_input_to_form_input(update_hoge_)
form = MutationValidationForm(data=data)
if form.is_valid():
form.prepare_data()
return form.save()
errors = ErrorType.from_errors(form)
return ErrorType(errors=errors)

Adding Queries/Mutations to your Schema

This architecture tries to consistently follow the graphene standard for defining schema. i.e. when defining your schema you create a class Query and class Mutation, then pass those to your schema schema = Schema(query=Query, mutation=Mutation)

Each app should build its Query and Mutation objects. These will then be imported in the schema.py, combined into a new Query class, and passed to schema.

# hoge/mutations/update_hoge.py

class UpdateHogeMutation:

update_hoge = UpdateHogeMutationType.Field()

# hoge/mutations/schema.py

from .mutations import update_hoge, create_hoge

class Mutation(update_hoge.Mutation,
create_hoge.Mutation):
pass

# common/schema.py

import graphene

import foo.schema
import hoge.schema

class Query(hoge.schema.Query, foo.schema.Query, graphene.GrapheneObjectType):
pass

class Mutation(hoge.schema.Mutation, foo.schema.Mutation, graphene.GrapheneObjectType):
pass

schema = graphene.Schema(query=Query, mutation=Mutation)

Directory Tree Overview

└── apps
├── common
│   ├── schema.py
│   └── types.py
├── foo
│   ├── mutations
│   │   └── create_or_update_foo.py
│   ├── queries
│   │   └── complex_foo_query.py
│   └── schema.py
└── hoge
├── mutations
│   ├── common.py
│   ├── create_hoge.py
│   └── update_hoge.py
├── queries
│   └── complex_query.py
├── schema.py
└── types.py

Went for a drive today (at Leo’s insistence) down to Chigasaki-Shi. Felt a little bad with Yokohama plates in Shonan plate territory.

Didn’t get out of the car, but man it’s so green and nice out there. Saw some huge koinobori too.

One of the common memes to come from covid19 is to post a before-after photo of a famous city or landmark. The before covid19 photo is the city as we’ve become accustomed to it: brown air full of smog. The after covid19 at the same location, but with naturally blue skies and clear air.

With everyone social distancing and automobile/truck traffic near zero we have been given a rare opportunity. We no longer have to imagine what our air and cities could be like if we didn’t drive pollution emitting vehicles everywhere, we can see, taste, and smell it with our own eyes.

Air pollution from cars and trucks have been suffocating our cities slowly, like one boil’s a frog, so we acclimate and brown air becomes “normal” and the way things have always been. With the burner temporary malfunctioning we can see just what a precarious position we’ve put ourselves in.

When this is all done and our lungs have acclimated to clean air we’ll have a choice: do we go back to the way things were and forget what we’ve experienced, or do we the courage to demand a change.

https://twitter.com/sistercelluloid/status/1249027255797460993

Watching videos like this one about UNIX system from 1982 is a great reminder that no matter what you're building today, we all stand on the shoulders of giants. Highly worth 20 minutes of your time.

https://youtu.be/XvDZLjaCJuw