Friday, 7 February 2014

Twisted Names – now with a (private) EDNS(0) message parser

This is the first in a series of articles describing my progress towards introducing EDNS(0) and DNSSEC support to Twisted Names.

In this article, I'll describe two new private classes designed to parse EDNS(0) messages.

But first a little background.

The EDNS protocol extensions depend on the use of a special DNS record which crucially contains the advertised maximum datagram size, extended return codes, a DNSSECOK flag, and a payload which contains zero or more extensible headers.

This special record is called an OPT record. It isn't a real record though; it doesn't convey any domain names or domain records. It's a hack to work around the lack of space in a standard 512 byte DNS message. All the header flags and fields that couldn't be shoehorned into the 96bit header section of a DNS message are instead encoded into an OPT "pseudo" record.

EDNS clients and servers append one OPT record to the "additional" records in a sent DNS message. This record is then extracted and interpreted by an EDNS aware receiver.

Twisted already has a class for parsing DNS records and it already has a class for parsing DNS messages. I've now created some wrappers which add the EDNS special behaviour to those existing APIs.

The first wrapper is a class called twisted.names.dns._OPTHeader, which wraps an instance of twisted.names.dns.RRHeader. This allows us to re-use the existing record parsing code, and means that the code in _OPTHeader is purely concerned with the special behaviour of an OPT record. Which seems neat! It also made the unit tests much more focused.

The second wrapper is a class called twisted.names.dns._EDNSMessage which is in trunk but not yet in a released version. This wraps an instance of twisted.names.dns.Message. And again, this allows us to re-use the existing parsing code while clearly separating the EDNS specific behaviour.

So Twisted Names can now interpret OPT records and EDNS messages, but we've decided to keep these private for the time being; until we are satisfied with the APIs.

This first version of _EDNSMessage has been written as a drop in replacement for dns.Message. But as a result it has inherited some of the ugliness of that class, namely:

Tom has some ideas about introducing a new Interface for message constructors and I've done some work on improving the abbreviated names, but I got stuck trying to figure out how to introduce the changes while maintaining backwards compatibility.

I'd originally thought those things were blockers, because I intended that users would pass curried message constructors / factories to the various protocols and client APIs, as a way of configuring / overriding EDNS flags and fields in their requests. But that's not a good API (as Tom points out in another comment).

So I've decided that those cleanups would be nice to have, but probably not necessary right now.

A much simpler strategy – and my current priority – is to just switch all Twisted names code (client, protocols, server) to use the new dns._EDNSMessage class in place of dns.Message. This shouldn't introduce any backwards incompatibilities and I'll explicitly disable the use EDNS during encoding (for now). That way Twisted Names clients and servers should continue to behave exactly as before.

But I'll then be free to start work on a DNS client which can be configured to send EDNS(0) requests and which is capable of negotiating with EDNS(0) servers. That's the key requirement of the first project milestone.

(twisted.names.server and the twistd dns plugin will remain non-EDNS for the timebeing.)

In fact I've already started that work, but first I had to fix a bug in the way twisted.names.server generates response messages. My proposed fix includes new functions and methods for generating response messages from a request message. (Previously, the request message was being re-used as the basis for the response, and inevitably, client specific flags and fields were leaking into the response.)

Happily, these new APIs will also allow me to explicitly disable EDNS(0) in twisted.names.server, until such time as we implement server side EDNS(0) support.

So that's all for now. In my next post I hope to be able to report that Twisted Names is using the new _EDNSMessage class internally and summarise some of the other work that I've done on Twisted Names in support of the EDNS(0) and DNSSEC project.

Thanks once again to The NLnet Foundation for funding this project.

Monday, 3 February 2014

The Bristol Twisted Sprint – a belated thankyou

I'd like to thank Luke, Rob and everyone at HybridCluster for organising the Bristol Twisted Sprint in December last year and for their hospitality on the day.

I got quite a bit of work done and it was a rare opportunity to meet Tom and JP, who had flown in from the Americas, and Hynek who had flown in from Berlin.

Earlier that week I'd been amazed to receive an email from Tom with code-reviews of three of my EDNS branches. He'd done them during his flight from Canada! Not sure I'd have had the stamina, given the lack of elbow room and the temptation of in-flight entertainment and complimentary booze. But Tom's a pro and it meant that we had lots to discuss on the day of the sprint. Tom had raised some concerns about my proposed mechanism for setting EDNS parameters from the high level twisted.names.client API – with which I agree. But I'm still struggling to implement the alternative mechanism that we discussed on the day. More on that in a later blog post.

It was also nice to see Hynek, who showed me an interesting custom DNS server which he'd written despite the lack of documentation and despite having to subclass and override the private _lookup method of twisted.names.authority. He laid it on pretty thick, and guilt–tripped me into writing some decent documentation for twisted.names.server. Conscience is what motivates me, so I'm pleased to report that we now have a little more narrative documentation for Twisted Names. And it looks pretty nice too, thanks to khorn's and glyph's work on porting the Twisted documentation to ReST and Sphinx.

JP adopted a "headmasterly" role; supervising things from a lectern at the front of the room (an improvised standing desk), and intimidating us (the pupils) with astonishing Typespeed scores between builds – the keyboard noise was deafening! But I'm exaggerating, JP was as helpful and as thoughtful as usual. Switching context effortlessly to assist us with all areas of Twisted — from development environment tips, to Twisted OpenSSL integration, to EDNS, to PyPy tests and benchmarking, to Tun/Tap APIs. Very impressive!

Let's hope there are more UK Twisted sprints in 2014.

Saturday, 1 February 2014

Twisted Names EDNS — a progress update (or lack thereof)

Apologies to those of you who have been eagerly awaiting news of EDNS and DNSSEC support in Twisted Names. I haven't really got a good excuse; just that I'm a bad communicator and I've been distracted by some mundane (and some exciting) non-computer activities. Sorry.

So, a progress report is long overdue and progress has been a little slower than expected, but let me try and compensate by promising a series of short updates and demos which I'll publish over the next few weeks, demonstrating what has been achieved so far.

Stay tuned!