DTLS and Finished messages posted June 2016

Ahsan asked me:

Sorry i am using DTLS 1.2 instead TLS 1.2. Kindly explain the structure of finished message, like how many bytes for "nonce", how many bytes are encrypted data and how many bytes for authentication tag.

I'm writing this because I want to clear things up: you can ask me pretty much anything and I'll do my best to answer here. So please ask away if you have a question in crypto! There is also a contact form for that.

Differences between DTLS and TLS

TLS is an application layer protocol. Some people say that it's a transport layer protocol. This is false. It runs in whatever program you are using on top of TCP/UDP. Although it is used to transport the real content of the application: it can be seen as an intermediate transport layer running in the application layer.

The TLS we know, the one that runs in our browser, typically runs on top of TCP. But in some situations, the developer might want to use UDP to exchange packets. Since UDP forgives packet loss (think multiplayer video games or audio/video conferences), it is important that TLS is setup accordingly to forgive those packet loss as well. For this, we use a similar but different specification than TLS: DTLS.

DTLS is what you use if you need TLS on top of UDP.

The main DTLS RFC documents the differences with TLS. Most of them are modification to the protocol so that the connection doesn't terminate/break if a message is lost, duplicated, out of order, etc...:

• records can't be split into several datagrams
• the sequence number is written in each record
• errors (duplicates, loss, out of order) are tolerated
• no stream cipher can be used (no state can be used if errors are tolerated)
• protections against DoS attacks (apparently DTLS has some problems with DoS attacks)
• ...

Finished messages

The simplest TLS handshake goes like this:

• the client sends its ClientHello packet
• the server replies with his ServerHello packet
• the client sends (his part of) the shared secret in a ClientKeyExchange packet

Now I omitted a bunch of packets that are usually part of the handshake as well. For example:

• the server usually sends his certificate after the ServerHello message.
• the server might also take part in the creation of the shared secret in some modes (including ephemeral modes)

But this is not what is interesting to us here.

After enough messages have been sent to compute the shared secret, a ChangeCipherSpec message is sent by both the client and the server to announce the beginning of traffic encryption. Followed directly by an encrypted Finished message authenticating all the previous handshake messages.

In my knowledge, the Finished message is the only encrypted message of a handshake. It is also the moment where the handshake is "authenticated" and where Man-In-The-Middle attacks usually stop.

Now what is in that Finished message?

Exactly the same things as in TLS. The TLS 1.2 RFC shines a bit more light on the subject:

struct {
    opaque verify_data[verify_data_length];
} Finished;

verify_data = PRF(master_secret, finished_label, Hash(handshake_messages)) [0..verify_data_length-1];

finished_label =
For Finished messages sent by the client, the string "client finished". For Finished messages sent by the server, the string "server finished".

Don't forget that this Finished structure is then encrypted before being sent in a record. The Hash and the PRF we already defined in previous posts, the handshake_messages value is what interest us: it is the concatenation of all the binary data received and sent during the handshake, in order, and not including this Finished one.

Now DTLS has the particularity that some messages are re-sent, out of order, etc... so duplicates must be ignored, real order must be preserved.

How do I know that?

Besides reading the RFC, you might often want to know what's happening for real. To be a bit more informative, let me tell you how I quickly get that kind of information when I'm curious:

• I setup a server key + certificate: openssl req -x509 -new -nodes -keyout key.pem -out server.pem.
• I start the server: openssl s_server -dtls1 -key key.pem -port 4433 -msg.
• I connect to it with a client: openssl s_client -dtls1 -connect localhost:4433 -msg.

The -msg argument will print out the exact content of the messages sent. In the case of the Finished message, it will show the unencrypted hexadecimal data sent. If you want to see the real encrypted data that is sent, you can use the -debug option.

You might also want to have a bit more information about every records. A good way to do this is to record the traffic with tcpdump: sudo tcpdump udp -i lo0 -s 65535 -w handshake.pcap and to open the .pcap file in Wireshark and enter udp && dtls in the filter area.