david wong

Hey! I'm David, cofounder of zkSecurity and the author of the Real-World Cryptography book. I was previously a crypto architect at O(1) Labs (working on the Mina cryptocurrency), before that I was the security lead for Diem (formerly Libra) at Novi (Facebook), and a security consultant for the Cryptography Services of NCC Group. This is my blog about cryptography and security and other related topics that I find interesting.

TLS, Pre-Master Secrets and Master Secrets posted March 2016

Everything you want to know about TLS 1.2 is in RFC 5246. But as you may know, if you've read RFCs before, it is not easy to parse (plus they have some sort of double spaces non-sense).

Before we can encrypt/MAC everything with keys to secure our connection, we need to go over a key exchange called the Handshake to safely agree on a set of keys for both parties to use. The handshake can currently use 5 different algorithms to do the key exchange: RSA, Diffie-Hellman, Elliptic Curve Diffie-Hellman and the ephemeral versions of the last two algorithms.

This blogpost is about what happens between this key exchange and the encryption/authentication of data.

The Pre-Master Secret

The pre-master key is the value you directly obtain from the key exchange (e.g. \(g^{ab} \pmod{p}\) if using Diffie-Hellman). Its length varies depending on the algorithm and the parameters used during the key exchange. To make things simpler, we would want a fixed-length value to derive the keys for any cipher suite we would want to use. This is the reason behind a pre master secret. The fixed-length value we'll call master secret. Here the RFC tells us how to compute it from the pre-master secret after having removed the leading zeros bytes.

master_secret = PRF(pre_master_secret, "master secret",
                    ClientHello.random + ServerHello.random)
                    [0..47];

The two random values ClientHello.random and ServerHello.random, sometimes called "nonces", are randomly generated and sent during the ClientHello of each parties. This is to bound the soon-to-be master key to this session. PRF stands for Pseudo-random function, basically some concrete construction that emulates a random oracle: given an input will produce an output computationally indistinguishable from a truly random sequence. But let's move on, and we will see later what exactly is that PRF.

The Master Secret

A master secret is always 48 bytes. So now that we have a fixed length value, we can derive 4 keys from it:

  • client_write_MAC_key
  • server_write_MAC_key
  • client_write_key
  • server_write_key

As you can probably guess, MAC keys are for the authentication and integrity with whatever MAC algorithm you chose in the cipher suite, write keys are for the symmetric encryption.

Interestingly, two keys are generated for every purpose: one key per side. This is mostly by respect of good practices. Always segregate the use of your keys.

The symmetric ciphers chosen in the handshake will dictate how long these keys we generate need to be. Note that AEAD ciphers that combine both authentication and encryption will not need MAC keys but will need two other keys instead: client_write_IV and server_write_IV. This is because their MAC keys are directly derived from the encryption keys.

The same PRF we used on the pre-master key will be used on the master-key over and over until enough bytes have been created for the keys. From the section 6.3 of the RFC:

key_block = PRF(SecurityParameters.master_secret,
                "key expansion",
                SecurityParameters.server_random +
                SecurityParameters.client_random);

The key_block value is then cut into enough keys.

That's it! Here's a recap:

Diffie-Hellman -> pre-master key -> 48bytes master key -> 4 variable-length keys.

The PRF

OK. Now that we got a nice global view of the process, let's dig deeper. The PRF used in TLS 1.2 is discussed here. It is quite different from the PRF used in TLS 1.1, see here.

Remember, for example how it was used to transform the pre-master key into a master key:

master_secret = PRF(pre_master_secret, "master secret",
                    ClientHello.random + ServerHello.random)
                    [0..47];

This is how the PRF function is used:

PRF(secret, label, seed) = P_<hash>(secret, label + seed)

If you want to follow along with code, here's the relevant golang code

P_hash(secret, seed) = HMAC_hash(secret, A(1) + seed) +
                       HMAC_hash(secret, A(2) + seed) +
                       HMAC_hash(secret, A(3) + seed) + ...

where + indicates concatenation, A() is defined as:

A(0) = seed
A(i) = HMAC_hash(secret, A(i-1))

This was a copy/paste from the RFC. To make it clearer: We use the label string ("master secret" in our example) concatenated with the two peers' random values as a seed.

We then MAC the seed with our pre-master secret as the key. We use the first output. Iterating the MAC gives us the subsequent values that we can append to our output.

\[ u_0 = label + serverHello.random + clientHello.random \]

\[ u_i = HMAC(secret, u_{i-1}) \]

\[ output = u_1 , u_2 , \cdots \] This goes on and on until the output is long enough to cover the 48 bytes of the master key (or the 4 keys if we're applying to PRF on the master key).

If P_256 is being used, then SHA-256 is being used. This means the output of HMAC will be 256 bits (32 bytes). To get the 48 bytes of the master key, two iterations are enough, and the remaining bytes can be discarded.

Well done! You've reached the end of my post. Now you can leave a comment or read something else.

Comments

Ahsan

Very useful information i have so many question about PRF and master secret key but this post cover the all of my question. Thanks for this useful info can you explain client finished message and how to verify client finished message.
I am using TLS 1.2 and my cipher is ECDHE_RSA_WITH_AES_128_SHA256.

david

the client finished message should be a normal TLS handshake message, except that the payload is encrypted this time and the thing should contain a MAC of the previous transcript from the client point of view.

Ahsan

Sorry i am using DTLS 1.2 instead TLS 1.2. Kindly explain the structure of finished message, like how many bytes for "nonce", how many bytes are encrypted data and how many bytes for authentication tag.

david

Ahsan: DTLS is a bit different from TLS. Not sure how it works exactly. Let me take a look. In the mean time you can also take a look there: https://tools.ietf.org/html/rfc6347#section-4.2

david

Ahsan I replied in a different blog post, check: http://cryptologie.net/article/353/dtls-and-finished-messages/

Boris

Thank you for very interesting article! Could you give code prf function on C++?

David

I have better, the source code in Go :)

https://github.com/golang/go/blob/master/src/crypto/tls/prf.go

and the tests:

https://github.com/golang/go/blob/master/src/crypto/tls/prf_test.go

Mike

Small quibble: around where you say "The pre-master key is the value you directly obtain from the key exchange" - shouldn't the b be an exponent?

david

yes Mike! Thanks for pointing that out :)

Mike

Excellent work, btw! Thanks for sharing!

Darshan K.

Quite a stupid query here.
If an attacker gets PRN during the ClientHello, wouldn't he be able to generate the PMS and Master-secret? If not, what protect this? I am sure I am missing a bit here.

david

PRN?

david

if you mean the random numbers in the clientHello and serverHello. These are not enough to generate the PMS. You also need a key, which is the shared secret computed out of the key exchange.

Raj

Very nicely written. I would like to share it with my co-workers to improve their knowledge

tsusanka

Still useful, thank you!

Christian

Thank you very much for your well written article. It really helped me to find a problem im my own TLS implementation.

Samsuddin Sikder

hi
you have provided go programming source code, do you have any link where i can see source code on C program.

[email protected]

darkangel

thanks for the infos, i have C# code for the prf(), if any one need it, just copy and paste, but not sure if it works or not, since i am having trouble decrypting the finish handshake message, but i think it should meet tls 1.0 requirement.

public static byte[] PRF(byte[] secret, byte[] label, byte[] seed, int outputLength)
{
List<byte> s1 = new List<byte>();
List<byte> s2 = new List<byte>();

int size = (int)Math.Ceiling((double)secret.Length / 2);

s1.AddRange(Utils.CopyArray(secret, 0, size));
s2.AddRange(Utils.CopyArray(secret, secret.Length - size, size));

var tbc = new List<byte>();
tbc.AddRange(label);
tbc.AddRange(seed);

var md5Result = MD5Hash(s1.ToArray(), tbc.ToArray(), outputLength);

var sha1Result = SHA1Hash(s2.ToArray(), tbc.ToArray(), outputLength);


var result = new List<byte>();
for (int i = 0; i < outputLength; i++)
result.Add((byte)(md5Result[i] ^ sha1Result[i]));

return result.ToArray();
}
private static byte[] MD5Hash(byte[] secret, byte[] seed, int outputLength)
{
int iterations = (int)Math.Ceiling((double)outputLength / 16);

HMACMD5 HMD5 = new HMACMD5(secret);

var result = new List<byte>();
byte[] A = null;
for (int i = 0; i <= iterations; i++)
if (A == null)
A = seed;
else
{
A = HMD5.ComputeHash(A);

var tBuff = new List<byte>();
tBuff.AddRange(A);
tBuff.AddRange(seed);

var tb = HMD5.ComputeHash(tBuff.ToArray());

result.AddRange(tb);
}

return result.ToArray();
}

private static byte[] SHA1Hash(byte[] secret, byte[] seed, int outputLength)
{
int iterations = (int)Math.Ceiling((double)outputLength / 20);

HMACSHA1 HSHA1 = new HMACSHA1(secret);
var result = new List<byte>();
byte[] A = null;

for (int i = 0; i <= iterations; i++)
if (A == null)
A = seed;
else
{
A = HSHA1.ComputeHash(A);

var tBuff = new List<byte>();
tBuff.AddRange(A);
tBuff.AddRange(seed);

var tb = HSHA1.ComputeHash(tBuff.ToArray());

result.AddRange(tb);
}

return result.ToArray();
}

Marcelo Gonçalves

Man, thanks for the explanation.

i Was searching for a good explanation about these topics.

Thank you !

SSL installation services

That'high, these secrets are fantastic. thanks for the incredible post.

Teo

Excellent, thank you

Saravanan M K

Correction#1:
============

In your example, the value u0 should be:

u0 = label + clientHello.random + serverHello.random

because that particular example explains the generation of 48 byte Master Secret.

For the generation of key block,

u0 = label + serverHello.random + clientHello.random

Correction#2:
============

> how to compute it from the pre-master secret after having removed the leading zeros bytes.

Leading zero bytes should be removed only for DH and DHE. For ECDH and ECDHE, they should be retained.

RFC8422 (and old RFC4992):
==========
5.10. ECDH, ECDSA, and RSA Computations

All ECDH calculations for the NIST curves (including parameter and
key generation as well as the shared secret calculation) are
performed according to [IEEE.P1363] using the ECKAS-DH1 scheme with
the identity map as the Key Derivation Function (KDF) so that the
premaster secret is the x-coordinate of the ECDH shared secret
elliptic curve point represented as an octet string. Note that this
octet string (Z in IEEE 1363 terminology), as output by FE2OSP (Field
Element to Octet String Conversion Primitive), has constant length
for any given field; leading zeros found in this octet string MUST
NOT be truncated.
==========

For RSA, since the first two bytes of the PMS is going to be client TLS version, it will always have non-zero value as the leading byte.

Chvash

Do you have a test vectors for TLS 1.2?
I am trying with
#client rand
# 6E2CC4958409EE76232DFB9CE9A319CA5D004FE5D60C786C7E550CB06E88F601}
#server rand
# 8B66324D4D029852FDAA670D6F0F32662BFC476ED202CA0DB5D896CA4F13B0CA
#PSK: 02B99BF7697763035E8883D4506061F2
I got master key as:
master key:
31e40fca4d311de40ce57d784b027e93342d5314f6b00f373fbf146889cb2f43
And A(1) as:
5f074dfdef4474d60a9dc9e52eac793517c69902e553186e88f4148f4e0d5e84
But I dont get where the follwing tested values are in there.
clientWriteKey =
00A02013931212D0637ED4F1651BC2A9

clientWriteIV =
1A5E165D3F8F404A7E381C687C8B8C73

serverWriteKey =
28F18DBA3553EE27FDE94FDCAA9F35BB

serverWriteIV =
DAE0DFFB22FEF2AB5B6A569A1D78AAC8

Any advice? Maybe my hashMac calculation is being done wrong. (Hmac with SHA256)

Naveen

Can anyone say how to find a(1) manually?

mh

Hi,

Regarding the PRF,
you have replaced A(i) from the RFC by u_i in your text; but the output should not be simply u_1 + u_2 + ...,
but instead HMAC_hash(secret, u_1 + seed) + HMAC_hash(secret, u_2 + seed) + ...
-> Two HMAC calls per iteration.

Cheers,
mh

Daw

Why do we share the PMS with the server? Why can't we share the master key or the 6 derived keys directly to the Server.(Of course after encrypting them with pub-key of Server) ?

Also why can't Server make the PMS and share it with Web Browser?

Burak

Thanks for the topic.
Can u explain the PRF for the master secret and the key_block when extended master secret extension is used.
master_secret = PRF(pre_master_secret, "extended master secret",
session_hash)
[0..47];
is computed like this but there is no information about the PRF used for the key_block derivation.

Original PRF is this as you mentioned in the topic but how it is modified for the EMS.
key_block = PRF(SecurityParameters.master_secret,
"key expansion",
SecurityParameters.server_random +
SecurityParameters.client_random);

leave a comment...