If you don't know about length extension attacks, it is a very simple and straight forward attack that let you forge a new hash by extending another one, letting you pretend that hashing had previously not been terminated.
The attack targets such hashes: SHA-256(key | message) where the key is secret and where | means concatenation.
This is because a SHA-2 hash (unless we're talking about the truncated versions) is literally a full copy of the state of the hash. It is not the state of hashing key and message, but rather key and message and some padding. Because like everything in the symmetric crypto world you need to pad to the block size. I believe this is 512 bits in the Secure Hash Algorithm 2.
The attack lets you take such a hash, and continue the hashing to obtain the hash of key | message | padding | more where more is whatever you want. And all of this without any knowledge of the secret key!
Interestingly, this comes from the way the Merkle-Damgard construction is applied (without a good finalization function). And because of this hash functions like MD4, MD5, SHA-1 and SHA-2 have all suffered from the same issues. You'd be glad to hear that this issue is fixed in any of the SHA-3 contestant (read: BLAKE2 and SHAKE and SHA-3 are fine). Keccak (SHA-3's winner) fixes it by using a Sponge construction, not letting you see a big part of the state (the capacity) while BLAKE2 fixes it by using the HAsh Iterative FrAmework (HAIFA), using a "number of bits hashed so far" (not including the padding) inside of the compression function.
While looking at the exact date length extension attacks were found (which I couldn't find), Samuel Neves came up with an interesting response.
It looks like the NIST was made aware, during the standardization process of SHA-2, that simple fixes would prevent length extension attacks.
This comment from John Kelsey (who later joined the NIST) is from 28 august 2001 (by the way it doesn't make sense to write dates as month/day/year. Nobody can understand it outside of the US. We have an ISO format that specifies a logical year-month-day). In it he talks about the attack, and proposes a simple fix:
Niels Ferguson suggested the following simple fix to me, some time ago: Choose some nonzero constant C0, of the same size as the hash function chaining variable. Hash messages normally, until we come to the last block in the padded message. XOR C0 into the chaining variable input into that last compression function computation. The resulting compression function output is used as the hash result. For concreteness, I propose C0 = 0xa5a5...a5, with the 0xa5 repeated until every byte is filled in. This should be interpreted in little-endian bit ordering.
Why did the NIST ignore this when it could have modified the draft before publication? I have no idea. Is this one more fuck up from their part?
Introduction
The Strobe Protocol Framework is a specification, available here, which you can use to implement a primitive called the Strobe Duplex Construction. The implemented Strobe object should respond to a dozen of calls that can be combined together to allow you to generate random numbers, derive keys, hash, encrypt, authenticate, and even build complex symmetric protocols.
The thing is sexy for several reasons:
- you only use a single primitive to do all of your symmetric crypto
- it makes the code size of your library extremely small, easy to fit in embedded devices and easy to audit
- on top of that it allows you to create TLS-like protocols
- every message/operation of your protocol depends on all the previous messages/operations
The last one might remind you of Noise which is a protocol framework as well that mostly focus on the asymmetric part (handshake). More on that later :)
Overview
From a high level point of view, here is a very simple example of using it to hash a string:
myHash = Strobe_init("hash")
myHash.AD("something to be hashed")
hash = myHash.PRF(outputLen=16)
You can see that you first instantiate a Strobe object with a custom name. I chose "hash" here but it could have been anything. The point is to personalize the result to your own protocol/system: initializing Strobe with a different name would give you a different hash function.
Here two functions are used: AD and PRF. The first one to insert the data you're about to hash, the second one to obtain a digest of 16 bytes. Easy right?
Another example to derive keys:
KDF = Strobe_init("deriving keys for something")
KDF.KEY(keyInput)
key1 = KDF.PRF(outputLen=16)
key2 = KDF.PRF(outputLen=16)
Here we use a new call KEY which is similar to AD but provides forward-secrecy as well. It is not needed here but it looks nicer and so I'll use it. We then split the output in two in order to form two new keys out of our first one.
Let me now give you a more complex example. So far we've only used Strobe to create primitives, what if I wanted to create a protocol? For example on the client side I could write:
myProtocol = Strobe_init("my protocol v1.0")
myProtocol.KEY(sharedSecret)
buffer += myProtocol.send_ENC("GET /")
buffer += myProtocol.send_MAC(len=16)
// send the buffer
// receive a ciphertext
message = myProtocol.recv_ENC(ciphertext[:-16])
ok = myProtocol.recv_MAC(ciphertext[-16:])
if !ok {
// reset the connection
}
Since this is a symmetric protocol, something similar should be done on the server side.
The code above initializes an instance of Strobe called "my protocol v1.0", and then keys it with a pre-shared secret or some key exchange output. Whatever you like to put in there. Then it encrypts the GET request and sends the ciphertext along with an authentication tag of 16 bytes (should be enough). The client then receives some reply and uses the inverse operations to decrypt and verify the integrity of the message. This is what the server must have done when it received the GET request as well. This is pretty simple right?
There's so much more Strobe can do, it is up to you to build your own protocol using the different calls Strobe provides. Here is the full list:
- AD: Absorbs data to authenticate.
- KEY: Absorbs a key.
- PRF: Generates a random output (forward secure).
- send_CLR: Sends non-encrypted data.
- recv_CLR: Receives non-encrypted data.
- send_ENC: Encrypts data.
- recv_ENC: Decrypts data.
- send_MAC: Produces an authentication tag.
- recv_MAC: Verifies an authentication tag.
- RATCHET: Introduce forward secrecy.
There are also meta variants of some of these operations which allow you to specify that what you're operating on is some frame data and not the real data itself. But this is just a detail.
How does it work?
Under its surface, Strobe is a duplex construction. Before I can explain that, let me first explain the sponge construction.
A sponge belongs to a field in cryptography called permutation-based cryptography. This is because at its core, it works on top of a permutation. The whole security of the thing is proven as long as your permutation is secure, meaning that it behaves like a random oracle. What's a permutation? Oh sorry, well, imagine the AES block cipher with a fixed key of 00000000000000000. It takes all the possible inputs of 128-bit, and it will give you all the possible outputs of 128-bit. It's a one-to-one mapping, for one plaintext there is always one ciphertext. That's a permutation.
SHA-3 is based on the sponge construction by the way, and it uses the keccak-f[1600] permutation at its core. Its security was assessed by long years of cryptanalysis (read: people trying to break it) and it works very similarly as AES: it has a series of steps that modify an input, and these steps are repeated many many times in what we call rounds. AES-128 has 10 rounds, Keccak-f[1600] has 24 rounds. The 1600 part of the name means that it has an input/ouput size of 1600 bits.
So here our permutation is Keccak-f[1600], and we imagine that our input/output is divided into two parts: the public part (rate) and the secret part (capacity). Intuitively we'll say that the bigger the secret part is, the more secure the construction is. And indeed, SHA-3 has several flavors that will use different sizes according to the security advertised.
The message is padded and split into multiple blocks of the same size as the public part. To absorb them into our sponge, we just XOR each blocks with the public part of the state, then we permute the state.
To obtain an output from this construction, we just retrieve the public part of our state. If it's not enough, we permute to modify the state of the sponge, then we collect the new public part so that it can be appended to the previous one. And we continue to do that until we have enough. If it's too much we truncate :)
And that's it! It's a sponge, we absorb and we squeeze. Makes sense right?
This is exactly how SHA-3 works, and the output is your hash.
What if we're not done though? What if we want to continue absorbing, then squeeze again, then absorb again, etc... This would give us a nice property: everything that we squeeze will depend on everything that has been absorbed and squeezed so far. This provides us transcript consistency.
The Keccak team said we can, and they created the Duplex construction. It's just something that allows us to absorb, to squeeze, to absorb, to squeeze, and on and on...
Building Strobe
"How is Strobe constructed on top of the Duplex construction?" you may ask. And I will give you an intuition of an answer.
Strobe has fundamentally 3 types of internal operations, that are used to build the operations we've previously saw (KEY, AD, Send_ENC, ...). They are the following:
- default:
state = input ⊕ state
- cbefore:
state = input
- cafter:
output, state = input ⊕ state
The default one simply absorbs the input with the state. This is useful for any kind of operation since we want them to affect the outcome of the next ones.
The cbefore internal operation allows you to replace the bits of the state with your input. This is useful when we want to provide forward-secrecy: if the state is later leaked, the attacker will not be able to recover a previous state since bits of the rate have been erased. This is used to construct the KEY, RATCHET and PRF operations. While KEY replaces the state with bits from a key, RATCHET and PRF replaces the state with zeros.
cafter is pretty much the same as the default operation, except that it also retrieves the output of the XOR. If you've seen how stream ciphers or one-time pads work, you might have recognized that this is how we can encrypt our plaintext. And if it wasn't more obvious to you, this is what will be used to construct the Send_ENC operations.
There is also one last thing: an internal flag called forceF that allows you to run the permutation before using any one of these internal operations. This is useful when you need to produce something from the Duplex construction: a ciphertext, a random number, a key, etc... Why? Because we want the result to depend on what happened previously, and since we can have many operations per block size we need to do this. You can imagine problems if we were not to do that: an encryption operation that would not depend on the previously inserted key for example.
Let's see some examples!
We'll start by keying our protocol. We first absorb the name of the operation (Strobe is verbose). We then permute (via the forceF flag) to start on a fresh block. Since the KEY operation also provides forward-secrecy, the cbefore internal operation is used to replace the bits of the state with the bits of the input (the key).
After that we want to encrypt some data. We'll absorb the name of the operation (send_ENC), we'll permute (forceF) and we'll XOR our plaintext with the state to encrypt it. We can then send that ciphertext, which is coincidentally also part of the new state of our duplex construction.
I'll give you two more examples. We can't just send encrypted data like that, we need to protect its integrity. And why not including some additional data that we want to authenticate:
You'll notice, AD does not need to permute the Strobe state, this is because we're not sending anything (or obtaining an output from the construction) so we do not need to depend on what has happened previously yet. For the send_MAC operation we do need that though, and we'll use the cafter internal operation with an input of 16 zeros to obtain the first 16 bytes of the state.
In these description, I've simplified Strobe and omitted the padding. There is also a flag that is differently set depending on who sent the first message. All these details can be learned through the specification.
Now what?
Go play with it! Here is a list of things:
Note that this is still a beta, and it's still experimental.
I've asked some TLS size questions on Twitter and got some nice results :]
People mostly got it right for the Client Hello. But it wasn't as easy for the Server Hello.
Client Hello → 212 bytes
Server Hello → 66 bytes
These are just numbers I got from a TLS 1.2 handshake with a random website. These numbers are influenced by the browser I use and the configuration of the server. But they should be close to that range anyway as the structure of a Client Hello or a Server Hello are quite simple.
A better question would be: what is the bigger message? and the Client Hello would always win. This is because the Server Hello only replies with one choice from the list of choices the client proposed. For example the server will choose only one ciphersuite from the 13 suites the client proposed. The server will choose one curve from the 3 different curves proposed by the client. The server will choose a single signature algorithm from the client's 10 propositions. And on and on...
Everyone (mostly) got this one!
Certificate → 2540 bytes
Obviously, this is the biggest message of the handshake by far. The number I got is from receiving two different certificates where each certificate is about a thousand bytes. This is because servers tend to send the full chain of certificates to the client, longer chains will increase the size of this message. Probably why there are propositions for a certification compression extension.
ServerKeyExchange → 338 bytes
ClientKeyExchange → 75 bytes
Both of these messages include the peer's public key during ephemeral key exchanges. But the ServerKeyExchange additionally contains the parameters of the key exchange algorithm and a signature of the server's public key. In my case, the signature was done with RSA-2048 and of size 256 bytes, while the NIST p256 public keys were of size 65 bytes.
Using ECDSA for signing, signatures could have been smaller. Using FFDH for the key agreement, public keys could have been bigger.
Tim Dierks also mentioned that using RSA-10000 would have drastically increased the size of the ServerKeyExchange.
Maybe a better question, again, would be which one is the bigger message.
People mostly got it right here!
The rest of the handshake is negligible:
ChangeCipherSpec is just 6 bytes indicating a switch to encryption, it will always be the same size no matter what kind of handshake you went through, most of its length comes from the record's header.
Finished is 45 bytes. Its content is a MAC of the handshake transcript, but an additional MAC is added to protect the integrity of the ciphertext (ciphertext expansion). Remember, Finished is the first (and only) encrypted message in a handshake.
