david wong

Hey ! I'm David, a security consultant at Cryptography Services, the crypto team of NCC Group . This is my blog about cryptography and security and other related topics that I find interesting.

Byte ordering and bit numbering in Keccak and SHA-3 April 2017

As I talked about in a previous blogpost, Keccak and SHA3-3 are different in their bit convention, which gave birth to quite an overly complicated specification. The lack of good explanations in both the reference implementations and the reference documents led me to write this blogpost.

Padding

Before going further, let me briefly re-explain the multi-rate padding also called pad10*1:

  1. take the input and append the delimiter (01 for SHA-3)
  2. append a 1 to start the padding
  3. append enough 0s followed by a final 1 so that the resulting output is a multiple of the rate

Here is a graphical example with input 0x9c = 1001 1100 and where the rate of our permutation is only 3 bytes:

padding

Now, I'll just acknowledge that most implementations, and even the specification, talk about using 0x80 for the final bit. This blogpost should answer why.

Theoretical bit numbering

Keccak is specified in term of bits, but is discreetly aware of byte arrays as well. This is important as the rounds of Keccak require us to XOR parts of the input with lanes of the state. These lanes can be represented as uint64 types for keccak-f[1600], which we'll be looking at in this blogpost.

It could be hard to understand what is really going on in FIPS 202, but a look at an older version of the specification shows that Keccak interprets byte arrays in big-endian, but with an LSB bit numbering.

If you look at the Appendix B.1 of FIPS 202. You can see that before an input can be interpreted by SHA-3, it needs to go through some transformations.

Here is our own example of how it works with an input of 0xaebcdf. First the bits are re-mapped, then a padding is added, finally it is split into as many lanes as possible (here we have two 16-bit lanes):

fips202

The same examples with bits:

fips202-bits

Two things that you must understand here:

  • the padding is applied after re-mapping the bits, not before! Since the padding is already defined for Keccak's internal ordering of its bits.
  • all of this is done so that a bit string is indexed as LSB first, not MSB first. It's weird but Keccak did it this way.

Why is the indexing of a bitstring so important?

Because we later apply the permutation function on the state, and some of the operations need to know what are the indexes of each bits. More on that later.

How to implement the re-mapping

It must be annoying to reverse all these bits right? What about not doing it? Well brace yourself, there is indeed a clever trick that would allow you to avoid doing this. And this trick is what is used in most of SHA-3's implementations. Here is a graphical explanation of what they do:

implementation

By reading the byte array as little-endian, and using an already reversed delimiter + padding, some magic happens!

The result is exactly the same but in reverse order.

If you aren't convinced look at the following diagram (which shows the same thing with bits) and look at the previous section result. Same, but the bitstream is readable in the reversed direction.

implementation-bit

This means that now the index 0 of Keccak's internal ordering is on the right, not on the left. How will it influence the round functions that apply on our lanes?

rounds

It turns out that a lot of operations are unchanged. Who would have known that XORs, ANDs and NOTs operations were not affected by the order of the bits! but only some rotations and bit positioning are. If you look at the reference implementations you will see that indeed, rotations have been reversed (compared to what the reference tells you to do).

And this is how a clever implementation trick made its way in a specification with no real explanation.

You're welcome!

comments (2)

SHA-3, Keccak and disturbing implementation stories April 2017

"Why is Java a big pile of crap?" said the article. And after reading through the argumentation, I would move to the comment section and read a divergent point of view. Who was right? I wondered. Although everyone knows that Java is indeed a huge pile of crap, many other articles follow the same path of disputing one biased opinion that might be right, wrong, but really is both. I often mused on if I would come to write such a piece, and I think today is the day.

Keccak's (and/or SHA-3's) specification makes no sense.

Yup, it makes no sense, and I have a list of points you'll have to read through to understand how exactly.

By the way, if you didn't know, Keccak was the winner of the SHA-3 competition and blablabla...

hash

Chocoweird indexing

First, let me introduce you to the internal state of Keccak.

internal state keccak

It is some sort of rectangular 3D object, and its different sub-objects have different names. Each little cube is a bit (a 1 or a 0). Cute.

I mean why not, AES was a square right? If we make it 3D we augment the security by one dimension. Sounds logical.

It gets weirder.

This is how the little cubes are indexed.

indexing keccak

If you've started to reason on how you could translate that thing into a struct, stop. Don't overthink it. The x and y coordinates don't matter: all operations are done modulo the other columns or rows or you name it... you could have the indexes x = 0 and y = 0 anywhere really; it wouldn't change a thing. This picture doesn't even appear in Keccak's specification, only in FIPS 202. It is probably a joke from the NIST.

Multiple of bytes? Nopecheese

A uint8_t array

...you must have thought, still trying to have a head start, still shooting to figure out how you would stuck these bits in your code.

But wait. The NIST loves to think about bits though, not bytes. That's often surprising to people who end up implementing their specs with a byte length instead of a bit length and it led to what DJB calls a bit attack.

SHA-3 was no exception, NIST said to Keccak: you shall be able to handle the darkest desires of our stupid developers. You must account for ALL INPUTS. You must accept non-multiple of bytes.

Keccak provided. You can now hash 7-bit inputs with SHA-3.

The poor user is given enough rope with which to hang himself -- something a standard should not do. —Rivest, 1992, commenting on the NIST/NSA “DSA” proposal

Bit indexing brainfudge

Let's talk about bit numbering, not byte order like big-endian and little-endian, no-no, bit numbering.

There are two ways to order bits in a byte (and we'll say here that a byte is an octet: it contains 8 bits):

  • MSB first: 0x02 = 0000 0010, 0x01 = 0000 0001, ...
  • LSB first: 0x40 = 0000 0010, 0x80 = 0000 0001, ...

Now what does this have to do with Keccak?

Keccak's bit numbering is LSB first, whereas NIST's one is MSB first (no kidding).

This means a re-mapping needs to take place when converting an input to the internal state of Keccak. This was all explained in the old specification of Keccak (see section 5). Unfortunately these explanations disappeared in the latest version of the spec, and the NIST ended up writing up the conversion in an appendix (B.1) of their own specification FIPS 202.

Let me tell you: FIPS 202's explanation makes no sense. They end up mixing the theoretical internal state of Keccak with methods on how you can implement the re-mapping without having to inverse the bits of each bytes. It took me a while to figure it out and I am not the only one (most questions about Keccak/SHA-3 on internet end up being about their bit re-ordering).

Conclusion

Is SHA-3 Complicated? Some people would say no. But in reality there is no way to understand how to implement SHA-3 without looking at a reference implementation. NIST standards should seriously take a look at the process of TLS 1.3: no-one has been seen copying a reference implementation. Implementers are independently coding up what they understand of the draft specifications, and if cross-testing doesn't work it's either because they failed to understand something or because the specification needs some more love.

Having said that, Keccak is really cool and some of its applications look promising.

comments (2)

One GCM implementation pitfall March 2017

If you look at Go's implementation of GCM, in particular this, you can see that the counter is set to nonce||1:

if len(nonce) == gcmStandardNonceSize {
    // Init counter to nonce||1
    copy(counter[:], nonce)
    counter[gcmBlockSize-1] = 1
} 

It needs to be. Without it, the first block of keystream is the encryption of 0 if the nonce is 0 (which can happen if nonces are generated from a counter). The encryption of 0 is also... the authentication key!

comment on this story

TLS 1.3 - Draft 19 March 2017

Draft 19 has been published, this is a quick post talking about what has changed. This is mosty taken from the draft changelog as well as reading the commits since version 18. I've tried to keep up with what was important, but I ignored the many typos and small fixes, as well as what I think doesn't matter from draft to draft (exporters, cookies, ...)

Add pre-extract Derive-Secret stages to key schedule

The key schedule part of TLS 1.3 is quite tricky. At different stage of the connection a "secret" is derived from different inputs, and used again to derive different keys and IVs (to encrypt/decrypt data). When a new phase begins (for example when handshake messages need to be encrypted, or when the handshake is over and application data messages need to be encrypted) a new secret is derived and new keys can be derived from that secret.

key schedule

Here you can see (the top arrow) that the Handshake secret is derived using the HKDF-Extract() function with the previous secret and the Diffie-Hellman key exchange output. Keys and IVs for both the Client and the Server are then derived from that secret (arrows pointing to the right). Next (all the way down) the Master secret is derived from that secret and the traffic keys, protecting the application data, can be derived from it.

Notice the Derive-Secret() function being used before re-using HKDF-Extract() again. This is new in Draft 19. This Derive-Secret() is the HKDF-Expand() function. If you know HKDF you know that it acts like a lot of different KDFs: in two steps. It extract the entropy given, and it then expands it. This was used to derive keys (you can see it with the arrows pointing on the right), but not to derive secrets. It is now fixed and that's why you can see it being used to derive the new Master secret. One of the positive outcome of this change is that HKDF can now more easily be replaced with another KDF.

Consolidate "ticket_early_data_info" and "early_data" into a single extension

This was an easy one.

The early_data extension was an empty extension used by a Client in the ClientHello message when it wanted to send 0-RTT data; and by a Server in the EncryptedExtensions message to confirm acceptance of the 0-RTT data.

The ticket_early_data_info was an extension that a Server would send in a ticket (for session resumption) to advertise to the Client that 0-RTT was available. It only contained one field: the maximum size of the data that should be sent as 0-RTT data.

Both are now merged under early_data which can be used for both purposes. Less extensions :) it's a good thing.

struct {} Empty;

struct {
   select (Handshake.msg_type) {
       case new_session_ticket:   uint32 max_early_data_size;
       case client_hello:         Empty;
       case encrypted_extensions: Empty;
   };
} EarlyDataIndication;

Change end_of_early_data to be a handshake message

To use the 0-RTT feature of TLS 1.3, a client can start sending 0-RTT data right after a ClientHello message. It is encrypted with a set of keys and IVs derived from the PSK and the ClientHello. The PSK will typically be some resumption secret from a previous connection, but it can also be a pre-shared key (yeah, you can't do 0-RTT if you don't already share some secret with the server). The image below is taken from the key schedule section.

key schedule early data

When a client is done sending this 0-RTT data, it can then finish the handshake by sending the remaining handshake messages (Certificate, CertificateVerify, Finished). These remaining handshake messages are encrypted (new in TLS 1.3) under a handshake key. To warn the server that the client is done sending 0-RTT data and that it is now using the handshake key to encrypt messages, it would previously (draft 18) send a end_of_early_data alert. This alert is now a handshake message! Which is what it should have been all along :)

Add state machine diagram

New diagrams representing the different client/server state machines made their way in draft 19! TLS 1.3 will officially be the first TLS RFC to provide so many diagrams. And they're beautiful too, what an artistic performance.

server workflow

tls 1.3 state machine client

client workflow

tls 1.3 state machine server

That's about it

I haven't spotted anything else major. Some SHOULDs became MUSTs, and some MUSTs became SHOULDs. At the moment, I believe the PSK and 0-RTT flows are quite hard to understand. Perhaps a diagram representing the flow of PSK from a server or client would be nice to have. Eric Rescorla has pointed out a few questions that should be answered to move the draft forward. I've seen so many issues, PR and commits these last weeks that I'm wondering when TLS 1.3 will be considered ready... but it's moving forward :)

comments (1)

NCC Con and the NCC T-shirt January 2017

After Real World Crypto, like every year, NCC Con follows next.

NCC Con is the NCC Group conference, a conference for its employees only, with a bunch of talks and drinks. I gave a talk on TLS 1.3 that I hope I can translate to a blog post at some point.

This year I also designed the NCC Group's Mascot! Which was given away as a T-shirt and a sticker to all the employees at the conference. It was a pretty surreal moment for me to see people around me, at the conference and in the casinos (it was in Vegas) wearing that shirt I designed :D

tshirt

sticker

And here is a wallpaper with my original submission

comments (2)

Real World Crypto 2017: Day 3 January 2017

(The notes for day 2 are here.)

The first talk on Quantum Computers was really interesting, but unfortunately mostly went over my head. Although I'm glad we had a pro who could come and talk to us on the subject. The take away I took from it was to go read the SMBC comics on the same subject.

After that there was a talk about TPMs and DAA for Direct Anonymous Attestation. I should probably read the wikipedia page because I have no idea what that was about.

Helena Handschuh from Cryptography Research talked about DPA Resistance for Real People. There are many techniques we use as DPA countermeasures but it seems like we still don't have the secret sauce to completely prevent that kind of issues, so what we really end up doing is rotating the keys every 3 encryption/decryption operations or so... That's what they do at Rambus, and at least what I've heard other people doing when I was at CHES this year. Mike Hamburg describes the way they rotate keys in his STROBE project a bit. Handschuh also talked about the two ways to certify a DPA-resistant product. There are evaluations like Common Criteria, which is usually the normal route, but now there is also validation. Don't ask me about that.

David Cash then entered the stage and delivered what I believe was the best talk of the conference. He started with a great explanation of ORE vs OPE. OPE (Order Preserving Encryption) allows you to encrypt data in a way that ciphertexts conserve the same lexicographic order, ORE (Order Revealing Encryption) does not, but some function over the ciphertexts end up revealing the order of the plaintexts. So they're kind of the same in the end and the distiction doesn't seem to really matter for his attacks. What matters is the distinction between Ideal ORE and normal ORE (and the obviously, the latter is what is used in the real world).

Ideal ORE only reveals the order while the ORE schemes we use also reveal other things, for example the MSDB (most significant different bits) which is the position of the first non-similar bit between two plaintexts.

Previous research focused on attacking a single column of encrypted data while their new research attacks columns of correlated data. David gives the example of coordinates and shows several illustrations of his attack revealing an encrypted image of the linux penguin, encrypted locations on a map or his go-about saved and encrypted by a GPS. Just by looking at the order of coordinates everything can be visually somewhat restored.

Just by analyzing the MSDB property, a bunch of bits from the plaintexts can be restored as well. It seemed like very bad news for the ORE schemes analyzed.

Finally, two points that seemed really important in this race for the perfect ORE scheme is that first: the security proofs of these constructions are considering any database data as uniformly random, whereas we know that we rarely need to store completely random data :) Especially columns are often correlated with one another. Second, even an (hypothetical) ideal ORE was vulnerable to their research and to previous research (he gave the example of encrypting the whole domain in which case the order would just reveal the plaintexts).

This is a pretty bad stab at ORE scheme in general, showing that it is intuitively limited.

Paul Grubbs followed with an explanation of BoPETS, a term that I believe he recently coined, meaning "Building on Property revealing EncrypTion". He gave a good list of products that I replicated below in a table.

Order Preserving Encryption SAP, Cipherbase, skyhigh, CipherCloud, CryptDB
Searchable Encryption Shadowcrypt, Mylar, kryptonostik, gitzero, ZeroDB, CryptDB
Deterministic Encryption Perspecsys, skyhigh, CipherCloud, CryptDB

They looked at Mylar and saw if they could break it from 3 different attacker setups: a snapshot setup (smash and grab attack), passive (attacker got in, and is now observing what is happening), active (attacker can do pretty much anything I believe).

Mylar uses some encrypted data blob/structure in addition to a "principal graph" containing some metadata, ACL, etc... related to the encrypted data. Grubbs showed how he could recover most of the plaintexts from all the different setups.

Tal Malkin interjected that these attacks would probably not apply to some different PPE systems like IBM OXT. Grubbs said it did not matter. Who's right, I don't know.

As for the active attacker problem, there seem to exist no intuitive solution there. If the attacker can do anything, you're probably screwed.

Raluca Ada Popa (Mylar) followed Grubbs by talking about her product Mylar and rejected all of Grubbs claims, saying that there were out of the scope of Mylar OR were attacking mis-use of the product. IIRC the same thing happened when CryptDB was "broken", CryptDB released a paper arguing that these were false claims.

After Mylar, Popa intend to release two new products with better security: Verena and Opaque.

David Mcgrew mentionned Joy and gave a long talk about detecting PRNG failures. Basically look for public values affected by a PRNG like signatures or the server/client random in TLS.

And that was it. See you next year.

If you have anything to say about my notes, the talks, or other people's notes, please do so in the comments :)

There was a HACS workshop right after RWC, and Tim Taubert wrote some notes on it here.

comment on this story

Real World Crypto 2017: Day 2 January 2017

Here we go again. Some really short notes as well for today. (The notes for day 1 are here.)

Trevor Perrin talked about Message Encryption from an historical point of view, from key directories to public key infrastructures and how to authenticate users to each other. Something interesting that Trevor talked about was CONIKS, some sort of Certificate Transparency-like protocol but for secure messaging (they call it key transparency).

when Alice wants to send a secure message to some other user, say Bob, her CONIKS client looks up Bob's key at the key directory, and verifies that this key has not changed unexpectedly over time. It also checks that this key is consistent with the key other clients are seeing for Bob. Only if these two consistency checks pass will the CONIKS client send Alice's message to Bob. The CONIKS client also performs these same checks for Alice's own key on a regular basis to ensure that the service provider is not tampering with Alice's key.

This sounds like an audit system (users can check what a key distribution server has been up to) + a gossip protocol (users can talk between them to verify consistency of the obtained public keys). Which seems like an excellent idea and makes me wonder why would Signal not use it.

djb mentioned the Self-Healing feature of ZRTP, similar to the recovery feature of Signal.

ZRTP caches symmetric key material used to compute secret session keys, and these values change with each session. If someone steals your ZRTP shared secret cache, they only get one chance to mount a MiTM attack, in the very next session. If they miss that chance, the retained shared secret is refreshed with a new value, and the window of vulnerability heals itself, which means they are locked out of any future opportunities to mount a MiTM attack. This gives ZRTP a "self-healing" feature if any cached key material is compromised.

Signal's self-healing property comes from the fact that an ephemeral Diffie-Hellman key agreement is continuously happening during communication. Like ZRTP it seems like it works out well only if the attacker is slow to act, thus it doesn't seem to be exactly comparable to backward secrecy (which might just be impossible in a protocol).

Later, someone (I don't know who from Felix Günther, Britta Hale, Tibor Jager and Sebastian Lauer because the program doesn't specify who the speaker is), presented on a 0-RTT system that would provide forward secrecy and anti-replayability. 0-RTT is one of the feature of TLS 1.3, which allows a client to start sending encrypted data to a server during its very first flight of messages. Unfortunately, and this was the topic of many discussions on TLS 1.3, these messages are replayable. The work builds on top of Math Green's work with Puncturable Encryption where the server (and the client?) use some key derivation system and remove parts of it after a message has been sent using the 0-RTT feature. I am not sure if this system is really efficient though, especially since the point of 0-RTT is to be able to be fast. If this solution isn't faster or, worse, slower than doing a normal TLS 1.3 handshake (1.5 round trips) then the 0-RTT has no meaning in life anymore.

It also seems like this wouldn't be applicable to the "ticket" way of doing 0-RTT in TLS 1.3, which basically encrypts the whole state and hand up the opaque blob to the client, this way the server doesn't store anything.

Hugo Krawczyk (the HKDF guy) talked about passwords and leaks with some Comic Sans MS (and there was this handy website to check if your username/password/... had been compromised). Hugo then presented some of his recent work on SPHINX, PPSS, X-PAKE, ... everything is listed with link to papers here.

SPHINX is a client-focused and transparent-to-the-server password manager (like all of them really). The desktop password manager uses some derivation parameter stored online or on a user's mobile phone to derive any website key from a master password. The online service or the mobile phone never sees anything (thanks to a simple blinding technique, reminding me of what Ari Juel did last year's RWC with PASS). Because of that, no offline attack are possible. The slides are here and are pretty self explanatory. I have to admit that the design makes a lot of sense to me. I dozed off for the second part of the talk but it was about "How to store a secret" and his PPSS thing (Password Protected Secret Sharing), same for the third part of the talk that was about X-PAKE, which I can imagine was a mix of his ideas with the PAKE protocol.

There were two talks about memory-hardness and proving that password hashing functions are memory-hard. It seemed like some people think it's important that these functions be data-independent as well (probably because in some cases cache attacks might be an issue). Most of the techniques here seemed to make sure that a minimum amount of memory was to be used at all time, and that this couldn't be reduced. I would have liked to see a comparison between Argon2 (the winner of the PHC), Blake 2 (which seems to be the thing people like and use) and Balloon Hashing (which seems to be Dan Boneh's new thing).

George Tankersley and Filippo Valsorda finished the day with a talk on Cloudflare and their CAPTCHA problem. A lot of attacks/spam seems to come from TOR, which has deteriorated the reputation of the TOR nodes' IPs. This means that when Cloudflare sees some traffic coming from TOR, it will present the user with a CAPTCHA to make sure they are dealing with a human. TOR users tend to strongly dislike Cloudflare because these CAPTCHA are shown for every different website, and for every time the TOR path is changed (10 minutes?). This, in addition to TOR already slowing down your browsing efficiency, has annoyed more than one person. Cloudflare is trying to remediate the problem by giving not one, but N tokens to the user for one CAPTCHA solved. By using blind signatures Cloudflare hopes to demonstrate its inability to deanonymize users by using a CAPTCHA token as a tracking cookie.

(I have been made aware of this problem in the past and have manually added TOR visitors as an "allowed country" in my Cloudflare's setup for cryptologie.net., which is one of the solution given to Cloudflare's customers.)

I believe the drafted solution is readable here if you're interested.

Here are more resources:

The notes for day 3 are here.

comment on this story

Real World Crypto 2017: Day 1 January 2017

Today was the first day of Real World Crypto. My favorite con (I think I've been saying that enough). I have avoided taking long notes about this one (as I did for the previous one). But fortunately a live stream was/is available here.

The Lechvin prize was given to Joan Daemen, co-inventor of AES and SHA3, and to Moxie Marlinspike and Trevor Perrin for their work on the development on secure messaging.

Daemen talked about how block cipher might become a thing from the past, replaced by more efficient and faster permutation constructions (like the permutation-baed sponge construction they developed for SHA3).

Moxie Marlinspike gave an brilliant speech. Putting that into words would only uglify whatever was said, you will have to take my words for it.

Rich Salz gave a touching talk about the sad history of OpenSSL.

Thai Duong presented his Project Wycheproof that test java cryptographic libraries for common cryptographic pitfalls. They have something like 80 test vectors (easy to export to test other languages) and have uncovered 40+ vulnerabilities. One is being commented here.

L Jean Camp gave a talk on some X.509 statistics across phishing websites and the biggest websites (according to some akamai ranking). No full ipv4 range stats. Obviously the phishing websites were not bothering with TLS. And the speaker upset several people by saying that phishing websites should not be able to obtain certificates for similar-looking domains. Adam Langley took the mic to explain to her how orthogonal these issues were, and dropped the mic with a "we will kill the green lock".

Quan Nguyen gave a nice talk about some fun crypto vulns. Unfortunately I was dozing off, but everyone seemed to have appreciated the talk and I will be checking these slides as soon as they come up. (Some "different" ways to retrieve the authentication key from AES-GCM)

Daniel Franke presented the NTS (Network Time Security) protocol. It looks like it could protect NTP. Is it a competitor of roughtime? On the roughtime page we can read:

The obvious answer to this problem is to authenticate NTP replies. Indeed, if you want to do this there‘s NTPv4 Autokey from six years ago and NTS, which is in development. A paper at USENIX Security this year detailed how to do it so that it’s stateless at the server and still mostly using fast, symmetric cryptography.

But that's what NTP should have been, 15 years ago—it just allows the network to be untrusted. We aim higher these days.

So I guess NTS is not coming fast enough, hence the creation of roughtime. I personally like how anyone can audit roughtime servers.

Tancrède Lepoint presented on CRYSTAL, a lattice-based key exchange that seems like a competitor to New Hope. He also talked about Open Quantum Safe that contains a library of post quantum primitives as well as a fork of OpenSSL making use of this library. Someone from the public appeared to be pretty angry not to be cited first in the research, but the session chair (Dan Boneh) smoothly saved us from an awkward Q/A.

Mike Hamburg came up with STROBE, a bespoke TLS-like protocol based on one sponge construction. It targets embedded devices but isn't really focusing on speed (?) It's also heavily influenced by BLINKER and tries to improve it. It kinda felt like a competitor of the Noise Protocol Framework but looking at the paper it seems more confusing than that and much more interesting as well. From the paper:

Strobe is a framework for building cryptographic two-party protocols. It can also be used for symmetric cryptosystems such as hashing, AEAD, MACs, PRFs and PRNGs. It is also useful as the symmetric part of a Schnorr-style signature scheme.

That's it. If anyone can point me to other notes on the talks I'd gladly post a list of links in here as well:

The notes for day 2 are here.

comments (2)

About Sweet32 November 2016

I've been thinking a lot about sweet32 recently. And I decided to try to reproduce their results.

First let me tell you that the attack is highly impractical. It requires the user to execute some untrusted javascript for dozens of hours without interruption. The other problem that I encountered was that I couldn't reach the amount of requests they were able to make a client send. In their paper, they claim to be able to reach up to 2,000 requests per second.

I tried to achieve such good numbers with "normal" browsers, and the amount of requests I was able to make was ridiculously low. Then I realized that they used a specific browser: Firefox Developer Edition. A browser made for developing websites. For some unknown reason, it was true that this specific browser was able to send an impressive amount of requests per second. Although I was never able to reach that magical number of 2,000. And even then, who really uses Firefox Developer Edition?

It should be noted that their attack was done in a lab, with a small distance between the client and the server, under perfect condition, when no other traffic was slowing down the attack, etc... I can't imagine this attack being practical at all in real settings.

Note that I can imagine different settings than TLS, at a different point in time in the future, being able to send enough requests per second that this attack would be deemed practical. And in that sense, Sweet32 should be taken seriously. But for now, and especially in the case of TLS, I wouldn't freak out if I see a 64-bit block cipher being used.

comments (1)

1/n-1 split to circumvent BEAST November 2016

A lot of attacks are theorized only to become practical years or decades later. This was the case with Bleichenbacher's and Vaudenay's padding oracle attacks, but also BEAST.

Realizing that Chosen-plaintext attacks were do-able on TLS -- because browsers would execute untrusted code on demand (javascript) -- a myriad of cryptanalysts decided to knock down on TLS.

POODLE was the first vulnerability that made us deprecate SSL 3.0. It broke the protocol in such a critical way that even a RFC was published about it.

BEAST was the one that made us move away from TLS 1.0. But a lot of embedded devices and products still use these lower versions and it would be unfair not to say that an easy patch can be applied to implementations of TLS 1.0 to counter this vulnerability.

BEAST comes from the fact that in TLS 1.0 the next message being encrypted with CBC will use the previous ciphertext's last block as IV. This makes the IV predictable and allow you to decrypt ciphertexts by sending many chosen plaintexts.

cbc encryption

the diagram of CBC for encryption taken from wikipedia. Here imagine that the IV is the previous ciphertext's last block.

The counter measures server-side are well known: move to greater versions of TLS. But if the server cannot fix this, one simple counter measure can be applied on the client-side (remember, this is a client-side vulnerability, it allows a MITM attacker to recover session IDs, cookies, etc...).

Again: BEAST works because the MITM attacker can predict the next IV. He can just observe the previous ciphertext block and craft the plaintext based on it. It's an interactive attack.

One way of preventing this is to send an empty message before sending each message. The empty message will produce a ciphertext (of essentially the MAC), which the attacker will not be able to predict. The message that the attacker asked the browser to encrypt will thus be encrypted with this unpredictable IV. The attacked is circumvented.

This counter measure is called a 0/n split.

diagram 0 split

Unfortunately a lot of servers did not like this countermeasures too much. Chrome pushed that first and kind of broke the web for some users. Adam Langley talks about them paying the price for fixing this "too soon". Presumably this "no data" message would be seen by some implementations as a EOF (End Of File value).

One significant drawback of the current proposed countermeasure (sending empty application data packets) is that the empty packet might be rejected by the TLS peer (see comments #30/#50/others: MSIE does not accept empty fragments, Oracle application server (non-JSSE) cannot accept empty fragments, etc.)

To fix this, Firefox pushed a patch called a 1/n-1 split, where the message to be sent would be split into two messages, the first one containing only 1 byte of the plaintext, and the second one containing the rest.

cbc split

If you look at a fixed client implementation sending messages over a negotiated TLS 1.0 connection, you will see that first it will send the first byte (in the screenshot below, the "G" letter), and then send the rest in a different TLS message.

cbc split burp

If you're curious, you can see that being done in code in the recent BearSSL TLS library of Thomas Porning.

static unsigned char *
cbc_encrypt(br_sslrec_out_cbc_context *cc,
    int record_type, unsigned version, void *data, size_t *data_len)
{
    unsigned char *buf, *rbuf;
    size_t len, blen, plen;
    unsigned char tmp[13];
    br_hmac_context hc;

    buf = data;
    len = *data_len;
    blen = cc->bc.vtable->block_size;

    /*
     * If using TLS 1.0, with more than one byte of plaintext, and
     * the record is application data, then we need to compute
     * a "split". We do not perform the split on other record types
     * because it turned out that some existing, deployed
     * implementations of SSL/TLS do not tolerate the splitting of
     * some message types (in particular the Finished message).
     *
     * If using TLS 1.1+, then there is an explicit IV. We produce
     * that IV by adding an extra initial plaintext block, whose
     * value is computed with HMAC over the record sequence number.
     */
    if (cc->explicit_IV) {
        /*
         * We use here the fact that all the HMAC variants we
         * support can produce at least 16 bytes, while all the
         * block ciphers we support have blocks of no more than
         * 16 bytes. Thus, we can always truncate the HMAC output
         * down to the block size.
         */
        br_enc64be(tmp, cc->seq);
        br_hmac_init(&hc, &cc->mac, blen);
        br_hmac_update(&hc, tmp, 8);
        br_hmac_out(&hc, buf - blen);
        rbuf = buf - blen - 5;
    } else {
        if (len > 1 && record_type == BR_SSL_APPLICATION_DATA) {
            /*
             * To do the split, we use a recursive invocation;
             * since we only give one byte to the inner call,
             * the recursion stops there.
             *
             * We need to compute the exact size of the extra
             * record, so that the two resulting records end up
             * being sequential in RAM.
             *
             * We use here the fact that cbc_max_plaintext()
             * adjusted the start offset to leave room for the
             * initial fragment.
             */
            size_t xlen;

            rbuf = buf - 4 - ((cc->mac_len + blen + 1) & ~(blen - 1));
            rbuf[0] = buf[0];
            xlen = 1;
            rbuf = cbc_encrypt(cc, record_type, version, rbuf, &xlen);
            buf ++;
            len --;
        } else {
            rbuf = buf - 5;
        }
    }

    /*
     * Compute MAC.
     */
    br_enc64be(tmp, cc->seq ++);
    tmp[8] = record_type;
    br_enc16be(tmp + 9, version);
    br_enc16be(tmp + 11, len);
    br_hmac_init(&hc, &cc->mac, cc->mac_len);
    br_hmac_update(&hc, tmp, 13);
    br_hmac_update(&hc, buf, len);
    br_hmac_out(&hc, buf + len);
    len += cc->mac_len;

    /*
     * Add padding.
     */
    plen = blen - (len & (blen - 1));
    memset(buf + len, (unsigned)plen - 1, plen);
    len += plen;

    /*
     * If an explicit IV is used, the corresponding extra block was
     * already put in place earlier; we just have to account for it
     * here.
     */
    if (cc->explicit_IV) {
        buf -= blen;
        len += blen;
    }

    /*
     * Encrypt the whole thing. If there is an explicit IV, we also
     * encrypt it, which is fine (encryption of a uniformly random
     * block is still a uniformly random block).
     */
    cc->bc.vtable->run(&cc->bc.vtable, cc->iv, buf, len);

    /*
     * Add the header and return.
     */
    buf[-5] = record_type;
    br_enc16be(buf - 4, version);
    br_enc16be(buf - 2, len);
    *data_len = (size_t)((buf + len) - rbuf);
    return rbuf;
}

Note that this does not protect the very first byte we send. Is this an issue? Not for browsers. But the next time you encounter this in a different setting, think about it.

comment on this story

What Diffie-Hellman parameters to use? October 2016

I see some discussions on some mailing lists about what parameters to use for Diffie-Hellman (DH).

It seems like the recent line of papers about weak Diffie-Hellman parameters (Logjam) and Diffie-Hellman backdoors (socat, the RFC 5114, the special primes, ...) has troubled more than one.

Less than two weeks ago, a study from Dorey et al. based on these previous results was released, uncovering many problems in how Diffie-Hellman is implemented (or even backdoored!) in the wild.

This is a non-problem. We don't need a RFC to choose Diffie-Hellman groups. A simple openssl gendh -out keyfile -2 2048 will generate a 2048-bit safe prime along with correct DH parameters for you to use. If you're worried about "special primes" issues, either make it yourself with this command, or pick a larger (let's say 4096-bit safe prime) from a list and verify that it's a safe prime. You can use this tool for that.

But since some people really don't want to do the work, here are some safe parameters you can use.

2048-bit parameters for Diffie-Hellman

Here's is the .pem file containing the parameters:

-----BEGIN DH PARAMETERS-----
MIIBCAKCAQEA7WJTTl5HMXOi8+kEeze7ftMRbIiX+P7tLkmwci30S+P6xc6wG1p4
SwbpPyewFlyasdL2Dd8PkhYFtE1xD3Ssj1De+P8T0UcJn5rCHn+g2+0k/CalysKT
XrobEzihlSLeQO1NsgBt1F1XCMO+6inLVvSGVbb3Cei4q+5Djnc7Yjjq0kxGY6Hd
ds/YQnyc1xdJU8NBi3zO1XY2Uk6BSd+NN5KnLh9zRq8t/b0RiIb/fY9mJ9BCtgPo
2m4AfJE8+5dE1ttpQAJFSlA8Ku3/9Vp8sMMWATVk2Q1z9PdkikKQYRfMPYDBSIa/
8Y2l9Hh7vNYOwXd4WF5Q55RHP46RB+F+swIBAg==
-----END DH PARAMETERS-----

You can parse it yourself by piping it to openssl dh -noout -text. It uses 2 as a generator and this big hexstring as a safe prime:

ed62534e5e473173a2f3e9047b37bb7ed3116c8897f8feed2e49b0722df44be3fac5ceb01b5a784b06e93f27b0165c9ab1d2f60ddf0f921605b44d710f74ac8f50def8ff13d147099f9ac21e7fa0dbed24fc26a5cac2935eba1b1338a19522de40ed4db2006dd45d5708c3beea29cb56f48655b6f709e8b8abee438e773b6238ead24c4663a1dd76cfd8427c9cd7174953c3418b7cced57636524e8149df8d3792a72e1f7346af2dfdbd118886ff7d8f6627d042b603e8da6e007c913cfb9744d6db694002454a503c2aedfff55a7cb0c316013564d90d73f4f7648a42906117cc3d80c14886bff18da5f4787bbcd60ec17778585e50e794473f8e9107e1

4096-bit parameters for Diffie-Hellman

Here's the .pem file:

-----BEGIN DH PARAMETERS-----
MIICCAKCAgEA66H+vEJiWaduRv/wwBhPkW1ivReF8yaq2W7166jcw62i069pkXlK
EeLtSDtGKiHu2YrjN03uf+q2urPhG+KXCMBgONlz7eGfPAD/xVSqK53jT8DFgqo6
Y1BccaJeU1iVy3GAVG1xS0aVPqyiNFDGpfd2WSuPIAi3tlgfD0Iu0sCBDLNcuA0d
4pCqBqEKOunD/vDcxp5jnVcj+36fozIkSQ3M2AzG6r44Qb0LFRrAuBM7QKs13COt
oGZwAJE/9GdfNC0jQsScrIVYV0MATv81mbFDAE6e0rLVqMeCdIY7gH0A0qWUVA4S
I3Mr5iPTYxFlA+vWuBPdZ1OXiQN5pzx0TTdnfkI6Q23rOeJG5eIa/LIZ/B+0OlqF
W8UwJL1eZoQGOtx9Al285OIiPktH0fJcpkfbMUmBG9r7xYuCw9vUQ1ed+BIQ366+
9mDTKjTOtmw7HahV48W/T8OPSoSFe/QgnX0VB7c6pnWZ2d4WFu2jVEeG3/pzqMM0
/VCFPXM6Zc0iat5V9qcn2OnOhaSjItuGEc75+8OIeEcdhAUq+PgkKoUIwCNv65RA
A17Skfgi3CLeLy5nKRd7nNv13PSXpjOkNYvRjjatEHZY9DTlfV0vqAlTZNjAt/yx
yNunxd5Sg1kxD1dZCkJXhAbcR8bPSU6OgjSDvGvRk+Dv3Qv5MtTWYisCAQI=
-----END DH PARAMETERS-----

And here is the hexstring value of the safe prime (note that it still uses 2 as a generator):

eba1febc426259a76e46fff0c0184f916d62bd1785f326aad96ef5eba8dcc3ada2d3af6991794a11e2ed483b462a21eed98ae3374dee7feab6bab3e11be29708c06038d973ede19f3c00ffc554aa2b9de34fc0c582aa3a63505c71a25e535895cb7180546d714b46953eaca23450c6a5f776592b8f2008b7b6581f0f422ed2c0810cb35cb80d1de290aa06a10a3ae9c3fef0dcc69e639d5723fb7e9fa33224490dccd80cc6eabe3841bd0b151ac0b8133b40ab35dc23ada0667000913ff4675f342d2342c49cac85585743004eff3599b143004e9ed2b2d5a8c78274863b807d00d2a594540e1223732be623d363116503ebd6b813dd675397890379a73c744d37677e423a436deb39e246e5e21afcb219fc1fb43a5a855bc53024bd5e6684063adc7d025dbce4e2223e4b47d1f25ca647db3149811bdafbc58b82c3dbd443579df81210dfaebef660d32a34ceb66c3b1da855e3c5bf4fc38f4a84857bf4209d7d1507b73aa67599d9de1616eda3544786dffa73a8c334fd50853d733a65cd226ade55f6a727d8e9ce85a4a322db8611cef9fbc38878471d84052af8f8242a8508c0236feb9440035ed291f822dc22de2f2e6729177b9cdbf5dcf497a633a4358bd18e36ad107658f434e57d5d2fa8095364d8c0b7fcb1c8dba7c5de528359310f57590a42578406dc47c6cf494e8e823483bc6bd193e0efdd0bf932d4d6622b
comments (3)

Looking for a crypto intern at Cryptography Services! October 2016

internship

The Cryptography Services team of NCC Group is looking for a summer 2017 intern!

We are looking for you if you're into cryptography and security! The internship would allow you to follow consultants on the job as well as lead your own research project.

Who are we? We are consultants! Big companies come to us and ask us to hack their stuff (legally), review their code and advise on their design. If we're not doing that, we spend our time reading papers, researching, attending conferences, giving talks and teaching classes, ... whatever floats our boat. Not one week is like the other! If you've spent some time doing cryptopals challenges you will probably like what we are doing.

We can't say much about who are the clients we work for, except for the public audits we sometimes do. For example we've performed public audits for TrueCrypt, OpenSSL, Let's Encrypt, Docker and more recently Zcash.

I was myself the first intern of Cryptography Services and I'd be happy to answer any question you might have =)

Take a look at the challenges here if you're interested!

comments (4)

Common x509 certificate validation/creation pitfalls September 2016

I wrote a gist here on certificate validation/creation pitfalls. I don't know if it is up for release but I figured I would get more input, and things to add to it, if I would just released it. So, go check it out and give me your feedback here!

Here's a copy of the current version:

Certificate validation/creation pitfalls

A x509 certificate, and in particular the latest version 3, is the standard for authentication in Public Key Infrastructures (PKIs). Think about Google proving that he's Google before you can communicate with him.

So. Heh. This x509 thing is a tad complicated. Trying to parse such a thing usually end up in the creation of a lot of different vulnerabilities. I won't talk about that here. I will talk about the other complicated thing about them: using them correctly!

So here's a list of pitfalls in the creation of such certificates, but also in the validation and use of them when encountering them in the wild wild web (or in your favorite infrastructure).

  1. KeyUsage
  2. Validity Dates
  3. Critical Extensions
  4. Hostname Validation
  5. Basic Constraints
  6. Name Constraint

KeyUsage

explanation: keyUsage is a field inside a x509 v3 certificate that limits the power of the public key inside the certificate. Can you only use it to sign? Or can it be used as part of a key Exchange as well (ECDH)? etc...

relevant RFC: Internet X.509 Public Key Infrastructure Certificate and Certificate Revocation List (CRL) Profile

relevant ASN.1:

KeyUsage ::= BIT STRING {
           digitalSignature        (0),
           nonRepudiation          (1),
           keyEncipherment         (2),
           dataEncipherment        (3),
           keyAgreement            (4),
           keyCertSign             (5),
           cRLSign                 (6),
           encipherOnly            (7),
           decipherOnly            (8) }

seen in attacks: KCI

best practice: Specify the KeyUsage at creation, verify the keyUsage when encountering the certificate. keyCertSign should be used if the certificate is a CA, keyAgreement should be used if a key exchange can be done with the public key of the certificate.

see more: Extended Key Usage

Validity Dates

explanation: a certificate is only to be valid in a specific interval of time. This interval is specified by the notBefore and notAfter fields.

relevant RFC: Internet X.509 Public Key Infrastructure Certificate and Certificate Revocation List (CRL) Profile

relevant ASN.1:

Validity ::= SEQUENCE {
    notBefore      Time,
    notAfter       Time }

best practice: Reject certificates that have a notBefore date posterior to the current date, or that have a notAfter date anterior to the current date.

Critical extensions

explanation: x509 certificate is an evolving standard, exactly like TLS, through extensions. To preserve backward compatibility, not being able to parse an extension is often considered OK, that is unless the extension is considered critical (important).

relevant RFC: https://tools.ietf.org/html/rfc5280#section-4.2

relevant ASN.1:

Extension  ::=  SEQUENCE  {
    extnID      OBJECT IDENTIFIER,
    critical    BOOLEAN DEFAULT FALSE,
    extnValue   OCTET STRING
                -- contains the DER encoding of an ASN.1 value
                -- corresponding to the extension type identified
                -- by extnID
}

best practice: at creation mark every important extensions as critical. At verification make sure to process every critical extensions. If a critical extension is not recognized, the certificate MUST be rejected.

Hostname Validation

explanation:

Knowing who you're talking to is really important. A x509 certificate is tied to a specific domain/organization/email/... if you don't check who it is tied to, you are prone to impersonation attacks. Because of reasons, these things can be seen in different places in the subject field or in the Subject Alternative Name (SAN) extension. For TLS, things are standardized differently and it will always need to be checked in the latter field.

This is one of the trickier issues in this list as hostname validation is protocol specific (as you can see TLS does things differently) and left to the application. To quote OpenSSL:

One common mistake made by users of OpenSSL is to assume that OpenSSL will validate the hostname in the server's certificate

relevant RFC:

relevant ASN.1:

TBSCertificate  ::=  SEQUENCE  {
    version         [0]  EXPLICIT Version DEFAULT v1,
    serialNumber         CertificateSerialNumber,
    signature            AlgorithmIdentifier,
    issuer               Name,
    validity             Validity,
    subject              Name,

seen in attacks:

best practice: During creation, check for the subject as well as the subject alternative name fields. During verification, check that the leaf certificate matches the domain/person you are talking to. If TLS is the protocol being used, check that in the subject alternative name field, only one level of wildcard is allowed and it must be on the leftmost position (*.domain.com is allowed, sub.*.domain.com is forbidden). Consult RFC 6125 for more information.

see more:

Basic Constraints

explanation: the BasicConstraints extension dictates if a certificate is a CA (can sign others) or not. If it is, it also says how many CAs can follow it before a leaf certificate.

relevant RFC: https://tools.ietf.org/html/rfc5280#section-4.2.1.9

relevant ASN.1:

id-ce-basicConstraints OBJECT IDENTIFIER ::=  { id-ce 19 }

BasicConstraints ::= SEQUENCE {
    cA                      BOOLEAN DEFAULT FALSE,
    pathLenConstraint       INTEGER (0..MAX) OPTIONAL }

seen in attacks: Moxie - New Tricks For Defeating SSL In Practice (2009)

best practice: set this field to the relevant value when creating a certificate. When validating a certificate chain, make sure that the pathLen is valid and the cA field is set to TRUE for each non-leaf certificate.

Name Constraints

explanation: the NameConstraints extension contains a set of limitations for CA certificates, on what kind of certificates can follow them in the chain.

relevant RFC: https://tools.ietf.org/html/rfc5280#section-4.2.1.10

relevant ASN.1:

id-ce-nameConstraints OBJECT IDENTIFIER ::=  { id-ce 30 }

NameConstraints ::= SEQUENCE {
     permittedSubtrees       [0]     GeneralSubtrees OPTIONAL,
     excludedSubtrees        [1]     GeneralSubtrees OPTIONAL }

GeneralSubtrees ::= SEQUENCE SIZE (1..MAX) OF GeneralSubtree

GeneralSubtree ::= SEQUENCE {
     base                    GeneralName,
     minimum         [0]     BaseDistance DEFAULT 0,
     maximum         [1]     BaseDistance OPTIONAL }

BaseDistance ::= INTEGER (0..MAX)

best practice: when creating a CA certificate, be aware of the constraints chained certificates should have and document it in the NameConstraints field. When verifying a CA certificate, verify that each certificate in the certificate chain is valid according to the requirements of upper certificates.

Out of scope

  • Certificate Chain Validation
  • Certificate Revocation

Acknowledgements

Thanks to Chris Palmer, Vincent Lynch and Jeff Jarmoc

comment on this story