david wong

Hey! I'm David, a security engineer at the Blockchain team of Facebook, previously a security consultant for the Cryptography Services of NCC Group. I'm also the author of the Real World Cryptography book. This is my blog about cryptography and security and other related topics that I find interesting.

Why I’m Writing A Book On Cryptography posted July 2020

I’ve now been writing a book on applied cryptography for a year and a half. I’m nearing the end of my journey, as I have one last ambitious chapter left to write: next-generation cryptography (a chapter that I’ll use to talk about cryptography that will become more and more practical: post-quantum cryptography, homomorphic encryption, multi-party computation, and zk-SNARKs).

I’ve been asked multiple times why write a new book about cryptography? and why should I read your book?. To answer this, you have to understand when it all started…

Diagrams are everything

Today if you want to learn about almost anything, you just google it. Yet, for cryptography, and depending on what you're looking for, resources can be quite lacking.

It all started a long time ago. For a class, I had to implement a differential power analysis attack, a breakthrough in cryptanalysis as it was the first side-channel attack to be published. A differential power analysis uses the power consumption of a device during an encryption to leak its private key. At the time, I realized that great papers could convey great ideas with very little emphasis on understanding. I remember banging my head against the wall trying to figure out what the author of the white paper was trying to say. Worse, I couldn’t find a good resource that explained the paper. So I banged my head a bit more, and finally I got it. And then I thought I would help others. So I drew some diagrams, animated them, and recorded myself going over them. That was my first screencast.

This first step in education was enough to make me want to do more. I started making more of these videos, and started writing more articles about cryptography on this blog (today totaling more than 500 articles).

we want to know

I realized early that diagrams were extremely helpful to understand complicated concepts, and that strangely most resources in the field shied away from them.

For example, anyone in cryptography who thinks about AES-CBC would immediately think about the following wikipedia diagram:

aes cbc

So here I was, trying to explain everything I learned, and thinking hard about what sorts of simple diagrams could easily convey these complex ideas. That’s when I started thinking about a book, years and years before Manning Publications would reach out to me with a book deal.

The applied cryptographer curriculum


I hadn’t started cryptography due to a long-life passion. I had finished a bachelor in theoretical mathematics and didn’t know what was next for me. I had also been programming my whole life, and I wanted to reconcile the two. Naturally, I got curious about cryptography, which seemed to have the best of both world, and started reading the different books at my disposal. I quickly discovered my life's calling.

Some things were annoying me though. In particular, the long introductions that would start with history. I was only interested in the technicalities, and always had been. I swore to myself, if I ever wrote a book about cryptography, I would not write a single line on Vigenère ciphers, Caesar ciphers, and others.

And so after applying to the masters of Cryptography at the university of Bordeaux, and obtaining a degree in the subject, I thought I was ready for the world. Little did I know. What I thought was a very applied degree actually lacked a lot on the real world protocols I was about to attack. I had spent a lot of time learning about the mathematics of elliptic curves, but nothing about how they were used in cryptographic algorithms. I had learned about LFSRs, and ElGamal, and DES, and a series of other cryptographic primitives that I would never see again.

When I started working in the industry at Matasano, which then became NCC Group, my first gig was to audit OpenSSL (the most popular TLS implementation). Oh boy, did it hurt my brain. I remember coming back home every day with a strong headache. What a clusterfuck of a library. I had no idea at the time that I would years later become a co-author of TLS 1.3.

sign

But at that point I was already thinking: this is what I should have learned in school. The knowledge I’m getting now is what would have been useful to prepare me for the real world. After all, I was now a security practitioner specialized in cryptography. I was reviewing real-world cryptographic applications. I was doing the job that one would wish they had after finishing a cryptography degree. I implemented, verified, used, and advised on what cryptographic algorithms to use.

This is the reason I’m the first reader of the book I’m writing. This is what I would have written to my past self in order to prepare me for the real world.

The use of cryptography is where most of the bugs are

My consulting job led me to audit many real world cryptographic applications like the OpenSSL, the encrypted backup system of Google, the TLS 1.3 implementation of Cloudflare, the certificate authority protocol of Let’s Encrypt, the sapling protocol of Zcash, the threshold proxy re-encryption scheme of NuCypher and dozens and dozens of other real-world cryptographic applications that I unfortunately cannot mention publicly.

Early in my job, I was tasked to audit the custom protocol a big corporation (that I can’t name) had written to encrypt their communications. It turns out that, they were signing everything but the ephemeral keys, which completely broke the whole protocol (as one could have easily replaced the ephemeral keys). A rookie mistake from anyone with some experience with secure transport protocols, but something that was missed by people who thought they were experienced enough to roll their own crypto. I remember explaining the vulnerability at the end of the engagement, and a room full of engineers turning silent for a good 30 seconds.

This story repeated itself many times during my career. There was this time where while auditing a cryptocurrency for another client, I found a way to forge transactions from already existing ones (due to some ambiguity of what was being signed). Looking at TLS implementations for another client, I found some subtle ways to break an RSA implementation, which in turned transformed into a white paper (with one of the inventor of RSA) leading to a number of Common Vulnerabilities and Exposures (CVEs) reported to a dozen of open source projects. More recently, reading about Matrix as part of writing my book, I realized that their authentication protocol was completely broken, leading to a complete break of their end-to-end encryption.

comic

There’s so many details that can unfortunately collapse under you, when making use of cryptography. At that point, I knew I had to write something about it. This is why my book contains many of these anecdotes.

As part of the job, I would review cryptography libraries and applications in a multitude of programming languages. I discovered bugs (for example CVE-2016-3959 in Golang’s standard library), I researched ways that libraries could fool you into misusing them (for example see my paper How to Backdoor Diffie-Hellman), and I advised on what libraries to use. Developers never knew what library to use, and I always found the answer to be tricky.

I went on to invent the disco protocol, and wrote a fully-featured cryptographic library in less than 1,000 lines of code in several languages. Disco only relied on two cryptographic primitives: the permutation of SHA-3 and curve25519. Yes, from only these two things in 1,000 lines of code a developer could do any type of authenticated key exchange, signatures, encryption, MACs, hashing, key derivation, etc. This gave me a unique perspective as to what a good cryptographic library was supposed to be.

I wanted my book to contain these kind of practical insights. So naturally, the different chapters contain examples on how to do crypto in different programming languages, using well-respected cryptographic libraries.

A need for a new book?

As I was giving one of my annual cryptography training at Black Hat, one student came to me and asked if I could recommend a good book or online course on cryptography. I remember advising the student to read the book from Boneh & Shoup and Cryptography I from Boneh on Coursera.

The student told me “Ah, I tried, it’s too theoretical!”. This answer stayed with me. I disagreed at first, but slowly realized that they were right. Most of these resources were pretty heavy in math, and most developers interacting with cryptography don’t want to deal with math. 
What else was there for them? The other two somewhat respected resources at the time were Applied Cryptography and Cryptography Engineering (both from Schneier). But these books were starting to be quite outdated. Applied Cryptography spent 4 chapters on block ciphers, with a whole chapter on cipher modes of operation but none on authenticated encryption. Cryptography Engineering had a single mention of elliptic curve cryptography (in a footnote).

On the other hand, many of my videos or blog posts were becoming good primary references for some cryptographic concepts.

I knew I could do something special.

Gradually, many of my students started becoming interested in cryptocurrencies, asking more and more questions on the subject. At the same time, I started to audit more and more cryptocurrency applications. I finally moved to a job at Facebook to work on Libra. Cryptocurrency was now one of the hottest field to work on, mixing a multitude of extremely interesting cryptographic primitives that so far had seen no real-world use case (zero knowledge proofs, aggregated signatures, threshold cryptography, multi-party computations, consensus protocols, cryptographic accumulators, verifiable random functions, verifiable delay functions, ... the list goes on)

libra

I was now in a unique position.

I knew I could write something that would tell students, developers, consultants, security engineers, and others, what modern applied cryptography was all about.

book

This was going to be a book with very little formulas, but filled with many diagrams.

This was going to be a book with little history, but filled with modern stories about cryptographic failures that I had witnessed for real.

This was going to be a book with little about legacy algorithms, but filled with cryptography that I've personally seen being used at scale: TLS, the Noise protocol framework, Wireguard, the Signal protocol, cryptocurrencies, HSMs, threshold cryptography, and so on.

This was going to be a book with little theoretical cryptography, but filled with what could become relevant: password-authentication key exchanges, zero-knowledge proofs, post-quantum cryptography, and so on.

Real-World Cryptography

When Manning Publications reached out to me in 2018, asking if I wanted to write a book on cryptography for them, I already knew the answer. I already knew what I wanted to write about. I had just been waiting for someone to give me the opportunity, the excuse to spend my time writting the book I had in mind.

Coincidentally, they had a series of "real-world" book, and so naturally I suggested that my book extend it.

book

Real-World Cryptography is available for free in early-access.

I want this to be the best book on applied cryptography. For this reason, if you have any feedback to give, please send it my way :)

The book should be ready in print for the end of the year.

Well done! You've reached the end of my post. Now you can leave me a comment or read something else.

Comments

Bill

"This is what I would have wrote to my past self in order to prepare me for the real world"

wrote -> written

Bill

Sorry. Feel free to delete my comments, just pointing out some typos

" but something that was missed by people who thought they were experimented enough to roll their own crypto"

experimented -> experienced

david

Hey Bill,

No worries, I appreciate! Thanks for pointing these out! You can also do it via the contact page if you're scared of spamming the comment section :)

Ilkin.

Really need this kind of book from Real world aspect. Also I am interested in, is it necessary to have prior knowledge to read the book, like math or crypto 101. Because of i am newbie self-learner in security field and have no background related with math or CS. I am really appreciate your hard work. Thanks so much.

David Jao

"This was going to be a book with little history, but filled with modern stories about cryptographic failures that I had witnessed for real."

Funny, there's a word that describes stories about past cryptographic failures perfectly. It's called "history."

History is important in cryptography because we often rely on history to tell apart the good from the junk. History is of utmost importance for those who study assumptions. Sure, you might have a proof of security, but unless you have a one-time pad, that proof depends on some assumptions, and those assumptions don't have proofs. The test of time is often all we have. How do we know factoring is hard? Because history says it is hard. (Or, depending on your point of view, factoring is easy, but building the necessary quantum computer is hard.)

The above is especially true for unstated assumptions. Side channel attacks work because they unveil a hidden assumption: the assumption that attackers can't access your internal state.

I'm not saying you need to cover ancient stuff like Caesar ciphers, but modern history is absolutely important for a cryptographer. The stories of our past can guide our future.

Alex

That looks wonderful, we'll definitely be reading your book. I wonder if you have an opinion on "Serious cryptography" book by Aumasson?

Glen Peterson

Nice article. Looking forward to the book.

Make sure to talk about legal requirements brewing in Australia and the U.S. to add backdoors for law enforcement and how such a law can be worded so as not to ruin encryption in those countries. For instance, if law enforcement wanted to look for child porn or catphishing at my day job, they could theoretically do so in the application logs, or in the database. There would still be privacy concerns, but it wouldn't compromise all legal uses of encryption the way requiring backdoored algorithms would. Even in that case, how would we know a request to look at our logs was legitimate?

My senator is the author of the EARN-IT act. Eye-roll!

Aim

Great work!
Can the diagrams and figures be used license free to show to students in a class?

Aim

Great work!
Can the diagrams and figures be used license free to show to students in a class?

Aim

Great work!
Can the diagrams and figures be used license free to show to students in a class?

Dennis

I'm going to buy this book just for the awesome illustrations. It looks absolutely amazing!!

Theo

Saw this link on Hackernews. Thought: yes, why would one write a book on crypto. There are many good books.
After reading your posting I thought: brilliant marketing!
Look forward to the book.

i want to understand quantum cryptography

hello,

will this become obsolete soon as quantum cryptography release?
danken

CryptoBook

Might be too late but it would be great for the book to include a section about encryption for data at rest and not just for data exchange. i.e. what are the modern techniques for full drive encryption, database at rest encryption, etc. In these cases key rotation is much more difficult.

david

> Can the diagrams and figures be used license free to show to students in a class?

I would hope so, probably something to ask the book editor (but you have my blessing in any case)

david

> Might be too late but it would be great for the book to include a section about encryption for data at rest

this is covered in the chapter on hardware cryptography

CryptoBook

> this is covered in the chapter on hardware cryptography

glad to hear it' covered.
"hardware" seems like an odd place to discuss encrypted data storage. I'm thinking about systems like LUKS, Keepass format, VeraCrypt and database level data at rest (like the work done by https://dgraph.io/blog/post/encryption-at-rest-dgraph-badger/ or https://github.com/cockroachdb/cockroach/blob/master/docs/RFCS/20171220_encryption_at_rest.md) or maybe even key/token management done in systems like Hashicorp Vault (https://github.com/hashicorp/vault).
In other words examples and background for the use case of "I have some data I need to store securely in my app: how should I build something or evaluate a solution"

Chris

Hi,

I'm a software engineer who completely agrees with you. After playing with libsodium and reading the associated paper. I cannot see where the dragons are. I know they are in there, but the paper and the documentation says I must do thing x but does not give me enough information on what happens when things break.

Good luck with the book writing.

Øystein

Hi, this looks really good and I want to buy a copy of the book

I don't understand.. is the book finished now so I can order it, or do I have to wait for a couple of months ? so that I don't buy half of the book

Thanks

Marco S. Zuppone

Wow! Now I want it :-)
It seems really cool.
A lot of books and courses about this topic are too much theoretical.
The book that I liked most so far about this topic is "Serious Cryptography.
A Practical Introduction to Modern Encryption" by Jean-Philippe Aumasson.
I'm looking forward to buy and read your book as well!
Kidn regards,
Marco - Stocktrader
https://msz.eu && https://msz.training

Soatok

A few years ago, we were stuck between recommending Applied Cryptography or Cryptography Engineering. Now we're going to be stuck between Serious Cryptography and Real-World Cryptography!

:3

I look forward to reading it in full. Skimming through the introduction available online was a treat.


Leave a comment