david wong

Hey! I'm David, cofounder of zkSecurity and the author of the Real-World Cryptography book. I was previously a crypto architect at O(1) Labs (working on the Mina cryptocurrency), before that I was the security lead for Diem (formerly Libra) at Novi (Facebook), and a security consultant for the Cryptography Services of NCC Group. This is my blog about cryptography and security and other related topics that I find interesting.

ASN.1 vs DER vs PEM vs x509 vs PKCS#7 vs .... posted April 2015

I was really confused about all those acronyms when I started digging into OpenSSL and RFCs. So here's a no bullshit quick intro to them.

PKCS#7

Or Public-Key Crypto Standard number 7. It's just a guideline, set of rules, on how to send messages, sign messages, etc... There are a bunch of PKCS that tells you exactly how to do stuff using crypto. PKCS#7 is the one who tells you how to sign and encrypt messages using certificates. If you ever see "pkcs#7 padding", it just refers to the padding explained in pkcs#7.

X509

In a lot of things in the world (I'm being very vague), we use certificates. For example each person can have a certificate, and each person's certificate can be signed by the government certificate. So if you want to verify that this person is really the person he pretends to be, you can check his certificate and check if the government signature on his certificate is valid.

TLS use x509 certificates to authenticate servers. If you go on https://www.facebook.com, you will first check their certificate, see who signed it, checked the signer's certificate, and on and on until you end up with a certificate you can trust. And then! And only then, you will encrypt your session.

So x509 certificates are just objects with the name of the server, the name of who signed his certificate, the signature, etc...

Example from wikipedia:

    Certificate
        Version
        Serial Number
        Algorithm ID
        Issuer
        Validity
            Not Before
            Not After
        Subject
        Subject Public Key Info
            Public Key Algorithm
            Subject Public Key
        Issuer Unique Identifier (optional)
        Subject Unique Identifier (optional)
        Extensions (optional)
            ...
    Certificate Signature Algorithm
    Certificate Signature

ASN.1

So, how should we write our certificate in a computer format? There are a billion ways of formating a document and if we don't agree on one then we will never be able to ask a computer to parse a x509 certificate.

That's what ASN.1 is for, it tells you exactly how you should write your object/certificate

DER

ASN.1 defines the abstract syntax of information but does not restrict the way the information is encoded. Various ASN.1 encoding rules provide the transfer syntax (a concrete representation) of the data values whose abstract syntax is described in ASN.1.

Now to encode our ASN.1 object we can use a bunch of different encodings specified in ASN.1, the most common one being used in TLS is DER

DER is a TLV kind of encoding, meaning you first write the Tag (for example, "serial number"), and then the Length of the following value, and then the Value (in our example, the serial number).

DER is also more than that:

DER is intended for situations when a unique encoding is needed, such as in cryptography, and ensures that a data structure that needs to be digitally signed produces a unique serialized representation.

So there is only one way to write a DER document, you can't re-order the elements.

And a made up example for an ASN.1 object:

OPERATION ::= CLASS
{
&operationCode INTEGER UNIQUE,

&InvocationParsType,

&ResponseParsAndResultType,

&ExceptionList ERROR OPTIONAL
}

And its DER encoding:

0110 0111 0010 110...

Base64

Base64 is just a way of writing binary data in a string, so you can pass it to someone on facebook messenger for exemple

From the openssl Wiki:

ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/ 
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
0000000000111111111122222222223333333333444444444455555555556666
0123456789012345678901234567890123456789012345678901234567890123

And if you see any equal sign =, it's for padding.

So if the first 6 bits of your file is '01' in base 10, then you will write that as B in plaintext. See an example if you still have no idea about what I'm talking about.

PEM

A pem file is just two comments (that are very important) and the data in base64 in the middle. For example the pem file of an encrypted private key:

-----BEGIN ENCRYPTED PRIVATE KEY-----
MIIFDjBABgkqhkiG9w0BBQ0wMzAbBgkqhkiG9w0BBQwwDgQIS2qgprFqPxECAggA
MBQGCCqGSIb3DQMHBAgD1kGN4ZslJgSCBMi1xk9jhlPxP3FyaMIUq8QmckXCs3Sa
9g73NQbtqZwI+9X5OhpSg/2ALxlCCjbqvzgSu8gfFZ4yo+Xd8VucZDmDSpzZGDod
X0R+meOaudPTBxoSgCCM51poFgaqt4l6VlTN4FRpj+c/WZeoMM/BVXO+nayuIMyH
blK948UAda/bWVmZjXfY4Tztah0CuqlAldOQBzu8TwE7WDwo5S7lo5u0EXEoqCCq
H0ga/iLNvWYexG7FHLRiq5hTj0g9mUPEbeTXuPtOkTEb/0ckVE2iZH9l7g5edmUZ
GEs=
-----END ENCRYPTED PRIVATE KEY-----

And yes the number of - are important

Well done! You've reached the end of my post. Now you can leave a comment or read something else.

Comments

Dicky

Thanks for such of brilliant explanation of all those acronym. It helps me as newbie to understand what all it about. Btw I have a question. How do I create a certificate file if I have ASN1 DER data? I would like to create pfx file at the end.

david

I think you can do something like `openssl asn1parse -in yourfile -inform der -out newfile -outform pem` to transform your der file into something else. I don't know for pfx, you might have some special too that could do that.

cris

I would say that PKCS does not stand for Public-Key CryptoSystem but Public Key Cryptography Standards...

david

yes! I fixed it, thanks Cris :)

cris

thanks David!! i appreciate it, you hot boy! :)

cris

I would say that PKCS does not stand for Public-Key CryptoSystem but Public Key Cryptography Standards...

cris

I would say that PKCS does not stand for Public-Key CryptoSystem but Public Key Cryptography Standards...

david

try hard

George

Great post David! Just to add to the definition of PKCS#7: it's also widely used as a container for multiple related certs, say in a certificate signing chain, where your end entity cert (the one you were issued) is packaged with all the intermediate and root CA (Certificate Authority, DigiCert/Verisign, etc.) certs that signed/authenticated your cert.

George

Great post David! Just to add to the definition of PKCS#7: it's also widely used as a container for multiple related certs, say in a certificate signing chain, where your end entity cert (the one you were issued) is packaged with all the intermediate and root CA (Certificate Authority, DigiCert/Verisign, etc.) certs that signed/authenticated your cert.

Marco

Thank you
Very usefull

Stan

Super useful post for me who just started diving into all this. Right now im signing using openssl on linux but the signature is verified in java and i keep losing 1-8 bytes randomly in a java signature length error e.g. like 253 instead of 256. Ever seen anything like that?

Bobby Tables

Great stuff, thank you ser

amil

Really good explanation about concepts!
May I translate your Article into Korean?
It's gonna help a lot of Korean people who want to understand the above concepts!
Thanks

david

of course feel free!

leave a comment...