Barefoot PKI 2: X.509 PKI

This post is part of a series introduced here.

Certificate PKI. For Simpletons.

In the previous post we saw how successful use of public key encryption (PKE) replaces one difficulty with another. We don't have to worry about sharing secret keys, but we do have to get trustworthy copies correspondents public keys. The solution to the problem — a trustworthy means of distributing keys — is called a Public Key Infrastructure. The X.509 PKI uses the certificates mentioned in the previous post: X.509 is the standard that defines the format and the meaning.

The certificate is the instrument that allows trust. Looking more formally:

1. A certificate is a structured text document which asserts — certifies — that a particular name will use a particular public key for a particular purpose over a particular period.

If you trust the assertion in the certificate , and trust the entity named to manage their private keys securely, you can make the public key magic work for you — encrypt, decrypt or check a signature and all the rest. Unfortunately, by itself, the plain certificate is worthless: anyone could draft such a document. I might generate a key pair; write myself a certificate that said that the public key belonged to Citibank; and start signing banker's drafts! So we need another stage:

2. Certificates are trusted if they are signed by a trusted certification authority (CA). Trust is a belief that the CA a) will follow a defined policy to ensure that it only signs "true" certificates, and b) keep its private key a secret so that no-one else can sign certificates that will validate with the CAs own public key.

Signed here means signed in the PKE sense. The signed document is a) the plain text document, and b) an encrypted hash of the document. If you know the CA's public key, you can decrypt the signature: if it comes out the same as the hash of the plain text, then the encryption must have been done using the CA's private key. If you trust the CA to keep their private key secret you can be sure that the document text offered to you is the same as that signed by the CA. If you trust the CA to only sign true certificates, you can be sure you have the public key you want to use.

(Diversion no. 1: You can see that this works off-line. A lot of the weirdness and toughness in X.509 PKI comes from the requirement for off-line working. It's a child of the eighties and it shows.)

(Diversion no. 2: Discussions of PKI tend to ignore the really important points like the diligence and process at the CA, and the ability of parties to look after their private keys. Consider that the most common use by far of certificates is to certify the identity of a public web server, and consider that that server probably has to be able to start-up, unattended, on an inaccessible hosting site and you can see that privacy of the private key is a bit of an issue.)

So: how do you know the CA's public key? You have to get it some way you can accept. CA keys arrive as so-called self-signed certificates: you can't check them, you just have to take care to get a true one. Browsers come with a long list of CA certificates pre-installed, Windows has a certificate store interface and tools to add CA and other certificates to it.

(Diversion no. 3: Check out the list of CA's your browser has decided that you need to trust. Then research Go Daddy's signing policy. That's a lot of why I think PKI doesn't work — you can't actually trust them to the level you want. Remember, you have no legal agreement with any of these CAs: You won't be able to sue e.g. Belgacom (who?) if you get robbed by a merchant abusing one of their certificates.)

It's a good deal more complicated than this. Certificates have limited validity and limited capability. Very often, intermediate CAs — CAs whose public key is signed by another, "higher" CA — do the actual signing, and you have to trust them too.

X.509 Certificates for Simpletons

So we can use certificates to build a PKI. Obviously these certificates need to be in a standard format and the universal format is X.509, an ITU standard which attempts to define a PKI. X.509 is old — pre-Internet — and as with so many international standards, it's oriented to a world which never happened. For our purposes, X.509 defines data formats for key parts of the certification process, most importantly the certificate itself. But we have other standards as well, notably Public Key Certificate Standards (PKCS) from RSA, which define the format of .Pnn files like the .p12 file which is crucial to Windows.

What is a certificate on the disk? It's a data-structure, unambiguously encoding all the values in such a robust way that the same encryption processes will yield the same result. The requirement for repeatable, checkable, bit-for-bit identicality, despite byte-endian reversals, network transmission, packing and unpacking, and fixed line lengths means that plain text won't do.

X.509 defines the structures using ASN.1. ASN.1 is a data-structure definition language with primitives like octet, integer, sequence etc. It doesn't define formats, but it does define content and order. For example, a simple certificate (not X.509) might be defined as a sequence consisting of a serial number which is an integer, a common name which is a character string, a public key which is .... That ties the content down very precisely, but it doesn't for example say whether the serial number is going to be a 128-bit GUID or a 32-bit long int. So X.509 defines the ASN.1 code for a number of data structures in the certificate process, including the certificate itself.

So those are the formats. Actually representing the content of a data structure is a job for an encoding method. There are many defined encodings for ASN.1, but X.509 uses just one, the DER which creates a binary octet stream. So an actual certificate, or a request, or whatever is a DER string in a file. In Windows, this carries the .CER extension. (Watch out: there is a CER encoding method, and CER is a superset of DER; nonetheless, .CER files contain .DER sequences.)

The DER format with the .CER extension is usual in Windows, but Unix goes one stage further. An early and now obselete use for X.509-style certificates was called Privacy Enhanced Mail (PEM), and it had to pass certificates over the old seven-bit, fixed-line-length mailers common at the time. The solution was the old favourite (really old — it goes back to uucp) base64 encoding, which encodes octet data in a set of 64 7-bit characters, with fixed line length. PEM is gone now, but .PEM files are still widely used — they're the OpenSSH default, because text is always convenient, and you can mix text and data without confusion. A .PEM file is a base64 representation of a DER octet sequence.

So that's the format, and it's pretty obvious that the public key, the name and the CA signature are going to be included in the certificate details. But there's more: the applicability period and the usage types.

The period just means something like : "I will use this key throughout 2006". NB that the validity of the certificate is not affected by the expiry of the applicability period. The key and the certificate may still be required long after its expiry, or the expiry of the key used to sign it: it may be needed to check a signature made in that year, or decrypt a message sent then. The validity refers to intended use rather than actual integrity of the certificate.

And finally, the usage types limit what the certificate can be used for. It may be an encryption key. It may be a signing key. It may authenticate the name of the entity for SSL challenge-response. These purposes are divided up by X.509 into capabilities, V3 capabilities and extended capabilities, with a detailed, extensible underlying structure (called OIDs ). However for all practical purposes they can be regarded as a set of check marks saying what the key can do. One capability to watch out for is "CA": the capability to sign certificates which will inherit the trust of the root certificate.

Other X.509 and PKI Data

The certificate is just one item. To ensure that certificates do what we want, other files and formats are defined by the basic workflow around certificate issue. The basic sequence of operations is linear:

• It starts with generation of a key pair. This is a standard cryptographic function — nothing to do with X.509 or the the CA — the user should do it locally to reduce the risk of exposing the private key. Not all applications can generate a key pair and so it's a standard CA function to do so, with the private key being returned (password encrypted) alongside the certificate in a PKCS12 (.P12) file.
• The application has to give something to the CA to sign. That is the job of the certificate request. PKI users which can generate a key can also generate a certificate request, otherwise the CA has to do it. The request is defined in X.509 — a DER-encoded ASN.1 data structure which contains all the user-specific content for the certificate, including the name, requested validity period, the "capabilities" requested and the public key to be used. It's a bit like an unsigned certificate. It's not confidential because it doesn't contain the private key, but CRs can be passworded to stop casual snooping. OpenSSL can generate CRs from a public key, by default in PEM format. Some public CAs, outside our current scope insist on DER requests.
• When a CA sees a request, it has to decide whether to sign it. Some tests are simple: few CAs will routinely sign a request that requests CA capability. Others are more complex: is the name requested legitimate? and does the period exceed our policy? Where theses choices are mechanical they are encoded into the setup of the CA software: The OpenSSL CA functions get this policy from the openssl.conf file.
• Signing produces a certificate. For some apps, that's sufficient. Others may just need the format translated (from PEM to DER, say). But apps that didn't generate their own private key will need the key pair back too, and, for Windows apps, this is the job of the .P12 file. The .P12 file is a DER stream, not defined by X.509 which contains a certificate and the corresponding (encrypted) private key. This file will install into the Windows certificate store with a double click.

Simple And that is really all there is to it. I've left out the the Certificate Revocation mechanism. All I'm going to say about that is that if you have a system which depends on X.509 certificate revocation to communicate withdrawal or ceasing of access, you have already failed.

Computer Investigation is not about Images of Children. Mostly.

This is just a rant really. I can't justify what I'm going to say, but it's been growing on me for a long time, and some publicity material at Infosec tipped me over. (You'll have to forgive me some circumlocutions, as I don't want to skew Adsense towards the P-word.)

As far as I can see, the whole field of computer forensics is obsessed with sexual images of children.

Is that fair? The material exists, and is just as vile as it seems, or worse. It fuels hideous organised violence and abuse. And the investigators are, after all, sincerely trying to shut down the obnoxious trade. So what can I object to?

I object to the way the topic is used.

• Used to stifle argument: "We must accept any invasion of privacy, because otherwise an abuser might not be punished (which is definitely not a matter of police convenience -- oh no.)"
• Used to garner prestige: "My work helps to protect society from this filth (and is definitely not one of the duller branches of system engineering or forensic accountancy.)"
• Used casually -- it always seems to be the first example picked.

I'm not an experienced investigator. But I've done a few, and everything I've done has been related to the text, dates or addresses of documents or email, or activity from attack tools. I've never so much as once had call to open a JPEG (GIFs as web design elements are another matter...)

Bridget Jones called it mentionitis: when fascinating topics — even un-admitted ones — keep cropping up. That was fiction but the effect is real, and I think it's what's happening here. So this is the challenge: if you are writing about computer forensics, lay off that area.

This stuff exists. It's a danger to my children. We're all against it, and some people have to investigate it. But there's no need to let it take over our un-thought thoughts. It's never necessary. If you want to talk about file types and signatures, take examples from Excel accounts disguised as DLLs. If you want to talk about image content, talk about copyright violation, or even voyeurs. If you want to talk about the rewards of the job, talk about the email bully forced to apologise. You'll feel better for it.

The Show

Infosec at Olympia this week and a fine show. The firms are still sorting out who gets a separate stand after all the mergers over the year.

The best novelty for me was Lockdown Networks with a box that watches for "port-up" events on your switches and makes VLAN choices based on what's plugged in. Printer? Put it on the printer LAN. Computer, no agent? Put it on the Contractor Convenience LAN. Looks like a real labour saver.

Still a very fragmented world, though. The nearest approach to a theme is that all the vulnerability and event scanners are putting workflow/annotation/presentation tools into their products. It's only after you buy these things that you discover that raw automatic events are unusable, and this is the vendors' first cut at making them useful.

More Colours

In the pre-dawn twilight, looking south west across saturated acid yellow of fresh flowering mustard rape into the darkest steely clouds of an approaching storm.

First rain all month.

Blind for Forty Years

I never noticed this before: The flowers on an oak tree are pretty bundles of loose catkins about an inch long. I checked with Mrs. U: she hadn't noticed either. But all the oaks we've checked are covered with them.

Barefoot PKI 1: Asymmetric Encryption

This post is the first is a series. I want to create a "barefoot PKI" to record my own experience and share the lessons. I found I needed to know more about PKI when I saw a sudden flurry of certificate requirements. Outlook users wanting to engage in S/MIME email. The MS Ops Manager implementation authenticating non-domain machines with certificates. Sensitive machines needing certificates to communicate over IPSEC. HTTPS-delivered applications needing server certificates.....

In the past I'd said "We don't do certificates." A good response, if you're a simpleton, because this is tricky stuff -- not conceptually difficult, but hard to keep straight and get all the bits lined up. But it's not good enough any more. So this is the Security Stories guide to PKI, for simpletons. By an authentic, accredited, self-taught simpleton.

Even for simpletons though, this is bigger than a single post. So I'm going to break it up.

1. Absolute minimum concepts for working with asymmetric encryption (Public/Private Key encryption), and some links to get more. (This post.)
2. The X.509 Public Key infrastructure
3. Simple PKI operations.
4. Tools and techniques and implementation with OpenSSL

I'm going to use OpenSSL for the practical stuff. This is not really a political statement: because a) It's what I actually used when I realised I was in trouble, b) it exposes a little more of the gory details, particularly in terms of getting things working with Microsoft systems, and c) what swung it for me was that it's an easy install -- I didn't feel that I was making ill-informed choices that would bind the firm into the future. I installed on Linux, but I'll be talking in terms of a Windows setup.

Asymmetric Encryption for Simpletons

Encryption used to be easy, and symmetric. Alice encrypted her plaintext message with a secret key (which is really just a string of bits) using a (hopefully) robust encryption algorithm. She sent the resulting unintelligible ciphertext to Bob. Bob knows the secret and can run the encryption in reverse to yield the plain text. This is the way it's always been -- the history of cryptography is mostly the history of better and clearer understanding of symmetric encryption.

To benefit from symmetric encryption Alice and Bob need to

1. Agree the encryption to be used. That's easy -- it's not a secret. Modern encryption doesn't rely on obscurity of the algorithm.
2. Keep the secret key secret, to prevent anyone else reading the message
3. But nonetheless share it between the two of them. That's difficult -- they need to have a means of communication which is secure so that the key won't leak. The historical solution has been a key of the day or the hour, in a code book. You don't need me to tell you how horrible that is, especially in a situation where Alice and Bob will never meet, or Alice is dealing with multiple mutually untrusting Bobs.

Public Key — asymmetric — encryption seems to have arrived in a bit of a rush between 1975 and 1980. There's still an encryption algorithm which uses a key. But there's no reverse process for decryption -- instead we use a key pair. The pair are chosen to be related by a property which at first sight seems magical: If you encrypt with the first key to get a ciphertext, you must encrypt again, using the same encryption algorithm with the second key, to get back to the original plaintext. And the same is true if you use the keys in the other order (though the intervening ciphertext would be different). It is spooky, but it does work and the smart types who fret about this sort of thing have not been able to find any conceptual weaknesses.

This magic offers a solution to the key sharing problem. Alice and Bob both create independent — different — key pairs. Each designates one of their keys as "public" and one as "private". The private key is kept secret — it never has to be shared with anyone. And the public key is actually published, to the whole world if necessary. To send a message to Bob, Alice encrypts it with his public key, knowing that the only person who can read it will be Bob, because the only way to decrypt needs the private key which only Bob knows.

It also offers some other wins:

• Alice can sign a file by encrypting it with her private key. Anyone can check that Alice's private key was used, by decrypting it with her public key and checking that it comes out right. Since Alice's private key must have been used, only Alice can have signed it.
• Once it's encrypted it can't be altered, or it won't decrypt right, so its authenticity is guaranteed — the text decrypted is definitely the text that was encrypted.

(Diversion: PK algorithms are relatively slow and require long keys of one or two thousand bits. Instead of using them directly:

• For encryption, the only message encrypted in the PK system is a randomly chosen key for the ordinary symmetric encryption which will protect the rest of the message — typically just 128 or 256 bits to protect a message that that may be megabytes long. So symmetric encryption is as important as it ever was.
• To avoid encrypting the entire document, a hash algorithm is used. A hash is a fixed-length digest of a message. It can't be used to recreate the message, but a change to the message is supposed to be very unlikely to produce the same hash. By convention therefore, Alice signs a hash. This is a little worrying as it appears the this promise is not entirely true in the case of the MD5 hash algorithm which is very widely used.

But the logic works much the same if you think of the PK crypto being used directly. The hash or the key exchange is a convenience only.)

To be secure, the public key must give no clue to the private key. As an example (there are other techniques) RSA encryption uses a key pair which are the (only) two prime factors of a very large number. Because large numbers are hard to factorise, the public key gives no hint of the private.

There are still a couple of problems: The first is practical: private keys have to be kept secret, or all the guarantees fail. The second feels a bit like we've gone round in a circle: we have to have a trustworthy means of finding out the public keys that their counter-parties will be using. It has to be trustworthy because a malicious person can read or spoof traffic if they can deceive either end about the right public key to use. The reason we haven't returned to the secret key distribution problem is simply that it doesn't have to be secret — just trusted.

Mechanisms set up to distribute and validate public keys are called Public Key Infrastructure (PKI). Certificates are the tool used to make it trustworthy. And X.509 is the standard for certificate formats. And that is quite enough for today.

Triple A

We have Authentication, Authorisation and Accounting. And the least of these is Accounting, because no-one bothers about computer time these days. But its presence in the triad reminds us that username and password authentication dates back to a time when the only reason to log on was to select the account to be billed for your FORTRAN session on the teletype.

Using a password to prove identity is a later loading of this basic idea. Which is why it doesn't work.

Log Lust Blog

Long train ride today, for a futile meeting.
Looking out of the window, I see it's not just in the south east that there have been many trees felled all along the track to control leaf fall.
Left behind: piles of logs, entire tree trunks. It's obvious they'll just be left. And I am consumed by lust to possess them.

Bedtime

As it's the holidays, the kiddies would stay on the PCs all night. To get them into bed, stern measures are needed:

First you need to set the accounts they are using (you're not letting them be admins, are you?) to have fixed logon hours. You can't do this through the GUI on XP Home, so you need a batch file of commands like this one to set times when it's possible to log on. Call it accounts.cmd -- you'll need to re-run it when anything changes:

net user mmadson /times:sunday,08:00:-21:00;monday-saturday,09:00-21:00;saturday,08:00-21:00

The /passwordreq:no directive can be useful here too.

Unfortunately, Windows won't enforce that logoff. (A domain would, but Windows itself will not.) So the second step is to force it. There would be any number of ways to deal with this, but I chose the ugliest: run the the Sysinternals psshutdown command at bedtime. I chose to run it from a command file so that I could get a log. Put this text into enforcer.cmd, make the obvious modifications and set it up as a Windows scheduled task. (In the control panel, under Performance and Maintenance.)

@echo off
echo "-start-" >>d:\at\log.txt
date /t >>d:\at\log.txt
time /t >>d:\at\log.txt
d:\at\psshutdown -o -f  >>d:\at\log.txt 2>&1
echo "-end-" >>d:\at\log.txt

I set it to run twenty minutes past the last log on time. Hey presto! Instant rage from the younger generation.

The obvious improvement is to only log off console sessions which are members of the kiddiewinks group. It's really annoying when it logs ME off! I think I could script that up, but I'm too idle.

Why is X.509 so grim?

I've been intending to assemble some notes about the way I've been using an OpenSSL install to knock off some certification requirements. I was going to start with a round-up of the ways that X.509 was needlessly confusing, redundant and bizarre. And I was going to say that large-scale, public, multi-party PKIs are wrong-headed and dangerous.

There's still room for the OpenSSL cookbook -- I'll do that. For such a long-lived product, the documentation just doesn't say what a typical newbie user will want to know. And there's room for me to come clean (if no-one else will) and say that the worst initial obstacle was that I got confused with OpenSSH!

But the rest has been done a long time ago. I wish I'd found it earlier. It's sufficient to say:

• X.509 PKI solves the wrong problem. It's not you. It really is too hard to get right.
• The semantics of the trust and reliance which X.509 attempts to create between unrelated parties are much more tricky than they look, do not address vital issues like counterparties' management of private keys and do not correspond with the ordinary requirements of law and regulation
• X.509 PKI only makes sense between parties who have a pre-existing legal structure (perhaps they're part of the same organisation), and have a means to deal with cancelling authorisation that does not depend on certificate revocation

Why? Witness Peter Gutman's PKI Introduction. And while you're at it, check out his other stuff too. I found the Encryption and Security Tutorial was particularly helpful.

New URL

The new URL makes much more sense. I doubt I've inconvenienced anyone, but if I have, I'm sorry, and thank you for finding your way here.

Obligatory Tribute to Homer Simpson

Just back from three days in a caravan on the Isle of Grain. Literally three days -- I slept at home, commuting with Comedy Dave and the dog, neither of whom were really happy with the caravan concept.

The spookily-named Allhallows Camp is great for bike-mad boys, even if they are a bit iffy about brakes. And the more mad son is very iffy on the brakes -- or at least that's a much more palatable theory than accepting that he's played too much "The Simpsons Hit and Run". Of course, when he runs into an occupied, parked van, and then sidesteps the inevitable argument with the Homer collision soundbite from the game: "Ow! My neck!" you have to wonder.