Understanding encryption fundamentals is the first step

Encryption is a fascinating science that blends mathematics, intrigue, and in our time, computers. Although encryption is about keeping information secret, there's nothing secret about the encryption process. In fact, the more public scrutiny an encryption algorithm is subjected to, the more secure it is regarded. If encryption is executed properly, the only thing you need to keep secret is the encryption key. Let me introduce you to encryption and show you the important difference between symmetric key encryption and public/private key encryption. We'll finish up with a brief discussion of common encryption protocols in use today and how various Windows components use encryption.

The Two Types of Encryption

Unencrypted data is referred to as plaintext. During encryption, you run the plaintext through an encryption algorithm along with an encryption key. After encryption, the information is still present, but it's been obscured into what's called ciphertext. You can restore ciphertext to the original plaintext by running it through a decryption algorithm along with a decryption key.

Symmetric key encryption is the type of encryption most people think of first. With symmetric key encryption, you use the same key to encrypt the plaintext into ciphertext and decrypt the ciphertext back to the original plaintext. Symmetric key encryption is fast and effective for keeping large amounts of information secret, but it runs into trouble when you need to share information between different parties. Symmetric key encryption is also known as shared secret encryption because for two different parties to exchange private information, they must both know the encryption key. This requirement means there has to be a secure means for sharing the key between the parties but not letting anyone else discover it. However, even if you come up with a secure way to exchange the key, you can run into other problems, such as what to do when a key gets compromised or when someone who knows the key gets fired. Public/private key encryption can address such problems.

With public/private key encryption, you typically have two keys. You can use either key to encrypt the information, but you must use the other key to decrypt it. To generate the keys, the encryption program finds two very long numbers that possess a mathematical relationship to each other. When the encryption program generates the key pair, it arbitrarily designates one key as private and the other key as public. You can freely share the public key, which allows any party to send a secret message to the key pair's owner. Public/private key encryption is also the foundation of digital signatures. You can use digital signatures to prove that a message, document, or packet was created by the sender and not modified in transit.

To create a digital signature, the sender first computes a hash (i.e., a mathematical digest) of the message. A hash is always the same length (usually a few bytes) whereas the message can be any length, even several gigabytes. Hashes are useful because any change to the message, even one bit, typically results in a different hash. (Given a great number of messages, two messages will eventually result in the same hash, which is called a hash collision, but this event isn't predictable.) After hashing the message, the sender encrypts the hash with the private key. The encrypted hash is the digital signature and gets sent along with the message. The recipient can verify the message's authenticity and integrity by checking the signature. The recipient decrypts the signature with the sender's public key, which yields the hash originally computed by the sender. Next, the recipient computes his or her own hash of the received message. If the two hashes match, the recipient knows the message truly came from the sender and wasn't changed in transit. The signature must have been encrypted with the sender's private key because the signature was successfully decrypted with the public key.

But how does the recipient know the sender's public key in the first place, especially if they've never met? Therein lies the question that illustrates the purpose of a public key infrastructure (PKI). A PKI typically uses certificates and Certification Authorities (CAs) to make it simple to obtain and verify public keys of different parties without any direct contact, much less a secure means of communication among those parties. I'll cover PKI and certificates in a future article.

Encryption Isn't Unbeatable

All encryption can be defeated if you have enough computing power and time. It's just a matter of trying to decrypt the ciphertext with every possible encryption key. That's why key length is so important. The longer the key, the more time and/or computing power it takes to find the right key. Because computers constantly get more powerful, encryption algorithms have to be updated so that encryption technology remains strong enough to make breaking the key impossible or at least impractical. DES encryption was proven to be too weak several years ago when the Electronic Frontier Foundation (EFF) challenged people to break it. That was the final nail in DES's coffin and soon we had the new Advanced Encryption Standard (AES).

Encryption can also be defeated if the software using encryption fails to implement the encryption or hash algorithm correctly. For example, in the 1990s, Windows NT's PPTP (a VPN protocol) was discovered to have several vulnerabilities due to improper implementation of the RC4 algorithm. These vulnerabilities were subsequently patched. More recently, the Wi-Fi Alliance saw its Wired Equivalent Privacy (WEP) standard consigned to the trash heap mainly because it failed to change encryption keys regularly. WEP's downfall led to the adoption of the Wi-Fi Protected Access (WPA) and 802.11i standards.

Encryption in Windows

Windows OSs extensively use both symmetric and public/private key encryption. For example, Windows encrypts the password hashes of user accounts with a symmetric key. That way, if attackers steal a copy of the SAM or Active Directory (AD), they'll have more difficulty deciphering users' passwords. To further secure the Windows system, you can use the Syskey utility to control where the symmetric key is stored. However, be careful with Syskey because you can lock yourself out of the system if you choose to store the key outside the system and later lose that key. Windows has its own PKI that's based on Certificate Services and integrated with AD and Group Policy. The Windows PKI leverages and integrates with many other Windows components as well. It also automates many of the laborious tasks associated with managing a PKI, such as certificate enrollment and renewal and certificate revocation list (CRL) publishing.

Two Windows components that heavily use encryption are IPsec and Encrypting File System (EFS). IPsec uses both symmetric and public/private key encryption to protect information sent over the network. IPsec uses certificates for the initial authentication of the two communicating computers, then uses DES or Triple DES (3DES) for the actual encryption of packets.

EFS also uses both types of encryption to protect files on a hard disk in case the computer or hard disk is stolen. Although public/private key encryption is often more desirable because you don't need to share a secret encryption key, symmetric key encryption is still very important because public/private key encryption is relatively slow. Therefore, to get the best of both worlds, you'll often see information encrypted with a symmetric key, which is then encrypted with the recipient's public key. EFS is such an example. For each encrypted file, EFS generates a symmetric file encryption key (FEK) and encrypts the file in blocks using DES, 3DES, or AES. Then, EFS encrypts the FEK with the public key of the user's EFS certificate and stores the FEK along with the file on the hard disk. When the user later accesses the file, EFS decrypts the FEK with the user's private key. To protect the private keys of user certificates, Windows has a special store in the registry where the private keys and other secrets are encrypted. (Keys can also be stored in the file system.) The keys are protected by a Master key, which is protected by a hash of the user's password. Thus, the user's password is the cornerstone protecting everything else, which is why using strong passwords is so important.

Knowledge Is Key

It pays to understand the fundamentals of encryption, hashing, certificates, and digital signatures so that you can safely use the encryption features in Windows OSs and other applications. This knowledge is key to keeping your secrets safe.