What Is Encryption Strength? How TLS Works, and the Computational Cost of DH, ECC, RSA, and Lattice Cryptography
"It's 256-bit, so it's secure" is only half right. Encryption strength depends not just on key length, but on how algorithm research into the underlying mathematical problems progresses.
How TLS Combines Cryptography
TLS (Transport Layer Security), which underpins HTTPS communication, isn't a single cryptographic algorithm — it's a "cipher suite" that combines multiple cryptographic techniques, each assigned a specific role. Taking a TLS 1.3 handshake as an example, the division of labor looks roughly like this.
| Key exchange | ECDHE (Elliptic Curve Diffie-Hellman, ephemeral). Generates a new key pair for every session, providing Forward Secrecy |
|---|---|
| Signing / authentication | RSA or ECDSA. The server certificate proves the server is actually the owner of that domain |
| Symmetric encryption | AES-GCM or ChaCha20-Poly1305. The actual communication data is encrypted with this fast symmetric cipher |
| Hash function | SHA-256 and similar. Used for key derivation (HKDF) and message authentication |
In other words, TLS takes a hybrid approach: "use public-key cryptography to safely share a key, then use that shared key with a fast symmetric cipher." Libraries like OpenSSL implement this combination of cipher suites and negotiate which algorithms both the server and client support.
The Kinds of Cryptography Used in OpenSSL
The public-key cryptography used for key exchange and signing bases its security on one of roughly three hard mathematical problems.
| DH (Diffie-Hellman) | The discrete logarithm problem (DLP) over a finite field. Relies on the difficulty of finding x from g^x mod p |
|---|---|
| ECC (Elliptic curve cryptography) | The discrete logarithm problem over an elliptic curve (ECDLP). Achieves the same security strength as DH with a much shorter key length |
| RSA | The problem of factoring large composite numbers. Relies on the difficulty of finding p, q from n = p × q |
| Lattice cryptography (post-quantum) | The Shortest Vector Problem (SVP) on a lattice, and Learning With Errors (LWE). Adopted in NIST's post-quantum cryptography standards (ML-KEM/Kyber, ML-DSA/Dilithium) because no efficient algorithm to solve them — even with a quantum computer — is currently known |
The Relationship Between Known Algorithms and Computational Cost
"Longer keys are more secure" is only true within the same algorithm family. In reality, the computational cost of the "best known algorithm" for solving each underlying problem is what actually determines the effective security strength.
| RSA (integer factorization) | The General Number Field Sieve (GNFS) is the best classical algorithm. Its running time is sub-exponential, on the order ofexp((64/9)^(1/3) · (ln n)^(1/3) · (ln ln n)^(2/3)), so it becomes only gradually harder to break as key length increases |
|---|---|
| DH/ECC (discrete logarithm) | Index calculus methods based on GNFS are the best classical algorithms over finite fields, while Pollard's rho method is the best over elliptic curves. Because the rho method is fully exponential (on the order of 2^(n/2)), ECC achieves an equivalent security strength to RSA with a much shorter key length |
| Lattices (SVP/LWE) | The LLL and BKZ algorithms are the best practical algorithms. Difficulty is designed to grow exponentially depending on the parameters chosen |
For example, RSA-3072 and a 256-bit ECC (elliptic curve) key are considered to offer roughly equivalent security strength (128-bit security). This comes down to the difference in computational complexity class between the two attack algorithms: the attack algorithm on the RSA side (GNFS) is sub-exponential, while the attack algorithm on the ECC side (Pollard's rho method) is fully exponential.
Estimates of encryption strength are calculated purely "against the best algorithms known today." If a faster classical algorithm for factoring or the discrete logarithm problem is ever discovered, simply increasing key length may not be enough to keep up. In fact, the biggest reason lattice cryptography was chosen for the NIST standard is the somewhat negative one that "no efficient algorithm for solving SVP/LWE on a quantum computer is currently known" — and there is no guarantee that situation won't change in the future. The security of cryptography rests not on a static number like "key length," but on the dynamic, ongoing competition of how far algorithm research into solving that underlying problem has progressed.
Summary
- TLS is a hybrid construction combining key exchange (e.g. ECDHE), signing (RSA/ECDSA), and symmetric encryption (AES, etc.)
- DH, ECC, and RSA base their security on the discrete logarithm problem, the elliptic curve discrete logarithm problem, and the integer factorization problem, respectively
- ECC achieves equivalent security to RSA with a shorter key length because of the difference in computational complexity class between the attack algorithms
- Encryption strength isn't a fixed value — it's a relative assessment that can change in the future as attack algorithm research advances
Check the Security of Your Communication Path Too
ShieldGuard NET scans your home network for misconfigurations and known vulnerabilities.
See ShieldGuard NET