IPsec Engine and Key Hierarchy
Overview
IPsec (Internet Protocol Security) is a suite of protocols that provides:
- confidentiality
- integrity
- authentication
- replay protection
at Layer 3 (Network Layer).
Unlike MACsec (Layer 2), IPsec protects IP packets, enabling:
- VPN tunnels
- site‑to‑site secure links
- host‑to‑host encrypted communication
- secure remote access
IPsec requires a dedicated hardware engine because:
- AES‑GCM or ChaCha20‑Poly1305 must run at line‑rate
- thousands of Security Associations (SAs) may be active
- replay protection must be implemented in hardware
- the CPU cannot process packets at high throughput
- the HSM cannot operate in the data path
IPsec in the System Architecture
The IPsec engine lives in the data path, typically inside a NIC or SoC. It receives keys from the HSM and processes packets at line‑rate.
Roles:
- CPU runs IKEv2
- HSM derives and protects keys
- IPsec Engine encrypts/decrypts packets
IPsec Protocol Stack
IPsec consists of two main protocols:
ESP — Encapsulating Security Payload
- encryption + authentication
- supports AES‑GCM and ChaCha20‑Poly1305
- used in both Transport and Tunnel mode
AH — Authentication Header
- authentication only
- rarely used today
Transport Mode
Protects only the IP payload.
Tunnel Mode
Encapsulates the entire IP packet — used for VPNs.
Key Hierarchy in IPsec
IPsec uses a multi‑layer key hierarchy managed by IKEv2.
IKE SA Keys
Derived during IKE_SA_INIT and IKE_AUTH:
- SK_d — used to derive Child SA keys
- SK_ai / SK_ar — authenticate IKE messages
- SK_ei / SK_er — encrypt IKE messages
These keys protect the IKEv2 control channel.
Child SA Keys
Each Child SA (ESP SA) includes:
- Encryption Key (AES‑GCM or ChaCha20)
- Integrity Key (if not using GCM)
- Salt / Nonce
- SPI (Security Parameter Index)
These are the keys used by the IPsec engine to process packets.
IKEv2 and HSM Interaction
The flow is:
The CPU never sees plaintext keys.
IPsec Engine Internals
The IPsec engine implements several hardware functions:
AES‑GCM Pipeline
- authenticated encryption
- deterministic latency
- line‑rate throughput
ChaCha20‑Poly1305 Pipeline
- modern alternative to AES
- efficient for lightweight hardware
SPI Lookup
- identifies the correct SA for each packet
- must be hardware‑accelerated
SA Switching
- enables seamless rekeying
- avoids packet loss
Packet Classification
- parses IP headers
- supports multiple tunnels and flows
Interaction Between HSM and IPsec Engine
The key flow is:
The IPsec engine never returns keys to the HSM or CPU.
Replay Protection
IPsec uses a Sequence Number (SN) per SA.
The IPsec engine implements:
- anti‑replay window
- monotonic SN counters
- hardware discard of out‑of‑window packets
- protection against SN wraparound
Replay protection must be hardware‑based to sustain line‑rate.
Threats and Mitigations
| Threat | Mitigation |
|---|---|
| Key extraction | HSM isolation, wrapped keys |
| Replay attacks | SN + hardware anti‑replay window |
| Forged packets | AES‑GCM / Poly1305 authentication |
| Side‑channel | Masking, balanced logic |
| Fault injection | Tamper sensors, redundant counters |
| CPU compromise | HSM enforces key usage policies |
