12. Cipher Suite Registration and Negotiation

This chapter addresses requirement F7: a cipher suite combining the AEAD of Chapter 8 and the hash of Chapter 9 shall be usable in a TLS 1.3 handshake. It specifies what a suite is, what registering one entails, how negotiation then proceeds, and what has to be true on both endpoints for a connection to result.

12.1 What a suite is

A TLS 1.3 cipher suite is a two-octet code point that names a pair: an AEAD algorithm for record protection and a hash for the key schedule and transcript. It carries nothing else. The key exchange and authentication methods that TLS 1.2 suites encoded were moved into extensions, which is why groups and signature algorithms are negotiated independently (§2.1).

A suite description, as the TLS implementation needs it, therefore comprises:

FieldPurpose
Code pointThe two octets on the wire
NameConfiguration and diagnostics
AEAD algorithmFetched for record protection
Handshake hashFetched for transcript and key schedule
Version boundsRestricts the suite to TLS 1.3
Strength metadataSecurity level filtering and ordering

Verified. These are the fields the reference implementation's suite entries carry. The first entry of tls13_ciphers[] pairs the RFC name and code point macros for TLS_AES_128_GCM_SHA256 with SSL_AES128GCM, SSL_HANDSHAKE_MAC_SHA256, version bounds of TLS1_3_VERSION at both ends, and strength values (ssl/s3_lib.c:39–56, tree openssl-3.5.8).

12.2 Registration

Registering a suite means making a description of the above shape available to the negotiation logic, and ensuring the algorithms it names are fetchable in the relevant library context. The second half is the work of Chapters 8 and 9 and is already specified. The first half is the subject of this section.

The design requirements for registration are as follows, independently of the mechanism by which the description reaches the negotiation logic:

  1. The code point must be unique within the deployment and drawn from a registry or the private-use range, on the reasoning given in §10.6. A suite is agreed by number, and two endpoints that attach different meanings to one number will complete a handshake and then fail to communicate — or, worse, communicate under mismatched assumptions.
  2. The named algorithms must resolve in the library context the SSL_CTX uses (constraint C6). A suite naming an algorithm that cannot be fetched is worse than a suite that is absent, because it can be selected and will then fail mid-handshake.
  3. The version bounds must confine it to TLS 1.3, since the suite structure and key schedule of earlier versions differ.
  4. The strength metadata must be accurate, for the security-level reasons given in §11.3.
  5. Ordering must be deliberate. The server selects from the client's offer according to its own preference ordering; a new suite placed above standard suites will be selected whenever a peer supports it, and placed below will be selected only when nothing else matches. Both are defensible; neither should happen by accident.

Reported, with a verified counterpart. The system this paper documents is reported by its author to add such suites through the provider interface. What this study verified independently is the structure of the suite description and the location of the suite table in the reference implementation (§12.1, and §4.4). Whether a given OpenSSL build obtains suite descriptions from a provider, from its built-in table, or from both is a property of that build; a deployment should establish it by inspection of the build in use, and Chapter 19 gives the test that settles the question empirically — openssl ciphers against a build with the provider activated and deactivated.

12.3 Negotiation

Once a suite exists on both endpoints, negotiation is ordinary and is worth stating because the failure modes are diagnosable only if the sequence is understood.

The client sends its supported suites in preference order in ClientHello, together with its groups and signature algorithms. The server intersects the client's list with its own, applies its preference policy, and returns a single selected suite in ServerHello. From that point both sides fix the AEAD and the hash, and the key schedule begins.

Three failure modes follow, and they are distinguishable:

No shared suite
The server finds an empty intersection and sends handshake_failure. The custom suite was offered by one side only — typically the provider is not activated on the other, or is activated in a different library context.
Suite selected, algorithm unavailable
The handshake proceeds past ServerHello and fails when an algorithm cannot be fetched. The suite description was present but the implementation was not — requirement 2 of §12.2 violated.
Suite selected, handshake fails at Finished
Both sides agreed and computed, but computed differently. This points at the hash: a transcript divergence, a duplication bug (§9.2), or a key-schedule width mismatch (§15).

The diagnostic value of separating these is high. The first is a configuration problem, the second a deployment problem, the third an implementation bug — and they are commonly reported identically, as "the handshake fails".

12.4 Both endpoints must change

A point that is obvious in principle and routinely underestimated in planning: a new suite is useless unilaterally (constraint C4). Unlike a path-1 implementation substitution, which is invisible to the peer, a new suite requires the peer to recognise the code point and to possess implementations of both named algorithms.

This bounds the applicability of the whole technique. A new suite is deployable where both endpoints are under one administration — internal services, an organisation's own clients, an embedded fleet — and is not deployable against the public web, where the peer is a browser whose suite list the deployment does not control. Chapter 16 develops the interoperability consequences.

12.5 Interaction with the other three paths

A suite introduces an AEAD and a hash. It does not introduce a group or a signature algorithm, which remain separately negotiated. A design that introduces all of them must therefore discharge the suite obligation and the two capability obligations; satisfying one does not imply the others.

This is the practical pay-off of the taxonomy in Chapter 4. A requirement phrased as "use our national algorithms in TLS" typically decomposes into a suite (for the cipher and hash), a TLS-GROUP entry (for the curve), and a TLS-SIGALG entry plus certificate work (for the signature) — three different mechanisms, of which only the first is the subject of this chapter.