1. Introduction

1.1 The problem

A TLS connection is often described as using "a cipher suite", as though the suite were a single choice. It is not. A TLS 1.3 connection simultaneously depends on at least four cryptographic decisions that are negotiated by different mechanisms, carried in different handshake fields, and — the point this paper develops — supplied to OpenSSL through different extension paths. The record layer needs an authenticated encryption algorithm. The key schedule needs a hash function, which also fixes the length of every secret derived during the handshake. The key exchange needs a group. The authentication step needs a signature algorithm matched to the certificate presented.

If an engineer wishes to introduce a new algorithm into this arrangement — a national standard block cipher, a curve mandated by a local regulator, a hash function chosen for migration reasons — the question that matters is not whether the algorithm can be implemented. It is whether the algorithm can be made to participate: advertised in the handshake, selected by negotiation, driven through its lifecycle by the TLS state machine, and agreed upon by a peer that was not modified.

OpenSSL 3 offers a mechanism that appears designed for exactly this. The provider interface allows a loadable module to supply algorithm implementations that the library will select at run time by name and by property query, without the application calling the module directly and without OpenSSL being recompiled. The companion study to this paper demonstrated the mechanism for a digest, showing that an algorithm supplied by a third-party module can be fetched and used through the ordinary EVP interface.

Extending that result from a digest to a TLS cipher suite is not a matter of repetition. A digest has one consumer and no negotiation. A cipher suite has a peer.

1.2 Scope

This paper is a design study of the following capability: a provider that supplies a block cipher in an AEAD mode, an elliptic-curve group, and a hash function, such that these algorithms can be used together in a TLS 1.3 connection.

The study covers:

• the cryptographic dependencies of a TLS 1.3 handshake, stated precisely enough to identify every point at which a provider-supplied algorithm is consumed (Chapter 2);
• the provider architecture as it stands in OpenSSL 3.5, including the dispatch table convention, parameter passing, and the library context (Chapter 3);
• the four extension paths by which a provider can influence a TLS connection, and how they differ (Chapter 4);
• the division of responsibility between libssl and libcrypto, which determines what provider code can and cannot reach (Chapter 5);
• a specified design for each of the three algorithm classes, with dispatch tables, context lifecycle, and parameter contracts (Chapters 7–11);
• cipher suite registration, negotiation and the key schedule (Chapters 12–15);
• interoperability, security analysis, proposed performance methodology, and a proposed validation strategy (Chapters 16–19).

The study does not cover: QUIC, DTLS, TLS 1.2 and earlier, post-quantum key encapsulation as a subject in its own right, hardware security module integration, or certificate path validation. Each is mentioned where it constrains the design, and each is excluded from the detailed treatment.

1.3 Why the provider interface rather than a patch

An engineer who wants a new algorithm in TLS has, in principle, two options: modify OpenSSL, or extend it. The second is preferable for reasons that are practical rather than aesthetic.

A patched OpenSSL must be maintained against a moving upstream. Every security release requires the patch to be rebased, retested and redeployed, and the patched library is no longer the library that distributions ship, package signatures cover, or auditors have reviewed. For a cryptographic library this is a significant operational cost, and it is paid indefinitely.

A provider, by contrast, is a separate shared object with its own lifecycle. It is loaded by configuration, it can be signed and distributed independently, and — critically — the ABI between the core and the provider is a table of function pointers with a negotiated content, not a struct layout. This is the property that makes the FIPS provider's separate validation lifecycle possible, and it is the same property that makes a third-party algorithm module maintainable.

There is a third consideration specific to cryptography. An organisation that must demonstrate which algorithm implementation was used — for compliance, for incident response, or for an audit — benefits from that implementation being a named, versioned, separately hashed artefact rather than a compile-time configuration of a larger library. The provider model makes the answer to "which code computed this ciphertext?" a question with a filename in it.

1.4 Contribution

The contributions of this paper are:

1. A taxonomy of the extension paths. Chapter 4 distinguishes four mechanisms that are frequently conflated in practitioner discussion: supplying an implementation for an algorithm TLS already knows, advertising a group through the TLS-GROUP capability, advertising a signature algorithm through TLS-SIGALG, and registering a cipher suite. They differ in mechanism and in reach, and the difference determines what a design can achieve without modifying libssl.
2. A specified provider design for three algorithm classes, at a level of detail sufficient for implementation: dispatch tables, context lifecycle, parameter contracts, and the ordering constraints imposed by the handshake (Chapters 7–11).
3. An analysis of the negotiation and key-schedule consequences of introducing a non-standard algorithm, including the transcript-hash coupling that makes the handshake hash a more invasive choice than the record cipher (Chapters 12 and 15).
4. A security analysis that treats the provider as part of the trusted computing base and enumerates what that implies (Chapter 17).
5. A validation strategy and a performance evaluation methodology, both stated as proposals with their threats to validity (Chapters 18 and 19).

1.5 Status of claims, and how to read this paper

Cryptographic engineering papers fail their readers most often by mixing three kinds of statement without marking the difference: what the system definitely does, what the author intends it to do, and what would be true if the design were implemented. This paper marks them explicitly, and the reader should rely on the marking.

Appendix D collects every load-bearing claim in the paper into a single table with its status and, where applicable, its source reference. A reader with limited time who wants to know what this paper actually establishes should read Chapter 4, then Appendix D.

1.6 Reference environment

All verified statements were checked in the following environment:

OpenSSL runtime     OpenSSL 3.5.8 25 Aug 2026 (library: OpenSSL 3.5.8)
OpenSSL headers     3.5.8 (libcrypto, libssl via pkg-config)
Source tree         openssl-3.5.8 (unpacked build tree)
Documentation       openssl-3.5.5 (pinned), OpenSSL_1_1_1g (pinned)
Platform            Linux x86_64, AlmaLinux 9.8
Compiler            gcc (available; no provider was compiled for this study)
Default providers   default (active)

The absence of a compiled artefact is deliberate and is stated here so that it is not mistaken for an omission. This is a design study; Chapter 20 lists what an implementation phase would need to produce for the design to be considered validated.

1.7 Structure of the paper

Chapters 2–5 establish the ground: what TLS needs, what a provider is, how the two can meet, and where the boundary between the two OpenSSL libraries falls. Chapter 6 states requirements. Chapters 7–15 are the design proper, moving from the provider's overall shape through each algorithm class to negotiation and the key schedule. Chapters 16–19 evaluate the design against interoperability, security, performance and testability. Chapter 20 concludes and states what remains.

Readers already familiar with the provider architecture may begin at Chapter 4. Readers interested only in what is provably true of stock OpenSSL should read Chapters 4 and 5 and Appendix D.