4. Four Paths from a Provider into a TLS Connection

Practitioner discussion of "adding an algorithm to TLS through a provider" tends to treat the task as one thing. It is four things, with different mechanisms and different degrees of independence from the TLS library itself. This chapter separates them. It is the analytical core of the paper, and the design chapters are organised around the distinctions drawn here.

4.1 The four paths

#PathMechanismWhat it adds
1Algorithm implementationOSSL_ALGORITHM returned from the provider's query functionA different implementation of something TLS already names
2Group advertisementTLS-GROUP capabilityA new key-exchange group, including a new code point
3Signature advertisementTLS-SIGALG capabilityA new signature algorithm, including a new code point
4Cipher suite registrationThe suite table consulted by libsslA new suite pairing an AEAD with a handshake hash

Paths 2 and 3 are the interesting ones architecturally, because they show that OpenSSL's designers did build a route by which a provider introduces something genuinely new to the protocol — not merely a faster version of an existing algorithm, but a new code point that appears on the wire. Path 1 is the most common and the least ambitious. Path 4 is the subject of this paper's title and is treated in Chapter 12.

4.2 Path 1: supplying an implementation

The simplest path is to implement an algorithm TLS already knows, under the name TLS already uses, and let property-based selection prefer it. A provider offering AES-256-GCM competes with the default provider's implementation; a property query, or the ordering of activated providers, determines which is chosen.

This path adds nothing to the protocol. No new code point appears on the wire, no peer needs to change, and interoperability is unaffected — the connection is an ordinary TLS_AES_256_GCM_SHA384 connection that happens to be computed by different code. That is precisely why it is useful: it is the path for hardware offload, for a formally verified implementation, or for a validated module. It is also the path with the sharpest failure mode, since a subtly wrong implementation of a standard algorithm produces connections that fail in ways the protocol has no vocabulary to describe.

The companion paper's edu provider is an instance of this path applied to a digest: it supplies EDU-SHA256, fetched by name, delegating to the default provider in a separate library context.

4.3 Paths 2 and 3: the capability mechanism

OpenSSL exposes a general mechanism by which a provider declares support for something the protocol layer will consume, called a capability. The core asks a provider for a named capability, and the provider answers by invoking a callback once per item it wishes to advertise, passing an OSSL_PARAM array that describes it.

Verified. Two capabilities relevant to TLS are recognised. The built-in providers dispatch on the capability name and answer TLS-GROUP and TLS-SIGALG, returning 0 — "we don't support this capability" — for anything else (providers/common/capabilities.c:335–347, tree openssl-3.5.8).

4.3.1 What a group advertisement contains

A group is advertised as a parameter array carrying its external (IANA) name, an internal algorithm name, the algorithm that implements it, a numeric code point, and security metadata. The built-in list is constructed by a macro:

#define TLS_GROUP_ENTRY(tlsname, realname, algorithm, idx)              \
    {                                                                   \
        OSSL_PARAM_utf8_string(OSSL_CAPABILITY_TLS_GROUP_NAME,          \
            tlsname, sizeof(tlsname)),                                  \
        OSSL_PARAM_utf8_string(OSSL_CAPABILITY_TLS_GROUP_NAME_INTERNAL, \
            realname, sizeof(realname)),                                \
        OSSL_PARAM_utf8_string(OSSL_CAPABILITY_TLS_GROUP_ALG,           \
            algorithm, sizeof(algorithm)),                              \
        OSSL_PARAM_uint(OSSL_CAPABILITY_TLS_GROUP_ID,                   \
            (unsigned int *)&group_list[idx].group_id),                \
        ...
    }

and used to build the list of supported groups:

TLS_GROUP_ENTRY("x25519",    "X25519",     "X25519", 28),
TLS_GROUP_ENTRY("x448",      "X448",       "X448",   29),
TLS_GROUP_ENTRY("secp256r1", "prime256v1", "EC",     22),

Verified. providers/common/capabilities.c:96–108 for the macro and :161–164 for these entries, tree openssl-3.5.8.

Three observations follow. The code point is supplied by the provider, as a parameter, which means a group that OpenSSL's own code does not know can be advertised and will appear in supported_groups. The internal name maps the wire name to an algorithm the provider implements, so the group name and the key-management algorithm need not coincide. And the EC entry shows that many named curves may share one implementing algorithm, distinguished by a curve parameter — the arrangement an elliptic-curve design in this paper will follow.

4.3.2 What a signature advertisement contains

Signature algorithms are advertised analogously, with the IANA name, the algorithm name, an OID, a code point and a security-bits value:

#define TLS_SIGALG_ENTRY(tlsname, algorithm, oid, idx)               \
    {                                                                \
        OSSL_PARAM_utf8_string(OSSL_CAPABILITY_TLS_SIGALG_IANA_NAME, \
            tlsname, sizeof(tlsname)),                               \
        OSSL_PARAM_utf8_string(OSSL_CAPABILITY_TLS_SIGALG_NAME,      \
            algorithm, sizeof(algorithm)),                           \
        OSSL_PARAM_utf8_string(OSSL_CAPABILITY_TLS_SIGALG_OID,       \
            oid, sizeof(oid)),                                       \
        OSSL_PARAM_uint(OSSL_CAPABILITY_TLS_SIGALG_CODE_POINT,       \
            (unsigned int *)&sigalg_constants_list[idx].code_point), \
        OSSL_PARAM_uint(OSSL_CAPABILITY_TLS_SIGALG_SECURITY_BITS,    \
        ...
    }

The most instructive fact about this path is what OpenSSL itself uses it for:

TLS_SIGALG_ENTRY("mldsa44", "ML-DSA-44", "2.16.840.1.101.3.4.3.17", 0),
TLS_SIGALG_ENTRY("mldsa65", "ML-DSA-65", "2.16.840.1.101.3.4.3.18", 1),
TLS_SIGALG_ENTRY("mldsa87", "ML-DSA-87", "2.16.840.1.101.3.4.3.19", 2),

Verified. providers/common/capabilities.c:292–302 for the macro and :316–318 for the ML-DSA entries, tree openssl-3.5.8.

Post-quantum signature algorithms — algorithms that did not exist when the TLS code was written — reach the handshake through the same capability mechanism available to a third-party provider. This is the strongest available evidence that paths 2 and 3 are genuine extension points rather than internal conveniences: OpenSSL uses them to introduce new cryptography to TLS, and the interface is not privileged.

4.4 Path 4: cipher suites

Cipher suites differ from groups and signature algorithms in how libssl obtains the list of what exists. Rather than querying providers for a capability, the TLS implementation consults a table of suite descriptions, each binding a code point to an AEAD algorithm, a handshake MAC, protocol version bounds and strength metadata.

Verified. In the reference tree the TLS 1.3 suites are held in a table declared static SSL_CIPHER tls13_ciphers[] whose length is taken with OSSL_NELEM at compile time (ssl/s3_lib.c:26 and :39, tree openssl-3.5.8). Each entry names an AEAD and a handshake MAC, as shown in §2.1. There is no TLS-CIPHERSUITE capability in ossl_prov_get_capabilities(), which answers only TLS-GROUP and TLS-SIGALG.

The design consequence is a real asymmetry, and it should be stated without softening: the three paths above place the provider in control of what is advertised, whereas the suite list is owned by the TLS library. A system that introduces a new suite must therefore arrange for that suite to be present in the list libssl consults, in addition to providing the algorithms behind it.

Reported. The system documented by this paper is reported by its author to add cipher suites combining block cipher, elliptic-curve and hash algorithms through the provider interface, and to use them in TLS communication. This study had no access to that implementation and did not reproduce the result. Chapter 12 specifies the registration and negotiation behaviour such a capability entails; Chapter 19 gives the test plan that would demonstrate it; Appendix D records the claim and its status.

4.5 Comparing the paths

PropertyPath 1
Implementation
Path 2
Group
Path 3
Sigalg
Path 4
Suite
New code point on the wireNoYesYesYes
Peer must also support itNoYesYesYes
Advertised by providern/aYesYesSee §4.4
Reachable without touching libsslYesYesYesSee §4.4
Affects key schedule widthNoNoNoYes, if the hash changes
Failure is visible asWrong resultsNo shared groupHandshake failureNo shared suite

4.6 Which path a requirement should take

The taxonomy yields practical guidance. If the requirement is a validated or accelerated implementation of a standard algorithm, path 1 is correct and the others are unnecessary complexity. If the requirement is a new curve or a new signature scheme — the common case for national standards and for post-quantum migration — paths 2 and 3 are the intended mechanism, and they do not require a modified TLS library. If the requirement is genuinely a new suite, pairing an AEAD with a handshake hash under a new code point, the work includes whatever is needed to make that suite visible to the negotiation logic, and that is the subject of Chapter 12.

A requirement is frequently stated as the fourth kind when it is really the first or second. An organisation that must use a particular national block cipher may find that its actual obligation is satisfiable by path 1 if a standard suite naming an equivalent-strength AEAD is acceptable, or that its curve requirement is a path-2 problem. Establishing which path a requirement truly needs is, in the experience this paper reflects, the single most valuable step in the design.