[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

3. Introduction to TLS

TLS stands for "Transport Layer Security" and is the successor of SSL, the Secure Sockets Layer protocol [SSL3] (see section Bibliography) designed by Netscape. TLS is an Internet protocol, defined by IETF(7), described in RFC 4346 and also in [RESCORLA] (see section Bibliography). The protocol provides confidentiality, and authentication layers over any reliable transport layer. The description, below, refers to TLS 1.0 but also applies to TLS 1.1 [RFC4346] (see section Bibliography) and SSL 3.0, since the differences of these protocols are minor. Older protocols such as SSL 2.0 are not discussed nor implemented in GnuTLS since they are not considered secure today. GnuTLS also supports X.509 and OpenPGP [RFC4880] (see section Bibliography).

[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

3.1 TLS Layers

TLS is a layered protocol, and consists of the Record Protocol, the Handshake Protocol and the Alert Protocol. The Record Protocol is to serve all other protocols and is above the transport layer. The Record protocol offers symmetric encryption, data authenticity, and optionally compression.

The Alert protocol offers some signaling to the other protocols. It can help informing the peer for the cause of failures and other error conditions. See The Alert Protocol, for more information. The alert protocol is above the record protocol.

The Handshake protocol is responsible for the security parameters' negotiation, the initial key exchange and authentication. See The Handshake Protocol, for more information about the handshake protocol. The protocol layering in TLS is shown in the figure below.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

3.2 The Transport Layer

TLS is not limited to one transport layer, it can be used above any transport layer, as long as it is a reliable one. A set of functions is provided and their purpose is to load to GnuTLS the required callbacks to access the transport layer.

These functions accept a callback function as a parameter. The callback functions should return the number of bytes written, or -1 on error and should set errno appropriately.

In some environments, setting errno is unreliable, for example Windows have several errno variables in different CRTs, or it may be that errno is not a thread-local variable. If this is a concern to you, call gnutls_transport_set_errno with the intended errno value instead of setting errno directly.

GnuTLS currently only interprets the EINTR and EAGAIN errno values and returns the corresponding GnuTLS error codes GNUTLS_E_INTERRUPTED and GNUTLS_E_AGAIN. These values are usually returned by interrupted system calls, or when non blocking IO is used. All GnuTLS functions can be resumed (called again), if any of these error codes is returned. The error codes above refer to the system call, not the GnuTLS function, since signals do not interrupt GnuTLS' functions.

For non blocking sockets or other custom made pull/push functions the gnutls_transport_set_lowat must be called, with a zero low water mark value.

By default, if the transport functions are not set, GnuTLS will use the Berkeley Sockets functions. In this case GnuTLS will use some hacks in order for select to work, thus making it easy to add TLS support to existing TCP/IP servers.

[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

3.3 The TLS Record Protocol

The Record protocol is the secure communications provider. Its purpose is to encrypt, authenticate and --optionally-- compress packets. The following functions are available:


To send a record packet (with application data).


To receive a record packet (with application data).


To get the direction of the last interrupted function call.

As you may have already noticed, the functions which access the Record protocol, are quite limited, given the importance of this protocol in TLS. This is because the Record protocol's parameters are all set by the Handshake protocol.

The Record protocol initially starts with NULL parameters, which means no encryption, and no MAC is used. Encryption and authentication begin just after the handshake protocol has finished.

[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

3.3.1 Encryption Algorithms Used in the Record Layer

Confidentiality in the record layer is achieved by using symmetric block encryption algorithms like 3DES, AES(8), or stream algorithms like ARCFOUR_128(9). Ciphers are encryption algorithms that use a single, secret, key to encrypt and decrypt data. Block algorithms in TLS also provide protection against statistical analysis of the data. Thus, if you're using the TLS protocol, a random number of blocks will be appended to data, to prevent eavesdroppers from guessing the actual data size.

Supported cipher algorithms:


3DES_CBC is the DES block cipher algorithm used with triple encryption (EDE). Has 64 bits block size and is used in CBC mode.


ARCFOUR is a fast stream cipher.


This is the ARCFOUR cipher that is fed with a 40 bit key, which is considered weak.


AES or RIJNDAEL is the block cipher algorithm that replaces the old DES algorithm. Has 128 bits block size and is used in CBC mode. This is not officially supported in TLS.

Supported MAC algorithms:


MD5 is a cryptographic hash algorithm designed by Ron Rivest. Outputs 128 bits of data.


SHA is a cryptographic hash algorithm designed by NSA. Outputs 160 bits of data.

[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

3.3.2 Compression Algorithms Used in the Record Layer

The TLS record layer also supports compression. The algorithms implemented in GnuTLS can be found in the table below. All the algorithms except for DEFLATE which is referenced in [RFC3749] (see section Bibliography), should be considered as GnuTLS' extensions(10), and should be advertised only when the peer is known to have a compliant client, to avoid interoperability problems.

The included algorithms perform really good when text, or other compressible data are to be transfered, but offer nothing on already compressed data, such as compressed images, zipped archives etc. These compression algorithms, may be useful in high bandwidth TLS tunnels, and in cases where network usage has to be minimized. As a drawback, compression increases latency.

The record layer compression in GnuTLS is implemented based on the proposal [RFC3749] (see section Bibliography). The supported compression algorithms are:


Zlib compression, using the deflate algorithm.


LZO is a very fast compression algorithm. This algorithm is only available if the GnuTLS-extra library has been initialized and the private extensions are enabled, and if GnuTLS was built with LZO support.

[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

3.3.3 Weaknesses and Countermeasures

Some weaknesses that may affect the security of the Record layer have been found in TLS 1.0 protocol. These weaknesses can be exploited by active attackers, and exploit the facts that

  1. TLS has separate alerts for "decryption_failed" and "bad_record_mac"
  2. The decryption failure reason can be detected by timing the response time.
  3. The IV for CBC encrypted packets is the last block of the previous encrypted packet.

Those weaknesses were solved in TLS 1.1 [RFC4346] (see section Bibliography) which is implemented in GnuTLS. For a detailed discussion see the archives of the TLS Working Group mailing list and the paper [CBCATT] (see section Bibliography).

[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

3.4 The TLS Alert Protocol

The Alert protocol is there to allow signals to be sent between peers. These signals are mostly used to inform the peer about the cause of a protocol failure. Some of these signals are used internally by the protocol and the application protocol does not have to cope with them (see GNUTLS_A_CLOSE_NOTIFY), and others refer to the application protocol solely (see GNUTLS_A_USER_CANCELLED). An alert signal includes a level indication which may be either fatal or warning. Fatal alerts always terminate the current connection, and prevent future renegotiations using the current session ID.

The alert messages are protected by the record protocol, thus the information that is included does not leak. You must take extreme care for the alert information not to leak to a possible attacker, via public log files etc.


To send an alert signal.


To map a gnutls error number to an alert signal.


Returns the last received alert.


Returns the name, in a character array, of the given alert.

[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

3.5 The TLS Handshake Protocol

The Handshake protocol is responsible for the ciphersuite negotiation, the initial key exchange, and the authentication of the two peers. This is fully controlled by the application layer, thus your program has to set up the required parameters. Available functions to control the handshake protocol include:


To initialize a priority set of ciphers.


To deinitialize a priority set of ciphers.


To associate a priority set with a TLS session.


To directly associate a session with a given priority string.


To set the appropriate credentials structures.


To set whether client certificate is required or not.


To initiate the handshake.

[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

3.5.1 TLS Cipher Suites

The Handshake Protocol of TLS negotiates cipher suites of the form TLS_DHE_RSA_WITH_3DES_CBC_SHA. The usual cipher suites contain these parameters:

The cipher suite negotiated in the handshake protocol will affect the Record Protocol, by enabling encryption and data authentication. Note that you should not over rely on TLS to negotiate the strongest available cipher suite. Do not enable ciphers and algorithms that you consider weak.

The priority functions, dicussed above, allow the application layer to enable and set priorities on the individual ciphers. It may imply that all combinations of ciphersuites are allowed, but this is not true. For several reasons, not discussed here, some combinations were not defined in the TLS protocol. The supported ciphersuites are shown in ciphersuites.

[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

3.5.2 Client Authentication

In the case of ciphersuites that use certificate authentication, the authentication of the client is optional in TLS. A server may request a certificate from the client -- using the gnutls_certificate_server_set_request function. If a certificate is to be requested from the client during the handshake, the server will send a certificate request message that contains a list of acceptable certificate signers. In GnuTLS the certificate signers list is constructed using the trusted Certificate Authorities by the server. That is the ones set using

Sending of the names of the CAs can be controlled using gnutls_certificate_send_x509_rdn_sequence. The client, then, may send a certificate, signed by one of the server's acceptable signers.

[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

3.5.3 Resuming Sessions

The gnutls_handshake function, is expensive since a lot of calculations are performed. In order to support many fast connections to the same server a client may use session resuming. Session resuming is a feature of the TLS protocol which allows a client to connect to a server, after a successful handshake, without the expensive calculations. This is achieved by using the previously established keys. GnuTLS supports this feature, and the example (see ex:resume-client) illustrates a typical use of it.

Keep in mind that sessions are expired after some time, for security reasons, thus it may be normal for a server not to resume a session even if you requested that. Also note that you must enable, using the priority functions, at least the algorithms used in the last session.

[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

3.5.4 Resuming Internals

The resuming capability, mostly in the server side, is one of the problems of a thread-safe TLS implementations. The problem is that all threads must share information in order to be able to resume sessions. The gnutls approach is, in case of a client, to leave all the burden of resuming to the client. I.e., copy and keep the necessary parameters. See the functions:

The server side is different. A server has to specify some callback functions which store, retrieve and delete session data. These can be registered with:

It might also be useful to be able to check for expired sessions in order to remove them, and save space. The function gnutls_db_check_entry is provided for that reason.

[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

3.6 TLS Extensions

A number of extensions to the TLS protocol have been proposed mainly in [TLSEXT] (see section Bibliography). The extensions supported in GnuTLS are:

and they will be discussed in the subsections that follow.

[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

3.6.1 Maximum Fragment Length Negotiation

This extension allows a TLS implementation to negotiate a smaller value for record packet maximum length. This extension may be useful to clients with constrained capabilities. See the gnutls_record_set_max_size and the gnutls_record_get_max_size functions.

[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

3.6.2 Server Name Indication

A common problem in HTTPS servers is the fact that the TLS protocol is not aware of the hostname that a client connects to, when the handshake procedure begins. For that reason the TLS server has no way to know which certificate to send.

This extension solves that problem within the TLS protocol, and allows a client to send the HTTP hostname before the handshake begins within the first handshake packet. The functions gnutls_server_name_set and gnutls_server_name_get can be used to enable this extension, or to retrieve the name sent by a client.

[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

3.7 Selecting Cryptographic Key Sizes

In TLS, since a lot of algorithms are involved, it is not easy to set a consistent security level. For this reason this section will present some correspondance between key sizes of symmetric algorithms and public key algorithms based on the most conservative values of [SELKEY] (see section Bibliography). Those can be used to generate certificates with appropriate key sizes as well as parameters for Diffie Hellman and SRP authentication.


Symmetric key size

RSA key size, DH and SRP prime size

ECC key size





























The first column provides an estimation of the year until these parameters are considered safe and the rest of the columns list the parameters for the various algorithms.

Note however that the values suggested here are nothing more than an educated guess that is valid today. There are no guarrantees that an algorithm will remain unbreakable or that these values will remain constant in time. There could be scientific breakthroughs that cannot be predicted or total failure of the current public key systems by quantum computers. On the other hand though the cryptosystems used in TLS are selected in a conservative way and such catastrophic breakthroughs or failures are believed to be unlikely.

NIST publication SP 800-57 [NISTSP80057] (see section Bibliography) contains a similar table that extends beyond the key sizes given above.

Bits of security

Symmetric key algorithms

RSA key size, DSA, DH and SRP prime size

ECC key size





















The recommendations are fairly consistent.

[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

3.8 On SSL 2 and Older Protocols

One of the initial decisions in the GnuTLS development was to implement the known security protocols for the transport layer. Initially TLS 1.0 was implemented since it was the latest at that time, and was considered to be the most advanced in security properties. Later the SSL 3.0 protocol was implemented since it is still the only protocol supported by several servers and there are no serious security vulnerabilities known.

One question that may arise is why we didn't implement SSL 2.0 in the library. There are several reasons, most important being that it has serious security flaws, unacceptable for a modern security library. Other than that, this protocol is barely used by anyone these days since it has been deprecated since 1996. The security problems in SSL 2.0 include:

Other protocols such as Microsoft's PCT 1 and PCT 2 were not implemented because they were also abandoned and deprecated by SSL 3.0 and later TLS 1.0.

[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

3.9 On Record Padding

The TLS protocol allows for random padding of records, to make it more difficult to perform analysis on the length of exchanged messages. (In RFC 4346 this is specified in section GnuTLS appears to be one of few implementation that take advantage of this text, and pad records by a random length.

The TLS implementation in the Symbian operating system, frequently used by Nokia and Sony-Ericsson mobile phones, cannot handle non-minimal record padding. What happens when one of these clients handshake with a GnuTLS server is that the client will fail to compute the correct MAC for the record. The client sends a TLS alert (bad_record_mac) and disconnects. Typically this will result in error messages such as 'A TLS fatal alert has been received', 'Bad record MAC', or both, on the GnuTLS server side.

GnuTLS implements a work around for this problem. However, it has to be enabled specifically. It can be enabled by using gnutls_record_disable_padding, or gnutls_priority_set with the %COMPAT priority string.

If you implement an application that have a configuration file, we recommend that you make it possible for users or administrators to specify a GnuTLS protocol priority string, which is used by your application via gnutls_priority_set. To allow the best flexibility, make it possible to have a different priority string for different incoming IP addresses.

To enable the workaround in the gnutls-cli client or the gnutls-serv server, for testing of other implementations, use the following parameter: --priority "%COMPAT".

This problem has been discussed on mailing lists and in bug reports. This section tries to collect all pieces of information that we know about the problem. If you wish to go back to the old discussions, here are some links:





[ << ] [ >> ]           [Top] [Contents] [Index] [ ? ]

This document was generated on July, 20 2009 using texi2html 1.76.