References

TCP Fast Open (TFO) mechanism is described in https://tools.ietf.org/html/draft-ietf-tcpm-fastopen-10.

Other considerations can be found in this white paper.

Quick guide

TCP fastopen [I-D.ietf-tcpm-fastopen] (TFO) allows data to be carried in the SYN packet. It also saves up to one RTT compared to standard TCP. TFO clients request a server cookie in the initial SYN packet at the
start of a new connection. The server returns a cookie in its SYN-ACK. The client caches the cookie and reuses it when opening subsequent connections to the same server. The cookie is stored by the client's TCP stack (kernel) and persists if either the client or server processes are restarted. TFO also falls back to a regular TCP handshake gracefully. The implementation in the Linux kernel also caches negative responses from servers and disables further attempts to use TFO temporarily on that specific connection. 

 

Message flow

 

Requesting Fast Open Cookie in connection 1:

   TCP A (Client)                                       TCP B(Server)
   ______________                                       _____________
   CLOSED                                                      LISTEN

   #1 SYN-SENT       ----- <SYN,CookieOpt=NIL>  ---------->  SYN-RCVD
   #2 ESTABLISHED    <---- <SYN,ACK,CookieOpt=C> ----------  SYN-RCVD
   (caches cookie C)
   Performing TCP Fast Open in connection 2:

   TCP A (Client)                                       TCP B(Server)
   ______________                                       _____________
   CLOSED                                                      LISTEN

   #1 SYN-SENT       ----- <SYN=x,CookieOpt=C,DATA_A> ---->  SYN-RCVD
   #2 ESTABLISHED    <---- <SYN=y,ACK=x+len(DATA_A)+1> ----  SYN-RCVD
   #3 ESTABLISHED    <---- <ACK=x+len(DATA_A)+1,DATA_B>----  SYN-RCVD
   #4 ESTABLISHED    ----- <ACK=y+1>--------------------> ESTABLISHED
   #5 ESTABLISHED    --- <ACK=y+len(DATA_B)+1>----------> ESTABLISHED


Implementation

 TFO is currently only available on Linux.

Adding support for this to existing name server implementations is relatively easy, but does require source code modifications. It is also controlled via the kernel parameter net.ipv4.tcp_fastopen, which is set to 1 by default which enables only client mode TFO. The changes required to support TFO are:

Client side

Server side