TCP Fast Open (TFO) mechanism is described in

Other considerations can be found in this white paper.




Client side

// Send first packet with sendto instead of connect
bytes_written = sendto(sockd, pkt_data, i, MSG_FASTOPEN, (struct sockaddr *) &serv_addr, sizeof (serv_addr));
// If the fastopen fails then it appears to fallback automagicly to regular 3 way handshake

Server side

// between bind and listen add
setsockopt(tcp, IPPROTO_TCP, TCP_FASTOPEN, &backlog, sizeof(backlog))

Kernel parameter

To enable TFO on the server machine, the kernel parameters need to be configured to support TCP_FASTOPEN in server mode. To enable it, run the following command as root:

sudo sysctl -w net.ipv4.tcp_fastopen=2

To act in pure server mode, set it to 2. If also running a TFO client on the machine, set it to 3.

More details from Documentation/networking/ip-sysctl.txt:

tcp_fastopen - INTEGER
	Enable TCP Fast Open feature (draft-ietf-tcpm-fastopen) to send data
	in the opening SYN packet. To use this feature, the client application
	must use sendmsg() or sendto() with MSG_FASTOPEN flag rather than
	connect() to perform a TCP handshake automatically.

	The values (bitmap) are
	1: Enables sending data in the opening SYN on the client w/ MSG_FASTOPEN.
	2: Enables TCP Fast Open on the server side, i.e., allowing data in
	   a SYN packet to be accepted and passed to the application before
	   3-way hand shake finishes.
	4: Send data in the opening SYN regardless of cookie availability and
	   without a cookie option.
	0x100: Accept SYN data w/o validating the cookie.
	0x200: Accept data-in-SYN w/o any cookie option present.
	0x400/0x800: Enable Fast Open on all listeners regardless of the
	   TCP_FASTOPEN socket option. The two different flags designate two
	   different ways of setting max_qlen without the TCP_FASTOPEN socket

	Default: 1

	Note that the client & server side Fast Open flags (1 and 2
	respectively) must be also enabled before the rest of flags can take

	See include/net/tcp.h and the code for more details.