Websocket Protocol | HFT

In the fast-paced world of high-frequency trading (HFT) and modern financial technology, the WebSocket protocol has become an indispensable tool. This article explores the significance of WebSockets in HFT and crypto trading, highlighting their broader applications in real-time data exchange.

The Evolution of Real-Time Communication

Before diving into WebSockets, it’s important to understand why they were developed. Traditional HTTP communication has limitations when it comes to real-time, bidirectional data exchange:

1. Connection Overhead: Each HTTP request-response cycle requires a new connection, adding latency.
2. Unidirectional Communication: HTTP is primarily designed for client-initiated requests, making server-initiated updates challenging.
3. Polling Inefficiency: Techniques like long-polling create unnecessary network traffic and server load.
4. Multiple TCP Connections: Servers might be forced to use multiple TCP connections for each client, complicating state management.
5. HTTP Header Overhead: Each HTTP request includes headers, creating additional network overhead.

WebSockets address these issues by providing a persistent, full-duplex communication channel over a single TCP connection.

WebSockets in High-Frequency Trading and Crypto Markets

Why WebSockets Matter in Trading

1. Ultra-Low Latency: Critical for executing trades at the speed required in HFT.
2. Real-Time Market Data: Instant updates on price changes, order book modifications, and trade executions.
3. Efficient Order Management: Quick submission and real-time status updates of orders.
4. Reduced Network Overhead: Lower data transfer compared to constant HTTP polling.
5. Crypto Exchange Support: Many major crypto exchanges now offer WebSocket APIs for automated trading.

Crypto Exchanges and WebSockets

Let’s look at an example from Bybit, a prominent crypto exchange:

const ws = new WebSocket('wss://stream.bybit.com/realtime');
jab
ws.onopen = () => {
    // Subscribe to order book updates
    ws.send(JSON.stringify({
        "op": "subscribe",
        "args": ["orderBook_200.100ms.BTCUSD"]
    }));
};

ws.onmessage = (event) => {
    const data = JSON.parse(event.data);
    if (data.topic && data.topic.startsWith('orderBook')) {
        console.log('Order book update:', data);
        // Process order book data
    }
};

This example demonstrates subscribing to real-time order book updates for the BTCUSD pair on Bybit.

Deep Dive into the WebSocket Protocol

WebSocket URIs

WebSocket URIs are similar to HTTP URIs but use different schemes:

• ws:// for unencrypted connections
• wss:// for connections encrypted with TLS

For example: wss://stream.example.com/realtime

The WebSocket Handshake

The WebSocket connection begins with an HTTP-based handshake, ensuring compatibility with existing web infrastructure. Here’s a detailed look at the handshake process:

Client Request:

GET /ws-endpoint HTTP/1.1
Host: example.com
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
Origin: http://example.com
Sec-WebSocket-Version: 13

Server Response:

HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=

The Sec-WebSocket-Key in the client request and Sec-WebSocket-Accept in the server response are crucial for the WebSocket handshake process.

WebSocket Frame Structure

Once the connection is established, data is exchanged in frames. Understanding the frame structure is crucial for efficient implementation:

0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-------+-+-------------+-------------------------------+
|F|R|R|R| opcode|M| Payload len |    Extended payload length    |
|I|S|S|S|  (4)  |A|     (7)     |             (16/64)           |
|N|V|V|V|       |S|             |   (if payload len==126/127)   |
| |1|2|3|       |K|             |                               |
+-+-+-+-+-------+-+-------------+ - - - - - - - - - - - - - - - +
|     Extended payload length continued, if payload len == 127  |
+ - - - - - - - - - - - - - - - +-------------------------------+
|                               |Masking-key, if MASK set to 1  |
+-------------------------------+-------------------------------+
| Masking-key (continued)       |          Payload Data         |
+-------------------------------- - - - - - - - - - - - - - - - +
:                     Payload Data continued ...                :
+ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +
|                     Payload Data continued ...                |
+---------------------------------------------------------------+

Key components include:

• FIN bit: Indicates if this is the final fragment in a message.
• Opcode: Defines the type of the frame (e.g., text, binary, close, ping, pong).
• MASK: Indicates whether the payload is masked.
• Payload length: The length of the frame’s payload data.
• Masking key: Used to unmask the payload (if masked).
• Payload data: The actual data being transmitted.

Fragmentation and Message Continuity

WebSockets support message fragmentation, allowing large messages to be split into multiple frames. This is particularly useful in trading scenarios where large amounts of market data need to be transmitted quickly.

std::vector<uint8_t> message_buffer;
bool is_fragmented = false;

void handle_frame(const WebSocketFrame& frame) {
    if (!is_fragmented && frame.opcode != CONTINUATION) {
        message_buffer.clear();
    }

    message_buffer.insert(message_buffer.end(), frame.payload.begin(), frame.payload.end());

    if (frame.fin) {
        process_complete_message(message_buffer);
        is_fragmented = false;
    } else {
        is_fragmented = true;
    }
}

It’s important to note that, according to the WebSocket protocol, message fragments must be delivered to the recipient in the order sent by the sender. Interleaving of message fragments is not allowed.

Control Frames: Ping, Pong, and Close

WebSockets use control frames for connection management:

• Ping and Pong: Used as a keep-alive mechanism and to verify that the remote endpoint is still responsive.
• Close: Initiates the closing handshake.

void handle_control_frame(const WebSocketFrame& frame) {
    switch (frame.opcode) {
        case PING:
            send_pong_frame();
            break;
        case CLOSE:
            perform_close_handshake();
            break;
        // ... handle other control frames
    }
}

Data Masking

In the WebSocket protocol, all frames sent from the client to the server must be masked. This is a security measure to prevent cache poisoning attacks. The masking process uses a simple XOR operation:

void mask_payload(std::vector<uint8_t>& payload, const uint8_t masking_key[4]) {
    for (size_t i = 0; i < payload.size(); ++i) {
        payload[i] ^= masking_key[i % 4];
    }
}

WebSockets in the OSI Model

To understand WebSockets in the context of network communication, it’s helpful to consider their place in the OSI (Open Systems Interconnection) model:

1. Physical Layer: The physical medium (e.g., Ethernet cables)
2. Data Link Layer: Ethernet frames
3. Network Layer: IP packets
4. Transport Layer: TCP segments
5. Session Layer: (Often combined with the Application layer in practice)
6. Presentation Layer: (Often combined with the Application layer in practice)
7. Application Layer: WebSocket frames

WebSocket operates at the Application layer, but it relies on the lower layers for actual data transmission. When a WebSocket frame is sent:

1. It’s encapsulated in a TCP segment
2. The TCP segment is encapsulated in an IP packet
3. The IP packet is encapsulated in an Ethernet frame
4. The Ethernet frame is transmitted over the physical medium

This layered approach allows WebSockets to leverage the reliability of TCP while providing a high-level, full-duplex communication channel.

Implementing WebSockets for HFT and Crypto Trading

When implementing WebSockets for high-frequency or crypto trading, consider these factors:

1. Low-Latency Libraries: Use optimized WebSocket libraries. For C++, consider libraries like Beast (part of Boost) or libwebsockets.
2. Efficient Parsing: Use fast JSON parsing libraries. For C++, the nlohmann/json library is popular:

#include <nlohmann/json.hpp>

void process_json_message(const std::vector<uint8_t>& message_data) {
    try {
        auto json = nlohmann::json::parse(message_data);
        // Process the JSON data
    } catch (const nlohmann::json::exception& e) {
        // Handle JSON parsing errors
    }
}

1. Connection Management: Implement robust reconnection logic and handle network issues gracefully.
2. Data Optimization: Minimize data transfer by subscribing only to necessary data streams.
3. Performance Monitoring: Implement latency monitoring and logging for performance optimization.
4. Security Considerations: Ensure proper implementation of the masking process for client-to-server messages.
5. Compliance with Exchange APIs: Carefully review and comply with the specific WebSocket API documentation of each exchange you’re connecting to.

Challenges and Considerations

While WebSockets offer significant advantages, they also come with challenges:

1. Connection Stability: Maintaining stable WebSocket connections over long periods can be challenging, especially in high-stakes trading environments.
2. Scalability: As the number of concurrent WebSocket connections increases, server-side scalability becomes a concern.
3. Firewall Issues: Some corporate firewalls may block WebSocket connections, necessitating fallback mechanisms.
4. Protocol Overhead: While less than HTTP, WebSockets still have some protocol overhead that needs to be managed in ultra-low-latency scenarios.
5. Data Synchronization: Ensuring data consistency across multiple WebSocket streams can be complex, especially in multi-exchange trading systems.
6. Security: While WebSockets use the same-origin policy and can be secured with TLS (wss://), developers must be vigilant about potential security vulnerabilities.
7. MTU Limitations: The Maximum Transmission Unit (MTU) of the network can limit the size of individual WebSocket frames, requiring efficient fragmentation strategies.

Conclusion

WebSockets have transformed real-time data exchange, especially in high-frequency trading and cryptocurrency markets. Their low-latency, full-duplex communication is ideal for fast data updates and order execution. As trading technology evolves, expertise in WebSockets will remain essential for building efficient, responsive systems. Staying updated with advancements in this field will be key for developers pushing the boundaries of real-time financial data exchange. For more on the protocol, refer to RFC6455.

Thank you for reading! If you enjoyed the article, please like and share your thoughts in the comments below.