KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#dsh #dsh-plugin #developer-tools #ai-agent #claude-code #codex #typescript #ai-agents #deepseek-harness #mcp #cli #rust

gaio

★ 919 Open GitHub ↗

High-performance, minimalist async-io (proactor) networking for Golang.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# gaio

[![GoDoc][1]][2] [![MIT licensed][3]][4] [![Build Status][5]][6] [![Go Report Card][7]][8] [![Coverage Statusd][9]][10]

[1]: https://godoc.org/github.com/xtaci/gaio?status.svg
[2]: https://pkg.go.dev/github.com/xtaci/gaio
[3]: https://img.shields.io/badge/license-MIT-blue.svg
[4]: LICENSE
[5]: https://img.shields.io/github/created-at/xtaci/gaio
[6]: https://img.shields.io/github/created-at/xtaci/gaio
[7]: https://goreportcard.com/badge/github.com/xtaci/gaio
[8]: https://goreportcard.com/report/github.com/xtaci/gaio
[9]: https://codecov.io/gh/xtaci/gaio/branch/master/graph/badge.svg
[10]: https://codecov.io/gh/xtaci/gaio

## Introduction
[中文文档](README_zh.md) | [中文介绍(知乎)](https://zhuanlan.zhihu.com/p/102890337)

In a typical Go network program, you accept a connection with `conn := lis.Accept()`, then spawn a goroutine to handle incoming data using `go func(net.Conn)`. Next, you allocate a buffer with `buf := make([]byte, 4096)` and wait for data with `conn.Read(buf)`.

For servers managing 10,000+ connections with frequent short messages (e.g., <512 bytes), context switching costs significantly exceed message receiving costs—each context switch requires over 1,000 CPU cycles, or approximately 600 ns on a 2.1 GHz processor.

By eliminating the one-goroutine-per-connection model through Edge-Triggered I/O Multiplexing, you can save the 2KB (R) + 2KB (W) stack space typically allocated per goroutine. Additionally, using an internal swap buffer eliminates the need for `buf := make([]byte, 4096)`, trading some performance for memory efficiency.

The gaio library implements the proactor pattern, effectively balancing memory constraints with performance requirements.

## How It Works

![alt text](https://raw.githubusercontent.com/xtaci/gaio/master/assets/gaio.jpg)

The `dup` function is used to copy the file descriptor from `net.Conn`:

```
NAME
       dup, dup2, dup3 - duplicate a file descriptor

LIBRARY
       Standard C library (libc, -lc)

SYNOPSIS
       #include <unistd.h>

int dup(int oldfd);
       int dup2(int oldfd, int newfd);

#define _GNU_SOURCE             /* See feature_test_macros(7) */
       #include <fcntl.h>              /* Definition of O_* constants */
       #include <unistd.h>

int dup3(int oldfd, int newfd, int flags);

DESCRIPTION
       The dup() system call allocates a new file descriptor that refers to the same open file description as the de‐
       scriptor oldfd.  (For an explanation of open file descriptions, see open(2).)  The new file descriptor  number
       is guaranteed to be the lowest-numbered file descriptor that was unused in the calling process.

After  a  successful return, the old and new file descriptors may be used interchangeably.  Since the two file
       descriptors refer to the same open file description, they share file offset and file status flags;  for  exam‐
       ple,  if  the  file  offset  is  modified by using lseek(2) on one of the file descriptors, the offset is also
       changed for the other file descriptor.

The two file descriptors do not share file descriptor flags (the close-on-exec flag).  The close-on-exec  flag
       (FD_CLOEXEC; see fcntl(2)) for the duplicate descriptor is off.
```

## Features

- **High Performance:** Battle-tested in High-Frequency Trading environments, achieving 30K–40K RPS on a single HVM server.
- **Scalability:** Designed for C10K+ concurrent connections, optimizing both parallelism and per-connection throughput.
- **Flexible Buffering:** Use `Read(ctx, conn, buffer)` with a nil buffer to leverage the internal swap buffer.
- **Non-Intrusive Integration:** Compatible with `net.Listener` and `net.Conn` (supports `syscall.RawConn`), enabling seamless integration into existing applications.
- **Efficient Context Switching:** Minimizes context switching overhead for small messages, ideal for high-frequency message exchanges.
- **Customizable Delegation:** Applications can control when to delegate `net.Conn` to gaio, such as after handshakes or specific `net.TCPConn` configurations.
- **Back-Pressure Handling:** Applications can control read/write request submission timing, enabling per-connection back-pressure management to throttle sending when necessary—particularly useful when transferring data from a faster source (A) to a slower destination (B).
- **Lightweight and Maintainable:** Approximately 1,000 lines of code, facilitating easy debugging and maintenance.
- **Cross-Platform Support:** Compatible with Linux and BSD.

## Conventions

- **Connection Delegation:** Once you submit an async read/write request for a `net.Conn` to `gaio.Watcher`, that connection becomes delegated to the watcher. Subsequent calls to `conn.Read` or `conn.Write` will return errors, but TCP properties set via `SetReadBuffer()`, `SetWriteBuffer()`, `SetLinger()`, `SetKeepAlive()`, and `SetNoDelay()` will be preserved.
  
- **Resource Management:** When you no longer need a connection, call `Watcher.Free(net.Conn)` to immediately close the socket and release resources. If you forget to call `Watcher.Free()`, the runtime garbage collector will clean up system resources when `net.Conn` is no longer referenced elsewhere. Similarly, failing to call `Watcher.Close()` will allow the garbage collector to clean up all related resources once the watcher is unreferenced.

- **Load Balancing:** For connection load balancing, create multiple `gaio.Watcher` instances and distribute `net.Conn` using your preferred strategy. For acceptor load balancing, use `go-reuseport` as the listener.

- **Safe Read Requests:** When submitting read requests with a nil buffer, the `[]byte` slice returned from `Watcher.WaitIO()` remains valid until the next `Watcher.WaitIO()` call.

## TL;DR

```go
package main

import (
        "log"
        "net"

"github.com/xtaci/gaio"
)

// this goroutine waits for all I/O events and sends back everything it receives
// in an async manner
func echoServer(w *gaio.Watcher) {
        for {
                // loop wait for any IO events
                results, err := w.WaitIO()
                if err != nil {
                        log.Println(err)
                        return
                }

for _, res := range results {
                        switch res.Operation {
                        case gaio.OpRead: // read completion event
                                if res.Error == nil {
                                        // send back everything, we won't start to read again until write completes.
                                        // submit an async write request
                                        w.Write(nil, res.Conn, res.Buffer[:res.Size])
                                }
                        case gaio.OpWrite: // write completion event
                                if res.Error == nil {
                                        // since write has completed, let's start read on this conn again
                                        w.Read(nil, res.Conn, res.Buffer[:cap(res.Buffer)])
                                }
                        }
                }
        }
}

func main() {
        w, err := gaio.NewWatcher()
        if err != nil {
              log.Fatal(err)
        }
        defer w.Close()
	
        go echoServer(w)

ln, err := net.Listen("tcp", "localhost:0")
        if err != nil {
                log.Fatal(err)
        }
        log.Println("echo server listening on", ln.Addr())

for {
                conn, err := ln.Accept()
                if err != nil {
                        log.Println(err)
                        return
                }
                log.Println("new client", conn.RemoteAddr())

// submit the first async read IO request
                err = w.Read(nil, conn, make([]byte, 128))
                if err != nil {
                        log.Println(err)
                        return
                }
        }
}

```

### More examples

<details>
	<summary> Push server </summary>
        package main

```go
package main

import (
        "fmt"
        "log"
        "net"
        "time"

"github.com/xtaci/gaio"
)

func main() {
        // by simply replace net.Listen with reuseport.Listen, everything is the same as in push-server
        // ln, err := reuseport.Listen("tcp", "localhost:0")
        ln, err := net.Listen("tcp", "localhost:0")
        if err != nil {
                log.Fatal(err)
        }

log.Println("pushing server listening on", ln.Addr(), ", use telnet to receive push")

// create a watcher
        w, err := gaio.NewWatcher()
        if err != nil {
                log.Fatal(err)
        }

// channel
        ticker := time.NewTicker(time.Second)
        chConn := make(chan net.Conn)
        chIO := make(chan gaio.OpResult)

// watcher.WaitIO goroutine
        go func() {
                for {
                        results, err := w.WaitIO()
                        if err != nil {
                                log.Println(err)
                                return
                        }

for _, res := range results {
                                chIO <- res
                        }
                }
        }()

// main logic loop, like your program core loop.
        go func() {
                var conns []net.Conn
                for {
                        select {
                        case res := <-chIO: // receive IO events from watcher
                                if res.Error != nil {
                                        continue
                                }
                                conns = append(conns, res.Conn)
                        case t := <-ticker.C: // receive ticker events
                                push := []byte(fmt.Sprintf("%s\n", t))
                                // all conn will receive the same 'push' content
                                for _, conn := range conns {
                                        w.Write(nil, conn, push)
                                }
                                conns = nil
                        case conn := <-chConn: // receive new connection events
                                conns = append(conns, conn)
                        }
                }
        }()

// this loop keeps on accepting connections and send to main loop
        for {
                conn, err := ln.Accept()
                if err != nil {
                        log.Println(err)
                        return
                }
                chConn <- conn
        }
}

```
</details>

## Documentation

For complete documentation, see the associated [Godoc](https://godoc.org/github.com/xtaci/gaio).

## Benchmarks

| Test Case | Throughput test with 64KB buffer |
|:-------------:|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Description | A client continuously sends 64KB of data to the server. The server reads the data and echoes it back. The client continues receiving until all bytes are successfully received. |
| Command | `go test -v -run=^$ -bench Echo` |
| Macbook Pro | 1695.27 MB/s 518 B/op 4 allocs/op|
| Linux AMD64 | 1883.23 MB/s 518 B/op 4 allocs/op|
| Raspberry Pi4 | 354.59 MB/s 334 B/op 4 allocs/op|

| Test Case | 8K concurrent connection echo test |
|:-------------:|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|Description| Starts 8192 clients, each sending 1KB of data to the server. The server reads and echoes back the data, and each client continues receiving until all bytes are successfully received.|
| Command | `go test -v -run=8k` |
| Macbook Pro | 1.09s |
| Linux AMD64 | 0.94s |
| Raspberry Pi4 | 2.09s |

## Testing Directives
On macOS, you need to increase the maximum open files limit to run the benchmarks.

```bash
sysctl -w kern.ipc.somaxconn=4096
sysctl -w kern.maxfiles=100000
sysctl -w kern.maxfilesperproc=100000
sysctl -w net.inet.ip.portrange.first=1024
sysctl -w net.inet.ip.portrange.last=65535

ulimit -S -n 65536
```

### Regression

![regression](https://raw.githubusercontent.com/xtaci/gaio/master/assets/regression.png)

X -> number of concurrent connections, Y -> time of completion in seconds

```
Best-fit values	 
Slope	8.613e-005 ± 5.272e-006
Y-intercept	0.08278 ± 0.03998
X-intercept	-961.1
1/Slope	11610
 
95% Confidence Intervals	 
Slope	7.150e-005 to 0.0001008
Y-intercept	-0.02820 to 0.1938
X-intercept	-2642 to 287.1
 
Goodness of Fit	 
R square	0.9852
Sy.x	0.05421
 
Is slope significantly non-zero?	 
F	266.9
DFn,DFd	1,4
P Value	< 0.0001
Deviation from horizontal?	Significant
 
Data	 
Number of XY pairs	6
Equation	Y = 8.613e-005*X + 0.08278
```

## FAQ
1. If you encounter an error like:

```
# github.com/xtaci/gaio [github.com/xtaci/gaio.test]
./aio_linux.go:155:7: undefined: setAffinity
./watcher.go:588:4: undefined: setAffinity
FAIL	github.com/xtaci/gaio [build failed]
FAIL
```

ensure that gcc/clang is installed.

## License

The `gaio` source code is available under the MIT [License](/LICENSE).

## References

* https://zhuanlan.zhihu.com/p/102890337 -- gaio小记
* https://github.com/xtaci/grasshopper -- A secure chained relayer for UDP made of gaio
* https://github.com/golang/go/issues/15735 -- net: add mechanism to wait for readability on a TCPConn
* https://en.wikipedia.org/wiki/C10k_problem -- C10K
* https://golang.org/src/runtime/netpoll_epoll.go -- epoll in golang 
* https://www.freebsd.org/cgi/man.cgi?query=kqueue&sektion=2 -- kqueue
* https://idea.popcount.org/2017-02-20-epoll-is-fundamentally-broken-12/ -- epoll is fundamentally broken
* https://en.wikipedia.org/wiki/Transmission_Control_Protocol#Flow_control -- TCP Flow Control 
* http://www.idc-online.com/technical_references/pdfs/data_communications/Congestion_Control.pdf -- Back-pressure

gaio

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README