tcpdp

KO | EN

gitlite — search

#javascript #python #react #ai #llm #golang #go #macos #java #windows #cli #mcp #ios #open-source

(★ 127)

tcpdp is TCP dump tool with custom dumper and structured logger written in Go.

tcpdp Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# tcpdp [![build](https://github.com/k1LoW/tcpdp/workflows/build/badge.svg)](https://github.com/k1LoW/tcpdp/actions) [![GitHub release](https://img.shields.io/github/release/k1LoW/tcpdp.svg)](https://github.com/k1LoW/tcpdp/releases) ![Coverage](https://raw.githubusercontent.com/k1LoW/octocovs/main/badges/k1LoW/tcpdp/coverage.svg) ![Code to Test Ratio](https://raw.githubusercontent.com/k1LoW/octocovs/main/badges/k1LoW/tcpdp/ratio.svg) ![Test Execution Time](https://raw.githubusercontent.com/k1LoW/octocovs/main/badges/k1LoW/tcpdp/time.svg)

tcpdp is TCP dump tool with custom dumper and structured logger written in Go.

`tcpdp` has 3 modes:

- TCP **Proxy** server mode
- **Probe** mode ( using libpcap )
- **Read** pcap file mode

## Usage

### `tcpdp proxy` : TCP proxy server mode

``` console
$ tcpdp proxy -l localhost:12345 -r localhost:1234 -d hex # hex.Dump()
```

``` console
$ tcpdp proxy -l localhost:55432 -r db.internal.example.com:5432 -d pg # Dump query of PostgreSQL
```

``` console
$ tcpdp proxy -l localhost:33306 -r db.example.com:3306 -d mysql # Dump query of MySQL
```

#### With server-starter

https://github.com/lestrrat-go/server-starter

``` console
$ start_server --port 33306 -- tcpdp proxy -s -r db.example.com:3306 -d mysql
```

#### With config file

``` console
$ tcpdp proxy -c config.toml
```

### `tcpdp probe` : Probe mode (like tcpdump)

``` console
$ tcpdp probe -i lo0 -t localhost:3306 -d mysql # is almost the same setting as 'tcpdump -i lo0 host 127.0.0.1 and tcp port 3306'
```

``` console
$ tcpdp probe -i eth0 -t 3306 -d hex # is almost the same setting as 'tcpdump -i eth0 tcp port 3306'
```

### `tcpdp read` : Read pcap file mode

``` console
$ tcpdump -i eth0 host 127.0.0.1 and tcp port 3306 -w mysql.pcap
$ tcpdp read mysql.pcap -d mysql -t 3306 -f ltsv
```

### `tcpdp config` Create config

``` console
$ tcpdp config > myconfig.toml
```

#### Show current config

``` console
$ tcpdp config
```

#### config format

``` toml
[tcpdp]
pidfile = "/var/run/tcpdp.pid"
dumper = "mysql"

[probe]
target = "db.example.com:3306"
interface = "en0"
bufferSize = "2MB"
immediateMode = false
snapshotLength = "auto"
internalBufferLength = 10000
filter = ""

[proxy]
useServerStarter = false
listenAddr = "localhost:3306"
remoteAddr = "db.example.com:3306"

[log]
dir = "/var/log/tcpdp"
enable = true
enableInternal = true
stdout = true
format = "ltsv"
rotateEnable = true
rotationTime = "daily"
rotationCount = 7
# You can execute arbitrary commands after rotate
# $1 = prev filename
# $2 = current filename
rotationHook = "/path/to/after_rotate.sh"
fileName = "tcpdp.log"

[dumpLog]
dir = "/var/log/dump"
enable = true
stdout = false
format = "json"
rotateEnable = true
rotationTime = "hourly"
rotationCount = 24
fileName = "dump.log"
```

## Installation

```console
$ go get github.com/k1LoW/tcpdp
```

## Architecture

### tcpdp proxy connection diagram

```
      client_addr
           ^
           |        tcpdp
+----------|---------------+
|          v               |
|  proxy_listen_addr       |
|         + ^              |
|         | |   +--------+ |
|         |<----+ dumper | |
|         | |<--+        | |
|         | |   +--------+ |
|         v +              |
|  proxy_client_addr       |
|          ^               |
+----------|---------------+
           |
           v
      remote_addr
```

### tcpdp probe connection diagram

```
                    server
+--------------------------+
|                          |
|                      +---+---+
|       <--------------| eth0  |----------->
|            interface +---+---+
|            /target     ^ |
|                        | |
|         tcpdp          | |
|        +--------+      | |
|        | dumper +------+ |
|        +--------+        |
+--------------------------+
```

### tcpdp read diagram

```
                  tcpdp
+--------+ STDIN +--------+ STDOUT
| *.pcap +------>+ dumper +-------->
+--------+       +--------+
```

## tcpdp.log ( `tcpdp proxy` or `tcpdp probe` )

| key | description | mode |
| --- | ----------- | ---- |
| ts | timestamp | proxy / probe / read |
| level | log level | proxy / probe |
| msg | log message | proxy / probe |
| error | error info | proxy / probe |
| caller | error caller | proxy / probe |
| conn_id | TCP connection ID by tcpdp | proxy / probe |
| target | probe target | proxy / probe |
| dumper | dumper type | proxy / probe |
| use_server_starter | use server_starter | proxy |
| conn_seq_num | TCP comunication sequence number by tcpdp | proxy |
| client_addr | client address | tcpdp.log, hex, mysql, pg | proxy |
| remote_addr | remote address | proxy |
| proxy_listen_addr | listen address| proxy |
| direction | client to remote: `->` / remote to client: `<-` | proxy |
| interface | probe target interface | probe |
| mtu | interface MTU (Maximum Transmission Unit) | probe |
| mss | TCP connection MSS (Max Segment Size) | probe |
| probe_target_addr | probe target address | probe |
| filter | BPF (Berkeley Packet Filter) | probe |
| buffer_size | libpcap buffer_size | probe |
| immediate_mode | libpcap immediate_mode | probe |
| snapshot_length | libpcap snapshot length | probe |
| internal_buffer_length | tcpdp internal packet buffer length | probe |

## Dumper

### mysql

MySQL query dumper

**NOTICE: MySQL query dumper require `--target` option when `tcpdp proxy` `tcpdp probe`**

| key | description | mode |
| --- | ----------- | ---- |
| ts | timestamp | proxy / probe / read |
| conn_id | TCP connection ID by tcpdp | proxy / probe / read |
| conn_seq_num | TCP comunication sequence number by tcpdp | proxy |
| client_addr | client address | proxy |
| proxy_listen_addr | listen address| proxy |
| proxy_client_addr | proxy client address | proxy |
| remote_addr | remote address | proxy |
| direction | client to remote: `->` / remote to client: `<-` | proxy |
| interface | probe target interface | probe |
| src_addr | src address | probe / read |
| dst_addr | dst address | probe / read |
| probe_target_addr | probe target address | probe |
| proxy_protocol_src_addr | proxy protocol src address | probe / proxy /read |
| proxy_protocol_dst_addr | proxy protocol dst address | probe / proxy /read |
| query | SQL query | proxy / probe / read |
| stmt_id | statement id | proxy / probe / read |
| stmt_prepare_query | prepared statement query | proxy / probe / read |
| stmt_execute_values | prepared statement execute values | proxy / probe / read |
| character_set | [character set](https://dev.mysql.com/doc/internals/en/character-set.html) | proxy / probe / read |
| username | username | proxy / probe / read |
| database | database | proxy / probe / read |
| seq_num | sequence number by MySQL | proxy / probe / read |
| command_id | [command_id](https://dev.mysql.com/doc/internals/en/com-query.html) for MySQL | proxy / probe / read |

### pg

PostgreSQL query dumper

**NOTICE: PostgreSQL query dumper require `--target` option `tcpdp proxy` `tcpdp probe`**

| key | description | mode |
| --- | ----------- | ---- |
| ts | timestamp | proxy / probe / read |
| conn_id | TCP connection ID by tcpdp | proxy / probe / read |
| conn_seq_num | TCP comunication sequence number by tcpdp | proxy |
| client_addr | client address | proxy |
| proxy_listen_addr | listen address| proxy |
| proxy_client_addr | proxy client address | proxy |
| remote_addr | remote address | proxy |
| direction | client to remote: `->` / remote to client: `<-` | proxy |
| interface | probe target interface | probe |
| src_addr | src address | probe / read |
| dst_addr | dst address | probe / read |
| probe_target_addr | probe target address | probe |
| proxy_protocol_src_addr | proxy protocol src address | probe / proxy /read |
| proxy_protocol_dst_addr | proxy protocol dst address | probe / proxy /read |
| query | SQL query | proxy / probe / read |
| portal_name | portal Name | proxy / probe / read |
| stmt_name | prepared statement name | proxy / probe / read |
| parse_query | prepared statement query | proxy / probe / read |
| bind_values | prepared statement bind(execute) values | proxy / probe / read |
| username | username | proxy / probe / read |
| database | database | proxy / probe / read |
| message_type | [message type](https://www.postgresql.org/docs/current/static/protocol-overview.html#PROTOCOL-MESSAGE-CONCEPTS) for PostgreSQL | proxy / probe / read |

### hex

| key | description | mode |
| --- | ----------- | ---- |
| ts | timestamp | proxy / probe / read |
| conn_id | TCP connection ID by tcpdp | proxy / probe / read |
| conn_seq_num | TCP comunication sequence number by tcpdp | proxy |
| client_addr | client address | proxy |
| proxy_listen_addr | listen address| proxy |
| proxy_client_addr | proxy client address | proxy |
| remote_addr | remote address | proxy |
| direction | client to remote: `->` / remote to client: `<-` | proxy |
| interface | probe target interface | probe |
| src_addr | src address | probe / read |
| dst_addr | dst address | probe / read |
| probe_target_addr | probe target address | probe |
| proxy_protocol_src_addr | proxy protocol src address | probe / proxy /read |
| proxy_protocol_dst_addr | proxy protocol dst address | probe / proxy /read |
| bytes | bytes string by hex.Dump | proxy / probe / read |
| ascii | ascii string by hex.Dump | proxy / probe / read |

## References

- https://github.com/jpillora/go-tcp-proxy
- https://github.com/dmmlabo/tcpserver_go

tcpdp

tcpdp Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

tcpdp

tcpdp Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README