s3sync

KO | EN

gitlite — search

#javascript #hacktoberfest #react #typescript #llm #android #machine-learning #deep-learning #nodejs #awesome #kubernetes #vue #ios #open-source

(★ 86)

♻️ Golang utility for syncing between s3 and local, similar to `aws s3 sync`

File Explorer

Download Latest Version (.zip)

- - ci.yml
- - example.go
- - example.go
- - example.go
.editorconfig
.gitignore
codecov.yml
docker-compose.e2e.yml
error.go
error_test.go
go.mod
go.sum
LICENSE
logger.go
Makefile
MIGRATION.md
option.go
option_test.go
README.md
renovate.json
s3path.go
s3path_test.go
s3sync.go
s3sync_test.go
util_test.go

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# s3sync

![CI](https://github.com/seqsense/s3sync/workflows/CI/badge.svg)
[![codecov](https://codecov.io/gh/seqsense/s3sync/branch/master/graph/badge.svg)](https://codecov.io/gh/seqsense/s3sync)

> Golang utility for syncing between s3 and local

## Migration guide

- [v2](MIGRATION.md#v2)

# Usage

Use `New` to create a manager, and `Sync` function syncs between s3 and local filesystem.

```go
import (
  "github.com/aws/aws-sdk-go-v2/config"
  "github.com/seqsense/s3sync/v2"
)

func main() {
  ctx := context.TODO()

// Load AWS config
  cfg, err := config.LoadDefaultConfig(ctx, config.WithRegion("ap-northeast-1"))
  if err != nil {
    panic(err)
  }

syncManager := s3sync.New(cfg)

// Sync from s3 to local
  syncManager.Sync(ctx, "s3://yourbucket/path/to/dir", "local/path/to/dir")

// Sync from local to s3
  syncManager.Sync(ctx, "local/path/to/dir", "s3://yourbucket/path/to/dir")

// Sync from s3 to s3
  syncManager.Sync(ctx, "s3://yourbucket/path/to/dir", "s3://anotherbucket/path/to/dir")
}
```

## Sets the custom logger

You can set your custom logger.

```go
import "github.com/seqsense/s3sync/v2"

...
s3sync.SetLogger(&CustomLogger{})
...
```

The logger needs to implement `Logf` methods. See the godoc for details.

## Sets up the parallelism

You can configure the number of parallel jobs for sync. Default is 16.

```
s3sync.New(cfg, s3sync.WithParallel(16)) // This is the same as default.
s3sync.New(cfg, s3sync.WithParallel(1)) // You can sync one by one.
```

# License

Apache 2.0 License. See [LICENSE](https://github.com/seqsense/s3sync/blob/master/LICENSE).

s3sync

File Explorer

Today's picks

Was this content helpful?

Recent Feedback

s3sync

File Explorer

Today's picks

Was this content helpful?

Recent Feedback

Download README