spotcheck

KO | EN

gitlite — search

#javascript #python #hacktoberfest #react #ai #typescript #llm #go #golang #android #machine-learning #rust #deep-learning #linux

★ 25 Open GitHub ↗

Quick, light weight querying of log files stored on AWS S3.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Spotcheck

Quick, light weight querying of log files stored on AWS S3.

- Downloads files from a S3 bucket for a single day.
- Concatenates all files into a single file.
- Unzips if necessary.
- Converts from a standard log format into JSON.

# Log file formats supported

- `s3`         - S3 website log
- `cloudfront` - Cloudfront log (File from each edge server)
- `cloudtrail` - Cloudtrail audit log

# Installation

npm install spotcheck -g

# Usage

# Create aws.json with your AWS API credentials.
    {
      "accessKeyId": "",
      "secretAccessKey": ""
    }

# Create report.json (see example below).

# Download data.
    spotcheck download report.json

Optional Parameters:
      --date   [date]      example: 04-08-2015

# Print counts from downloaded data.
    spotcheck report [path to json file]

Parameters (must use one):
      --format [format]    example: s3
      --field  [field]     example: user-agent

# Report format

Example `report.json`
````
{
  "Bucket": "",
  "Format": "s3",
  "Prefix": "",
  "Output": "report.log"
}
````

# More Robust Alternatives

Spotcheck is not intended to be a robust log parsing solution. It is designed
more to make one off requests and queries easy, while leaving the heavy lifting
up to more robust and full feature tools like: Loggly / Splunk / Hadoop / etc.

spotcheck

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

spotcheck

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README