directory_stat_exporter

KO | EN

gitlite — search

#javascript #python #hacktoberfest #typescript #android #machine-learning #deep-learning #nodejs #macos #awesome #cli #pytorch #mcp #openai

directory_stat_exporter

(★ 8)

Directory Statistics Exporter for Prometheus

directory_stat_exporter Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# MOVED TO CODEBERG

## Migrating from GitHub to Codeberg

Many developers are shifting from GitHub to Codeberg, a European, community-driven platform. This move is largely fueled by a desire for greater data sovereignty and a commitment to open-source ethics.
Why Developers are Leaving GitHub
- Corporate Ownership: Concerns over Microsoft’s influence on data privacy and potential censorship.
- Legal Risks: Hosting with US-based companies subjects data to American jurisdiction, specifically laws like the Patriot Act and FISA, which allow broad government surveillance and data access.
- Performance: Increasing reports of GitHub becoming bloated or unreliable, particularly with CI/CD tools.
- AI Ethics: Friction regarding GitHub Copilot and the use of open-source code for training proprietary models.

## Why Choose Codeberg?
- Non-Profit Governance: Operated by a European non-profit dedicated to transparency, not shareholders.
- Legal Protections: Benefiting from stricter European data protection regulations.
- Seamless Transition: Offers a familiar Gitea-based interface that minimizes the learning curve.
- Community First: Focuses on serving the developer community rather than monetizing user data.

[https://codeberg.org/olibf/directory_stat_exporter](https://codeberg.org/olibf/directory_stat_exporter)

# directory statistics for prometheus
get simple directory statistics (metrics) for prometheus. if you want to know if a batch job did not process all the files in a directory or a certain folder contains files with error information, this is the right place. the directory_stat_exporter provides minimal metrics to see how many documents are in a folder and the timestamp (unix time in seconds) of the oldest one. with this information, you can decide, depending on the specs of an interface, if everything is ok or not. the analysis of a directory can be generated from only the directory itself, without the subdirectories, or also including all the subdirectories. depending on your needs.

the purpose of this exporter is to provide metrics so prometheus can generate an alert if there are files waiting longer than a certain time.

## features
- all done without prometheus libraries
  - don't know if this is a good decesion or a bad one.
  - that's not really a feature, is it. oh, well, it's a fun project anyway.
- super simple, the only things provided so far:
  - number of files in directory, with or without subdirectories
  - last modified timestamp of oldest file in directoy, with or without subdirs.
  - for calculation reasons the current timestamp

## exports
- `dirstat_files_in_dir`: number of files in directory
- `dirstat_oldest_file_time`: timestamp (unix time) of oldest file in dir
- `dirstat_current_timestamp`: the current timestamp. because it's not provieded by prometheus (or I was not able to find it.)

## todos
- test handling of unc paths in windows (yes, it's targeted for windows.)
- better logging (really? do I need logging here?)
- better error handling
  - it should be fault tolerant and rather give useful metrics if an error occurs
- make information gathering concurrent, so more directories can be handled in the same time
- add the disk usage of the current directory as a metric
  - think of usefulness first. do we really need this? -> so far: no.
- add performance measurements:
  - how fast is the gathering
  - where are the limits on local drives
  - what are the limits on remote directories 
- work out how to handle dependencies propperly.

## notes to self
- *important* stack items correctly (types and help text must only appear once in a metric export / per request)
- labels must not contain a single backslash... I replaced all backslashes now with forward slashes. -> there must be a better solution
  - e.g. add labels to the configuration and give them meaningful names.

## problems
- large directories might not be handled well
  - might use lot of memory, because whole directory is read once (untested)

## configuration
### service port
`serviceport: "9999"`

this does not really need more explanation

### cache time
`cachetime: 5`

time of interval in minutes. must be an integer value.

this is the time interval which is used to poll the updates from the directories. it does not make sense to me to poll the directories every 15 seconds. if it is set to `0` then caching is disabled.

### directories
```
directories:
     - name: "tmp"
       path: /tmp
       recursive: true
```
|key|value|
|---|-----|
|path|a path to the directory to be monitored. this can be a relative or an absolute path. for windows it can also be a UNC path.|
|name|a useful name. must be unique. not sure if special characters and spaces are a good idea. I have to check that.|
|recursive|should subdirectories also be analysed. can be `true` or `false`|

## usage
in your GOPATH type
`go get gopkg.in/yaml.v2`

`go get github.com/codestoke/directory_stat_exporter`

you'll find the binary in the `bin` directory of your GOPATH. the `config.yml` must be in the same directory of the executable (there are no other options yet)

currently, this generates the version in the master branch. this might not be very useful and can be buggy.

you can always download the source from a release, unpack it in your GOPATH and then compile it with

`go build`

## TLS and basic authentication

directory_stat exporter supports TLS and basic authentication. This enables better
control of the various HTTP endpoints.

To use TLS and/or basic authentication, you need to pass a configuration file
using the `--web.config.file` parameter. The format of the file is described
[in the exporter-toolkit repository](https://github.com/prometheus/exporter-toolkit/blob/master/docs/web-configuration.md).

have fun!

cheers, Oli

directory_stat_exporter

directory_stat_exporter Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

directory_stat_exporter

directory_stat_exporter Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README