hadoop-tsdb-connector

KO | EN

gitlite — search

#python #llm #golang #android #deep-learning #docker #java #awesome #awesome-list #pytorch #ios #mcp #openai #open-source

hadoop-tsdb-connector

(★ 15)

No description available.

hadoop-tsdb-connector Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

The aim of the  TSDB Connector for Hadoop project is

Developing a solution to push metrics to TSDB which is highly reliable and a built-in hadoop solution.

Every machine in hadoop cluster has a set of metrics associated with it which convey the state of the Hadoop process / system at that specific time. Currently there are external agents which are used to push these "metrics" to the database (TSDB ) .

We propose a solution to push these metrics from within the hadoop process itself providing more reliability inc ase the external agent stops working and

# Compatible Versions:-
This system was tested with Hadoop versions :-
1. Hadoop 1.2.1 
2. Hadoop 2.4.1

and OpenTSDB 2.0

The 2 files here are  TSDBSink-1.2.1.java and TSDBSink-2.4.1.java for Hadoop 1.2.1 and Hadoop 2.4.1 Version respectively

# Instructions :-

To use this Sink you need to place the files in the following folder

For Hadoop 1.2.1
<hadoop-1.2.1 source folder>/src/core/org/apache/hadoop/metrics2/sink/TSDBSink.java

For Hadoop 2.4.1
<hadoop-2.4.1 source folder >/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/sink/TSDBSink.java

and compile hadoop.

Change the hadoop-metrics2.properties file and add the following in the sink

*.sink.tsdb.class=org.apache.hadoop.metrics2.sink.TSDBSink

*.period=10

namenode.sink.tsdb.servers="TSDB Server hostname":4242

datanode.sink.tsdb.servers="TSDB Server hostname":4242

# Limitations:-

There are some limitations to this project. In case the TSDB server is restarted , the metrics are stopped .

# Roadmap:-

1. We are currently working on adding code to reset the connection in such a case upto a certain number of retires

2. We are also adding the option for resetting connection in the hadoop dfsadmin utility.

hadoop-tsdb-connector

hadoop-tsdb-connector Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

hadoop-tsdb-connector

hadoop-tsdb-connector Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README