DataFinder

KO | EN

gitlite — search

#python #java #python3 #arduino #golang #machine-learning #rust #html #flask #javascript #seismology #nodejs

★ 12 Open GitHub ↗

Data management system

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

> [!IMPORTANT]  
> The software is no longer maintained and is provided only for reference.

# Introduction
The data management system DataFinder primarily targets the management of 
scientific technical data. Data can be attached with individual meta data that 
is based on a free-definable data model to achieve data structuring 
and ordering. Moreover, the system is able to handle large amounts of data and 
can be easily integrated in existing working environments. The system is based 
upon a client-server-architecture and uses open, stable standards.

The server side consists of the meta data server that stores the meta data 
as well as the system configuration. The server is accessed with the standardized
protocol WebDAV - an extension of the HTTP protocol - by the client side.
Data can be stored on the same server as the meta data or separated from it. 
The concept of separated meta data and data storage allows the flexible usage 
of heterogeneous storage resources. The access to these storage resources is 
achieved by using standardized data transfer interfaces. For example, FTP, File System, 
GridFTP, or WebDAV are currently supported.

# Infrastructure Installation
The DataFinder system consists of two clients one handling the data management related
and one handling the administrative tasks (e.g. data modeling, configuration of
storage resource, etc.). The minimum requirement concerning the server infrastructure is a 
WebDAV server which conforms to the WebDAV basic specification (RFC4918). The
WebDAV server stores configuration data, the logical data structure, and
optionally the managed data files. Examples of supported WebDAV servers are:
* Catacomb (open source)
* Apache + mod_dav (open source)
* Subversion

Currently, the best option is the Catacomb WebDAV server which implements additional
WebDAV specification (e.g. searching, access control) which in-turn 
are required for corresponding functionality of the DataFinder clients. For more
information on the Catacomb WebDAV server visit:
http://catacomb.tigris.org/

# Client Installation
## Source Distribution Installation
* Required Packages:
  * Python >= 2.6
  * Pyparsing >= 1.5.2
* GUI clients only
   * PyQt 3.18.1 (Qt 3.3.8, SIP 4.10)
   * PyQt 4.7.3 (Qt 4.6.2, SIP 4.10)
* Win32 only
   * PyWin32 214 (Win32 only) 
* Optional Packages (at least the WebDAV library or the Subversion package is required):
  * Python WebDAV Library >= 0.3.0
  * paramiko >= 1.7.4
  * nose 0.11.1
  * pysvn >=1.7.2
  * boto >= 2.0rc1
* Install dependencies.
  * Extract archive.
  * Change to the created directory.
  * Run to verify your environment: $> python setup.py test
    (Set --nosecommand to the complete path of the nose Python start script if not found; e.g., --nosecommand=/usr/local/bin/nosetests)
  * Generate additional files $> python setup.py gen
  * Run to install: $> python setup.py install

Script API distribution:
* Try the script examples in contrib/script_examples.

GUI client distribution:
* Start administrative client with 'python datafinder-admin-client.py'.
* Start data management client with 'python datafinder-client.py'.
 
## Binary Distribution Installation
* Extract archive.
* Change to the created directory.
* Start administrative client with 'datafinder-admin-client[.exe]'.
* Start data management client with 'datafinder-client[.exe]'.
 
## Binary Windows Distribution Installation
* Start the corresponding installer.
* Start clients with the help of the created symbolic links.

# Getting started
The DataFinder requires - as mentioned above - at least a WebDAV server conforming to the WebDAV base
specification RFC4918. After installation and starting of a corresponding 
WebDAV server please follow these steps:
* Start the administrative client.
* Cancel the login dialog and create a new configuration with 
  'File->Create Configuration...'.
* Chose 'File->Connect...' and connect to the created configuration (configuration URL and credentials are already inserted into the login dialog).
* The environment variable 'DF_START' can be set to the configuration URL.
* Start the DataFinder data management client.

More detailed information is available on: https://github.com/DLR-SC/DataFinder/wiki

DataFinder

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README