russtat

KO | EN

gitlite — search

#javascript #python #hacktoberfest #react #ai #typescript #llm #go #golang #android #machine-learning #rust #deep-learning #linux

★ 16 Open GitHub ↗

Russian statistical database EMISS python frontend

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

![](https://raw.githubusercontent.com/S0mbre/russtat/master/icons/main.ico)

# russtat: Python / PostgreSQL access to the Russian Federal statistics
*russtat* utilizes the power of Python to download and process the massive public data release by the Russian Statistics Office from the [EMISS website](https://fedstat.ru/). The original XML-formatted datasets (close to 7,000 in total) are parsed and saved as JSON files which can then be fed into a ready-made PostgreSQL database to give additional power. Alternativaly, you can utilize the JSON files with your own software, in any way you like!

## Download
*russtat* source code and documentation are hosted on [Github](https://github.com/S0mbre/russtat)

## Features:
* parallel non-blocking data retrieval and processing using the multiprocessing library
* fool-proof XML parsing with default values for missing data, string trimming, type conversions and exception handling
* ready SQL script to create the PostgreSQL database from scratch
* database routine handling
* extensive API documentation with Doxygen

## License and Copyrights
All source data are taken from the EMISS website at https://fedstat.ru/. No data is modified by this application.

The EMISS website publishes all materials under the [Creative Common Attribution 3.0 license](https://creativecommons.org/licenses/by/3.0/). The copyright owners are the Russian [Federal State Statistic Service](https://eng.gks.ru/) (ROSSTAT) and the Russian [Ministry of Digital Development, Communications and Mass Media](https://digital.gov.ru/en/) (MINKOMSV'AZ').

## Installation

### Requirements
You must have the following applications / packages installed in your system:

* Python 3.7+ (the app was written and tested with Python 3.8)
* Python packages: 
	- pip
	- psycopg2
* Git (should be pre-installed on most modern Linux and Mac systems, alternatively install from the [git website](https://git-scm.com/downloads))

#### 1. Clone repo

To get the latest version from Github, run:
  ```bash
  git clone https://github.com/S0mbre/russtat .
  ```
  
#### 2. Install the required packages

I recommend (as many do) installing packages into python's virtual environment using *virtualenv* or the inbuilt *venv*:
  
  Create a new virtual environment (assuming your projects root folder is 'myprojects'):
  
  **Linux / Mac**
  ```bash
  cd myprojects
  venv russtat
  cd russtat
  . ./bin/activate
  ```
  
  **Windows**
  ```bash
  cd myprojects
  venv russtat
  cd russtat
  scripts\activate.bat
  ```
  
  This step is, of course, optional. You can skip it if you don't want to use virtual environments for some reason or other. 
  
  Then just run:
  ```bash
  cd russtat
  python -m pip install -r requirements.txt
  ```
  
  If you're using a virtual environment, you can deactivate it after closing the app with `deactivate`.

## Usage
Run `python russtat.py` to start the application. Please modify the `main()` function is that file first to suit your purpose!

See the documentation in russtat/doc/ref to find out more!

russtat

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README