ulb-zeitungsprojekt-hp1

KO | EN

gitlite — search

#javascript #hacktoberfest #ai #llm #go #rust #deep-learning #linux #docker #java #vue #security #ios #openai

ulb-zeitungsprojekt-hp1

(★ 12)

Training data from "Hauptphase I" of project "Digitalisierung historischer deutscher Zeitungen"

ulb-zeitungsprojekt-hp1 Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Training Data Zeitungsdigitalisierung HP I

Data of the DFG-Project "Zeitungsdigitalisierung Hauptphase I" at University and States Library Saxony-Anhalt (2019-2021).

In the progress of the project, there were 550.000 newspaper pages ocr-ed, at first with standard `frk+deu` model config. Finally, after the training process, all pages again with the final training model.

OCR was done with Tesseract 4.x, training used a slightly adopted version of tesstrain.

## Training data

Training data pairs are located in `data` - subfolder. They  consist of more than 16.000 line images (tif-format) and corresponding textual groundtruth transcriptions.

*Attenzione*  
Downloading / cloning this repository might take some time depending on your network connection!

## Additional resources

Additional resources can be found inside the `resources` directory. Besides rudimentary `*.number` and `.punc` files it also contains the `*.wordlist` file that might be used final `*. traineddata`.

The wordlist contains more than 25.000 entries (double checked). Feel free to use it as extension for custom models related to german historical newspapers (1870-1945).

Further, it includes several Tesseract 4 unicharset-files, grabbed from <https://github.com/tesseract-ocr/langdata_lstm>.

## Training

### Installation Prerequisities

First of all, you need a tesseract installation, since tesstrain is like a middle-ware to run the actual training with tesseract's `lstmtrain` itself. Please install tesseract from `ppa:alex-p/tesseract-ocr` or compile yourself. The training at ULB used tesseract 4.x

Next, get a base model to start from.

* `frk`: <https://github.com/tesseract-ocr/tessdata_best/raw/master/frk.traineddata>
* `Fraktur`: <https://github.com/tesseract-ocr/tessdata_best/raw/master/script/Fraktur.traineddata>
* `GT4Hist`: <https://ub-backup.bib.uni-mannheim.de/~stweil/ocrd-train/data/Fraktur_5000000/Fraktur_5000000_0.466.traineddata>

Please note:  
_The language configuration files that can be installed from official ubuntu 18.04-Repository **do not** fit for training!_

### Run Training

*Please note*  
Base model configurations have to be placed in the tesseract `tessdata` configuration location, i.e. `/usr/share/tesseract-ocr/4.00/tessdata/` for tesseract 4.x.

The `TRAIN_DATA_PATH` must be an absolute local path. Feel free to use the training pairs from the `data`-folder or use your own.

Start training by executing `./train-local.sh`.

## License

<a rel="license" href="http://creativecommons.org/licenses/by/3.0/de/"><img alt="Creative Commons License" style="border-width:0" src="https://i.creativecommons.org/l/by/3.0/de/88x31.png" /></a><br />This work is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by/3.0/de/">Creative Commons Attribution 3.0 Germany License</a>.

ulb-zeitungsprojekt-hp1

ulb-zeitungsprojekt-hp1 Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

ulb-zeitungsprojekt-hp1

ulb-zeitungsprojekt-hp1 Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README