torch_random_fields

KO | EN

gitlite — search

#javascript #python #ai #llm #golang #go #linux #java #windows #awesome #kubernetes #pytorch #security #openai

torch_random_fields

(★ 18)

A highly optimized library for building markov random fields with pytorch.

torch_random_fields Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

`torch_random_fields` is a library for building markov random fields (MRF) with complex topology \[1\] \[2\] with `pytorch`, it is optimized for batch training on GPU.

The key features include:

- Easy to **plug** into your research code
- Support for **batch** acceleration of any random field with arbitary **binary** or **ternary** connections on the **GPU**
- Fast training/inference with **top-K** logits, do not worry about too large label space
- Support for **context-aware** transition matrix and **low-rank** factorization

You may cite this project by:

```
@inproceedings{
  wang2022regularized,
  title={Regularized Molecular Conformation Fields},
  author={Lihao Wang and Yi Zhou and Yiqun Wang and Xiaoqing Zheng and Xuanjing Huang and Hao Zhou},
  booktitle={Advances in Neural Information Processing Systems},
  editor={Alice H. Oh and Alekh Agarwal and Danielle Belgrave and Kyunghyun Cho},
  year={2022},
  url={https://openreview.net/forum?id=7XCFxnG8nGS}
}
```

## Cases

### Linear-Chain CRF

Check out the [tutorial](tests/test_linear_chain_crf.ipynb).

The well known linear-chain CRF which is heavily adopted in sequence labeling (POS-tagging, chunking, NER, etc.) is supported.

### Top-K Skip-Chain CRF

Check out the [tutorial](tests/test_skip_chain_crf.ipynb).

In `torch_random_fields`, any random field with arbitary topology is supported. To be more precise, we require binary connections, although in some case ternary connections are also supported (yes, I am lazy).

Here we show a case of Dynamic Skip-Chain CRF, where:

- Some nodes (e.g., two nodes with the same words) are connected, which looks **skipping** the linear connection \[3\]
- Only the **top-3** labels for each node are kept, greatly speeding up training and inference \[4\]

### Ising/Potts Model

Ising model (or Potts model) is widely used in statistical physics and computational biology \[5\]. In this case, the random variables form a grid, but it can be fully connected.

## Features

### Learning

- **Linear-Chain CRF**: 
  - [x] maximum likelihood estimation
  - [x] structured perceptron
  - [x] piecewise training
  - [x] pseudo-likelihood
- **General CRF**: 
  - [x] structured perceptron
  - [x] piecewise training
  - [x] pseudo-likelihood

### Inference

- **Linear-Chain CRF**: 
  - [x] viterbi decoding
  - [x] **batch** loopy belief propagation
  - [x] **batch** mean field variational inference

- **General CRF**: 
  - [x] **batch** loopy belief propagation
  - [x] naive mean field variational inference
  - [ ] **batch** naive mean field inference

### Sampling

- [x] Gibbs Sampling

## Acknowledgement

Some implementation borrows from these great projects with modifications:

- linear-chain crf: [pytorch-crf](https://github.com/kmkurn/pytorch-crf), [fairseq](https://github.com/facebookresearch/fairseq)
- loopy belief propagation: [pystruct](https://github.com/pystruct/pystruct)

## Reference

\[1\] [An Introduction to Conditional Random Fields](https://arxiv.org/abs/1011.4088) (Sutton and McCallum, 2010)

\[2\] [Graphical Models, Exponential Families, and Variational Inference](https://people.eecs.berkeley.edu/~wainwrig/Papers/WaiJor08_FTML.pdf) (Wainwright and Jordan, 2008)

\[3\] [A Skip-Chain Conditional Random Field for Ranking Meeting Utterances by Importance](https://aclanthology.org/W06-1643/) (Galley, 2006)

\[4\] [Fast Structured Decoding for Sequence Models](http://papers.neurips.cc/paper/8566-fast-structured-decoding-for-sequence-models.pdf) (Sun, 2020)

\[5\] [Improved contact prediction in proteins: Using pseudolikelihoods to infer Potts models](https://arxiv.org/abs/1211.1281) (Ekeberg, 2013)

torch_random_fields

torch_random_fields Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

torch_random_fields

torch_random_fields Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README