sparse-structured-attention

KO | EN

gitlite — search

#javascript #hacktoberfest #react #ai #typescript #rust #deep-learning #linux #docker #windows #kubernetes #awesome-list #ios #mcp

sparse-structured-attention

(★ 224)

Sparse and structured neural attention mechanisms

파일 탐색기

최종 버전 다운로드 (.zip)

- - __init__.py
  - _fused.pyx
  - _fused_jv.pyx
  - _isotonic.pyx
  - base.py
  - fused.py
  - isotonic.py
  - oscar.py
  - sparsemax.py
  - test_attention.py
  - test_fused.py
  - test_oscar.py
  - test_sparsemax.py
- MANIFEST.in
- setup.py
.gitignore
.travis.yml
fusedmax.png
LICENSE
README.md

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Sparse and structured attention mechanisms
[![Build Status](https://travis-ci.org/vene/sparse-structured-attention.svg?branch=master)](https://travis-ci.org/vene/sparse-structured-attention)
[![PyPI version](https://badge.fury.io/py/torchsparseattn.svg)](https://badge.fury.io/py/torchsparseattn)

--------------------------------------------------------------------------------

Efficient implementation of structured sparsity inducing
attention mechanisms: fusedmax, oscarmax and sparsemax.

**Note**: If you are just looking for sparsemax, I recommend the implementation in the [entmax](https://github.com/deep-spin/entmax).

Currently available for pytorch >= 0.4.1. (For older versions, use a previous
release of this package.) Requires python >= 2.7, cython, numpy, scipy.

Usage example:

```python

In [1]: import torch
In [2]: import torchsparseattn
In [3]: a = torch.tensor([1, 2.1, 1.9], dtype=torch.double)
In [4]: lengths = torch.tensor([3])
In [5]: fusedmax = torchsparseattn.Fusedmax(alpha=.1)
In [6]: fusedmax(a, lengths)
Out[6]: tensor([0.0000, 0.5000, 0.5000], dtype=torch.float64)
```

For details, check out our paper:

> Vlad Niculae and Mathieu Blondel
> A Regularized Framework for Sparse and Structured Neural Attention
> In: Proceedings of NIPS, 2017. 
> https://arxiv.org/abs/1705.07704

sparse-structured-attention

파일 탐색기

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

sparse-structured-attention

파일 탐색기

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README