shared-tensor

KO | EN

gitlite — search

#python #react #ai #typescript #llm #golang #android #machine-learning #rust #deep-learning #cli #kubernetes #pytorch #openai

(★ 35)

A distributed, shared tensor with high performance approximate updates for machine learning

shared-tensor 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# shared-tensor
A distributed, shared tensor with high performance approximate updates for machine learning

## Example use:

require 'sharedtensor'
    
    original_tensor = torch.Tensor(4,5,6,2)

-- Tries to connect to a shared tensor.  If the connection
    -- succeeds, use that, else start a new shared tensor with
    -- values from "original_tensor".
    z = sharedtensor.createOrFetch("192.168.0.1", 9925, original_tensor)

while true do
      z:copyToTensor(current_values)
      new_values = ...  -- do learning on this...
      z:addFromTensor(new_values.csub(curent_values));
    end

## Usage notes:

The transfer uses compression which assumes latency for approximate updates is most important.  It also assumes the magnitude of all values is similar.  The synchronisation is fully ascynchronous, and while it will always eventually converge over time, values may overshoot temporaraly.

All machines must be able to connect to one another.  They will form a tree formation internally.  Nodes will pass around other nodes IP addresses, so all must be on an IPv4 network with no NAT or port mappings etc.  Multiple tensors must use different ports.

## TODO:

* Way to limit network bandwidth based on fixed bitrate or fixed quality setting.  Currently simply fills all bandwidth.

* Nicer handling of disconnections and errors in general.  Reconnection logic would be cool.

* Tree "design" to allow for variable latency links or network topologies where some systems can't see others.

* More complete examples, including `char-rnn`

* Diagrams in documentation.

* Support "table sync" so that a table of tensors all get synched, but tensors can have different magnitudes

* Try different compression methods in the real world

* build AWS image/integration.

* Do the actual delta compression in a cuda kernel.  Should save *lots* of CPU.

## Want to help out?

Pull requests accepted!

shared-tensor

shared-tensor 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

shared-tensor

shared-tensor 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README