ReImproveJS - GitLite

KO | EN

gitlite — search

#javascript #python #ai #machine-learning #rust #deep-learning #docker #nodejs #java #awesome #kubernetes #awesome-list #pytorch #openai

ReImproveJS

(★ 122)

A framework using TensorFlow.js for Deep Reinforcement Learning

파일 탐색기

최종 버전 다운로드 (.zip)

- - abstractagent.md
  - academy.md
  - convolutionalneuralnetwork.md
  - datasource.md
  - dqagent.md
  - learningdatalogger.md
  - memory.md
  - model.md
  - neuralnetwork.md
  - qaction.md
  - qagent.md
  - qmatrix.md
  - qstate.md
  - qtransition.md
  - result.md
  - teacher.md
  - typedwindow.md
- - layertype.md
  - teachingstate.md
- - academyconfig.md
  - academystepinput.md
  - agentconfig.md
  - agenttrackinginformation.md
  - buildagentconfig.md
  - convolutionallayer.md
  - convolutionalnetworklayer.md
  - denselayer.md
  - dqagentconfig.md
  - dropoutlayer.md
  - flattenlayer.md
  - graphedge.md
  - graphnode.md
  - layer.md
  - layerconfig.md
  - maxpooling2dlayer.md
  - memento.md
  - mementotensor.md
  - memoryconfig.md
  - neuralnetworklayer.md
  - qactiondata.md
  - qagentconfig.md
  - qstatedata.md
  - teachertrackinginformation.md
  - teachingconfig.md
  - totflayerconfig.md
- globals.md
- README.md
- build-npm.sh
- - academy.ts
  - memory.ts
  - model.ts
  - networks.ts
  - teacher.ts
- reimprove.ts
- academy.spec.ts
- agent.spec.ts
- model.spec.ts
- network.spec.ts
- q.spec.ts
- reimprove.spec.ts
- teacher.spec.ts
.gitignore
.npmignore
.travis.yml
_config.yml
CHANGELOG.md
LICENSE
package-lock.json
package.json
README.md
tsconfig.json

# 설치 가이드

1. 코드 내려받기

git clone https://github.com/BeTomorrow/ReImproveJS

깃허브에서 프로젝트 코드 전체를 내 컴퓨터로 내려받습니다.

cd ReImproveJS

방금 내려받은 프로젝트 폴더 안으로 이동합니다.

2. Node.js

쉬움 추천

사전 준비물

Git GitHub에서 프로젝트 코드를 내려받으려면 필요합니다.
Node.js npm 명령어를 쓰려면 Node.js가 설치되어 있어야 합니다. LTS(장기지원) 버전을 추천합니다.

<script src="https://cdn.jsdelivr.net/npm/reimprovejs@0/dist/reimprove.js"></script>

이 명령어를 터미널에 그대로 입력해 실행하세요.

$ npm install reimprovejs

package.json에 명시된 라이브러리들을 내려받아 설치합니다.

✅ 명령어 실행 후 터미널에 나타나는 주소(보통 http://localhost:3000 형태)를 브라우저에서 열어보세요.

이 레포의 README에 적힌 실제 명령어를 그대로 가져왔습니다.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# This repository is deprecated. I cannot maintain it anymore, and cannot update issues.

# ReImproveJS

> A framework using TensorFlow.js for Deep Reinforcement Learning

[Documentation](docs/README.md) | [NPM](https://www.npmjs.com/package/reimprovejs) | [Wiki](https://github.com/Pravez/ReImproveJS/wiki) | [Changelog](CHANGELOG.md)

[![npm version](https://badge.fury.io/js/reimprovejs.svg)](https://badge.fury.io/js/reimprovejs)
[![Build Status](https://travis-ci.org/Pravez/ReImproveJS.svg?branch=master)](https://travis-ci.org/Pravez/ReImproveJS)

`ReImproveJS` is a little library to create Reinforcement Learning environments with Javascript.
It currently implements DQN algorithm, but aims to allow users to change easily algorithms, like for instance A3C or Sarsa.

The library is using TensorFlow.js as a computing background, enabling the use of WebGL to empower computations.

Getting started
------------------

Installation
------------

ReImproveJS is available as a standalone or as a NPM package.
As usual, you can use the CDN 
```html
<script src="https://cdn.jsdelivr.net/npm/reimprovejs@0/dist/reimprove.js"></script>
```

or if you have your local version

```html
<script src="/path/to/your/lib/root/reimprove.js"></script>
```
You can also install it through NPM.

```bash
$ npm install reimprovejs
```

Usage
-----------

With ReImproveJS, you have an environment organized as if your agents were part of a "school". The idea is that you are managing
an `Academy`, possessing `Teachers` and `Agents` (Students). You add `Teachers` and assign `Agents` to them. At each step of
your world, you just need to give the `Academy` each `Teacher`'s input, which will handle everything concerning learning.

Because you are in Reinforcement Learning, you need a Neural Network model in order for your agents to learn. TFJS's `Model` is
embedded into a wrapper, and you just need to precise what type of layers you need, and that's all !
For instance :

```javascript

const modelFitConfig = {              // Exactly the same idea here by using tfjs's model's
    epochs: 1,                        // fit config.
    stepsPerEpoch: 16
};

const numActions = 2;                 // The number of actions your agent can choose to do
const inputSize = 100;                // Inputs size (10x10 image for instance)
const temporalWindow = 1;             // The window of data which will be sent yo your agent
                                      // For instance the x previous inputs, and what actions the agent took

const totalInputSize = inputSize * temporalWindow + numActions * temporalWindow + inputSize;

const network = new ReImprove.NeuralNetwork();
network.InputShape = [totalInputSize];
network.addNeuralNetworkLayers([
    {type: 'dense', units: 32, activation: 'relu'},
    {type: 'dense', units: numActions, activation: 'softmax'}
]);
// Now we initialize our model, and start adding layers
const model = new ReImprove.Model.FromNetwork(network, modelFitConfig);

// Finally compile the model, we also exactly use tfjs's optimizers and loss functions
// (So feel free to choose one among tfjs's)
model.compile({loss: 'meanSquaredError', optimizer: 'sgd'})

```

Now that our model is ready, let's create an agent...

```javascript

// Every single field here is optionnal, and has a default value. Be careful, it may not
// fit your needs ...

const teacherConfig = {
    lessonsQuantity: 10,                   // Number of training lessons before only testing agent
    lessonsLength: 100,                    // The length of each lesson (in quantity of updates)
    lessonsWithRandom: 2,                  // How many random lessons before updating epsilon's value
    epsilon: 1,                            // Q-Learning values and so on ...
    epsilonDecay: 0.995,                   // (Random factor epsilon, decaying over time)
    epsilonMin: 0.05,
    gamma: 0.8                             // (Gamma = 1 : agent cares really much about future rewards)
};

const agentConfig = {
    model: model,                          // Our model corresponding to the agent
    agentConfig: {
        memorySize: 5000,                      // The size of the agent's memory (Q-Learning)
        batchSize: 128,                        // How many tensors will be given to the network when fit
        temporalWindow: temporalWindow         // The temporal window giving previous inputs & actions
    }
};

const academy = new ReImprove.Academy();    // First we need an academy to host everything
const teacher = academy.addTeacher(teacherConfig);
const agent = academy.addAgent(agentConfig);

academy.assignTeacherToAgent(agent, teacher);

```

And that's it ! Now you just need to update during your world emulation if the agent gets rewards, and
feed inputs to it.

```javascript
// Nice event occuring during world emulation
function OnSpecialGoodEvent() {
    academy.addRewardToAgent(agent, 1.0)        // Give a nice reward if the agent did something nice !
}

// Bad event
function OnSpecialBadEvent() {
    academy.addRewardToAgent(agent, -1.0)        // Give a bad reward to the agent if he did something wrong
}

// Animation loop, update loop, whatever loop you want
async function step(time) {
    
    let inputs = getInputs();          // Need to give a number[] of your inputs for one teacher.
    await academy.step([               // Let the magic operate ...
        {teacherName: teacher, agentsInput: inputs}
    ]);
    
}

// Start your loop (/!\ for your environment, not specific to ReImproveJS).
requestAnimationFrame(step);
```

Rewards are reset to 0 at each new step.

__Please be careful__ : Convolutional networks are implemented and operational as models, but currently not 
fully implemented and tested in the Reinforcement Learning, so please __do not use them__ for now.

Exemples
-----------------

Here an exemple made by [@RGBKnights](https://github.com/RGBKnights) : https://gist.github.com/RGBKnights/756b5f51465cc22d0ca39205979ad2a1