flutter_tiktoken

KO | EN

gitlite — search

#python #hacktoberfest #ai #typescript #go #rust #linux #nodejs #java #cli #kubernetes #vue #pytorch #openai

flutter_tiktoken

(★ 9)

flutter_tiktoken is a flutter offline package for a fast BPE tokeniser for OpenAI models.

flutter_tiktoken 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

## flutter_tiktoken

flutter_tiktoken is a flutter offline package for a fast BPE tokeniser for OpenAI models.

flutter_tiktoken is a copy package in [https://github.com/mvitlov/tiktoken](https://github.com/mvitlov/tiktoken) and fixed the [issue](https://github.com/mvitlov/tiktoken/issues/5) build spent many time (more than one hour) when build in release mode.

Open Ai Official Calcuate           |  flutter_tiktoken
:-------------------------:|:-------------------------:
![Snip20230621_3](https://github.com/JerryFans/flutter_tiktoken/assets/14149080/96a7153e-dfd5-40ee-a1cc-c9efc205a844) | ![Simulator Screenshot - iPhone 14 - 2023-06-21 at 18 43 10](https://github.com/JerryFans/flutter_tiktoken/assets/14149080/dcd5162f-92bf-49ab-b9e8-5364c3c5c7df)

## Usage

init flutter_tiktoken
```dart
void main() async {
  WidgetsFlutterBinding.ensureInitialized();
  await TiktokenDataProcessCenter().initData();
  runApp(const MyApp());
}
```

calcuateTokens
```dart
static int _calcuateTokens(
      {required String text, String modelName = 'gpt-4'}) {
    final encoding = encodingForModel(modelName);
    final numTokens = encoding.encode(text).length;
    return numTokens;
  }
```

flutter_tiktoken

flutter_tiktoken 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

flutter_tiktoken

flutter_tiktoken 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README