chatbot-ui-llama.cpp

KO | EN

gitlite — search

#javascript #python #hacktoberfest #react #ai #typescript #llm #go #golang #android #machine-learning #rust #deep-learning #linux

chatbot-ui-llama.cpp

★ 18 Open GitHub ↗

A static web ui for llama.cpp server. The llama.cpp chat interface for everyone. base on chatbot-ui

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Chatbot UI

## Chatbot UI for llama.cpp server

### How to use

`step1` get the web

download from [release](https://github.com/yportne13/chatbot-ui-llama.cpp/releases) page and unzip it

clone this repository, and then:

```
npm install
npm run build
```

and you will get a `out/` dir.

`step2` get llama.cpp server

if you are windows user, go to https://github.com/ggerganov/llama.cpp/releases to download a server.

or if you are not windows user, go to https://github.com/ggerganov/llama.cpp, clone, and `make`.

`step3` run

for windows

```
 .\server.exe --model /PATH_TO_MODEL --path THE_OUT_DIR_YOU_JUST_BUILD
```

or else

```
 ./server --model /PATH_TO_MODEL --path THE_OUT_DIR_YOU_JUST_BUILD
```

and go to your browser: http://localhost:8080

### Config

change the language: in next-i18next.config.js, change defaultLocale

add more model: types/openai.ts

## below are the origin chatbot-ui readme

## News

Chatbot UI 2.0 will launch on Monday January 8th, 2024.

See a [preview](https://x.com/mckaywrigley/status/1738273242283151777?s=20).

This repo will be completely overhauled with the updated codebase.

The old codebase will be available in a new repo.

There will be several hours of downtime on ChatbotUI.com while the new version is deployed.

We recommend exporting your data before the update!

## About

Chatbot UI is an open source chat UI for AI models.

See a [demo](https://twitter.com/mckaywrigley/status/1640380021423603713?s=46&t=AowqkodyK6B4JccSOxSPew).

![Chatbot UI](https://raw.githubusercontent.com/yportne13/chatbot-ui-llama.cpp/main/public/screenshots/screenshot-0402023.jpg)

## Updates

Chatbot UI will be updated over time.

Expect frequent improvements.

**Next up:**

- [ ] Sharing
- [ ] "Bots"

## Deploy

**Vercel**

Host your own live version of Chatbot UI with Vercel.

[![Deploy with Vercel](https://vercel.com/button)](https://vercel.com/new/clone?repository-url=https%3A%2F%2Fgithub.com%2Fmckaywrigley%2Fchatbot-ui)

**Docker**

Build locally:

```shell
docker build -t chatgpt-ui .
docker run -e OPENAI_API_KEY=xxxxxxxx -p 3000:3000 chatgpt-ui
```

Pull from ghcr:

```
docker run -e OPENAI_API_KEY=xxxxxxxx -p 3000:3000 ghcr.io/mckaywrigley/chatbot-ui:main
```

## Running Locally

**1. Clone Repo**

```bash
git clone https://github.com/mckaywrigley/chatbot-ui.git
```

**2. Install Dependencies**

```bash
npm i
```

**3. Provide OpenAI API Key**

Create a .env.local file in the root of the repo with your OpenAI API Key:

```bash
OPENAI_API_KEY=YOUR_KEY
```

> You can set `OPENAI_API_HOST` where access to the official OpenAI host is restricted or unavailable, allowing users to configure an alternative host for their specific needs.

> Additionally, if you have multiple OpenAI Organizations, you can set `OPENAI_ORGANIZATION` to specify one.

**4. Run App**

```bash
npm run dev
```

**5. Use It**

You should be able to start chatting.

## Configuration

When deploying the application, the following environment variables can be set:

| Environment Variable              | Default value                  | Description                                                                                                                               |
| --------------------------------- | ------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------- |
| OPENAI_API_KEY                    |                                | The default API key used for authentication with OpenAI                                                                                   |
| OPENAI_API_HOST                   | `https://api.openai.com`       | The base url, for Azure use `https://<endpoint>.openai.azure.com`                                                                         |
| OPENAI_API_TYPE                   | `openai`                       | The API type, options are `openai` or `azure`                                                                                             |
| OPENAI_API_VERSION                | `2023-03-15-preview`           | Only applicable for Azure OpenAI                                                                                                          |
| AZURE_DEPLOYMENT_ID               |                                | Needed when Azure OpenAI, Ref [Azure OpenAI API](https://learn.microsoft.com/zh-cn/azure/cognitive-services/openai/reference#completions) |
| OPENAI_ORGANIZATION               |                                | Your OpenAI organization ID                                                                                                               |
| DEFAULT_MODEL                     | `gpt-3.5-turbo`                | The default model to use on new conversations, for Azure use `gpt-35-turbo`                                                               |
| NEXT_PUBLIC_DEFAULT_SYSTEM_PROMPT | [see here](utils/app/const.ts) | The default system prompt to use on new conversations                                                                                     |
| NEXT_PUBLIC_DEFAULT_TEMPERATURE   | 1                              | The default temperature to use on new conversations                                                                                       |
| GOOGLE_API_KEY                    |                                | See [Custom Search JSON API documentation][GCSE]                                                                                          |
| GOOGLE_CSE_ID                     |                                | See [Custom Search JSON API documentation][GCSE]                                                                                          |

If you do not provide an OpenAI API key with `OPENAI_API_KEY`, users will have to provide their own key.

If you don't have an OpenAI API key, you can get one [here](https://platform.openai.com/account/api-keys).

## Contact

If you have any questions, feel free to reach out to Mckay on [Twitter](https://twitter.com/mckaywrigley).

[GCSE]: https://developers.google.com/custom-search/v1/overview

chatbot-ui-llama.cpp

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README