SearchGPT

KO | EN

gitlite — search

#hacktoberfest #react #llm #rust #nodejs #java #windows #awesome #kubernetes #awesome-list #vue #pytorch #mcp #open-source

(★ 168)

GPT Enhanced with Real-Time Web Browsing 🔗

File Explorer

Download Latest Version (.zip)

- - config.yaml
- fetch_web_content.py
- llm_answer.py
- locate_reference.py
- main.py
- retrieval.py
- serper_service.py
- web_crawler.py
.gitignore
LICENSE
README.md
README_CN.md
requirements.txt

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

SearchGPT: GPT with Real-Time Web Browsing
==========================================

Languages / 语言: [English](https://github.com/Wilson-ZheLin/SearchGPT) | [中文](https://github.com/Wilson-ZheLin/SearchGPT/blob/main/README_CN.md)

GPT's extensive knowledge base covers a wide range of topics but is limited to historical data. To address this, we've integrated web browsing into GPT, connecting its advanced language processing to the internet. 🔗

This upgrade enables access to the latest information, enhancing the model's relevance and efficiency in real-time data retrieval, and is faster than web-browsing on the GPT web page.

You may consider this as a tiny implementation of the `ChatGPT search` or `new Bing`, primarily for search engines and Q&A.

Please give me a star if you like it! 🌟

Features
--------
* Rapid real-time web search through [Serper (Google API)](https://serper.dev)

* Automatically adjusts web search and response **language** based on input

* **Multi-threading** to extract main content from web pages, reducing embedding costs

* Semantic search with [OpenAI Embedding](https://platform.openai.com/docs/guides/embeddings/what-are-embeddings) and [ChromaDB](https://www.trychroma.com)

* LLM responses generated with references and web sources

* Supports customization of [models](https://platform.openai.com/docs/models), **AI roles**, and **output formats**

* Matches each quoted sentence in the AI’s response to the **title, snippet, and link of the source website**

Architecture
------------

![Project Architecture](https://github.com/Wilson-ZheLin/GPT-4-Web-Browsing/assets/145169519/d313835d-5b57-4391-8b79-be81fdd4fa8a)

Demo
----

https://github.com/Wilson-ZheLin/GPT-4-Web-Browsing/assets/145169519/6406d20c-efec-4cb9-ad51-510f51ce5212

Getting Started
---------------

### Prerequisites

To run `main.py` or `llm_answer.py`, you'll need:
* Stable connection to OpenAI (may be instability in China)
* [Python 3.11.5](https://www.python.org/downloads/) (not necessarily the same)
* [Serper API Key](https://serper.dev)
    * Serper: 2,500 free queries (very adequate and fast)
* [OpenAI API Key](https://openai.com/blog/openai-api)
    * OpenAI: First $5 is free (GPT-3.5-turbo-16k  recommended)
    
### Installation
1. Install the required packages (consider adding `-i https://pypi.tuna.tsinghua.edu.cn/simple` in China):

```
pip install -r requirements.txt
```

2. Save your **API Keys** in `config.yaml`

3. run `main.py` or `llm_answer.py`, where you can change the query

License
-------

This project is licensed under the [MIT License](./LICENSE).

SearchGPT

File Explorer

Today's picks

Was this content helpful?

Recent Feedback

SearchGPT

File Explorer

Today's picks

Was this content helpful?

Recent Feedback

Download README