txtai
π‘ μλ§¨ν± κ²μ, LLM μ€μΌμ€νΈλ μ΄μ λ° μΈμ΄ λͺ¨λΈ μν¬νλ‘μ°λ₯Ό μν μ¬μΈμ AI νλ μμν¬
νμΌ νμκΈ°
μ΅μ’ λ²μ λ€μ΄λ‘λ (.zip)- build.yml
- docs.yml
- minimal.yml
- Dockerfile
- api.py
- Dockerfile
- workflow.py
- Dockerfile
- Dockerfile
- Dockerfile
- Dockerfile
- configuration.md
- index.md
- methods.md
- cluster.md
- configuration.md
- customization.md
- index.md
- mcp.md
- methods.md
- openai.md
- security.md
- ann.md
- cloud.md
- database.md
- general.md
- graph.md
- index.md
- scoring.md
- vectors.md
- format.md
- index.md
- indexing.md
- methods.md
- query.md
- agent.excalidraw
- agent.png
- api-dark.png
- api.excalidraw
- api.png
- architecture-dark.png
- architecture.excalidraw
- architecture.png
- cloud-dark.png
- cloud.excalidraw
- cloud.png
- embeddings-dark.png
- embeddings.excalidraw
- embeddings.png
- examples-dark.png
- examples.excalidraw
- examples.png
- faq.excalidraw
- faq.png
- flows-dark.png
- flows.excalidraw
- flows.png
- format-dark.png
- format.excalidraw
- format.png
- further-dark.png
- further-ghdark.png
- further.excalidraw
- further.png
- indexing-dark.png
- indexing.excalidraw
- indexing.png
- install-dark.png
- install.excalidraw
- install.png
- llm.excalidraw
- llm.png
- logo.png
- models.excalidraw
- models.png
- pipeline-dark.png
- pipeline.excalidraw
- pipeline.png
- query-dark.png
- query.excalidraw
- query.png
- rag-dark.png
- rag.excalidraw
- rag.png
- schedule-dark.png
- schedule.excalidraw
- schedule.png
- search-dark.png
- search.excalidraw
- search.png
- task-dark.png
- task.excalidraw
- task.png
- why-dark.png
- why.excalidraw
- why.png
- workflow-dark.png
- workflow.excalidraw
- workflow.png
- main.html
- audiomixer.md
- audiostream.md
- microphone.md
- texttoaudio.md
- texttospeech.md
- transcription.md
- filetohtml.md
- htmltomd.md
- segmentation.md
- tabular.md
- textractor.md
- tokenizer.md
- urlretrieve.md
- caption.md
- imagehash.md
- objects.md
- llm.md
- rag.md
- entity.md
- labels.md
- reranker.md
- similarity.md
- summary.md
- translation.md
- hfonnx.md
- lemur.md
- mlonnx.md
- trainer.md
- index.md
- console.md
- export.md
- file.md
- image.md
- index.md
- retrieve.md
- service.md
- storage.md
- template.md
- url.md
- workflow.md
- index.md
- schedule.md
- cloud.md
- examples.md
- faq.md
- further.md
- index.md
- install.md
- models.md
- observability.md
- poweredby.md
- usecases.md
- why.md
- 01_Introducing_txtai.ipynb
- 02_Build_an_Embeddings_index_with_Hugging_Face_Datasets.ipynb
- 03_Build_an_Embeddings_index_from_a_data_source.ipynb
- 04_Add_semantic_search_to_Elasticsearch.ipynb
- 05_Extractive_QA_with_txtai.ipynb
- 06_Extractive_QA_with_Elasticsearch.ipynb
- 07_Apply_labels_with_zero_shot_classification.ipynb
- 08_API_Gallery.ipynb
- 09_Building_abstractive_text_summaries.ipynb
- 10_Extract_text_from_documents.ipynb
- 11_Transcribe_audio_to_text.ipynb
- 12_Translate_text_between_languages.ipynb
- 13_Similarity_search_with_images.ipynb
- 14_Run_pipeline_workflows.ipynb
- 15_Distributed_embeddings_cluster.ipynb
- 16_Train_a_text_labeler.ipynb
- 17_Train_without_labels.ipynb
- 18_Export_and_run_models_with_ONNX.ipynb
- 19_Train_a_QA_model.ipynb
- 20_Extractive_QA_to_build_structured_data.ipynb
- 21_Export_and_run_other_machine_learning_models.ipynb
- 22_Transform_tabular_data_with_composable_workflows.ipynb
- 23_Tensor_workflows.ipynb
- 24_Whats_new_in_txtai_4_0.ipynb
- 25_Generate_image_captions_and_detect_objects.ipynb
- 26_Entity_extraction_workflows.ipynb
- 27_Workflow_scheduling.ipynb
- 28_Push_notifications_with_workflows.ipynb
- 29_Anatomy_of_a_txtai_index.ipynb
- 30_Embeddings_SQL_custom_functions.ipynb
- 31_Near_duplicate_image_detection.ipynb
- 32_Model_explainability.ipynb
- 33_Query_translation.ipynb
- 34_Build_a_QA_database.ipynb
- 35_Pictures_are_worth_a_thousand_words.ipynb
- 36_Run_txtai_in_native_code.ipynb
- 37_Embeddings_index_components.ipynb
- 38_Introducing_the_Semantic_Graph.ipynb
- 39_Classic_Topic_Modeling_with_BM25.ipynb
- 40_Text_to_Speech_Generation.ipynb
- 41_Train_a_language_model_from_scratch.ipynb
- 42_Prompt_driven_search_with_LLMs.ipynb
- 43_Embeddings_in_the_Cloud.ipynb
- 44_Prompt_templates_and_task_chains.ipynb
- 45_Customize_your_own_embeddings_database.ipynb
- 46_Whats_new_in_txtai_6_0.ipynb
- 47_Building_an_efficient_sparse_keyword_index_in_Python.ipynb
- 48_Benefits_of_hybrid_search.ipynb
- 49_External_database_integration.ipynb
- 50_All_about_vector_quantization.ipynb
- 51_Custom_API_Endpoints.ipynb
- 52_Build_RAG_pipelines_with_txtai.ipynb
- 53_Integrate_LLM_Frameworks.ipynb
- 54_API_Authorization_and_Authentication.ipynb
- 55_Generate_knowledge_with_Semantic_Graphs_and_RAG.ipynb
- 56_External_vectorization.ipynb
- 57_Build_knowledge_graphs_with_LLM_driven_entity_extraction.ipynb
- 58_Advanced_RAG_with_graph_path_traversal.ipynb
- 59_Whats_new_in_txtai_7_0.ipynb
- 60_Advanced_RAG_with_guided_generation.ipynb
- 61_Integrate_txtai_with_Postgres.ipynb
- 62_RAG_with_llama_cpp_and_external_API_services.ipynb
- 63_How_RAG_with_txtai_works.ipynb
- 64_Embeddings_index_format_for_open_data_access.ipynb
- 65_Speech_to_Speech_RAG.ipynb
- 66_Generative_Audio.ipynb
- 67_Whats_new_in_txtai_8_0.ipynb
- 68_Analyzing_Hugging_Face_Posts_with_Graphs_and_Agents.ipynb
- 69_Granting_autonomy_to_agents.ipynb
- 70_Getting_started_with_LLM_APIs.ipynb
- 71_Analyzing_LinkedIn_Company_Posts_with_Graphs_and_Agents.ipynb
- 72_Parsing_the_stars_with_txtai.ipynb
- 73_Chunking_your_data_for_RAG.ipynb
- 74_OpenAI_Compatible_API.ipynb
- 75_Medical_RAG_Research_with_txtai.ipynb
- 76_Whats_new_in_txtai_9_0.ipynb
- 77_GraphRAG_with_Wikipedia_and_GPT_OSS.ipynb
- 78_Accessing_Low_Level_Vector_APIs.ipynb
- 79_RAG_is_more_than_Vector_Search.ipynb
- 80_Distilling_Knowledge_into_Tiny_LLMs.ipynb
- 81_OpenCode_as_a_txtai_LLM.ipynb
- 82_Agentic_College_Search.ipynb
- 83_TxtAI_got_skills.ipynb
- 84_Agent_Tools.ipynb
- 85_Progressive_Distillation.ipynb
- agent_quickstart.py
- article.py
- baseball.py
- benchmarks.py
- books.py
- images.py
- rag_quickstart.py
- similarity.py
- wiki.py
- workflow_quickstart.py
- workflows.py
- __init__.py
- bash.py
- edit.py
- embeddings.py
- factory.py
- function.py
- glob.py
- grep.py
- read.py
- skill.py
- todo.py
- write.py
- __init__.py
- base.py
- factory.py
- model.py
- placeholder.py
- __init__.py
- annoy.py
- factory.py
- faiss.py
- ggml.py
- hnsw.py
- milvus.py
- numpy.py
- pgvector.py
- sqlite.py
- torch.py
- turbovec.py
- zvec.py
- __init__.py
- factory.py
- ivfsparse.py
- pgsparse.py
- __init__.py
- base.py
- __init__.py
- factory.py
- json.py
- messagepack.py
- __init__.py
- agent.py
- caption.py
- embeddings.py
- entity.py
- extractor.py
- labels.py
- llm.py
- objects.py
- openai.py
- rag.py
- reranker.py
- segmentation.py
- similarity.py
- summary.py
- tabular.py
- textractor.py
- texttospeech.py
- transcription.py
- translation.py
- upload.py
- workflow.py
- __init__.py
- application.py
- authorization.py
- base.py
- cluster.py
- extension.py
- factory.py
- route.py
- __init__.py
- base.py
- __init__.py
- base.py
- compress.py
- factory.py
- tar.py
- zip.py
- __init__.py
- base.py
- factory.py
- hub.py
- storage.py
- __init__.py
- __main__.py
- base.py
- __init__.py
- base.py
- labels.py
- questions.py
- sequences.py
- texts.py
- tokens.py
- __init__.py
- base.py
- factory.py
- image.py
- serialize.py
- __init__.py
- orm.py
- statement.py
- __init__.py
- aggregate.py
- base.py
- error.py
- expression.py
- token.py
- __init__.py
- base.py
- client.py
- duckdb.py
- embedded.py
- factory.py
- rdbms.py
- sqlite.py
- __init__.py
- action.py
- autoid.py
- configuration.py
- documents.py
- functions.py
- indexes.py
- indexids.py
- reducer.py
- stream.py
- transform.py
- __init__.py
- base.py
- errors.py
- explain.py
- hybrid.py
- ids.py
- query.py
- scan.py
- terms.py
- __init__.py
- base.py
- __init__.py
- base.py
- factory.py
- networkx.py
- query.py
- rdbms.py
- topics.py
- __init__.py
- base.py
- cls.py
- factory.py
- last.py
- late.py
- lemur.py
- max.py
- mean.py
- muvera.py
- __init__.py
- models.py
- onnx.py
- registry.py
- tokendetection.py
- __init__.py
- audiomixer.py
- audiostream.py
- microphone.py
- signal.py
- texttoaudio.py
- texttospeech.py
- transcription.py
- __init__.py
- filetohtml.py
- htmltomd.py
- safeopen.py
- segmentation.py
- tabular.py
- textractor.py
- tokenizer.py
- urlretrieve.py
- __init__.py
- caption.py
- imagehash.py
- objects.py
- __init__.py
- factory.py
- generation.py
- huggingface.py
- litellm.py
- litert.py
- llama.py
- llm.py
- opencode.py
- rag.py
- __init__.py
- crossencoder.py
- entity.py
- labels.py
- lateencoder.py
- questions.py
- reranker.py
- similarity.py
- summary.py
- translation.py
- __init__.py
- hfonnx.py
- hftrainer.py
- lemur.py
- mlonnx.py
- __init__.py
- base.py
- factory.py
- hfmodel.py
- hfpipeline.py
- nop.py
- tensors.py
- __init__.py
- base.py
- bm25.py
- factory.py
- normalize.py
- pgtext.py
- sif.py
- sparse.py
- terms.py
- tfidf.py
- __init__.py
- base.py
- errors.py
- factory.py
- messagepack.py
- pickle.py
- serializer.py
- __init__.py
- download.py
- library.py
- resolver.py
- sparsearray.py
- template.py
- __init__.py
- external.py
- factory.py
- huggingface.py
- litellm.py
- litert.py
- llama.py
- m2v.py
- sbert.py
- words.py
- __init__.py
- base.py
- factory.py
- sbert.py
- __init__.py
- base.py
- recovery.py
- __init__.py
- base.py
- console.py
- export.py
- factory.py
- file.py
- image.py
- retrieve.py
- service.py
- storage.py
- stream.py
- template.py
- url.py
- workflow.py
- __init__.py
- base.py
- execute.py
- factory.py
- __init__.py
- version.py
- __init__.py
- testdense.py
- testsparse.py
- __init__.py
- testapiagent.py
- testapiembeddings.py
- testapipipeline.py
- testapiworkflow.py
- testauthorization.py
- testcluster.py
- testdependency.py
- testencoding.py
- testextension.py
- testmcp.py
- testopenai.py
- __init__.py
- testclient.py
- testcustom.py
- testdatabase.py
- testduckdb.py
- testencoder.py
- testrdbms.py
- testsql.py
- testsqlite.py
- __init__.py
- testmodels.py
- testpooling.py
- __init__.py
- testaudiomixer.py
- testaudiostream.py
- testmicrophone.py
- testtexttoaudio.py
- testtexttospeech.py
- testtranscription.py
- __init__.py
- testfiletohtml.py
- testsegmentation.py
- testtabular.py
- testtextractor.py
- testtokenizer.py
- testurlretrieve.py
- __init__.py
- testcaption.py
- testimagehash.py
- testobjects.py
- __init__.py
- testgenerator.py
- testlitellm.py
- testlitert.py
- testllama.py
- testllm.py
- testopencode.py
- testrag.py
- testsequences.py
- __init__.py
- testentity.py
- testlabels.py
- testreranker.py
- testsimilarity.py
- testsummary.py
- testtranslation.py
- __init__.py
- testonnx.py
- testquantization.py
- testtrainer.py
- __init__.py
- __init__.py
- testkeyword.py
- testsparse.py
- __init__.py
- testcustom.py
- testexternal.py
- testhuggingface.py
- testlitellm.py
- testlitert.py
- testllama.py
- testm2v.py
- testsbert.py
- testvectors.py
- testwordvectors.py
- __init__.py
- testsbert.py
- testvectors.py
- __init__.py
- testagent.py
- testapp.py
- testarchive.py
- testcloud.py
- testconsole.py
- testembeddings.py
- testgraph.py
- testlibrary.py
- testoptional.py
- testserialize.py
- testworkflow.py
- utils.py
- .coveragerc
- .gitignore
- .pre-commit-config.yaml
- .pylintrc
- apps.jpg
- CITATION.cff
- demo.gif
- LICENSE
- logo.png
- Makefile
- mkdocs.yml
- pyproject.toml
- README.md
- setup.py
# μ€μΉ κ°μ΄λ
1. μ½λ λ΄λ €λ°κΈ°
git clone https://github.com/neuml/txtai
κΉνλΈμμ νλ‘μ νΈ μ½λ μ 체λ₯Ό λ΄ μ»΄ν¨ν°λ‘ λ΄λ €λ°μ΅λλ€.
cd txtai
λ°©κΈ λ΄λ €λ°μ νλ‘μ νΈ ν΄λ μμΌλ‘ μ΄λν©λλ€.
2. 곡μ μ€μΉ μ€ν¬λ¦½νΈ
μ¬μ μΆμ²μ¬μ μ€λΉλ¬Ό
- Python 3 pip λͺ λ Ήμ΄λ₯Ό μ°λ €λ©΄ Pythonμ΄ νμν©λλ€.
pip install txtai
PyPIμ λ°°ν¬λ ν¨ν€μ§λ₯Ό λ°λ‘ μ€μΉν©λλ€. μμ€ ν΄λ‘ μ΄ νμ μμ΅λλ€.
μ€μΉ ν μ ν°λ―Έλμ μ΄κ³ , νλ‘κ·Έλ¨μ λ²μ νμΈ λͺ
λ Ή(μ: --version)μΌλ‘ μ μ μ€μΉλλμ§ νμΈνμΈμ.
μ΄ λ ν¬μ READMEμ μ ν μ€μ λͺ λ Ήμ΄λ₯Ό κ·Έλλ‘ κ°μ Έμμ΅λλ€.
3. Docker
μ¬μμ¬μ μ€λΉλ¬Ό
- Git GitHubμμ νλ‘μ νΈ μ½λλ₯Ό λ΄λ €λ°μΌλ €λ©΄ νμν©λλ€.
- Docker Desktop 컨ν μ΄λλ₯Ό λΉλνκ³ μ€ννλ €λ©΄ νμν©λλ€. μ€μΉ ν μ€νν΄μ λ°±κ·ΈλΌμ΄λμ μΌλμΈμ.
β οΈ μ΄ νλ‘μ νΈλ κ·λͺ¨κ° ν° μ μ₯μλΌ, μ΄ λ°©λ²μ΄ μ€μ ν΅μ¬ μ νμ΄ μλλΌ λ΄λΆ νμ ν¨ν€μ§λ₯Ό κ°λ¦¬ν€λ κ²μΌ μ μμ΅λλ€. README μ 체λ₯Ό ν¨κ» νμΈν΄λ³΄μΈμ.
docker build -f docker/api/Dockerfile -t txtai .
Dockerfileμ κΈ°λ°μΌλ‘ μ€ν κ°λ₯ν μ΄λ―Έμ§λ₯Ό λΉλν©λλ€.
docker run -p 8080:80 txtai
λΉλλ μ΄λ―Έμ§λ₯Ό μ€μ 컨ν μ΄λλ‘ μ€νν©λλ€.
ν°λ―Έλμ docker compose ps λ₯Ό μ
λ ₯ν΄ μ»¨ν
μ΄λλ€μ΄ Up μνμΈμ§ νμΈνμΈμ. READMEμ ν¬νΈ λ²νΈκ° μ νμλ€λ©΄ λΈλΌμ°μ μμ http://localhost:ν¬νΈλ²νΈ λ‘ μ μν΄λ³΄μΈμ.
4. Python
μ¬μμ¬μ μ€λΉλ¬Ό
pip install txtai
PyPIμ λ°°ν¬λ ν¨ν€μ§λ₯Ό λ°λ‘ μ€μΉν©λλ€. μμ€ ν΄λ‘ μ΄ νμ μμ΅λλ€.
μλ¬ λ©μμ§ μμ΄ μ€νλκ³ ν°λ―Έλμ μλ΄ λ¬Έκ΅¬κ° μΆλ ₯λλ©΄ μ μμ
λλ€.
μ΄ λ ν¬μ READMEμ μ ν μ€μ λͺ λ Ήμ΄λ₯Ό κ·Έλλ‘ κ°μ Έμμ΅λλ€.
5. Make
보ν΅μ¬μ μ€λΉλ¬Ό
- Git GitHubμμ νλ‘μ νΈ μ½λλ₯Ό λ΄λ €λ°μΌλ €λ©΄ νμν©λλ€.
- Make Linux/macOSλ λ³΄ν΅ κΈ°λ³Έ μ€μΉλμ΄ μμ΅λλ€. Windowsλ λ³λ μ€μΉ(μ: MSYS2, WSL)κ° νμν©λλ€.
make
μμ±λ λΉλ μ€μ μ λ°νμΌλ‘ μ€μ μ»΄νμΌμ μ§νν΄ μ€ν νμΌμ λ§λλλ€.
μλ¬ μμ΄ λλλ©΄ μ±κ³΅μ
λλ€. μμ±λ μ€ν νμΌμ μ§μ μ€νν΄λ³΄μΈμ.
// repository documentation
Was this content helpful?
(0 ratings)