llama.cpp Build Guide

Category

Local AI tooling

Provider

ggml

Type

repo

Level

Advanced

Topic

Local AI / LLM Engineering / RAG

Track

Local AI / LLM Engineering / RAG

Section

Learning path

Format

Documentation

Status

publishable

Commercial

link-only

Featured

no

Fast start

no

Sequence

nan

Priority

B

Primary source

mega_open_hub

Sources

mega_open_hub

ID

56257ad11396242e

llama.cpp

ggml-org

Core local inference stack for CPU / GPU quantized deployment and experimentation.

LlamaIndex RAG Guide

LlamaIndex

Conceptual and practical entry point for ingestion, indexing, and retrieval-augmented generation workflows.

FAISS

Meta

Library for efficient similarity search and clustering of dense vectors at large scale.

pgvector

pgvector

PostgreSQL extension for embedding similarity search when you want one operational database for app + vectors.

Open WebUI Docs

Open WebUI

Offline-first self-hosted AI interface that works well as a local front-end for models and knowledge tools.

vLLM OpenAI-Compatible Server

vLLM

High-throughput server for serving open models behind an OpenAI-compatible API.

Ollama Docs

Ollama

Official documentation for running and integrating local models with a simple developer workflow.

Haystack

deepset

Solid framework for retrieval pipelines, agents, evaluation, and production patterns.

Qdrant documentation

Qdrant

Simple, strong choice for local and hybrid vector search systems.

TRL documentation

Hugging Face

Useful for supervised fine-tuning, preference tuning, and training experiments.

Open WebUI Quick Start

Open WebUI

Fastest path to standing up Open WebUI with Docker for a self-hosted local AI stack.

Ollama API Introduction

Ollama

API docs for integrating local generation into apps, automations, and RAG pipelines.