Giving Zhimalab a Knowledge Base: A LanceDB + DeepSeek Field Report

Recently I had an idea: let Zhimalab visitors “ask” the site’s content — like “what sections does the site have” or “how do I set up the environment in the Python tutorial”. Relying on an LLM guessing blindly obviously won’t work; you have to retrieve first and then answer — that is, RAG (retrieval-augmented generation). After two days of tinkering I finally got it working, so I’m writing down the tech choices and pitfalls for anyone who wants to self-host a knowledge base.

Why these three components

RAG’s core is three things: chunking, storing vectors, and generating. My picks:

• Generation: DeepSeek. Great value, strong Chinese quality, and I’d already configured an API key on the server.
• Vector store: LanceDB. An embedded database that needs no separate service — very friendly to this 2GB-RAM server.
• Embedding: fastembed + BGE Chinese model. This is the easiest thing to trip over — DeepSeek has no official embedding API, so you either compute locally or pay a third party extra. I chose local: fastembed runs inference via ONNX without the heavyweight torch, and the small BGE model handles Chinese well.

Installation

The server runs Python 3.14 + uv. I initially worried lancedb had no 3.14 package, but a quick check was reassuring: lancedb uses the stable cp310-abi3 ABI wheel, so any Python 3.10+ can use it directly; fastembed is pure Python anyway. So installation is one command:

cd /opt/zhimalab-api
uv add lancedb fastembed openai

Wiring into FastAPI

I encapsulated retrieval and generation in a standalone app/rag.py module, then exposed two endpoints in main.py:

@app.get("/rag/ask")     # Q&A: retrieval + DeepSeek generation
@app.get("/rag/search")  # search: return only relevant snippets, no model call

Ingesting documents goes through a CLI script: rag.py add <directory>, which chunks and vectorizes documents into LanceDB; the two endpoints and the CLI share the same database directory.

Pitfall log (the most valuable part)

1. HuggingFace is blocked. Models have to be downloaded from HF and the server couldn’t reach it. I added the env var HF_ENDPOINT=https://hf-mirror.com to use a domestic mirror.
2. XetHub 401. Newer huggingface_hub defaults to the XetHub CAS channel for downloads, which returns a 401 against mirror sites directly — you must add HF_HUB_DISABLE_XET=1 to force plain HTTP. This error is very sneaky; at first I thought the mirror was down.
3. Permission problems. The service runs as the www-data user, but the vector store’s default path was under /root (permissions 700), which www-data couldn’t enter at all. In the end I moved both the data directory and the model cache to /var/lib/zhimalab-rag and granted access.
4. systemd doesn’t read .bashrc. I had put the key in root’s .bashrc, but the service couldn’t see it at all — systemd doesn’t load shell config files. The right way is to write it into the service’s EnvironmentFile (/opt/zhimalab-api/.env).
5. Slow startup. After a service restart you wait about 8 seconds (onnxruntime loading); don’t test immediately after restarting, or you’ll get Connection refused.

Measured results

I put a site-introduction article into the store and asked it:

Q: What sections does Zhimalab have?

A: According to the reference material, the Zhimalab (zhimalab.tech) website mainly includes the following sections: blog, frontend tools, interactive tutorials, AI lab and kids’ games.

Retrieval score 0.62, and the answer came entirely from the material — no hallucination.

Next steps

• Batch-ingest the site’s articles and tutorials into the knowledge base
• Add auth and rate limiting to the /rag endpoints, then consider exposing them publicly
• Try deepseek-reasoner for more complex reasoning Q&A

The skeleton of the knowledge base is in place; the rest is just filling it with content.