repowise indexes a multi-million-line monorepo on a single machine, with its full git history. This page gives the commands for a repository of a few million lines and tens of thousands of commits, from the first clone to keeping the index current.
A machine with 8 vCPUs, 32 GB of RAM and 50 GB of free disk indexes a
4-million-line repository with its full history; give it 64 GB of RAM when
PostgreSQL runs on the same machine. The first index is a one-time cost. After
it, repowise update re-analyses only what each commit changed, and the
health walk, dead code and deep git history run in parallel worker processes.
1. Prepare the checkout
Index a normal clone with its history:
git clone https://gitlab.example.com/group/monorepo.git
cd monorepo
git rev-list --count HEAD # how many commits you haveUse a full clone and choose how much history to read with --commit-limit
(below), rather than trimming the clone. A shallow clone (--depth or
--shallow-since) leaves cut-off commits at its boundary, and git reports each
of them as adding every file in the repository, which slows the history pass.
If you already have a shallow clone, run git fetch --unshallow before the
first index.
C and C++ include paths. repowise resolves #include lines through
compile_commands.json when it finds one at the repo root or in build/,
alongside the CMake and Bazel target files it reads on its own. A CMake
configure step is enough to write it; nothing has to compile:
cmake -S . -B build -DCMAKE_EXPORT_COMPILE_COMMANDS=ONFor Bazel, generate it with a compile-commands extractor and leave the file at the repo root. Python and Scala need no setup: Python imports resolve from the tree, and Scala resolves through the shared JVM index, reading SBT and Mill build files where it needs them.
Exclusions. Leave out what nobody edits: vendored libraries, generated
code, large test fixtures. Put the patterns in .repowiseIgnore at the root
(gitignore syntax, read on every run) or pass them once with -x:
cat > .repowiseIgnore <<'EOF'
third_party/
contrib/
**/generated/
tests/fixtures/data/
EOFGit submodules are skipped by default; --include-submodules indexes them.
2. Choose the database
| SQLite (default) | PostgreSQL | |
|---|---|---|
| Best for | One machine, one user | A shared server, the web UI and MCP for a team |
| Setup | Nothing: .repowise/ in the repo | PostgreSQL with the pgvector extension available |
For PostgreSQL:
pip install "repowise[postgres]"
export REPOWISE_DB_URL="postgresql+asyncpg://repowise:secret@db.internal:5432/repowise"The schema is created on first use. After the first index on PostgreSQL, run
ANALYZE; once in that database so the query planner sees the new table
sizes. For long-lived connections through a firewall,
REPOWISE_POSTGRES_KEEPALIVE=1 turns on keepalive probes and a command
timeout (Environment variables).
3. First index
REPOWISE_PARSE_WORKERS=4 repowise init --yes --no-prose --no-editor-setup \
--commit-limit 25000| Flag | Why |
|---|---|
--no-prose | Builds every page from code, with no model and no key. Add the model-written concept pages later with repowise generate. |
--commit-limit 25000 | Reads the newest 25,000 commits for file history, ownership, co-change and the commit list. Set it above git rev-list --count HEAD to read the full history. Saved to config. |
REPOWISE_PARSE_WORKERS=4 | Parse processes. Each holds its own parser, so this is the main memory lever; the default is the CPU count, at most 8. |
--no-editor-setup | On a server, writes nothing outside .repowise/. Drop it on a developer machine to wire up editors and agents. |
--yes | Takes every default without asking. |
For the fastest first pass, index the graph and essential history first and add the rest later, with no model calls either way:
repowise init --yes --no-prose --mode fast
repowise update --fullAn interrupted first index continues where it stopped with
repowise init --resume.
Full history
commit_limit sets how far back churn, ownership, co-change and the commit
list see; Change Records cover the newest 10,000. The default is the newest
5,000 commits, there is no upper bound, and depth is cheap: deep history is
read four regions at a time with git's pack memory bounded. On git/git (61,453
non-merge commits), reading the full history instead of 5,000 commits adds a
few seconds and about 100 MB to the index.
To deepen an existing index without re-indexing:
repowise config set commit_limit 25000
repowise update --index-only --defer-docs --refresh-gitFiles left out
Source files up to 8 MB are parsed: those over 2 MB one at a time, outside the
parallel worker pool, so memory stays bounded. Source files over 8 MB,
minified bundles, files with no parser over 500 KB, and generated files are
left out of the index. The console names the
first few source files; every one of them, with its size and reason, is in
.repowise/skipped_files.txt, rewritten on each init and update:
# Files repowise did not index: path, size, reason (tab-separated)
src/parser/grammar_tables.cpp 9,310 KB over the 8,192 KB limit
tests/data/fixtures.tsv 48,112 KB over the size limit for files without a parser4. Keep it current
repowise update diffs the new commit against the last indexed one and
re-analyses only what changed. Pick a trigger (Auto-sync
has the details):
| Setup | Trigger |
|---|---|
| Developer machine | The post-commit hook init installs (repowise hook install) |
| Shared server on GitLab | The GitLab push webhook, with the 15-minute poller as a safety net |
| Server-owned clone | managed_checkout: true, so each sync fetches and fast-forwards the clone first |
Source edits, added and removed files, documentation and config edits patch
the graph in place, C and C++ included: on a 62,000-file C++ repository an
update takes 52 s where a full graph rebuild takes 303 s. A change to a build
file the graph reads (CMakeLists.txt, BUILD, BUILD.bazel) in a way that
changes the build rebuilds the whole graph on that update, because include and
target resolution depend on it.
Memory and time levers
In order of effect:
REPOWISE_PARSE_WORKERS=2 repowise init ... # fewer parse processes, less memory
repowise init --mode fast ... # index only; `repowise update --full` adds the rest
repowise init -x 'third_party/' ... # exclude what you never edit
repowise init --max-file-pages 2000 ... # cap file pages, highest importance firstAfter the first index, repowise config list --suggested reads what it stored
and names the settings that fit this repository, each with its numbers and the
command: a deeper commit_limit when many files have no commit in the window,
exclude_patterns for directories holding only vendored or generated code,
and every file page when the automatic cap left production files out.
To index the monorepo together with its smaller sibling repositories behind one MCP server, see Workspace setup.