index

$npx mdskill add softspark/ai-toolkit/index

Reindexes knowledge base for semantic search via vector store.

  • Solves the need to keep semantic search up-to-date with KB changes.
  • Depends on Qdrant vector store and an indexing pipeline.
  • Uses content hashing to detect new, changed, or deleted documents.
  • Executes make index or make index-full commands via Bash.

SKILL.md

.github/skills/indexView on GitHub ↗
---
name: index
description: "Reindexes KB for semantic search via vector store (Qdrant). Triggers: reindex KB, rebuild index, vector reindex, refresh embeddings."
effort: low
disable-model-invocation: true
argument-hint: "[path or scope]"
allowed-tools: Bash
---

# Knowledge Base Indexing

$ARGUMENTS

Reindex the knowledge base for semantic search.

> **Prerequisite**: This command requires a vector store (e.g., Qdrant) and an indexing pipeline configured for your project. If not configured, this command provides guidance on setup.

## Usage

```
/index              # Incremental index (detect changes)
/index --full       # Full rebuild
```

## Execution

### Direct Execution
```bash
# Incremental index (auto-detects changes)
make index

# Full rebuild
make index-full
```

### Docker Execution
```bash
docker exec {app-container} make index
docker exec {app-container} make index-full
```

## Change Detection

The indexer uses content hashing to detect changes:

| Scenario | Action |
|----------|--------|
| New document | Index |
| Changed content | Reindex |
| No changes | Skip |
| Deleted document | Remove from index |

## Frontmatter Validation

Before indexing, ensure all KB documents have valid frontmatter:

```yaml
---
title: "Document Title"
service: {service-name}
category: reference|howto|procedures|troubleshooting|decisions|best-practices
tags: [tag1, tag2]
last_updated: "YYYY-MM-DD"
---
```

## Troubleshooting

| Problem | Solution |
|---------|----------|
| Index not updating | Check file timestamps, run full rebuild |
| Missing documents | Verify frontmatter is valid |
| Slow indexing | Check embedding service performance |
| No vector store | Set up Qdrant or compatible vector DB |

## Rules

- **MUST** require explicit user permission before running `make index` or `make index-full` — never self-trigger
- **NEVER** trigger a full rebuild to "clean up" unless the user asked for it
- **CRITICAL**: validate KB frontmatter before indexing — abort on invalid documents rather than indexing a broken state
- **MANDATORY**: respect change-detection hashes; do not force reindexing of unchanged documents

## Gotchas

- Content-hash change detection keys on the file's content AND path. A **moved** document (same content, new path) looks new to the indexer — both the old path vector and the new one will exist until a full rebuild. Plan a full rebuild after mass reorganizations.
- Deleting a document on disk does not automatically remove its vectors from Qdrant; the indexer emits tombstones only if run with a directory scan. Without `--delete-missing`, orphan vectors stay for weeks.
- Embedding providers rate-limit by requests-per-minute AND by tokens-per-minute. A reindex of 1000+ docs hits the token cap first and stalls silently — watch for 429s in the indexer log before concluding "slow indexing".
- `make index-full` truncates the collection before re-embedding; if the embedding job crashes mid-way, the collection is left partially populated with no query-time indicator of the gap.

## When NOT to Use

- For searching the KB — use `/research-mastery` or call `smart_query()` via the rag-mcp tool
- For fixing indexing bugs — use `/debug` on the indexer pipeline
- To evaluate RAG quality after reindexing — use `/evaluate`
- When no vector store is configured — document the gap, do not invent one

More from softspark/ai-toolkit

SkillDescription
a11y-validateAccessibility validator: WCAG 2.1 AA, EN 301 549, EAA. Triggers: a11y, accessibility, WCAG, EAA, ARIA, contrast, keyboard, screen reader.
agent-creatorCreates new specialized agents with frontmatter, tools, delegation. Triggers: new agent, create agent, agent scaffold, specialized agent.
ai-toolkit-rulesMandatory engineering, security, testing, git, performance, quality, and response rules. Claude MUST load this skill for every technical, coding, debugging, review, architecture, DevOps, data, or file-editing task in Chat or Cowork.
api-patternsREST/GraphQL API design: naming, versioning, pagination, idempotency, OpenAPI. Triggers: API design, REST, GraphQL, OpenAPI, Swagger, idempotency, rate limit.
architecture-auditAudits codebase for architectural friction, shallow modules; proposes RFCs. Triggers: improve architecture, shallow modules, deepen modules, reduce coupling.
biz-scanScans codebase for revenue opportunities, KPIs, monetization gaps. Triggers: business metrics, KPI, analytics gaps, monetization, revenue.
briefingExecutive daily briefing aggregating reports from all agents into decision-focused summary. Triggers: briefing, daily summary, status across system, executive update.
chaosInjects controlled faults for resilience testing on non-prod. Triggers: chaos, fault injection, latency injection, dependency kill, resilience test.
ci-cd-patternsCI/CD: GitHub Actions, GitLab CI, Jenkins, caching, blue-green, canary. Triggers: CI, CD, pipeline, GitHub Actions, workflow YAML, release, canary, rollout.
content-moderation-patternsContent moderation with Claude: pre-filter vs LLM-classify, categories, thresholds, HITL. Triggers: moderation, safety filter, policy enforcement, content classifier.