Debtlint

Debtlint is a technical debt detection tool focused on duplicated logic, built for the era of AI code generation. It identifies repeated or near-duplicate code segments across a codebase and reports them for refactoring.

Vision

As coding assistants become mainstream, "silent" duplication appears quickly:

• existing utility functions get rewritten,
• business patterns are repeated,
• logic becomes fragmented across multiple files.

Debtlint provides a programmatic, fast, and cost-efficient way to prevent this drift and keep architecture healthy over time.

Objectives

• Detect exact duplications and structural similarities.
• Surface duplicated blocks with file and line locations.
• Provide actionable reports (terminal + JSON) for local use and CI.
• Integrate easily into team workflows (local, CI, agents).
• Stay as language-agnostic as possible.

Install

cargo install debtlint

Or from the repository:

cargo install --path .

Quick start

1. Create a debtlint.config.json at the root of the project:

{
  "src": ".",
  "excludes": ["target", "node_modules", ".git"]
}

2. Optionally train and save your own vocabulary as .debtlint-vocab.json:

debtlint --save-vocab .debtlint-vocab.json

3. Run Debtlint (reuse the vocabulary to skip BPE training):

debtlint --load-vocab .debtlint-vocab.json

Or run without a saved vocabulary (trains BPE on the fly):

debtlint

How It Works

The Debtlint pipeline follows four main steps:

1. Code ingestion Read tracked source files under src from debtlint.config.json, while respecting excludes.

2. BPE encoding Train (or load) a Byte Pair Encoding vocabulary to turn source into token sequences that capture recurring patterns.

3. Winnowing-based duplicate detection Build hashed shingles over the token stream, select fingerprints with a sliding window, then extend matching regions into duplicated blocks.

4. Reporting Print a human-readable summary by default, or JSON for CI and agents (--report-format json).

Why BPE

BPE is historically a compression and tokenization technique, notably used to segment large amounts of text efficiently. In Debtlint, this logic helps highlight repetitive structures and grammatical blocks of code.

Core reference: Byte pair encoding - Wikipedia

More detail: doc/bpe.md

CLI

debtlint [OPTIONS]

Flag	Default	Description
--vocab-size	1000	Max vocabulary size (base + merges)
--min-frequency	2	Min pair frequency to create a merge
--load-vocab <PATH>	—	Load a vocabulary JSON and skip BPE training
--save-vocab <PATH>	—	Save the trained vocabulary as JSON
--output-encoded <PATH>	—	Write encoded token sequences as JSON
--shingle-size	50	Shingle size in tokens (Winnowing)
--window-size	3	Sliding window size in shingles
--min-tokens	20	Minimum duplicated block length in tokens
--report-format	text	text or json

Examples:

# Human-readable report
debtlint

# JSON for CI / agents
debtlint --load-vocab debtlint-vocab.json --report-format json

# Train and persist a vocabulary
debtlint --vocab-size 2000 --save-vocab vocab.json

Configuration is read from debtlint.config.json in the current directory.

GitHub Actions Integration

Use the action in this repository (action.yml at the root) to analyze pull requests. See action/README.md for inputs and details.

name: Check duplication

on:
  pull_request:
    branches: [main]

jobs:
  debtlint:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      pull-requests: write

    steps:
      - uses: actions/checkout@v4

      - name: Run debtlint
        uses: PoCInnovation/Debtlint@v1
        with:
          vocab-path: .debtlint-vocab.json

Agent integration

Debtlint can sit in an agent loop after code generation:

• generate code,
• run debtlint --report-format json,
• refactor reported duplicates,
• re-run until clean (or after a fixed number of iterations).

An agent skill lives in skills/debtlint.

License

This project is licensed under the Apache License 2.0.

Contributing

Contributions are welcome. Read the guide: CONTRIBUTING.md

Project presentation video

Our PoC team ❤️

Developers

Enzo Ruffino	Loan Riyanto	Rayan

Manager

Laurent Gonzalez	Lohan

Organization

🚀 Don't hesitate to follow us on our different networks, and put a star 🌟 on PoC's repositories

Made with ❤️ by PoC