The GitHub Actions part is straightforward. If a duplicate-code checker has a CLI, meaningful exit codes, and predictable output, GitHub Actions can run it. The more important decisions are what you scan, what constitutes a failure, and what to do when an existing repository already contains duplication.

Those decisions determine whether the check becomes a useful engineering constraint or just another red CI job that everyone learns to ignore.

CI Should Enforce a Policy, Not Just Run a Command

The simplest duplicate-code policy is also the strictest: if any duplicate code is found, fail the build. That works well for a new project or a repository that is already clean, but it can be a bad adoption strategy for a mature codebase.

Suppose you add duplicate detection to a five-year-old project and the first run finds 150 existing duplicate groups. The detector has done exactly what you asked, but CI can no longer pass until someone fixes five years of accumulated debt. Teams faced with that situation usually end up choosing between stopping feature work, accepting a permanently failing check, or disabling the check again. None of those outcomes are especially useful.

Before adding the workflow, decide what you actually want CI to enforce. You may want zero duplication, you may initially want visibility without enforcement, or you may want to accept existing duplication while preventing new duplication from being introduced. Those are three different policies even though they can all use the same detector.

A Minimal GitHub Actions Workflow

GitHub Actions does not need any special understanding of duplicate-code detection. At its simplest, the job checks out the repository, runs the detector over the Python source, and interprets the result.

For the examples here I am using Arid, the Python duplicate-code checker I built to provide focused, Python-aware duplicate detection without requiring a general-purpose linter. Arid has an official GitHub Action, so a minimal workflow looks like this:

name: Duplicate code

on:
  pull_request:
  push:
    branches: [main]

jobs:
  duplicate-code:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4

      - name: Check for duplicate Python code
        uses: sponge-b0b/arid@v2.2.3
        with:
          paths: .

That is enough for a basic enforcing check. The Action handles the Python setup and installation needed to run Arid, scans the repository, and fails when active duplicate findings are present.

For a new or already-clean project, this may be all you need.

Keep Detection Policy in the Project

One of the easiest ways to make CI irritating is to hide important tool configuration inside the workflow file. Developers then run one thing locally while CI quietly runs something else.

A better separation of responsibilities is to let GitHub Actions decide when the tool runs and let the project configuration decide how it runs.

For example:

[tool.arid]
min-lines = 6
exclude = [
    "generated/**",
]

The workflow itself can remain unchanged:

- name: Check for duplicate Python code
  uses: sponge-b0b/arid@v2.2.3
  with:
    paths: .

A developer can reproduce the same policy locally with:

arid .

That property matters more than it may appear. CI failures are much easier to work with when the developer can reproduce the exact same check before pushing another commit. The workflow should automate the repository's quality policy, not define a second, hidden version of it.

Decide Whether Findings Should Fail CI

Enforcement does not have to begin on day one. In an existing codebase, it can be useful to run duplicate detection for a while before making findings blocking. That gives you a chance to understand what the detector reports, adjust exclusions or thresholds, and decide what level of duplication the team actually wants to enforce.

With the Arid Action, that can be done by disabling failure on findings:

- name: Report duplicate Python code
  uses: sponge-b0b/arid@v2.2.3
  with:
    paths: .
    fail-on-findings: "false"

The important distinction is that this changes the policy for duplicate findings, not the meaning of every possible failure. A completed scan that finds duplication is different from a scan that cannot complete because of invalid configuration, unreadable source, or another operational problem.

That distinction is worth preserving in any CI tool. "We found something and chose not to block on it" is a policy decision. "The analysis did not complete" is a different state entirely.

For many mature repositories, a reasonable adoption path is to begin in reporting mode, tune the check, and only then turn it into a required gate. There is little value in introducing a quality check so aggressively that the first lesson developers learn is how to bypass it.

Brownfield Repositories Need a Different Strategy

Reporting-only mode is useful during evaluation, but eventually you may want to enforce duplicate-code policy without first eliminating every duplicate that already exists.

That is where baselining becomes useful.

A baseline records the findings that already exist and treats them as acknowledged debt. Future scans can then distinguish between duplication the repository already had and duplication introduced later. The CI policy becomes "do not add new duplicate code" rather than "repair the entire history of the repository before the next pull request can merge."

That is often a much more practical constraint for a brownfield project. Existing debt remains visible and can be reduced over time, but it does not prevent the team from enforcing a better rule going forward.

There are enough subtleties around baseline creation, maintenance, stale findings, and debt reduction that baselines deserve a separate article. The important point here is that CI enforcement does not have to be binary. You do not have to choose between fixing everything immediately and ignoring duplication forever.

Duplicate Detection Is a Corpus-Level Problem

There is another CI mistake that is less obvious: scanning only the files changed by the pull request.

That optimization makes sense for many tools. A formatter can format one file. A linter can inspect one file. A type checker may be able to limit part of its work to a dependency boundary.

Duplicate detection is different because a duplicate is a relationship between two or more regions of code.

Suppose a pull request adds this:

def normalize_items(items):
    result = []

    for item in items:
        if item is None:
            continue

        result.append(str(item).strip())

    return result

The duplicate may already exist in src/legacy/importer.py. If CI analyzes only the newly changed file, there is nothing to compare it with. The file appears unique because the rest of the corpus has been removed from the analysis.

This is one of those cases where a performance optimization can quietly change the question being asked. "Does this changed file contain something that duplicates code elsewhere in the repository?" cannot be answered by looking only at the changed file.

Arid separates those concerns with its focus option. The full corpus can still be analyzed while reporting is restricted to findings that touch the area you care about:

- name: Check package for new duplicate code
  uses: sponge-b0b/arid@v2.2.3
  with:
    paths: .
    focus: src/package

The distinction is important: focus the report, not the detector. You can reduce noise without throwing away the context required to detect duplication correctly.

CI Results Should Be Useful to Both Humans and Automation

A command returning success or failure is enough to build a gate, but CI usually benefits from richer output.

The Arid Action writes a Markdown report to the GitHub job summary by default, which means developers do not have to dig through raw workflow logs just to see what was found. It also exposes structured outputs such as duplicate-group counts, duplicated lines, duplication percentage, affected files, occurrence counts, scan completeness, and the underlying scan exit code.

For example:

- name: Check for duplicate Python code
  id: arid
  uses: sponge-b0b/arid@v2.2.3
  with:
    paths: .

- name: Show duplicate metrics
  run: |
    echo "Duplicate groups: ${{ steps.arid.outputs.duplicate-groups }}"
    echo "Duplicate lines: ${{ steps.arid.outputs.duplicate-lines }}"

You may never need those outputs, and that is fine. Their value is that another piece of automation does not have to parse terminal text intended for a human.

Human-readable output and machine-readable output serve different purposes. Trying to make one format do both usually produces output that is unpleasant for humans and fragile for machines.

Add SARIF Only If It Improves the Workflow

GitHub code scanning understands SARIF, so Arid can also publish duplicate findings through that interface. If your team already uses GitHub code scanning for static-analysis results, putting duplicate findings there can make sense.

A workflow enabling SARIF looks like this:

name: Duplicate code

on:
  pull_request:
  push:
    branches: [main]

permissions:
  contents: read
  security-events: write

jobs:
  duplicate-code:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4

      - name: Check for duplicate Python code
        uses: sponge-b0b/arid@v2.2.3
        with:
          paths: .
          sarif: "true"

I would not enable SARIF merely because it exists. If developers already get everything they need from the job summary, adding another reporting surface may not improve anything.

The useful architectural property is that reporting remains separate from analysis. Arid performs the scan once and can derive the job summary, Action outputs, and optional SARIF result from that same analysis. You should not need to run the detector three times because three consumers want the result in three different formats.

A Reasonable Adoption Sequence

For a new or clean repository, I would start with the simple enforcing workflow:

name: Duplicate code

on:
  pull_request:
  push:
    branches: [main]

jobs:
  duplicate-code:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4

      - name: Check for duplicate Python code
        uses: sponge-b0b/arid@v2.2.3
        with:
          paths: .

For an established repository where you do not yet know what the check will uncover, I would begin with the same workflow but set fail-on-findings to false. Once the configuration is producing useful results, decide whether the repository should enforce zero duplication or whether existing findings should be baselined so that only new duplication fails CI.

That progression is intentionally boring. Good CI policy usually is.

The important part is that the detector does not dictate the adoption model. A greenfield project, a project evaluating the tool, and a brownfield project with years of historical duplication can all use the same underlying analysis while enforcing different policies.

Is Arid the Right Tool for This?

Not necessarily.

If Pylint R0801 already performs well enough for your repository and fits the rest of your tooling, there may be no reason to replace it. If you have a multi-language monorepo and want one clone detector for everything, a Python-specific tool is probably the wrong abstraction. If your real requirement is semantic clone detection—finding code that behaves similarly even though it is written differently—you need a detector designed for that problem.

Arid deliberately has a narrower scope. It detects exact duplicated Python source after configurable Python-aware normalization, and it is designed to run alongside tools such as Ruff rather than become another general-purpose linter.

That narrowness is why I built it. I had already moved most of my Python linting workflow to Ruff but still wanted corpus-level duplicate detection. Carrying a general-purpose linter primarily for one expensive check felt like the wrong dependency boundary, so I built a tool whose responsibility begins and ends with duplicate-code detection.

The local workflow is intentionally unremarkable:

ruff check .
arid .

The GitHub Action simply makes the second command part of the repository's normal engineering process.

The YAML Is the Easy Part

Adding duplicate-code detection to GitHub Actions takes only a few lines. Deciding what those lines should mean is the more interesting part.

A good CI check answers a clear policy question. Are duplicate findings always forbidden? Are you still evaluating the signal? Are existing findings accepted while new ones are blocked? Are you analyzing the whole repository even when you only want to report findings in a particular area?

Those choices determine whether duplicate detection becomes useful or merely becomes present.

The goal is not another green checkmark in a pull request. The goal is a constraint developers understand, can reproduce locally, and can keep green without working around it. Once the policy is clear, GitHub Actions is just the mechanism that makes it consistent.

Arid is an open-source Python duplicate-code checker written in Rust, designed as a focused replacement for Pylint R0801 and to complement Ruff.

GitHub: https://github.com/sponge-b0b/arid

GitHub Marketplace: https://github.com/marketplace/actions/arid-duplicate-code-check

About the Author

Bob Taylor is a software engineer and architect who builds developer tools and AI systems. He is currently developing Arid and Polaris.

A collection of linting tools and plugins becomes one fast executable. Configuration gets simpler. Checks that used to take long enough to interrupt the development loop become cheap enough to run constantly. Ruff now provides more than 900 lint rules, along with formatting, import sorting, automatic fixes, caching, pyproject.toml configuration, and editor integrations.

For many projects, that's enough to eliminate a significant portion of the traditional Python tooling stack.

Then you go looking for duplicate-code detection.

Ruff has rules that detect specific forms of duplication—duplicate exception handlers, duplicate entries in __all__, duplicate set values, and similar local mistakes—but it does not provide a general cross-file duplicate-code detector comparable to Pylint's R0801.

That leaves Ruff users with an interesting choice. You can add a broader analysis tool back into the stack to recover one capability, ignore duplication entirely, or pair Ruff with a tool focused specifically on the problem.

I ended up choosing the third option.

Ruff Doesn't Need to Do Everything

It's tempting to think of adopting a tool as choosing a winner.

If Ruff replaces Flake8, isort, Black, and a collection of plugins, the natural question becomes: why can't it replace everything else too?

I don't think that's necessarily the right goal.

Ruff is already extraordinarily broad. Its rules are inspired by or compatible with a long list of existing Python tools, but Ruff reimplements those rules in Rust as first-party functionality rather than invoking the original tools underneath.

That architecture is a big part of what makes Ruff useful, but there's still a difference between adding another lint rule and adding another kind of analysis.

Consider unused imports:

import pathlib

A linter can inspect that module and determine whether pathlib is used.

Now consider this:

def normalize_customer(customer):
    name = customer.name.strip()
    email = customer.email.lower().strip()
    return name, email

There may be absolutely nothing wrong with that function when considered by itself.

The problem appears when essentially the same implementation exists somewhere else in the repository.

Duplicate detection therefore asks a different question:

Where else in this corpus does this code occur?

That's inherently a broader problem than many ordinary lint rules. The unit of reasoning isn't necessarily one statement, function, or file. It's the source corpus.

That doesn't mean Ruff could never perform this kind of analysis. It means I wouldn't judge Ruff by whether every conceivable form of Python analysis eventually becomes a Ruff rule.

Sometimes composition is a perfectly good architecture.

What Are the Options?

If you're using Ruff and want duplicate-code detection, there are several reasonable approaches.

The first is to do nothing.

That isn't sarcasm. Not every project needs an automated duplication check. Small projects may make duplication obvious during review, and some teams simply don't consider it important enough to justify another tool.

The second option is Pylint.

Pylint's R0801, or duplicate-code, is the established Python solution I used before building Arid. You don't have to restore an entire Pylint configuration either. If all you want is duplicate detection, you can run that checker specifically:

pylint --disable=all --enable=duplicate-code .

If that's fast enough for your project, it's a reasonable solution.

I covered that path, and why I eventually moved away from it, in A Fast Alternative to Pylint R0801.

The third option is a dedicated duplicate-code tool.

That's the approach I eventually wanted for my own projects: let Ruff handle the broad linting job and use something else for the analysis Ruff doesn't provide.

The Ruff + Arid Model

That second part is why I built Arid.

Arid is an open-source Python duplicate-code checker written in Rust. It isn't a Ruff replacement, and I deliberately don't want it to become one.

My normal workflow is:

ruff check .
arid .

That's basically the entire idea.

Ruff owns linting. Arid owns duplicate-code detection.

There's some appeal in how boring that is.

Instead of building another Python quality platform with formatting, imports, complexity analysis, security rules, type checking, duplication, and whatever else I can squeeze into it, Arid can remain narrowly concerned with one question.

That lets Ruff continue doing the things Ruff is exceptionally good at while Arid specializes in something Ruff currently doesn't attempt.

What Counts as Duplicate Code?

This deserves some precision because "duplicate code" can describe very different kinds of analysis.

Consider:

def calculate_total(items):
    total = 0

    for item in items:
        total += item.price

    return total

and:

def calculate_order_value(products):
    amount = 0

    for product in products:
        amount += product.price

    return amount

A sophisticated semantic clone detector might decide those functions represent the same computation.

Arid won't.

Arid detects exact duplicated Python source after configurable Python-aware normalization. It can exclude things such as comments, docstrings, imports, and function signatures from duplicate identity, but it isn't attempting to prove that differently written programs mean the same thing.

That boundary is deliberate.

Semantic similarity is interesting, but it's also a much harder problem with different tradeoffs around false positives and explainability. I wanted something closer to the practical duplicate-code check I had been getting from Pylint.

When Arid reports duplication, I want the reason to be unsurprising.

Fast Tools Change When You Use Them

Performance matters here for a reason that has less to do with benchmark bragging rights than it might appear.

A quality check that takes 30 seconds isn't necessarily useless.

But you probably don't run it after every small change.

A check that takes a fraction of a second can become part of the normal development loop:

ruff check .
arid .
pytest

That difference changes behavior.

Fast checks move earlier.

Instead of discovering duplication when CI runs—or during a larger quality pass before merging—you can discover it while you're still working on the code that introduced it.

This was one of the things I liked about Ruff in the first place. Speed isn't merely about spending fewer seconds waiting for software. It changes which checks are cheap enough to run routinely.

I wanted duplicate detection to have the same property.

How Fast Is Arid?

For Arid 2.0, I benchmarked against Pylint 4.0.6 using pinned corpora and Hyperfine. The Pylint runs isolate its duplicate-code functionality rather than comparing Arid against a complete Pylint analysis.

Running serially, Arid measured:

Requests — 191.19x faster

Pydantic — 219.06x faster

Polaris — 249.68x faster

There's an important limitation on what those numbers mean.

They do not mean "Arid is 200x faster than Pylint" in some general sense. Pylint performs many analyses that Arid doesn't perform at all.

The benchmark asks a much narrower question: when both tools are being used for the duplicate-code job Arid was built to replace, how much does that analysis cost?

That's the comparison I care about.

And although Arid supports worker-based parallelism, I use serial execution for the canonical comparison. I'd rather have a boring benchmark whose assumptions are easy to understand than manufacture the largest number possible.

One pyproject.toml, Two Responsibilities

Using focused tools doesn't have to mean returning to configuration-file sprawl.

Ruff supports pyproject.toml, ruff.toml, and .ruff.toml. Arid supports configuration through [tool.arid] in pyproject.toml.

So a project can keep both tools in the same configuration file:

[tool.ruff]
line-length = 100

[tool.ruff.lint]
select = ["E", "F", "B", "UP"]

[tool.arid]
min-lines = 6

They're independent tools, but they don't have to feel like unrelated infrastructure scattered around the repository.

That matters to me because "use focused tools" can become bad advice if every capability introduces another config format, another runtime, another reporting convention, and another collection of strange CI semantics.

Composition works best when the boundaries are narrow and predictable.

The Existing-Code Problem

There's another issue that becomes especially visible when adding a new quality check to an established Ruff workflow.

Imagine you've spent months cleaning up your Ruff configuration. CI is green. Developers have fast feedback.

Then you add duplicate detection and discover 200 existing duplicate groups.

Congratulations: your new quality gate is now permanently red.

The obvious responses aren't great.

You can fix all 200 findings before enabling the check, which may turn a tooling improvement into a refactoring project nobody has time for.

You can suppress the check broadly, which defeats the point.

Or you can establish a baseline.

Arid can record the duplicate findings that already exist and treat them as accepted debt while continuing to reject newly introduced duplication.

That creates a migration path:

existing duplicate debt
        ↓
record baseline
        ↓
reject new duplication
        ↓
refactor old duplication over time
        ↓
prune resolved baseline entries

I think this is a much more general lesson about static-analysis adoption.

A tool doesn't become useful merely because it can identify everything that's wrong. On an existing codebase, it also needs a practical answer to the question: what are we supposed to do with all the problems that existed yesterday?

Ruff itself recognizes a version of this adoption problem. Its documentation describes --add-noqa and --add-ignore workflows for introducing new rules while suppressing existing violations.

Arid's baseline mechanism solves the analogous problem for findings whose identity can span multiple files and locations.

I'll cover that workflow in more depth separately, because it's useful well beyond the Ruff + Arid combination.

The CI Version Is Still Boring

Locally:

ruff check .
arid .

In CI, the same separation works.

Arid 2.0 provides an official GitHub Action:

- uses: sponge-b0b/arid@v2.0.0
  with:
    paths: .

You can also invoke the CLI directly if that's preferable.

Arid can emit human-readable text as well as JSON, Markdown, and SARIF, and it can produce multiple report formats from a single analysis. That lets a CI run serve different consumers without rescanning the project for each representation.

For example, a developer may want console output while GitHub code scanning consumes SARIF and some downstream automation consumes JSON.

The detector runs once.

Again, this is less interesting as a feature checklist than as a property of composition. If two tools are going to live beside each other in CI, both need to behave predictably as components of a larger system.

Focused Tools Still Need Good Contracts

This became more important to me while building Arid 2.0.

A small CLI can get away with being designed entirely around humans:

scan some files
print some text
return an exit code

But developer tools increasingly have other software consuming their results: CI systems, editors, reporting services, scripts, and coding agents.

Once that happens, "small tool" doesn't mean "informal interface."

Arid 2.0 has versioned report schemas, stable finding fingerprints, structured operational errors, deterministic capability discovery, and explicit completion state for partial analysis.

Those aren't duplicate-detection algorithms.

They're contracts.

And I think they're part of what makes the focused-tool model viable. If I'm going to assemble a development workflow from specialized components, those components need to be easy for both humans and software to reason about.

A Unix pipe is only elegant because both sides agree on what goes through it.

Could Ruff Eventually Add Duplicate Detection?

Of course.

Ruff has grown enormously, and its current documentation lists more than 900 lint rules. There is no technical law saying duplicate-code analysis must forever live outside Ruff.

If Ruff eventually implements the kind of duplicate detection I need, I'll evaluate it like any other option.

That doesn't make building a companion tool pointless.

Software ecosystems don't have to converge toward one executable that does everything. Sometimes a focused tool is useful permanently. Sometimes it demonstrates demand for a capability that eventually moves elsewhere. Sometimes competing implementations discover different tradeoffs.

I'm comfortable with all three outcomes.

Arid doesn't need Ruff to remain incomplete in order to justify existing. It needs to solve a real problem well enough to be useful today.

Should a Ruff User Add Arid?

Only if you care about duplicate-code detection.

If you don't, Ruff doesn't have a problem that needs fixing.

If you do, I'd think about the decision this way:

Use Pylint R0801 if you're already happy with Pylint or its dedicated duplicate-code pass performs well enough for your project.

Use a more sophisticated clone-analysis tool if you're trying to detect semantic or structurally similar implementations rather than exact normalized duplication.

Consider Arid if you want a focused, fast duplicate-code check that fits naturally beside Ruff and can be used locally and in CI without turning duplication into another general-purpose linting stack.

For me, that last option produced a development loop I like:

ruff check .
arid .

Two tools. Two responsibilities.

I don't think good tooling architecture is measured by how few executable names appear in a CI file. It's measured by whether each piece solves a useful problem, composes cleanly with the others, and stays out of the developer's way.

Ruff already does that extraordinarily well.

Duplicate-code detection doesn't need to be inside Ruff to work the same way.

Arid is an open-source Python duplicate-code checker written in Rust and designed to complement Ruff.

About the Author

Bob Taylor is a software engineer and architect who builds developer tools and AI systems. He is currently developing Arid and Polaris.

GitHub

Ruff can replace a large part of the traditional Python linting stack, and it does that work extremely quickly. For a lot of projects, adopting Ruff means there is much less reason to run several different linters over the same source tree.

There is one check I wasn't able to move out of Pylint, though: duplicate-code detection.

Pylint reports duplicated code as R0801, or duplicate-code. It comes from Pylint's similarities checker and looks for similar lines across Python source.

It's a useful check. The problem for me wasn't what R0801 did. It was having to run Pylint to get it.

Why I Still Wanted R0801

Duplicate code is easy to dismiss as a style problem until you've maintained enough of it.

Two copies of the same implementation mean two places that may need to change when the behavior changes. Three copies mean three. Eventually one gets fixed while another doesn't, and code that looked identical gradually stops behaving identically.

Pylint's own documentation makes essentially the same point: duplicated logic increases the number of places that have to be found, changed, and tested, and can make code harder to understand during review.

So turning off R0801 wasn't the solution I wanted.

I wanted to keep the check without paying for a much broader analysis pass just to get that one result.

Running Only Pylint's Duplicate-Code Check

If you already have Pylint installed, the first thing to try is simply running less Pylint.

You can enable only duplicate-code detection:

pylint --disable=all --enable=duplicate-code .

That is a perfectly reasonable solution, particularly if Pylint is already part of your project and its performance is acceptable.

There are also useful controls around what contributes to similarity. Pylint supports ignoring comments, docstrings, imports, and function signatures when constructing the source representation used for duplicate detection.

So before replacing anything, I'd start there.

If a dedicated R0801 pass is fast enough for your repository, you may already have the solution you need.

For me, it wasn't.

Why Duplicate Detection Can Become Noticeable

Duplicate-code detection is different from many ordinary lint rules.

A rule such as "this import is unused" can largely reason about a particular file or syntax tree. Duplicate detection has to compare source across a corpus because the code in one file may duplicate code somewhere else entirely.

That distinction becomes more important as a repository grows.

It also complicates the obvious answer to performance problems: just split the work across files.

Cross-file analysis needs a global view of the source being compared. There are real-world projects that have ended up separating Pylint's duplicate-code checker into its own single-process CI job because partitioning files among workers can change the detected clusters.

That doesn't mean Pylint is a bad tool. Pylint does far more than duplicate detection.

But it does raise a reasonable question if R0801 is the reason you're still running it:

Do you need a general-purpose linter for this particular job?

A Focused Alternative

That question is why I built Arid.

Arid is a Python duplicate-code checker written in Rust. It isn't intended to replace Pylint as a whole, and it isn't trying to replace Ruff.

It replaces one job:

Pylint R0801

with a tool dedicated to duplicate-code detection.

The workflow I use is simply:

ruff check .
arid .

Ruff handles the broad linting work. Arid handles duplication.

That separation is useful beyond performance. A focused tool can make its configuration, reporting, CI behavior, and machine interfaces specifically about the problem it's solving rather than accommodating an entire linting framework.

How Much Faster?

I don't think "written in Rust" is a benchmark, so I maintain a pinned performance campaign for Arid.

For Arid 2.0, I compared it with Pylint 4.0.6 while isolating Pylint's duplicate-code functionality rather than comparing Arid with an entire Pylint lint run.

Running serially, the results were:

Requests — 191.19x faster

Pydantic — 219.06x faster

Polaris — 249.68x faster

Those numbers aren't meant to establish that Pylint itself is "200x slower." That would be an unfair comparison because Pylint performs many checks Arid doesn't even attempt.

They answer a much narrower question:

If the job is duplicate-code detection, what does the focused implementation cost compared with Pylint's implementation of that job?

That's the comparison that mattered to me.

What Arid Actually Detects

There is another important qualification.

"Duplicate code" can mean several different things.

At one extreme is exact textual duplication. At the other are sophisticated clone detectors trying to recognize code that is structurally or semantically similar despite substantial differences in its source.

Arid isn't trying to solve the entire code-clone research problem.

It detects exact duplicated Python source after configurable Python-aware normalization. Depending on the configuration, comments, docstrings, imports, and function signatures can be excluded from duplicate identity.

That makes it much closer in purpose to the R0801 workflow I wanted to replace.

If you're looking for semantic clone detection—two differently written implementations that happen to do the same thing—Arid isn't the tool for that.

I think being explicit about that boundary is important. A focused tool is only useful if its focus matches the problem you actually have.

What About an Existing Codebase Full of Duplication?

This is where replacing a checker and actually adopting one become different problems.

Suppose you run duplicate detection on a mature project for the first time and discover 300 existing findings.

Technically, the tool worked.

Practically, you've just created 300 reasons for your team not to enable it in CI.

You could fix everything before enforcing the check, but that's often unrealistic. You could ignore duplicate detection entirely, but then new duplication continues accumulating.

The more useful approach is a baseline.

With Arid, existing findings can be recorded as accepted debt. CI can then reject new duplication without requiring you to eliminate everything that existed before the check was introduced.

As existing duplication gets refactored, stale baseline entries can be identified and pruned.

That changes adoption from:

fix all existing duplication
        ↓
enable the check

into:

record existing duplication
        ↓
prevent new duplication
        ↓
improve old duplication over time

For mature codebases, I think the second model is considerably more realistic.

I'll cover that workflow separately because it turns out to be a more general engineering problem than just configuring a duplicate-code checker.

Using It in CI

For a basic local check, Arid doesn't require much:

uv tool install arid
arid .

Arid 2.0 also has an official GitHub Action for CI:

- uses: sponge-b0b/arid@v2.0.0
  with:
    paths: .

V2 can produce text, JSON, Markdown, and SARIF reports, including multiple representations from a single analysis.

That matters if, for example, you want readable output for developers, structured JSON for another tool, and SARIF for code scanning. The source doesn't need to be analyzed independently for every consumer.

Again, none of that makes duplicate detection inherently better. It makes the detector easier to incorporate into the systems surrounding it.

Should You Replace Pylint R0801?

Not necessarily.

If you're already using Pylint extensively, R0801 performs well enough for your repository, and you like its behavior, replacing it solely because another implementation is faster may accomplish very little.

I'd consider a dedicated alternative when the situation looks more like this:

• you've moved most linting to Ruff;

• duplicate-code detection is one of the remaining reasons you're running Pylint;

• R0801 has become noticeable on your repository;

• you want duplicate detection in a fast local feedback loop;

• you need baseline-based adoption for an existing codebase; or

• you want structured duplicate-code output for CI or other tooling.

That's the situation Arid was built for.

I didn't start the project because I thought Python needed another general-purpose linter. Quite the opposite. I started it because I wanted one useful capability without carrying a general-purpose linter along just to get it.

If that's your problem too, Arid may be useful.

If it isn't, keep using R0801.

The goal isn't to replace a tool that already works for you. It's to avoid running more tooling than the problem actually requires.

Arid is an open-source Python duplicate-code checker written in Rust and designed to complement Ruff.

About the Author

Bob Taylor is a software engineer and architect who builds developer tools and AI systems. He is currently developing Arid and Polaris.

GitHub

I use Pylint's R0801 duplicate-code detection on Polaris, a fairly large Python project I've been building. I also use Ruff, which handles most of the Python linting I care about and handles it very quickly.

Unfortunately, Ruff doesn't detect duplicate code. So every time I wanted that one check, I was back to waiting for Pylint.

Eventually I got tired of waiting and built Arid, a focused Python duplicate-code checker written in Rust. The idea was deliberately simple: do the job I was using Pylint R0801 for, do it accurately, and do it fast enough that I wouldn't mind running it all the time.

The first versions of Arid proved that idea worked. Arid 2.0 is about something different: what does a fast detector need around it before it becomes a tool you can comfortably build into real development workflows? That question ended up defining the release.

The Detector Didn't Need Reinventing

Major versions have a way of encouraging major rewrites: new architecture, new algorithm, new semantics. Everything is better because everything is new.

I didn't do that.

Arid 2.0 uses the same basic detection model as 1.2. It still detects exact duplicated Python source after configurable Python-aware normalization. It still reports DUP001. Comments, docstrings, imports, and function signatures can be excluded from duplicate identity. The detector isn't suddenly trying to find semantically equivalent code or fuzzy AST clones.

That's intentional. The detector was already solving the problem I wanted it to solve, so instead of redesigning the part that worked, I concentrated v2 on the things surrounding it: stable machine contracts, CI integration, baseline management, focused workflows, incomplete-analysis handling, project control, and better support for external tooling.

Arid 2.0 isn't really a new detector. It's the same detector with a much more useful development workflow around it.

Fast Still Matters

None of that would matter much if Arid stopped being fast.

The v2 performance campaign used the same pinned benchmark corpora and Hyperfine methodology used to qualify Arid 1.2. Against Pylint 4.0.6, running serially, Arid 2.0 measured:

Those aren't comparisons against an entire Pylint run. The benchmark isolates Pylint's duplicate-code functionality so the comparison is actually about the job Arid replaces.

I also compared v2 directly against the qualified Arid 1.2 implementation. The additional v2 functionality introduced only low-single-digit overhead across the canonical corpora.

Could I have spent more time trying to recover that few percent? Sure. Would anybody using Arid notice the difference between roughly 219x faster than Pylint and slightly more than 219x faster than Pylint? Probably not. At some point optimization becomes an excellent way to avoid working on things users actually need.

Stable Identity for a Finding

One of those things is identity.

Suppose Arid finds the same duplicated code today and tomorrow, but somebody inserts 20 lines near the top of the file. The physical line numbers changed.

The duplicate didn't.

Or perhaps a file moves to another directory. Maybe the order of occurrences changes, or the same duplicate appears in another file. If external tooling identifies findings using locations, those findings become surprisingly unstable.

Arid 2.0 gives every finding a versioned fingerprint:

arid-finding-v1:sha256:...

That fingerprint identifies the normalized duplicate content independently of path, physical line number, occurrence ordering and multiplicity, structural metadata, output format, and worker mode. The same identity is exposed in SARIF through a versioned partial fingerprint.

This isn't particularly exciting when you're looking at a CLI report. It becomes considerably more useful when a CI system, reporting service, coding agent, or other tool needs to reason about the same finding across multiple runs.

Focus the Report, Not the Analysis

Large projects create another problem. Sometimes I don't care about every duplicate in the repository. I'm working on one package, directory, or file and want to know what's relevant to the thing I'm changing.

The obvious implementation is to scan only that path, but that's wrong for duplicate detection.

Suppose I'm working in:

src/payments/

and some code there duplicates code in:

src/customers/

If I analyze only src/payments/, I've removed half of the evidence.

Arid 2.0 therefore separates what gets analyzed from what gets reported:

arid . --focus src/payments

Arid still performs whole-corpus duplicate detection. Baseline enforcement still happens against the complete result, and only afterward does focus filtering determine which groups are reported. If a focused finding also occurs outside the focused path, those occurrences remain part of the finding.

In other words, focus changes what you ask Arid to show you without changing the corpus Arid uses to determine whether the code is duplicated. That's an important distinction for CI jobs and coding agents operating on a specific part of a larger repository.

Existing Duplicate Debt Is a Lifecycle

Baselines were already part of Arid before v2. The idea is straightforward: perhaps you're introducing duplicate-code enforcement into a mature project that already has 300 duplicate groups.

You could fix all 300 before adopting the tool.

Or don't adopt the tool.

Neither is especially compelling.

A baseline gives you a third option: accept the existing debt temporarily while preventing new duplicate debt from being introduced. Arid 2.0 extends that into an actual lifecycle.

arid . --baseline-status arid-baseline.json

can distinguish accepted duplicate debt, active/new findings, and stale baseline entries. Then:

arid . --prune-baseline arid-baseline.json

removes stale acceptance when the corresponding duplication no longer exists. It never silently accepts new debt.

That gives a team a useful progression:

existing duplication
        ↓
baseline it
        ↓
prevent new duplication
        ↓
refactor existing duplication over time
        ↓
prune stale baseline entries
        ↓
smaller baseline

You don't have to make an old codebase perfect before you're allowed to stop making it worse. I suspect that principle applies to considerably more than duplicate code.

Failure Doesn't Have to Mean "Tell Me Nothing"

Source analysis has another annoying edge case. Imagine scanning 3,000 Python files and one cannot be read, parsed, or normalized. Should the entire analysis disappear?

Sometimes yes. If you're enforcing a complete quality gate, an incomplete analysis cannot be treated as success. But that doesn't mean the useful results from the other 2,999 files need to vanish.

Arid 2.0 adds:

arid . --keep-going --json

Independent source failures are collected while valid files continue through detection. The important part is that Arid doesn't pretend partial analysis is complete analysis.

A report-v4 result explicitly says:

{
  "complete": false
}

and includes structured source errors. The process still exits with operational status 2, and incomplete reports cannot be emitted as SARIF.

The intent is simple: produce as much useful information as you safely can, but be explicit about how complete that information is. A human can inspect the partial result, and an automated consumer can make its own decision. Neither has to guess whether the analysis silently skipped something.

One Scan, Several Consumers

A CI pipeline often wants more than one representation of the same result. Maybe developers want readable console output, the build system wants JSON, GitHub code scanning wants SARIF, and the job summary wants Markdown.

The inefficient answer is to run the analyzer four times.

Arid 2.0 can instead produce multiple outputs from one in-memory report:

arid . \
  --format text \
  --report json=artifacts/arid.json \
  --report markdown=artifacts/arid.md \
  --report sarif=artifacts/arid.sarif

The source isn't reparsed four times and duplicate detection isn't repeated four times. The analysis happens once, and the result is rendered for the consumers that need it.

Obvious in retrospect? Probably. Still worth doing.

Machine-Readable Means Having a Contract

Once other software starts consuming CLI output, "it happens to be JSON" isn't enough.

Arid 2.0 introduces report schema v4 with explicit fields for things like the schema version, tool version, analysis metadata, completion state, structured errors, and finding fingerprints. The schema itself is published:

schemas/report-v4.schema.json

Arid also publishes contracts for capabilities and fatal JSON-mode operational errors. And:

arid --capabilities

allows tooling to discover deterministic build capabilities without first discovering or analyzing a project.

This is partly about ordinary CI integration, but there's another consumer I care about more now than I would have a few years ago: coding agents.

I use AI heavily in my own development workflow. An agent interacting with a tool shouldn't have to scrape human-readable console output and hope a sentence doesn't change in the next release. If we're increasingly going to have software using software on our behalf, the interfaces between those tools need to become more explicit, not less.

JSON gives us a machine-readable format. Publishing the schema tells the consumer what that format actually promises.

Arid Now Has an Official GitHub Action

Of course, the easiest integration is the one you don't have to assemble yourself.

Arid 2.0 ships an official composite GitHub Action:

- uses: sponge-b0b/arid@v2.0.0
  with:
    paths: .

The Action installs the exact Arid release associated with its tag and performs one scan. It can expose core metrics as outputs, write a job summary, and produce SARIF when configured.

Before v2, Arid could certainly be used in CI. Now there's a supported integration that makes doing it considerably simpler.

Sometimes You Want to Analyze Code That Isn't on Disk

Arid normally discovers and analyzes Python files in a project, but editors, coding agents, and other tools frequently have source that doesn't exist on disk yet—or source that differs from what's currently there.

V2 adds virtual Python source through standard input:

cat src/example.py | arid . --stdin-path src/example.py

The virtual source goes through the same Python parser and normalizer as disk-backed source. If an equivalent disk path exists, the virtual version replaces it for that scan. Otherwise it can be added to the corpus when the resolved project context permits it. Arid never writes that source to disk.

That means another tool can effectively ask, "If this were the contents of src/example.py, what duplicates would exist?" without first modifying the working tree. That's useful for editors and automation, and yes, it's particularly useful for coding agents.

Explicit When You Need It

Convention is great until automation needs certainty.

Arid still supports its existing nearest-config behavior, but v2 adds explicit control over project and configuration context:

arid . --config path/to/pyproject.toml
arid . --no-config
arid . --project-root path/to/project
arid . --show-config
arid . --list-files

For somebody running Arid manually in a normal repository, most of this can stay invisible. For CI, monorepos, editor integrations, and agents, being able to ask exactly which project and configuration are being used becomes considerably more important.

The common case can still rely on convenient defaults. The less common cases now have a way to be explicit.

Not Everything Became Public

There was one place where v2 deliberately became less extensible.

Arid is primarily a CLI application, but its Rust crate naturally exposes Rust code too. In v2, I narrowed the semver-supported Rust surface to a small crate-root application API. Implementation modules, detector internals, and reporting internals are no longer promises to downstream Rust consumers.

That decision fits something I've been thinking about a lot lately: every public interface creates an obligation. Once implementation details become supported API, changing your own internals becomes somebody else's breaking change.

If Arid were intended to be a general duplicate-detection framework, that would be a different conversation. It isn't, and not everything another developer could call needs to become something they should depend on.

What Didn't Change May Matter More

For a major release, the compatibility list is almost as important as the feature list.

Arid 2.0 preserves the things ordinary users depend on:

• exact normalized duplicate semantics;

• DUP001;

• normal CLI invocation;

• [tool.arid] configuration;

• normalization behavior;

• source suppression;

• existing baseline-v1 files;

• serial execution by default;

• worker controls;

• the 0 / 1 / 2 exit meanings;

• pre-commit integration;

• supported release platforms.

For CLI-only users who don't consume Arid's machine contracts or Rust internals, upgrading from 1.2 may require no migration work at all.

The intentional breaking changes are concentrated where a major version gives us room to make contracts cleaner: report JSON, SARIF finding identity, and the supported Rust API. That's the kind of major version I prefer: break what you have a good reason to break and leave everything else alone.

I Tried It on Real Projects

I don't want Arid's correctness story to be based entirely on unit tests and a repository containing six carefully selected Python files.

The v2 validation campaign exercised Arid against Black, Django, mypy, Rich, Unicode and space-containing paths, and combinations of the new workflow features.

For equivalent settings, canonical duplicate groups from Arid 2.0 were compared directly with qualified Arid 1.2 results across Black, Django, mypy, and Rich. No detector-semantic regression was found.

Validation also covered focus behavior, baseline-before-focus ordering, virtual-source replacement without disk mutation, controlled malformed source with --keep-going, multi-output on Django, worker determinism, and the published GitHub Action.

Performance matters, but so does knowing that the fast answer is still the right answer.

Where Arid Fits

Arid isn't trying to replace Ruff. Quite the opposite.

My normal mental model is:

ruff check .
arid .

Ruff handles the broad Python linting problem extraordinarily well. Arid handles one problem Ruff currently doesn't: duplicate code.

Because Arid is deliberately focused on that problem, I can make decisions around its detection model, reporting, baselines, and automation without turning it into another general-purpose linter. That's still the philosophy behind v2:

Cleaner contracts. Better automation. Same focused detector.

Try It

With uv:

uv tool install "arid==2.0.0"
arid .

Or with pip:

python -m pip install "arid==2.0.0"
arid .

If you're already using Ruff and want duplicate-code detection, try Arid on a real project. If you're still running Pylint primarily because you need R0801, I'm particularly interested in what you think.

I'm also interested in people integrating static-analysis tools into CI, editors, or coding-agent workflows. A large part of Arid 2.0 exists because once a command-line tool starts participating in larger development systems, speed isn't the only thing that matters anymore.

Arid started because I didn't want to wait for Pylint.

Version 2.0 is what happened after the detector became fast enough that speed stopped being the most interesting problem.

Arid is an open-source Python duplicate-code checker written in Rust. Version 2.0 is available now.

Documentation: Arid 2.0 release notes

About the Author

Bob Taylor is a software engineer and architect who builds developer tools and AI systems. He is currently developing Arid and Polaris.

GitHub

I know. I'm one of them.

Give us a small enough problem and eventually somebody will propose an interface, a factory, a plugin architecture, dependency injection, and perhaps a message bus just in case the three functions need to communicate asynchronously someday.

You know. For flexibility.

I've been thinking about this while building Arid, a Python duplicate-code checker written in Rust.

Arid has a deliberately narrow job:

Detect duplicated Python source code quickly and accurately.

That's it.

It doesn't format Python.

It doesn't sort imports.

It doesn't type-check anything.

It doesn't detect dead code.

It doesn't scan JavaScript.

And unless something changes dramatically, it isn't going to make coffee either.

That narrow scope created an interesting architectural question:

How much architecture does a small tool actually need?

My answer has increasingly become:

Enough to make the current problem clean. No more.

That sounds like YAGNI.

It is.

But I think there's a more useful way to look at it.

Every Abstraction Is a Prediction

Suppose Arid had started with this:

Language
    ├── Python
    ├── JavaScript
    ├── TypeScript
    └── ...

Looks reasonable.

Maybe even responsible.

After all, duplicate-code detection isn't inherently a Python problem. Why couple the architecture to Python?

So we introduce something like:

LanguageFrontend
    parse()
    normalize()
    classify()

Python implements it today.

JavaScript can implement it tomorrow.

Look at us. Future-proof already.

Except there is currently no JavaScript version of Arid.

There is no TypeScript version.

There is no requirement for either one.

There isn't even a roadmap item for another language.

What exactly did the abstraction buy us?

It bought us an interface.

It bought us indirection.

It bought us a contract that future implementations now have to fit.

It bought us tests for a generic concept that currently has exactly one implementation.

And perhaps most importantly, it quietly made a prediction:

Arid will need multiple language frontends.

Maybe that's true someday.

Maybe it isn't.

Either way, we've paid for part of that future before we know whether it's coming.

That's the thing about abstractions.

They're not free flexibility. They're bets about where the software is going.

Architecture Has Carrying Costs

The cost of an abstraction isn't just the code required to create it.

Code is usually the cheap part.

The real cost is that somebody has to understand it.

Consider a hypothetical detector architecture:

DuplicateDetector
    ├── ExactDetector
    ├── StructuralDetector
    ├── SemanticDetector
    └── FuzzyDetector

Nice.

Except Arid has one detector.

It performs exact duplicate detection after configurable Python-aware normalization.

There is no structural detector.

There is no semantic detector.

There is no fuzzy detector.

In fact, those are explicitly outside Arid's current scope.

So if I introduce DuplicateDetector, what have I modeled?

Not the software that exists.

I've modeled software I can imagine.

That's a subtle but important difference.

Now every developer reading the code has additional questions:

Why is this an interface?

Are there other implementations?

Can the implementation change at runtime?

Am I expected to add new detectors this way?

What guarantees does the abstraction make?

Which behavior belongs to the interface and which belongs to the implementation?

The abstraction has increased the number of concepts required to understand the system without increasing what the system can do.

That's architectural debt too.

We just don't usually call it that because the code looks clean.

But Don't Just Put Everything in main()

There is an obvious bad interpretation of this argument:

Small software doesn't need architecture.

I don't believe that.

Arid 1.1.0 has an architecture.

The main application pipeline is roughly:

discover
   ↓
read
   ↓
normalize
   ↓
build corpus
   ↓
detect duplicates
   ↓
apply baseline
   ↓
build report
   ↓
render output

Those are real boundaries because they represent different responsibilities in the problem.

File discovery shouldn't know how suffix arrays work.

Duplicate detection shouldn't know how Python comments are parsed.

Normalization shouldn't know how Markdown reports are rendered.

Reporting shouldn't decide what counts as a duplicate.

Those separations aren't predictions about hypothetical future products.

They're properties of the problem Arid solves today.

That's the distinction I care about.

Good architecture separates things that are actually different. Overarchitecture separates things because they might become different someday.

Those are not the same thing.

Concrete Is Not a Dirty Word

At some point, "concrete" became suspicious in software design.

If one module directly calls another module, perhaps we're too tightly coupled.

Better introduce an interface.

But coupling isn't automatically bad.

Incorrect coupling is bad.

Arid's duplicate detector depends on Arid's corpus representation.

Of course it does.

That's the data it detects duplicates in.

The normalization layer produces Arid's normalized representation.

Again: yes.

That's its job.

Those relationships aren't architectural mistakes waiting to be abstracted away.

They're the architecture.

The question shouldn't be:

How do I eliminate coupling?

It should be:

Are these things coupled for a reason that belongs to the domain?

If the answer is yes, hiding that relationship behind another interface doesn't necessarily improve anything.

Sometimes it just makes the coupling harder to see.

One Implementation Is a Clue

I don't subscribe to a hard rule that an interface must always have multiple implementations.

There are legitimate reasons to put a boundary in front of a single implementation.

External systems are an obvious example.

Testing can be another.

A meaningful architectural seam can exist before the second implementation arrives.

But one implementation should at least make you ask a question:

What variation am I modeling?

If the answer is:

Well, someday we might...

I become suspicious.

Someday is responsible for a lot of software.

Someday we'll support another database.

Someday we'll have multiple cloud providers.

Someday this will become a distributed system.

Someday users will write plugins.

Someday we'll support seventeen programming languages.

Maybe.

When someday becomes a requirement, we can design for someday with considerably more information than we have now.

And if the current design makes that future literally impossible without rewriting the entire system, that's worth considering.

But there is an enormous amount of territory between:

Don't make the future impossible.

and:

Implement the future now.

We seem to confuse those surprisingly often.

The Plugin System Nobody Asked For

Plugin systems are one of my favorite examples.

Imagine adding plugins to Arid.

What can a plugin do?

Add a language?

Change normalization?

Replace duplicate detection?

Add output formats?

Filter findings?

Modify configuration?

Now we need a plugin API.

Then we need to decide which internal concepts are public.

Then those concepts need stability guarantees.

Then plugins need version compatibility.

Then failures need isolation.

Then documentation.

Then testing.

Then somebody writes a plugin that depends on behavior we thought was an implementation detail.

Congratulations.

Our tiny duplicate-code checker now has an ecosystem to govern.

For what requirement?

There isn't one.

A plugin architecture would not make Arid more flexible today.

It would create an obligation to remain flexible tomorrow.

That's different.

Generalization Can Make the Current Problem Worse

There's another cost to premature abstraction that bothers me more than the extra code.

It can make the abstraction less correct.

Arid's frontend is Python-aware for a reason.

A comment isn't simply "text following a comment delimiter."

A docstring isn't simply "a string."

A function signature has Python-specific syntax.

Structural context depends on Python syntax.

If I had started by demanding a language-neutral abstraction, I would have needed to decide what all programming languages have in common before I had completely solved the Python problem.

What is a Function in the generic model?

What is a Comment?

What is a Docstring in a language that doesn't have docstrings?

What is StructuralScope across Python, Rust, JavaScript, SQL, and whatever somebody asks for next?

Now we're not merely implementing duplicate detection.

We're designing a theory of programming languages.

All because somebody might want TypeScript someday.

No thanks.

Arid can understand Python correctly.

If another language becomes a real requirement later, then we can compare two concrete implementations and discover which concepts are genuinely shared.

That's usually a much better time to generalize.

The second implementation teaches you things the first one can't.

Duplication Isn't Always Worse Than the Wrong Abstraction

This is where DRY can get us into trouble.

We're trained to see duplication and eliminate it.

I'm literally building a tool that finds duplicated code, so I'm probably supposed to be careful here.

But eliminating duplication by creating the wrong abstraction can be worse than the duplication itself.

Two pieces of code can look similar today and evolve for completely different reasons tomorrow.

Combine them too early and you've coupled their futures.

The same thing happens architecturally.

We see two concepts that might eventually share behavior, so we manufacture a common parent before we understand either one.

Then reality arrives.

One implementation needs a special case.

Then another.

The abstraction starts accumulating flags.

Then optional methods.

Then configuration.

Eventually the "generic" abstraction is mostly a complicated description of the differences it was supposed to hide.

Sometimes duplication is information.

It tells you:

These things look similar.

It does not necessarily tell you:

These things are the same concept.

That's a decision we still have to make.

Extensibility Is a Feature

We often talk about extensibility as though every system should have as much of it as possible.

I don't think that's true.

Extensibility is a product capability.

Like any other capability, it has users, requirements, costs, and tradeoffs.

If third-party developers need to extend your system without modifying it, extensibility may be essential.

If your organization has five implementations behind a stable contract, abstraction may be essential.

If you're publishing a framework whose entire purpose is to support unknown use cases, flexibility may be the product.

But Arid is a CLI that finds duplicate Python code.

Its value isn't proportional to the number of ways it can be extended.

Its value comes from doing its one job correctly, quickly, and predictably.

That changes the architecture I want.

Small Doesn't Mean Crude

Architectural restraint doesn't mean throwing everything into one file until it becomes unbearable.

Look at Arid's internal model.

A prepared file owns its original source, normalized source, normalized lines, and segments.

A normalized line carries things the detector and reporting pipeline genuinely need: its range in normalized text, original source line, whether it's effective, and its structural context and scope.

A duplicate occurrence identifies a file and a normalized range.

A duplicate group contains its effective size and occurrences.

These are small types.

They exist because the domain has those concepts.

That's very different from introducing types whose primary purpose is to make the architecture look sophisticated.

The test I increasingly like is:

Can I explain why this abstraction exists without talking about a hypothetical future?

If I can say:

We need this boundary because parsing Python and detecting repeated normalized sequences are different responsibilities.

Good.

If I say:

We need this because eventually we may support arbitrary parser backends selected dynamically from third-party plugins...

I'm going to need considerably more evidence.

What Happens When Requirements Change?

The obvious objection is:

Isn't this shortsighted? What happens when Arid needs another language?

Then I change the architecture.

Seriously.

Architecture is not a one-time ceremony performed before implementation begins.

It's the structure of a living system.

If Arid someday has a legitimate requirement to support Rust source, I'll have something incredibly valuable that I don't have today:

a second real language implementation.

Then I can look at Python and Rust and ask:

What is actually common?

What must vary?

Where does the boundary belong?

Which assumptions in the Python implementation were language-specific?

Which concepts really are universal?

The abstraction designed from those answers is likely to be better than the one I invent today while staring at a single Python implementation.

Will refactoring cost something?

Of course.

So does maintaining an unnecessary abstraction for three years waiting for a requirement that never arrives.

Architecture is tradeoffs.

There is no option where we pay nothing.

You Can Always Add Code Later

This sounds ridiculously obvious, but software development sometimes behaves as if there's a code shortage.

There isn't.

If a requirement appears later, we're allowed to write more code.

If a second implementation appears, we're allowed to extract an interface.

If users need plugins, we're allowed to design a plugin model.

If the pipeline needs asynchronous execution, we're allowed to introduce it.

If Arid needs another language, we're allowed to refactor the frontend.

We don't receive bonus points for having predicted every requirement five years early.

In fact, predictions made too early can make the real requirement harder to implement because now it has to fit the imaginary one.

The goal isn't to avoid changing the architecture.

The goal is to make the architecture easy to change when reality gives us a reason to change it.

Those are very different objectives.

Build the Architecture You Can Defend

Arid 1.1.0 isn't architecturally small because I don't care about architecture.

It's small because I do.

Its pipeline has boundaries.

Its Python-specific parsing is isolated from duplicate detection.

Its internal representation carries the information downstream stages actually need.

Detection doesn't decide presentation.

Structural metadata describes findings without changing duplicate identity.

Those are architectural decisions.

But there is no generic language framework.

There is no detector hierarchy.

There is no plugin system.

There is no dependency-injection framework.

There is no async runtime.

Not because those things are bad.

Because Arid doesn't currently have problems that they solve.

That's the standard I want to apply more often:

Don't ask whether an abstraction could be useful. Ask what requirement makes it necessary.

If you can't name one, maybe don't build it yet.

Your small tool doesn't need a framework.

It needs an architecture that makes the problem it actually solves obvious.

Build that.

When the problem changes, change the architecture.

Arid is an open-source Python duplicate-code checker written in Rust. This article discusses the architecture of Arid 1.1.0. Its intentionally small application pipeline can be seen in lib.rs, and its core domain representation is in model.rs.

About the Author

Bob Taylor is a software engineer and architect who builds developer tools and AI systems. He is currently developing Arid and Polaris.

GitHub

You have some source files. Find sequences of lines that occur more than once.

How hard could it be?

I asked essentially that question when I started building Arid, a duplicate-code checker for Python.

The answer, as it often is in software, was: it depends on what you mean.

What exactly is a duplicate?

Consider these two functions:

def save_customer(value):
    # Persist the customer
    result = serialize(value)
    database.save(result)

def save_account(value):
    # Store the account
    result = serialize(value)
    database.save(result)

As text, they're different.

As executable logic, they're the same.

If I change the comment again, is it suddenly different code?

What about the function name?

What about a docstring?

Imports?

Blank lines?

Decorators?

Formatting?

At some point duplicate-code detection stops being a string-comparison problem and becomes a language problem.

That was one of the first lessons I learned building Arid.

First, Define "Duplicate"

There are a lot of ways two pieces of code can be similar.

These are obviously identical:

value = calculate()
save(value)

value = calculate()
save(value)

Now change the variable name:

value = calculate()
save(value)

result = calculate()
save(result)

Are those duplicates?

A human can look at them and reasonably say yes.

Arid says no.

That's deliberate.

Arid 1.1.0 detects exact duplicate source after configured normalization. It is not trying to determine whether two pieces of code are semantically equivalent, structurally similar, or suspiciously alike.

Changing value to result changes the code being compared.

Changing a literal from 10 to 20 changes it.

Changing an expression changes it.

That's an important boundary because "find code that means roughly the same thing" is a very different problem from "find source that has actually been duplicated."

The first problem starts taking you toward AST similarity, clone classification, semantic analysis, and eventually a fairly interesting discussion about what "equivalent" even means.

I wasn't trying to solve that problem.

I wanted a fast replacement for the duplicate-code functionality I was using in Pylint.

Pylint's similarity checker already has useful concepts for this. By default, it can exclude comments, docstrings, imports, and function signatures from similarity calculation. Those options are part of Pylint's similarity checker.

Arid keeps that general idea.

The interesting part is figuring out how to do it correctly.

A # Is Not Necessarily a Comment

Let's start with comments.

This looks easy:

value = 42  # this is a comment

Remove everything after #.

Done.

Until this shows up:

value = "# this is not a comment"

Now our sophisticated duplicate-code checker has helpfully converted valid Python source into:

value = "

Excellent.

You can keep adding increasingly clever text-processing rules, or you can ask Python what the thing actually is.

Arid takes the second approach.

Its Python frontend tokenizes the source and identifies tokens whose kind is actually Comment. The comment's source range can then be removed from the representation used for duplicate matching.

So:

value = "# this is not a comment"
other = 42  # actual comment

normalizes to:

value = "# this is not a comment"
other = 42

when comments are ignored.

The distinction seems obvious when you see the example.

But that's the point.

The distinction is obvious because you understand Python syntax.

A text processor doesn't.

A String Is Not Necessarily a Docstring

Docstrings get more interesting.

Consider:

def calculate():
    """Calculate the current value."""
    value = 42
    return value

If docstrings are configured to be ignored, we don't want that string to participate in duplicate identity.

Now consider:

def calculate():
    value = 42
    """This is an ordinary string expression."""
    return value

Should that string disappear too?

No.

They're both string literals.

Only one is a docstring.

The difference isn't the quotes. The difference is where that expression exists in the Python program.

Arid identifies structural docstrings as string-expression statements in the docstring position of a module, class, or function body.

So this:

"""module documentation"""

class Customer:
    """class documentation"""

    def save(self):
        """method documentation"""
        persist()

can have all three docstrings excluded from matching.

But this:

def save():
    persist()
    """ordinary string expression"""
    finish()

keeps the string expression.

You can't make that distinction reliably by looking for triple quotes.

You have to understand the syntax tree.

And now our simple "compare some lines" project has a parser.

That escalated quickly.

Function Signatures Are Worse Than They Look

Ignoring function signatures sounds simple too.

Remove the def line:

def calculate(value):

Except Python doesn't require a function declaration to fit on one line.

It can look like this:

def calculate(
    value: dict[str, tuple[int, int]],
    multiplier: int = 2,
) -> list[int]:
    return transform(value, multiplier)

Or:

async def calculate(
    value: dict[str, tuple[int, int]],
) -> list[int]:
    return await transform(value)

There can be nested brackets, type annotations, default values, and plenty of colons inside those expressions before you reach the colon that actually terminates the function signature.

So "remove everything through the first colon" isn't going to last very long.

Arid walks the parser tokens beginning at def or async, tracks bracket nesting, and finds the colon that terminates the declaration at nesting level zero.

Then it masks that source range.

The body remains.

There's another detail I like here.

Decorators remain significant.

Given:

@transactional
def save(value):
    persist(value)

and:

@cached
def save(value):
    persist(value)

ignoring function signatures doesn't silently erase the decorators.

Arid removes the function declaration.

It doesn't remove everything vaguely associated with the function.

That distinction is easier to maintain when syntax tells you where the boundaries actually are.

Then There Are Imports

Imports have their own collection of small traps.

This:

import os

is easy.

This:

from package import (
    first,
    second,
    third,
)

takes several physical lines.

Imports can also occur inside control flow or functions:

if enabled:
    import optional_backend
    run()

If imports are ignored, Arid removes the import statement while retaining:

if enabled:
    run()

Then somebody writes:

import os; run()

If you remove only the AST range belonging to import os, you're left with:

; run()

Wonderful.

Or:

run(); import os

becomes:

run();

So Arid's frontend deliberately consumes the appropriate adjacent semicolon when an ignored statement shares a logical line with retained code.

The result in both cases is:

run()

It's a tiny implementation detail.

It's also exactly the kind of tiny implementation detail that separates "works on my example" from "works on Python source."

Normalization Is Not Parsing

At this point it would be easy to let the parser take over the entire design.

I didn't want that either.

Arid uses Python syntax to answer questions that require knowledge of Python:

• Is this actually a comment?

• Is this string expression actually a docstring?

• What source range belongs to this import?

• Where does this function signature end?

• Is this code associated with a module, class, or function?

Then that knowledge crosses a boundary.

The Python frontend converts parser-specific information into ordinary source ranges and structural regions owned by Arid.

The normalization layer doesn't operate on AST nodes.

The duplicate detector certainly doesn't.

The basic relationship is:

Python source
    ↓
parser + tokenizer
    ↓
source masks + structural regions
    ↓
normalized lines
    ↓
duplicate detector

I think that distinction matters.

Python syntax determines what participates in comparison. It does not perform the comparison.

The parser is there because text alone can't reliably tell me which text should be ignored.

Once that question has been answered, the rest of Arid doesn't need to know which parser answered it.

The Source You Compare Is Not the Source You Report

Normalization creates another problem.

Suppose the original code is:

def calculate(
    first: int,
    second: int,
) -> int:
    # Add the values
    result = first + second

    return result

With comments and signatures ignored, the meaningful normalized representation might effectively be:

result = first + second
return result

That's useful for matching.

It's terrible for reporting if you lose the relationship to the original file.

A developer doesn't want a diagnostic that says:

Duplicate found at normalized line 2.

Normalized line 2 does not exist in the file they're editing.

Arid therefore keeps the original physical source-line location on every normalized line.

The detector works with the normalized representation.

The report maps the result back to the original Python source.

That means ignored comments, docstrings, signatures, imports, and blank lines can disappear from duplicate identity without making the diagnostic point somewhere imaginary.

This sounds like bookkeeping.

It is bookkeeping.

Bookkeeping is architecture when getting it wrong makes the product useless.

Four Lines Isn't Always Four Lines

There's another deceptively small question:

What does --min-lines 4 mean?

Four physical lines?

Four normalized lines?

Four lines containing actual code?

Consider:

values = [
    first,
    second,
]

The closing bracket is a normalized physical line.

But should ] count as one of the four meaningful lines required to declare a duplicate?

Arid distinguishes a normalized line from an effective line.

A line is effective when it contains at least one alphanumeric character or underscore.

So punctuation-only lines can remain part of a repeated source sequence without artificially helping that sequence satisfy the configured minimum duplicate size.

Blank lines don't count either.

That means a reported four-line duplicate means four effective normalized lines satisfied the threshold, even if the corresponding physical source range spans more lines.

Again, this isn't difficult because the algorithm is exotic.

It's difficult because apparently simple words like line turn out to require definitions.

Python-Aware Does Not Mean Semantic

This is probably the most important boundary in Arid's normalization model.

Using a Python parser does not mean Arid performs semantic duplicate detection.

These two blocks:

value = calculate()
save(value)

result = calculate()
save(result)

are not duplicates to Arid.

Nor are:

if ready:
    execute()

and:

if is_ready:
    execute()

They may represent the same pattern.

They may deserve refactoring.

They may even have been created by copy and paste followed by renaming a variable.

Arid still says they're different.

Why?

Because the parser is being used to interpret Python syntax accurately, not to erase meaningful Python source until everything vaguely similar starts matching everything else.

There's a line somewhere between useful normalization and inventing equivalence.

As of Arid 1.1.0, that line sits in a fairly conservative place.

Comments, structural docstrings, imports, and function signatures can be configured out.

What remains must match exactly.

Could Arid eventually detect renamed-variable clones, structural clones, or fuzzy AST similarity?

Sure.

It could also become an IDE, package manager, database, and small accounting system.

The question isn't whether those things can be built.

The question is whether they're the problem Arid is supposed to solve.

For now, they're not.

Description Is Not Identity

Arid does use the Python structure for one other purpose: describing what it found.

A duplicate can be reported as declarative or executable and associated with module, class, or function scope.

For example, repeated class-level assignments might be described differently from repeated executable logic inside functions.

But that metadata doesn't change duplicate identity.

Two blocks don't become equal because they're both executable.

They don't stop being equal because one occurs in a different structural context.

First Arid answers:

Is this source duplicated?

Then it can help answer:

What kind of source did I find?

I deliberately keep those questions separate.

The moment classification starts deciding whether something "really counts" as duplication, the tool starts making domain judgments it doesn't have enough information to make.

Repeated declarative code might be framework boilerplate.

It might be accidental duplication.

It might be exactly what the project wants.

Arid doesn't know.

Neither does the AST.

The Algorithm Wasn't the First Hard Part

Arid ultimately uses a generalized suffix array and longest-common-prefix analysis to find repeated normalized sequences.

That's the algorithmically interesting part of the project, and I'll get into it separately.

But something surprised me while building the tool.

Before you can efficiently find repeated sequences, you have to decide what the sequence is.

That decision contains a surprising amount of the product's behavior.

Do comments matter?

Which strings are docstrings?

Do imports matter?

What is a function signature?

What happens to decorators?

What does a line mean?

How do normalized lines map back to physical source?

Does structural context affect equality?

How much difference is allowed before code stops being "the same"?

Those aren't suffix-array questions.

They're product-definition questions.

And they're language questions.

The duplicate detector can only be as correct as the representation you give it.

Feed it bad normalization quickly and all you've built is a very fast way to produce bad answers.

The Parser Isn't the Point

I started Arid because Pylint's duplicate-code checker was too slow for my workflow.

I expected performance to be the interesting problem.

And performance certainly mattered.

But building the tool reinforced something I keep running into in software architecture:

Before optimizing the solution, define the problem precisely.

"Find duplicate code" isn't precise enough.

"Find exact repeated Python source after removing a configurable set of syntactically understood constructs" is considerably closer.

Once that definition existed, a lot of architectural decisions became easier.

Use Python syntax where Python syntax matters.

Turn that knowledge into a small internal representation.

Keep parser details out of the detector.

Preserve the mapping back to the source developers actually edit.

And don't quietly turn exact duplicate detection into semantic similarity because the parser happens to make that possible.

It turns out duplicate-code detection is harder than comparing text.

Not because comparing text is hard.

Because deciding which text means what is.

Arid is an open-source Python duplicate-code checker written in Rust. This article describes the normalization model in Arid 1.1.0. The implementation is available in normalize.rs, and the Python syntax frontend is in python.rs.

About the Author

Bob Taylor is a software engineer and architect who builds developer tools and AI systems. He is currently developing Arid and Polaris.

GitHub

I was working on Polaris, a fairly large Python project, and doing what I normally do: running code-quality tools against it. Ruff handles most of what I want from a Python linter these days, and it handles it very quickly.

There was one problem.

Ruff doesn't detect duplicated blocks of code.

That isn't an oversight in my configuration. Ruff simply doesn't support project-wide duplicate-code detection today.

Pylint does.

Its R0801 checker has been finding similar code for years, and Pylint also exposes the same capability through its standalone symilar tool. It can ignore comments, docstrings, imports, and function signatures while looking for repeated blocks.

There was just one small problem.

It was slow.

On Polaris, it was really slow.

Eventually I got tired of waiting for it.

So I built Arid.

How Hard Could It Be?

Those may be some of the most dangerous words in software development.

At first glance, duplicate-code detection doesn't sound particularly complicated. Read some files, compare some lines, find the parts that repeat, print them out.

Done.

Except we aren't comparing text files. We're analyzing Python source code.

Suppose these two functions contain the same executable logic:

def save_customer(value):
    # Persist the current customer
    result = serialize(value)
    database.save(result)

def save_account(value):
    # Store the account
    result = serialize(value)
    database.save(result)

Are they duplicates?

If comments are ignored and function signatures are ignored, they probably should be.

Now add docstrings. Imports. Blank lines. Decorators. Multiline function signatures. Parenthesized expressions. Different physical source ranges. Same-file duplicates that overlap one another.

Suddenly "compare some lines" needs a little more definition.

Pylint already has an answer to many of these questions, and I wasn't trying to invent a completely different meaning for duplicate code. Arid started with the intent of Pylint's R0801: find repeated Python source while allowing things such as comments, docstrings, imports, and signatures to be excluded from the comparison.

But I didn't need to reproduce Pylint's implementation.

That distinction turned out to matter.

One Job

Once I decided to build the tool, I had one significant advantage over Pylint.

Arid only needed to do one thing.

Pylint is a general-purpose static-analysis system. Duplicate-code detection is one capability among many.

Arid has exactly one production responsibility:

Find duplicated Python source code.

That's it.

No formatting. No import sorting. No type checking. No complexity analysis. No security scanner. No dead-code detector.

Ruff already does a lot of those things exceptionally well. I have no interest in building a slower, less capable Ruff just so Arid can have a longer feature list.

In fact, one of the rules I settled on for Arid is fairly simple:

If a feature naturally belongs in Ruff, it probably doesn't belong in Arid.

That decision had architectural consequences.

Arid supports one programming language, so it doesn't have a generic language abstraction.

It has one detector, so it doesn't have a detector hierarchy.

It doesn't have a plugin system.

It doesn't have a reporter registry.

It doesn't have a dependency-injection framework.

It doesn't have an async runtime.

And I did not create a parser abstraction just in case I wake up one morning three years from now and decide to replace the parser.

I realize this may be shocking.

The duplicate-code checker somehow manages to function anyway.

Those choices are explicit in Arid's technical architecture. Parser-specific knowledge stays in the Python frontend, while everything downstream operates on Arid-owned data structures rather than parser AST or token types.

That isn't an argument against abstraction.

It's an argument for paying for abstraction when you actually have a requirement for it.

Every abstraction has a cost. More types. More indirection. More concepts somebody has to understand. More extension points that have to remain stable. More opportunities to design for a future that never arrives.

If Arid eventually develops requirements that justify one of those abstractions, then I'll have a reason to build it.

Until then, I have a duplicate-code checker to write.

Small Scope Does Not Mean a Trivial Problem

Keeping the product small didn't make the underlying problem simple.

Arid still has to understand enough Python syntax to distinguish source that participates in duplicate identity from source that may be ignored.

The basic pipeline became:

Python source
    ↓
file discovery
    ↓
Python parse + tokenize
    ↓
Python-aware filtering + structural classification
    ↓
normalized source lines
    ↓
global exact-duplicate index
    ↓
maximal repeated blocks
    ↓
duplicate findings + metrics

The Python frontend uses syntax to identify comments, docstrings, imports, function declarations, structural scope, and their source ranges. It converts that parser-specific information into Arid's internal representation.

After that boundary, the duplicate-detection engine doesn't know what a Python AST node is.

That was deliberate.

The parser is a dependency.

Python is the domain.

Those aren't the same thing.

I want Arid to understand Python, but I don't want the entire application architecture to understand the implementation details of whichever parser happens to provide that information.

There's another important distinction in the design: structural information describes a duplicate, but it does not determine whether two blocks are duplicates.

Arid can tell you that repeated code is executable logic inside a function or declarative code associated with a class. That can be useful when deciding what deserves attention.

But "this looks like framework boilerplate" is a judgment.

So is "this duplicate is harmless."

So is "you need to refactor this."

Arid doesn't pretend to know your application's intent. Its job is to detect the duplication accurately and give you enough objective information to make your own decision.

Exact Means Exact

I also didn't want duplicate findings based solely on the assumption that two hashes being equal means two pieces of source code are equal.

Hashes can be useful internally.

They are not proof.

Arid v1 defines duplication as exact equality after the configured Python-aware normalization. Hashing can be an implementation technique, but equality eventually has to resolve to actual equality.

The global detection engine uses a generalized suffix array with longest-common-prefix analysis to identify repeated normalized source sequences. Arid then turns those candidates into maximal duplicate groups while handling things like overlapping occurrences and deterministic canonicalization.

That probably deserves an article of its own.

The important point here is that once duplicate detection became the entire problem instead of one feature inside a larger tool, I could choose an architecture specifically for that problem.

Fast Was Still the Point

All of this architectural discussion can make the project sound much more philosophical than it actually was.

Let's not rewrite history.

I built Arid because I was tired of waiting for Pylint.

So eventually I had to answer the obvious question:

Is it actually faster?

For Arid 1.0, I built a reproducible benchmark suite around three real Python repositories at fixed revisions: Requests, Pydantic, and Polaris.

The benchmark isolates Pylint's duplicate-code checker, pins tool versions and repository revisions, records environment metadata, uses repeated Hyperfine measurements, and distinguishes comparisons with approximately equivalent semantics from comparisons against tools whose clone-detection semantics differ.

Against Pylint 4.0.6, the published Arid 1.0 measurements were:

On the pinned Polaris benchmark, Arid completed the duplicate scan in about 442 milliseconds.

Pylint took about 117 seconds.

That's the difference between a check I don't mind running and one that interrupts my workflow.

But benchmark numbers need context.

Pylint is not standing still.

Its upcoming 4.1 release includes significant work on the duplicate-code checker: reuse of an already-parsed AST when running inside Pylint, rolling hashes with caching, and changes intended to avoid quadratic behavior in problematic inputs. The Pylint project reports improvements ranging from roughly 1.5× on small projects to 20× on large ones, along with lower memory usage.

Good.

I hope it gets even faster.

Arid doesn't need Pylint to be bad in order for Arid to be useful.

And those benchmark numbers are measurements of particular versions of two pieces of software on particular corpora.

They are not a law of physics.

If Arid's entire identity were "Pylint 4.0.6 is slow," the project would have a fairly short shelf life.

The original motivation was performance.

The resulting tool has a broader reason to exist: focused, Python-aware duplicate-code detection that can sit next to Ruff without requiring a general-purpose linter for that one remaining job.

Performance is still the reason I started.

It just isn't the only engineering property I care about now.

Yes, I Built It With AI

There's another part of the story worth being explicit about.

I built Arid with ChatGPT as an AI coding partner.

The same is true of Polaris.

I use AI extensively in my development workflow. It helps generate code, examine designs, write tests, review changes, reason about problems, and accelerate implementation.

I'm not particularly interested in pretending otherwise.

What I have found, though, is that generating code and engineering software are not the same activity.

The difficult parts don't disappear because an LLM can produce Rust.

You still have to decide what the product is supposed to do.

You have to define the invariants.

You have to recognize a bad abstraction when one appears.

You have to decide whether a result is actually correct.

You have to build validation that doesn't merely prove the implementation agrees with itself.

You have to benchmark honestly.

You have to reject unnecessary code.

And sooner or later, you have to decide that the thing is stable enough to put 1.0.0 on it.

AI changes how I write software.

It does not remove the need to engineer it.

That experience probably deserves an article too.

The Tool I Wanted to Use

Arid began with a very unremarkable engineering problem.

I had a useful tool.

One part of it was too slow for the way I wanted to work.

The tool I preferred to use for the rest of my Python linting didn't provide that capability.

So I wrote the missing piece.

I didn't need another Python linter.

I needed duplicate-code detection that was fast enough that I wouldn't think twice about running it.

That constraint eventually led to a Rust implementation, Python-aware normalization, exact matching, deterministic output, a suffix-array-based detector, a reproducible benchmark suite, and a deliberately small architecture.

But none of those things were the original idea.

The original idea was much simpler:

I was tired of waiting.

Arid is open source and available on GitHub. If duplicate-code detection is part of your Python workflow, give it a try. Feedback, bug reports, and contributions are welcome.

About the Author

Bob Taylor is a software engineer and architect who builds developer tools and AI systems. He is currently developing Arid, a fast Python duplicate-code checker written in Rust, and Polaris, an AI-assisted portfolio intelligence platform.

GitHub