Vibecheck, Rushil Jaiswal

01 The problem

A generation of founders is shipping products built largely by AI coding agents, and a lot of that code goes to production without anyone who could recognise a hardcoded key reading it.

The existing tools do not help these people. Semgrep output assumes you know what a taint source is. The gap is not detection, it is translation.

Vibecheck runs the real scanners, Semgrep, Gitleaks, OSV, and uses an LLM only for the parts LLMs are good at: explaining what a finding means in business terms, and writing the patch. The model is never the thing deciding whether a vulnerability exists.

02 How it works

The engine clones and reads; it never runs the code it is scanning. Scanners are bounded subprocesses, the clone is deleted as soon as findings are written, and only then does an LLM see anything. Schematic, boxes illustrative, not to scale.

03 Scan some code

The heuristic pass, ported line-for-line from engine/app/scanners/heuristics_scanner.py and running in your browser. Paste code. This is the cheap regex layer that runs alongside Semgrep, Gitleaks and OSV, not a replacement for them.

heuristics_scanner · per-line rules · nothing executed

Code to scan

Deliberately lower-confidence: these are candidate findings. In the real pipeline an LLM dedupes everything down to the top eight and writes the plain-English explanation.

04 The parts that were actually hard

Never execute the target repository. A scanner that runs untrusted code is a remote code execution service with extra steps. Vibecheck only clones and reads, with size and file-count caps, and runs each scanner as a bounded subprocess.
Deduping to eight findings. Semgrep on a real repo returns hundreds of results, most of them noise to this audience. Ranking down to the eight that matter, and being willing to drop the rest, is most of the product.
Writing for someone who cannot read a stack trace. Each finding gets what is wrong, why it matters in real-world terms, and a copy-pasteable prompt for an AI coding agent to fix it.
Private repos without holding the keys longer than needed. Signed-in users connect GitHub and the engine clones with the user’s token, scoped to that scan.

Honest limits. Static analysis has a false-positive rate and this pipeline inherits it, then compounds it by having an LLM write confident prose about each finding. The auto-fix PR is a starting point that a human must review. I would not merge one unread, and the product should say that louder than it currently does. It is in free test mode with the billing code present but ungated.