Depthfirst is finding new security vulnerabilities in open-source software every week. And we believe that finding these vulnerabilities is only valuable if it leads to a safer internet. We created this policy to clarify how we handle these public discoveries: ensuring maintainers have the time they need to fix issues, while ensuring users aren't left vulnerable.

Depthfirst achieved a ~90% improvement on the CyberGym vulnerability-exploitation benchmark by redesigning the system around the model rather than relying on naïve prompting. By adding accurate situational context, real-time runtime instrumentation, and a modular multi-agent architecture, they raised success rates from the historical 20–28% range to 53%. The core takeaway: LLM capability is often bottlenecked by system design, and thoughtfully engineered agents can unlock far more performance than model upgrades alone.

We recently discovered a textbook example of this in Langfuse, a leading open-source LLM engineering platform with 16k stars on Github. A subtle flaw in its background job controls allowed any authenticated user to access highly sensitive administrative functions, creating a significant business risk