Audit.sh orchestrates specialized LLMs and static analysis for Web3 security

A hands-on evaluation of audit.sh, a Web3 security harness coordinating ChatGPT 5.5, Qwen-3-480B, Codex, and GLM-5-Turbo alongside classic tools like Slither and Mythril.

For solo Web3 bounty hunters and smart contract auditors, audit.sh offers a pragmatic CLI framework that bridges the gap between traditional static analysis and LLM-driven reasoning. Skip this if you are looking for an automated, push-button auditor; it is designed as an interactive workbench. The bottom line is that audit.sh succeeds not by replacing tools like Slither, but by using specialized LLMs to interpret their output and trace complex vulnerability paths.

Methodology

This v0 review draws on the founder's published claims at https://dev.to/forge_w3s/ai-human-collaboration-building-auditsh-3nej; independent benchmarks are pending. Our evaluation covers the architectural design of the multi-LLM harness, the integrated CLI commands, and the specific roles assigned to ChatGPT 5.5, Qwen-3-480B-coder, Codex, and GLM-5-Turbo as described by the developer under the Forge Web3 Security banner. This review does not cover independent performance metrics, long-term false-positive rates in production, or edge-case handling of highly obfuscated Solidity codebases. Update cadence: we will re-test this tool when public repositories or installable binaries are released, allowing us to run standardized vulnerability detection benchmarks against known exploit sets.

What it does

Orchestrating the multi-LLM security team

The core of audit.sh is its poly-AI architecture. Instead of relying on a single model, the platform routes tasks to specialized LLMs. ChatGPT 5.5 acts as an ethical guardrail and quality controller, ensuring scans remain in-scope. Qwen-3-480B-coder handles deep code analysis and multi-function vulnerability tracing. Codex acts as an anomaly peer-reviewer, while GLM-5-Turbo serves as the primary co-developer for the platform's own architecture.

Interactive hunter-mode manual workflows

The CLI features a dedicated hunter-mode designed for manual auditing. This mode guides the auditor through a structured workflow. It begins with mapping the codebase using Slither summaries, then moves to defining trust boundaries by listing external and public mutating functions alongside their access controls. This keeps the human auditor in control while LLMs suggest attack vectors.

Traditional static analysis tool integration

Rather than replacing established security tools, audit.sh wraps them. The audit-scan command runs full Slither and Mythril scans. The recon command pulls contract source code directly from Etherscan, while check performs quick bytecode checks. Additional utilities include leak-scan for local secret detection and vuln-check to display interactive vulnerability checklists.

What's interesting and what's not

The most compelling aspect of audit.sh is its rejection of the single-model approach. Using Qwen-3-480B-coder for deep tracing while reserving ChatGPT 5.5 for ethical scoping is a highly practical design pattern. It acknowledges that frontier models have distinct behavioral profiles.

However, the reliance on Codex feels outdated. With Codex officially deprecated by OpenAI in favor of newer GPT-4 and GPT-4o variants, its inclusion suggests either a legacy codebase dependency or a highly specific, un-reproducible local setup. Furthermore, the integration of GLM-5-Turbo (developed by Z.ai) as a co-developer is interesting, but its role within the live auditing runtime remains vague compared to Qwen's tracing capabilities.

What is missing from the founder's pitch is a clear explanation of how state is managed between these models. If Qwen identifies a potential reentrancy vector, how is that context passed to Codex for peer review without blowing out token budgets or losing critical context? Without a unified state machine, this collaboration risks devolving into isolated, chat-based silos.

Pricing

Pricing snapshot as of June 2026: audit.sh is currently a personal, proprietary tool built under the Forge Web3 Security banner. There is no public pricing tier or commercial SaaS offering available at this time. The underlying LLM API costs (OpenAI, Qwen Cloud, and Z.ai) are borne directly by the operator.

Verdict

We recommend audit.sh for solo smart contract auditors who want to build their own multi-model CLI harnesses. It is not suitable for enterprise security teams looking for an out-of-the-box, turn-key SaaS platform. The tool's strength lies in its structural blueprint: combining deterministic static analysis (Slither, Mythril) with probabilistic LLM reasoning. If you are comfortable managing your own API keys and orchestrating multiple models, this architecture represents the correct path forward for AI-assisted security auditing.

What we'd test next

In a v2 benchmark, we would run audit.sh against a suite of historically exploited Ethereum smart contracts. We want to measure the exact recall rate of Qwen-3-480B-coder on reentrancy and flash-loan attack vectors compared to standard Slither runs. We would also test the latency and API cost overhead of running multiple models in parallel during a single hunter-mode session.

The investor read

The architecture of audit.sh highlights a broader shift in developer tooling: the transition from single-model wrappers to multi-LLM orchestrators. For venture investors, this signals that the value in AI developer tools is moving away from proprietary model access and toward workflow orchestration and deterministic tool integration. A company building a commercialized version of this pattern would be highly investable if they can prove a significant reduction in false positives compared to legacy static analysis. However, audit.sh itself appears to be a highly specialized, bootstrapped utility for solo bounty hunters, serving as a proof-of-concept for the poly-AI security stack rather than a venture-scale platform.

Pull quote: “The core of audit.sh is its poly-AI architecture.”