pip-audit vs Safety: Which Python Dependency Scanner Actually Catches What Matters

Hook: I ran pip-audit and Safety against the same deliberately vulnerable requirements.txt in a clean sandbox. One of them worked. The other refused to run at all — and that failure is itself the most useful finding in this post.

The test file

Eight pinned packages, each a real, old, genuinely vulnerable release — not a toy example:

Django==3.2.4
Flask==1.0
Jinja2==2.10.1
PyYAML==5.3.1
requests==2.19.1
urllib3==1.23
cryptography==3.2
Pillow==8.1.0

Running pip-audit

pip-audit resolves your requirements file, checks every package (including transitive dependencies it has to install to resolve the tree) against the OSV database and PyPI's advisory feed, and reports what it finds. Because two of these pins (requests and urllib3) have a real version conflict if you let pip resolve dependencies freely, I ran it with --no-deps, which skips dependency resolution and trusts your pins as-is — the right mode for an already-locked requirements file:

pip install pip-audit
pip-audit -r requirements.txt --no-deps -f json > pip-audit-out.json

It completed in a few seconds. Here's the real output, summarized — unique advisory IDs per package (pip-audit sometimes lists the same advisory twice under different source aliases, so these are deduplicated), with the earliest fixed version it suggested:

Notice idna==2.7 on that list. It's not in requirements.txt at all — it's a transitive dependency that requests pulled in, and pip-audit caught it anyway. That's the single most important thing this run demonstrated: your direct pins are not your whole attack surface. A scanner that only checks what you wrote in requirements.txt misses whatever those packages quietly bring with them.

Running Safety — and what actually happened

This is where the comparison got more interesting than planned. Current Safety (3.8.1) ships as a client for Safety's own hosted vulnerability platform, not a self-contained local scanner:

pip install safety
safety check -r requirements.txt --output json

In this sandbox, that command failed outright: httpx.ConnectError: [Errno -2] Name or service not known — it couldn't resolve Safety's own API host. Rolling back to the last pre-rewrite release, Safety 2.3.5, which historically used a bundled/offline-friendlier vulnerability feed, hit the same wall: Check your network connection, unable to reach the server. Both versions, two different architectures, same outcome — Safety would not produce a single result without reaching a host of its own, while pip-audit, querying PyPI's own JSON API and the OSV database, ran to completion with no extra configuration.

This is a real operational difference, not a coverage nuance: if your CI runner, build agent, or air-gapped environment can reach PyPI (which it almost certainly already needs to, to install anything) but doesn't separately allowlist Safety's service, pip-audit works and Safety silently doesn't. That's worth knowing before you wire either one into a pipeline that gates a merge or a deploy.

What this means for picking a scanner

Based on what each tool actually is, not just this one run:

• pip-audit is free, open-source, and backed by OSV (an open, community-maintained vulnerability database Google helps run) plus PyPI's own advisory data. It needs no account and no API key. In this test it also demonstrated it checks resolved transitive dependencies, not just your direct pins.
• Safety, since its v2→v3 rewrite, is built around a commercial, curated vulnerability database with its own hosted service and login — which can mean richer, more actively curated advisories for paying users, but also means the tool has an external dependency and a cost model that pip-audit simply doesn't.

For a production service where "did this scan actually run" matters as much as "what did it find," that operational gap — zero dependencies vs. a network call to a third party's API — is worth weighing alongside raw advisory counts.

Try it yourself

Point either tool at your own requirements.txt and see what comes back — pip-audit in particular is a five-minute, zero-signup check:

pip install pip-audit
pip-audit -r requirements.txt --no-deps

If you're pinning exact versions (you should be), --no-deps avoids resolver conflicts and audits exactly what you've locked in, transitive dependencies included.

Mumuni Mohammed

Experienced DevOps Engineer proficient in Python, Docker, Kubernetes, CI/CD, and cloud technologies. Specialized in automating processes and building scalable infrastructure for seamless development and deployment.