Cariddi: Take a list of domains, crawl urls and scan for endpoints, secrets, api keys, file extensions, tokens and more

🧭 Overview

Cariddi is a fast, pipeline-first web crawler written in Go by edoardottt. You hand it a list of live hosts on stdin, and it crawls each one hunting for the things a tester actually cares about: endpoints, secrets, hidden parameters, juicy file extensions, error messages, and useful information — all in a single pass.

What makes cariddi stand out is that it’s designed to be a stage in a chain, not a standalone GUI toy. It reads from stdin, writes clean output (including JSON), and every feature is a flag you can toggle. Drop it after subfinder | httpx and it becomes the “deep look” step that turns a list of live domains into a list of things worth attacking.

| 🔐 OffSec Tip
Cariddi’s philosophy is “take a list of domains, crawl them, find sensitive stuff.” If your recon already ends in a list of live hosts, cariddi is the natural next command.

Repo: github.com/edoardottt/cariddi

📦 Installation

Golang (recommended — always latest):

1

go install -v github.com/edoardottt/cariddi/cmd/cariddi@latest

Homebrew:

1

brew install cariddi

Snap:

1

sudo snap install cariddi

Arch (pacman):

1

pacman -Syu cariddi

NixOS:

1

nix-shell -p cariddi

From source (Linux):

1
2

git clone https://github.com/edoardottt/cariddi.git
cd cariddi && go get ./... && make linux

From source (Windows):

1
2

git clone https://github.com/edoardottt/cariddi.git
cd cariddi && go get ./... && .\make.bat windows

Verify:

1

cariddi -version

🚀 Basic Usage

Cariddi always reads targets from stdin — one URL per line. That’s the whole interface.

1
2
3
4
5

# Single target
echo https://example.com | cariddi

# A list of targets
cat urls.txt | cariddi

Plain crawl with no flags just walks the site and prints discovered URLs. The power comes from turning on hunt modes.

| 🔐 OffSec Tip
Because input is always stdin, cariddi composes with everything. There is no “target file” flag — piping is the API.

🚩 All CLI Flags

🎯 Hunt modes (what to look for)

🧩 Custom pattern files

⚙️ Crawl configuration

🚫 Ignore / scope control

🪪 Headers & User-Agent

📤 Output

ℹ️ Info

| 🔐 OffSec Tip
Run cariddi -examples any time you forget a flag — the tool ships with its own quick reference.

📖 Every Flag Explained (the ones that matter)

🌐 -intensive — go wide

Normal crawling stays on the exact host you fed it. -intensive expands scope to the whole subdomain tree (*.target.com). Use it when your target’s root domain is in scope, not just one host.

1

echo https://target.com | cariddi -intensive

🧃 -ext [1-7] — juicy files by aggressiveness

The number is a sensitivity dial, not a category. 1 returns only the most interesting extensions (.env, .git, .sql, backups…); 7 returns a much broader (noisier) set.

1

cat urls.txt | cariddi -ext 2

🔑 -s / -e — secrets and endpoints

-s runs cariddi’s secret regexes over every response body. -e extracts endpoints (including those buried in JS). They stack.

1

cat urls.txt | cariddi -s -e

📏 -md — depth control

Without a depth cap, cariddi follows links as far as they go. On big sites that’s slow. Cap it:

1

cat urls.txt | cariddi -md 2

🚧 -i / -it / -ie — staying in scope

Keep the crawler out of logout links, huge media, or out-of-scope paths:

1

cat urls.txt | cariddi -i "logout,signout,static" -ie "png,jpg,gif,woff,svg"

🧪 Practical Examples

Crawl and hunt secrets:

1

cat urls.txt | cariddi -s

Endpoints with your own custom endpoint list:

1

cat urls.txt | cariddi -e -ef endpoints_file.txt

Grab juicy files across a target:

1

cat urls.txt | cariddi -ext 2

Faster, but polite (200 workers with a 2s delay):

1

cat urls.txt | cariddi -c 200 -d 2

Route everything through Burp for inspection:

1

cat urls.txt | cariddi -proxy http://127.0.0.1:8080

Authenticated crawl:

1

cat urls.txt | cariddi -headers "Cookie: session=abc123;;Authorization: Bearer eyJ..."

🧠 Advanced Examples

🌊 The “everything” pass

Turn on every hunt mode at once for a full deep-dive on a small, in-scope list:

1

cat urls.txt | cariddi -s -e -err -info -ext 2 -c 50

🧩 Custom secret patterns

Create a file of regexes (one per line) for the client’s internal token formats:

1
2

(?i)svc_[a-z0-9]{32}
(?i)internal-api-[a-f0-9]{40}

Then:

1

cat urls.txt | cariddi -s -sf custom_secrets.txt

🥷 Stealthy crawl

Random UA, delay, capped concurrency, capped depth:

1

cat urls.txt | cariddi -rua -d 3 -c 5 -md 2

💾 Cache to resume

-cache writes to .cariddi_cache so re-runs don’t re-fetch everything:

1

cat urls.txt | cariddi -s -e -cache

🎯 Real Pentest Workflow

The canonical recon chain — subdomains → live hosts → deep crawl → attack surface:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

# 1. Enumerate subdomains
subfinder -d target.com -all -silent > subs.txt

# 2. Keep only what's actually live and serving HTTP
cat subs.txt | httpx -silent -mc 200,301,302,403 > live.txt

# 3. Deep crawl for secrets, endpoints, errors, info, juicy files
cat live.txt | cariddi -s -e -err -info -ext 2 -c 50 -json > cariddi.json

# 4. Slice the JSON into actionable lists
jq -r 'select(.secret != null) | .secret'   cariddi.json > secrets.txt
jq -r 'select(.endpoint != null) | .url'     cariddi.json > endpoints.txt

# 5. Fuzz the parameters/endpoints you found
ffuf -w endpoints.txt -u https://target.com/FUZZ -mc 200,301,302,403

| 🔐 OffSec Tip
Run cariddi with -json in any automated workflow. Human-readable output is for eyeballing; JSON is for piping into jq and the next tool.

🤖 Automation Examples

Reusable shell function:

1
2
3
4
5
6
7

crd() {
  # usage: crd target.com
  subfinder -d "$1" -all -silent \
    | httpx -silent -mc 200,301,302,403 \
    | cariddi -s -e -ext 2 -c 50 -json \
    | tee "cariddi_$1.json"
}

Loop over a whole scope file, one program per line:

1
2
3
4

while read domain; do
  echo "[*] $domain"
  echo "https://$domain" | cariddi -s -e -ot "results/$domain"
done < scope.txt

🧾 JSON Output Explanation

With -json, cariddi emits structured records to stdout instead of plain text. Each finding carries the URL it came from plus a field describing what kind of finding it is — a matched secret, an endpoint, an error string, an info leak, or a juicy-extension URL. That structure is what lets you split one crawl into many targeted wordlists with jq (see below).

For reporting, -oh results.html gives you a shareable HTML report, and -ot results writes a plain TXT log — both can run alongside -json.

🔧 jq Filtering Examples

1
2
3
4
5
6
7
8
9
10
11

# All discovered URLs
cariddi ... -json | jq -r '.url'

# Only the secrets
cariddi ... -json | jq -r 'select(.secret != null) | .secret'

# Only endpoints
cariddi ... -json | jq -r 'select(.endpoint != null) | .url' | sort -u

# Everything that looked like an error (potential info leak)
cariddi ... -json | jq -r 'select(.error != null) | {url, error}'

| 🔐 OffSec Tip
Field names in the JSON can vary by cariddi version. Pipe one record through jq '.' first to see the exact keys before you script against them.

🗡️ Combine with Katana

Katana is a heavier crawler with headless + JS support. A common pattern is to let Katana collect the URL surface, then hand cariddi the job of hunting over it:

1

katana -u https://target.com -d 3 -silent | cariddi -s -e

Or the reverse — use cariddi as the discovery crawler and Katana as a second, JS-aware pass. Either way they complement rather than duplicate.

🕰️ Combine with gau

Feed cariddi historical URLs from Wayback/Common Crawl so it hunts over old, forgotten paths too:

1

gau --subs target.com | cariddi -s -e -ext 2

📡 Combine with httpx

httpx is the gatekeeper — only send cariddi hosts that actually respond, so you don’t waste crawl time on dead entries:

1

subfinder -d target.com -silent | httpx -silent -mc 200,301,302,403 | cariddi -s -e

📜 Combine with subjs

cariddi already parses JS, but subjs can pre-collect script URLs from a host list to widen coverage before the crawl:

1

cat live.txt | subjs | cariddi -s -e

💥 Combine with ffuf

Turn cariddi’s discovered endpoints into a targeted fuzz wordlist:

1
2
3
4
5

cat live.txt | cariddi -e -json \
  | jq -r 'select(.endpoint != null) | .url' \
  | sed 's#https\?://[^/]*/##' | sort -u > paths.txt

ffuf -w paths.txt -u https://target.com/FUZZ -mc 200,204,301,302,403

⚡ One-liners

1
2
3
4
5
6
7
8

# Full chain: subs -> live -> deep crawl -> JSON
subfinder -d target.com -silent | httpx -silent | cariddi -s -e -ext 2 -json > out.json

# Just the secrets, from a domain, in one line
gau --subs target.com | cariddi -s -json | jq -r 'select(.secret!=null).secret' | sort -u

# Stealthy, proxied through Burp
cat urls.txt | cariddi -rua -d 3 -c 5 -proxy http://127.0.0.1:8080

⚠️ Common Mistakes

• No stdin. cariddi has no -u/-l target flag; it only reads stdin. cariddi https://x.com does nothing useful — you must echo or cat into it.
• Skipping httpx first. Crawling dead or non-HTTP hosts wastes time. Filter to live hosts before piping in.
• -intensive on out-of-scope roots. It expands to *.target.com. Make sure the whole subdomain tree is in scope before you use it.
• Concurrency too high, no delay. -c 200 with no -d will get you rate-limited or blocked. Tune to the target.
• Grepping JSON. Use -json + jq, not -plain + grep, when you need to script against results.
• Assuming -ext 7 is “best”. Higher numbers mean more (noisier) extensions, not more valuable ones. Start at 1–2.

💡 Pro Tips & Performance

• Tune the trio: -c, -d, -t. Concurrency, delay, and timeout together decide speed vs. stealth vs. reliability. On a fragile target: -c 5 -d 2 -t 15. On a beefy CDN-backed one: -c 100.
• Always cap depth in automation. -md 2 or -md 3 keeps runtime bounded on huge sites.
• Use -cache for iterative work. Re-running the same scope while tuning flags? Cache saves you the re-fetch.
• Ignore aggressively. -i "logout,signout" -ie "png,jpg,css,woff,svg" cuts noise and avoids self-inflicted session kills.
• -rua for blends. Random User-Agent per request helps avoid trivial UA-based blocking.
• Store responses when it matters. -sr keeps raw responses so you can re-analyze offline without re-hitting the target.

🏁 Conclusion

Cariddi earns its place as the “deep look” stage of a recon pipeline. It doesn’t try to be a subdomain tool or an HTTP prober — it assumes you already have a list of live hosts and does one job extremely well: crawl them and surface the sensitive stuff. Secrets, endpoints, parameters, errors, and juicy files, all in one pass, all pipeable as JSON.

Wire it in right after subfinder | httpx, feed the output through jq into ffuf, and you’ve got a repeatable chain that turns a raw scope into a ranked attack surface. Keep a custom secrets file (-sf) per client, respect the target with sane -c/-d values, and let the pipeline do the rest.

| 🔐 OffSec Tip
The best cariddi command is the one that fits inside a pipe. If you’re copy-pasting URLs into it by hand, you’re not using it the way it was built.