# ICIJ Offshore Leaks Database

## 1. Summary

The International Consortium of Investigative Journalists (ICIJ) publishes a
combined database of offshore entities from the Panama Papers, Paradise Papers,
Pandora Papers, Bahamas Leaks, and Offshore Leaks. ~800,000+ offshore entities
with their officers, intermediaries, and addresses.

## 2. Access Methods

- **Bulk download (primary):** `https://offshoreleaks-data.icij.org/offshoreleaks/csv/full-oldb.LATEST.zip` (~70 MB ZIP, refreshed periodically)
- **Search UI (human):** `https://offshoreleaks.icij.org/`
- **Auth:** None
- **Note:** The previous Open Refine reconciliation endpoint at
  `/reconcile` now returns 404. ICIJ has removed it. The bulk ZIP is the
  remaining stable access path. The skill's `fetch_icij_offshore.py` caches
  the ZIP locally (default `~/.cache/hermes-osint/icij/`, refreshes after
  30 days) and searches it offline.

## 3. Data Schema

Key fields emitted by `fetch_icij_offshore.py`:

| Column | Type | Description |
|--------|------|-------------|
| `node_id` | int | ICIJ canonical node ID |
| `name` | str | Entity / officer / intermediary name |
| `node_type` | str | entity / officer / intermediary / address |
| `country_codes` | str | Semicolon-separated ISO codes |
| `countries` | str | Country names |
| `jurisdiction` | str | Offshore jurisdiction (BVI, Panama, etc.) |
| `incorporation_date` | str | YYYY-MM-DD |
| `inactivation_date` | str | YYYY-MM-DD (if struck) |
| `source` | str | Panama Papers / Paradise Papers / Pandora Papers / etc. |
| `entity_url` | str | Link to ICIJ page |
| `connections` | str | Semicolon-separated node IDs of related entities |

## 4. Coverage

- Worldwide offshore entity records
- Earliest records: 1970s (Bahamas Leaks). Most data 1990–2018.
- NOT updated in real-time — new leaks added when ICIJ publishes them
- ~810,000 offshore entities + ~750,000 officers + ~150,000 intermediaries

## 5. Cross-Reference Potential

- **SEC EDGAR** ↔ `name` (public companies with offshore arms)
- **USAspending** ↔ `name` (federal contractors with offshore structure)
- **OFAC SDN** ↔ `name` (sanctioned entities using offshore vehicles)

Join key: normalized entity/officer name. `node_id` is canonical for cross-
referencing within ICIJ. Connections graph traversal is in-script (BFS over
`connections`).

## 6. Data Quality

- Offshore entity names sometimes appear in multiple leaks with slight variations
- Officers may be nominees (front persons), not beneficial owners
- Some entries have minimal info (just a name + jurisdiction)
- The connections graph is incomplete — some relationships are documented in
  source materials but not in the structured database
- Inactive/struck-off entities are still included with `inactivation_date`

## 7. Acquisition Script

Path: `scripts/fetch_icij_offshore.py`

```bash
# Search by entity name (case-insensitive substring across the bulk DB)
python3 SKILL_DIR/scripts/fetch_icij_offshore.py --entity "EXAMPLE CORP" \
    --out data/icij.csv

# Search by officer (individual person)
python3 SKILL_DIR/scripts/fetch_icij_offshore.py --officer "SMITH JOHN" \
    --out data/icij.csv

# Search by jurisdiction (filter on cached results)
python3 SKILL_DIR/scripts/fetch_icij_offshore.py --officer "SMITH" \
    --jurisdiction "BRITISH VIRGIN ISLANDS" --out data/icij_bvi.csv

# Force a fresh download (default refresh window is 30 days)
python3 SKILL_DIR/scripts/fetch_icij_offshore.py --entity "EXAMPLE CORP" \
    --force-refresh --out data/icij.csv
```

First call downloads the ~70 MB ZIP under `~/.cache/hermes-osint/icij/`
(or `$HERMES_OSINT_CACHE/icij/`). Subsequent calls reuse the cache for 30 days.

## 8. Legal & Licensing

- Public record as published by ICIJ under explicit publication
- No copyright on the underlying facts (entity names, jurisdictions)
- ICIJ asks for attribution if used in derivative reporting
- **Ethical note**: Presence in this database does NOT imply wrongdoing. Many
  offshore structures are legal. The database is a research tool, not a list of
  criminals.

## 9. References

- Database: https://offshoreleaks.icij.org/
- About the data: https://offshoreleaks.icij.org/pages/about
- Methodology: https://www.icij.org/investigations/panama-papers/
- API hints: Open Refine reconciliation endpoint at `https://offshoreleaks.icij.org/reconcile`