Caching Data Files for Optimal Performance

For documentation on older releases of Watchman (v0.31.x series), please visit the older docs website in our GitHub repository.

Leverage local data file caching to accelerate Watchman startups, reduce network dependency, and ensure reliable operation even with unstable remote sources. This feature is particularly valuable for large-scale deployments.

By setting INITIAL_DATA_DIRECTORY to a local directory Watchman will look for the following files.

OFAC

  • add.csv - Address
  • alt.csv - Alternate ID
  • sdn.csv - Specially Designated National
  • sdn_comments.csv - Specially Designated National Comments

Download the files from Data Center - SDN List

US Consolidated Screening List

  • consolidated.csv - US Consolidated Screening List

Download the US Consolidated Screening List

EU Consolidated Screening List

  • eu_csl.csv - EU Consolidated Screening List

Download the EU Consolidated Screening List

UK Consolidated Screening List

  • UK_Sanctions_List.csv - UK Sanctions List

Download the UK Sanctions List

UN Consolidated Sanctions List

  • un_consolidated.xml - UN Consolidated Sanctions List

Download from UN Sanctions

FinCEN 311 Special Measures

  • fincen_311.html - FinCEN 311/9714 Special Measures page (parsed for actions)

Point FINCEN_311_DOWNLOAD_URL or place the HTML snapshot in your INITIAL_DATA_DIRECTORY.

Senzing / OpenSanctions formatted lists

For any opensanctions_* or custom Download.Senzing entries, the Location (URL or file://...) is used. When using INITIAL_DATA_DIRECTORY, relative file references or exact filenames referenced in the location can be resolved from that directory for startup without network access. Periodic refreshes will still attempt live fetches unless the location is a stable local file path.

Live downloads with improved reliability

The INITIAL_DATA_DIRECTORY approach above supplies pre-parsed files at startup. For environments that prefer (or require) live downloads with resilience against flaky sources, consider the companion project moov-io/watchman-cache. It acts as a reverse proxy + cache that speaks Watchman’s existing *_DOWNLOAD_TEMPLATE and *_DOWNLOAD_URL variables, handles 302 redirects, and can serve stale content for days during origin outages. See the Download reliability section in the configuration documentation.