Skip to main content

Source: ebisu/docs/adr/0009-intelligence-platform-maritime-domain-awareness.md | ✏️ Edit on GitHub

ADR-0009: Intelligence Platform for Maritime Domain Awareness

Date: 2024-01-07 Status: Accepted

Context​

The Ebisu vessel database has evolved from a simple vessel registry to a complex Master Data Management (MDM) system aggregating data from multiple Regional Fisheries Management Organizations (RFMOs). During implementation, we discovered that:

  1. Different sources report conflicting data about the same vessel
  2. Vessel attributes change over time (names, flags, ownership)
  3. Some vessels appear to intentionally obscure their identity
  4. Data quality varies significantly between sources
  5. "Duplicate" data from multiple sources actually increases confidence

Traditional database approaches treat these as data quality problems to be "fixed." However, in the maritime domain, these patterns often indicate:

  • Illegal, Unreported, and Unregulated (IUU) fishing
  • Sanctions evasion
  • Flag hopping to avoid regulations
  • Ownership obfuscation
  • Other maritime security risks

Decision​

We will architect Ebisu as an Intelligence Platform for Maritime Domain Awareness rather than a simple vessel database. This means:

  1. Conflicts are preserved as intelligence, not resolved automatically
  2. Duplicates are confirmations that increase trust scores
  3. Temporal changes are tracked as vessel history, not overwritten
  4. Trust scoring uses intelligence principles (source reliability, information credibility)
  5. Risk patterns emerge from data conflicts and inconsistencies

Implementation Approach​

1. Admiralty Code System​

Adopt NATO/military intelligence rating system:

  • Source Reliability: A (completely reliable) to F (cannot be judged)
  • Information Credibility: 1 (confirmed) to 6 (cannot be judged)

2. Conflict Intelligence​

-- Conflicts stored with hypotheses
CREATE TABLE vessel_intelligence_assessments (
    assessment_id UUID PRIMARY KEY,
    vessel_uuid UUID,
    field_name TEXT,
    reports JSONB, -- All conflicting reports
    source_reliability CHAR(1), -- A-F
    info_credibility INTEGER, -- 1-6
    competing_hypotheses JSONB,
    deception_probability DECIMAL(3,2),
    risk_factors TEXT[]
);

3. Trust Scoring Evolution​

-- Trust based on:
-- 1. Number of sources (more = better)
-- 2. Source authority levels
-- 3. Data consistency across sources
-- 4. Temporal stability
-- 5. Conflict patterns (many conflicts = lower trust)

4. RFMO Import Standards​

Each RFMO import must follow these stages:

  1. Preserve ALL data - including unusual values like "APPLIED NOT YET RECEIVED"
  2. Match/create vessel - using IMO > IRCS > MMSI > Name hierarchy
  3. Record in vessel_sources - track which sources have this vessel
  4. Store RFMO ID - in vessel_external_identifiers as proper type
  5. Capture attributes - all measurements, types, registrations
  6. Track conflicts - when sources disagree
  7. Calculate trust - based on all evidence

Consequences​

Positive​

  1. Better risk detection - Patterns of deception become visible
  2. Data integrity - No information is lost or "corrected" away
  3. Audit trail - Complete history of what each source reported when
  4. Trust-based MDM - Data quality is quantified, not assumed
  5. Intelligence value - System provides maritime domain awareness, not just vessel lookups

Negative​

  1. Increased complexity - More tables and relationships to manage
  2. Storage requirements - Keeping all historical data and conflicts
  3. Query complexity - Need to consider trust scores and conflicts
  4. User training - Users must understand conflicts aren't errors

Neutral​

  1. Paradigm shift - From "single truth" to "assessed intelligence"
  2. Different workflows - Analysts review conflicts rather than auto-resolution
  3. New capabilities - System can now support risk scoring and pattern analysis

Implementation Notes​

DO NOT​

  • Automatically resolve conflicts
  • Delete "duplicate" data
  • Overwrite historical values
  • Treat disagreements as errors
  • Simplify by picking "most recent"

ALWAYS​

  • Preserve all source data
  • Track temporal changes
  • Score source reliability
  • Analyze conflict patterns
  • Calculate confidence levels

Example: Flag Conflicts​

When ICCAT reports a vessel flag as Panama but IOTC reports it as Liberia:

  1. Store both values with timestamps
  2. Create competing hypotheses (flag change, data lag, dual registration)
  3. Assess based on other evidence (port state data, ownership, history)
  4. Calculate deception probability
  5. Update vessel risk score

References​

  • NATO Admiralty Code System (STANAG 2511)
  • Intelligence Community Directive 206 (Sourcing Requirements)
  • Analysis of Competing Hypotheses (Richards J. Heuer Jr.)
  • Maritime Domain Awareness frameworks (US Coast Guard)
  • ADR-0001: PostgreSQL-First Architecture (data model foundation)
  • ADR-0047: Ebisu Data Warehouse Integration (vessel MDM approach)
  • ADR-0055: Ebisu Unified Import Architecture (RFMO handling)