Skip to main content

Source: ocean/docs/ebisu-replication-setup.md | ✏️ Edit on GitHub

Ebisu Data Warehouse Integration - Setup Guide

This guide covers the setup and configuration of logical replication between the Ebisu master database (Crunchy Bridge) and tenant databases (Neon).

Prerequisites​

1. Network Configuration​

Crunchy Bridge (Ebisu) Configuration​

  1. Log into your Crunchy Bridge dashboard
  2. Navigate to your Ebisu cluster settings
  3. Go to the "Network" or "Firewall" section
  4. Add inbound rules for Neon's IP ranges:
    • Obtain your Neon project's static IP range from the Neon dashboard
    • Add these IPs to the Crunchy Bridge allowlist
    • Ensure port 5432 (PostgreSQL) is open

Neon Configuration​

  1. Neon databases allow outbound connections by default
  2. If using IP allowlisting on Neon, ensure Crunchy Bridge IPs are allowed

2. Database Configuration​

On Crunchy Bridge (Ebisu)​

  1. Ensure wal_level is set to logical:

    • This can be configured in the Crunchy Bridge dashboard
    • Requires a database restart if changing

  2. Run the replication setup script:

    # Connect to your Ebisu database
    psql $EBISU_CONNECTION_STRING

    # Run the setup script
    \i ebisu-database/scripts/setup_replication_user.sql

  3. Create the publication:

    # Run the migration
    \i ebisu-database/src/db/migrations/0001_setup_logical_replication.sql

3. Environment Variables​

Add these to your Supabase Edge Functions environment:

# Ebisu connection details
EBISU_HOST=your-ebisu-host.db.provider.com
EBISU_PORT=5432
EBISU_DATABASE=ebisu
EBISU_REPLICATOR_PASSWORD=<secure-password-from-setup>

# Encryption key for vault
VAULT_KEY_ID=<your-vault-key-id>

# Optional: Slack webhook for alerts
SLACK_WEBHOOK_URL=https://hooks.slack.com/services/...

Implementation Steps​

Phase 0: Foundation Setup ✅​

  1. Tenant Metadata Registry: Migration created at supabase/migrations/20250102000000_add_tenant_metadata_registry.sql
  2. Provisioning Integration: Updated to populate metadata during tenant creation
  3. Replication User: Script at ebisu-database/scripts/setup_replication_user.sql

Phase 1: Publisher Configuration ✅​

  1. Publication Creation: Migration at ebisu-database/src/db/migrations/0001_setup_logical_replication.sql
  2. Drizzle Integration: Replication manager at ebisu-database/src/replication/index.ts

Phase 2: Subscriber Implementation ✅​

  1. Sync Edge Function: Created at supabase/functions/sync-tenant-data/
    • Handles initial data seeding
    • Creates subscriptions
    • Manages replication lifecycle
  2. Automatic Setup: Integrated into tenant provisioning flow

Phase 3: Security Hardening ✅​

  • Read-only constraints automatically applied to replicated tables
  • Tenant applications cannot modify master data

Phase 4: Monitoring & Alerting ✅​

  1. Monitor Edge Function: Created at supabase/functions/monitor-replication/
  2. GitHub Action: Scheduled job at .github/workflows/monitor-replication.yml
  3. Slack Alerts: Configured for replication issues

Testing the Setup​

  1. Create a new tenant:

    # This will automatically trigger replication setup
    curl -X POST $SUPABASE_URL/functions/v1/provision-tenant-resources \
      -H "Authorization: Bearer $SERVICE_ROLE_KEY" \
      -d '{"region": "us-east-1"}'

  2. Check replication status:

    curl -X POST $SUPABASE_URL/functions/v1/sync-tenant-data \
      -H "Authorization: Bearer $SERVICE_ROLE_KEY" \
      -d '{"tenantId": "...", "action": "check"}'

  3. Monitor health:

    curl $SUPABASE_URL/functions/v1/monitor-replication \
      -H "Authorization: Bearer $SERVICE_ROLE_KEY"

Troubleshooting​

Common Issues​

  1. Connection Failed

    • Verify network connectivity between Crunchy Bridge and Neon
    • Check firewall rules on both sides
    • Ensure replicator user has correct permissions

  2. Subscription Creation Failed

    • Check if tables exist on both publisher and subscriber
    • Verify publication exists on Ebisu
    • Ensure no existing subscription with same name

  3. Replication Lag

    • Check network latency between regions
    • Monitor Ebisu database load
    • Consider increasing resources if needed

Recovery Procedures​

Reset a failed subscription:

curl -X POST $SUPABASE_URL/functions/v1/sync-tenant-data \
  -H "Authorization: Bearer $SERVICE_ROLE_KEY" \
  -d '{"tenantId": "...", "action": "reset"}'

Security Considerations​

  1. Connection Strings: Always encrypted using Supabase Vault
  2. Network: Use IP allowlisting on both sides
  3. Permissions: Replicator user has minimal SELECT-only access
  4. Monitoring: Automated alerts for any issues

Future Enhancements​

  1. Schema Evolution: Tools for managing schema changes across all subscribers
  2. Selective Replication: Allow tenants to choose which data sets to replicate
  3. Performance Optimization: Connection pooling for replication checks