Skip to main content

Source: ocean/docs/STAGING_ENVIRONMENT_STRATEGY.md | ✏️ Edit on GitHub

Staging Environment Strategy

This document outlines the strategy for creating a high-fidelity, cost-effective staging environment for the Ocean/Ebisu platform. The approach is based on replicating the production architecture and leveraging existing automation.

Core Principles​

  1. Mirror Production Architecture: The staging environment should mimic the production setup (PostgreSQL Logical Replication from a publisher to subscribers) to ensure issues are caught before they reach users.
  2. Leverage Existing Automation: The ebisu-database repository contains the scripts and documentation that serve as a blueprint. We will adapt, not reinvent, this automation.
  3. Isolate Environments: Staging and production must be completely separate, from the database instances to the authentication provider and API keys.
  4. Cost-Effectiveness: Use low-cost, non-production-grade cloud resources for the staging infrastructure.

Staging Architecture Components​

A complete staging environment requires a parallel version of each piece of the production stack:

  1. Staging Ebisu Publisher DB: A simple PostgreSQL instance that acts as the master database for the staging environment.
  2. Staging Ebisu Subscriber DB: A second PostgreSQL instance that acts as a tenant database, receiving data from the publisher via logical replication.
  3. Staging Supabase Project: A separate, free-tier Supabase project for isolated user authentication and for hosting the staging versions of the database orchestration Edge Functions.
  4. Vercel Preview Deployments: The ocean frontend and API layer will continue to be deployed via Vercel, with preview branches configured to point to the staging backend components.

Actionable Plan​

Step 1: Provision Staging Database Infrastructure​

The first step is to create the core database resources.

  • Action: Provision a small, single cloud VM (e.g., AWS EC2 t3.micro, DigitalOcean Droplet) running two PostgreSQL instances. The existing ebisu-database/docker-compose.yml provides a perfect template for this using Docker.
    • Instance 1: staging-ebisu-publisher
    • Instance 2: staging-ebisu-subscriber
  • Justification: This provides the necessary database infrastructure at a very low cost, perfectly simulating the separate publisher/subscriber model.

Step 2: Adapt the Publisher Setup Script​

The ebisu-database repository contains a script for setting up the production Crunchy Bridge publisher. We will adapt this for our generic staging publisher.

  • Action: Create a new script, ebisu-database/scripts/configure-staging-postgres.sh.
  • Implementation: This script will perform the same logical steps found in the REPLICATION_SETUP.md's "Manual Setup" section, using standard psql commands:
    1. Ensure the PostgreSQL instance's wal_level is set to logical.
    2. Connect to the staging-ebisu-publisher database.
    3. CREATE ROLE replicator WITH LOGIN REPLICATION PASSWORD '...';
    4. CREATE PUBLICATION ebisu_master_data FOR ALL TABLES;
    5. GRANT SELECT ON ALL TABLES IN SCHEMA public TO replicator;
  • Automation: This script should be integrated into a CI/CD pipeline that runs whenever the staging branch of ebisu-database is updated.

Step 3: Deploy and Configure Staging Services​

With the databases ready, the application and orchestration layers can be configured.

  1. Create Staging Supabase Project:

    • Action: Create a new, free-tier Supabase project named ocean-staging-auth.
    • Configure: Deploy the existing Edge Functions (provision-tenant-resources, monitor-replication, etc.) to this project.

  2. Set Staging Environment Variables:

    • Action: In both the new Supabase project's secrets and the Vercel project settings for "Preview" deployments, create a complete set of environment variables for staging. These must be kept separate from production keys.
    # Staging Database
    STAGING_EBISU_HOST=<IP of your staging VM>
    STAGING_EBISU_PORT=5432
    STAGING_EBISU_DATABASE=staging_ebisu_publisher
    STAGING_EBISU_REPLICATOR_PASSWORD=<your new replicator password>
    STAGING_EBISU_CERT_PEM=<self-signed cert if needed, or empty>

    # Staging Supabase Auth
    NEXT_PUBLIC_SUPABASE_URL=<URL of your staging Supabase project>
    NEXT_PUBLIC_SUPABASE_ANON_KEY=<Anon key of your staging Supabase project>

    # Other Services
    STRIPE_API_KEY=<Stripe Test Mode Key>

End-to-End Staging Workflow​

The resulting workflow provides a seamless development and testing experience:

  1. DB Schema Change: A developer merges a migration into the staging branch of ebisu-database. The CI/CD pipeline automatically applies this migration to the staging-ebisu-publisher database.
  2. App Feature Change: A developer opens a pull request in the ocean repository.
  3. Vercel Deployment: Vercel builds a preview deployment. It automatically uses the STAGING_* and NEXT_PUBLIC_SUPABASE_* variables configured for preview builds.
  4. Testing Replication: The developer can make an API call to the provision-tenant-resources endpoint on the staging Supabase instance. This function connects the staging-ebisu-subscriber to the staging-ebisu-publisher, and logical replication begins.
  5. Validation: The Vercel preview deployment can now read replicated data from the staging-ebisu-subscriber, providing a full, high-fidelity test of the entire system.