Skip to main content

Source: ocean/docs/E2E_TESTING.md | ✏️ Edit on GitHub

E2E Testing with Stagehand

This document explains how to run end-to-end tests on deployed Vercel URLs using Stagehand, an AI-powered browser automation tool.

Overview​

We use Stagehand for E2E testing because it provides:

  • AI-powered element selection and interaction
  • Better handling of dynamic content
  • Comprehensive error reporting
  • Screenshot capabilities
  • Network and console error tracking

Running Tests​

Quick Start​

Test all deployed URLs:

pnpm test:e2e

Run tests in headless mode:

pnpm test:e2e:headless

What the Tests Do​

  1. Environment Check: Verifies that Supabase environment variables are loaded
  2. Form Filling: Uses AI to fill out the signup form with test data
  3. Form Submission: Submits the form and waits for navigation
  4. Error Detection: Captures console errors, network failures, and form validation errors
  5. Screenshots: Takes screenshots before submission and on errors

Test Results​

Test results are saved to:

  • tests/e2e/reports/signup-test-{timestamp}.json - Detailed test results
  • tests/e2e/screenshots/ - Visual captures of form states

Troubleshooting Signup Form Issues​

1. Check Environment Variables​

Visit /api/health on your deployed URL to see:

  • Which environment variables are present
  • Whether Supabase connection is working
  • Debugging information

Example: https://ocean-goldfish.vercel.app/api/health

2. Common Issues​

Form Not Submitting:

  • Environment variables not set in Vercel dashboard
  • Supabase rate limiting (check error messages)
  • Network connectivity issues

Missing Environment Variables:

  1. Go to Vercel Dashboard → Your Project → Settings → Environment Variables
  2. Add:
    • VITE_SUPABASE_URL
    • VITE_SUPABASE_ANON_KEY
  3. Redeploy the project

Validation Errors:

  • Check console for specific field validation failures
  • Review screenshots in tests/e2e/screenshots/

3. Manual Testing Steps​

If automated tests fail, manually verify:

  1. Open the deployed URL
  2. Open browser DevTools (F12)
  3. Check Console for errors
  4. Check Network tab for failed requests
  5. Try submitting the form and observe the response

Advanced Usage​

Testing Specific URLs​

Modify TEST_URLS in tests/e2e/test-signup.mjs:

const TEST_URLS = ['https://your-deployment.vercel.app']

Using BrowserBase (Cloud Testing)​

For running tests in the cloud, set up BrowserBase:

  1. Get API key from https://browserbase.com
  2. Set BROWSERBASE_API_KEY environment variable
  3. Update Stagehand config:
const stagehand = new Stagehand({
  env: 'BROWSERBASE',
  apiKey: process.env.BROWSERBASE_API_KEY,
})

CI/CD Integration​

Add to your GitHub Actions workflow:

- name: Run E2E Tests
  run: pnpm test:e2e:headless
  env:
    BROWSERBASE_API_KEY: ${{ secrets.BROWSERBASE_API_KEY }}

Next Steps​

  1. Add more test scenarios (error cases, edge cases)
  2. Integrate with CI/CD pipeline
  3. Set up scheduled test runs
  4. Add performance monitoring
  5. Create visual regression tests