Data Sources

Atlas supports multiple data sources simultaneously, allowing you to aggregate events from different providers. This guide covers how to configure and work with each adapter type.

Overview

Data sources are configured in map.config.ts:

map.config.ts

dataSources: [
  {
    id: 'source-1',
    name: 'Source One',
    adapter: 'static',
    adapterConfig: { path: '/data/source1.json' },
    enabled: true,
  },
  {
    id: 'source-2', 
    name: 'Source Two',
    adapter: 'supabase',
    adapterConfig: {},
    enabled: true,
  },
]

Users can toggle individual sources on/off using the source toggle control.

Input coercion (alias handling)

All adapters validate data through the same pipeline. Enum fields (e.g. siteType, eventType, primarySourceTypes) accept canonical values from the type system. In addition, common shorthands are automatically coerced to canonical values before validation, so you do not need to normalize them in your adapter.

Coercion is defined in @/core/schemas via alias maps (e.g. SITE_TYPE_ALIASES, EVENT_TYPE_ALIASES, MEDIA_TYPE_ALIASES). Examples:

siteType: "military_airfield" → "military_air_base", "naval_base" → "military_naval_base"
eventType: "ufo_sighting" → "sighting", "ce1" → "close_encounter"
media[].type: "photo" → "image", "mp4" → "video"

To add a new alias: edit the appropriate alias map in apps/atlas/src/core/schemas/event.ts (e.g. SITE_TYPE_ALIASES). Key = incoming value, value = canonical enum string. The same coercion then applies to static, Supabase, and API adapters.

Static JSON Adapter

The simplest way to get started. Load events from a JSON file.

Setup

Create a JSON file in public/datasets/demo/:

public/datasets/demo/my-events.json

{
  "events": [
    {
      "id": "event-001",
      "status": "published",
      "createdAt": "2024-01-15T00:00:00Z",
      "updatedAt": "2024-01-15T00:00:00Z",
      "temporal": {
        "date": "2024-01-15",
        "dateCertainty": "exact"
      },
      "location": {
        "name": "Phoenix, Arizona",
        "country": "United States",
        "latitude": 33.4484,
        "longitude": -112.074,
        "siteType": "urban",
        "locationSensitivity": "standard"
      },
      "summary": "Description of the event."
    }
  ]
}

Configure the adapter:

map.config.ts

{
  id: 'my-data',
  name: 'My Events',
  adapter: 'static',
  adapterConfig: {
    path: '/datasets/demo/my-events.json',
  },
  enabled: true,
}

Best Practices

Keep files under 5MB for optimal performance
Use meaningful IDs for events
Validate JSON before deploying

Supabase Adapter

Connect to a Supabase PostgreSQL database for production deployments.

Database Setup

Create a Supabase project at supabase.com
Create the events table:

Supabase SQL Editor

CREATE TABLE events (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  status TEXT NOT NULL DEFAULT 'draft',
  created_at TIMESTAMPTZ DEFAULT NOW(),
  updated_at TIMESTAMPTZ DEFAULT NOW(),
  
  -- Temporal
  temporal JSONB NOT NULL,
  
  -- Location
  location JSONB NOT NULL,
  
  -- Optional sections
  summary TEXT,
  classification JSONB,
  witnesses JSONB,
  sensor_evidence JSONB,
  object_characteristics JSONB,
  movement JSONB,
  investigation JSONB,
  response_impact JSONB,
  source_data JSONB,
  environment JSONB,
  media JSONB,
  related_events JSONB
);

-- Enable RLS
ALTER TABLE events ENABLE ROW LEVEL SECURITY;

-- Public read access for published events
CREATE POLICY "Public read access"
  ON events FOR SELECT
  USING (status = 'published');

Set environment variables:

.env.local

NEXT_PUBLIC_SUPABASE_URL=https://xxx.supabase.co
NEXT_PUBLIC_SUPABASE_ANON_KEY=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...

Configure the adapter:

map.config.ts

{
  id: 'database',
  name: 'Production Database',
  adapter: 'supabase',
  adapterConfig: {
    // Uses env vars by default, or override here
    // url: 'https://your-project.supabase.co',
    // anonKey: 'your-anon-key',
  },
  enabled: true,
}

Row Level Security

Always enable RLS to control data access:

Supabase SQL Editor

-- Only show published events publicly
CREATE POLICY "Public can view published"
  ON events FOR SELECT
  USING (status = 'published');

-- Authenticated users can view all
CREATE POLICY "Authenticated view all"
  ON events FOR SELECT
  TO authenticated
  USING (true);

API Adapter

Fetch events from any REST API that returns data in the expected format.

API Requirements

Your API endpoint should:

Accept GET requests
Return JSON with an events array
Support CORS if hosted on a different domain

Configuration

map.config.ts

{
  id: 'external-api',
  name: 'External Data',
  adapter: 'api',
  adapterConfig: {
    endpoint: 'https://api.example.com/events',
    
    // Optional: API key authentication
    apiKey: process.env.API_KEY,
    
    // Optional: custom headers
    headers: {
      'X-Custom-Header': 'value',
    },
  },
  enabled: true,
}

Building an API

If building your own API, return data in this format:

{
  "events": [
    {
      "id": "...",
      "temporal": { "date": "2024-01-15", "dateCertainty": "exact" },
      "location": { /* ... */ },
      // ... other fields
    }
  ],
  "meta": {
    "total": 150,
    "page": 1,
    "pageSize": 100
  }
}

Multiple Sources

Combine multiple sources to aggregate data from different providers:

map.config.ts

dataSources: [
  {
    id: 'official',
    name: 'Official Reports',
    color: '#3b82f6', // Blue
    adapter: 'supabase',
    adapterConfig: {},
    enabled: true,
  },
  {
    id: 'community',
    name: 'Community Reports',
    color: '#10b981', // Green
    adapter: 'api',
    adapterConfig: { endpoint: 'https://community-api.example.com' },
    enabled: true,
  },
  {
    id: 'historical',
    name: 'Historical Archive',
    color: '#8b5cf6', // Purple
    adapter: 'static',
    adapterConfig: { path: '/data/historical.json' },
    enabled: false, // Off by default
  },
]

Each source gets a distinct marker color, and users can toggle them independently.

Creating Custom Adapters

For specialized data sources, you can create custom adapters.

Implementation

src/adapters/my-adapter.ts

import type { DataAdapter, MapEvent, DataSourceConfig } from './types';

export class MyAdapter implements DataAdapter {
  readonly id: string;
  readonly name: string;
  
  constructor(config: DataSourceConfig) {
    this.id = config.id;
    this.name = config.name;
  }
  
  async getMapEvents(): Promise<MapEvent[]> {
    // Fetch and transform your data
    const response = await fetch('...');
    const data = await response.json();
    return this.transformData(data);
  }
  
  async getMapEventById(id: string): Promise<MapEvent | null> {
    const mapEvents = await this.getMapEvents();
    return mapEvents.find(i => i.id === id) || null;
  }
  
  async getFilterOptions() {
    const mapEvents = await this.getMapEvents();
    return {
      countries: [...new Set(mapEvents.map(i => i.location?.country))],
      siteTypes: [...new Set(mapEvents.map(i => i.location?.siteType))],
      // ... other options
    };
  }
  
  private transformData(raw: any[]): MapEvent[] {
    return raw.map(item => ({
      id: item.id,
      status: 'published',
      temporal: { date: item.date, dateCertainty: 'exact' },
      location: {
        name: item.location,
        country: item.country,
        latitude: item.lat,
        longitude: item.lng,
        siteType: 'unknown',
        locationSensitivity: 'standard',
      },
      // ... map other fields
    }));
  }
}

Registration

src/adapters/index.ts

import { MyAdapter } from './my-adapter';

export function createAdapter(config: DataSourceConfig): DataAdapter {
  switch (config.adapter) {
    case 'my-adapter':
      return new MyAdapter(config);
    // ... other cases
  }
}

Adapter Utilities

Atlas exports utility functions for building custom adapters:

Using adapter utilities

import {
  // Filter and extract utilities
  applyMapEventFilters,
  extractFilterOptions,
  
  // Security utilities
  validateSecureUrl,
  checkRateLimit,
  sanitizeErrorMessage,
  
  // Error handling
  AdapterError,
  handleAdapterError,
  classifyError,
  
  // Constants
  QUERY_LIMITS,
  HTTP_HEADERS,
} from '@repo/map/adapters';

Filter Utilities

import { applyMapEventFilters, extractFilterOptions } from '@repo/map/adapters';

// Extract available filter options from events
const options = extractFilterOptions(mapEvents);
// { countries: ['USA', 'UK'], siteTypes: ['urban', 'rural'], ... }

// Apply filters to events
const filtered = applyMapEventFilters(mapEvents, {
  countries: ['USA'],
  siteTypes: ['urban'],
});

Security Utilities

import { validateSecureUrl, checkRateLimit } from '@repo/map/adapters';

// Validate URLs before fetching (prevents SSRF)
if (!validateSecureUrl(endpoint)) {
  throw new Error('Invalid endpoint URL');
}

// Implement rate limiting
if (!checkRateLimit('my-adapter')) {
  throw new Error('Rate limit exceeded');
}

Error Handling

import { AdapterError, handleAdapterError } from '@repo/map/adapters';

try {
  const response = await fetch(endpoint);
  if (!response.ok) {
    throw new AdapterError('FETCH_FAILED', 'Failed to fetch data', {
      status: response.status,
    });
  }
} catch (error) {
  // Classify and handle the error appropriately
  const handled = handleAdapterError(error);
  console.error(handled.userMessage);
  return [];
}

Performance Tips

Large Datasets

For datasets over 1000 events:

Use Mapbox clustering mode
Implement server-side pagination
Consider viewport-based loading

Caching

The data context caches fetched data. To refresh:

const { refresh } = useData();
await refresh();

Error Handling

All adapters should handle errors gracefully:

async getMapEvents(): Promise<MapEvent[]> {
  try {
    const response = await fetch(this.endpoint);
    if (!response.ok) throw new Error('Fetch failed');
    return await response.json();
  } catch (error) {
    console.error('Failed to fetch events:', error);
    return []; // Return empty array on error
  }
}

Data Sources

Atlas supports multiple data sources simultaneously, allowing you to aggregate events from different providers. This guide covers how to configure and work with each adapter type.

Overview

Data sources are configured in map.config.ts:

map.config.ts

dataSources: [
  {
    id: 'source-1',
    name: 'Source One',
    adapter: 'static',
    adapterConfig: { path: '/data/source1.json' },
    enabled: true,
  },
  {
    id: 'source-2', 
    name: 'Source Two',
    adapter: 'supabase',
    adapterConfig: {},
    enabled: true,
  },
]

Users can toggle individual sources on/off using the source toggle control.

Input coercion (alias handling)

Coercion is defined in @/core/schemas via alias maps (e.g. SITE_TYPE_ALIASES, EVENT_TYPE_ALIASES, MEDIA_TYPE_ALIASES). Examples:

siteType: "military_airfield" → "military_air_base", "naval_base" → "military_naval_base"
eventType: "ufo_sighting" → "sighting", "ce1" → "close_encounter"
media[].type: "photo" → "image", "mp4" → "video"

Static JSON Adapter

The simplest way to get started. Load events from a JSON file.

Setup

Create a JSON file in public/datasets/demo/:

public/datasets/demo/my-events.json

{
  "events": [
    {
      "id": "event-001",
      "status": "published",
      "createdAt": "2024-01-15T00:00:00Z",
      "updatedAt": "2024-01-15T00:00:00Z",
      "temporal": {
        "date": "2024-01-15",
        "dateCertainty": "exact"
      },
      "location": {
        "name": "Phoenix, Arizona",
        "country": "United States",
        "latitude": 33.4484,
        "longitude": -112.074,
        "siteType": "urban",
        "locationSensitivity": "standard"
      },
      "summary": "Description of the event."
    }
  ]
}

Configure the adapter:

map.config.ts

{
  id: 'my-data',
  name: 'My Events',
  adapter: 'static',
  adapterConfig: {
    path: '/datasets/demo/my-events.json',
  },
  enabled: true,
}

Best Practices

Keep files under 5MB for optimal performance
Use meaningful IDs for events
Validate JSON before deploying

Supabase Adapter

Connect to a Supabase PostgreSQL database for production deployments.

Database Setup

Create a Supabase project at supabase.com
Create the events table:

Supabase SQL Editor

CREATE TABLE events (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  status TEXT NOT NULL DEFAULT 'draft',
  created_at TIMESTAMPTZ DEFAULT NOW(),
  updated_at TIMESTAMPTZ DEFAULT NOW(),
  
  -- Temporal
  temporal JSONB NOT NULL,
  
  -- Location
  location JSONB NOT NULL,
  
  -- Optional sections
  summary TEXT,
  classification JSONB,
  witnesses JSONB,
  sensor_evidence JSONB,
  object_characteristics JSONB,
  movement JSONB,
  investigation JSONB,
  response_impact JSONB,
  source_data JSONB,
  environment JSONB,
  media JSONB,
  related_events JSONB
);

-- Enable RLS
ALTER TABLE events ENABLE ROW LEVEL SECURITY;

-- Public read access for published events
CREATE POLICY "Public read access"
  ON events FOR SELECT
  USING (status = 'published');

Set environment variables:

.env.local

NEXT_PUBLIC_SUPABASE_URL=https://xxx.supabase.co
NEXT_PUBLIC_SUPABASE_ANON_KEY=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...

Configure the adapter:

map.config.ts

{
  id: 'database',
  name: 'Production Database',
  adapter: 'supabase',
  adapterConfig: {
    // Uses env vars by default, or override here
    // url: 'https://your-project.supabase.co',
    // anonKey: 'your-anon-key',
  },
  enabled: true,
}

Row Level Security

Always enable RLS to control data access:

Supabase SQL Editor

-- Only show published events publicly
CREATE POLICY "Public can view published"
  ON events FOR SELECT
  USING (status = 'published');

-- Authenticated users can view all
CREATE POLICY "Authenticated view all"
  ON events FOR SELECT
  TO authenticated
  USING (true);

API Adapter

Fetch events from any REST API that returns data in the expected format.

API Requirements

Your API endpoint should:

Accept GET requests
Return JSON with an events array
Support CORS if hosted on a different domain

Configuration

map.config.ts

{
  id: 'external-api',
  name: 'External Data',
  adapter: 'api',
  adapterConfig: {
    endpoint: 'https://api.example.com/events',
    
    // Optional: API key authentication
    apiKey: process.env.API_KEY,
    
    // Optional: custom headers
    headers: {
      'X-Custom-Header': 'value',
    },
  },
  enabled: true,
}

Building an API

If building your own API, return data in this format:

{
  "events": [
    {
      "id": "...",
      "temporal": { "date": "2024-01-15", "dateCertainty": "exact" },
      "location": { /* ... */ },
      // ... other fields
    }
  ],
  "meta": {
    "total": 150,
    "page": 1,
    "pageSize": 100
  }
}

Multiple Sources

Combine multiple sources to aggregate data from different providers:

map.config.ts

dataSources: [
  {
    id: 'official',
    name: 'Official Reports',
    color: '#3b82f6', // Blue
    adapter: 'supabase',
    adapterConfig: {},
    enabled: true,
  },
  {
    id: 'community',
    name: 'Community Reports',
    color: '#10b981', // Green
    adapter: 'api',
    adapterConfig: { endpoint: 'https://community-api.example.com' },
    enabled: true,
  },
  {
    id: 'historical',
    name: 'Historical Archive',
    color: '#8b5cf6', // Purple
    adapter: 'static',
    adapterConfig: { path: '/data/historical.json' },
    enabled: false, // Off by default
  },
]

Each source gets a distinct marker color, and users can toggle them independently.

Creating Custom Adapters

For specialized data sources, you can create custom adapters.

Implementation

src/adapters/my-adapter.ts

import type { DataAdapter, MapEvent, DataSourceConfig } from './types';

export class MyAdapter implements DataAdapter {
  readonly id: string;
  readonly name: string;
  
  constructor(config: DataSourceConfig) {
    this.id = config.id;
    this.name = config.name;
  }
  
  async getMapEvents(): Promise<MapEvent[]> {
    // Fetch and transform your data
    const response = await fetch('...');
    const data = await response.json();
    return this.transformData(data);
  }
  
  async getMapEventById(id: string): Promise<MapEvent | null> {
    const mapEvents = await this.getMapEvents();
    return mapEvents.find(i => i.id === id) || null;
  }
  
  async getFilterOptions() {
    const mapEvents = await this.getMapEvents();
    return {
      countries: [...new Set(mapEvents.map(i => i.location?.country))],
      siteTypes: [...new Set(mapEvents.map(i => i.location?.siteType))],
      // ... other options
    };
  }
  
  private transformData(raw: any[]): MapEvent[] {
    return raw.map(item => ({
      id: item.id,
      status: 'published',
      temporal: { date: item.date, dateCertainty: 'exact' },
      location: {
        name: item.location,
        country: item.country,
        latitude: item.lat,
        longitude: item.lng,
        siteType: 'unknown',
        locationSensitivity: 'standard',
      },
      // ... map other fields
    }));
  }
}

Registration

src/adapters/index.ts

import { MyAdapter } from './my-adapter';

export function createAdapter(config: DataSourceConfig): DataAdapter {
  switch (config.adapter) {
    case 'my-adapter':
      return new MyAdapter(config);
    // ... other cases
  }
}

Adapter Utilities

Atlas exports utility functions for building custom adapters:

Using adapter utilities

import {
  // Filter and extract utilities
  applyMapEventFilters,
  extractFilterOptions,
  
  // Security utilities
  validateSecureUrl,
  checkRateLimit,
  sanitizeErrorMessage,
  
  // Error handling
  AdapterError,
  handleAdapterError,
  classifyError,
  
  // Constants
  QUERY_LIMITS,
  HTTP_HEADERS,
} from '@repo/map/adapters';

Filter Utilities

import { applyMapEventFilters, extractFilterOptions } from '@repo/map/adapters';

// Extract available filter options from events
const options = extractFilterOptions(mapEvents);
// { countries: ['USA', 'UK'], siteTypes: ['urban', 'rural'], ... }

// Apply filters to events
const filtered = applyMapEventFilters(mapEvents, {
  countries: ['USA'],
  siteTypes: ['urban'],
});

Security Utilities

import { validateSecureUrl, checkRateLimit } from '@repo/map/adapters';

// Validate URLs before fetching (prevents SSRF)
if (!validateSecureUrl(endpoint)) {
  throw new Error('Invalid endpoint URL');
}

// Implement rate limiting
if (!checkRateLimit('my-adapter')) {
  throw new Error('Rate limit exceeded');
}

Error Handling

import { AdapterError, handleAdapterError } from '@repo/map/adapters';

try {
  const response = await fetch(endpoint);
  if (!response.ok) {
    throw new AdapterError('FETCH_FAILED', 'Failed to fetch data', {
      status: response.status,
    });
  }
} catch (error) {
  // Classify and handle the error appropriately
  const handled = handleAdapterError(error);
  console.error(handled.userMessage);
  return [];
}

Performance Tips

Large Datasets

For datasets over 1000 events:

Use Mapbox clustering mode
Implement server-side pagination
Consider viewport-based loading

Caching

The data context caches fetched data. To refresh:

const { refresh } = useData();
await refresh();

Error Handling

All adapters should handle errors gracefully:

async getMapEvents(): Promise<MapEvent[]> {
  try {
    const response = await fetch(this.endpoint);
    if (!response.ok) throw new Error('Fetch failed');
    return await response.json();
  } catch (error) {
    console.error('Failed to fetch events:', error);
    return []; // Return empty array on error
  }
}

On this page

On this page