Skip to content
Migrating to data_exports

Migrating to data_exports

ShipHero provides three GraphQL queries for Data Exports:

QueryStatusResponse
lakehero_data_exportDeprecatedFlat file list
recurring_data_exportDeprecatedFlat file list
data_exportsRecommendedFiles grouped by type, run, module, and table

These are separate GraphQL queries, not API versions selected through a version parameter.

Why migrate

The data_exports query allows integrations to:

  • Request FULL, DELTA, or both export types explicitly.
  • Retrieve the latest eligible export or multiple runs within a time range.
  • Filter exports by module.
  • Identify completed exports with a run_id.

The same query works for free and paid accounts. The account plan changes the export frequency, not the API contract.

New query

query DataExports(
  $exportTypes: [DataExportType!]!
  $timestampFrom: ISODateTime
  $timestampTo: ISODateTime
  $modules: [DataExportModuleName!]
) {
  data_exports(
    export_types: $exportTypes
    timestamp_from: $timestampFrom
    timestamp_to: $timestampTo
    modules: $modules
  ) {
    data {
      full {
        run_id
        modules {
          name
          tables {
            name
            files {
              signed_url
              size
              created_at
            }
          }
        }
      }
      delta {
        run_id
        modules {
          name
          tables {
            name
            files {
              signed_url
              size
              created_at
            }
          }
        }
      }
    }
  }
}

Omit both timestamps to request the latest eligible export. Provide timestampFrom and timestampTo to retrieve exports within a specific period.

How to migrate

Migration is the same for lakehero_data_export and recurring_data_export because both use the previous flat response format. Update the integration as follows:

  1. Replace the previous query with data_exports.
  2. Replace date and run_hour_utc with timestamp_from and timestamp_to.
  3. Select FULL, DELTA, or both using export_types.
  4. Read tables from full[].modules[].tables[] or delta[].modules[].tables[].
  5. Use the containing full or delta collection instead of detecting the export type from the filename or URL.
  6. Use run_id to identify runs that have already been processed.
  7. Process every entry in tables[].files. A table may contain multiple Parquet files; process all returned files together to load the complete table.

Data Exports runs start on September 25, 2026. If your migration needs older records, load the most recent FULL export without timestamp filters. See Historical data availability for details.

For recurring DELTA loads, request a time range and process every returned run. Each DELTA export covers an eight-hour window (previously 48 hours), so exports will contain overlapping changes. Requesting only the latest DELTA may skip intermediate exports. Make loads idempotent to handle the overlap.

Frequency and access

  • Free accounts receive weekly FULL exports and DELTA exports every six hours.
  • Paid accounts receive daily FULL exports and DELTA exports every 30 minutes.
  • All Data Export queries require the view:data_exports scope and a user with Public API and Data Exports enabled.
  • Signed URLs expire after 24 hours. Run the query again to generate new URLs when necessary.