Skip to main content
Pipelines is the health view for your data feeds: what synced, when, and whether anything needs attention, so stale or failed data never goes unnoticed. It exists because a feed can fail quietly: actuals can stop arriving and every report keeps showing the last good numbers as if nothing happened. Pipelines makes that visible.

​
What it shows

  • The status of each ingestion run: success, in progress, or failed.
  • Data freshness against an expected cadence, with a clear signal when a feed falls behind.
  • The detail behind a failure, including per-row errors, so you can fix the source and re-run.

​
Freshness monitoring

Each feed has an expected cadence (for example, daily for HiBob people or for GL actuals). Pipelines compares the last successful sync against that cadence and surfaces a freshness state, so a feed that should have arrived but did not is flagged rather than silently served as current.
Freshness is judged against expectations, not against the clock alone. A feed that legitimately runs weekly is not flagged for being a day old, while a daily feed that goes quiet is.

​
Failure modes and re-running

1

Spot the failed run

The run shows as failed with the stage it failed in (sync, or curation).
2

Read the reason

Sync failures usually point at the source (expired credentials, an unreachable warehouse, a rejected query). Curation failures are reported per row, with the specific reason such as an unmapped value, a bad date cast, or a missing required field.
3

Fix and re-run

Correct the credential, the source data, or the field map, then re-run. Because the raw payload is retained and curation is idempotent, a re-run reprocesses cleanly without a re-upload and replaces the prior rows from that batch rather than duplicating them.

​
Common questions

No. Validation is per row, so the good rows curate and the bad ones are flagged individually with a reason. You fix and re-run only what failed.
If the underlying feed has not refreshed, reports show the last curated data. Check the freshness state here first; a stale feed is the usual cause.

​
Troubleshooting

  • A run is stuck in progress: confirm the source is reachable, then re-run. Pipelines is the place to confirm a run truly completed rather than stalling after a partial step.
  • Repeated curation failures on the same field: the field map or a dimension mapping is likely wrong. See Dimensions and GL Mapping.
  • A feed is healthy but numbers look off: verify the mapping landed values on the right canonical fields before assuming an ingestion problem.