Pipeline control panel

Data

Runs the document-processing pipeline on request: resolves which documents actually exist for a set of records, checks what has already been processed, dispatches only the remainder, replays failures, and confirms the results reached their destination.

Used when

When a batch of records needs its documents processed, when a previous run left failures behind, or when someone needs to know whether a given account's documents made it through.

Loads first

• Its own dispatch workflow, followed step by step rather than improvised.
• The integration guide for the source system holding the document rows.
• The read patterns for the store that records what has already been processed.

Returns

• The source environment, the processing target, and the run identifier.
• Input row counts, unique record counts, and how many fall outside supported handling.
• Document-metadata counts, storage verification counts, and any location that could not be read.
• The already-processed count and the count remaining to dispatch.
• Dry-run and live dispatch counts, the queues targeted, and the manifest written.
• Verification of the processed results, and a summary of any follow-up requests raised.