Building a data import job
Data import jobs in Future Ordering are responsible for creating import entities that later jobs in the same flow can process. Each job gathers or transforms source data and uploads entities for downstream jobs to consume.
Requirements and best practices
Calling and responding
Basic Authentication is the only supported external authentication method for your job's endpoint. Provide the credentials to Future Ordering when you register the job. Requests your job sends back to Future Ordering are ordinary API calls and follow the standard API conventions, including the required X-Api-Version: 0 header.
An import job runs asynchronously. When the orchestration system calls your endpoint, respond immediately with 202 Accepted to confirm that the job has been accepted, then process the request in the background.
The initial request contains SAS URLs pointing at the source data for each entity. Fetching from these URLs requires no token, but they are temporary. Use them during the run rather than storing them for later.
Handling data
If your job is not the first in a flow, use a dynamic data model. You cannot predict what metadata earlier jobs in the flow have added, and a fixed model risks discarding it.
When your job finishes, upload all data to the orchestrator, including the entities you did not change. Only leave an entity out when you intend to remove it, for example removing a store from a store import. GZIP the data when uploading if your application supports it.
Reporting progress and errors
Report progress at key milestones - when the job starts, at critical steps and on completion - so that administrators can monitor a run while it is in progress.
If your application encounters an unrecoverable error, send an error report with all the diagnostic information available. Error reports are visible in the Navigator application, so never include personal or secret data in them.