Synchronizing content requests
When building an integration a large part is about synchronizing the results of content requests as they become completed.
States
The state of a content request be changed from any value to any other value, which means that a completed request can sometimes become confirmed again.
This means that a content request can be completed more than once and that your integration should plan for this. For example if your integration automatically publishes a page you might want to:
- Keep the page available but update it when the request is completed again.
- Unpublish the page and republish it when the request is completed again.
The exact route to take depends on the goals of your integration.
Getting all request changes
The recommended way to to fetch requests is based on their last state change which allows you to monitor the progress of your requests.
For the HTTP API this means calling the endpoint for querying content with a query that does a few things:
- Sets a criteria to limit results to content requests sent via the API (
source
=api
) - Sets a criteria that fetches requests where
lastStateChange
is after the last state change seen by the sync - Sorts the results by
lastStateChange
, so that if a sync fails the next sync can restart correctly
The API will paginate the results and an integration can either rely on the
page
query parameter to fetch several pages or tweak the criteria for
completed
when a page has been handled.
In pseudo-code a synchronization method might look like this:
while true
let lastStateChangeSeen = getLastStateChangeSeen()
let requests = fetchChangedSince(lastStateChangeSeen)
if requests.length == 0
// Nothing completed - nothing more to do exit the do-loop
break
done
for request of requests
storeContentRequest(request)
lastStateChangeSeen = request.lastStateChange;
done
setLastStateChangeSeen(lastStateChangeSeen);
done
Running as synchronization based on such a pseudo-code at a set interval will fetch completed content requests as they become available. The interval can be 5 minutes, 1 hour or even once a day. If you want to handle content requests as they change state the HTTP API supports callbacks via webhooks.
Getting completed requests
If you are only interested in when things are completed the easiest way is to perform a query for requests since either:
- The last time the integration fetched completed requests
- The completion time of the last completed content request
Similar to fetching all requests construct a query that:
- Sets a criteria to limit results to content requests sent via the API (
source
=api
) - Sets a criteria that fetches requests where
completed
is after the previously picked date - Sorts the results by
completed
, so that if a sync fails and option 2 was used the next sync can restart correctly
In pseudo-code this would look something like this:
while true
let dateOfLastCompleted = getLastCompletedInDatabase()
let requests = fetchCompletedSince(dateOfLastCompleted)
if requests.length == 0
// Nothing changed state - nothing more to do exit the do-loop
break
done
for request of requests
storeContentRequest(request)
done
done
Error handling
An import part of building a stable synchronization is error handling, things such as logging when something goes wrong and allowing the synchronization to proceed.
It is recommended that your system at a minimum logs cases where it can't access the API and if it is unable to store a received content request.
In addition to this we recommend pinging us with an import state whenever your system imports a finished content request. Using the HTTP API this can be done via the endpoint for setting import state.
Your integration may ping us both after a successful import and if an error occurs.
In pseudo-code a method for storing content requests might look like:
if request.state == 'completed'
try
storeStateAndInfoForRequest(request)
transferContent(request, targetEntity);
updateImportState(request, 'success')
catch ex
logError(ex)
updateImportState(request, 'fail')
else
try
storeStateAndInfoForRequest(request)
catch ex
logError(ex)
In this pseudo-code storeStateAndInfoForRequest
is a method that performs
an API call to update the import state with a short time out and a catch-all
for any errors. This is recommended so that a synchronization does not lock
for a long time while waiting for the call to complete and that the call does
not break the synchronization if it errors.