Synchronizing content requests

When constructing an integration, a significant aspect involves synchronizing the outcomes of content requests as they progress towards completion.

States

The state of a content request can transition from any value to any other, allowing for example a completed request to sometimes revert to a confirmed state.

This flexibility implies that a content request may achieve completion multiple times, and it's imperative for your integration to accommodate this scenario. For instance, if your integration automatically publishes a page, you might consider:

• Maintaining the page's availability but updating it when the request reaches completion again.
• Unpublishing the page and then republishing it when the request attains completion once more.

The choice you make depends on the specific objectives of your integration.

Retrieving Request State Changes

To effectively monitor the progress of your requests, it's essential to retrieve changes based on the last state transition. In the context of the HTTP API, this involves calling the endpoint for querying content with specific criteria:

• Track State Changes: Set criteria to retrieve requests where the last state change (lastStateChange) occurred after the previous state change observed during synchronization. This dynamic filtering ensures you stay up-to-date with the latest modifications.

• Sort Results: Arrange the results by the timestamp of the last state change (lastStateChange). This sorting approach facilitates seamless restarts in case of synchronization failures.

For efficient data handling, the API employs pagination. Integrations can choose to rely on the page query parameter to fetch multiple pages of results or adjust criteria accordingly when a page has been processed.

In pseudo-code, a synchronization method might resemble the following example:

while true
  let lastStateChangeSeen = getLastStateChangeSeen()
  let requests = fetchChangedSince(lastStateChangeSeen)

  if requests.length == 0
    // Nothing completed - nothing more to do exit the do-loop
    break
  done

  for request of requests
    storeContentRequest(request)
    lastStateChangeSeen = request.lastStateChange;
  done

  setLastStateChangeSeen(lastStateChangeSeen);
done

You can run a synchronization process based on the provided pseudo-code at specified intervals to retrieve completed content requests as they become available. These intervals can vary, whether you choose a frequency of every 5 minutes, hourly, or even once a day.

However, if you prefer to handle content requests immediately as they change states, the HTTP API offers callback support through webhooks. This way, your system can be instantly notified and respond to state transitions in real-time.

Getting completed requests

If your primary interest is to receive notifications when requests are completed, the simplest approach is to execute a query for requests based on either:

1. The last time your integration fetched completed requests.
2. The completion time of the most recent completed content request.

Similar to the process for fetching all requests, you can construct a query that:

• Specifies criteria to restrict the results to content requests sent via the API (source = api).
• Defines criteria to fetch requests where the completion time is after the previously recorded date.
• Sorts the results by the completion time. This sorting ensures that in case of a synchronization failure, when option 2 is employed, the subsequent synchronization can resume correctly.

In pseudo-code, the procedure might resemble the following:

while true
  let dateOfLastCompleted = getLastCompletedInDatabase()
  let requests = fetchCompletedSince(dateOfLastCompleted)

  if requests.length == 0
    // Nothing changed state - nothing more to do exit the do-loop
    break
  done

  for request of requests
    storeContentRequest(request)
  done
done

Error handling

Effective error handling is a crucial aspect of developing a robust synchronization process. This includes activities like logging issues when they arise and ensuring the synchronization can continue seamlessly.

We strongly advise your system to log incidents where it encounters difficulties in accessing the API or storing received content requests.

Additionally, we recommend notifying us of important states whenever your system imports a completed content request. You can accomplish this through the HTTP API using the endpoint for setting the import state.

Your integration should notify us both after a successful import and in case of errors.

In pseudo-code, a method for storing content requests might resemble the following:

if request.state == 'completed'
  try
    storeStateAndInfoForRequest(request)

    transferContent(request, targetEntity);

    updateImportState(request, 'success')
  catch ex
    logError(ex)

    updateImportState(request, 'fail')
else
  try
    storeStateAndInfoForRequest(request)
  catch ex
    logError(ex)

In the pseudo-code, the storeStateAndInfoForRequest method is responsible for making an API call to update the import state. It includes a short timeout period for the API call and a catch-all for handling any errors. This approach is strongly recommended to prevent synchronization from becoming stuck while waiting for the API call to complete and to ensure that the synchronization process doesn't break in case of an error during the API call.