Polling for Mentions

A common integration use case is to continually populate a display with Mentions from a Brandwatch Query. This, for example, is something that we ourselves do in some of our Vizia tiles.

The behavior that we want to achieve is as follows:

1. On application start up, we want to retrieve the latest 100 mentions.
2. At every polling interval, we want to get the latest mentions since the last time that we polled.

IIn this tutorial we will assume that the application has already logged in and has a token. We also recommend that you read the section in our documentation about Mentions, in addition to this tutorial.

Getting the latest 100 mentions

When our application runs for the first time, we want to get the latest 100 mentions for our query. We use the following parameters to achieve this:

Here is the request as a cURL command:

curl -X GET 'https://api.brandwatch.com/projects/1998232810/data/mentions?queryId=1999544643&startDate=2019-01-16&endDate=2019-01-17&pageSize=100&page=0&orderBy=added&orderDirection=desc'

This will return a list of Mentions. The following example response has had the Mentions data trimmed for brevity.

{
  "results": [
    {
      "added": "2019-01-16T23:36:52.237+0000",
    },
    {
      "added": "2019-01-16T23:36:52.237+0000",
    },
  ],
  "resultsPage": 0,
  "resultsPageSize": 100,
  "resultsTotal": 55,
  "startDate": "2019-01-16T00:00:00.000+0000",
  "endDate": "2019-01-17T00:00:00.000+0000"  
}

Each mention in the response results will have an added field. This is the date the mention was added to the query. In your application, find the mention with the most recent added value and store it (2019-01-16T23:56:28.157+0000 in this case). You will want to use this value for the next time you poll for mentions.

Polling for new Mentions

Because different servers within the BCR platform may process data at different times, we can't rely purely on the added value to retrieve all newly added mentions - we must add a buffer to allow for this.

Usually we will want a “buffer” of five minutes. So take that added value (2019-01-16T23:56:28.157+0000) and subtract 5 minutes from it so you have 2019-01-16T23:51:28.157+0000. Now you can use this with the sinceAdded parameter to find mentions that have been added within the last 5 minutes of the most recent mention.

We then add this to our new request (NB the date has been URL encoded):

curl -X GET 'https://api.brandwatch.com/projects/1998232810/data/mentions?queryId=1999544643&startDate=2019-01-16&endDate=2019-01-17&pageSize=100&page=0&orderBy=added&orderDirection=desc&sinceAdded=2019-01-16T23%3A51%3A28.157%2B0000'

This then returns the newest 100 mentions since we last polled. Repeat this process in order to continually poll for new Mentions in your integration. Due to the 5 minute buffer you will see some of the same mentions you saw in the previous request, so make sure to deduplicate as you need to (using the queryId and resourceId values for each mention). If the query is low volume this might simply be the latest mention.

You may also find you receive a full page of mentions and need to read multiple pages (see the Paging through historical Mentions tutorial for details of how to do that)

We can now repeat this process with the new nextCursor value, until no more results are returned, to retrieve all available Mentions.

Finding only new mentions

When a query is "backfilled" the added field will be reset for each mention. This may be useful for you, as it helps you detect if the backfill produced any different mentions you had not seen before. However if you do not want this behaviour you can apply another parameter to only return "new" mentions.

We can add that to our request like this:

curl -X GET 'https://api.brandwatch.com/projects/1998232810/data/mentions?queryId=1999544643&startDate=2019-01-16&endDate=2019-01-17&pageSize=100&page=0&orderBy=added&orderDirection=desc&sourceType=new'

This will then only return "new" mentions and not those created during a backfill.