Webhooks Intermittently Don't Fire

Dashboard / Writing room
#1

Hey Prismic support,

Is there something going on webhooks? We've been tracking an issue with our publishing pipeline for the last week or so, and it looks like there may be some kind of intermittent failure with Prismic's webhooks. Its difficult to tell for certain because the webhook logs don't have a timestamp, and it seems to be happening randomly, but every day or two a webhook seems to be failing to fire when a post is published. After some testing, we're confident that our publishing pipeline is working properly, but we've had several publishing events not hit our backend through the webhook we setup. Is there something going on with webhooks on your end that could explain this?

To give you a rough idea of what our publishing pipeline looks like, we have Prismic hit a webhook whenever a publish or release happens. This triggers a background job that fetches any new documents from Prismic's API, and then compares the results to what we have in our database and performs an upsert.

We've confirmed that our background job workers, the webhook endpoint, and our DB upserts are all working properly. Which leads me to the conclusion that the webhooks must be failing to fire for some reason.

Any help or guidance would be much appreciated.

3 Likes
#3

I've been having the same problem! While doing some testing a few days ago to verify the webhook workflow actually works I noticed occasionaly the webhook endpoint wasn't being hit. It's very intermittent. I had actually set up 2 webhooks to fire, one going to a server and one going to my local machine, and sometimes only 1 of the 2 endpoints was being hit.

This is a big problem for scheduled releases because no one would be checking/monitoring it the same way an editor could check the page got updated after he just published it.

#5

Hi there.

Can you provide us with your repository names and a screenshot of the logs of the failed events triggered (in a private message if necessary) in order to check our logs for any errors?

Looking forward to your reply,
Fares

#6

Hey @Fares, thanks for the response.

In my case the repository name is Taste. As for logs of failed events, there are none, that's kind of the problem. Prismic doesn't indicate that they're was a failed webhook event, it just doesn't hit the webhook at all. Like @martin.kzh mentioned, I too have two separate webhooks that fire on publish, and often when there's such a failure to trigger it'll trigger for one or the other, but not always both.

#7

I have managed to document this. The repository I did it on is https://mynewsite.prismic.io/, but I think this happens on any repository (I've seen it on a different one). I've used 2 webhooks to show the difference, but trigger failure potentially happens in any case.

Screenshots and explanation here: Prismic webhook test

(I've blacked out my local server IP)

P.S. The webhook logs really need timestamps!

1 Like
#8

Hi Martin,

Thank you for your contribution, and documenting this issue.
So I have create an issue in our tracker with all the details mentioned and forward it to our dev team.

And I find adding timestamps is essential for debugging issues, the I have created a feature request and forwarded it to the @activations-team.

#10

Hi @martin.kzh @devon.noeldetilly Could you figure out the type of trigger event that is being dropped? is it only document publishing?

#11

In my case it's publishing. I haven't tested with other events.

1 Like