Hi! Loving Prismic so far, but running into a bit of a snag.
I have an external system that I need to keep up-to-date information about what documents are available in Prismic. I'm using webhooks for this, and it mostly works fine. However, I need a way to "catch up" if the webhooks fail for some period of time.
This is easy enough to do this with published documents - I can just query for all documents here "document.last_published_date" greater than last time I checked. This gives me all the new and changed documents.
However, when a document is unpublished, I can never access the information about that document from the API again. This is problematic, for example, if there is a bug in the webhook or my site was down when the unpublish action happened.
Basically, I need a way to "catch up" to the latest ref state and find out "what documents have been unpublished since ".
Is this simply not possible? The only thing I can think of is to "dangerouslyGetAll" and make a list of all currently active documents, then compare that to what my system thinks is published, and look for any that my system thinks are published but don't show up in the results of "dangerouslyGetAll". This seems problematic, though, because "dangerouslyGetAll" will get more and more expensive the longer I'm in operation, and the more content I have.
Is there any other way to find out via the REST API that some previously published documents have been unpublished?
And it turns out that there is already a retry mechanism in our Webhooks where it will try 5 times, and if still getting an error code, the Webhook would get disabled.
So based on what was said, there will be no issue and need to catch up, all you will need is to activate the webhook manually.
Please let us know if you need any further assistance,
Fares
I was aware of the retry. If it fails 5 times, and gets disabled, I will miss webhooks until I can fix the problem and reactivate them. There will be data that is missed during this window.
I don't think there is a good way to sync "what has been unpublished since (some datetime)".
I appreciate the retry feature, but there should be some way to ask the API about unpublished documents.
Well, it seems there is something I didn't understand correctly here; for me, there will be no lost/missed webhooks if the webhook is disabled, and once you activate back again, you will get all the webhook events that weren't delivered.
There is one thing to be improved on our side to implement some sort of notification system where it sends an email, for example, if the webhook gets disabled.
Please let me know if there is still something unclear.
Interesting. I didn't see anything in the docs about catching up all missed webhooks when re-enabling a webhook.
If I manually disable a webhook, and then re-enable it later, I won't get all the webhooks I missed, right?
So this is an interesting difference in behavior. You're saying if a webhook is disabled for errors, then later re-enabled, it will at that time be sent all the messages it missed while it was disabled? Are you certain that's the way it works (it will be very hard to test)?
If that is the case, it should really be documented! I apologize if it's in the documentation and I just haven't found it.
As for the "catching-up" behavior I asked for, I still would find value in a feature to query Prismic for unpublished documents. If this isn't on the roadmap, I'd vote for it to be added - it would be a very nice-to-have feature for my workflow (keeping an external system fully in-sync with Prismic).
However, you've answered my question and I can work with the current feature-set as-is. Thanks for following up.