Query unpublished (draft, archive) documents

I'm synchronizing my content between Prismic and Algolia through a webhook. This content is localized, so I need to have separated indexes in Algolia (one for each locale). For better performances, I need to know which index to update when a document is published, archived or edited.
Each time the webhook is triggered, I get the list of updated documents (by IDs) but I don't get the list of edited locales, so I'm querying each document ID to get it's lang.

My problem is that when a document is not published, the API does not return it to me. Is there any parameter to request archived or draft documents?

3 Likes

Hi there,

Thanks for reaching out.

You can put the documents that you need to fetch in a release then you can get the locale of the documents.

Best,
Fares

2 Likes

This is being tracked as an open feature request.

If you have another use-case for this feature, you can 'Flag' this topic to reopen. Please use the :heart: button to show your support for the feature and check out our Feature Request Guidelines.

Any updates on this? We are creating static pages and when a page is deleted we only get the id of the deleted page but cannot fetch it to get the slug and then delete the file with that slug on our static storage.

3 Likes

Hello @adolfo.yanes, we don't have any news about this feature. If this changes we'll announce it in our progress page What's new - Prismic

Any update on this? Currently there is no way to revalidate content that gets unpublished, since the webhook only returns the document id but there is no way to query the unpublished document.

Yes, it isn't possible to know which documents get unpublished. You'd need to use a workaround to see if the document exists in a release or not.

I have the same issue. I use a webhook to revalidate pages in next.js. Each page has an UID which is used to construct the page URL. Because I couldn't figure out the UID on unpublish, I had to resort to storing page data in a database. I query the database by document ID and it returns the UID. This way I can figure out which UID/URL to revalidate on unpublish. (there are some free database providers, I used https://www.cockroachlabs.com)

Some suggestions that would solve this problem on Prismics side:

  1. Allow to send (all) document data or document UID to the webhook
  2. Allow to delay unpublish for x seconds after the webhook has been called

If anyone is interested in a quick and dirty example solution for the problem I have (use at your own risk):

Summary
// pages/api/revalidate.ts
import type { NextApiRequest, NextApiResponse } from 'next';
import { Client } from 'pg';

import { linkResolver, createClient } from '~/prismicio';

/**
 * TODO: THIS IS A HACK:
 * ? remove postgres logics when prismic allows
 * ? to figure out links/data/uid of unpublished pages
 */

const storeDocumentData = async (
  updatedData: { uid: string; id: string }[],
) => {
  const pgClient = new Client(process.env.REVALIDATE_DATABASE);
  await pgClient.connect();

  await pgClient.query(`
    UPDATE storedDocuments SET data='${JSON.stringify(updatedData)}' 
    WHERE ID = (SELECT ID FROM storedDocuments ORDER BY ID LIMIT 1);
  `);

  await pgClient.end();
};

const getStoredDocumentData = async () => {
  const pgClient = new Client(process.env.REVALIDATE_DATABASE);
  await pgClient.connect();

  const storedDataString = (
    await pgClient.query(
      `SELECT data
       FROM storedDocuments
       ORDER BY ID LIMIT 1;
      `,
    )
  ).rows[0].data;

  await pgClient.end();

  return JSON.parse(storedDataString) as {
    uid: string;
    id: string;
  }[];
};

const handler = async (req: NextApiRequest, res: NextApiResponse) => {
  // Check for secret to confirm this is a valid request
  if (req.body.secret !== process.env.REVALIDATE_SECRET) {
    return res.status(401).json({ message: 'Unauthorized' });
  }

  try {
    const client = createClient();
    const storedDocumentData = await getStoredDocumentData();
    const documents = await client.getAllByIDs(req.body.documents);

    const revalidatePromises = req.body.documents.flatMap(
      async (id: string) => {
        const foundUpdatedDocumentIndex = documents.findIndex(
          (doc) => doc.id === id,
        );
        const foundUpdatedDocument = documents[foundUpdatedDocumentIndex];
        const storedDocumentIndex = storedDocumentData.findIndex(
          (doc) => doc.id === id,
        );
        const storedDocument = storedDocumentData[storedDocumentIndex];

        if (
          storedDocument?.uid &&
          foundUpdatedDocument?.uid &&
          storedDocument.uid !== foundUpdatedDocument.uid
        ) {
          // uid has changed (uid is used as URL)
          const updatedData = [...storedDocumentData];
          storeDocumentData(updatedData);
          updatedData[storedDocumentIndex].uid = foundUpdatedDocument.uid;
          return [
            res.revalidate(
              linkResolver({ type: 'page', uid: storedDocument.uid }),
            ),
            res.revalidate(
              linkResolver({ type: 'page', uid: foundUpdatedDocument.uid }),
            ),
          ];
        }

        if (foundUpdatedDocument?.uid) {
          if (!storedDocument) {
            // new document
            storeDocumentData([
              ...storedDocumentData,
              {
                id: foundUpdatedDocument.id,
                uid: foundUpdatedDocument.uid,
              },
            ]);
          }
          return res.revalidate(
            linkResolver({ type: 'page', uid: foundUpdatedDocument.uid }),
          );
        }

        if (storedDocument) {
          // found no updated document so the page is unpublished
          const updatedData = [...storedDocumentData];
          delete updatedData[storedDocumentIndex];
          storeDocumentData(updatedData);
          return res.revalidate(
            linkResolver({ type: 'page', uid: storedDocument.uid }),
          );
        }

        throw Error('Revalidate bug');
      },
    );

    await Promise.all(revalidatePromises);
    return res.status(200).send('OK');
  } catch (err) {
    // eslint-disable-next-line no-console
    console.error(err);
    return res.status(500).send('Error revalidating');
  }
};

export default handler;
// revalidate.js
const { createClient } = require('@prismicio/client');
const { Client } = require('pg');
const smConfig = require('./sm.json');

const fetch = (...args) =>
  import('node-fetch').then(({ default: fetch }) => fetch(...args));

global.fetch = fetch;

const cleanPageData = (pageData) => ({
  uid: pageData.uid,
  id: pageData.id,
});

const insertInitialPageData = async () => {
  if (!process.env.REVALIDATE_DATABASE) return;
  const client = createClient(smConfig.apiEndpoint);
  const pgClient = new Client(process.env.REVALIDATE_DATABASE);
  await pgClient.connect();

  await pgClient.query(`
    DROP TABLE storedDocuments;
  `);

  await pgClient.query(`
    CREATE TABLE storedDocuments (
      id serial PRIMARY KEY,
      data TEXT
    );
  `);

  const documents = await client
    .getAllByType('page')
    .then((docs) => docs.map(cleanPageData));

  await pgClient.query(`
    INSERT INTO storedDocuments(data)
    VALUES ('${JSON.stringify(documents || [])}');
  `);

  await pgClient.end();
};

insertInitialPageData();
// update your build command e.g.
next build && node ./revalidate.js

Don't forget to return { notFound: true } in your dynamic next pages

interface Params extends ParsedUrlQuery {
  uid: string[];
}

export const getStaticProps: GetStaticProps<{}, Params> = async ({ previewData, params }) => {
  const client = createClient({ previewData });

  const asyncPage = client
    .getByUID('page', params?.uid?.[0] || '')
    .catch((err: Error) => {
      if (err.message.includes('No documents were returned')) {
        return null;
      }
      throw Error(err.message);
    });

  const asyncGlobalData = fetchGlobalData(client);

  const [page, globalData] = await Promise.all([asyncPage, asyncGlobalData]);

  return page
    ? {
        props: { page, globalData },
      }
    : { notFound: true };
};

export const getStaticPaths: GetStaticPaths = async () => {
  const client = createClient();
  const documents = await client.getAllByType('page');

  return {
    paths: documents.map((doc) => prismicH.asLink(doc, linkResolver)),
    fallback: 'blocking',
  };
};

2 Likes

We're very interested in this feature, for the same reasons as @bram.zijp1.Query unpublished (draft, archive) documents - #7 by Pau

1 Like

Same here. Very interested about this feature like @bram.zijp1 described....

EDIT :
And I think that returning the uid would still not b enough is some cases. For example if we have nested routes like [type]/[uid].tsx. Here I would only have the uid, i also need the doctype. I think it could be better to have the ability to query unpublished/archived documents.

Agree, the option to query unpublished documents through the api by setting a flag unpublished=true or something would be the best solution.

3 Likes

Bumping this because we're having the same issue:

Using webhooks to trigger Next.JS on demand revalidations.

We have page paths on the documents themselves.

When we PUBLISH a document, it works as intended, as we get the Document ID and grab our path to revalidate.

When we UNPUBLISH, we cannot query the document by it's document id as it is in 'archived'.

So we cannot get the path in this manner. If there's a way to query it, we'd love to know!

Just discovered this issue. I'm quite surprised one can query "never-published" docs (preview endpoint), but not archived (published-then-unpublished) docs.

In our case, we render static listing pages (just the first page). If a document is archived we need to revalidate the specific listing (based on the doc's type), but since the webhook req contains only an unqueryable ID, we can't get any details on what was deleted and thus need to revalidate ALL the listings.

The only solution I can think of right now is to:

  1. Store the "previous ref" on each webhook call.
  2. If the doc is not found in the present ref, fetch it again using the stored ref.

This assumes the refs are never flushed, which I cannot confirm.


Edit:

I've just come to realise the same issue prevents us from revalidating the URLs of archived pages during ISR, so we can't setup /404 redirects. To revalidate in NextJS we need the URL. To get the URL we need the doc. If the doc was archived, there is no way to know its URL (unless we have the previous ref saved as described above).