I think I found what's the problem. I didn't define PrismicProvider in gatsby-ssr, only in gatsby-browser. This means that some of my Prismic settings only loaded after initial load (crucially my html serializer) -> Understanding the Gatsby lifecycle | by Dennis Brotzky | Narative | Medium
For reference, I looked at this starter blog and I saw that it was included in both the gatsby ssr and gatsby browser, which wasn't clear before that