Hey,
I hope you are all well and safe.
I'm using the integration field to connect to a youtube channel. Everything works well, but I can't figure out how to paginate the results, there are more than 50 videos in the channel?
The way its working now is with a nodejs server running on Heroku that fetch all the data from the Youtube API and mutates the data so it follows the pattern for the integration field. I can share my code if that helps.
Hope someone can help in here.
Thanks,
Kristian
Hi Kristian,
If you could share your code that would be great. I can also show you can example now.
https://if-test-custom-api.herokuapp.com/cat1/150/50?page=2
Note:“cat1” it s just a name you can put whatever you want
In this example you can see we have 150 is the total number of items in the catalogue, so we set the result_size as 150. Then in the url we give the page size of 50 and call the second page so we get the results starting from 50.
Does this make sense?
Thanks for you replay.
It make sense what you saying but what I don’t undertand with the Youtube API is that it has a nextPageToken parameter you can pass on the API url, put not from a to b.
Do i need to fetch and store all the videos in an array and then create a custom pagination that controls what data prismic gets back?
app.get('/videos', (req, res) => {
let formattedData = {};
let formattedResults = [];
let curIndex = 0
const url = 'https://www.googleapis.com/youtube/v3/search?channelId=UCCNhmdvSnRNmuxTIYFybk5A&part=snippet,id&order=date&maxResults=1';
fetch(`${url}&key=${process.env.GOOGLE_API_KEY}`)
.then(response => response.json())
.then(json => {
json.items.forEach(item => {
let date = item.snippet.publishedAt;
let TS = new Date(date).getTime();
let TSFLOAT = parseInt(TS);
let id = item.id.videoId;
if(typeof id === 'string' || id instanceof String) {
formattedResults.push({
'id': id,
'title': item.snippet.title,
'description': item.snippet.description,
'image_url': item.snippet.thumbnails.default.url,
'last_update': TSFLOAT,
'blob': {
'id': id,
'thumb': item.snippet.thumbnails
}
});
} else {
console.log(id, item.snippet.title);
}
});
formattedData = {
"results_size": 150,
"results": formattedResults
};
res.json(formattedData);
});
});Hello Kristian,
Here is a code that should work. I permit myself to remove the if else condition and use type=video in the url. Also, i m not using env var for the API key only for my tests. You can reput your env var (process.env.GOOGLE_API_KEY). For the pagination, we need to play with the nextPageToken yes.
const express = require('express');
const app = express();
const port = 3000;
const fetch = require('node-fetch');
const apiKey = 'YOUR_API_KEY';
const url = 'https://www.googleapis.com/youtube/v3/search?channelId=UCCNhmdvSnRNmuxTIYFybk5A&part=snippet,id&order=date&maxResults=50&type=video';
function getResult(token) {
if (!token) {
return fetch(`${url}&key=${apiKey}`).then(response => response.json());
} else {
return fetch(`${url}&key=${apiKey}&pageToken=${token}`)
.then(response => response.json());
}
}
app.get('/videos', async (req, res) => {
let formattedData = {};
let formattedResults = [];
let json = undefined;
const page = req.query.page;
if (page && page > 0) {
let token = undefined;
for (let i = 0; i < page; i++) {
json = await getResult(token)
.then(json => {
token = json.nextPageToken;
return json;
})
}
} else {
json = await getResult()
}
json.items.forEach(item => {
let date = item.snippet.publishedAt;
let TS = new Date(date).getTime();
let TSFLOAT = parseInt(TS);
let id = item.id.videoId;
formattedResults.push({
'id': id,
'title': item.snippet.title,
'description': item.snippet.description,
'image_url': item.snippet.thumbnails.default.url,
'last_update': TSFLOAT,
'blob': {
'id': id,
'thumb': item.snippet.thumbnails
}
});
});
formattedData = {
"results_size": 150,
"results": formattedResults
};
res.json(formattedData);
});
app.listen(port, () => console.log(`Example app listening at http://localhost:${port}`));
The only issue with that solution is the fact that you need to fetch the first x - 1 page to get the x page. A better solution would be to store all data somewhere in your system as a cache (with expiration after like 30 minutes or 1 hour). And then return only the cached result. But that’s an other subject. For small YouTube channel it’s ok to fetch few times to get the data that you want.
Also, don’t hardcode results_size in the response. I would use pageInfo.totalResults from the YouTube API response to populate this field.
Thank you,
Raphaël
Hey Raphael,
Thanks so much for your reply, really helpful.
Seems to be working on my locale version but when I deploy it and try to sync in Prismic I get a 503 status error and page 1. Does that has something to do with the limits of quotas?
Or is the problem that you mentioned with the first page?
Cheers,
Kristian
Sorry I’ve found the error, so it does sync now, thanks again.
I’m not sure how it is suppose to work in Prismic, but I can see a ist of videos but not in order if I compare them to the Youtube channel and I doesn’t load more if scroll to bottom in the list. But I can search all the videos so looks like it has fetched all of them? Do you have any ideas?
Many thanks!
Hey Kristian,
Sorry about the long delay in the reply. This slipped through the cracks, we won’t let that happen again.
The integration field works as you’ve described I believe, in that there is no scroll more feature, you will need to search for items in the catalogue as we show in the intro article.
As for the order, how does the order compare to json catalogue that you load in the integration field?
Threads close after a period of inactivity. Flag this thread to re-open it and continue the conversation.