Pagination In NodeJs: 01 – Using Offset based pagination (skip and limit)

Prerequisites: Basic knowledge of Nodejs and Mongodb.

What is pagination ?

Ans: Consider that you have around 1 million records that you want to fetch. But fetching 1 million record at once will take alot of time and its not good practice to fetch huge data at once. So, there comes Pagination where users can fetch data in pages. This is achieved by using options skip and limit, giving clients full control of the page(data) they are getting back.

When to use Pagination?

As its statement describes, Pagination should be used:

What are the ways in which you can implement Pagination in NodeJs?

Well, there could be multiple ways, which you could use to implement Pagination in nodejs but in this post i am gonna talk about two main ways in which we can implement Pagination. Those two ways are as follows:

In this post I am gonna talk only about Offset Based Pagination. For Cursor based pagination you can check out my this post.

Offset based pagination:

It is one of the most common pagination methods we are using for decades. It simply uses limit and offset in SQL queries to paginate the data from database.

In NOSQL database, it will be limit and skip. Let’s implement it in our application and see the advantages/disadvantages of it. Implementation for offset pagination is straightforward,

 


  const fetchCompanies = async (req, res) => {
   constlimit=parseInt(req.query.limit);
   constoffset=parseInt(req.query.skip);
   try {

    consttradesCollection=await Trades.find().skip(offset).limit(limit);

    consttradesCollectionCount=await Trades.count();
    consttotalPages=Math.ceil(tradesCollectionCount/limit);
    constcurrentPage=Math.ceil(tradesCollectionCount%offset);

    res.status(200).send({
     data: tradesCollection,
     paging: {
     total: tradesCollectionCount,
     page: currentPage,
     pages: totalPages,
    },
   });

 } catch (e) {
  console.log("Error", e);
  res.status(500).send({
  data: null,
});

An important line here is,

const tradesCollection = await Trades.find().skip(offset).limit(limit);

MongoDB has skip and limit operators to implement offset based pagination. On sending the response, it is recommended to send them along with pagination data.

res.status(200).send({
 data: tradesCollection,
 paging: {
  total: tradesCollectionCount,
  page: currentPage,
  pages: totalPages,
 },
});

Drawbacks of Offset based pagination:

1. Offset pagination doesn’t scale for large datasets. Using SQL offset or NOSQL skip operators. It scans the record one by one and skip or offset it. If your database has a million records, just like we see in this tutorial, offset-based pagination can affect scalability.
2. If you have real-time data, offset-based pagination will be unreliable and problematic. There will be skipping of data or duplicate data. Read more