What does adding the oplogReplay option do for a tailable cursor in mongodb? Why did it fix my timeout issue?

Vote count: 0

I was trying to tail the oplog in mongo and was getting a timeout error. The oplog collection has over 7 millions documents in it. However, as a last ditch effort I added the option oplogReplay: true to the cursor options and the timeout error went away and I was reading the oplog stream.

The nodejs mongo docs are pretty cryptic about what this option actually does, only that it's for tailable cursors which is why I over looked it:

oplogReplay {Boolean}, sets an internal flag, only applicable for tailable cursor.

My guess is it either skips the initial scan of the whole oplog and goes right to the end, or it scans from the other end of the oplog.

My code looks something like this:

const tstamp = new MongoDB.Timestamp(0, Math.floor(new Date().getTime() / 1000));
const filters = {
  ts: { $gt: tstamp }
};
const curOpts = {
    tailable: true,
    timeout: false,
    oplogReplay: true,  //<- What does this actually do?
    awaitData: true,
    numberOfRetries: 60 * 60 * 24,
    tailableRetryInterval: 1000
};

mongodb.connect(url).then(db => {
    const cur = db.collection('oplog.rs').find(filters, curOpts);
    const stream = cur.stream();
    stream
      .on('data', log => {
        console.log('Data!')
        console.log(log)
      })
      .on('close', () => {
        console.log('Stream closed....')
        db.close()
      })
      .on('error', err => {
        console.log('Stream error....')
        console.error(err)
        db.close()
      });
}));

Related reseach

I also found this question and answer which is related. I found that after I solved my original issue though.

And when I was researching the problem, I found that the oplog scan times out after 30 seconds and this is/was causing issues from some.

Another question and answer related to my timeout issue. It seems that creating an index on the ts field of the oplog is a very bad idea and impossible on newer versions of mongo.

asked 40 secs ago

CovertIII

Let's block ads! (Why?)

What does adding the oplogReplay option do for a tailable cursor in mongodb? Why did it fix my timeout issue?