Make this code faster

Going through a MongoDB database (Javascript)

Hello there.

I have an issue while trying to add a few fields to every entry in a MongoDB database, using Javascript. First, i tried to get all the documents and apply to them my function with find().forEach(myFunction)

But I realised that i was entering the function around 2 millions times while I only have 1 million entries in my database. I tried to reduce it, to discover the number starts getting bigger when I have more than 101 entry in my database. I then discovered that find() returns entries by batches, with the default batch size being 101 entry, and looks like the entries are not cleared in between each iteration, so if find() returns 102 entries, i will go through the 101 first ones, then through the 102nd, and then trough the 2nd to 101st (since the 1st one was overwritten by the 101st). BUT, increasing the batch size would help since according to the doc, batches are ALWAYS limited to 4MB, which is obviously too little when dealing with a million entries.

I then tried to get a cursor from find() and going through my entries with cursor.hasNext(), but it seems to work the same way.

So, is there any way to only call myFunction the exact same number of times than entries in my database ?
It would be a great optimisation to do.

Thanks !

var myFunction = function(){ //... } var cur = db.countries.find(); while(cur.hasNext()) { myFunction(; } // db.countries.find().forEach(myFunction); works the same way

Reactions 0 reactions

No reactions yet, be the first: comment or tweak the code

Comment on this code:

Please Login or Register to leave a comment.