#302

New

Use case: take a list of users and do some operation on batches of them in a loop as efficient as possible.

Reported by Volkan Ozcelik on JavaScript: from Zero to Hero · 03/10/2018 23:39:47

Assigned to:
Volkan Ozcelik
Priority:
Normal
Status:
New
Category:
Use Cases To Be Moved
Version:
None

ok good. Then setInterval will not work, since mongodb (actually mongoose) is a promise-based api.

simplified pseudocode coming:
… tbd (edited)

Colton Ehrman [7:31 PM]

setInterval(() => {
User.find({}, (err, users) => {
console.log(users);
});
}, 1000);

doesnt this work?

volkan [7:33 PM]
nope.
You want to do the next fetch after you fetch the users, not back-to-back.

what if `User.find` takes 3 seconds?

Colton Ehrman [7:34 PM]
what next fetch?

volkan [7:35 PM]
```
const processUsers = (users) => new Promise((resolve, reject) => {
console.log(users);

// giving CPU some time to breathe. setTimeout(() => resolve(true), 100);

});

const run = () => {
User.find({}, (err, users) => {
processUsers(users).then(() => run)
});
}

run()
it’s be roughly something like this.
do not do the next iteration unless you finish processing your former task.
I’m not handling edge cases of course; you’ll have to design the `reject` case, the timeout case and a bunch of other edge cases.
and `console.log(users)` will likely be an async db update action, so you’ll have to create a `Promise` for that too
Again instead of `Promise`s I’d use `async/await` : it’ll make it a bit easier to follow.

would i be safe just doing a `setInterval` on server boot and just loop through my users on every interval and performing the actions

volkan [7:27 PM]
It depends.
how frequently do you get a new user. once a day or so, or once an hour or so? or once a second or so?

Colton Ehrman [7:28 PM]
well i am not currently live yet, haha
so never

volkan [7:28 PM]
:slightly_smiling_face:
okay. so l’d start simple.
followup question:
where are you storing the stats?
in memory?
in a db?

Colton Ehrman [7:29 PM]
currently i have mongodb in place

volkan [7:30 PM]
ok good. Then setInterval will not work, since mongodb (actually mongoose) is a promise-based api.
simplified pseudocode coming:
… tbd (edited)

Colton Ehrman [7:31 PM]

setInterval(() => {
User.find({}, (err, users) => {
console.log(users);
});
}, 1000);

doesnt this work?

volkan [7:33 PM]
nope.
You want to do the next fetch after you fetch the users, not back-to-back.

what if `User.find` takes 3 seconds?

Colton Ehrman [7:34 PM]
what next fetch?

volkan [7:35 PM]
```
const processUsers = (users) => new Promise((resolve, reject) => {
console.log(users);

// giving CPU some time to breathe. setTimeout(() => resolve(true), 100);

});

const run = () => {
User.find({}, (err, users) => {
processUsers(users).then(() => run)
});
}

run()
it’s be roughly something like this.
do not do the next iteration unless you finish processing your former task.
I’m not handling edge cases of course; you’ll have to design the `reject` case, the timeout case and a bunch of other edge cases.
and `console.log(users)` will likely be an async db update action, so you’ll have to create a `Promise` for that too
Again instead of `Promise`s I’d use `async/await` : it’ll make it a bit easier to follow.

Colton Ehrman [7:38 PM]
i havent used async/await before
but thank you for the example! that helps a bunch :smile:

volkan [7:39 PM]
Happy to help.
You can use `Promise`s too :wink: that’d work; you can even do it with good-old callbacks (continuation-passing style) but that’d require some extra bookkeeping and attention :wink:
^ actually that could make a good “use case” too
@Colton Ehrman do you think it’d be usefulif I cover something similar in a lesson.
I mean I can start with setInterval and explain why it’d fail, then go with callbacks, turn it to promises, and then finish it with async/await — guess it’d be a nice thing to sum up a few flow control techniques.

Colton Ehrman [7:41 PM]
Yes! Very much! :+1:

volkan [7:41 PM]
okie dokie. Noting it down!

Attachments

No attachment has been uploaded, yet.


Loading comments...