Why push notifications are getting scheduled but not getting sent? What could be cause of this?
Until very recently Push Notifications were getting sent.
Now I am able to create them & they are getting scheduled but not getting sent.
Tried restarting countly as well as restarting server over 20 times. No use.
Can anyone please help?
Comments
Could you attach relevant part of
log/countly-api.log? It must have some error.Here is sample log.
2019-11-19T13:02:13.262Z: INFO [push:store] SG counting common 1 stores X 1 apps
2019-11-19T13:02:13.720Z: INFO [push:store] SG counting 1 stores X 1 apps: got results for 5c28d5e68926b1763281597c / ap
2019-11-19T13:02:13.721Z: INFO [push:store] SG counting common 1 stores X 1 apps: done
2019-11-19T13:02:13.722Z: INFO [push:endpoints] Returning full audience for 5dd3e7d5574c490933d695d1: {"total":53143,"count":{"unknown":53143}}
2019-11-19T13:02:13.726Z: INFO [push:endpoints] Saved full audience for 5dd3e7d5574c490933d695d1: {"_id":"5dd3e7d5574c490933d695d1","apps":["5c28d5e68926b1763281597c"],"appNames":["MPSC Toppers"],"platforms":["a"],"source":"api","result":{"status":0,"total":0,"processed":0,"sent":0,"errors":0,"error":null,"errorCodes":{},"resourceErrors":[],"aborts":[],"nextbatch":null},"expiryDate":"2019-11-26T13:02:13.258Z","date":"2019-11-19T13:02:13.258Z","tz":false,"tx":false,"auto":false,"created":"2019-11-19T13:02:13.259Z","test":false,"v":18081,"build":{"total":53143,"count":{"unknown":53143}}}
2019-11-19T13:02:20.404Z: INFO [jobs:manager] Trying to start job {"_id":"5dd3e7db574c490933d695e3","name":"push:schedule","created":1574168539646,"status":0,"started":null,"finished":null,"duration":0,"data":{"mid":"5dd3e7d5574c490933d695d1"},"next":1574168539637}
2019-11-19T13:02:20.424Z: INFO [push:store] Note 5dd3e7d5574c490933d695d1 pushing users for 1 apps with date undefined over undefined
2019-11-19T13:03:02.510Z: INFO [jobs:job] Replacing job push:process ({"cid":"5c28d69f8926b1763281597f","aid":"5c28d5e68926b1763281597c","field":"ap"}) with date 1574168539637: {"_id":{},"status":0,"name":"push:process","next":{"$gte":1574168539637},"data":{"cid":"5c28d69f8926b1763281597f","aid":"5c28d5e68926b1763281597c","field":"ap"}}
2019-11-19T13:03:02.549Z: INFO [jobs:job] Replacing job push:process ({"cid":"5c28d69f8926b1763281597f","aid":"5c28d5e68926b1763281597c","field":"ap"}) with date 1574168539637: no future jobs to move
2019-11-19T13:03:02.554Z: INFO [jobs:job] Replacing job push:process ({"cid":"5c28d69f8926b1763281597f","aid":"5c28d5e68926b1763281597c","field":"ap"}) with date 1574168539637: found 0 existing jobs
2019-11-19T13:03:02.557Z: INFO [push:store] Note 5dd3e7d5574c490933d695d1 pushFetched results: [{"inserted":53143,"next":1574168539637,"collection":"push_5c28d5e68926b1763281597c_ap","field":"ap","cid":"5c28d69f8926b1763281597f"}]
2019-11-19T13:03:02.875Z: INFO [jobs:manager] Trying to start job {"_id":"5dd3e8068646fa0663c7f8bd","name":"push:process","created":1574168582555,"status":0,"started":null,"finished":null,"duration":0,"data":{"cid":"5c28d69f8926b1763281597f","aid":"5c28d5e68926b1763281597c","field":"ap"},"next":1574168539637}
Unhandled rejection for {} with reason {"errno":"ENOMEM","code":"ENOMEM","syscall":"spawn"} stack Error: spawn ENOMEM
at ChildProcess.spawn (internal/child_process.js:313:11)
at exports.spawn (child_process.js:508:9)
at Object.exports.fork (child_process.js:109:10)
at new ResourceFaçade (/usr/local/countly/api/parts/jobs/resource.js:194:27)
at ResourcePool.resources.(anonymous function).RES.ResourcePool [as construct] (/usr/local/countly/api/parts/jobs/manager.js:571:24)
at ResourcePool.getResource (/usr/local/countly/api/parts/jobs/resource.js:481:33)
at Manager.getResource (/usr/local/countly/api/parts/jobs/manager.js:588:34)
at IPCFaçadeJob._run (/usr/local/countly/api/parts/jobs/job.js:989:36)
at Manager.runIPC (/usr/local/countly/api/parts/jobs/manager.js:401:23)
at Manager.run (/usr/local/countly/api/parts/jobs/manager.js:523:25)
at job.prepare.then (/usr/local/countly/api/parts/jobs/manager.js:358:26)
at
at process._tickCallback (internal/process/next_tick.js:189:7)
2019-11-19T13:03:02.894Z: ERROR [core:api] Logging unhandled rejection
Trace
at process.on (/usr/local/countly/api/api.js:166:13)
at emitTwo (events.js:126:13)
at process.emit (events.js:214:7)
at emitPendingUnhandledRejections (internal/process/promises.js:108:22)
at process._tickCallback (internal/process/next_tick.js:190:7)
Hi again, your server seems to be out of memory. The best solution would be to move to a bigger server in terms of RAM or enable swap. Note that enabling swap is generally a very bad idea, but sometimes there's not much else to do, therefore we have a script for that: https://github.com/Countly/countly-server/blob/master/bin/scripts/make.swap.sh
Artem, Thanks a lot. It worked. I moved my server from 2GB to 4GB and it worked.
Thanks Countly!
You guys are amazing.
Please sign in to leave a comment.