Bank Feed Service temporary outage resolved
Blog post · 18 Apr, 2012
READ ON
The main queued-job processor on the Bank Feed Service failed for 8 hours today. At 7:30am 18th April NZST / 3:30pm 17th April EDT an application job process ballooned in size, which resulted in monitoring software being shutdown, along with the job processors. Because the process monitoring software running on the server itself was down, this went unnoticed until this afternoon.
This means that during the 8 hour period between 7.30am - 3.30pm Wed 18th April NZST / 3.30pm - 11.30pm Tue 17th April EDT the Bank Feed service wouldn’t have been returning requests for account updates and addition etc.
We’re reviewing our monitoring configuration to prevent this from happening again, on a server level (to ensure the job processes can’t and won’t bring everything down around it) and on an overall systems level (so we’re thoroughly and repeatedly alerted if the monitoring software itself goes down).
We’ll be dropping a line to people who are interested in joining the new PocketSmith mobile alpha by this weekend - if you’d like to be involved, please get in touch with us!
