Non-JSON reply, HTTP status 503

[vird]

edge-core-js can crash app with unhandled rejection

Error: Non-JSON reply, HTTP status 503, /api/v2/store/ca41***/aedb***
at response.json.catch.jsonError (/usr/src/app/node_modules/edge-core-js/lib/core/storage/storage-servers.js:55:17)
at process._tickCallback (internal/process/next_tick.js:68:7)

There are no specific steps to reproduce, seems there is no retry logic here

edge-core-js/src/core/storage/storage-servers.js

Line 54 in 8131738

response.json().catch(jsonError => {

Seems sync is called without proper error wrapping

[swansontec]

Ah, I see the problem. The logic is supposed to rotate servers when anything goes wrong, but the rotation logic only applies to network errors, not errors where we can reach the server but it fails.

We need to do two things to fix this:

1. Get that server back online.
2. Fix the core's to do proper rotation in cases like this.

[swansontec]

#289