This incident has been resolved. The preliminary root cause is a sudden spike in concurrent executions, which affected the DB capacity. After increasing the capacity and processing the backlog the system is back to normal. A more thorough investigation and mitigation are to follow to improve system stability due to surge in concurrent requests.