Digital asset exchange Coinbase confirms that it experienced an outage on June 1, 2020, that reportedly affected the operations of Coinbase.com (mainly for regular or retail traders), Coinbase Pro for professional or institutional traders, and it also impacted the exchange’s mobile applications.
Not all cryptocurrency trades performed via Coinbase’s platform were impacted by the outage, the San Francisco-based exchange claims in a June 6 blog post.
Coinbase clarified:
“Trading through the API, which accounts for the majority of trading volume, remained functional throughout this time. We quickly discovered the root cause and remediated the issue.”
At about 16:05 PDT, the Bitcoin (BTC) price reached (around) $10,000. The exchange says that it saw a 5x increase in traffic or user activity on its platform, due to the significant increase in BTC price.
According to Coinbase, the surge in traffic impacted its internal services, leading to an increase in “latency between services.”
Coinbase claims:
“This led to process saturation of the web servers responsible for our API, where the number of incoming requests was greater than the number of listening processes, causing the requests to either be queued and timeout, or fail immediately. Our request error rate spiked to 50%, causing customers to experience errors when interacting with coinbase.com and our mobile apps.”
Coinbase explained that their platform’s health check is served by these “saturated” processes, which reportedly led to several instances being flagged as “unhealthy and taken out of the load balancer, further exacerbating this issue.”
To address this issue, Coinbase says it redeployed the API at 16:20 PDT, which increased the machines serving the exchange’s higher than expected traffic. After completing the deployment, the previous deploy’s instances “were taken out of rotation,” which created yet another two minute outage because of “instances saturating and being marked unhealthy,” Coinbase claims.
However, this was “handled automatically by our autoscaling,” the exchange said.
Coinbase confirmed:
“We’re working on reducing the impact of price-related traffic spikes though pre-scaling and caching. Longer term we’re planning to improve our deployment process to mitigate some of the auto-scaling issues we experienced.”
It’s worth noting that these outages have become quite common for Coinbase, which is concerning considering the exchange was seeking an $8 billion valuation last year.
Coinbase was heavily criticized recently by many cryptocurrency and blockchain industry participants for trying to sell its blockchain security and analysis software to the US Internal Revenue Service (IRS) and the US Drug Enforcement Agency (DEA) only a year after it acquired intelligence agency Neutrino.
Both the DEA and the IRS reportedly plan to acquire licenses from the US-based exchange for an analytics platform (Coinbase Analytics).
The IRS noted:
“Coinbase Analytics (fka Neutrino) allows for the analysis and tracking of cryptocurrency flows across multiple blockchains that criminals are currently using. Coinbase Analytics also provides some enhanced law enforcement sensitive capabilities that are not currently found in other tools on the market. This action will result in a Firm Fix Priced purchase order, Period of Performance: One base year from date of award with one 12-month option.”
Public records reveal that the exchange hasn’t yet received the awards.