Solana suffers 7 hour outage as bots invade the network

Published at: May 2, 2022

The Solana network suffered a seven-hour outage overnight between Saturday and Sunday due to a large number of transactions from the nonfungible token (NFT) minting bots.

A record-breaking four million transactions, or 100 gigabits of data per second, congested the network causing validators to be knocked out of consensus resulting in Solana going dark at roughly 8:00 pm UTC on Saturday.

It wasn’t until seven hours later on Sunday, 3:00 am UTC that validators were able to successfully restart the main network.

Validator operators successfully completed a cluster restart of Mainnet Beta at 3:00 AM UTC, following a roughly 7 hour outage after the network failed to reach consensus.Network operators an dapps will continue to restore client services over the next several hours. https://t.co/ezqEYQYKWl

— Solana Status (@SolanaStatus) May 1, 2022

The bots hoarded a popular application used by Solana NFT projects to launch collections called Candy Machine. In a Twitter post by Metaplex, the company confirmed that traffic from bots on their app was partially to blame for the network crash.

Today #Solana mainnet-beta went down partially due to botting on the Metaplex Candy Machine program. To combat this, we have merged and will soon deploy a botting penalty to the program as part of a broader effort to stabilize the network. https://t.co/QaAZT3VxXz

— Metaplex (@metaplex) May 1, 2022

Metaplex shared that it would be implementing a 0.01 Solana (SOL), or $0.89 at the time of writing, charge on wallets that attempt to complete an invalid transaction which the firm said “is typically done by bots that are blindly trying to mint.”

The outage caused the price of SOL, the blockchain’s native coin, to crash by nearly 7% to $84, although trading since has seen prices recover to just over $89.

The most recent outage marks the seventh time this year that Solana has suffered partial or full outages according to its own status reporting. Between January 6-12, 2022, the network was plagued with issues causing partial outages and degraded performance for between 8 and 18 hours.

Solana said “high compute transactions” caused a reduction in network capacity to “several thousand” transactions per second (TPS), much lower than the advertised 50,000 TPS.

Later in January, over 29 hours of partial outages and network instability were recorded between the 21st and 22nd of the month, with excessive duplicate transactions again causing network congestion and outages on the blockchain.

Related: Scalability or stability? Solana network outages show work still needed

In September 2021, Solana was hit with a major outage with the network offline for over 17 hours. Solana attributed that outage to a distributed denial-of-service (DDoS) attack on an initial decentralized exchange (DEX) offering with bots spamming the network with 400,000 per second. Industry observers commented on what has been often touted as an “Ethereum killer.”

At this point Solana should start posting business hours

— pseudotheos in (@pseudotheos) May 1, 2022

Solana was the second network to strain under notable transaction volume related to NFTs over the weekend. The Ethereum transaction cost surged to an average of over $450 due to a release of 55,000 NFTs by Yuga Labs with some users paying up to 5 Ether (ETH), or $14000, in gas fees for transactions and much more to mint one of the NFTs.

Update: Headline changed removing "7th outage in 2022", along with clarifying explanations that past outages included both partial and full outages.

Tags
Blockchain
Solana
Transactions
Bot
Consensus
Related Posts
Solana hit with another network incident causing degraded performance
The Solana blockchain has suffered a third incident in just a few months that clogged the network and caused transactions to fail, with users debating whether it was caused by another DDos attack or if it was just network issues. The scale and nature of the incident is hard to ascertain, with Coinbase, Wu Blockchain and Redditors reporting there had been an incident causing the network to slow and transactions to fail. However Solana Labs co-founder Anatoly Yakovenko denied there had been a DDoS on this occasion. The latest incident comes less than a month after a previous attack which …
Blockchain / Jan. 5, 2022
Report crowns Solana for using least energy per transaction, but there's a catch
Solana (SOL), one of the most active proof-of-stake (PoS) blockchains, appears to be a PoS protocol consuming the lowest amount of electricity per transaction, according to a new report. The Crypto Carbon Ratings Institute (CCRI), a research startup focused on the environmental impact of cryptocurrencies, released on Wednesday a new report calculating the electricity consumption and carbon footprint of major PoS blockchains. The CCRI specifically analyzed PoS networks including Cardano, Solana, Polkadot, Avalanche, Algorand and Tezos. According to the CCRI’s findings, the Solana blockchain consumed 0.166 watt-hours (Wh) of electricity per transaction within the study, becoming the most energy-efficient PoS …
Blockchain / Feb. 3, 2022
The blockchain trilemma: Can it ever be tackled?
Are L2 networks the saviors? The emergence of a new tier of blockchains called layer-2 chains that have leveraged new cryptographic techniques like zero-knowledge-proofs (ZKPs) has helped address the scalability enigma. While the world was busy solving the blockchain trilemma at the L1 level, there were a bunch of smart developers who understood that optimization need not necessarily happen at the L1 level. One can create a network on top of L1 networks to solve fundamental problems. There are a few L2 networks, such as Polygon, Immutable and the Bitcoin lightning network, which use concepts such as sharding and rollups …
Decentralization / Dec. 7, 2022
The state of Solana: will the layer-1 protocol rise again in 2023?
About two months after the FTX collapse, the Solana network is stronger than ever, according to Austin Federa, the head of strategy and communications at the Solana Foundation. Federa defines the recent SOL token price crash as a short-term market reaction to the perceived connection between Solana and the defunct crypto exchange FTX. While FTX founder Sam Bankman-Fried was invested in many Solana-based projects, Federa pointed out he didn't have any influence on the network’s operations and fundamentals. “The external perception was that there was a very close relationship between the Solana network and FTX, which wasn't the case," Federa …
Blockchain / Jan. 27, 2023
Solana CEO hoses down claims network outages caused by on-chain voting
Anatoly Yakovenko, the founder and CEO of Solana Labs has downplayed claims that Solana's network outages were being caused by a high volume of validator messages and its on-chain voting system clogging its consensus layer. While the Solana Foundation confirmed in a Feb. 27 post that the “root cause” of the recent 20-hour network outage is still not clear, the CEO responded to speculation that Solana’s decision to include on-chain votes as transactions is a “massive design flaw” that has led to its many outages. The controversial thread in question was posted by Twitter user DBCryptoX earlier on Feb. 27 …
Blockchain / Feb. 28, 2023