Dreams in the Gulf Stream: Solana’s mempool-less protocol

Introduction

Blockchains have many components, and one of the most well-known and vital parts is mempools, or memory pools. This is partly because the two largest blockchains, Bitcoin and Ethereum, utilize mempools.
Consequently, the presence of a mempool has become synonymous with the concept of a "real" blockchain, leading some to argue that a blockchain cannot function without one. However, this conventional approach has inherent limitations: transactions residing in mempools are not processed quickly enough, and depending on the fees paid, could remain in the mempool for extended periods before being included in blocks. This delay creates bottlenecks, particularly during times of high network congestion, and exposed transactions to Maximal Extractable Value (MEV).

The Solana blockchain took a new approach in removing the mempool from their blockchain entirely and thus increased scalability, reduced block confirmation times and made for an overall easier and smoother user experience. In this article, we will cover what mempools are, how they are used, how Solana’s approach differs from other blockchains and what we can learn from it.

A brief background of mempools will help one better understand and appreciate Solana’s different take on this.

So what is a mempool?

A mempool, also known as a memory pool, transaction pool, or transaction queue, is an in-memory data structure within blockchain nodes that temporarily stores pending transactions before they are included in blocks. This temporary storage acts as a waiting area where transactions reside until they are validated and incorporated into the blockchain by miners (in the case of Proof of Work chains) or validators (in the case of Proof of Stake chains).

The mempool is a crucial component of the blockchain ecosystem, ensuring that all submitted transactions are recorded and processed in an orderly manner.

The prioritization of transactions within a mempool is primarily based on the fees attached to each transaction. Transactions with higher fees are given priority and are included in the next available block more quickly than those with lower fees. This fee-based prioritization mechanism incentivizes users to pay higher fees to expedite the processing of their transactions, especially during times of network congestion when the number of pending transactions can be significant.

This fee-based system ensures that miners or validators are compensated for their work in maintaining the network, aligning economic incentives with network security and efficiency

Prior to a transaction being added to the mempool, it must first pass a series of validation checks performed by a node. These checks ensure that the transaction conforms to the standard transaction specifications set by the blockchain protocol. If a transaction meets all the necessary criteria, it is added to the mempool; if it fails any of the checks, it is disregarded.

Sample checks done by nodes prior to transaction inclusion in the mempool includes but isn’t limited to:

• ●  Is the signature valid (i.e., the transaction has not been tampered with)?

• ●  Are all required elements – such as the To: and From: addresses – present and valid?

• ●  Is the From: address an external account (and not a contract account)?

• ●  If the transaction transfers value, does the From: address balance cover the value?

• ●  Does the transaction have a gas limit that’s below the block limit?

• ●  Does the transaction fit into the local mempool?

  The mempool in a blockchain network functions much like a waiting room in a medical office, where pending transactions are temporarily held before being included in a block. Before a transaction can enter this waiting room, it must first pass through a reception area where a receptionist, akin to a node in the blockchain, checks to ensure that all necessary requirements are met. This preliminary screening ensures that only valid and properly formatted transactions are allowed into the mempool.

Just as a receptionist verifies a patient's appointment details and personal information, a node performs various validation checks on a transaction. These checks include verifying the transaction's signature to ensure it hasn't been tampered with, confirming the presence and validity of required elements such as the "To:" and "From:" addresses, and ensuring the sender has sufficient funds to cover the transaction amount. Only after these checks are satisfactorily completed is the transaction added to the mempool, similar to a patient being admitted to the waiting room after passing the reception checks.

Once in the waiting room, patients must wait for their turn to see the doctor, which parallels how transactions wait in the mempool until they are picked up for inclusion in a block. This waiting period can vary depending on the transaction fee paid and the current level of network congestion. Higher-fee transactions are often prioritized, much like how urgent cases might be seen by a doctor sooner.

In these blockchain networks, there isn't a singular, global mempool; rather, each node maintains its own mempool. This means that the set of pending transactions can vary between nodes, as each one independently validates and stores transactions before they are included in a block. To facilitate better coordination and synchronization, Bitcoin Improvement Proposal 35 (BIP0035) introduced and allowed a node to expose its mempool to other nodes, enabling them to share information about pending transactions.

In blockchains like Bitcoin and Ethereum, the protocol used for propagating transactions is known as a gossip protocol. This protocol is a distributed propagation method where each node periodically sends messages to a subset of other random nodes.
In blockchain-specific implementations, this propagation involves the use of a hashed probabilistic data structure called a bloom filter. The bloom filter efficiently represents a hashed value for each node's mempool, which when compared with other node’s hashed values, helps identify any extra or missing transactions from other nodes, allowing for quick synchronization.

In summary, the use of mempools, while integral to the functioning of many blockchains, presents scalability challenges. Transactions must wait in line, often prioritized by the fees attached to them, leading to slower overall processing times. This can result in significant delays, especially for lower-fee transactions, which may languish in the mempool for an indeterminate period and impacts user experience.

However, mempool-based blockchains have certain advantages; namely, they ensure orderly transaction processing, maintain the sequence and integrity of transactions as they are included in blocks. This methodical approach provides a reliable and predictable framework for transaction verification and inclusion.

What is Gulf Stream protocol?

One of the eight key innovations of the Solana blockchain is Gulf Stream, a transaction caching and forwarding protocol that eliminates the need for a mempool. As covered above, traditional blockchain systems typically use a mempool to temporarily hold pending transactions before they are included in a block.

Gulf Stream revolutionizes this process by forwarding transactions directly to the current validator leader and the next few validator leaders, thereby making the mempool redundant in this network. This method enhances transaction throughput and reduces latency.

Each elected validator leader is responsible for producing four consecutive blocks within a limited timeslot of approximately 400 milliseconds. By preemptively caching transactions with the current and upcoming validator leaders, Gulf Stream ensures that these leaders have immediate access to pending transactions, allowing them to quickly include these transactions in the blocks they produce.

The seamless propagation of transactions from one validator to the next is facilitated by another Solana innovation called Turbine, but we won’t go into it as the functioning of this mechanism is beyond the scope of this article.
Turbine plays a crucial role, in tandem with Gulf Stream, to ensure that transactions are rapidly and securely propagated across the network.

The knowledge and selection of current and upcoming leaders via a leader schedule, as well as the known quantity of consecutiveness of block production, allows transactions to be forwarded directly to those validator, as opposed to transactions being added to a mempool awaiting inclusion in the next block. This process significantly enhances transaction processing efficiency.

A block is considered “final” when an additional 31 blocks are built on top of it. This concept of finality is important as it provides certainty that the transactions within the block are permanently recorded on the blockchain and cannot be altered or rolled back.

The proactive transaction forwarding mechanism of Gulf Stream is further strengthened by a stake-weighted validator selection process for transaction processing.

In this system, validators with the most stake (funds locked up with a validator to ensure the security of the Solana network) are given priority for processing transactions. This prioritization ensures that validators who have the most to lose, and are thus highly incentivized to maintain network integrity, are at the forefront of transaction validation.

Additionally, the stake-weighted selection process acts as an effective Sybil resistance mechanism, which is critical for preventing denial-of-service (DoS)-type attacks on the Solana network by deterring malicious actors who might try to flood it with excessive transactions.

Conclusion

In this article, we covered the history of mempools, their function, and how they work in various blockchains. We also explored how Solana’s different approach helps make it the fastest, most secure, and user-friendly blockchain.

Gulf Stream alone wouldn't achieve its significant impact; hence, Solana Labs introduced eight different innovations that, working in concert, create the blockchain's current performance and efficiency. These innovations collectively contribute to Solana's unique capabilities.

Not all blockchains need a mempool, as Solana has demonstrated for years. In fact, some Directed Acyclic Graph (DAG)-based protocols also operate without a traditional mempool, further highlighting the diverse approaches to blockchain architecture.