How To Use Sharding Without Sacrificing Security

When you partition a large database into many smaller and even quicker but more manageable parts, it is called Sharding in layman’s term. It simply means that you have a big database that you break down into many tiny databases that can be proliferated across a number of servers.

There’s this fuss around related to the validation mechanism in Blockchain implementation. With sharding, this validation process only strengthens. Through this process, only a rather smaller group of nodes aka shard, not the entire network node, will do the validation for each transaction.

In case of blockchain, the scene is not so complex. Blockchain network gets divided logically into a number of smaller nodes or shards. When a user creates a transaction, the validation of the shard will be carried out by the peers in the shard group. However, there’s something smarter than just sharding out in world with more potential, it goes by the name Adaptive State Sharding.

Elrond, the first blockchain company to introduce and apply the application of Adaptive state Sharding. The extent of this tech implementation can be measured by the fact each shard is capable of carrying more than 3750 transactions per second.

But why the Term Adaptive?

When there is a catapult increase in throughput, new shards get activated on its own, all thanks to its adaptive nature, achieving linear scalability in Blockchain.

This network has been under development for nearly two years. In the meantime, the blockchain space has witnessed an increase of interest and hence dedicated resources; thus, incredible amounts of development has already taken place on existing infrastructures, particularly Ethereum. A third pillar of the network’s infrastructure is interoperability. The blockchain is in compliance with Ethereum Virtual Machine (EVM). As Ethereum has been the foundation for many independent blockchain developers, this Network will be the most attractive solution to existing projects that seek more scalability. To further ease the on-boarding process, the Elrond blockchain offers reverse compatibility with Ethereum’s ERC-20 token standard.

This network is delivering scalability, security, and interoperability in a truly decentralized network. The core features of the blockchain have proven to be functional during network tests. The project’s test net will be live soon, and developers will have an early opportunity to build upon, arguably, the most advanced blockchain to date.

Here’s a breakdown of the layers that make up this blockchain.

Cryptographic Layer

At the very base of any blockchain is the cryptographic layer. This is essentially the DNA of the blockchain as this layer is the design for the terms of transaction and block verification.

The network utilizes Schnorr Scheme for transaction signature and verification; the use of Schnorr signatures creates a smaller data footprint. Beyond being simple and efficient, the Schnorr Scheme utilizes a well-researched and battle-tested algorithm. This means of creating a digital signature is already present in various altcoins and proposals have been made to integrate it in the Bitcoin network as-well.

As block verification requires an aggregated signature from multiple validators, a multi-signature scheme is needed. Thus, the Network will use the Boneh-Lynn-Shacham (BLS) multi-signature scheme for block signature and verification.

Core and Execution Layer

In a blockchain, blocks, transactions, and accounts all exist in the form of data; all this data is assembled in the blockchain’s core. Data models in the network’s core can enable temporary or permanent storage.

Transactions and addresses exist as data; by decoupling the data model from the access point, the core buffers transactions before they can be assembled into a block.

The execution aspect of the layer is to ensure once data inputs have been validated and the blockchain fulfills its transactional dues. The execution layer processes transactions and then assembles them into blocks; the executed transactions and blocks are then maintained across all nodes to ensure consistency. Thus, the execution layer also sustains node synchronization in parallel on all shards; the synchronization between shards is done at the Metachain.

Communication Layer

This network is a decentralized protocol and thus effective communication between nodes is of essence. The communication layer is the design for messaging and broadcasting, and the channels for the communication.

For security purposes, the network randomly repositions nodes from various shards. Moreover, it’s imperative that all nodes remain in synchrony on transactional throughput. The network must be able to relay communication within the nodes of a shard and within all the shards in the network and the communication layer enables this. The usage of both intra-shard and cross-shard communication channels also ensures that the network processes compound transactions properly.

Put simply, while the core and execution layer processes the data and transfers, the communication layer provides the channel for movement once the processing is complete.

The communication layer ensures node connectivity, relays the back-and-forth transfer of data within the network, provides a channel for information request, and handles the broadcasting of blocks, transactions, and receipts within the nodes of a single shard and across all the shards within the network. The communication is based on a highly optimized usage of libp2p from IPFS. Network sharding ensures that communication is limited to where it is needed. Specifically, intra-shard communication takes place between only interested nodes. This enables interoperability among the chains without causing a data flow bottleneck.

Consensus Layer

Blockchains are able to function as secure and decentralized networks by maintaining a consensus. This is a fundamental aspect of any robust blockchain. After the core and execution layer has processed network functions and the communication layer has relayed it to shards, the consensus layer acts as a mechanism to ensure the throughput validates only honest data.

Network’s Secure Proof of Stake (SPOS) is based on practical Byzantine Fault Tolerance (pBFT). Byzantine faults are the faulty or dishonest data on a decentralized network. As the name suggests, pBFT exists to ensure a consensus layer is able to withstand said faults. pBFT consensus assumes that no more than 1/3 of the nodes in a network will be malicious. Thus, in Elrond Network, a block is only validated when 2/3 + 1 of the validator nodes aggregate a signature.

In the SPOS mechanism, each shard comprises validators whose eligibility is determined via stake, which is held in a smart contract, and rating. A lower rating decreases the selection odds of a validator. With each block, a new validator group is selected at random through a roulette-wheel selection. The provably decentralized nature of random selection ensures that attack vectors can never predict who the validator will be, and thus they cannot know which nodes to take over to maneuver an attack.

Based on pBFT’s leader-follower model, SPOS assigns the first node in a validator group as the block proposer. Thus, both the proposer and the validators are selected at random, allowing the consensus layer to offer optimal security to the network.

A benefit of the SPOS is that, unlike proof-of-work protocols, it is eco-friendly. As nodes are selected based on a stake and rating, there is no concern for network hash being concentrated by two or three mining farms. Instead, anyone can become a node, and as this network requires minimal resources to allow active node participation, the blockchain remains truly decentralized.

Adaptive State Sharding

Adaptive State Sharding comprises the sharding of network/communication, transaction/computation, and state/storage. To get a deep-rooted grasp of what Adaptive State Sharding entails, it’s important to understand the purpose of each single form of sharding.

Transaction/Computation Sharding: Transaction sharding is a mechanism being pursued by many projects. It tackles the caveats of PoW, in which each node must approve every transaction. In transaction sharding, nodes are split into groups (shards) that process different transactions in parallel.

Network/Communication Sharding: Data (messages) is partitioned across shards.

Stage/Storage Sharding: Blockchains must store data of the entire history of the transactions they process. Rather than having every node store a copy of all the data, state/storage sharding distributes the storage burden across different groups of nodes (shards).

Elrond Network integrates all three forms of sharding and the result of this is a network that scales proportional to the number of validators and shards. A metachain coordinates the shards and ensures new shards are activated as more nodes join the network. Shards can be added seamlessly because wallets are divided between the shards through a hierarchical binary tree model. Addition of a shard passes half the address space of a parent shard to the sibling shard. Meanwhile, removal of subsequent shards merges the address space from sibling shards back to the parent shards.

Cross-Shard Transactions

Elrond Network allows transactions to be sent from one shard to another. As the network utilizes an asynchronous model, validation and processing is first done in the sending shard and then in the receiving shard. When a transaction is dispatched, the Metachain secures it by notarizing the block from the sending shard through the creation and proposition of a new metablock, which are the blocks created on the Metachain.

The metablock contains the following information about each shard-block: sender shard ID, receiver shard ID, and shard-block hash.

In a cross-shard transaction, the receiving shard fetches the transaction’s relevant shard-block (the blocks created in shards, not Metachain) hash from the metablock, requests the shard-block from the sending shard, parses the transaction list, requests missing transactions (if any), and then finally executes the same shard-block in itself and sends this shard-block to the metachain’s resulting metablock. Once this is notarized by the Metachain, the transaction is finalized.

Wrap Up

Elrond Network is the result of highly focused research and development as-well-as the implementation of various novel blockchain infrastructural solutions. Each layer plays a vital role — from processing network throughput to communicating it or validating it. The network is able to expand the capacity of certain layers by adding new shards via its native use of Adaptive State Sharding. The Secure Proof of Stake (SPOS) consensus ensures that, despite the decentralized nature of the blockchain, the implementation of a stake and rating system for block validation candidacy, along with a provably random means to select block proposer and validators from the candidates, the network remains secure.

Elrond Network is pinned on the principles of scalability, decentralization, and interoperability. With the ability to process 3,750 transactions per second per shard, and ensuring even a simple laptop can run a node, the network lives up to its first two pillars. The Elrond Virtual Machine’s compatibility with the EVM delivers interoperability, ensuring that the expansion of blockchain’s utility, in industries and use-cases, that demand greater throughput, can be easily transferred from other networks to Elrond. Find more information at https://elrond.com and become part of our growing community.

How To Use Sharding Without Sacrificing Security was originally published in Hacker Noon on Medium, where people are continuing the conversation by highlighting and responding to this story.