Building a Blockchain: The Grey Paper

Background

The blockchain hype train is at an all time high! There are countless articles and news stories calling it the greatest invention since the internet. On the other hand, there is an equally large camp saying that although the tech is great, there is no real use case.

Until recently, I’ve been somewhere in the middle. Intrigued by blockchain I began researching it. After reading several articles and white papers, I felt comfortable with some of the terms and concepts. However, I still didn’t have that “aha” moment. The content I was reading was too high level, and didn’t clearly convey the power and potential of building distributed, decentralized applications.

Two weeks ago Blockchain Academy TLV kicked off its inaugural session. Blockchain Academy’s premise is to promote active and productive technical blockchain discussion.

My aim is to explain what I’ve learned so far on my blockchain journey, and to walk you through an Ethereum inspired blockchain implementation I’ve been working on the past 2 weeks.

What I cannot create, I do not understand

During Blockchain TLV’s first meeting, Leonid Beder discussed blockchain basics, and live coded a basic blockchain implementation in Typescript. The implementation included the following functionality:

Blockchain: a continuously growing list of records, called blocks, which are linked and secured using cryptography. Each block typically contains a hash pointer as a link to a previous block, a timestamp and transaction data. We defined an initial class with the following interface:

• We’d store all the blocks in the blockchain.
• We’d store the pool of all of the transactions waiting to be processed.
• Method for creating new blocks
• Method for submission of new transactions into the pool.

Blocks: Every block in our chain will contain

• Block number (i.e. “3rd block in chain)
• transactions
• Timestamp — block creation time
• Nonce — arbitrary number that can only be used once
• The hash of the previous block — by using the previous hash, we make the blockchain immutable

Transactions: We defined a transaction class that contained

• The public address of the sender
• The public address of the recipient
• The value of the transaction.

Submit a Node: A blockchain is a distributed, decentralized database, where every node on the network holds a complete copy of transaction history. Since we’re developing a permissions-less blockchain — everyone is free to join. This method allows us to add nodes to the network.

Submit a Transaction: A blockchain serves as a record of transaction history. Although a transaction can be defined broadly, it’s most popular use case is a transfer of funds from user A to user B. This method lets us create transactions, waiting to be mined.

Mine a Block: Transactions aren’t reflected across the network immediately. We need to invest computational power into validating transactions. This is called Proof of Work. Mining a block is the action of performing “Proof Of Work” on all of the transactions that have recently been emitted but not yet validated. No free lunches in cryptocurrencies ;)

Consensus: Consensus is the process of all nodes reaching agreement on the current state of the network. We choose “the longest chain” (Greedy Heaviest Observed Subtree) approach. This approach states that all nodes will accept the blockchain of the node with the most blocks. The rationale behind this, is that the longest blockchain has the most computational power invested in it (remember, we require blocks to be validated via PoW), and is therefore the most reliable / believable.

Check out the workshops source code here.

Although it was a lot of new material to process in a hour, I was hooked :) Towards the end of the workshop Leonid announced a surprise!

I like surprises

There would be a competition to see who could make the most creative and “awesome” pull request to the basic blockchain we had just built.

My work over the past 2 weeks

I had been itching to learn more about blockchain, and believe that building one is the best way to do that. I set out to build a blockchain inspired by the Ethereum implementation with the following features:

• Accounting system - users will be able to register accounts (external accounts) with a balance, and initiate transfers of funds. Additionally, users can register contract accounts and deploy smart contracts across the network
• Transaction-based state machine
• Secured and validated transactions and state transitions
• Enable users to write smart contracts and decentralized applications where they can create their own arbitrary rules for ownership, transaction formats and state transition functions.

Accounts

The global shared state of nodes on the network is composed of accounts. Accounts holds important information such as

• address: this will serve as the name of the account, similar to what we did in the workshop.
• balance: amount of funds owned by the account
• nonce: If the account is an externally owned account, this number represents the number of transactions sent from the account’s address.

My blockchain implementation has 2 different types of accounts:

• External accounts, which are controlled by private keys (via RSA encryption)
• Contract accounts, which are controlled by their contract code.

Single Shared State Across Network

Accounts are public, and you can view any account given its address, and node it was registered on. I made accounts “global” by informing all nodes of the creation of an account on the network. So when account transactions are dispatched to a node, I propagate that transaction to all nodes on the network. This way, the blockchain is truly decentralized, and we don’t depend on single nodes for valuable information like account existence or account balances. Essentially we have created a singular “shared-state” between nodes.

We register each account on every node, so that it is “global”Differences between External and Contract Accounts

The main difference between both types of accounts is rooted in how they can communicate with other accounts across the network. An external account can communicate in 2 ways

1. Messaging other external accounts (i.e. a transaction which moves funds from account A to B).
2. Invoking method execution on contract accounts. These methods vary, as they are solely dependent on the contract author and the contract that he / she created.

It is important to clarify that contract accounts, in contrast to external accounts, cannot explicitly dispatch transactions to the network. Instead, contract accounts can only emit transactions in response to other transactions they have received.

External Accounts

Authenticating accounts is done via RSA encryption. Upon account initialization a private, public key pair is created and written to disk.

Keys being created in the ExternalAccounts constructor. On the right, we view the public key

These keys are used to digitally sign outgoing transaction requests with the account credentials. A more robust implementation would always ask an account, whether or not he wants to sign the request before sending it. You can view a rudimentary implementation of this in requestAuthForTransaction.sh.

requestAuthForTransaction.sh

An assumption I’ve made for this implementation is that all accounts agree to sign all transactions requests with their credentials.

Contract Accounts

Contract accounts are controlled by their code. They simply execute a specific piece of code when invoked, and have direct control over their balance.

One of the challenges here was figuring out how to support this in a dynamic, distributed, decentralized, trust less manner. Ethereum uses a globally accessible virtual machine (given a pointer to the contract’s byte code) to execute the contract’s code. But JS doesn’t have a globally accessible VM out of the box for all the nodes to refer to. I initially tried to deploy a contract to all nodes on the network as a stringified JSON object, so that contract execution could be invoked independently on each node by parsing the received object. Unfortunately, JSON.stringify ignores methods and functions, therefore stripping smart contracts of their functionality on writes. My workaround for sending contracts dynamically (not hard coded on every node instance) across the network was as follows:

Deploying a Contract

1. Write a smart contract as a plain JS object.

2. Wrap it with parenthesis to make it a string, and remove all newlines so it can be processed via the built in JS eval method when other nodes receive it.

3. Send the string as data to all nodes on the network.

Deploying a contract across the network

This allows nodes to create contracts on the fly and deploy them across the network. Now, a node can receive a contract it didn’t know about before, and execute its code! 😀😀😀

Writing contract updates

Initially we can parse the stringified contract by using the built-in JS eval method, which takes a string and evaluates it as a JS statement.

After executing a method that mutates contract state (i.e. changes balance of contract) we want to store the contract in its entirety, without losing access to our methods.

Therefore, we will use the JSON stringify method, and pass a custom “replacer” function, instructing it on how to stringify functions.

In the future, to read the specially stringified version of the contract we will pass a custom “reviver” to the JSON.parse method.

Emitting Transactions via Contract Execution

Some contract mutations may only update the internal state of the contract. Other contract executions can “emit” transactions that effect the state of the network. For example, a contract holding funds (contracts govern their own funds) can send them to a specific user upon a certain condition being met. These emitted transactions are listened for, and placed into the mining queue, similar to other transactions in the network. They must be validated before being written to the blockchain. The flow looks like this:

An emitted transaction via a contract mutation request, appended to the mempoolSmart Contracts with Permissions

Calling a smart contract simply invokes a method that results in a transaction emission. Contracts are deployed globally and we’d like an easy way to control who can invoke contract methods. We’d like to enable users to build contracts that support user permissions. For example in an election, we’d only want eligible users to be able to vote (we can determine eligibility however we see fit.). Because all requested transactions are digitally signed via RSA encryption by the requester, we can safely check the user who requested to execute the contract and decide whether or not he is authorized to do so.

DAOVotingContract.ts example with permissionsTransaction Validation

During the first workshop we implemented a raw version of transactions. Because we had no accounts, no form of identification, and no balances to update, they had no meaning. After implementing the above, we can now verify the legality, or “correctness” of a requested transaction

State Transition Function

Ethereum’s white paper describes a cryptocurrency ledger as a state transition system, where state is composed of ownership status of currency within the system. In my implementation every node has accounts who hold a specific balance. But how does state (balance) change over time? Ethereum specifies a State Transition Function as follows:

Illustration from the Ethereum White Paper

In our implementation we will take a similar approach to validating state transitions. We will:

1. Check if a transaction adheres to the requested structure and has all the necessary values.
2. All transactions have a senderNodeId and a senderAddress. Although this is an unsafe assumption, for the current implementation and for lack of a proper user client, we will assume that all accounts agree and digitally sign all outgoing transaction requests with their credentials. Before being submitted to the mining queue, we will verify this digital signature.
3. Check that the nonce matches the nonce in the sender’s account.

blockchain.ts

After verifying the transaction via the stateTransitionValidation method we can add the transaction to the mining queue and mine. It is important to note that these transactions have not yet mutated the network state. Upon consensus, if these transactions belong to the longest chain, and it is a transaction that moves funds we will validate that the sender has an adequate account balance. An example of filtering illegal transactions (sender A sending more funds then he owns) can be see in adequateFunds.sh.

adequateFunds.sh

To visualize this process:

Example Contracts

Now that our blockchain has accounts, validated transactions and a way to create and propagate contracts across the network we can look at some real contracts. These contracts are pretty straight forward, and showcase the flexibility and potential of authoring smart contracts on the blockchain.

Counter Contract

To start things off let’s look at a simple contract that acts as a counter. Every time the contract’s incrementValue method is called we will increment the counter. Notice, that anyone can invoke this contract code.

We invoke the contract code by dispatching a mutateContract/:CounterContract PUT request. This contract has its own internal state which it governs. When incrementValue is called the contract state emits a transaction, requesting to mutate the contract state. After being mined, the mutation is written to the blockchain, so that we have a record of who used the contract (initiated the mutation request) and how the contract was used.

This contract can be executed via the Postman client or by running the counter_contract.sh script in the shell-scripts directory.

Transfer funds after arbitrary date Contract

A more realistic scenario for a smart contract is moving funds from account A to account B. The following contract is based on the scenario that Bob wants to transfer 400 coins to Alice, after an arbitrary date has passed. Instead of using a third party service, or escrow system, he can create a smart contract that checks when the date has passed, and initiate a transfer of funds.

When the contract’s moveFunds method is invoked it will check the date, see that it has passed and transfer funds to Alice.

This contract can be executed via the Postman client or by running the move_funds_after_data_contract.sh script in the shell-scripts directory.

DAO Vote for Bonus Contract

This scenario involves a DAO (Decentralized autonomous organization) that wants to give one of its employees a bonus. Because this DAO is a completely fair organization 😀😀😀, we should vote on who is deserving of the bonus.

But remember, the blockchain is a public ledger! Therefore, the moment we upload to contract to the network, anyone can see it, and make calls to the contract code. We’d like a way to restrict the voting to only the DAO’s employees. As I mentioned earlier, every request with an accounts credentials, is digitally signed by that account, with their private key. This way, it is easy to verify the voters true origin. So this contract has an “authorizedVoters” field which we will check against, every time an account tries to cast a vote.

This contract can be executed via the Postman client or by running the voting_permissions_contract.sh script in the shell-scripts directory.

These are only several examples of smart contracts. The current structure allows anyone to upload any contract (as long as it adheres to parsable format I am using), deploy it to the network in session, and start executing it. We can create contracts for elections, distributed file storage and more.

Conclusion

Programming this proof-of-concept blockchain was quite a challenge! It is only the tip of the iceberg in terms of ways it can be improved and secured. I had a great time researching the concepts, and learning about blockchain implementations and mechanisms.

A huge thank you, to Leonid Beder for the graceful blockchain introduction, and to Kin Contributors, Orbs and Samsung Next for organizing Blockchain Academy.

You can check out my full implementation here.

If you’d like to continue the blockchain discussion, feel free to reach out via Linkedin.

Lastly, if you have made it this far, thanks for sticking with it 🤗🤗🙌🙌🖖🏼✌️🏼

Blockchain - Hacker Noon

Building a Blockchain: The Grey Paper was originally published in Hacker Noon on Medium, where people are continuing the conversation by highlighting and responding to this story.