Blockchain Benchmarking Hyperledger Besu with Hyperledger Caliper

During my internship with Web3 Labs, I had the unique opportunity to delve into the world of Blockchain Benchmarking using Hyperledger Caliper, a tool designed to measure the performance of Ethereum Clients like Hyperledger Besu.

Benchmarking is the compass that can guide Blockchain Developers towards network optimization. It’s about understanding the limits and capabilities of a system under various conditions. I have specifically been working on Besu’s Private Network Setups using QBFT, IBFT and Clique.

Setting up Caliper to Benchmark Besu

For this post we will be benchmarking Besu’s QBFT Consensus Algorithm with an ERC-20 Workload.

Here are the steps to start with Caliper:

1. Install the Caliper NPM Package

npm install --only=prod @hyperledger/caliper-cli@0.5.0

2. Check if it is installed correctly.

npx caliper --version

3. Bind it to the latest version of Besu.

npx caliper bind --caliper-bind-sut besu:latest

4. Clone the caliper-benchmarks repository. This repository comes with some predefined Workloads against which you can run benchmarks.

git clone https://github.com/hyperledger/caliper-benchmarks.git

cd caliper-benchmarks

6. Start your QBFT network. More details on that can be found here.

7. Setup a network config file inside the `caliper-benchmarks` folder that we cloned earlier. A new folder called QBFT-Network can be created at the following path.

mkdir networks/besu/QBFT-Network

8. Inside this `NetworkConfig` File, we want it to point towards the Web Socket URL, where our Besu Network is running. For this example, it is pointing to, `ws://51.159.75.80:8546`. Also, the ABI for the ERC-20 smart contract has been left out from here, this can be added inside the `abi` field.

{
"caliper": {
"blockchain": "ethereum",
"command" : {}
},
"ethereum": {
"url": "ws://51.159.75.80:8546",
"contractDeployerAddress": "0xf17f52151EbEF6C7334FAD080c5704D77216b732",
"contractDeployerAddressPrivateKey": "ae6ae8e5ccbfb04590405997ee2d52d2b330726137b875053c36d94e974d162f",
"fromAddress": "0xf17f52151EbEF6C7334FAD080c5704D77216b732",
"fromAddressPrivateKey": "ae6ae8e5ccbfb04590405997ee2d52d2b330726137b875053c36d94e974d162f",
"transactionConfirmationBlocks": 10,
"contracts": {
"Erc20": {
"address": "0xd9bB4402537D044709BC80b666F522A6F0ce6435",
"estimateGas": true,
"gas": {
"transfer": 800000
},
"abi":
[

]
}
}
}
}

9. We will be benchmarking the `transfer` method inside the ERC-20 smart contract. To set up this workload, the following code can be put inside a new folder called `ERC-20` and be used. The 4 main files that we require here are the following:

transfer.js

'use strict';

const OperationBase = require('./operation-base');
const SimpleState = require('./simple-state');

class Transfer extends OperationBase {
constructor() {
super();
}

createSimpleState() {
const accountsPerWorker = this.numberOfAccounts / this.totalWorkers;
return new SimpleState(this.workerIndex, this.moneyToTransfer, accountsPerWorker);
}

async submitTransaction() {
const transferArgs = this.simpleState.getTransferArguments();
await this.sutAdapter.sendRequests(this.createConnectorRequest('transfer', transferArgs));
}
}

function createWorkloadModule() {
return new Transfer();
}

module.exports.createWorkloadModule = createWorkloadModule;

simple-state.js

'use strict';

const Dictionary = 'abcdefghijklmnopqrstuvwxyz';

class SimpleState {

constructor(workerIndex, moneyToTransfer, accounts = 0) {
this.accountsGenerated = accounts;
this.moneyToTransfer = moneyToTransfer;
this.accountPrefix = this._get26Num(workerIndex);
}

_get26Num(number){
let result = '';

while(number > 0) {
result += Dictionary.charAt(number % Dictionary.length);
number = parseInt(number / Dictionary.length);
}

return result;
}

getTransferArguments() {
return {
target: "0xf17f52151EbEF6C7334FAD080c5704D77216b732",
amount: this.moneyToTransfer
};
}
}

module.exports = SimpleState;

operation-base.js

'use strict';

const { WorkloadModuleBase } = require('@hyperledger/caliper-core');

const SupportedConnectors = ['ethereum'];

class OperationBase extends WorkloadModuleBase {

constructor() {
super();
}

async initializeWorkloadModule(workerIndex, totalWorkers, roundIndex, roundArguments, sutAdapter, sutContext) {
await super.initializeWorkloadModule(workerIndex, totalWorkers, roundIndex, roundArguments, sutAdapter, sutContext);

this.assertConnectorType();
this.assertSetting('moneyToTransfer');
this.moneyToTransfer = this.roundArguments.moneyToTransfer;
this.simpleState = this.createSimpleState();
}

createSimpleState() {
throw new Error('Simple workload error: "createSimpleState" must be overridden in derived classes');
}

assertConnectorType() {
this.connectorType = this.sutAdapter.getType();
if (!SupportedConnectors.includes(this.connectorType)) {
throw new Error(`Connector type ${this.connectorType} is not supported by the benchmark`);
}
}

assertSetting(settingName) {
if(!this.roundArguments.hasOwnProperty(settingName)) {
throw new Error(`Simple workload error: module setting "${settingName}" is missing from the benchmark configuration file`);
}
}

createConnectorRequest(operation, args) {
switch (this.connectorType) {
case 'ethereum':
return this._createEthereumConnectorRequest(operation, args);
default:
// this shouldn't happen
throw new Error(`Connector type ${this.connectorType} is not supported by the benchmark`);
}
}

_createEthereumConnectorRequest(operation, args) {
return {
contract: 'Erc20',
verb: operation,
args: Object.keys(args).map(k => args[k]),
readOnly: false
};
}
}

module.exports = OperationBase;

config.yaml

simpleArgs: &simple-args
moneyToTransfer: 1

test:
name: ERC20
description: >-
To benchmark transferring tokens of an ERC20 contract.
workers:
number: 1
rounds:
- label: 1000 transfers with 100tps
description: >-
Transfer ERC20 Tokens between accounts.
txNumber: 1000
rateControl:
type: fixed-load
opts:
transactionLoad: 100
startingTps: 100
workload:
module: benchmarks/scenario/simple/ERC-20/transfer.js
arguments:
<< : *simple-args

10. Running the final command!

npx caliper launch manager \

--caliper-benchconfig benchmarks/scenario/ERC-20/config.yaml \

--caliper-networkconfig networks/besu/QBFT-Network/NetworkConfig.json \

--caliper-flow-skip-install \

--caliper-workspace .

This should start a benchmark run with ERC-20 token transfers on your QBFT network.

More Workloads, Common Errors, etc.

The code for this ERC-20 workload and also the ERC-721 workload that runs a benchmark for minting and transferring ERC-721 tokens can be found here.

A common problem faced while trying to achieve a high TPS number while benchmarking using Caliper is the nonce issue which happens due to one transaction failing, and the remaining failing because there is no Nonce fix from Caliper’s side. This can be fixed by resending the failed transaction with the corrected nonce to ensure that the further transactions don’t fail. More on that can be found in this PR to Caliper.

One of the most significant challenges I encountered was ensuring the accuracy of the benchmarking setup. It was crucial to simulate real-world conditions accurately to obtain meaningful data. This meant configuring the network settings meticulously, understanding the nuances of the ERC-20 workload, and ensuring that every step, from installation to execution, was executed flawlessly.

Another challenge was dealing with unexpected issues, such as the Nonce Issue, which threatened the reliability of our results. Tackling these issues required a combination of technical expertise, problem-solving skills, and creativity. It was a process of trial and error, learning from each setback, and adapting our approach.

The reason for testing the QBFT network with Caliper for this blog post is because QBFT is the recommended enterprise-grade consensus protocol designed for private enterprise networks developed by ConsenSys, and supported by Web3 Labs through our Hyperledger Besu support.

At Web3 Labs, we offer comprehensive support for QBFT/IBFT2 through our Hyperledger Besu support program. Our expertise in blockchain development and deployment allows us to assist organisations in setting up and maintaining private network infrastructures based on QBFT and IBFT2 protocols. Whether you need guidance in implementing, or require production support for your private blockchain network, our team is here to help.