Hyperledger Web3j: Truly decode support for dynamic Solidity structs

In Solidity, dynamic structs are complex data types that can store multiple elements of varying sizes, such as arrays, mappings, or other structs. The system encodes these dynamic structs into binary format using Ethereum's ABI (Application Binary Interface) encoding rules. The system encodes the structs whenever it stores or passes them in transactions.

Decoding this binary data is crucial for interpreting the state or output of a smart contract. This process involves understanding how Solidity organizes and packs data, particularly in dynamic types, to accurately reconstruct the original struct from its binary representation. This understanding is key to developing robust and interoperable decentralized applications.

Decoding dynamic structs in an external development environment that interacts with a blockchain network is challenging. These structs can include arrays, mappings, and nested structs of different sizes. They require careful handling to keep data accurate during encoding and decoding. In Hyperledger Web3j, we addressed this by creating object classes that match the expected struct format in the blockchain environment.

These object classes are designed to inherit from the org.web3j.abi.datatypes.DynamicStruct class, which is part of the ABI module. The developers designed this class to handle the complexities of encoding and decoding dynamic structs and other Solidity data types. The ABI module leverages Hyperledger Web3j's type-safe mapping to ensure easy and secure interactions with these complex data structures.

However, when the goal is to extract a specific value from encoded data, creating a dedicated object can add unnecessary complexity. This approach can also use up extra resources. To address this, our contributors, calmacfadden and Antlion12, made significant improvements by extending the org.web3j.abi.TypeReference class.

Their enhancements allow dynamic decoding directly within the class, removing the need to create extra objects. This change simplifies the process of retrieving specific values from encoded data. This advancement reduces overhead and simplifies interactions with blockchain data.

Decoding dynamic struct before enhancement

To clarify, here’s a code example that shows how you could decode dynamic structs using Hyperledger Web3j before the enhancements.

 
/**
* create the java object representing the solidity dinamyc struct
* struct User{ 
* uint256 user_id; 
* string name;
* }
*/
public static class User extends DynamicStruct {
   public BigInteger userId;

   public String name;

   public Boz(BigInteger userId, String name) {
       super(
               new org.web3j.abi.datatypes.generated.Uint256(data),
               new org.web3j.abi.datatypes.Utf8String(name));
       this.userId = userId;
       this.name = name;
   }

   public Boz(Uint256 userId, Utf8String name) {
       super(userId, name);
       this.userId = userId.getValue();
       this.name = name.getValue();
   }
}
/**
* create the function which should be able to handle the class above
* as a solidity struct equivalent 
*/
public static final org.web3j.abi.datatypes.Function getUserFunction = new org.web3j.abi.datatypes.Function(
               FUNC_SETUSER,
               Collections.emptyList(),
             Arrays.<typereference<?>>asList(new TypeReference() {}));

</typereference<?>

Now as the prerequisite is done, the only thing left is to call do the decode and here is an example:


@Test
public void testDecodeDynamicStruct2() {
   String rawInput =
           "0x0000000000000000000000000000000000000000000000000000000000000020"
                   + "000000000000000000000000000000000000000000000000000000000000000a"
                   + "0000000000000000000000000000000000000000000000000000000000000040"
                   + "0000000000000000000000000000000000000000000000000000000000000004"
                   + "4a686f6e00000000000000000000000000000000000000000000000000000000
";

   assertEquals(
  FunctionReturnDecoder.decode( 
          rawInput, 
         getUserFunction.getOutputParameters()),
          Collections.singletonList(new User(BigInteger.TEN, "John")));
}

In the above test, we decoded and asserted that the rawInput is a User struct having the name John and userId 10.

Decoding dynamic struct with new enhancement

With the new approach, declaring an equivalent struct object class is no longer necessary. When the method receives the encoded data, it can immediately decode it by creating a matching reference type. This simplifies the workflow and reduces the need for additional class definitions. See the following example for how this can be implemented:


public void testDecodeDynamicStruct2() {
   String rawInput =
           "0x0000000000000000000000000000000000000000000000000000000000000020"
                   + "000000000000000000000000000000000000000000000000000000000000000a"
                   + "0000000000000000000000000000000000000000000000000000000000000040"
                   + "0000000000000000000000000000000000000000000000000000000000000004"
                   + "4a686f6e00000000000000000000000000000000000000000000000000000000
";

TypeReference dynamicStruct =
       new TypeReference(
               false,
               Arrays.asList(
                       TypeReference.makeTypeReference("uint256"),
                       TypeReference.makeTypeReference("string"))) {};


List decodedData =
     FunctionReturnDecoder.decode(rawInput,					     
           Utils.convert(Arrays.asList(dynamicStruct)));

List decodedDynamicStruct = 				
((DynamicStruct) decodedData.get(0)).getValue();


assertEquals(decodedDynamicStruct.get(0).getValue(), BigInteger.TEN);
assertEquals(decodedDynamicStruct.get(1).getValue(), "John");}

In conclusion, Hyperledger Web3j has made great progress in simplifying the decoding of dynamic Solidity structs. This addresses one of the most challenging parts of blockchain development. By introducing object classes like org.web3j.abi.datatypes.DynamicStruct and enhancing the org.web3j.abi.TypeReference class, the framework now provides a more efficient and streamlined method for handling these complex data types.

Developers no longer need to create dedicated struct classes for every interaction, reducing complexity and resource consumption. These advancements not only boost the efficiency of blockchain applications but also make the development process easier and less prone to errors. This ultimately leads to more reliable and interoperable decentralized systems.