Web3 Foundation
https://web3.foundation
PythonAsyncIOQUICJamPackPyring+5 more
Tessera: Python JAM Client
Case Study: Implementing the Join-Accumulate Machine Protocol in Python
Project Overview
Transforming vision into reality
Client Information
Organization
Web3 Foundation
Website
web3.foundationProject Timeline
Start Date
April 2025
Completion
Invalid Date
Duration
Ongoing Development
Project Overview
A technical case study documenting the design and development considerations for Tessera, a Python-based client implementation for the JAM (Join-Accumulate Machine) protocol. This study explores how Tessera aims to interpret and implement the JAM specification outlined in the Graypaper, focusing on core protocol mechanics, state management, PVM interaction, consensus participation (client-side), networking, and cryptographic requirements necessary for a functional JAM node.
INTRODUCTION
Tessera: A Python Implementation of the JAM Protocol
Tessera represents an effort by ChainScore Labs to create a functional and spec-compliant client for the JAM (Join-Accumulate Machine) protocol, as detailed in the Graypaper. Written primarily in Python, Tessera aims to provide an accessible, well-documented implementation for researchers, developers, and network participants interested in interacting with or validating the JAM network. This case study outlines the architectural considerations, implementation approach, and challenges involved in building such a client.
PROTOCOL-UNDERSTANDING
Core JAM Concepts in Tessera
Implementing Tessera requires a deep understanding and accurate translation of the JAM Graypaper's core concepts into Python data structures and logic. Key areas of focus include:
- Hybrid Architecture: Modeling the distinction between on-chain (Accumulation) and in-core (Refinement) computation logic and their interaction.
- Service-Oriented Model: Implementing data structures and logic for service accounts, including state (storage, preimages), code (refine/accumulate entry points), balance, and gas limits.
- Coretime Concept: Handling core assignment, authorization checks (verifying authorizer logic outputs), and coretime accounting from a client perspective.
- Block & State Structure: Defining Python classes/objects for blocks, headers, extrinsics (tickets, preimages, reports, etc.), and the partitioned state tuple (α, β, γ...).
- State Transition Function (Y): Implementing the block-level state transition logic, ensuring deterministic updates based on prior state and block contents.
NETWORKING
Peer-to-Peer Networking Layer
Connecting to the JAM Network
Tessera requires a robust networking layer to discover peers, synchronize chain data, and propagate messages (blocks, transactions, consensus votes). This involves selecting or implementing a suitable P2P library (like LibP2P) and defining message types and protocols based on JAM's anticipated requirements.
- Peer Discovery: Implementing mechanisms like Discv5 or Kademlia DHT for finding other JAM nodes.
- Message Serialization: Defining and implementing serialization/deserialization formats (e.g., SSZ, RLPx, or a JAM-specific format) for network messages.
- Gossip Protocols: Implementing gossip logic for propagating new blocks, transactions (if applicable directly), and consensus messages efficiently.
- Request/Response Protocols: Defining protocols for requesting specific blocks, state parts, or historical data from peers during synchronization.
- Transport Layer: Utilizing secure and reliable transport protocols (e.g., TCP with TLS/Noise).
CONSENSUS-CLIENT
Consensus Mechanism Interaction (Client-Side)
While Tessera might not initially act as a block producer or finality voter, it must understand and verify the outputs of JAM's consensus mechanisms: Safrole for block production/fork choice and Grandpa for finality. This involves processing and validating consensus-related messages received from the network.
- Safrole Verification: Validating block author eligibility based on sealing keys derived from tickets/VRFs. Implementing the fork-choice rule based on Safrole's criteria (e.g., highest valid ticket score, chain length).
- Grandpa Verification: Processing Grandpa votes (prevotes, precommits) gossiped on the network. Verifying vote signatures against the known validator set. Identifying finalized blocks based on supermajority agreement (2/3+).
- Validator Set Management: Tracking the current and upcoming validator sets (κ, λ, γκ) based on state transitions and epoch changes defined in Safrole.
- Epoch Processing: Handling epoch transitions, including validator set rotation and Safrole state updates (roots, sealers).
BLOCK-PROCESSING
Block Processing and State Transition
Applying the JAM State Machine
A core function of Tessera is processing incoming blocks and applying the state transition function (Y) defined in the Graypaper. This involves validating block structure, processing extrinsics, executing/verifying PVM code (Accumulation), and updating the local state representation.
- Block Validation: Verifying header fields (parent hash, state root, extrinsic root), seal signature (Hs), VRF output (Hu), consensus markers (He, Hw, Ho).
- Extrinsic Processing: Handling different extrinsic types (Tickets ET, Preimages EP, Reports EG, Assurances EA, Disputes ED) according to their specific validation rules and state effects.
- State Transition (Y): Implementing the logic for updating each state component (α, β, γ...) based on the prior state and the processed block B, as defined in Graypaper section 4.2.1 and subsequent sections.
- Accumulation Logic Verification: Executing or verifying the results of the 'Accumulate' entry point for relevant service accounts affected by the block's work reports (W*).
- State Root Calculation: Calculating the posterior state root (H'r) after applying state changes, using the specified Merkle Trie structure, and comparing it to the expected root.
PVM-INTERACTION
PVM (PolkaVM/RISC-V) Integration
Executing and Verifying Service Code
JAM utilizes a PVM based on RISC-V (RV64EM) for executing service logic (Refine and Accumulate). Tessera needs to integrate with a PVM execution environment. This could involve embedding a Python-based RISC-V interpreter/JIT, or interfacing with an external PVM implementation (potentially written in a more performant language like Rust or C++ via FFI). The client must handle PVM initialization, gas metering, host function calls (Ω functions), and result/state change verification.
- Execution Environment: Integrating a RISC-V (RV64EM compatible) interpreter, JIT, or external process interface.
- Gas Metering: Implementing instruction-level gas counting according to PVM specifications to enforce computational limits.
- Host Function Interface (Ω): Providing implementations for all host functions callable from within the PVM (e.g., state access, historical lookups, cryptographic operations, logging).
- State Interaction: Allowing the PVM to securely read relevant parts of the JAM state and propose state modifications (primarily for Accumulate).
- Refinement Verification: Potentially re-executing or verifying proofs of refinement computation performed in-core by guarantors.
- Accumulation Execution/Verification: Executing the 'Accumulate' entry point for services based on work reports within the defined gas limits.
STATE-MANAGEMENT
State Management and Storage
Tessera needs an efficient way to store and query the JAM state, which is represented as a complex structure committed to via a Merkle Trie. This likely involves using a persistent key-value store (like LMDB or RocksDB) optimized for Merkle Trie operations. The implementation must handle state updates atomically per block, support efficient trie root calculation, and potentially implement state pruning.
- Data Store: Utilizing a performant key-value database (e.g., LMDB, RocksDB) suitable for large datasets and frequent updates.
- Merkle Trie Implementation: Implementing or using a library for the specific binary Patricia Merkle Trie structure defined in the JAM Graypaper (Appendix D), optimized for Python.
- Atomic Updates: Ensuring state changes from a block are applied atomically; either all succeed or none do, maintaining state consistency.
- State Root Calculation: Efficiently calculating the state root after each block application.
- State Pruning: Implementing strategies to prune old state data that is no longer needed (beyond finality + archival requirements) to manage storage growth.
- Preimage/Lookup Handling: Storing and managing preimage data (components p and l of service accounts) and implementing the historical lookup logic (Λ).
CRYPTOGRAPHY
Cryptographic Primitives
Implementing JAM's Security Foundations
Correct and performant implementation of JAM's cryptographic primitives is crucial for security and interoperability. Tessera needs to integrate Python libraries or bindings for:
- Hashing: Blake2b-256 (H) and Keccak-256 (Hk) for various commitments and identifiers.
- Signatures (Ed25519): Used for assurances (EA) and potentially other validator messages. Requires key generation, signing, and verification.
- Signatures (BLS12-381): Used for Grandpa finality votes and BEEFY commitments. Requires key generation, signing, verification, and aggregation.
- Signatures (Bandersnatch Schnorr-like): Used for block seals (Hs). Requires specific curve operations.
- VRFs (Bandersnatch RingVRF): Used for ticket generation (ET) and potentially Safrole leader selection. Requires proof generation and verification, and output derivation (Y).
- Serialization/Merklization: Implementing the specific encoding (Appendix C) and Merklization (Appendix D/E) rules for state and block components.
Metrics
Statistics
1M+
TPS
6s
Block Time
~682MB/s
Data Availability Throughput
~12s
Finality Time
CHALLENGES-PYTHON
Challenges and Considerations for Python
Building a high-performance blockchain client in Python presents unique challenges:
- Performance Bottlenecks: Python's Global Interpreter Lock (GIL) can limit CPU-bound parallelism. Cryptographic operations and PVM execution can be significantly slower than in compiled languages like Rust or C++.
- Memory Management: Handling large state tries and network buffers efficiently requires careful memory management to avoid excessive consumption or garbage collection pauses.
- Cryptographic Library Availability: Mature and performant Python libraries for newer primitives like Bandersnatch curve operations (VRFs, signatures) might be scarce or require bindings to C/Rust libraries.
- Concurrency Model: Leveraging `asyncio` effectively for network I/O and concurrent task handling is crucial, but requires careful design to avoid blocking operations.
- Potential Solutions: Utilizing optimized libraries (e.g., `numpy` for some math), implementing critical sections in C/Cython/Rust via FFI, aggressive caching, efficient data structures, and potentially multi-processing for CPU-bound tasks where possible.
LONG-RUN
Towards a Pythonic JAM Client
Tessera aims to provide a valuable contribution to the JAM ecosystem by offering a Python client implementation. While facing inherent performance challenges, careful architecture, leveraging optimized libraries, and potentially using FFI for critical components can lead to a functional and useful client for development, testing, and potentially certain network participation roles. This project underscores ChainScore Labs' commitment to exploring and implementing cutting-edge blockchain protocols.
Start Your Journey
Ready to Create Something Amazing?
Let's discuss how we can help you achieve similar results with a custom solution tailored to your specific needs and goals.
Start Your Project Today
Book a 30-minute strategy session with our experts to discuss your project goals, timeline, and technical requirements.
Explore Live Project
See the final implementation in action and explore the features we built for Web3 Foundation.
A project by
Web3 Foundation
100%
Success Rate
24/7
Support