- For details about the technical architecture and to contribute, see the GitHub repository.
- For questions, reach out on the Polygon R&D Discord.
What is the Chain Indexer Framework?
The Chain Indexer Framework is a blockchain data indexing framework for building event-driven data pipelines on EVM blockchains. It is built on Kafka and provides the core logic and helper functions needed to collect raw blockchain data, transform it, and make it queryable by a dApp backend. The framework handles three stages:- Fetch raw blockchain data via RPC and store it in a Kafka stream.
- Transform the raw data using the framework’s helper functions, based on your dApp’s requirements.
- Store the transformed data in a database (such as Postgres or MongoDB) that your dApp’s API layer can query.
Why use it?
Without an indexer, querying blockchain data requires iterating through every block and transaction. This is slow, expensive in RPC calls, and becomes impractical as chain history grows. The Chain Indexer Framework solves this by indexing historical blocks once and maintaining a live stream of new events. Once indexed, the data is immediately accessible to any number of application layers without repeated RPC queries.Key properties
| Property | Description |
|---|---|
| Open source | Fork, modify, and host on your own infrastructure. No third-party rate limits. |
| TypeScript | Easy to install via NPM and integrate into existing Node.js projects. |
| Modular architecture | Producers, transformers, and consumers are separate modules. Debug and extend each independently. |
| One-time historical indexing | Index historical blocks once; the data remains available in the Kafka data warehouse. |
| Event-triggered actions | Set triggers on specific blockchain events to drive notifications or UI updates. |
| Scalable | Handles increased data volumes as your dApp grows. |
Use cases
- Wallet services: transaction history, balance history, and real-time updates.
- dApp backends: real-time access to contract events and token transfers.
- Analytics: monitoring smart contract interactions and token transfer patterns.
- Cross-chain services: indexing data from multiple EVM chains for cross-chain features.
- Oracle support: efficient access to specific onchain data points.
- NFT marketplaces: tracking ownership changes, price history, and token attributes.
How it works
The framework uses a producer-transformer-consumer pipeline:- Producers connect to the blockchain RPC and push raw block and event data into a Kafka topic.
- Transformers consume from Kafka, apply your dApp-specific logic, and output structured data.
- Consumers read the structured data and write it to a queryable database.