⏯️Run the Indexer

How to run Guru Network Indexer

Quickstart

Prerequisites

Ensure you have the following installed:

  • Docker

  • Docker Compose

Step-by-Step Instructions

  1. Prepare Environment Variables

    • Copy the env.sample file and rename it to .env.

    • Edit the .env file to set the following variables according to your needs:

      • CHAIN_ID: The ID of the blockchain network you are indexing.

      • DATABASE: The name of the database that will be created in ClickHouse.

      • PROVIDER_URL: The URL of your RPC provider, such as a public node, Infura, or a local node.

    • See Environment Variables

  2. Start Indexing

    • Open your terminal and navigate to the directory containing your docker-compose.yml file.

    • Run the command:

      docker-compose up

    • This command will start indexing the Ethereum blockchain into ClickHouse.

  3. Verify Database Content

    • Once the indexing process begins, you can inspect the database to see the tables created for each entity type and a few service-related tables.

Running certain command

Install Ethereum ETL:

pip3 install -r requirements.txt
CLICKHOUSE_URL=clickhouse+http://default:@localhost:8123/ethereum alembic upgrade head

Export blocks and transactions (Schema, Reference):

ethereumetl export_blocks_and_transactions --start-block 0 --end-block 500000 \
--blocks-output blocks.csv --transactions-output transactions.csv \
--provider-uri https://mainnet.infura.io/v3/${INFURA_API_KEY}

Export ERC20 and ERC721 transfers (Schema, Reference):

ethereumetl export_token_transfers --start-block 0 --end-block 500000 \
--provider-uri file://$HOME/Library/Ethereum/geth.ipc --output token_transfers.csv

Export traces (Schema, Reference):

ethereumetl export_traces --start-block 0 --end-block 500000 \
--provider-uri file://$HOME/Library/Ethereum/parity.ipc --output traces.csv

Stream blocks, transactions, logs, token_transfers continually to console (Reference):

ethereumetl stream --start-block 500000 -e block,transaction,log,token_transfer --log-file log.txt \
--provider-uri https://mainnet.infura.io/v3/7aef3f0cd1f64408b163814b22cc643c

Find other commands here.

Supported export destinations here.

Linters/formatters

Install

pip install black ruff mypy

Run

all-at-once run and fix:

ruff check --fix . && black . && mypy .

Or one-by-one:

  • Check and auto-fix with: ruff --fix .

  • Check typing: mypy .

  • Auto-format all: black .

Useful Links

PreviousIndexerNextEnvironment Variables

Last updated