Skip to content

💡 For CDK fork ID9 chains NOT attached to the AggLayer (Isolated), they can ignore section 4.

💡 For CDK fork ID9 chains attached to the AggLayer, follow steps in sections 1 to 5. This is a coordinated effort between Polygon and the Implementation Provider.

1. Summary of the Procedure

To initiate a CDK chain upgrade, the Implementation Provider can request support from Polygon by submitting the “Request Help for an Issue with an Existing CDK Chain” through the service desk.

CDK Service Desk Example Request
CDK Service Desk Example Request

Polygon will then collaborate with the Implementation Provider to schedule the UTC timing and dates for the upgrade.

This planning enables the Implementation Provider to schedule testnet and mainnet maintenance windows for the respective networks, ensuring proper communication and coordination with their communities.

See Example Maintenance Communication to Network Partners Implementation Providers can prepare for the customer network chains.

The high-level steps in the collaborative process are:

  1. Implementation Provider halting the sequencer,
  2. Implementation Provider can send & execute the upgrade transaction,
  3. Implementation Provider upgrades components to Fork 12 stack,
  4. Implementation Provider restarts the components.
  5. Polygon updates the Permissionless node attached to the Agglayer for that chain.

2. CDK Components Versions

Please read carefully to fully understand the new architecture before starting the process: https://docs.polygon.technology/cdk/getting-started/cdk-erigon/

The table below lists the CDK Fork ID 9 components and the new CDK FEP Fork ID 12 components.

💡 Please note: Latest CDK FEP Fork ID 12 components are listed in the Polygon Knowledge Layer

CDK Components Fork ID 9 CDK Components Fork ID 12
CDK Validium node
Sequence sender
Aggregator		 0.6.7+cdk.1 CDK Erigon RPC & CDK node cdk-erigon:v2.1.x
CDK node
Sequence sender
Aggregator		 cdk:v0.3.x
Tx pool manager zkevm-pool-manager Tx pool manager zkevm-pool-manager use latest tag
Prover v6.0.0 Prover zkevm-prover v8.0.0-RC14
CDK data availability v0.0.7 CDK data availability cdk-data-availability use latest tag
zkEVM rollup node v6.0.0 zkEVM rollup node N/A
Contracts v6.0.0 Contracts zkevm-contracts
Bridge service v0.4.2-cdk.1 Bridge service zkevm-bridge-service
Bridge UI Polygon Portal Bridge UI Polygon Portal

3. Implementation Provider Preparation Steps

The Implementation Provider must prepare in advance for the upgrade to ensure a smooth transition from fork ID 9 to fork ID 12. Failure to complete these steps ahead of time could result in delays or even cancellation of the scheduled upgrade.

  1. The Implementation Provider downloads CDK Fork 12 components in advance so they are ready to deploy.
  2. Map to the latest prover files which can be found here: https://storage.googleapis.com/zkevm/zkproverc/v8.0.0-rc.9-fork.12.tgz
  3. Scale up the number of provers in advance. It is recommended that you at least double the number of provers up and running for the scheduled upgrade maintenance window.
    • Ensure all (majority) of the network batches are verified before starting the upgrade process, otherwise there will be additional downtime as we wait for the network to be ready.
  4. Setting up your CDK Erigon node.
    • Please see the cdk-erigon readme for information on prereqs, configuration files and running cdk-erigon.
  5. Required CDK-Node pre syncing.
  6. Run CDK-Node aggregator component in sync only mode in advance to populate its database.
  7. Run CDK-Node l1infotreesync component in advance to populate the sequence sender database.
  8. Required Erigon pre-syncing:
    • Generate data stream files from the current cdk-validium-node database.
    • Use the Erigon tool (from cdk-erigon repo) to serve the DS:
      • go run ./zk/debug_tools/datastream-host --file /path/to/directory
      • Start CDK-Erigon, which reads from this datastream (provide zkevm.l2-datastreamer-url: 127.0.0.1:6900 in the config).
    • Wait for it to sync to the tip.
    • CDK-Erigon can be stopped. The generated files will be used later during the upgrade process.

The whole process should look more or less like this: 

# PREREQUISITES: Install GO 1.23
WORK_DIR=/tmp

cd $WORK_DIR
git clone https://github.com/0xPolygonHermez/zkevm-node.git
cd $WORK_DIR/zkevm-node/tools/datastreamer
vim config/tool.config.toml
# Edit [StateDB] section with your StateDB credentials
make generate-file

cd $WORK_DIR
git clone https://github.com/0xPolygonHermez/cdk-erigon.git
cd $WORK_DIR/cdk-erigon
go run ./zk/debug_tools/datastream-host \
    --file $WORK_DIR/zkevm-node/tools/datastreamer/datastream

# Bring up Erigon pointing to that DS (localhost:6900 if running locally)
# and let it fully sync to the end of the DS.

4. Polygon Preparation Steps

  1. Polygon will collaborate with the Implementation Provider to agree on the scheduled and dates for the upgrade which will incorporate any required time locks.
  2. For chains attached to the Polygon Agglayer, Polygon will handle steps to upgrade the permissionless node.
  3. See example communication that Implementation Providers can use to prepare their customer network partners and communities.

5. Operational Steps

Please Note: To avoid creating reorgs or other unwanted situations, it’s important that all L2 transactions are verified before performing a fork upgrade. This means all batches should be closed, sequenced, and verified on L1.

Please Note: IP Partner Rollup admin can send & execute the fork ID upgrade transaction.

  1. Stop the sequencer.

  2. Reconfigure the node to enforce sequencer stops at a specific batch_num:

    • Get the current batch number from StateDB:
    -- Note resulting value as X:
SELECT batch_num, wip FROM state.batch WHERE wip IS true;
    • Edit node configuration:
    # SEQUENCER CONFIG
# Where X is the batch number obtained from the SQL query:
Sequencer.Finalizer.HaltOnBatchNumber = X+1
# Optional: Reduce batch time to avoid excessive downtime
Sequencer.Finalizer.BatchMaxDeltaTimestamp = "120s" # 1800s

    # SEQUENCE SENDER CONFIG
# Optional: Reduce sending time to avoid excessive downtime
SequenceSender.WaitPeriodSendSequence = "10s" # 60s
SequenceSender.LastBatchVirtualizationTimeMaxWaitPeriod = "30s" # 600s

    # AGGREGATOR CONFIG
# Recommended: Reduce verify interval to avoid excessive downtime
Aggregator.VerifyProofInterval = "5m"
    • Restart the sequencer, sequence sender, and aggregator to apply these configs.
    • Check that the sequencer halts when reaching batch X+1.
    • Wait until all pending batches are virtualized and verified (X):
    -- Both queries should return X
SELECT max(batch_num) FROM state.virtual_batch;
SELECT max(batch_num) FROM state.verified_batch;

  3. Stop all services (node, prover/executor, bridge).

  4. Implementation Provider Rollup admin: Upgrade Smart Contract.

    • Upgrade rollup to fork 12.
    • Wait for the Tx to be finalized.

Steps to Deploy CDK FEP Fork 12 Components

  1. It is recommended to back up your DAC node by taking a snapshot of the DAC database.

    • Locate Your PostgreSQL Instance:
      • Identify the PostgreSQL host, port, database name (dac_db), and the username (master_user) configured for your DAC node. These values can typically be found in your environment’s configuration or .env files.
      • Run the pg_dump Command:
      • Use the pg_dump command to create a backup of your DAC database. Replace the placeholders with your specific values:
      • pg_dump -U master_user -d dac_db -h 127.0.0.1 -p 5432 > dac.db.sql
      • Verify the Backup:
      • After running the command, ensure that the dac.db.sql file has been created and contains the database snapshot.

  2. With the network stopped, repeat Erigon sync to get it fully synced to the current state.

    • This instance is ready to act as Sequencer and/or RPC. Clone the whole Erigon config/datadir as many times as instances are needed. Pick one to be the new Sequencer (by setting the environment variable CDK_ERIGON_SEQUENCER=1), and configure all other instances (permissionless RPCs) to point to the Sequencer:
    zkevm.l2-sequencer-rpc-url: "http://sequencer-fqdn-or-ip:8123"
zkevm.l2-datastreamer-url: "sequencer-fqdn-or-ip:6900"

  3. Start the stateless Executor.

  4. Start CDK-Erigon Sequencer
    • On a fork upgrade, once the upgrade Tx is finalized you can start the sequencer. Once started, check logs and ensure new blocks are generated with new forkid.
      • You would expect to see starting block #5796790 as forkid 12. If you see the starting block 5796790 as fork id 9 there is a problem. If the new block is on the old fork id 9, you need to resync sequencer from scratch or get one of the rpc datadirs (that were synced till the halt and are currenctly stopped) and replace it to become the new sequencer.
  5. Verify in the sequencer’s logs that new blocks are being generated with fork ID 12.
  6. Start the Pool Manager (if used/needed).
  7. Start CDK-Erigon RPCs (if used/needed).
  8. Start the Bridge.
  9. Start the CDK aggregator and Sequence Sender components.
  10. Start the stateless Prover.

Polygon Steps for CDK Chains Attached to the Agglayer

Polygon’s DevOps team will be accountable for upgrading the Agglayer permissionless nodes during the upgrade process.

Post-Upgrade Validations

  1. Implementation Provider will test the batch lifecycle.
  2. Implementation Provider will test the bridge.

Example Maintenance Communication to Network Partners

There is a planned maintenance window upgrade of the xxxx network on the following dates. This is to upgrade the xxx network from Fork ID9 to Fork ID12.

Maintenance Event:  xxx Network Upgrade from Fork ID9 to Fork ID12

Date: TBD by Implementation Provider

Time: 00:00 PM EDT / 00:00 PM UTC

Duration: 2 Hours

Things to Note:

  • This upgrade is from Fork ID9 to Fork ID12. The change log can be viewed here.
  • Partners should not update their nodes until after the xxx network upgrade is confirmed as complete.
  • New Node Version: cdk-erigon:v2.1.x

Upgrade Instructions for Community Partners (Testnet/Mainnet)

Update FROM node version 0.6.7+cdk.1 up to cdk-erigon:v2.1.x.

Instructions to Update Nodes

  1. Stop the RPC, Synchronizer, and Executor components.
  2. Update the node (RPC and Synchronizer) to cdk-erigon:v2.1.x.
  3. Update the Prover/Executor to v8.0.0-RC12-fork.12.
  4. Start the components in this order:
    1. Prover/Executor
    2. Synchronizer
    3. RPC

Contract Upgrade procedure for isolated networks

Resources

For fork.12 upgrades, it is need to upgrade PolygonZkEVMGlobalExitRootV2 and the PolygonRollupManager. Please refer to the Upgrade Banana SC section in order to do the SCs upgrade

Verifier

Add rollup type

Upgrade rollup

Upgrade Smart Contracts to Banana

Params

  • example parameters
    • the script deploys the new implementation and any EOA can do that
  • On top of those parameters, it is needed a file from previous deployment
    • this file is generated automatically by hardhat when you deploy a network
    • placed in $root-sc_repo/.oppenzeppelin/${network-name}.json
    • remember to copy this file into thew folder mentioned above and then run the upgradeBanana script

Output script

  • Similar to this one
    • Take the scheduleData, send a tx to timelockAddress with the proper EOA that has proxyAdmin rights
    • wait timelockDelay
    • Take the executeData, send a tx to timelockAddress with the proper EOA that has proxyAdmin rights

Apply upgrade

  • cast send $TIMELOCK_ADDR $SCHEDULE_DATA --private-key $ADMIN_KEY --rpc-url $L1_URL
  • Wait timelock delay
  • cast send $TIMELOCK_ADDR $EXECUTE_DATA --private-key $ADMIN_KEY --rpc-url $L1_URL