Sequencer endpoints timing out during DevNet validator onboarding (GKE)

App Development
1

Hi,

I am onboarding a Canton Network validator on GKE Autopilot (australia-southeast1) running Splice v0.5.18. Our validator successfully connects to all 14 DevNet scan nodes but cannot reach any sequencer endpoints — all connections time out.

What is working:

  • All 14 DevNet scan SVs return version 0.5.18

  • Participant node connects and authenticates successfully

  • Validator passes all init steps up to domain registration

Where it fails:

The validator gets stuck at RegisterSynchronizer with this error:

SYNC_SERVICE_BAD_CONNECTIVITY: The provided sequencer connections are inconsistent:
connection timed out after 30000ms: sequencer-0.sv-1.dev.global.canton.network.cumberland.io:443
connection timed out after 30000ms: sequencer-0.sv-1.dev.global.canton.network.tradeweb.com:443
PKIX path building failed: sequencer-0.sv-2.dev.global.canton.network.cumberland.io:443

We tested sequencer connectivity directly from inside the cluster:

bash

for host in \
  sequencer-0.sv-1.dev.global.canton.network.cumberland.io \
  sequencer-0.sv-2.dev.global.canton.network.cumberland.io \
  sequencer-0.sv-1.dev.global.canton.network.tradeweb.com \
  sequencer-0.sv-1.dev.global.canton.network.digitalasset.com \
  sequencer-0.sv.dev.global.canton.network.digitalasset.com \
  sequencer-0.sv-1.dev.global.canton.network.sync.global; do
  printf "%s: " "$host"
  curl -fsS --ipv4 --connect-timeout 10 --max-time 15 "https://$host:443" 2>/dev/null | head -1 \
    || echo "unreachable/timeout"
done

All six sequencer endpoints return unreachable/timeout with no response at all — not even a connection refused.

Our egress IP 35.244.95.187 has been whitelisted by the Canton Foundation for scan API access. Is sequencer access controlled by a separate whitelist? If so, what is the process to get 35.244.95.187 whitelisted for sequencer access on DevNet?

2

You’re querying the wrong migration id. sequencer-0 is for migration id 0. DevNet is on migration id 1. You can find this at SV Network Status - Canton Foundation which is also what the docs suggest here Kubernetes-Based Deployment of a Validator node — Splice documentation

3

Thank you @cocreature — that was exactly the issue. Setting domain-migration-id = 1 in the validator config caused the scan nodes to return sequencer-1.* endpoints instead of sequencer-0.*, and the validator successfully connected to all 14 sequencers and registered with the Global Synchronizer.