Blockchain process latency is too High, result in lagging block height

[toolchainx]

I am using reth node for Ethereum mainnet node

1. I am using google cloud platform persistent ssd disk.
2. the usage of CPU and the memory is not High.
3. I use prysm as beacon chain client.

Occasionally, I came across the error from prysm said following error, but I am not sure if this is the root cause.

time="2024-11-04 16:17:49" level=warning msg="Block processing failure" error="could not notify the engine of the new payload: timeout from http.Client: received an undefined execution engine error" firstSlot=10326035 firstUnprocessed=10326039 lastSlot=10326041 prefix=initial-sync root=0x8850ded968f409ce24f5859b843b34b1cafb96dfca77af3ce584071a68a5f95f
time="2024-11-04 15:25:20" level=error msg="Could not handle p2p pubsub" error="could not notify the engine of the new payload: timeout from http.Client: received an undefined execution engine error" prefix=sync topic="/eth2/6a95a1a9/beacon_block/ssz_snappy"

reth docker image version

ghcr.io/paradigmxyz/reth:v1.1.0

Hope someone can shed some light on why the Blockchain process latency is so high occasionnally

[jenpaff]

Thanks a lot for your report, apologies you've ran into issues. Would you be able to share the exact GCP spec?

[toolchainx]

Sure, we run reth node as Kubernetes pod on GKE, the Kubernetes node GCP spec:
SSD persistent disk
Machine type: n2-highmem-16
CPU platform: Intel Cascade Lake