Fix concurrency issues

[markspanbroek]

Switches nim-ethers to http subscriptions instead of websocket subscriptions. This changes the timing of subscriptions and unearths some concurrency issues. This PR fixes those issues.

Includes:

• fixes to nim-ethers: Fix http subscription polling nim-ethers#94
• transactions are confirmed with .confirm(1), previously we used .confirm(0), which didn't work
• many fixes to tests that had hidden assumptions about timing

This is split off from the work in #988

[benbierens]

I'm expecting this to make our tests even slower. But it does seem correct.

[veaceslavdoina]

I've performed some tests to check if slowness on Windows can be related to IPv6.

Results

Branch	PRC Endpoint	Run status	make testAll	IPv6
fix-concurrency-issues	127.0.0.1	✅	66m21.431s	🟢
	localhost	❌	149m10.858s	🟢
	localhost	❌	153m0.365s	🟢
	localhost	❌	152m45.421s	🟢
	localhost	✅	57m50.468s	🔵
	localhost	✅	58m8.511s	🔵
	localhost	✅	58m4.198s	🔵
	127.0.0.1	✅	56m48.298s	🔵

🟢 - IPv6 is enabled or primary
🔵 - IPv6 is disabled or secondary

details

Server

Cloud	Region	Instance
Hetzner	Germany, Nuremberg	CPX41 - 8 vCPU, 16 GB RAM, 240 GB SSD

Commands

# Clone
git clone https://github.com/codex-storage/nim-codex.git repos/nim-codex && cd repos/nim-codex

# Branch
git switch -C fix-concurrency-issues origin/fix-concurrency-issues

# Update
time make -j 8 update # 7m55.096s

# Build
time make -j 8 # 8m36.045s

# Tests
time make -j8 testAll # 66m21.431s

# --> localhost
find tests -type f -name "*.nim" -exec sed -i 's/127.0.0.1:8545/localhost:8545/g' {} \;
grep -r 'localhost:8545' tests

# --> 127.0.0.1
find tests -type f -name "*.nim" -exec sed -i 's/localhost:8545/127.0.0.1:8545/g' {} \;
grep -r '127.0.0.1:8545' tests

Disable IPv6

Guidance for configuring IPv6 in Windows for advanced users

:: Prefer IPv4 over IPv6 in prefix policies
netsh interface ipv6 show prefixpolicies
netsh interface ipv6 set prefix ::/96 60 3
netsh interface ipv6 set prefix ::ffff:0:0/96 55 4

:: Permanently
:: Prefer IPv4 over IPv6
:: reg add HKLM\SYSTEM\CurrentControlSet\Services\Tcpip6\Parameters /v DisabledComponents /t REG_DWORD /d 0x20 /f

:: Disable IPv6
:: reg add HKLM\SYSTEM\CurrentControlSet\Services\Tcpip6\Parameters /v DisabledComponents /t REG_DWORD /d 0xFF /f

:: restart
:: shutdown -r -t 0

:: reg query HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip6\Parameters /v DisabledComponents

:: check
ping localhost -n 1

:: Pinging WIN-IP4MNH4JT01 [127.0.0.1] with 32 bytes of data:
:: Reply from 127.0.0.1: bytes=32 time<1ms TTL=128

Enable IPv6

:: Prefer IPv6 over IPv4 in prefix policies
netsh interface ipv6 show prefixpolicies
netsh interface ipv6 set prefix ::/96 1 3 store=persistent
netsh interface ipv6 set prefix ::ffff:0:0/96 35 4 store=persistent

:: delete the key
:: reg delete HKLM\SYSTEM\CurrentControlSet\Services\Tcpip6\Parameters /v DisabledComponents /f

:: restart
:: shutdown -r -t 0

:: reg query HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip6\Parameters /v DisabledComponents

:: check
ping localhost -n 1

:: Pinging WIN-IP4MNH4JT01 [::1] with 32 bytes of data:
:: Reply from ::1: time<1ms

RPC request time

# localhost - 0m0.326s
time curl -m 5 -X POST \
  -s http://localhost:8545 \
  -H 'Content-Type: application/json' \
  -d '{"jsonrpc":"2.0","method":"eth_syncing","params":[],"id":1}'

# 127.0.0.1 - 0m0.116s
time curl -m 5 -X POST \
  -s http://127.0.0.1:8545 \
  -H 'Content-Type: application/json' \
  -d '{"jsonrpc":"2.0","method":"eth_syncing","params":[],"id":1}'

We can fix that for CI, but it will require to update the docs and perform one more step, locally as well.

      - name: Prefer IPv4 over IPv6 in prefix policies
        if: matrix.os == 'windows'
        run: |
          netsh interface ipv6 set prefix ::/96 60 3
          netsh interface ipv6 set prefix ::ffff:0:0/96 55 4