Merge espresso code #324
Closed
Merge espresso code #324
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Increase the timeout limit for the e2e validation check
Add SwitchSequencer
This flag in gotestsum causes some bugs which multiple the test time. gotestyourself/gotestsum#417 gotestyourself/gotestsum#423
This reverts commit 6bef1c3.
- Add scripts to debug more easily locally. - Skip tests by substring match instead of exact match. - Don't run coverage on CI, we never look at it. - Move all Espresso tests to Espresso CI job.
This enables the --hide-summary argument
Sync nitro v3.2.1
Bumps [nitro-testnode](https://github.com/EspressoSystems/nitro-testnode) from `07a25f1` to `77d1447`. - [Commits](EspressoSystems/nitro-testnode@07a25f1...77d1447) --- updated-dependencies: - dependency-name: nitro-testnode dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Example call to espresso-crypto function stub in batch poster.
This commit includes a log line and example call to the espresso-crypto
function stubs in batch_poster.go
* Result of `make fmt`
Run `make fmt` to clean up formatting not apparent in editor
* Make current lib.rs c abi compatible
* result of make fmt
* Add C ABI ffi between rust crypto lib and espressocrypto go module.
This commit adds FFI support between exsisting Rust crypto helper code
and the exsisting espressocrypto go module in native.go These functions
will now serve as utility functions to use in the integration that will
handle ffi to the Rust code.
This commit also moves the espresso-crypto-helper code to the
espressocrypto directory as we no longer need it for the arbitrator.
* Build native.go successfully
* Update the espresso-crypto and add test for merkle proof verification
* update flake.nix to new go version and fix errors
* Current state of work
The current state of my work revolves around editing the Makefile to
include the new espressocrypto ffi functions in the build process.
Additionally, this commit restores some espressocrypto code to the
arbitrator in an attempt to get a docker image built via the test node.
I felt as though this would be easier for this PR as opposed to ripping
out all of the code from the arbitrator.
* Update the flake.lock
* Add the namespace proof test
* Current state of build system changes
This commit represents the current state of the build system after
testing today.
Currently things compile locally, but not when building an image from
the docker file. There are errors related to duplicate function
definitions from what seem to be the FFI lib that stylus exposes.
* Build nitro docker image sucessfully
- Modify makefile to have a file target. Since the build artifact of our
static library is a file we should use this as the make target
(instead of only having PHONY targets).
- Use rust 1.81 to build all shared rust libraries in docker.
Our crate requires a minimum version of 1.81. However the other
libraries like libstylus.a are currently compiled with 1.75 and this
seems to cause issues with duplicate symbol definitions during linking.
For example:
8.718 go build -ldflags="-extldflags '-ldl' -X github.com/offchainlabs/nitro/cmd/util/confighelpers.version= -X github.com/offchainlabs/nitro/cmd/util/confighelpers.datetime= -X github.com/offchainlabs/nitro/cmd/util/confighelpers.modified=" -o target/bin/nitro "/workspace/cmd/nitro"
42.74 # github.com/offchainlabs/nitro/cmd/nitro
42.74 /usr/local/go/pkg/tool/linux_amd64/link: running clang failed: exit status 1
42.74 /usr/bin/ld: /workspace/espressocrypto/lib/espresso-crypto-helper/target/release/libespresso_crypto_helper.a(std-1c4b19562077c20d.std.1f2242ed6435445e-cgu.0.rcgu.o): in function `__rdl_alloc':
42.74 /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/std/src/alloc.rs:397: multiple definition of `__rdl_alloc'; /workspace/execution/gethexec/../../target/lib/libstylus.a(std-90f6ddbf82de36ec.std.409886f6357001f0-cgu.0.rcgu.o):/rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/alloc.rs:389: first defined here
Bump the rust version of the prover-header-builder to 1.81 and compile
the espresso crypto library in the same container.
* Fix LDFLAGS, remove unnecessary CFLAGS
* Fix some merge errors
- We need rust 1.81 to compile the espresso crypto lib
- Re-add makefile rules
* Fix merge errors
* Remove FFI call in batch poster to expidite merge.
* Update bf submodule in arbitrator to match upstream version (This was set to a different commit during my erroneous merge)
* Remove extra change in espressocrypto_wasm.go
* Fix error with temporary espressocryptowasm directory preventing CI from running
* fix error of missing function definitions in espressocryptowasm directory to allow CI to run
* Re-add nosec comment about gosec error that will be removed soon
* result of make fmt
---------
Co-authored-by: Zach Showalter <zacshowa@gmail.com>
Co-authored-by: ImJeremyHe <yiliang.he@qq.com>
Still using the legacy dev node
* Use path context instead of git context With the default git context the repository is re-cloned at the start of the docker build and the .dockerignore file is ignored. > Be careful because any file mutation in the steps that precede the > build step will be ignored, including processing of the .dockerignore > file since the context is based on the Git reference. https://github.com/docker/build-push-action?tab=readme-ov-file#git-context This leads to some `.git` worktree files showing up inside the dockerbuild with triggers a bug in yarn. See yarnpkg/yarn#7537 (comment) for more details about the yarn bug.
* Use the latest espresso dev node - Fix an issue that may cause the validation reorg - Waiting for L1 to be ready can avoid some potential bugs in dev node
* Add espresso proof validation to batch poster Adds calls to the espresso validation code to the batch poster to validate the namespace proof and merkle proof in a TEE * result of make fmt * Fix lint errors * Add better logging * Fix batch poster after merge of upstream/integration
* Remove espresso validation in the replay binary * Remove espresso-crypto related dependencies in arbitrator * remove espressocryptowasm * Update Makefile and Dockerfile to remove references to unused espresso crypto wasm lib * remove batch_poster.go.orig file * Fixes from make fmt * Remove espresso_ops_test * Remove additional code related to espresso validation in the staker * Remove changes on stateless block validator * Undo the stopwaiter Previous modifications on this was because we had to stop and start the seuquencers to function the escape hatch. Right now we don't need that anymore. These changes are from the upstream * Rename test flag --------- Co-authored-by: ImJeremyHe <yiliang.he@qq.com>
* Add attestation quote in batch poster * write file * update batch poster * use read file * update to userDataHash * reduce file permissions * nit
* Submit multiple messages in one transaction * Remove the espresso_transaction_payload_signature_test.go * Update go client * Remove signature and data signer * Remove arbitrator/espresso-crypto-helper * Add size limit to hotshot transaction --------- Co-authored-by: Sneh Koul <snehkoul1999@gmail.com>
We no longer need the customizations in nitro-contracts (except potentially some deployment tools) so this commit resets it to the before the celestia contracts were added celestiaorg/nitro-contracts@4f8c65c and makes the nitro code work with that version by removing the hotshot / light client address passed to the OSP.
* Remove the legacy code and fix the test
Sneh1999
force-pushed
the
merge-espresso-code
branch
from
5d23352
to
07b1e84
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #309
This PR:
Merging our code with celestia's v3.2.1-rc.1 branch
This PR does not
There is a lot of code we will still have to delete but we should probably address it in future commits