Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add REST endpoint to fetch manifest without fetching data #929

Closed
wants to merge 4 commits into from

Conversation

vpavlin
Copy link
Contributor

@vpavlin vpavlin commented Oct 1, 2024

Closes #920

I attempted to add the manifest/{cid}/network endpoint to fix #920

Seems like nph reformated the whole file - do you use it? If so which version and with what config?:) If not, I'll try to disable it and push PR without this mess. The actual changes are on lines 104 and 220

@AuHau
Copy link
Member

AuHau commented Oct 1, 2024

Yeah, we are unfortunately not yet using nph :-(

@emizzle
Copy link
Contributor

emizzle commented Oct 1, 2024

I'm am very much in favour of using it. We'd need a code base-wide effort to implement it initially, which may be a bit of an effort (worthwhile effort though).

@benbierens
Copy link
Contributor

I'm seeing a bunch of unrelated changes, I think. Formatting and ordering of procs. Makes the more tricky to review the functional changes. Mostly I'd like to ask you: when is it useful to fetch the manifest? I would like to have an API that gives a lot more info about the state of the node and much finer control.

@vpavlin
Copy link
Contributor Author

vpavlin commented Oct 1, 2024

Disabled nph to make the PR look sane:D

@benbierens

when is it useful to fetch the manifest?

For me particularly, I'd like to leverage the altruistic data storing - but I don't want to store anything that people throw at me:) So I'd like to first know the size of the dataset they are sking me to store. Very specifically, I've built a tiny demo for Codex + Waku here https://app.radicle.xyz/nodes/seed.radicle.garden/rad:z47KBuinFp7miXRVPGz6cVHTTnr94 and part of it is a public node with a small Waku node "in front of it" which caches content based on cid passed via Waku message - I'd like to be able to verify the size of the data is < X MB before I actually pull the data to the node.

I assume there will be other use cases for knowing the manifest and data size before actually triggering the data pull in the future, but this is what I have at the moment:)

@cskiraly
Copy link
Contributor

cskiraly commented Oct 2, 2024

I'm am very much in favour of using it. We'd need a code base-wide effort to implement it initially, which may be a bit of an effort (worthwhile effort though).

If we decide to do some reformatting, please, please do it as a separate patch.
If squash merge is used, please also do it as a separate PR, so that it ends up being a separate patch.

@AuHau
Copy link
Member

AuHau commented Oct 22, 2024

@vpavlin so @benbierens added this feature in his API work and now we have released it, see here: https://api.codex.storage/#tag/Data/operation/downloadNetworkManifest

Does that satisfy your need? Can we close this PR?

@vpavlin
Copy link
Contributor Author

vpavlin commented Oct 22, 2024

Yup, I think that's perfect:) I am only disapponted because I was hoping to have commit in Codex repo:-P

@vpavlin vpavlin closed this Oct 22, 2024
@AuHau
Copy link
Member

AuHau commented Oct 22, 2024

Yup, I think that's perfect:) I am only disapponted because I was hoping to have commit in Codex repo:-P

Oh man, I am sorry. I will be watching for your next contribution ;-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

REST API to fetch manifest
5 participants