API: various issues with R package

[rickhelmus]

Hi guys,

As it seems that with version 6 the CLI only exports a limited amount of data, I will have to start from scratch and a build a new interface in patRoon that uses the API. All seems quite well when using Swagger, and I'm happy to see things are quite well organized and accessible now. I noticed there is an OpenAPI export of the R package, but I seem to run in quite a few issues. Is this still in active development and is it better to wait a bit?

Anyway, below are some issues I encountered. I started of with the test code.

Project creation:

> sirius_api$projects_api$CreateProjectSpace(project_id, project_dir)
Error: lexical error: inside a string, '\' occurs before a character which it may not.
           ,"location":             "C:\sir_test\test2.sirius"        
                     (right here) ------^

I think backslashes are not properly escaped when retrieved from the server (input is with forward slashes). The project is created fine though.

Running a simple job:

data <- normalizePath("../../demo-data/mgf/laudanosine.mgf")
sirius_api$projects_api$ImportPreprocessedData(project_id, input_files=data)

> job_submission <- sirius_api$jobs_api$GetDefaultJobConfig(include_config_map = F)
Warning messages:
1: In initialize(...) :
  Initializing SpectralMatchingType with DUMMY_ENUM. Use one of the valid values: INTENSITY, GAUSSIAN, MODIFIED_COSINE. If you did not manually initialize SpectralMatchingType, this may already be overwritten by an enum loaded from a JSON config.
2: In initialize(...) :
  Initializing InstrumentProfile with DUMMY_ENUM. Use one of the valid values: QTOF, ORBITRAP. If you did not manually initialize InstrumentProfile, this may already be overwritten by an enum loaded from a JSON config.
3: In initialize(...) :
  Initializing ConfidenceMode with DUMMY_ENUM. Use one of the valid values: OFF, EXACT, APPROXIMATE. If you did not manually initialize ConfidenceMode, this may already be overwritten by an enum loaded from a JSON config.
> job_submission$spectraSearchParams$enabled <- FALSE
> job_submission$formulaIdParams$enabled <- TRUE
> job_submission$structureDbSearchParams$enabled <- TRUE
> job_submission$canopusParams$enabled <- FALSE # fails if disabled...
> job_submission$msNovelistParams$enabled <- FALSE
> job <- sirius_api$jobs_api$StartJob(project_id, job_submission)
Error in if (local_var_response$status_code >= 200 && local_var_response$status_code <=  : 
  missing value where TRUE/FALSE needed

The first warnings appear harmless. All works fine if I keep canopus enabled (for which I have no interest atm).

Trying to get some results:

> sirius_api$features_api$GetStructureCandidates(project_id, aligned_feature_id) # errors
[[1]]
Error in sprintf("\"mcesDistToTopHit\":\n            %f\n                    ",  : 
  invalid format '%f'; use format %s for character objects
> sirius_api$features_api$GetFingerprintPrediction(project_dir, aligned_feature_id, formula_id) # fails
Error in if (local_var_response$status_code >= 200 && local_var_response$status_code <=  : 
  missing value where TRUE/FALSE needed
> sirius_api$features_api$GetStructureCandidatesByFormula(project_id, aligned_feature_id, formula_id)
[[1]]
Error in sprintf("\"mcesDistToTopHit\":\n            %f\n                    ",  : 
  invalid format '%f'; use format %s for character objects

All of these works fine with the swagger interface.

Thanks,
Rick

[mfleisch]

Hey Rick,
thanks for your feedback.

In SIRIUS 6 we switched to an embedded database for storage and a separate public API to make tools and data accessible from the outside. The old file/folder based storage solution is entirely gone. The summary tsv/csv/xlsx files can still be exported though. This clean separation allows us to maintain a backward compatible public API without blocking further development. Switching to the new API with patRoon should be stable for a long time then.

The API specification is stable and will not not change anymore but will be extended over time. There are a few parts that are flagged as experimental but they are probably not even part of the SDK anyways (mostly endpoints to access masstraces and feature quality).

During November we have a project going on that will heavily utilize the R SDK and likely lead to bug fixes and increased test coverage. So it's currently a good time for feedback and bug reports :-)

Since SIRIUS 6, canopus prediction are a mandatory input to perform structure database search the classed became a feature of the improved confidence score estimation. So not enabling canopus will cause structure db search to fail. It is nevertheless a bit weird that your are getting these type errors when requesting results. We will look into that as well as into the issue with badly escaped file paths.

Best
Markus

[rickhelmus]

Hi Markus,

Thanks for your detailed reply and clarifications! I am quite happy to hear that stability is a major goal here, as this was sometimes a bit of a maintenance struggle for me in the past ;-) The summary files are a bit too limited for me now, as I also need more detailed info like MS2 annotations and fingerprints. But in any case, porting everything to the API makes most sense.

Glad to hear the R SDK will receive more testing to iron out further issues. Once things are stabilized a bit further I will continue, and potentially add some noise again ;-)

Thanks,
Rick