Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

♻️ OpenTelemetry Enhancements: Filter out healthchecks, add catalog internals instrumentation #6752

Merged
merged 160 commits into from
Nov 20, 2024

Conversation

mrnicegyu11
Copy link
Member

What do these changes do?

  • Modifies the opentelemetry-collector-config to drop healthchecks and redis PING statements that clutter the view in jaeger. conceptually it is up for discussion if we should filter them or not, up for discussion, let me know if you would want to keep healthchecks around.
  • Adds (mostly as example code to copy-paste) some custom spans in the list_services call of the catalog.

How to test

By modifying the .env-devel file locally as such:

CATALOG_TRACING=null
TRACING_OPENTELEMETRY_COLLECTOR_BATCH_SIZE=1
TRACING_OPENTELEMETRY_COLLECTOR_PORT=
TRACING_OPENTELEMETRY_COLLECTOR_ENDPOINT=

opentelemetry can be disabled and one can verify that simcore still works without issues. Calls to the _tracer are interpreted as a no-op automatically.

Dev-ops checklist

Dustin Kaiser and others added 30 commits September 18, 2024 16:11
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Pedro Crespo-Valero <32402063+pcrespov@users.noreply.github.com>
Co-authored-by: Andrei Neagu <neagu@itis.swiss>
…the node is a frontend node (ITISFoundation#6434)

Co-authored-by: Dustin Kaiser <8209087+mrnicegyu11@users.noreply.github.com>
Co-authored-by: Andrei Neagu <neagu@itis.swiss>
Co-authored-by: Andrei Neagu <neagu@itis.swiss>
Co-authored-by: Andrei Neagu <neagu@itis.swiss>
@mrnicegyu11 mrnicegyu11 added t:enhancement Improvement or request on an existing feature a:catalog catalog service labels Nov 18, 2024
@mrnicegyu11 mrnicegyu11 added this to the Event Horizon milestone Nov 18, 2024
@mrnicegyu11 mrnicegyu11 self-assigned this Nov 18, 2024
@GitHK GitHK self-requested a review November 18, 2024 14:56
Copy link

codecov bot commented Nov 18, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 66.05%. Comparing base (5cf0635) to head (0ae4170).
Report is 1 commits behind head on master.

❗ There is a different number of reports uploaded between BASE (5cf0635) and HEAD (0ae4170). Click for more details.

HEAD has 30 uploads less than BASE
Flag BASE (5cf0635) HEAD (0ae4170)
unittests 31 1
Additional details and impacted files
@@             Coverage Diff             @@
##           master    #6752       +/-   ##
===========================================
- Coverage   88.50%   66.05%   -22.45%     
===========================================
  Files        1549      662      -887     
  Lines       62410    32610    -29800     
  Branches     2125      265     -1860     
===========================================
- Hits        55238    21542    -33696     
- Misses       6844    11007     +4163     
+ Partials      328       61      -267     
Flag Coverage Δ
integrationtests 64.87% <ø> (+0.01%) ⬆️
unittests 89.50% <100.00%> (+3.06%) ⬆️
Components Coverage Δ
api ∅ <ø> (∅)
pkg_aws_library ∅ <ø> (∅)
pkg_dask_task_models_library ∅ <ø> (∅)
pkg_models_library ∅ <ø> (∅)
pkg_notifications_library ∅ <ø> (∅)
pkg_postgres_database ∅ <ø> (∅)
pkg_service_integration ∅ <ø> (∅)
pkg_service_library ∅ <ø> (∅)
pkg_settings_library ∅ <ø> (∅)
pkg_simcore_sdk 77.44% <ø> (-7.84%) ⬇️
agent ∅ <ø> (∅)
api_server ∅ <ø> (∅)
autoscaling ∅ <ø> (∅)
catalog 89.50% <100.00%> (+0.07%) ⬆️
clusters_keeper ∅ <ø> (∅)
dask_sidecar ∅ <ø> (∅)
datcore_adapter ∅ <ø> (∅)
director ∅ <ø> (∅)
director_v2 76.45% <ø> (-14.50%) ⬇️
dynamic_scheduler ∅ <ø> (∅)
dynamic_sidecar 59.82% <ø> (-29.96%) ⬇️
efs_guardian ∅ <ø> (∅)
invitations ∅ <ø> (∅)
osparc_gateway_server 79.68% <ø> (-5.47%) ⬇️
payments ∅ <ø> (∅)
resource_usage_tracker ∅ <ø> (∅)
storage ∅ <ø> (∅)
webclient ∅ <ø> (∅)
webserver 59.73% <ø> (-29.01%) ⬇️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5cf0635...0ae4170. Read the comment docs.

---- 🚨 Try these New Features:

Copy link
Contributor

@matusdrobuliak66 matusdrobuliak66 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!!

@mrnicegyu11 mrnicegyu11 changed the title ♻️ OpenTemetry Enhancements: Filter out healthchecks, add catalog internals instrumentation ♻️ OpenTelemetry Enhancements: Filter out healthchecks, add catalog internals instrumentation Nov 19, 2024
Copy link
Contributor

@bisgaard-itis bisgaard-itis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for the effort. Just a few suggestions from my side.

@mrnicegyu11
Copy link
Member Author

As agreed with @bisgaard-itis we don't add manual telemetry instrumentation for now,he will try to push a magic decorator in the servicelib that we can use to do this :--) Much cleaner, less noisy!

Copy link

sonarcloud bot commented Nov 20, 2024

@mrnicegyu11 mrnicegyu11 merged commit 793318d into ITISFoundation:master Nov 20, 2024
51 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a:catalog catalog service t:enhancement Improvement or request on an existing feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants