Skip to content

OpenTelemetry-Go Contrib vulnerable to denial of service in otelhttp due to unbound cardinality metrics

High severity GitHub Reviewed Published Oct 12, 2023 in open-telemetry/opentelemetry-go-contrib • Updated Feb 19, 2024

Package

gomod go.opentelemetry.io/contrib/instrumentation/github.com/emicklei/go-restful/otelrestful (Go)

Affected versions

< 0.44.0

Patched versions

0.44.0
gomod go.opentelemetry.io/contrib/instrumentation/github.com/gin-gonic/gin/otelgin (Go)
< 0.44.0
0.44.0
gomod go.opentelemetry.io/contrib/instrumentation/github.com/gorilla/mux/otelmux (Go)
< 0.44.0
0.44.0
gomod go.opentelemetry.io/contrib/instrumentation/github.com/labstack/echo/otelecho (Go)
< 0.44.0
0.44.0
gomod go.opentelemetry.io/contrib/instrumentation/gopkg.in/macaron.v1/otelmacaron (Go)
< 0.44.0
0.44.0
gomod go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace (Go)
< 0.44.0
0.44.0
gomod go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp (Go)
< 0.44.0
0.44.0

Description

Summary

This handler wrapper https://github.com/open-telemetry/opentelemetry-go-contrib/blob/5f7e6ad5a49b45df45f61a1deb29d7f1158032df/instrumentation/net/http/otelhttp/handler.go#L63-L65
out of the box adds labels

  • http.user_agent
  • http.method

that have unbound cardinality. It leads to the server's potential memory exhaustion when many malicious requests are sent to it.

Details

HTTP header User-Agent or HTTP method for requests can be easily set by an attacker to be random and long. The library internally uses httpconv.ServerRequest that records every value for HTTP method and User-Agent.

PoC

Send many requests with long randomly generated HTTP methods or/and User agents (e.g. a million) and observe how memory consumption increases during it.

Impact

In order to be affected, the program has to configure a metrics pipeline, use otelhttp.NewHandler wrapper, and does not filter any unknown HTTP methods or User agents on the level of CDN, LB, previous middleware, etc.

Others

It is similar to already reported vulnerabilities

Workaround for affected versions

As a workaround to stop being affected otelhttp.WithFilter() can be used, but it requires manual careful configuration to not log certain requests entirely.

For convenience and safe usage of this library, it should by default mark with the label unknown non-standard HTTP methods and User agents to show that such requests were made but do not increase cardinality. In case someone wants to stay with the current behavior, library API should allow to enable it.

The other possibility is to disable HTTP metrics instrumentation by passing otelhttp.WithMeterProvider option with noop.NewMeterProvider.

Solution provided by upgrading

In PR open-telemetry/opentelemetry-go-contrib#4277, released with package version 0.44.0, the values collected for attribute http.request.method were changed to be restricted to a set of well-known values and other high cardinality attributes were removed.

References

References

Published by the National Vulnerability Database Oct 12, 2023
Published to the GitHub Advisory Database Oct 16, 2023
Reviewed Oct 16, 2023
Last updated Feb 19, 2024

Severity

High

CVSS overall score

This score calculates overall vulnerability severity from 0 to 10 and is based on the Common Vulnerability Scoring System (CVSS).
/ 10

CVSS v3 base metrics

Attack vector
Network
Attack complexity
Low
Privileges required
None
User interaction
None
Scope
Unchanged
Confidentiality
None
Integrity
None
Availability
High

CVSS v3 base metrics

Attack vector: More severe the more the remote (logically and physically) an attacker can be in order to exploit the vulnerability.
Attack complexity: More severe for the least complex attacks.
Privileges required: More severe if no privileges are required.
User interaction: More severe when no user interaction is required.
Scope: More severe when a scope change occurs, e.g. one vulnerable component impacts resources in components beyond its security scope.
Confidentiality: More severe when loss of data confidentiality is highest, measuring the level of data access available to an unauthorized user.
Integrity: More severe when loss of data integrity is the highest, measuring the consequence of data modification possible by an unauthorized user.
Availability: More severe when the loss of impacted component availability is highest.
CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H

EPSS score

0.213%
(60th percentile)

Weaknesses

CVE ID

CVE-2023-45142

GHSA ID

GHSA-rcjv-mgp8-qvmr

Credits

Loading Checking history
See something to contribute? Suggest improvements for this vulnerability.