Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

failed mapping AST context canceled #14384

Open
RobbanHoglund opened this issue Oct 4, 2024 · 1 comment
Open

failed mapping AST context canceled #14384

RobbanHoglund opened this issue Oct 4, 2024 · 1 comment

Comments

@RobbanHoglund
Copy link

Describe the bug
After upgrade to Loki 3.2.0 it is repeatedly logging this for some queries:
ts=2024-10-04T01:09:28.649297614Z caller=spanlogger.go:111 middleware=QueryShard.astMapperware org_id=fake traceID=35d59f6204919421 user=fake caller=log.go:168 level=warn msg="failed mapping AST" err="context canceled" query="{application="myapp",level=~"INFO|WARN|ERROR"} "

To Reproduce
Steps to reproduce the behavior:

  1. Started Loki 3.2.0
  2. From Grafana explore do {application="myapp",level=~"INFO|WARN|ERROR|TRACE"}

Expected behavior
If this is an actual problem in Loki 3.2.0 it would be good to get more information about what is the root cause.

Environment:

  • Infrastructure: bare-metal
  • Deployment tool: bash scripts

Screenshots, Promtail config, or terminal output
If applicable, add any output to help explain your problem.
Image
Image

@jammiemil
Copy link

Also seeing this in a cluster running 3.2.0, however its only happening on one of our Read pods (Running SSD via Helm), the same pod is reporting High query latency (upto 50s), and reporting a fair amount of context cancelled in the logs, All the other read pods are behaving just fine and returning query in sub-seconds

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants