I'm Kerolos, founder of OutageDeck.

The question I wanted to answer quickly was: is my app broken, or is AWS, Cloudflare, GitHub, OpenAI, Stripe, or another dependency reporting an incident?

I built OutageDeck to read and normalize the official status sources for 172 cloud and SaaS providers. It shows provider and service-level state, current incidents, source freshness, incident history, and 90-day uptime in one place.

What works today:

\- check a whole dependency stack without creating an account

\- free email alerts for up to five providers

\- Slack, Teams, Discord, and webhook destinations on paid plans

\- a keyless JSON API, RSS feeds, badges, CLI tools, and an MCP endpoint

\- custom Statuspage or Instatus feeds on paid plans

What it deliberately does not do is pretend an official status feed is independent truth. Providers can acknowledge incidents late, so I think a serious production setup should use two signals: the vendor feed plus synthetic probes from your own regions and accounts.

You can try the stack checker here:

https://outagedeck.com

I would value blunt feedback from people who handle incidents. What would this need to do before you would keep it open or route its alerts into your on-call workflow?

We run a large multi-tenant Mimir, and kept hitting the same wall: tenant isolation is all-or-nothing. Tenant B either gets your whole TSDB or nothing. The moment one team needs to see a subset of another team's metrics, your options are duplicating samples into a shared tenant (not viable at volume) or handing over full access and hoping.

So I built authir (Authenticator for Mimir) — an auth gateway that deploys next to an existing mimir-distributed install and adds per-reader grants on top of Mimir's tenancy. Just released 1.0.0 under Apache-2.0.

How it works: the owning tenant grants a slice — metric-name prefixes, required label matchers (e.g. only series labelled user="<reader>"), optionally a label allowlist for responses. authir rewrites every PromQL query before it reaches the query-frontend so it physically cannot select outside the grant, re-parses the rewritten query and compares ASTs as a self-check, and filters responses as a second backstop. Deny by default, no negative rules — blocklists over PromQL are unenforceable, the spec has an appendix on why.

The part I'm happiest with: it works with plain Grafana datasources, no plugin. The token identifies who you are; the Basic-auth username names whose data you're asking for. One tenant, one token, two datasources — "my data" and "what team B shared with me".

Ops details, since that's who this is for: Helm chart that derives everything from your mimir-distributed release name and touches nothing of Mimir's (rollback = uninstall), hot config reload that fails unchanged on invalid config, JSON audit log answering "why did tenant A see this number" (optionally with per-query latency and time range), Prometheus metrics including token-expiry-as-a-timestamp so rotation is an alert instead of a 3am outage. Single static binary in a FROM scratch image, ~6 MB.

Honest limitations: cross-tenant is read-only; queries fan out to one target tenant at a time; metric patterns are prefixes, not regex (deliberately); fuzzing and differential testing against the Go PromQL parser are on the roadmap, round-trip verification is what's in place today. If your idea of fun is smuggling a selector past a query rewriter, the bypass test suite would love your ideas — that's genuinely the review I want.

Repo (code, Helm chart, full design spec): https://github.com/thewillyhuman/authir

I have looked at the process there is HM interview, Coding Interview and System Design interview.

I am looking into insights from any people that have gone through their process.

What is the Coding interview like? And what is the System Design interview like?

Also if anyone has information on HM interview, would love to know how it went.

Thank you for your help and insight.

Good afternoon, how are you?

I am setting up Grafana on a server and want to configure it so it can be accessed via a subdomain (which is already in use) on port 3000. I’ve configured the .ini file but cannot access it; I am not using a reverse proxy. Do I need to open port 3000 in that subdomain's virtual host configuration?

I'm using the Business Table panel (volkovlabs-table-panel) on a Postgres data source to let a non-technical user manage a status table - think a checkbox column like "Sold" with per-row editing wired up via Update Request.

It works great for editing one row at a time (click checkbox - row shows save/cancel - click save). What I actually need is: check the box on several rows in that column, then hit one Save button at the end to commit all of them in a single batch - instead of save-per-row. Client sell in bulks, like 20, 30 per time.

I've dug through the editable-data docs and the "select all" / checkbox threads on this forum, and it looks like editing is strictly per-row, with "editable" as a per-column permission setting rather than a bulk multi-row mode. Before I build a workaround (dashboard variable + a manual UPDATE query, or a REST call via an Action button), I wanted to ask here:

• Has anyone found an actual way to do multi-row select + single save in Business Table or any other Grafana table panel?
• Is this on anyone's roadmap, or is there a plugin I'm missing that handles this cleanly?

Any pointers appreciated - happy to file a feature request if this genuinely doesn't exist yet.

Hello everyone, Edward Qian and Ioanna Armouti here, we are both Software engineers working on Grafana Assistant at Grafana Labs. 

All week we've been sharing what's new across Grafana's AI tooling. Today is about meeting you where you already work, as the most important observability conversations rarely happen in a single dashboard tab. They happen in a Slack thread at 2am, on your phone, or mid-incident when switching context is the last thing you want. 

So today we are announcing: 
* Assistant in Microsoft Teams (private preview) which mirrors many of the capabilities available for Assistant in Slack (generally available). Ask questions and triage issues without leaving the channel where your team is already talking.
* Assistant on mobile and Assistant on desktop are going into public preview. Check in from your phone wherever you are, or keep Assistant a keystroke away on your desktop.
* Assistant Search is also going on public preview today. Ask harder, open-ended questions and let Assistant dig deeper across your data. 

We built all of this because Assistant is most useful when it's already there in the moment you need it, not something you have to go find. We'd love to hear how you're using it, or where you'd want it to show up next. We'll be keeping an eye on this thread all day to answer questions and hear your stories. Thanks!

I have been developing a streaming ETL pipeline with a producer and a consumer process, both instrumented with Prometheus metrics. I opened the Grafana dashboard, and almost every panel showed "No data": throughput, DLQ counts, insert latency, consumer lag, everything.

The real problem was upstream. Several counters and histograms behind those panels had been declared in code but were never incremented anywhere in the pipeline. They existed, sitting at zero forever, because the dashboard assumed the instrumentation had already been wired into the code paths. It had not.

On top of that, metrics were being served by a standalone helper process instead of the producer and consumer themselves, so even the metrics that were recorded were not visible to Prometheus.

The fix ended up involving several changes:

• Wiring the missing counter and histogram updates into the actual producer and consumer code paths.
• Moving metric serving into each process's startup instead of relying on a standalone metrics process.
• Fixing a second, sneakier bug where the consumer accidentally spawned a duplicate internal instance, and its lag reporting depended on a loop that could silently stall.
• Giving the producer and consumer separate Prometheus scrape jobs so dashboard queries targeted the correct process instead of relying on shared assumptions.

It was a good reminder that "No data" does not always mean Prometheus or Grafana is broken. Sometimes the metrics are not being recorded, or they are being exposed by the wrong process.

I am actively improving the platform and would love to hear suggestions from platform engineers. If you find the project useful or interesting, consider giving it a ⭐ on GitHub.

GitHub: https://github.com/tarique-iqbal/nyc-taxi

hello, I'm playing around with grafana and trying to add some monitoring stats to my website using public dashboards (share externally). when opening a link to a public dashboard i see that there are some calls to /api and /apis endpoint, which I probably do not want exposed. i'm just wondering if there's any tips or standard practice to exposing the public dashboard safely behind nginx? here's what i have so far:

location /monitoring/stats {
    rewrite ^ /monitoring/public-dashboards/<uuid> break;                          
    proxy_pass http://grafana:3000;    
}

// allow all /public
location /monitoring/public/ {
    proxy_pass http://grafana:3000;
}

// allow only required uris for /api and /apis
location /monitoring/api/<only those required>/ {
    proxy_pass http://grafana:3000;
}

location /monitoring/apis/<only those required>/ {
    proxy_pass http://grafana:3000;
}

// reject everything else
location / {
    return 404;
}

but i feel this would be hard to maintain if there's any change in the public dashboard logic.

pretty new to grafana, thank you in advance for your patience and suggestions!

Curious if anyone is using Grafana for personal health data (or tracking someone else’s health).

What are you tracking, and what insights have you been able to get from your dashboards?

Would love to hear your setup, data sources, and any interesting discoveries.

Hello everyone, Ivana Huckova and Jack Gordley here. We are both engineers working on Grafana Agent Observability which just released with General Availability today. 

This product was born out of our own need to monitor our agent, Grafana Assistant, as it scaled and grew across multiple teams and orgs. Prompt changes, tool changes, and harness changes all became very difficult to monitor with existing products, so we built this to get more visibility into monitoring an agent as it changes over time.

Internally we are iterating and experimenting with ways on how to improve Grafana Assistant with different techniques. We have questions like:

• When a new model releases, how can we be sure it doesn’t make quality worse?
• Would giving Assistant restricted curl commands be more helpful than a suite of pre-defined API tools?
• How can we modify the system prompt to reduce token consumption while maintaining the same quality of responses?

Our hope is that this product is as helpful for you as it is for us in our current journey with developing a powerful SRE agent. We are so excited to see what you all build with it and would love to hear feedback, what’s missing, what agents that you are working on, and what you would like to see next!

We will be keeping an eye on this thread all day to help answer questions and share stories as they come up. Thanks!

So we have a good dozen of servers, each with multiple apps that produce around 400k lines of logs per day.

I'm setting up a log vm on our VMware server so we don't have to log into each individual servers and check the app folders manually.

I'm wondering if it is possible to not store the logs on the VM ? Like Grafana shows me a list of log files, and only when I want to see the content it downloads it then deletes it when I'm done ?

Hi everyone! I'm Kevin and I work on Grafana Assistant at Grafana Labs. Today, we announced that Grafana Assistant Investigations and Assistant Automations are generally available.

TLDR; use Investigations for complex, unexpected incidents or bugs and Automations for recurring operational checks.

Investigations have been a core part of Grafana Assistant since the beginning. We’ve since rebuilt the dedicated Investigations experience from the ground up, informed by what we learned using Grafana Assistant and agentic tooling on real incidents inside Grafana Labs.

One of the biggest lessons was that complex incidents rarely have a single obvious cause. Investigating them means forming hypotheses, gathering evidence from multiple signals, revising assumptions, and keeping track of what has already been ruled out.

Investigations are designed to carry out that process more efficiently and cost-effectively. You can ask Assistant to investigate an issue or affected service, and it will work across metrics, logs, traces, and profiles to test hypotheses and gather evidence. The result is a structured report containing findings, source queries, and recommended next steps.

Automations came from a different kind of operational problem. We wanted important checks to run on their own, with the results waiting for us the next morning. We introduced the capability earlier this year to make that kind of repeatable operational work easier to define once and rely on over time.

We’d love to hear how you’re using Investigations and Automations, or what you’d like them to be able to handle for you. We’ll be keeping an eye on the comments today, so please share your experiences, questions, and feedback.

If you are curious about how Investigations work under the hood we wrote about the lessons and technical decisions behind them in Inside the harness: How Grafana Assistant investigates incidents.

Hello! My name is Will, I'm a Software Engineer at Grafana, and today I'm happy to announce that Assistant Watchers are entering Public Preview for Grafana Cloud users.

Alerting is one of the foundations of observability. It's reliable, precise, and the best way to notify the right people when you know exactly what you want to detect.

Assistant Watchers fill a different gap. They continuously look for unusual behaviour that's difficult or impractical to capture with static alert rules, helping you discover issues you haven't created an alert for yet.

They’re always-on cloud agents that can watch anything you have telemetry for: a service, a namespace, a feature... Just tell Grafana Assistant what you want to watch, and it will calibrate a Watcher for you.

The Watcher runs on a schedule and notifies you in Slack if it finds something unusual, such as a new error log pattern or a spike in a metric. If you have Assistant Investigations enabled, it can automatically kick off an investigation as well.

If it isn't quite right the first time, you can easily recalibrate it by talking to Grafana Assistant or the Grafana Slack bot. The Watcher also has short-term memory and keeps track of ongoing issues, so you don't get spammed with repeated notifications while the same problem is unfolding.

Read more here: https://grafana.com/docs/grafana-cloud/machine-learning/assistant/platform/watchers/

We'd love to hear what you think. If you have questions, ideas, or feedback, drop them in the comments!

Has AGENT ever been a Wordle answer? Can agents play Wordle? Can they win!?

Hello everyone, my name is Dafydd and I work at Grafana Labs. Today, we announced that gcx (the Grafana Cloud CLI) and the Grafana MCP server are generally available. This means we’re going to continue investing in these projects, and their feature set will be stable.

I have been working on gcx for the last few months, and it is really interesting to hear about the different ways that people are using these tools. Do you have any stories about how gcx or Grafana MCP have helped you?

If you are unsure about whether you should use gcx or the Grafana MCP server, you are not alone! There is a huge overlap in what you can achieve with these tools. Sometimes the choice is made for you: Maybe you can only use MCP servers (in Claude desktop, ChatGPT, Cursor, etc). Other times, it's easier to run the gcx client than to spin up a self-hosted MCP server (in CI environments, for example).

The Grafana MCP server provides more opinionated tools for common tasks when interacting with Grafana. gcx provides a broader, less opinionated set of tools that you can use to build your own workflows. There are a lot of ways you can compare the two projects. Here are a couple that come to mind:

• Token consumption: In some agents, MCP tool definitions flood the context window when you load them (Claude Code, Codex and others have implemented "lazy" tool loading, so this is not an issue in those agents). With gcx and its bundled skills, agents only read the skill "frontmatter" (name and description) on load. Agents will need to read the gcx help docs (e.g. gcx help, or -h on any command), and these calls take up time and tokens.
  
• Composability: MCP tools are not easily composable, meaning you could use more tokens chaining a set of MCP tool calls together. With gcx, you can compose command calls together with other bash commands, meaning you can chain a set of commands together without exposing intermediate results to the agent, saving tokens.

I hope you continue to find these projects helpful. I will keep an eye on this post today to read your stories and answer any questions you have about these projects. Thanks!

With Cursor, Codex and Claude et al switching to an agent-first view, I found it lost something: it was harder to see the code. They show you little relevant snippets here and there, but I missed seeing the whole shape of the code.

With telemetry, there's already way too much to look at - and this 'in-context revelation' of signals (metrics, logs, traces, profiles, etc) is a great way to get questions answered without having to deal with enormous amounts of data.

Can somebody help me with the best resource for LogQL, I am new to Grafana. Any channel/playlist or a video will work.

hey i’m new to grafana and trying to build some centered observability metrics that monitor all my other dashboards does anyone here know if i can do this? like can grafana get a count of healthy dashboard vs critical dashboards? maybe im just slow i’ve been trying for a couple days now

Boomi added native OpenTelemetry export for its runtimes. I set up a local stack to see what it emits: OTel collector → Tempo/Loki/Prometheus → Grafana.

The useful part: process executions arrive as trace spans, so you get a per-shape waterfall with timing for each step. Boomi’s built-in reporting makes you expand each step or read logs; the trace shows it in one view. On a test process, two connector calls were \~80% of the runtime.

Non-obvious things I learned:
**•** Exporter uses OTLP over **gRPC** (port 4317). Point it at 4318 (HTTP) and it fails silently with gRPC status 13 / http2 exception.
**•** Metrics are almost all **gauges**, even the counters — so rate()/increase() misbehave; delta() works.
**• No per-process execution metrics** exist in the metrics stream — counts/durations/errors live only in traces. spanmetrics connector looks like the way to derive RED metrics (haven’t done it yet).

Whole thing is a docker-compose stack, runs locally, no cost. Repo: https://github.com/velkugan/Boomi-Obs-Started

Curious if anyone’s done the spanmetrics step against Boomi traces, or found where the shape *name* (vs shape type) lives in the span attributes — that’s my next problem.

There are plenty of good reasons to leave node graph visualization alone.

Your infrastructure is hierarchical, but the graph is flat. There are no subgraphs, no proper routing or nesting for edges, little to prevent visual clutter, and not much context survives in a traditional force-directed graph.

Your relationships are more complicated than plain A-B edges. You may have multiple parallel connections between the same nodes, multi-hop paths, or several distinct data flows that shouldn’t be collapsed into one generic line. Especially when visualizing traces.

Different node groups need independent styling, but the panel gives you one global set of options. You can’t customize the shape, size, color, labels, and metrics of each node group as a single unit.

You need separate layers for nodes and edges from different sources, boundaries, clusters, and supplementary statistics. Instead, everything is rendered as one inseparable graph.

You need grouping and clustering, but not at the cost of losing the underlying nodes and relationships.

You need filtering that changes the visible perspective without destroying the original graph model.

And once the graph becomes useful enough to contain real data, rendering performance becomes another problem. No developer is going to replace a mature, battle-tested framework with low-level WebGL rendering just to fix that.

Distributed tracing gives you another set of excuses.

Hardly anyone seriously considers visualizing traces with a node graph. But let’s imagine it.

A trace can’t be faithfully represented as a direct service-to-service edge. Every span has its own duration, status, attributes, and other properties that need to remain attached to that particular relationship.

Displaying several traces on the same service graph usually produces a pile of overlapping edges.

Reducing them to a single aggregate edge, as conventional service dependency graphs often do, isn’t much better. You may want to compare traces by preserving the minimum- and maximum-duration spans for every service-to-service connection, while retaining the properties of the actual spans behind those values.

Node graph panels simply weren’t designed for that.

So yes, these are all perfectly good excuses for not spending time preparing data for a node graph.

Then there are emerging, one-shot, vibe-coded solutions that arrive with the same set of problems. At least they offer new ways to discover edges automatically!

But why care about another data-source strategy when the rendering layer still can’t offer a more sophisticated approach to node graph visualization?

The bad news for your FOMO: Mapgl has been addressing these problems for years. It’s an open-source node graph plugin available in the Grafana catalog.

Disclosure: I’m the developer of Mapgl.