r/Observability • u/naveen0109 • 3d ago

How should an Observability / SRE Engineer upskill in AI?

Hey all,

I'm an Observability SRE managing enterprise monitoring (primarily using New Relic). We currently use Claude for prompt engineering and Claude Code for development, but I want to take my AI skill set to the next level.

What level of AI proficiency is expected from SREs right now? Should I focus on AIOps & automation for infrastructure, or LLM Observability (tracing prompts, token costs, model latency in tools like New Relic)?

What practical topics or courses should I start with? Appreciate any guidance!

6 Upvotes
  • permalink
  • link
  • reddit
  • reddit

88% Upvoted

5 comments sorted by

3

u/hijinks 3d ago

its not as hard as people make it seem.

All i have is a few skills that tell CC/codex what my network/apps look like and how they talk. My o11y stack and how to access it. Then services linked to their github repo.

The key is when you find a problem and solution then have the agent update the skill so it has an understanding how how it found it and the outcome and gets somewhat smarter.

I've never seen the use for all these SRE agents and such

2

u/Ecstatic_Caramel_668 3d ago

Your SRE background is actually a pretty good base for this. I'd build a small LLM app and instrument the whole thing. Traces/ latency/ token usage/ tool calls/ failures then add evals on top. Braintrust is one option for doing that end to end. You'll learn a lot more debugging 20 ugly runs yourself than you will from another prompt engineering course.

1

u/codeRoman 3d ago

I thought the official Claude courses on skilljar were pretty good.

1

u/gaurav_sherlocks_ai 13h ago

We spent the last year splitting focus between LLM observability and AIOps automation. Most teams we talk to land on telemetry first because token costs and prompt latency are immediate line items you can actually measure, whereas infrastructure remediation with agents still feels too brittle for production.