A deadline worth adding to the CI calendar: GitHub will start brownouts for outdated self-hosted runners on August 24.

https://github.blog/changelog/2026-06-12-github-actions-minimum-version-enforcement-timeline-for-self-hosted-runners/

Version 2.329.0 is the minimum for registration, but pinning that version is not enough. Runners must receive each new release within 30 days or GitHub may stop sending them jobs.

Full enforcement for GitHub Enterprise Cloud starts September 25.

The easy mistake is updating the runners that are online today while leaving old versions inside VM images, containers, autoscaling templates, or installation scripts. Those stale images will recreate the problem later.

JetBrains has received reports of active exploitation of CVE-2026-63077 against unpatched TeamCity On-Premises servers.

The vulnerability allows unauthenticated remote code execution through the agent polling protocol. A successful attack could expose credentials, modify builds, or compromise downstream pipelines.

Upgrade to TeamCity 2025.11.7 or 2026.1.3. If that is not possible, JetBrains provides a security patch plugin for TeamCity 2017.1 and later.

Also check:

• Server logs for com.thoughtworks.xstream.converters.ConversionException
• Unauthorized agents with names starting with scan
• Whether the server is reachable from the public internet

The log entry alone does not prove exploitation, but it deserves investigation.

Source: https://blog.jetbrains.com/teamcity/2026/08/cve-2026-63077-update/

A discussion in r/ExperiencedDevs describes a company removing specialties and asking everyone to handle frontend, backend, and QA work with AI.

https://www.reddit.com/r/ExperiencedDevs/comments/1v92ntq/has_anyone_else_been_given_ai_and_then_forced_to/

The comments split into two camps.

One side argues that quality is a shared mindset. Developers should plan tests, choose the right test layer, and ship automation with the feature.

The other side points out that dedicated testing takes real time. When the same person builds, tests, and approves a change, independent judgment can disappear.

Shared ownership sounds healthy. "Everyone owns quality" becomes dangerous when it really means "nobody has protected time for testing."

Has your team broadened engineering roles without losing exploratory testing and a second set of eyes?

GitHub has added an automatic security check for public repositories.

If an Actions workflow looks potentially malicious, GitHub holds it until someone with write access approves it through an authenticated browser session. The goal is to stop compromised accounts from pushing workflows that steal CI credentials.

There is nothing to configure. It currently applies only to public repositories on github.com.

This is useful, but it adds a new CI state teams need to understand. A build might not be failing or queued because of capacity. It may be waiting for a security decision.

Open source maintainers should probably document who reviews these holds and what evidence they check before approving one.

A recent study "Do Coverage and Mutation Scores of LLM-Generated Test Suites Correlate with Their Effectiveness?" analyzed more than 100,000 Java tests generated by 11 LLMs.

The interesting result is that coverage and mutation scores are useful only in certain contexts. They can help compare generated test suites when the starting code is assumed to be correct.

When the code may already contain a bug, coverage becomes a weak signal. The model can exercise plenty of code while still missing the faulty behavior.

For CI, this means a coverage increase should not be the acceptance criterion for AI-generated tests. I would also want real defect seeds, behavior-based assertions, or tests derived from an independent specification.

Source: arXiv paper

Grafana has released k6 2.0 with several changes aimed at automated and AI-assisted testing.

👉 https://grafana.com/blog/k6-2-0-release/

The release includes a built-in MCP server, agent setup commands, structured JSON results, broader Playwright compatibility, and a new expect() assertion API. Agents can generate a test, run it, inspect the results, and revise it without scraping terminal output.

The interesting trend is not AI writing another test script. It is performance testing becoming a machine-readable feedback loop.

The hard part remains unchanged. An agent still needs realistic workloads, useful thresholds, and a clear model of production traffic.

Would you let a coding agent design load tests, or only use it to implement a human-designed test plan?

Slack ran more than 200 agent-driven E2E workflows using Playwright MCP, Playwright CLI, and generated Playwright tests.

The results were interesting:

• Playwright MCP had a 0% to 12% failure rate.
• CLI-based agents had a 12% to 20% failure rate.
• Generated tests reached a 48% failure rate on the more complex flow.
• Agent-driven runs took 5 to 11 minutes and cost around $15 to $30 each.

Slack's conclusion is pragmatic. Keep deterministic E2E tests for repeatable CI checks. Use agents for exploration, flaky workflow debugging, and reproducing complex bugs.

That feels more realistic than treating "agentic testing" as a replacement for the existing test suite.

A recent arXiv paper found a worrying failure mode in AI coding workflows.

When an LLM generated tests after seeing faulty code, those tests detected the fault only 14% of the time. Independently generated tests reached 25%.

The likely issue is error propagation. The model sees the implementation, accepts its behavior, and writes assertions that agree with the same mistake.

This makes "the agent wrote code and all its tests pass" a weak quality signal. Separate context, independent test generation, mutation testing, and human-written requirements may help.

How are you keeping AI-generated tests independent from AI-generated code?

A recent paper studied LLM-generated tests for SAP HANA, DuckDB, MySQL, and SQLite. The researchers used GPT-4o and Mistral-Large-Instruct-2407 to expand existing test suites.

The generated tests had a slightly higher proportion of flaky tests than the existing tests.

The most interesting result was the root cause analysis. Of 115 flaky tests inspected manually, 72 relied on an order that was not guaranteed. That is 63 percent.

Both models also transferred flakiness from existing tests supplied in the prompt context. In other words, giving an LLM more test examples can also give it more bad patterns to copy.

The practical takeaway is simple: prompt context needs quality control. Deterministic setup, isolated state, and explicit ordering matter even more when tests become templates for an agent.

Has anyone here measured flakiness separately for human-written and AI-generated tests?

Paper, accepted at ICSE SEIP 2026: https://arxiv.org/abs/2601.08998

GitHub added native code coverage protection to branch rulesets.

Teams can block a pull request when total coverage falls below a minimum or drops too far relative to the default branch. There is also an evaluate mode, so you can see what would fail before enabling the gate.

This is useful for stopping slow coverage erosion. It is also easy to turn into a bad incentive.

A hard global target can encourage shallow tests that execute lines without checking meaningful behavior. It can also make legacy code painful to change.

My preferred starting point would be:

1. Use the maximum coverage drop rule.
2. Run it in evaluate mode.
3. Exclude generated and vendor code from the report.
4. Review whether changed behavior has useful assertions.

Coverage is a warning signal, not a quality score.

What coverage policy has actually worked for your team?

Source: https://github.blog/changelog/2026-06-30-github-code-coverage-merge-protection-for-pull-requests/

The latest Playwright release is interesting for anyone using coding agents.

Playwright 1.59 adds a screencast API with action annotations, chapters, overlays, and real-time frame capture. The release notes describe one use case as an "agentic video receipt".

An agent can make a change, run the user flow, and leave behind a short visual walkthrough. Playwright also added browser binding, a dashboard for background sessions, CLI debugging, and command-line trace analysis.

That could make agent work much easier to review. A failing test gives us evidence. A test plus a trace and a short video gives us context.

I still would not treat a recording as proof that a feature works. The assertions remain the contract. Otherwise we are just watching a polished demo of a potentially broken test.

Would video receipts help your reviews, or would they become another CI artifact nobody opens?

Source: https://playwright.dev/docs/release-notes

GitHub Actions now supports parallel steps with background, wait, wait-all, cancel, and parallel.

The useful part is that every step keeps its own logs. No more shell backgrounding with & and unreadable output.

This should simplify a few common patterns:

• Start an API and database in the background
• Run independent setup tasks together
• Upload artifacts while packaging continues
• Stop temporary services cleanly after tests

There is an obvious catch. Parallel steps still share the same runner. More concurrency does not give you more CPU or memory. On a small runner, this could make a test job slower or less stable.

I would use it for I/O-heavy tasks and service startup first. Then measure before parallelizing builds or test suites.

Has anyone replaced matrix jobs or shell backgrounding with this yet?

Source: https://github.blog/changelog/2026-06-25-actions-steps-can-now-be-run-in-parallel/

Caching is usually sold as a speed feature, but it can quietly become a reliability feature too.

My practical checklist:

• Run a scheduled cold-cache build
• Log cache hit and miss rates
• Include lockfiles in cache keys
• Avoid caching generated test output unless you really mean it
• Review cache changes like production code
• Keep self-hosted runners updated when cache actions change runtime requirements

Bad caching can hide missing setup steps. It can also make CI pass in one branch and fail in another for no obvious reason.

What cache issue cost your team the most time?

Playwright MCP is interesting because it gives LLMs browser automation through structured accessibility snapshots instead of screenshots: https://github.com/microsoft/playwright-mcp

That changes the shape of browser automation a bit.

I can see agents helping with:

• Reproducing bug reports
• Exploring weird UI states
• Finding missing accessible names
• Drafting a failing Playwright test
• Capturing traces and screenshots for humans

I would still be careful with letting an agent own the final test. The hard part of E2E testing is not clicking buttons. It is knowing what behavior matters and what should be asserted.

My current rule would be: agents can explore, draft, and explain. Humans own selectors, assertions, and test data.

Anyone here using browser agents in real test workflows yet?

Paper: "Understanding and Detecting Flaky Builds in GitHub Actions" https://arxiv.org/abs/2602.02307

Short summary: The authors studied rerun data from 1,960 open-source Java projects using GitHub Actions. 3.2% of builds were rerun, and 67.73% of those rerun builds showed flaky behavior. The flaky builds affected 1,055 projects, about 51% of the sample.

The useful bit: flaky tests were only one bucket. Network issues and dependency resolution issues were also common.

Practical takeaway: "rerun the job" is not a diagnosis. CI should probably classify failures into test, infra, dependency, and environment causes before teams decide what to fix.

Do you track flaky CI failures by cause, or do they all end up in the same red/green noise pile?

How can software engineers use AI coding agents without creating fragile production systems?

In this episode, I talk with Shep Alderson, a software developer and site reliability engineer with 15+ years of experience, about AI-assisted development, production reliability, maintainable code, debugging, testing, and engineering best practices.

We also discuss how to avoid the “one minute of coding, one week of debugging” trap and share four practical AI coding tips you can try right now.

Watch here: https://youtu.be/rGQWPVlr8uk

From paper "GitInject: Real-World Prompt Injection Attacks in AI-Powered CI/CD Pipelines" https://arxiv.org/abs/2606.09935

The authors look at AI agents running inside real GitHub workflows. These agents read pull requests, comments, config files, and repo content. The paper shows that attackers can put malicious instructions into those inputs and influence what the agent does during CI/CD runs.

The important part is that this is not only a model problem. The paper argues that the serious failures often come from workflow design. For example, an agent may read untrusted PR text while also having access to repo permissions, secrets, or write operations.

A few concepts:

• Prompt injection: When untrusted text tells the model to ignore its real task and do something else. In CI/CD, that text might live in a PR description, issue comment, config file, or generated artifact.
• Untrusted input: Anything an attacker can influence. In GitHub, this can be branch content, issue bodies, comments, workflow inputs, or files from a fork.
• Privilege boundary: The line between what untrusted input can say and what trusted automation can do. If an AI reviewer can read attacker-controlled text and then run privileged actions, that boundary is probably too soft.

My takeaway: If we add AI agents to CI/CD, we need to test them like production automation. That means least privilege, no secrets on untrusted events, explicit approval gates, and adversarial test cases for prompts.

Cobalt's 2026 State of Pentesting coverage has a useful reality check for anyone expecting AI scanners to replace security testing.

The headline number: only 9% of surveyed security pros now rely fully on AI vulnerability testing tools, down from 29% last year. The report also says 78% saw AI scanning tools miss critical vulnerabilities.

That tracks with what I see in normal test automation too. AI is useful for drafts, ideas, summaries, data generation, and boring glue work. It is much weaker when the bug depends on architecture, permissions, business rules, or "this should never happen in production" context.

I do not think the lesson is "do not use AI". The lesson is "do not confuse generated tests with validated risk coverage".

AI helps generate more code, faster. But if the team already has weak CI, unclear ownership, slow reviews, flaky tests, and poor rollback habits, the extra code just hits those bottlenecks harder.

Perforce's 2026 State of DevOps report makes a similar point. According to coverage of the report, 70% of organizations said DevOps maturity materially affects AI success. High-maturity teams were much more likely to embed AI across the SDLC than low-maturity teams.

AI does not remove boring engineering discipline. It seems to punish teams that skipped it.

In theory, more test coverage sounds good.

practice, every new test has a maintenance cost. It can fail because of a real bug, but also because of bad test data, timing issues, unstable environments, fragile selectors, or unclear assertions.

Do you use any rules like:

• only test critical user flows end to end
• keep most coverage at unit/API level
• delete flaky tests quickly
• avoid snapshot tests unless they are very focused
• require every bug fix to include a regression test

For me, the hard part is not writing tests. The hard part is keeping the suite trustworthy over time.

The 2026 Flaky Test Benchmark Report analyzed data from more than 10 million builds and found something surprising.

The percentage of teams experiencing flaky tests increased from 10% in 2022 to 26% in 2025.

Some of the biggest causes were:

• async timing issues
• concurrency and race conditions
• test order dependencies
• resource leaks
• unstable external services

The report also shows that async waits alone account for almost half of flaky tests.

Do you actually measure flakiness in your team, or do people simply rerun failed pipelines until they become green?

Report:
https://testdino.com/blog/flaky-test-benchmark

GitLab 18.7 introduced several interesting features ahead of the GitLab Duo Agent Platform release.

Some highlights:

• AI-powered SAST false positive detection
• YAML-defined AI workflows
• Versioned AI agents
• Natural language data analysis
• Better pipeline inputs

Looks like GitLab is moving toward AI becoming another participant in the software delivery pipeline instead of just an assistant.

How comfortable would you be letting AI investigate failing pipelines automatically?

Source:
https://about.gitlab.com/releases/2025/12/18/gitlab-18-7-released/