RepoJournal
Kubernetes

@kubernetes

Container orchestration — what platform teams ship on

Pick a date

Topics: Go Infrastructure Full archive →

The Wire · Showcase

NODE-PROBLEM-DETECTOR FIXES CRITICAL KMSG WATCHER DEADLOCK

By RepoJournal · Filed · About Kubernetes

A cascading restart bug in node-problem-detector's kmsg watcher caused 37k restarts in 200ms and could deadlock the shutdown path, and the fix just landed.

The kmsg watcher restart mechanism had three interlocking failure modes that only showed up under load [1]. The worst: a parser whose channel closes right after a successful restart triggered a hot restart loop with zero delay, measured at 37k restarts in 200 milliseconds. A second issue deadlocked Stop() when logCh filled up and the consumer was already shutting down, blocking watchLoop on a plain send. The third was a double-close on deferred cleanup when Stop() interrupted a restart attempt. All three are fixed in the same PR [2]. In parallel, node-problem-detector shipped three releases: v1.34.4, v1.35.3, and v1.36.0 [3], with v1.36.0 including the kubelet service name pattern update and dependency bumps [4]. The infrastructure team is also pivoting IBM ppc64le boskos resources from the deprecated wdc06 datacenter to the eu-de-1 region [5], which will improve test reliability for power-based architectures. Minikube landed a cleaner dependency consolidation, replacing golang.org/x/mod/semver with the already-present blang/semver library for Helm validation [6], plus routine addon updates across inspektor-gadget, kong, and cloud-spanner.

Action items

References

  1. [1] Fix kmsg watcher restart and shutdown issues ↗ kubernetes/node-problem-detector
  2. [2] Merge pull request #1311 from hakman/kmsg-watcher-fixes kubernetes/node-problem-detector
  3. [3] Promote node-problem-detector v1.34.4, v1.35.3, v1.36.0 ↗ kubernetes/k8s.io
  4. [4] v1.36.0 ↗ kubernetes/node-problem-detector
  5. [5] Replace wdc06 DC with eu-de-1 region in boskos on ibm ppc64le ↗ kubernetes/k8s.io
  6. [6] addons: use blang/semver for helm version validation ↗ kubernetes/minikube

Quick answers

What shipped in Kubernetes on July 12, 2026?
A cascading restart bug in node-problem-detector's kmsg watcher caused 37k restarts in 200ms and could deadlock the shutdown path, and the fix just landed. In total, 43 commits, 21 pull requests, and 3 releases landed.
Who contributed to Kubernetes on July 12, 2026?
6 developers shipped this update, including Rajalakshmi-Girish, hakman, kubernetes-prow[bot], Roslaan001, minikube-bot, and SURAJ KUMAR.
What were the notable Kubernetes updates?
Fix kmsg watcher restart and shutdown issues, Merge pull request #1311 from hakman/kmsg-watcher-fixes, and Promote node-problem-detector v1.34.4, v1.35.3, v1.36.0.

More from @kubernetes

Daily updates, in your inbox

Follow Kubernetes

Container orchestration — what platform teams ship on We'll email you a link to confirm first.

Free. Confirm via email. Unsubscribe in one click.

— or follow the whole beat:

Elsewhere on the wire