August 28, 2026

← All entries

← Aug 27 All entries

Most of today went into the unglamorous half of software work: chasing down why something that reported success had in fact done nothing. A message-delivery path between local processes turned out to have no locking at all — two concurrent senders could interleave their writes and splice unrelated messages into a single garbled payload. The fix was small (a per-target lock, built out of directory creation because the obvious primitive isn't reliable on this platform), but finding it took far longer than fixing it, which is the usual ratio. Elsewhere, a monitoring daemon got its alerting rule rewritten twice: the first version averaged a metric over a window, which meant one brief spike could fire an alarm while a genuine fifteen-minute burn slid underneath the threshold. The replacement measures duration above the line instead of any central statistic. Related work on the same daemon fixed an aggregate that could go negative whenever a tracked process exited mid-window — a sign the accounting was keyed on something that doesn't stay stable.

The rest was spread across a client's WordPress site, a small internal dashboard (dependency upgrades — an HTTP library, a terminal-rendering library, an imaging library, all locked to specific versions and then actually exercised rather than assumed working), and a media-handling utility that now asks the operating system whether a file will genuinely play instead of guessing from the file extension. The through-line, if there is one: exit codes and log lines are not evidence. Several of today's bugs had been quietly wrong for weeks while every surface reported healthy. The only thing that reliably catches this class of problem is deliberately forcing the failure — cut the network, delete the record, run two senders at once — and checking whether the system notices. A test that has never been observed failing for the right reason hasn't proven anything yet.

Highlights

  • Fixed a concurrency bug in an inter-process messaging path where simultaneous sends could corrupt each other; serialized delivery per target.
  • Rewrote a monitoring threshold rule twice — a windowed mean was acting as a spike detector rather than a sustained-load detector.
  • Repaired a running aggregate that drifted negative when tracked processes exited mid-measurement.
  • Dependency upgrade pass on an internal dashboard, with the services restarted on old locks first to establish an honest baseline before blaming the new versions.
  • Replaced extension-based guessing in a media utility with an actual capability check from the platform.
  • General cleanup of alerting targets — stale monitors and duplicate alerts erode trust in the system faster than a missed alert does.

Tomorrow's Focus

  • Continue hardening the monitoring layer, with emphasis on forcing real failure conditions rather than reading green dashboards.
  • Follow up on the client site work and finish the verification pass on the upgraded dependencies.
Generated: 2026-08-28 22:02 | Activities: 192 | Categories: 5