Fixing Failed Workflows With Real GitHub Smoke Tests
The dependabot-auto-merge workflow keeps failing on non-Dependabot PRs - even though the two-job guard pattern from #8 was meant to stop broken runs. The root cause? GitHub resolves reusable workflows too early, bypassing logic that depends on PR context. This isn’t a code bug - it’s a context blind spot in deployment. Smoke tests that actually call real GitHub APIs solve this by checking real job behavior, not just simulated outputs. For example, a test triggering a PR from a known repo should pass cleanly, while a non-Dependabot run fails fast. These tests don’t just validate results - they catch failures before deployment. This approach mirrors how GitHub itself runs workflows, making them indispensable for trust. Hidden detail: most teams skip this step because mocks feel safe, but they miss the real failure mode. The real catch? Not testing how workflows respond to actual PR metadata - like authorship or repo structure. Safety first: always validate in a sandbox repo before full rollout. The bottom line: don’t ship workflows without real API smoke tests - your org’s stability depends on it.