{"id":2053,"date":"2026-11-14T10:50:00","date_gmt":"2026-11-14T10:50:00","guid":{"rendered":"https:\/\/devopsschool.org\/blog\/?p=2053"},"modified":"2026-11-15T00:50:00","modified_gmt":"2026-11-15T00:50:00","slug":"iamops-nightly-job-team-story","status":"publish","type":"post","link":"https:\/\/devopsschool.org\/blog\/iamops-nightly-job-team-story\/","title":{"rendered":"The Quarterly Report That Vanished: An IAMOps Team Story"},"content":{"rendered":"<p>The team in this story is invented for illustration, but you may recognise every beat. Six engineers run the delivery platform for a mid-sized online retailer. They are good at their jobs, a little tired, and proud of how quickly they ship. Their cloud account has grown for five years, and so has the pile of permissions nobody has looked at since. On a Tuesday in March, one tidy-up changes how they think about IAMOps, the craft of drafting and refining permission rules for staff and software.<\/p>\n<h2>The tidy-up<\/h2>\n<p>It starts innocently. A team member, annoyed by an audit finding, picks one broad role and removes everything it has not used in the last thirty days. The role belongs to a reporting service. The change is reviewed, merged and applied. Nothing breaks that day or the next. The team feels quietly pleased.<\/p>\n<p>Then the end of the quarter arrives. A finance job that runs four times a year starts, reaches for a storage location, and is refused. By morning, a report that executives rely on is missing. The permission removed had not been used for thirty days because it had not been needed for ninety. Usage data had a blind spot, and the team had walked straight into it.<\/p>\n<h2>The retrospective<\/h2>\n<p>The meeting that follows is surprisingly constructive. Nobody argues about blame; they argue about method. The platform lead draws a loop on the whiteboard: set, verify, refine. &#8220;We did the third step without the second,&#8221; she says. The window they used to verify was far shorter than the slowest job in the system.<\/p>\n<p>The team reads the Ops Atlas page on authorisation as code, which one of them has bookmarked, and recognises their mistake in its list of pitfalls: periodic jobs, calls that leave thin traces and permissions that hand roles to other identities all look unused when they are not. The write-up they lean on is <a href=\"https:\/\/www.xopsschool.com\/iamops\/\">the Atlas entry that treats permissions as code to be tested<\/a>. It also gives them vocabulary for what to do next.<\/p>\n<h2>Four changes the team makes<\/h2>\n<h3>Owners and an inventory<\/h3>\n<p>They list every service account and automation token, and attach a person to each. Three belong to nobody. One is a leftover from a migration two years earlier, and its permissions are broader than any human on the team holds.<\/p>\n<h3>Policy in a repository<\/h3>\n<p>Permissions move out of the console and into files. Every change now has an author, a reviewer and a test. The tests include cases that should be allowed and cases that must be refused, such as &#8220;the reporting job must read the finance bucket&#8221; and &#8220;the reporting job must never write to production databases&#8221;.<\/p>\n<h3>A longer window, and a conversation<\/h3>\n<p>Before removing anything, they check usage over at least a full quarter and speak to the job&#8217;s owner. If the job is seasonal, the permission stays but gets a comment explaining why and a review date.<\/p>\n<h3>Short-lived credentials for the pipeline<\/h3>\n<p>The deployment pipeline switches from a stored key to a token issued per run. They are careful with the trust condition, using an exact repository and environment rather than a wildcard, since a loose pattern would admit any repository in their organisation.<\/p>\n<h2>A detail that surprised them<\/h2>\n<p>While writing tests, the team discovered a rule that was meant to block deletions outside business hours. The condition was written as &#8220;not equal to the maintenance window&#8221;, and a request that arrived without the expected time attribute sailed through, because the negated test matched when the value was absent. Nobody had noticed for years. The fix was a positive condition plus a test that sent a request with the attribute missing. It took ten minutes and removed a risk they had been carrying without knowing.<\/p>\n<h2>What changed after a quarter<\/h2>\n<p>The numbers are small but telling. The pile of stored keys shrank by nearly half. Across their roles, the typical share of permitted actions in real use almost doubled. Getting new access took longer, creeping up by twelve hours, which the team accepted, since each request now came with a clear owner and an expiry date. Those figures echo a worked example in the Atlas rather than a measurement from any particular company.<\/p>\n<h2>The conversation with a newcomer<\/h2>\n<p>A few months later a junior engineer asks why the team will not simply give her broad access &#8220;to get started&#8221;. The lead tells the quarterly-report story, then says something the team now repeats: narrow first, widen on request, and always put a date on the exception. The junior engineer writes it on a sticky note, which stays on her monitor for the rest of the year.<\/p>\n<h2>Lessons worth borrowing<\/h2>\n<ul>\n<li>Removal is a change like any other, so give it a test, a reviewer and a rollback.<\/li>\n<li>Measure over a window longer than your slowest recurring job.<\/li>\n<li>Write tests for refusals, not only for approvals.<\/li>\n<li>Keep an exception register with owners and expiry dates.<\/li>\n<\/ul>\n<h2>Where to sharpen the skills<\/h2>\n<p>The reasoning habits that rescued this team are the same ones taught in <a href=\"https:\/\/devopsschool.org\/blog\/devops-problem-solving-how-to-build-critical-thinking-skills\/\">our guide to building critical thinking for DevOps problem solving<\/a>, and the self-audit approach in <a href=\"https:\/\/devopsschool.org\/blog\/devops-knowledge-gaps-students-must-fill-a-practical-guide\/\">the practical list of knowledge gaps DevOps students need to close<\/a> is a good way to find where your own team&#8217;s blind spots lie. Learners looking for a first exercise could replay the story: build a small sandbox, create a role with too much power, and measure what it really uses.<\/p>\n<p>The Tuesday in March did not end the team&#8217;s habit of moving fast. It gave them a map of the places where speed needed a guardrail, and the missing report became the most useful incident they never had to write up as an outage.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>An invented platform team removes a role&#8217;s unused permissions, loses a seasonal job, and rebuilds its approach with owners, tests and short-lived credentials.<\/p>\n","protected":false},"author":5,"featured_media":2054,"comment_status":"","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[489],"tags":[],"class_list":["post-2053","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-devops-careers"],"_links":{"self":[{"href":"https:\/\/devopsschool.org\/blog\/wp-json\/wp\/v2\/posts\/2053","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devopsschool.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devopsschool.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devopsschool.org\/blog\/wp-json\/wp\/v2\/users\/5"}],"replies":[{"embeddable":true,"href":"https:\/\/devopsschool.org\/blog\/wp-json\/wp\/v2\/comments?post=2053"}],"version-history":[{"count":1,"href":"https:\/\/devopsschool.org\/blog\/wp-json\/wp\/v2\/posts\/2053\/revisions"}],"predecessor-version":[{"id":2146,"href":"https:\/\/devopsschool.org\/blog\/wp-json\/wp\/v2\/posts\/2053\/revisions\/2146"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devopsschool.org\/blog\/wp-json\/wp\/v2\/media\/2054"}],"wp:attachment":[{"href":"https:\/\/devopsschool.org\/blog\/wp-json\/wp\/v2\/media?parent=2053"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devopsschool.org\/blog\/wp-json\/wp\/v2\/categories?post=2053"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devopsschool.org\/blog\/wp-json\/wp\/v2\/tags?post=2053"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}