According to most companies that sell security tools, the world is a dangerous place. Hackers are everywhere. They’re crawling around in your EC2 instances, in your employee laptops, in your Gmail inboxes. They’re hiding behind your watercoolers and under your meetings tables and in your breakout booths. Half of your employees are hackers. Hell, your CISO is probably a hacker too — you should fire her and buy more tools.
Billions of dollars are spent around this issue. State-of-the-art WAFs, AI-assisted endpoint protections, eye-wateringly priced security logging solutions that take entire teams just to set up and manage — and yet almost zero care seems to be directed towards one of the simplest and most cost-effective defenses: least privilege.
I’d like to state upfront that this won’t be a persuasive article. I have no grand idea to sell you, no concrete answer to this problem that we’re facing. I’m the Developer Advocate for a company that makes least privilege workflow software, which means the bulk of my job is arguing in comment sections about why least privilege even matters. This article, if anything, is a cry for help.
All I can really do is share the things that we’ve discovered while working in this space, and hope that you’ll share your own insights too. And maybe then we can start thinking about how to solve this thing.
So then why do so few organizations bother to properly scope employee privilege?
Problem 1: They think it’s about trust
Imagine, for the sake of argument, that there is a lava pit in the middle of your office. Most of your co-workers are so used to the lava pit that they don’t even notice it’s there, while a few others have actually created entire geothermic systems that require the lava pit to function. Occasionally someone will fall into the lava pit, suffer grievous injuries, and then be sent into lava pit awareness training.
One day you meet with one of the infrastructure teams to discuss how maybe instead of having lava pit awareness training, we should just get rid of the lava pit. The outrage is immediate: we’ve always had the lava pit! Too many systems rely on it! Getting rid of the lava pit would mean redesigning the entire floor!
Out of all these objections, one of them seems to stand out the most: but we trust our people to not fall into it.
Of course, it’s a ridiculous argument. No one chooses to fall into the lava pit, just as no one chooses to get phished. But it’s one of the more difficult points to argue against. As an engineer, it’s hard to not feel patronized when your lava pit is taken away. Sure, maybe other people in the company are incompetent goons — but you’d never fall into the pit. You’re too smart for that.
Accepting least privilege means admitting to yourself that no matter how smart and good at computers you might be, sometimes shit happens — and you are not immune to it. This is a difficult realization for anyone.
Problem 2: Move fast, break things
No one ever got seed funding for not getting hacked.
Many growth-focused companies tend to sacrifice security in favor of pushing out their product quicker. The narrative is usually “we’ll make it secure once we’ve actually built it”. The issue is that security debt works just the same way that tech debt does — all these oversights and shortcuts accumulate to eventually make any sort of positive change almost impossible. Giving people sweeping amounts of access results in them creating workflows that rely on that over-privileged access, and as more time passes those processes become more ingrained. Removing people’s access is a hell of a lot harder than just not having given them that access in the first place.
This is a problem that mostly applies to startups and rapidly scaling companies (large business have no problem putting red tape around every single access request and process — most of their executives probably have KPIs based around doing so).
Problem 3: What’s your job again?
Implementing least privilege requires actually knowing what all your coworkers do.
That’s an easy enough thing if you work in an organization where everyone has clearly defined roles and never has to step outside those roles to help someone else or meet a changing requirement. But you don’t work in that organization, because it doesn’t exist and we live in Hell.
Any good least privilege architecture must assume that people will sometimes need to do things outside their roles. Overly broad permissions are dangerous, while extremely scoped permissions can limit cross-team functionality and kill creativity. It’s not an easy balance to strike, and ultimately this requires a lot of communication between the security team and everyone else — which, let’s face it, is a whole other article.
Problem 4: It’s all business, baby
This is a section that I hesitate to add, and if it’s included in the final article then it’s only due to the good grace and patience of the people who pay me. But there’s a problem in the business of security: and it’s the business part.
In an earlier company I worked at, I once met with the CFO to request budget approval for a new security hire. I will never forgot what he told me: “What’s the ROI on security?”
I could have replied “what’s the ROI on looking both ways when you cross the street?”, but I didn’t. That was a line that came to me months later, when I was still the only security hire and still bitter about the meeting.
At the time, I thought the CFO was naive and had no business sense. But he was right: there is no ROI on security, only on security optics.
Good security can save you lots of money years down the line, but businesses don’t work on a timescale of years. They work in quarters. Investors want to see what you’ve achieved in the past three months, and telling them that you’ve become unhackable by buying a machine-learning anti-virus is a hell of a lot more impressive than telling them you’ve taken away Dylan’s root access to all the customer data. The first one lets you show off slides full of cool graphs and mean-looking falcon logos, and the second one gets people asking: why the **** did Dylan have root access to all the customer data?
The problem, of course, is that buying the cool anti-virus is often a lot cheaper and less problematic than taking away Dylan’s access. So, here we are.
Problem 5: Implementation
In security, the most interesting (and difficult) problems tend to involve people, not technology. Unfortunately, most least privilege solutions are blatantly anti-people.
No one wants to fill out a Google document and then figure out who to email it to just to get access to a basic part of their job. No one wants to look at tickets on a kanban board just to figure out who they need to give access to. And no one wants to go through all the IAM roles to discover that the penetration tester from two years ago still has AWS access just because no one remembered to revoke it.
So that’s why we’ve created Common Fate, an open-source developer workflow tool that helps you request and approve specific, timeboxed permissions with just a few clicks.
What? Did you think we were going to write this whole thing without advertising our product even once? Nah.
So what do we do about all this?
Honestly? I don’t know. On a more granular level, each of the above problems have their own solution. Talk to your coworkers about how you trust them, you just don’t want to take unnecessary risks. Make sure to do good security design before you need to pay KPMG $25 million to fix it for you. Talk to your coworkers about what their jobs actually are. Check out our Github repo, etc. etc.
On a more holistic level, many of these problems are underpinned by a common factor: an organization’s inability or unwillingness to properly map out their structure and commit to designing better permission workflows. The inability part can be solved with a bit of time and strategy — the unwillingness part, not so much.
