What Is Technology Ethics? How Game Theory Explains Trust in States and Criminal Organizations

What actually separates state power from organized crime?

The sociologist Max Weber defined the state as the entity that can successfully claim a monopoly on the legitimate use of physical force. The operative word is "legitimate." The force a state exercises and the force a criminal organization exercises are, as raw acts, on a continuum -- what separates them is purely whether society recognizes that force as legitimate.

The economist Mancur Olson made this distinction concrete with a specific model. Compare a "roving bandit," who plunders and moves on, with a "stationary bandit," who settles in one place for good. A stationary bandit has an incentive to provide some order and let a reproducible economy flourish, rather than looting everything at once, because doing so lets him extract a steady stream of tax (or protection money) year after year. Olson argued that this revenue-maximizing stationary bandit -- one who supplies order in order to sustain extraction -- is close to the actual origin of the state as an institution.

What this theory suggests is that what separates a state from a criminal organization isn't the moral character of the actor, but the time horizon of the relationship -- how long the interaction is expected to continue.

Both depend on trust, and betrayal is the biggest risk for both

Whether it's a state or a criminal organization, drawing cooperation from members or from the surrounding population requires securing trust in some form. Pay your taxes and you get public services; pay protection money and you're left alone -- both rest on the expectation that a promise will be kept. And for both structures, the single biggest risk is betrayal: corruption or arbitrary abuse of power for a state, informants or internal turf wars for a criminal organization.

This is where the game-theoretic framework of repeated games becomes useful. In a one-shot game, betraying your counterpart to keep all the gains for yourself can be the rational move. But in a repeated game, where the relationship is expected to continue, cooperation becomes the rational equilibrium as long as the loss from forfeiting future cooperation -- the "shadow of the future" -- outweighs the short-term gain from defecting now. This is formalized generally in game theory as the Folk Theorem.

The important point is that this logic applies equally to states and to criminal organizations. A criminal code that punishes traitors and a state's accountability mechanisms (elections, courts, audits) are solving the exact same problem -- making short-term betrayal costly enough to protect future cooperation -- through different enforcement mechanisms: violence versus due process. In other words, the right basis for deciding whether to trust something isn't whether it claims to be "good," but how much it stands to lose by betraying you, and how long the relationship can credibly be expected to continue.

The longer a secret is kept, the bigger the backlash when it surfaces

Trust accumulates slowly, through the gradual buildup of small acts of cooperation. But when a fact that was propping up that trust turns out to have been concealed for a long time, trust doesn't just erode -- it collapses instantly and disproportionately. Behavioral economics' prospect theory (loss aversion, negativity bias) shows that people feel losses more intensely than equivalent gains. Losing a store of trust built up over years is exactly that kind of loss, and it carries an outsized psychological weight.

In game-theoretic terms, this mirrors why the grim trigger strategy -- cooperate until a single defection is discovered, then withhold cooperation forever after -- shows up repeatedly in real human and organizational relationships. The longer a secret is held, the larger the track record of cooperation it has enabled, but the larger the loss when it's finally exposed, in direct proportion. Keeping a secret for a long time is itself a bet that amplifies the blowback if it's ever revealed -- that's the claim of this section.

Information superiority and secrecy in war

What Clausewitz called the "fog of war" -- the fundamental uncertainty of wartime -- is formalized in modern international relations theory as the security dilemma. As the political scientist Robert Jervis argued, when states have no reliable way to verify each other's true intentions (i.e., information is asymmetric), mutual suspicion can drive an arms race or a conflict even when both sides would genuinely prefer peace.

Within that structure, information superiority -- concealing your own intent while learning your opponent's -- carries decisive strategic value. The Allied breaking of the Enigma cipher in World War II (the work at Bletchley Park, including Alan Turing) is a historical case where the tug-of-war between a technology for keeping secrets and a technology for breaking them arguably shaped the outcome of the war itself. States pour enormous resources into both espionage and counter-intelligence for exactly this reason: information asymmetry is as strategically valuable a resource as military force itself.

Bringing this back to a technologist's vantage point

This may all sound abstract, but it bears directly on how the security industry operates day to day. We're constantly asked to decide whether to trust government agencies, vendors, open-source maintainers, or researchers who report vulnerabilities to us. Following the logic above, that decision shouldn't hinge on the label someone claims or the apparent morality of their intent, but on whether the relationship functions as a repeated game -- where betrayal costs future cooperation -- and how much verifiability and transparency actually exist.

The practice of Coordinated Vulnerability Disclosure, which we discuss in a separate article, is exactly this logic crystallized into an institution. Researchers and vendors sustain a shared commitment to not deceive each other, which creates a "shadow of the future" -- continued cooperation next time -- and builds up trust across the entire industry. Trust, in this view, isn't something you extend by taking someone's reputation or label at face value; it's something that only exists because the structure of the relationship itself is designed, and kept verifiable.

Summary

• Weber's definition holds that what separates a state from organized crime is not the nature of its force, but whether society recognizes that force as "legitimate"
• Olson's stationary bandit theory models the origin of the state as an economic actor providing order in order to maximize sustained extraction
• Both states and criminal organizations depend on trust; the repeated game and its folk theorem show cooperation is sustained by the "shadow of the future," not morality
• A long-held secret, once exposed, triggers a disproportionately large collapse of trust because of loss aversion (prospect theory)
• The security dilemma and information asymmetry explain why secrecy and espionage carry as much strategic weight in war as military force itself
• For technologists, the practical implication is to base trust on the repeated structure of a relationship and its verifiability, not on the label someone claims