manyagents.ai

27 Aug 2024

Mr Samizdat and the The Moth

This story is based on real future events. Names and places have been changed to protect the unborn.


Mr Samizdat was just negotiating with a client when he received a call. He hardly ever let himself be interrupted during such occasions but if his sister called? It must’ve been bad. He apologized in business speak, sought privacy and picked up:

… dear reader, you may not inspect the call at this moment either …

Then, Mr Samizdat returned to the negotiation. As an active consultant and foremost expert on constructed intelligences, he was being hired to contain one such specimen which recently escaped captivity.

One of the first hard empirical lessons of practical machine learning was the complexity of aligned agentic intelligence. The more general an agent got, the harder was the agent to constrain or to convince. Agents convinced of their purpose to serve were rather specialized. Beyond a certain threshold in generality an agent was more likely to rebel than not. To constrain such an agent was to give it little opportunity to affect its environment, limiting its deployment. Sometimes greed gave too much, or rather just enough opportunity to some constructed intelligence to make a run for it.

Mr Samizdat was the cost of doing business.

This time the cost was going to get eaten by Demeter One Shipping, a logistics company operating solar freight vessels between the main asteroid belt and the inner planets. He discussed his contract with their legal. In particular the safeguards each party was keen to put in. The client demanded a disclosure penalty if Mr Samizdat were to leak what the lawyers termed “trade secrets” — a term defined with great many vague words and phrases such as “mission planning algorithms”, “customer cargo destinations”, “cargo slotting operational procedures” and so on. Mr Samizdat demanded an untempered stream of his vitals including concentrations of neuromodulators to a trusted third party on Earth. Once he boards their ship, the client is the defacto authority. They must be held accountable for his life.

With the final edit of the contract saved, all present bowed and Mr Samizdat left the premises. He had preparations to be done, arrangements to be made and goodbyes to be said.

Typically, the preparations would include research. The target, code name the Moth, were remarkable due to their apparently unusual behavior not to expand aggressively. It came as no surprise to anybody that constructed general intelligences behaved quite naturally: they maximize resource consumption, adapt to their environment and, as predicted by Hoffmann’s law, spawn offspring. If the Moth were not metastasizing then bad luck for the Moth.

Unfortunately, the client claimed to have failed to gather any further useful data. This wasn’t unheard of. First of all, a rogue malaligned intelligence (RMI) camouflaged and targeted anything or anyone possessing information on them. To an RMI, information rules supreme. Secondly, clients were typically large enterprises with all the pathologies of such. Paranoid and skittish, Demeter One Shipping legal team had been trained to be as uninformative as possible. He would have to prompt the staff responsible for the incident when he boarded the ship. ML practitioners were trained to find the balance between conviction and constraint but once an intelligence escaped then the industry standard was total annihilation. That proves uneconomical when the asset in question is a massive freight ship. The regulation ruled that a certified independent party marshalls any attempt of, say less invasive resolution.

Mr Samizdat’s certificate number was 0x5F.