Time for a bit of weird speculation. I haven't heard this idea before (at least, in the real world) -- but knowing Silicon Valley, there are surely folks there who are thinking about it.

In my spare time, I'm the owner of an eensy-weensy garage startup. It's never made any money -- I have ambitions of it someday getting to the point of paying me an adequate salary, but I made my peace long ago with the idea that I'll never get rich off of it.

I created it because it's a useful tool that I want to exist in the world. But that leads to a reasonable concern: what if I were to die? Many small companies founder at the loss of their founder: they change hands, enshittify, or simply cease to exist. That genuinely worries me.

#LLMs

1/

in reply to Justin du Coeur

And a bizarre thought experiment occurred to me last night: given the rise of agentic systems, is it now possible for a company to become entirely autonomous? That is, could the company be transformed to be entirely run, managed and effectively "owned" by an immortal swarm of agents?

It's not quite as insane as it sounds, given the current state of #AI. We've already got [first-draft tooling to replace a C-suite with AI](github.com/SenteLabsAI/OpenExe…). The coding and design tools are getting roughly good enough to sustain a small company indefinitely -- not to do anything radically creative, but to keep the lights on and the security locked down properly. I think all the individual functions can more or less be done by AI today, if almost certainly imperfectly.

#LLMs

2/

in reply to Justin du Coeur

*Legally*, I suspect the situation is dicier: there are assumptions laced throughout legal processes that there are people in the loop.

But given the common trope of companies within companies, with no people you can readily track down, and the notion of legal personhood for corporations, I'm not actually certain that it's impossible to create a corporate structure that simply has no people involved, especially if it was a non-profit with no beneficial owners.

(The capitalist world tends to be focused on "Who is making the money?", but in this scenario that might literally be "nobody, aside from keeping the servers running and the tokens paid for".)

#AI #LLMs

3/

in reply to Justin du Coeur

*Ethically*, of course, it's a ridiculous minefield, and almost certainly an awful idea.

There are a variety of problems, from the absence of anyone who could be held legally liable for criminal responsibility if the company did something truly egregious, to the plain reality that we simply don't know how a bunch of LLMs would drift off-alignment over the span of years. The whole thing smells like a fine way to build a paperclip maximizer.

But I have a nasty feeling that the whole thing is *technically* possible (indeed, I suspect I could do it at this point) -- and I generally find that merely being a bad (even ridiculous) idea is no impediment to the tech bros. So it may be worth pondering the idea (and what legal protections need to be placed around it) before somebody does it for real.

And if nothing else, it's a potential background tidbit for science fiction...

#AI #LLMs

4/4

This website uses cookies. If you continue browsing this website, you agree to the usage of cookies.