Regulating AI Before We Understand AGI: Why California Backed Away From SB 1047
Why did California step back from SB 1047 while trying to regulate AI before society agreed on what AGI means?
Build a working map of accountability for the case involving OpenAI; the next step is to separate a technical restriction, enforceability, and the responsibility of the company, platform, and user.
What to watch for
Key takeaways
The “ToTheMoon is your guide in the world of modern technologies!” topic becomes clearer once this point is included: the SB 1047 veto is not a fight between supporters and opponents of safety but between two fears: missing a dangerous model and smothering new approaches with premature rules.
The “How lawmakers in different countries regulate AI” topic becomes clearer once this point is included: the reason for the conflict is understandable: if regulation is based only on compute scale and training cost, it will hit companies that have not created a dangerous system but have crossed a formal threshold. For a startup, such a requirement can become a ban on experimentation; for a giant, it is simply another legal expense.
The decision in “Even the "godmother" of AI doesn't know what AGI is” depends on one criterion: the researcher hailed at a San Francisco conference as the "progenitor" and even the "god" of AI admitted she herself does not know what people mean when they say "AGI": everyone loads the word with something different — a shaky foundation for a law.
The discussion of “China and investment in state-space models: the next step in AI?” yields a practical test: more and more researchers doubt that simply scaling transformers will get to strong AI by itself — hence China's growing interest in alternative architectures and state-space models: a bet not on more of the same, but on something different.
The boundary of the “AI vs AGI: nobody understands what it is (except Sam Altman)” case is defined by this point: even market participants have not agreed on what counts as AGI: some mean a system able to do most human intellectual tasks, others a model that delivers economic results without constant supervision.
The “Andrej Karpathy explains what AI is as simply as possible” topic becomes clearer once this point is included: Karpathy — who worked at Tesla and on the early GPTs at OpenAI — explains models through the archiver analogy: a huge volume of data gets "compressed" as in WinRAR, and the interesting properties emerge in generalization — like a model trained on Spanish that starts speaking English from a few examples.
The “What do you think of AI legislation? Do politicians help or get in the way?” scene leads to a working conclusion: AI regulation is no longer a single law: it is simultaneously a fight over safety, investment, architectures, and the right to decide which company even gets to try the next approach.
The “Dubbing TV series with AI” issue should be assessed with one constraint in mind: an AI-dubbed series, a retouched photo, and a fully synthetic video can all receive the same "created with AI" label although the model's role in each is completely different — labeling without gradations tells the viewer less than it seems.
The boundary of the “AI in healthcare. Can the insurance companies be beaten?” case is defined by this point: a model can help sort documents or explain a term, but it must not quietly replace the physician, the insurer, and the rules of access to treatment: any automation here runs straight into the question of who answers if the recommendation is wrong and a person relied on it.
The practical meaning of “Will AI startups still be able to raise funding?” is that state restrictions are joined by the market's own: OpenAI can require investors not to back direct competitors, locking in its advantage not only through model quality but through access to capital.
What this episode is about
The veto of California's SB 1047 was not a dispute between supporters and opponents of safety. It was a conflict between two fears: missing a dangerous model and suffocating new approaches with premature rules. At the same time, the market is looking beyond transformers, introducing content labels, and tightening the relationship between OpenAI and investors.
California attempted to create one of the strictest liability regimes for developers of large models, but SB 1047 was vetoed. The reason for the conflict is understandable: if regulation is based only on compute scale and training cost, it will hit companies that have not created a dangerous system but have crossed a formal threshold. For a startup, such a requirement can become a ban on experimentation; for a giant, it is simply another legal expense.
The problem is that even market participants have not agreed on what counts as AGI. Some describe a system capable of performing most human intellectual tasks; others mean a model that produces economic results without constant supervision.
At the same time, more researchers doubt that scaling transformers alone will automatically reach that point. Interest in alternative architectures and state-space models is therefore growing in China.
Regulators are meanwhile choosing more concrete targets: deepfakes, watermarks, and mandatory removal of false content after a complaint. The harm is easier to demonstrate here, but the solution remains imperfect. A dubbed television series, a retouched photograph, and a fully synthetic video can all receive the same label even though AI played completely different roles in each.
In healthcare, the cost of error is higher still. A model can help analyze documents or explain a term, but it should not silently replace the physician, the insurer, and the rules governing access to treatment. Any automation in this area immediately becomes a question of responsibility: who is accountable if the recommendation is wrong and a person acts on it?
Government restrictions are joined by restrictions from the market itself. OpenAI can ask investors not to back direct competitors, preserving its advantage not only through model quality but also through access to capital.
AI regulation is therefore no longer a single law. It is simultaneously a fight over safety, investment, architectures, and the right to determine which company will even be allowed to try the next approach.
AI regulation can no longer be reduced to one law: it simultaneously shapes safety, investment, acceptable architectures, and a company’s ability to test the next approach.
Episode transcript
The episode is in Russian; below is an English reading guide to the transcript (the full EN transcript is a machine translation). Voice matching applied to 96 segments: 52 identified, 6 mixed, 22 probable, and 16 unresolved.
Read transcript on a separate page
Loading…