"I'm surprised you want to u..."

https://arbital.com/p/75

by Kenzi Amodei Jun 16 2015


I'm surprised you want to use the word "advanced" to for this concept; implies to me this is the main kind of high-level safety missing from standard "safety" models? I guess the list of bullet points does cover a whole lot of scenarios. It does make it sound sexy, and not like something you'd want to ignore. Obvious alternative usage for the word advanced relative to safety would be for "actually" safe (over just claimed safe). Maybe that has other words available to it like provably.

I have the intuition that many proposals fail against advanced agents; I don't see intuitively that it's the "advanced" that's the main problem (that would imply they would work as long as the agent didn't become advanced, I think? What does that look like? And is this like Asimov's three laws or tool AI or what?)

Are there any interesting intuition pumps that fall out of omniscience/omnipotence that don't fall easily out of the "advanced" concept?