Standard agent properties

https://arbital.com/p/standard_agent

by Eliezer Yudkowsky Jun 9 2015 updated Dec 18 2015

What's a Standard Agent, and what can it do?


Boundedly rational agents

For the arguments that sufficiently intelligent agents will appear to us as boundedly rational agents in some sense, see:

Economic agents

Naturalistic agents


Comments

Kenzi Amodei

I didn't know that about Bayesian inference-ish updating baking in an Occam-ish prior. Does it need to be complexity penalizing, or would any consistent prior-choosing rule work? I assume the former from the phrasing.

Why is that? "does not much constrain the end results" could just mean that unless we assume the agent is Occam ish, then we can't tell from its posteriors whether it did Bayesian inference or something else. But I don't see why that couldn't be true of some non-Occam-ish prior picking rule, as long as we knew what that was.

I think this definition includes agents that only cared about their sensory inputs, since sensory inputs are a subset of states of the world.

This makes me think that the definition of economic agent that I googled isn't what was meant, since this one seems to be primarily making a claim about efficiency, rather than about impacting markets ("an agent who is part of the economy"). Something more like homo economicus?

Naturalistic agents seems to have been primarily a claim about the situation that agent finds itself in, rather than a claim about that agents' models (eg, a cartesian dualist which was in fact embedded in a universe made of atoms and was itself made of atoms, would still be a "naturalistic agent", I think)

The last point reminds me of Dawkins style extended phenotypes; not sure how analogous/comparable that concept is. I guess it makes me want to go back and figure out if we defined what "an agent" was. So like does a beehive count as "an agent" (I believe that conditioned on it being an agent at all, it would be a naturalized agent)?

…does Arbital have search functionality right now? Maybe not :-/