{
localUrl: '../page/Vingean_reflection.html',
arbitalUrl: 'https://arbital.com/p/Vingean_reflection',
rawJsonUrl: '../raw/1c1.json',
likeableId: '319',
likeableType: 'page',
myLikeValue: '0',
likeCount: '2',
dislikeCount: '0',
likeScore: '2',
individualLikes: [
'StevenDee',
'AlexRay'
],
pageId: 'Vingean_reflection',
edit: '10',
editSummary: '',
prevEdit: '9',
currentEdit: '10',
wasPublished: 'true',
type: 'wiki',
title: 'Vingean reflection',
clickbait: 'The problem of thinking about your future self when it's smarter than you.',
textLength: '3262',
alias: 'Vingean_reflection',
externalUrl: '',
sortChildrenBy: 'likes',
hasVote: 'false',
voteType: '',
votesAnonymous: 'false',
editCreatorId: 'EliezerYudkowsky',
editCreatedAt: '2016-06-21 01:55:03',
pageCreatorId: 'EliezerYudkowsky',
pageCreatedAt: '2015-12-18 23:28:03',
seeDomainId: '0',
editDomainId: 'ConnorFlexman',
submitToDomainId: '0',
isAutosave: 'false',
isSnapshot: 'false',
isLiveEdit: 'true',
isMinorEdit: 'false',
indirectTeacher: 'false',
todoCount: '0',
isEditorComment: 'false',
isApprovedComment: 'true',
isResolved: 'false',
snapshotText: '',
anchorContext: '',
anchorText: '',
anchorOffset: '0',
mergedInto: '',
isDeleted: 'false',
viewCount: '364',
text: '[summary: [1c0 Vinge's Principle] implies that when an agent is designing another agent (or modifying its own code), it needs to approve the other agent's design without knowing the other agent's exact future actions. [1c1] is reasoning about cognitive systems, especially cognitive systems very similar to yourself (including your actual self), under the constraint that you can't predict the exact future outputs.\n\nIn [-1mq], this appears as the rule that we should talk about our successor's actions only inside of quantifiers.\n\n"Vingean reflection" may be a much more general issue in the design of advanced cognitive systems than it might appear at first glance. An agent reasoning about the consequences of *its current code*, or considering what will happen if it *spends another minute thinking,* can be viewed as doing Vingean reflection. Vingean reflection can also be seen as the study of how a given agent *wants* thinking to occur in cognitive computations, which may be importantly different from how the agent *currently* thinks.]\n\n[1c0 Vinge's Principle] implies that when an agent is designing another agent (or modifying its own code), it needs to approve the other agent's design without knowing the other agent's exact future actions.\n\nDeep Blue's programmers decided to run Deep Blue, *without* knowing Deep Blue's exact moves against Kasparov or how Kasparov would reply to each move, and without being able to visualize the exact real-outcome instead. Instead, by reasoning about the way Deep Blue was searching through game trees, they arrived at a well-justified but abstract belief that Deep Blue was 'trying to win' (rather than trying to lose) and reasoning effectively to that end.\n\n[1c1] is reasoning about cognitive systems, especially cognitive systems very similar to yourself (including your actual self), under the constraint that you can't predict the exact future outputs. We need to make predictions about the consequence of operating an agent in an environment via reasoning on some more abstract level, somehow.\n\nIn [-1mq], this appears as the rule that we should talk about our successor's actions only inside of quantifiers.\n\n"Vingean reflection" may be a much more general issue in the design of advanced cognitive systems than it might appear at first glance. An agent reasoning about the consequences of *its current code*, or considering what will happen if it *spends another minute thinking,* can be viewed as doing Vingean reflection. A reflective, self-modeling chess-player would not choose to spend another minute thinking, if it thought that its further thoughts would be trying to lose rather than win the game - but it can't predict its own exact thoughts in advance.\n\nVingean reflection can also be seen as the study of how a given agent *wants* thinking to occur in cognitive computations, which may be importantly different from how the agent *currently* thinks. If these two coincide, we say the agent is [1fx reflectively stable].\n\n[1mq] is presently the main line of research trying to slowly get started on formalizing Vingean reflection and reflective stability.\n\nFurther reading:\n\n- http://intelligence.org/files/VingeanReflection.pdf\n- http://intelligence.org/files/TilingAgentsDraft.pdf',
metaText: '',
isTextLoaded: 'true',
isSubscribedToDiscussion: 'false',
isSubscribedToUser: 'false',
isSubscribedAsMaintainer: 'false',
discussionSubscriberCount: '2',
maintainerCount: '1',
userSubscriberCount: '0',
lastVisit: '2016-02-21 16:32:39',
hasDraft: 'false',
votes: [],
voteSummary: [
'0',
'0',
'0',
'0',
'0',
'0',
'0',
'0',
'0',
'0'
],
muVoteSummary: '0',
voteScaling: '0',
currentUserVote: '-2',
voteCount: '0',
lockedVoteType: '',
maxEditEver: '0',
redLinkCount: '0',
lockedBy: '',
lockedUntil: '',
nextPageId: '',
prevPageId: '',
usedAsMastery: 'false',
proposalEditNum: '0',
permissions: {
edit: {
has: 'false',
reason: 'You don't have domain permission to edit this page'
},
proposeEdit: {
has: 'true',
reason: ''
},
delete: {
has: 'false',
reason: 'You don't have domain permission to delete this page'
},
comment: {
has: 'false',
reason: 'You can't comment in this domain because you are not a member'
},
proposeComment: {
has: 'true',
reason: ''
}
},
summaries: {},
creatorIds: [
'EliezerYudkowsky'
],
childIds: [
'Vinge_principle',
'reflective_stability',
'tiling_agents',
'reflective_consistency'
],
parentIds: [
'ai_alignment'
],
commentIds: [],
questionIds: [],
tagIds: [
'stub_meta_tag',
'Vingean_uncertainty'
],
relatedIds: [],
markIds: [],
explanations: [],
learnMore: [],
requirements: [],
subjects: [],
lenses: [],
lensParentId: '',
pathPages: [],
learnMoreTaughtMap: {},
learnMoreCoveredMap: {},
learnMoreRequiredMap: {},
editHistory: {},
domainSubmissions: {},
answers: [],
answerCount: '0',
commentCount: '0',
newCommentCount: '0',
linkedMarkCount: '0',
changeLogs: [
{
likeableId: '0',
likeableType: 'changeLog',
myLikeValue: '0',
likeCount: '0',
dislikeCount: '0',
likeScore: '0',
individualLikes: [],
id: '14237',
pageId: 'Vingean_reflection',
userId: 'EliezerYudkowsky',
edit: '10',
type: 'newEdit',
createdAt: '2016-06-21 01:55:03',
auxPageId: '',
oldSettingsValue: '',
newSettingsValue: ''
},
{
likeableId: '0',
likeableType: 'changeLog',
myLikeValue: '0',
likeCount: '0',
dislikeCount: '0',
likeScore: '0',
individualLikes: [],
id: '14236',
pageId: 'Vingean_reflection',
userId: 'EliezerYudkowsky',
edit: '9',
type: 'newEdit',
createdAt: '2016-06-21 01:53:41',
auxPageId: '',
oldSettingsValue: '',
newSettingsValue: ''
},
{
likeableId: '0',
likeableType: 'changeLog',
myLikeValue: '0',
likeCount: '0',
dislikeCount: '0',
likeScore: '0',
individualLikes: [],
id: '8871',
pageId: 'Vingean_reflection',
userId: 'EliezerYudkowsky',
edit: '8',
type: 'newChild',
createdAt: '2016-03-22 01:18:54',
auxPageId: 'reflective_consistency',
oldSettingsValue: '',
newSettingsValue: ''
},
{
likeableId: '0',
likeableType: 'changeLog',
myLikeValue: '0',
likeCount: '0',
dislikeCount: '0',
likeScore: '0',
individualLikes: [],
id: '5421',
pageId: 'Vingean_reflection',
userId: 'EliezerYudkowsky',
edit: '8',
type: 'newChild',
createdAt: '2016-01-17 06:02:32',
auxPageId: 'tiling_agents',
oldSettingsValue: '',
newSettingsValue: ''
},
{
likeableId: '0',
likeableType: 'changeLog',
myLikeValue: '0',
likeCount: '0',
dislikeCount: '0',
likeScore: '0',
individualLikes: [],
id: '5348',
pageId: 'Vingean_reflection',
userId: 'EliezerYudkowsky',
edit: '8',
type: 'newEdit',
createdAt: '2016-01-16 06:50:58',
auxPageId: '',
oldSettingsValue: '',
newSettingsValue: ''
},
{
likeableId: '0',
likeableType: 'changeLog',
myLikeValue: '0',
likeCount: '0',
dislikeCount: '0',
likeScore: '0',
individualLikes: [],
id: '5347',
pageId: 'Vingean_reflection',
userId: 'EliezerYudkowsky',
edit: '7',
type: 'newEdit',
createdAt: '2016-01-16 06:50:42',
auxPageId: '',
oldSettingsValue: '',
newSettingsValue: ''
},
{
likeableId: '0',
likeableType: 'changeLog',
myLikeValue: '0',
likeCount: '0',
dislikeCount: '0',
likeScore: '0',
individualLikes: [],
id: '4514',
pageId: 'Vingean_reflection',
userId: 'EliezerYudkowsky',
edit: '6',
type: 'newChild',
createdAt: '2015-12-28 20:08:45',
auxPageId: 'reflective_stability',
oldSettingsValue: '',
newSettingsValue: ''
},
{
likeableId: '0',
likeableType: 'changeLog',
myLikeValue: '0',
likeCount: '0',
dislikeCount: '0',
likeScore: '0',
individualLikes: [],
id: '4224',
pageId: 'Vingean_reflection',
userId: 'EliezerYudkowsky',
edit: '0',
type: 'newAlias',
createdAt: '2015-12-19 01:50:36',
auxPageId: '',
oldSettingsValue: '',
newSettingsValue: ''
},
{
likeableId: '0',
likeableType: 'changeLog',
myLikeValue: '0',
likeCount: '0',
dislikeCount: '0',
likeScore: '0',
individualLikes: [],
id: '4212',
pageId: 'Vingean_reflection',
userId: 'EliezerYudkowsky',
edit: '6',
type: 'newEdit',
createdAt: '2015-12-18 23:39:31',
auxPageId: '',
oldSettingsValue: '',
newSettingsValue: ''
},
{
likeableId: '0',
likeableType: 'changeLog',
myLikeValue: '0',
likeCount: '0',
dislikeCount: '0',
likeScore: '0',
individualLikes: [],
id: '4211',
pageId: 'Vingean_reflection',
userId: 'EliezerYudkowsky',
edit: '5',
type: 'newEdit',
createdAt: '2015-12-18 23:39:09',
auxPageId: '',
oldSettingsValue: '',
newSettingsValue: ''
},
{
likeableId: '0',
likeableType: 'changeLog',
myLikeValue: '0',
likeCount: '0',
dislikeCount: '0',
likeScore: '0',
individualLikes: [],
id: '4210',
pageId: 'Vingean_reflection',
userId: 'EliezerYudkowsky',
edit: '4',
type: 'newEdit',
createdAt: '2015-12-18 23:38:08',
auxPageId: '',
oldSettingsValue: '',
newSettingsValue: ''
},
{
likeableId: '0',
likeableType: 'changeLog',
myLikeValue: '0',
likeCount: '0',
dislikeCount: '0',
likeScore: '0',
individualLikes: [],
id: '4209',
pageId: 'Vingean_reflection',
userId: 'EliezerYudkowsky',
edit: '3',
type: 'newTag',
createdAt: '2015-12-18 23:35:35',
auxPageId: 'stub_meta_tag',
oldSettingsValue: '',
newSettingsValue: ''
},
{
likeableId: '0',
likeableType: 'changeLog',
myLikeValue: '0',
likeCount: '0',
dislikeCount: '0',
likeScore: '0',
individualLikes: [],
id: '4202',
pageId: 'Vingean_reflection',
userId: 'EliezerYudkowsky',
edit: '3',
type: 'newEdit',
createdAt: '2015-12-18 23:29:05',
auxPageId: '',
oldSettingsValue: '',
newSettingsValue: ''
},
{
likeableId: '0',
likeableType: 'changeLog',
myLikeValue: '0',
likeCount: '0',
dislikeCount: '0',
likeScore: '0',
individualLikes: [],
id: '4201',
pageId: 'Vingean_reflection',
userId: 'EliezerYudkowsky',
edit: '2',
type: 'newEdit',
createdAt: '2015-12-18 23:28:47',
auxPageId: '',
oldSettingsValue: '',
newSettingsValue: ''
},
{
likeableId: '0',
likeableType: 'changeLog',
myLikeValue: '0',
likeCount: '0',
dislikeCount: '0',
likeScore: '0',
individualLikes: [],
id: '4198',
pageId: 'Vingean_reflection',
userId: 'EliezerYudkowsky',
edit: '1',
type: 'newChild',
createdAt: '2015-12-18 23:28:13',
auxPageId: 'Vinge_principle',
oldSettingsValue: '',
newSettingsValue: ''
},
{
likeableId: '0',
likeableType: 'changeLog',
myLikeValue: '0',
likeCount: '0',
dislikeCount: '0',
likeScore: '0',
individualLikes: [],
id: '4197',
pageId: 'Vingean_reflection',
userId: 'EliezerYudkowsky',
edit: '1',
type: 'newEdit',
createdAt: '2015-12-18 23:28:03',
auxPageId: '',
oldSettingsValue: '',
newSettingsValue: ''
},
{
likeableId: '0',
likeableType: 'changeLog',
myLikeValue: '0',
likeCount: '0',
dislikeCount: '0',
likeScore: '0',
individualLikes: [],
id: '4160',
pageId: 'Vingean_reflection',
userId: 'EliezerYudkowsky',
edit: '0',
type: 'newTag',
createdAt: '2015-12-18 21:09:44',
auxPageId: 'Vingean_uncertainty',
oldSettingsValue: '',
newSettingsValue: ''
},
{
likeableId: '0',
likeableType: 'changeLog',
myLikeValue: '0',
likeCount: '0',
dislikeCount: '0',
likeScore: '0',
individualLikes: [],
id: '4158',
pageId: 'Vingean_reflection',
userId: 'EliezerYudkowsky',
edit: '0',
type: 'newParent',
createdAt: '2015-12-18 21:09:39',
auxPageId: 'ai_alignment',
oldSettingsValue: '',
newSettingsValue: ''
},
{
likeableId: '0',
likeableType: 'changeLog',
myLikeValue: '0',
likeCount: '0',
dislikeCount: '0',
likeScore: '0',
individualLikes: [],
id: '4156',
pageId: 'Vingean_reflection',
userId: 'EliezerYudkowsky',
edit: '0',
type: 'deleteParent',
createdAt: '2015-12-18 21:09:22',
auxPageId: 'Vingean_uncertainty',
oldSettingsValue: '',
newSettingsValue: ''
},
{
likeableId: '0',
likeableType: 'changeLog',
myLikeValue: '0',
likeCount: '0',
dislikeCount: '0',
likeScore: '0',
individualLikes: [],
id: '4154',
pageId: 'Vingean_reflection',
userId: 'EliezerYudkowsky',
edit: '0',
type: 'newParent',
createdAt: '2015-12-18 21:09:06',
auxPageId: 'Vingean_uncertainty',
oldSettingsValue: '',
newSettingsValue: ''
}
],
feedSubmissions: [],
searchStrings: {},
hasChildren: 'true',
hasParents: 'true',
redAliases: {},
improvementTagIds: [],
nonMetaTagIds: [],
todos: [],
slowDownMap: 'null',
speedUpMap: 'null',
arcPageIds: 'null',
contentRequests: {}
}