{
localUrl: '../page/flee_from_surprise.html',
arbitalUrl: 'https://arbital.com/p/flee_from_surprise',
rawJsonUrl: '../raw/552.json',
likeableId: '2958',
likeableType: 'page',
myLikeValue: '0',
likeCount: '6',
dislikeCount: '0',
likeScore: '6',
individualLikes: [
'EricBruylant',
'NateSoares',
'MalcolmMcCrimmon',
'SzymonSlawinski',
'MahmudulNuman',
'NoahLuxton'
],
pageId: 'flee_from_surprise',
edit: '5',
editSummary: '',
prevEdit: '4',
currentEdit: '5',
wasPublished: 'true',
type: 'wiki',
title: 'Shift towards the hypothesis of least surprise',
clickbait: 'When you see new evidence, ask: which hypothesis is *least surprised?*',
textLength: '5729',
alias: 'flee_from_surprise',
externalUrl: '',
sortChildrenBy: 'likes',
hasVote: 'false',
voteType: '',
votesAnonymous: 'false',
editCreatorId: 'DonyChristie',
editCreatedAt: '2016-11-03 02:10:16',
pageCreatorId: 'NateSoares',
pageCreatedAt: '2016-07-06 06:15:23',
seeDomainId: '0',
editDomainId: 'AlexeiAndreev',
submitToDomainId: '0',
isAutosave: 'false',
isSnapshot: 'false',
isLiveEdit: 'true',
isMinorEdit: 'false',
indirectTeacher: 'false',
todoCount: '0',
isEditorComment: 'false',
isApprovedComment: 'true',
isResolved: 'false',
snapshotText: '',
anchorContext: '',
anchorText: '',
anchorOffset: '0',
mergedInto: '',
isDeleted: 'false',
viewCount: '5935',
text: 'The [1zh log-odds form of Bayes' rule] says that strength of belief and strength of evidence can both be measured in [3y2 bits]. These evidence-bits can also be used to measure a quantity called "Bayesian surprise", which yields [fixme: One final, if this is the last thing in the path]yet another intuition for understanding Bayes' rule.\n\nRoughly speaking, we can measure how surprised a hypothesis $H_i$ was by the evidence $e$ by measuring how much probability it put on $e.$ If $H_i$ put 100% of its probability mass on $e$, then $e$ is completely unsurprising (to $H_i$). If $H_i$ put 0% of its probability mass on $e$, then $e$ is _as surprising as possible._ Any measure of $\\mathbb P(e \\mid H_i),$ the probability $H_i$ assigned to $e$, that obeys this property, is worthy of the label "surprise." Bayesian surprise is $-\\!\\log(\\mathbb P(e \\mid H_i)),$ which is a quantity that obeys these intuitive constraints and has some other interesting features.\n\nConsider again the [-54v] problem. Consider the hypotheses $H$ and $\\lnot H$, which say "the oyster will contain a pearl" and "no it won't", respectively. To keep the numbers easy, let's say we draw an oyster from a third bay, where $\\frac{1}{8}$ of pearl-carrying oysters are blue and $\\frac{1}{4}$ of empty oysters are blue.\n\nImagine what happens when the oyster is blue. $H$ predicted blueness with $\\frac{1}{8}$ of its probability mass, while $\\lnot H$ predicted blueness with $\\frac{1}{4}$ of its probability mass. Thus, $\\lnot H$ did better than $H,$ and goes up in probability. Previously, we've been combining both $\\mathbb P(e \\mid H)$ and $\\mathbb P(e \\mid \\lnot H)$ into unified likelihood ratios, like $\\left(\\frac{1}{8} : \\frac{1}{4}\\right)$ $=$ $(1 : 2),$ which says that the 'blue' observation carries 1 bit of evidence $H.$ However, we can also take the logs first, and combine second.\n\nBecause $H$ assigned only an eighth of its probability mass to the 'blue' observation, and because [1y6 Bayesian update works by eliminating incorrect probability mass], we have to adjust our belief in $H$ by $\\log_2\\left(\\frac{1}{8}\\right) = -3$ bits away from $H.$ (Each negative bit means "throw away half of $H$'s probability mass," and we have to do that 3 times in order to remove the probability that $H$ failed to assign to $e$.)\n\nSimilarly, because $\\lnot H$ assigned only a quarter of its probability mass to the 'blue' observation, we have to adjust our belief in $H$ by $\\log_2\\left(\\frac{1}{4}\\right) = -2$ bits away from $\\lnot H.$\n\nThus, when the 'blue' observation comes in, we move our belief (measured in bits) 3 notches away from $H$ and then two notches back towards $H.$ On net, our belief shifts 1 notch away from $H$.\n\n\n\n_$H$ assigned 1/8th of its probability mass to blueness, so it emits $-\\!\\log_2\\left(\\frac{1}{8}\\right)=3$ bits of surprise pushing away from $H$. $\\lnot H$ assigned 1/4th of its probability mass to blueness, so it emits $-\\!\\log_2\\left(\\frac{1}{4}\\right)=2$ bits of surprise pushing away from $\\lnot H$ (and towards $H$). Thus, belief in $H$ moves 1 bit towards $\\lnot H$, on net._\n\nIf instead $H$ predicted blue with probability 4% (penalty $\\log_2(0.04) \\approx -4.64$) and $\\lnot H$ predicted blue with probability 8% (penalty $\\log_2(0.08) \\approx -3.64$), then we would have shifted a bit over 4.6 notches towards $\\lnot H$ and a bit over 3.6 notches back towards $H,$ but we would have shifted the same number of notches _on net._ This is why it's only the _relative_ difference between the number of bits docked from $H$ and the number of bits docked from $\\lnot H$ that matters.\n\nIn general, given an observation $e$ and a hypothesis $H,$ the number of bits we need to dock from our belief in $H$ is $\\log_2(\\mathbb P(e \\mid H)),$ that is, the log of the probability that $H$ assigned to $e.$ This quantity is never positive, because the logarithm of $x$ for $0 \\le x \\le 1$ is in the range $[-\\infty, 0]$. If we negate it, we get a non-negative quantity that relates $H$ to $e$, which is 0 when $H$ was certain that $e$ was going to happen, and which is infinite when $H$ was certain that $e$ wasn't going to happen, and which is measured in the same units as evidence and belief. Thus, this quantity is often called "surprise," and intuitively, it measures how surprised the hypothesis $H$ was by $e$ (in bits).\n\nThere is some correlation between Bayesian surprise and the times when a human would feel surprised (at seeing something that they thought was unlikely), but, of course, the human emotion is quite different. (A human can feel surprised for other reasons than "my hypotheses failed to predict the data," and humans are also great at ignoring evidence instead of feeling surprised.)\n\nGiven this definition of Bayesian surprise, we can view Bayes' rule as saying that _surprise repels belief._ When you make an observation $e,$ each hypothesis emits repulsive "surprise" signals, which shift your hypothesis. Referring again to the image above, when $H$ predicts the observation you made with $\\frac{1}{8}$ of its probability mass, and $\\lnot H$ predicts it with $\\frac{1}{4}$ of its probability mass, we can imagine $H$ emitting a surprise signal with a strength of 3 bits away from $H$ and $\\lnot H$ emitting a surprise signal with a strength of 2 bits away from $\\lnot H$. Both those signals push the belief in $H$ in different directions, and it ends up 1 bit closer to $\\lnot H$ (which emitted the weaker surprise signal).\n\nIn other words, whenever you find yourself feeling surprised by something you saw, think of the _least surprising explanation_ for that evidence — and then award that hypothesis a few bits of belief.\n',
metaText: '',
isTextLoaded: 'true',
isSubscribedToDiscussion: 'false',
isSubscribedToUser: 'false',
isSubscribedAsMaintainer: 'false',
discussionSubscriberCount: '1',
maintainerCount: '1',
userSubscriberCount: '0',
lastVisit: '',
hasDraft: 'false',
votes: [],
voteSummary: [
'0',
'0',
'0',
'0',
'0',
'0',
'0',
'0',
'0',
'0'
],
muVoteSummary: '0',
voteScaling: '0',
currentUserVote: '-2',
voteCount: '0',
lockedVoteType: '',
maxEditEver: '0',
redLinkCount: '0',
lockedBy: '',
lockedUntil: '',
nextPageId: '',
prevPageId: '',
usedAsMastery: 'true',
proposalEditNum: '0',
permissions: {
edit: {
has: 'false',
reason: 'You don't have domain permission to edit this page'
},
proposeEdit: {
has: 'true',
reason: ''
},
delete: {
has: 'false',
reason: 'You don't have domain permission to delete this page'
},
comment: {
has: 'false',
reason: 'You can't comment in this domain because you are not a member'
},
proposeComment: {
has: 'true',
reason: ''
}
},
summaries: {
Summary: 'The [1zh log-odds from of Bayes' rule] says that strength of belief and strength of evidence can both be measured in [3y2 bits]. These evidence-bits can also be used to measure a quantity called "Bayesian surprise", which yields [fixme: One final, if this is the last thing in the path]yet another intuition for understanding Bayes' rule.'
},
creatorIds: [
'NateSoares',
'AlexeiAndreev',
'DonyChristie'
],
childIds: [],
parentIds: [
'bayes_rule'
],
commentIds: [
'96l'
],
questionIds: [],
tagIds: [
'b_class_meta_tag'
],
relatedIds: [],
markIds: [],
explanations: [
{
id: '5180',
parentId: 'flee_from_surprise',
childId: 'flee_from_surprise',
type: 'subject',
creatorId: 'NateSoares',
createdAt: '2016-07-10 23:35:16',
level: '2',
isStrong: 'true',
everPublished: 'true'
}
],
learnMore: [],
requirements: [
{
id: '5165',
parentId: 'bayes_log_odds',
childId: 'flee_from_surprise',
type: 'requirement',
creatorId: 'NateSoares',
createdAt: '2016-07-10 22:17:15',
level: '2',
isStrong: 'true',
everPublished: 'true'
},
{
id: '5301',
parentId: 'bayes_rule',
childId: 'flee_from_surprise',
type: 'requirement',
creatorId: 'AlexeiAndreev',
createdAt: '2016-07-16 16:12:58',
level: '2',
isStrong: 'true',
everPublished: 'true'
},
{
id: '5645',
parentId: 'blue_oysters',
childId: 'flee_from_surprise',
type: 'requirement',
creatorId: 'AlexeiAndreev',
createdAt: '2016-07-26 17:15:08',
level: '2',
isStrong: 'false',
everPublished: 'true'
},
{
id: '5809',
parentId: 'bayes_rule_elimination',
childId: 'flee_from_surprise',
type: 'requirement',
creatorId: 'AlexeiAndreev',
createdAt: '2016-08-02 00:56:44',
level: '2',
isStrong: 'false',
everPublished: 'true'
}
],
subjects: [
{
id: '5180',
parentId: 'flee_from_surprise',
childId: 'flee_from_surprise',
type: 'subject',
creatorId: 'NateSoares',
createdAt: '2016-07-10 23:35:16',
level: '2',
isStrong: 'true',
everPublished: 'true'
},
{
id: '5644',
parentId: 'bayes_rule',
childId: 'flee_from_surprise',
type: 'subject',
creatorId: 'AlexeiAndreev',
createdAt: '2016-07-26 17:14:38',
level: '2',
isStrong: 'false',
everPublished: 'true'
},
{
id: '5810',
parentId: 'bayes_log_odds',
childId: 'flee_from_surprise',
type: 'subject',
creatorId: 'AlexeiAndreev',
createdAt: '2016-08-02 00:58:13',
level: '1',
isStrong: 'false',
everPublished: 'true'
}
],
lenses: [],
lensParentId: '',
pathPages: [],
learnMoreTaughtMap: {},
learnMoreCoveredMap: {
'1lz': [
'1xr',
'1yc',
'1zh',
'1zm',
'220',
'56j',
'6cj'
]
},
learnMoreRequiredMap: {
'552': [
'25y'
]
},
editHistory: {},
domainSubmissions: {},
answers: [],
answerCount: '0',
commentCount: '0',
newCommentCount: '0',
linkedMarkCount: '0',
changeLogs: [
{
likeableId: '3685',
likeableType: 'changeLog',
myLikeValue: '0',
likeCount: '3',
dislikeCount: '0',
likeScore: '3',
individualLikes: [],
id: '20347',
pageId: 'flee_from_surprise',
userId: 'DonyChristie',
edit: '5',
type: 'newEdit',
createdAt: '2016-11-03 02:10:16',
auxPageId: '',
oldSettingsValue: '',
newSettingsValue: ''
},
{
likeableId: '0',
likeableType: 'changeLog',
myLikeValue: '0',
likeCount: '0',
dislikeCount: '0',
likeScore: '0',
individualLikes: [],
id: '18244',
pageId: 'flee_from_surprise',
userId: 'EricBruylant',
edit: '0',
type: 'newTag',
createdAt: '2016-08-03 17:14:09',
auxPageId: 'b_class_meta_tag',
oldSettingsValue: '',
newSettingsValue: ''
},
{
likeableId: '0',
likeableType: 'changeLog',
myLikeValue: '0',
likeCount: '0',
dislikeCount: '0',
likeScore: '0',
individualLikes: [],
id: '18047',
pageId: 'flee_from_surprise',
userId: 'AlexeiAndreev',
edit: '0',
type: 'newSubject',
createdAt: '2016-08-02 00:58:13',
auxPageId: 'bayes_log_odds',
oldSettingsValue: '',
newSettingsValue: ''
},
{
likeableId: '0',
likeableType: 'changeLog',
myLikeValue: '0',
likeCount: '0',
dislikeCount: '0',
likeScore: '0',
individualLikes: [],
id: '18045',
pageId: 'flee_from_surprise',
userId: 'AlexeiAndreev',
edit: '0',
type: 'newRequirement',
createdAt: '2016-08-02 00:56:45',
auxPageId: 'bayes_rule_elimination',
oldSettingsValue: '',
newSettingsValue: ''
},
{
likeableId: '0',
likeableType: 'changeLog',
myLikeValue: '0',
likeCount: '0',
dislikeCount: '0',
likeScore: '0',
individualLikes: [],
id: '17551',
pageId: 'flee_from_surprise',
userId: 'AlexeiAndreev',
edit: '4',
type: 'newEdit',
createdAt: '2016-07-26 17:18:04',
auxPageId: '',
oldSettingsValue: '',
newSettingsValue: ''
},
{
likeableId: '0',
likeableType: 'changeLog',
myLikeValue: '0',
likeCount: '0',
dislikeCount: '0',
likeScore: '0',
individualLikes: [],
id: '17550',
pageId: 'flee_from_surprise',
userId: 'AlexeiAndreev',
edit: '3',
type: 'newEdit',
createdAt: '2016-07-26 17:16:43',
auxPageId: '',
oldSettingsValue: '',
newSettingsValue: ''
},
{
likeableId: '0',
likeableType: 'changeLog',
myLikeValue: '0',
likeCount: '0',
dislikeCount: '0',
likeScore: '0',
individualLikes: [],
id: '17549',
pageId: 'flee_from_surprise',
userId: 'AlexeiAndreev',
edit: '0',
type: 'newRequirement',
createdAt: '2016-07-26 17:15:09',
auxPageId: 'blue_oysters',
oldSettingsValue: '',
newSettingsValue: ''
},
{
likeableId: '0',
likeableType: 'changeLog',
myLikeValue: '0',
likeCount: '0',
dislikeCount: '0',
likeScore: '0',
individualLikes: [],
id: '17548',
pageId: 'flee_from_surprise',
userId: 'AlexeiAndreev',
edit: '0',
type: 'newSubject',
createdAt: '2016-07-26 17:14:39',
auxPageId: 'bayes_rule',
oldSettingsValue: '',
newSettingsValue: ''
},
{
likeableId: '0',
likeableType: 'changeLog',
myLikeValue: '0',
likeCount: '0',
dislikeCount: '0',
likeScore: '0',
individualLikes: [],
id: '16939',
pageId: 'flee_from_surprise',
userId: 'AlexeiAndreev',
edit: '0',
type: 'deleteSubject',
createdAt: '2016-07-16 20:50:35',
auxPageId: 'bayes_rule',
oldSettingsValue: '',
newSettingsValue: ''
},
{
likeableId: '0',
likeableType: 'changeLog',
myLikeValue: '0',
likeCount: '0',
dislikeCount: '0',
likeScore: '0',
individualLikes: [],
id: '16861',
pageId: 'flee_from_surprise',
userId: 'AlexeiAndreev',
edit: '0',
type: 'newSubject',
createdAt: '2016-07-16 16:15:22',
auxPageId: 'bayes_rule',
oldSettingsValue: '',
newSettingsValue: ''
},
{
likeableId: '0',
likeableType: 'changeLog',
myLikeValue: '0',
likeCount: '0',
dislikeCount: '0',
likeScore: '0',
individualLikes: [],
id: '16859',
pageId: 'flee_from_surprise',
userId: 'AlexeiAndreev',
edit: '0',
type: 'newRequirement',
createdAt: '2016-07-16 16:12:58',
auxPageId: 'bayes_rule',
oldSettingsValue: '',
newSettingsValue: ''
},
{
likeableId: '0',
likeableType: 'changeLog',
myLikeValue: '0',
likeCount: '0',
dislikeCount: '0',
likeScore: '0',
individualLikes: [],
id: '16542',
pageId: 'flee_from_surprise',
userId: 'NateSoares',
edit: '0',
type: 'newTeacher',
createdAt: '2016-07-10 23:35:16',
auxPageId: 'flee_from_surprise',
oldSettingsValue: '',
newSettingsValue: ''
},
{
likeableId: '0',
likeableType: 'changeLog',
myLikeValue: '0',
likeCount: '0',
dislikeCount: '0',
likeScore: '0',
individualLikes: [],
id: '16543',
pageId: 'flee_from_surprise',
userId: 'NateSoares',
edit: '0',
type: 'newSubject',
createdAt: '2016-07-10 23:35:16',
auxPageId: 'flee_from_surprise',
oldSettingsValue: '',
newSettingsValue: ''
},
{
likeableId: '0',
likeableType: 'changeLog',
myLikeValue: '0',
likeCount: '0',
dislikeCount: '0',
likeScore: '0',
individualLikes: [],
id: '16514',
pageId: 'flee_from_surprise',
userId: 'NateSoares',
edit: '0',
type: 'newRequirement',
createdAt: '2016-07-10 22:17:15',
auxPageId: 'bayes_log_odds',
oldSettingsValue: '',
newSettingsValue: ''
},
{
likeableId: '0',
likeableType: 'changeLog',
myLikeValue: '0',
likeCount: '0',
dislikeCount: '0',
likeScore: '0',
individualLikes: [],
id: '16230',
pageId: 'flee_from_surprise',
userId: 'NateSoares',
edit: '2',
type: 'newEdit',
createdAt: '2016-07-08 15:58:16',
auxPageId: '',
oldSettingsValue: '',
newSettingsValue: ''
},
{
likeableId: '0',
likeableType: 'changeLog',
myLikeValue: '0',
likeCount: '0',
dislikeCount: '0',
likeScore: '0',
individualLikes: [],
id: '15580',
pageId: 'flee_from_surprise',
userId: 'NateSoares',
edit: '0',
type: 'newParent',
createdAt: '2016-07-06 06:15:51',
auxPageId: 'bayes_rule',
oldSettingsValue: '',
newSettingsValue: ''
},
{
likeableId: '0',
likeableType: 'changeLog',
myLikeValue: '0',
likeCount: '0',
dislikeCount: '0',
likeScore: '0',
individualLikes: [],
id: '15578',
pageId: 'flee_from_surprise',
userId: 'NateSoares',
edit: '1',
type: 'newEdit',
createdAt: '2016-07-06 06:15:23',
auxPageId: '',
oldSettingsValue: '',
newSettingsValue: ''
}
],
feedSubmissions: [],
searchStrings: {},
hasChildren: 'false',
hasParents: 'true',
redAliases: {},
improvementTagIds: [],
nonMetaTagIds: [],
todos: [],
slowDownMap: 'null',
speedUpMap: 'null',
arcPageIds: 'null',
contentRequests: {
lessTechnical: {
likeableId: '3699',
likeableType: 'contentRequest',
myLikeValue: '0',
likeCount: '3',
dislikeCount: '0',
likeScore: '3',
individualLikes: [],
id: '147',
pageId: 'flee_from_surprise',
requestType: 'lessTechnical',
createdAt: '2016-11-08 00:39:20'
},
moreWords: {
likeableId: '4147',
likeableType: 'contentRequest',
myLikeValue: '0',
likeCount: '1',
dislikeCount: '0',
likeScore: '1',
individualLikes: [],
id: '209',
pageId: 'flee_from_surprise',
requestType: 'moreWords',
createdAt: '2019-02-24 12:15:56'
}
}
}