"1. I propose that this concept be called "unex..."

https://arbital.com/p/7qs

by Leon D Feb 3 2017 updated Feb 17 2017


  1. I propose that this concept be called "unexpected surprise" rather than "strictly confused":
  1. The section on "Distinction from frequentist p-values" is, I think, both technically incorrect and a bit uncharitable.

    • It's technically incorrect because the following isn't true:

      The classical frequentist test for rejecting the null hypothesis involves considering the probability assigned to particular 'obvious'-seeming partitions of the data, and asking if we ended up inside a low-probability partition.

      Actually, the classical frequentist test involves specifying an obvious-seeming measure of surprise , and seeing whether is higher than expected on . This is even more arbitrary than the above.

    • On the other hand, it's uncharitable because it's widely acknowledged one should try to choose to be sufficient, which is exactly the condition that the partition induced by is "compatible" with for different , in the sense that for all the considered .

      Clearly is sufficient in this sense. But there might be simpler functions of that do the job too ("minimal sufficient statistics").

      Note that being sufficient doesn't make it non-arbitrary, as it may not be a monotone function of .

  2. Finally, I think that this concept is clearly "extra-Bayesian", in the sense that it's about non-probabilistic ("Knightian") uncertainty over , and one is considering probabilities attached to unobserved (i.e., not conditioning on the observed ).

    I don't think being "extra-Bayesian" in this sense is problematic. But I think it should be owned-up to.

    Actually, "unexpected surprise" reveals a nice connection between Bayesian and sampling-based uncertainty intervals:

    • To get a (HPD) credible interval, exclude those that are relatively surprised by the observed (or which are a priori surprising).
    • To get a (nice) confidence interval, exclude those that are "unexpectedly surprised" by .