Chronology |
Current Month |
Current Thread |
Current Date |

[Year List] [Month List (current year)] | [Date Index] [Thread Index] | [Thread Prev] [Thread Next] | [Date Prev] [Date Next] |

*From*: John Denker <jsd@av8n.com>*Date*: Fri, 1 Oct 2021 11:22:36 -0700

On 9/22/21 8:47 PM, Paul Nord wrote:

You'd really enjoy David MacKay's lecture 10 on Information theory. He

really rips into physicists regarding "old school" methods. He's a

physicist, actually. So he should know.

Yeah. I've actually discussed these topics with David MacKay.

He definitely knows what he's talking about.

As for the video, I just now got around to watching it. IMHO

the first 25½ minutes is the least interesting part. I'm not

interested in hearing about all the "old school" methods that

don't work.

The rest of the video is great.

https://youtu.be/mDVE0M-xQlc?t=1537

===============

Tangential point: Beware that the word "Bayesian" means different

things to different people. There are some absolute kooks who

call themselves "Bayesians" as if that automatically proved the

correctness of whatever they feel like saying.

The correct approach is just math. As MacKay points out, the math

was used to good effect by Laplace, years before Rev. Bayes came

along. You can call it Bayesian or not.

I don't call myself a Bayesian, for the same reason that I don't

call myself a Laplacian or a Newtonian.

This terminological snafu is one of the reasons why it is hard to

google for information on the topic. Sometimes the right answer

will be called Bayesian, and sometimes not.

================

As MacKay points out, we start with a probability in the form of

a product of factors. We can take the logarithm, which gives us a

sum of terms, with exactly the same meaning. It's a sum of log

probabilities, or log improbabilities.

Note that the weighted sum of log improbabilities looks an

awful lot like a cross entropy. It also looks a lot like a

Kullback-Leibler information divergence. So you sometimes see

those terms appearing in articles on this subject.

The weighting factors in the aforementioned weighted sum play

the same role as Bayesian priors. So once again, it's just math.

You can call it Bayesian or not. The math doesn't care.

- Next by Date:
**Re: [Phys-L] Bayesian Inference in Half-Life measurement** - Next by thread:
**Re: [Phys-L] Bayesian Inference in Half-Life measurement** - Index(es):