*If* the folks at CERN are chasing significance, *and I’m not saying they are*, they are doing it in a vastly more subtle way than the experiments I’ve been discussing here. For starters, they are not satisfied with the usual criterion, namely that an effect is real if there’s only a 5% chance that it’s a coincidence. Particle physicists demand a ‘5-σ result’ before claiming they discovered a new particle. In other words, they say an effect is real if there’s only a 0.00005% chance that it’s a coincidence.

They have not reached this point, so they did not, yesterday, claim to have discovered the Higgs. They said they found ‘tantalizing hints’ of the Higgs. For example, the ATLAS detector has gotten a 3.6-σ result, meaning there’s a chance of less than 0.01% of being a coincidence. The CMM detector, doing an independent experiment, has gotten a 2.8-σ result, meaning there’s a chance of less than 1% of being a coincidence. So, they will continue the experiment and collect more data to see if these hints hold up.

For a good detailed but nontechnical intro, I recommend this:

• Ethan Siegel, The Large Hadron Collider, the Higgs, and hope, *Starts With a Bang!*, 13 December 2011.

My figures come from here:

• Peter Woit, Today’s Higgs results, *Not Even Wrong*, 13 December 2011.

You ask:

Can we trust statistics when it comes to natural laws?

We can’t trust anything except this: *repeatedly re-examining all our beliefs in an intelligent and well-balanced way, using all the tools of reason at our disposal, trying hard to lapse neither into complacency nor paranoia*. If we keep doing this, we can become quite sure (though never 100% sure) about some rather unobvious things.

And when we’re doing this, there’s something a lot worse than using statistics: namely, *not* using statistics.

I use the term to communicate things like response of a marketing campaign and if I showed error bars and and used words like “detectable” to my audience, it would not be accepted and moved on. It would be a point of discussion and probably a tangent.

Just came across your blog and enjoy it.

Thanks

]]>Hi Thomas: I have spent a significant amount of my life rebelling, and I have the abysmal GPA with little income to show it, so I’m very motivated to be a ‘good student’. I am halfway through the semester, I’ve memorized two formulas, and none – zero – of the problems I’ve been presented have required the usage of those formulae, nor has my instructor managed to give me any idea of a standard or typical situation in which to utilize them, or any standard methodology I should use in applying them. I have flung my book at the wall several times, and am just learning it from YT at this point. Frustrated. MUCH. I don’t understand why, and I’m not getting how, and so yeah I’ve given up trying to make any sense of what I’m being told in class.

]]>No, the person who wrote this is strong-willed and will not merely “adapt”. She will, however, do what it takes to pass the course!

]]>By the way, I think every statistician would agree that confidence intervals (error bars) are much more informative and intuitive than p-values and these concepts can be regarded as technically interchangeable if computed appropriately (i.e, given an estimate and p-value we can compute the confidence interval, or vice versa).

I also think that “error bar” is a more appropriate term than “confidence interval,” since the concept has nothing to do with one’s psychological state of mind.

]]>Jacob Cohen, The earth is round (

p< 0.05).American Psychologist49 (1994), no. 12, 997-1003.

It’s short, and a great read.

]]>“It struck me as bull to start with, admittedly, but since my grade depended on it, I grinned and swallowed.” sounds to me like a description of the switching from “learning” to “adapting” I had in mind. “At least my eyes are open now” sounds to me more like a strengthening of it (leading to “treating statistics as a necessary but unpleasant piece of bureaucratic red tape, and then doing whatever it takes to achieve the appearance of a significant result”), than like a switching back.

]]>