Messages
(i) So we're trying to use half an hour each Thursday, e.g. 12:29 to 12:59, in the Integreat Room 821, 8th floor, for "extra talking" (no new theorems and proofs), details of exercises, questions & answers, etc. It went fine today, Thu 14-xi, partly with the Master Theorem of Ch4, optimal (conditional) testing in expo families. For some of the themes we discussed, check 4.31, 4.32 (the Master Theorem itself), 4.33, Stories iv.6, iv.7.
(ii) Next week we round of Ch 7, on CDs and ccs, and start Ch 8 Light, the last chapter in the course. For Ch 7, we do 7.10, with Old Egypt lifetimes data placed on the site, and also the following: Consider the classic y_1, ..., y_n iid \N(\xi,\sigma^2). First find good clear exact CDs and ccs, for \xi and for \sigma. Then work also with the "Wilks Theorem Recipe" for these two.
Ikke rom 600, men 8de etasje, Integreat-rommet ved kaffemaskinene.
New message: something complicated with Room 600, after all, so we try ROOM 821, 8th floor, Integreat Room.
Our teaching room on Thursdays 10:15 to 12:00 is not available afterwards, but I've managed to book *Room 600*, 6th floor, for Thursdays, starting today Nov 14 onwards. So we can grab a piece of lunch and a cup of coffee and use this opportunity, either from 12:15 and half an hour, or from 12:30, if we wish.
The point is to have time for more details, Questions & Answers, going through themes from previous exercises and stories, etc. The intention is *not* to fill in with even more theorems and proofs and techniques and tools and stories from Romerriket and Old Egypt and Pushkin and Platon and Bach vs. Reger (but read more in the Nils-Emil book).
As listed in the curriculum document, the exercises to be worked through for Ch 7 are 1, 2, 3, 6, 9, 10, 12, 14.
You may also glance through Céline Cunen's "Confidence Curves for Dummies" in FocuStat's summary report (page 5 here):
https://www.mn.uio.no/math/english/research/projects/focustat/summary-report-(january-2019)/
Dennis Christensen teaches from Ch 6, Bayes, on Thursday Nov 7th. On Monday 11th, Nils does more from Ch 6, perhaps rounding off, before we do a light version of Ch 7, Confidence Distributions.
For Mon 11th and Thu 14th, do the Ch 6 exercises that Dennis didn't go through, but also, and perhaps first: Exercise 5.51 (the "we can do things", points (a), (b)), and Exam 2022 set, exercises 2, 3, 1, in more or less that order. For 5.51 (a), take p = 0.10.
There will be some *extra half hours*, for some of the Mondays and Thursday, with possibilities for more details, questions and answers, further exercises (if wished for), etc.; we come back to this after Nils' trip to the Hamburg Symposium on Climate Sciences Statistics.
Hope you all had a chance to reflect on Minister Tung's digitalforelesning last week; it's also on YouTube.
Mon Nov 4th I round off Ch 5, likelihood, KL and least false, "how to do it", a few regression models.
On Thu 7th, Dennis Christensen goes through a decent chunk of the Ch 6 curriculum, where the exercises list is 1, 2, 3, 4, 6, 8, 15, 16, 27, 28; again, much more on Bayes in the stk4021 course, so for this course this is meant to be a modest volume.
Pretty soon we'll be diving into previous exam sets and a few more Stories. Start examining exam sets from 2023, 2022, 2021. We've done 2023, 1 and 2, and the next we'll do are 2022, nos 2, 3, 1.
It's defined as obligatory for the Integreat PhDs to come to Minister Tung's lecture, which unfortunately clashes in time with our course -- so no teaching Thu 31st October. So see you all on Monday Nov 4th, with adjusted timeplan and more information.
Integreat Digitalisation Lecture 2024: Karianne Tung
Norwegian Minister of Digitalisation and Public Governance Karianne Tung holds the opening lecture on digitalisation status in Norway.
Time and place: Oct. 31, 2024 10:30 AM – 11:30, Sophus Lies auditorium.
https://www.integreat.no/events/public-events/conferences/annual-digitalisation-lecture/index.html
Probability Proofs for Stirling (and More):
the Ubiquitous Role of √2π
I think all of this is understandable for stk4011 students (whether this makes it "exam relevant" or not).
To be arXiv'd and submited to a journal in a day or two.
https://www.mn.uio.no/math/english/research/projects/focustat/publications_2/stirling11.pdf
One might statistically predict a perhaps higher correlation between the December 2024 exam set in this course, in terms of themes & style, and earlier Nils LH exam sets, compared to those composed by other lecturers. We'll also go through most of these in the course of November. So check here, for the occasions:
2023 (check also the Oblig), 2019, 2014 (both Exam Project and the firetimers).
Otherwise a good way to prepare for the December 2024 exam is to solve all 777 Exercises, and all questions for the 77 Statistical Stories, in the forthcoming Hjort-Stoltenberg book.
https://www.mn.uio.no/math/english/research/projects/focustat/lecture-notes-and-exercises-for-various-courses-gi/
a. Check the stk4011 exam set December 2023. H?yesterett October 2024 says it's ok for Nils to have used that exam set's Problem 1 in the October 2024 Oblig. But for Monday we do Problem 2, which August will do on the blackboard. Also plod ahead through the Ch 5 Exercises 1, 2, 6, 14, 15, 16, 17, 18, 23, 29, 30, 31, 42, 44, 46, 51. Gently asterisked: 2, 14, 15, 17, 42, 44.
b. For Thursday 31st, Halloween day & fagligpedagogisk dag: 09.15 –10.00: "Hvordan lyve med statistikk" (Aud. 1, VB), with Dennis Christensen. After that, Dennis is Nils Substitute for stk4011, with *Ch 6* material, Bayes, where the curriculum subset of exercises is 1, 2, 3, 4, 6, 8, 15, 16, 27, 28.
c. Nils is anderswo engagiert, singing here, 11.15 –12.30: "Tunes of Time: A musical lecture about the history of English" (Aud. 1, Sophus Bugge).
The curriculum document has been updated, as of 24-x-24, with lists of exercises for all chapters 1, 2, 3, 4, 5, 6, 7, 8. Note that these lists related to the PartOne-October2024.pdf, for Chapters 5, 6, 7, 8. There will be one more updated version, after a few more weeks, regarding the appopriate list of Statistical Stories.
Over the remaining weeks of the course we'll also sample some earlier exam sets -- more information to come soon.
On Thursday 24th I will go through aspects of the Oblig (which you know has deadline Monday 21st!).
On Mon 21-x, we first spend some time rounding off the last parts of Ch 4, with more sufficiency, copleteness, optimal conditional testing. Then we start the important Ch 5, on likelihood theory, the last very-important-part of the course -- after this we'll have somewhat light versions of Chs 6, 7, 8. Key words for Ch 5 are likelihood, maximum likelihood estimators, Kullback-Leibler, normal approximations, Wilks theorems, applications of these tools.
For this week we plod ahead with the list of exercises given on the curriculum list: Exercises 1, 2, 6, 14, 15, 16, 17, 23, 28, 29, 30, 41, 43, 45, 50.
I've uploaded an updated PartOne, namely PartOneAgain.pdf, as of 18-x-2024. Both versions, PartOne of mid August and PartOneAgain of mid October, will be retaind at the course website.
The changes are not big, and ind...
Oblig!, deadline is Monday October 21.
This week we're rounding off Ch 4, with key words testing, power, Neyman-Pearson, sufficiency, factorisation theorem, uniformly most powerful tests, conditional testing. Certain methods work particularly well for the exponential family class.
The "gently asterisked" subset of exercises for Ch4 is 1, 2, 5, 7, 16, 18, 24, 25, 26, 32, 34, 41. There will soon be an updated version of the curriculum document.
Nils is anderswo engagiert, as it turns out, so Emil Stoltenberg will teach Thursday October 10 (10-12). The topics are inside Ch 4, so a bit of testing and a bit of Neyman-Pearson and then attention on sufficiency and the factorisation theorem. Nils is teaching as usual next week, with the rest of Ch 4.
Otherwise, happy work with the Oblig.
A generation of Norwegian statisticians and actuarial science people grew up learning the basics from Erling Sverdrup's then famous Lov og tilfeldighet, I & II (these two were enough for four-five courses). The sentence above, which I hope I recall correctly, is from Sverdrup's Bind I, when explaining what hypothesis testing is about.
The lecture today is cancelled due to a doctor's appointment.
We've rounded off Ch 3. There will be approx 2 exercises related to order statistics: (i) the Oblig 2023 no. 1; (ii) Story i.v, about 548 boys and 481 girls born at Rikshospitalet in Oslo, and their birthweights.
Thanks to Vilde U who told us about n = 700,000 necessary for the empirical kurtosis to be inside [5.80, 6.20], with data from the exponential; she's right (as I found out), check R script com351c. You may fiddle a bit with that code to check sample kurtosis from other distributions, and for many such the convergence to true kurtosis is drastically faster; it's slow here, since the tail is heavy and the kurtosis big.
Now two weeks for Ch 4, with approximately these exercises: 1, 2, 3, 4, 5, 7, 8, 9, 12, 16, 18, 24, 25, 26, 27, 29, 32, 34, 35, 41. Gently asterisked: 1, 2, 5, 7, 16, 18, 24, 25, 26, 32, 34, 41.
Sigurd does 4.3.
Oblig comes: from Oct 7 to Oct 21.
We've started Ch 3, where the tentative subset of exercises is 1, 2, 4, 5, 6, 7, 8, 9, 11, 12, 15, 16, 17, 18, 24, 27, 30, 31, 32, 34. On Thu Sept 26, Maham I does 3.9, on Mon Sept 30, Carl Fredrik A does 3.8. Key words in front of us are: (i) confidence; (ii) large-sample theory of Ch 2 to help us to aproximate confidence intervals; (iii) order statistics basics; (iv) moment estimators; (v) multiple linear regression.
On Mon Sept 30 we first do the New Haven Story iv.1, with linear regression used also for prediction. The Oblig, from Oct 7, will have something half-similar. Note the R script com105a, with New Haven analysis (and note that even though b is clearly significant positive, predictions for start - 5 years and end + 5 years have a big overlap).
I define 3.8 and 3.12, done by Carl Fredrik and Vilde, as important, since it's "not easy, not too hard, various details, and instructive". Then we round off Ch3 with (ii), (iii), (iv) from the ...
We've been more or less through the Handball story (Who wins?, v.6) and the Football story (the Turn-around operation, v.7). After having started Ch3, with the basics of the linear regression model, including prediction, work through as much as you can for Stories iv.1 (New Haven temperatures) and iv.3 (How special are You?).
The Irelevant curated datasets are now on the course site -- the first, newhaven_data, temperatures and years; the second. sleep14, weight-of-body and weight-of-brain for 56 mammals.
Here's another probability proof of Stirling's 1730 formula, which I stumbled upon yesterday. Let X_n be gamma(n,1). Then Z_n=(X_n-n)/\rootn tends to Z, the standard normal. Work a bit with the mean of Trunc(Z_n), and see that it is \rootn e^(-n) n^n/n!. Which has to tend to the mean of Trunc(Z), 1/\sqrt{2 \pi}. End of proof.
https://www.facebook.com/groups/1589206911336271/posts/3853571988233074/
So perhaps a subset of the December 2024 exam questions might have been mentioned or pointed to earlier in the FocuStat group.
After rounding off Ch2 (with CLT, LLN, Lindeberg, delta method, Cramer-Slutsky) we aim at doing Ch3 for Weeks 6 and 7. A tentative list of curriculum exercises is this one: 1, 2, 4, 5, 6, 7, 8, 9, 11, 12, 15, 16, 17, 18, 24, 27, 30, 31, 32. On Mon Sept 23, Aslak does 1 and 2.
I have updated the curriculum list document (as of 16-ix), also tentatively bringing in a little asterisking to indicate those that are somewhat more fundamental ("hi there, I can be used on lots of occasions") than the others ("I'm here for a single good illustration").
We've also done two Statistical Stories, and will soon enough have more; also these will be listed in the curriculum document.
The Oblig dates are tentatively set to t_0 = Mon Oct 7, where the exercise set is made available, and t_1 = Mon Oct 21, deadline for handing in your pdf'd reports.
The exercise list for Ch 2 is more or less 1, 2, 6, 7, 8, 9, 11, 12, 13, 14, 18, 19, 21, 29, 32, 33, 41, 42, 43, 52, 54, 55, and we plod on, for Weeks 4 and 5, with these. Keywords are CLT, LLN, Lindeberg, the delta method, and results will be used here and there in the rest of the course.
For Week 5, work through as much as you can also for Statistical Stories v.7 (how Belgium went from 0-2 to 3-2, football) and v.6 (how exciting was it, really, minute by minute, Norway-Denmark women's handball, November 2022)? Data for Story v.6 are on the website. Another dataset there has match results for 117 matches, where the point is to have a Poisson dependence model: results (x,y) have a positive correlation. Sigurd does 2.39, proving the Stirling formula via the Poisson and CLT.
Football story, Belgium Breaks a 48 Year Old Curse (Apparently):
https://www.mn.uio.no/math/english/research/projects/focustat/the-focustat-blog!/belgiaja...
We have managed to find student representatives (and I hope these mail addresses work): Vilde Hansteen Ung (vildehun@uio.no) and David Andreas Sand (davidasa@uio.no).
If you wish to communicate something to me, or have questions regarding the course, the simplest algorithm is to approach me (e.g. during teaching breaks) -- but you may also talk to Vilde and/or David, who will then talk with me. Vilde and David: we agree on a cup of coffee within a few weeks.
We've started on Ch 2, large-sample theory, and will use more or less Weeks 4 and 5 to go through the main material. The tentative list of exercises is 1, 2, 6, 7, 8, 9, 11, 12, 13, 14, 18, 19, 21, 29, 32, 33, 41, 42, 43, 52, 54, 55 (and soon we'll start with a few Statistical Stories). For Mon Sept 9, we're aiming for 1, 2, 6, 7, 8, 9, 11, from which Vilde does no.6. If she sees this in time: let X_n be Gamma(a_n, b_n), with a_n going to 1 and b_n going to 1. Show that X_n tends to the unit exponential in distribution.
On Thu Sept 12 we go on, with these, from Ch 2: 2, 9 (continue from what Vilde showed us), 11, August does a sutable subset of the union of 13, 14, 18 (try your own g(p)), 19, 21. After that we're ready to push for CLT, the delta method, Lindeberg, the LLN -- those are key words for the most important themes of the chapter.
Our plan is to go through Chs 1 + 2 in the course of five weeks, and we're indeed rounding off Ch 1 after the Monday of Week 3. Remaining exercises from Ch 1, for Thursday Week 2 and Monday Week 3, are (more or less, depending on time):
24, 30, 31, 33, 40, 41, 43, 45, 49, 50.
Hedvig B does 1.40 (f,g,h). H?kon does chi-squared things from 1.43(d)-(h), Vilde something next week. For Ch 2 the intended full list is
1, 2, 6, 7, 8, 9, 11, 12, 13, 14, 18, 19, 21, 29, 32, 33, 41, 42, 43, 52, 54, 55.
We'll also soon start with a few of the Statistical Stories.
The plan is to use *a bit less than 5 weeks* for Chs 1, 2, then about 2 weeks for each of Chs 3, 4, 5. If things go well, we round off Ch1 on Mon Sept 2 and start on Ch2 Thu Sept 5.
Ch 1 has a multitude of exercises, of which we might be defining about a total of 25 as curriculum. After as much as we muster in Week 1 of 1, 2, 3, 4, 7, 8, 9, 10, 12, 15, 16, we go for
17, 18, 21, 23, 24, 29, 30, 31, 33,
then the last batch with 40, 41, 43, 45, 49, 50.
(a) For Week 2, Monday, Aslak H does 1.12(e), Maham I does 1.17, on the blackboard; for Thursday, David S does 1.30(c),(d),(e).
(b) I pointed to a somewhat heavy-handed way today Mon Aug 26 to find prodd and preven for the Poisson; it works, but an easier way is this. Write down power series sums for prodd and preven, with prodd + preven = 1. Show that preven - prodd = \exp(-2\theta), then show preven = \half(1 + \exp(-2\theta)), prodd = \half(1 - \exp(-2\theta)...