STAT 656 — Bayesian Data Analysis Fall 2024

Hello, if you have any need, please feel free to consult us, this is my wechat: wx91due

STAT 656 — Bayesian Data Analysis Fall 2024 

Course Description: Bayesian data analysis refers to practical inferential methods that use probability models for both observable and unobservable quantities. The flexibility and generality of these methods allow them to address complex real-life problems that are not amenable to other techniques. This course will provide a modern introduction to Bayesian data analysis, its powerful applications and the challenges it presents. Topics covered include: 

First half of the class: fundamentals of Bayesian inference for single and multiparameter models, hierarchical models, regression, model checking, Monte Carlo and MCMC methods 

Second half of the class: more modern topics such as posterior approximation through variational Bayes, amortized inference, Bayesian inference in simulation models, Causal inference, Bayesian nonparametrics. 

Specific topics and the course outline are subject to change as the semester progresses. All topics will be motivated by problems from the physical, life, social, and management sciences. Conceptual understanding and inference via computer simulation will be emphasized throughout the course. 

Learning Outcomes: 

• Fluency in the principles and techniques of Bayesian data analysis.
• Ability to read classic and modern research papers on Bayesian statistics 
• Apply Bayesian methodology to solve real-life problems. 
• Utilize R and RStan for Bayesian computation, visualization, and analysis. 

– R is freely available for download here. Free manuals are available here. 

– Assistance for R is available at the help desk in MATH G175.

• Discuss what is learned in lectures and assignments through a final project, an oral presentation and written report.

Prerequisites: Introductory graduate-level courses in probability and statistics, such as STAT 516 and 517, and familiarity with computing. Students should have some programming experience using a language such as R, Python, or MATLAB. The ability to write, debug, and execute programs in at least one of the above languages is required. Also required is familiary and fluency with probability and linear algebra. Advanced graduate-level statistics courses, such as STAT 519, STAT 525, STAT 528, and STAT 532 are recommended, but not required. 

Course Outline: To be finalized 

• Week 1: Introduction to Bayesian inference and R (Chapters 1 – 3 of BDA). 

Lecture 1 (8-22-2023): Overview of Bayesian statistics and R. 

Handout (Due 8-22-2023): Background survey. 

Lecture 2 (8-24-2023): Bayesian inference from Binomial and Multinomial data. Elicitation of prior knowledge from a subject-matter expert. Analytic, discrete approximation, and Monte Carlo posterior calculation methods. 

Homework 1 (Due 9-5-2023): Problems based on Chapters 1 – 3 of BDA. 

• Week 2: Standard parametric models (Chapters 2 – 3 of BDA). 

Lecture 3 (8-29-2023): Normal Data and the general exponential family. 

Lecture 4 (8-31-2023): The Multivariate Normal distribution. 

• Week 3: Bayesian regression (Chapter 14 of BDA). 

Lecture 5 (9-5-2023): Bayesian linear regression. 

Lecture 6 (9-7-2023): Stan, and the Bayesian analysis of generalized linear models. 

Homework 2 (Due 9-19-2023): Problems based on Chapters 3, 5, and 14 of BDA. 

• Week 4: Hierarchical models (Chapter 5 of BDA).

Lecture 7 (9-12-2023): Exchangeability and setting up hierarchical models. 

Lecture 8 (9-14-2023): Hierarchical normal models, computation with hierarchical normal models. ˆ Week 5: Model checking and improvement (Chapters 6 and 7 of BDA). 

Lecture 9 (9-19-2023): Posterior predictive checks, evaluating and comparing models. 

Lecture 10 (9-21-2023): Empirical Bayes – Homework 3 (Due 10-10-2023): Problems based on Chapters 6, 7, 10, and 11 of BDA. 

• Week 6: Introduction to Bayesian computation (Chapters 10 and 11 of BDA). 

Lecture 11 (9-26-2023): An introduction to Markov Chain Monte Carlo 

Lecture 12 (9-28-2023): The Metropolis-Hastings algorithm – Project Description: Due 10-3-2023.

• Week 7: Continuation of Bayesian computation (Chapter 11 of BDA). 

Lecture 13 (10-3-2023): Gibbs sampling 

Lecture 14 (10-5-2023): MCMC practicalities and advanced techniques. ˆ Week 8: Advanced MCMC algorithms (Chapter 12 of BDA). – October Break (10-10-2023): No class. 

Lecture 15 (10-12-2023): Hamiltonian Monte Carlo 

Homework 4 (Due 10-24-2023): Problems based on Chapters 11 and 12 of BDA. 

• Week 9: Modal and distributional approximations (Chapter 13 of BDA). 

Lecture 16 (10-17-2023): Variational inference. 

Lecture 17 (10-19-2023): Applications of variational inference. 

• Week 10: Paper presentations. 

Lecture 18 (10-24-2023): Paper presentations 

Lecture 19 (10-26-2023): Paper presentations

• Week 11: Paper presentations. 

Lecture 20 (10-31-2023): Paper presentations 

Lecture 21 (11-2-2023): Paper presentations 

Project Progress Report: Due 11-26-2023. 

• Week 12: Modal and distributional approximations (Chapter 13 of BDA). 

Lecture 22 (11-7-2023): Finite mixture models 

Lecture 23 (11-9-2023): Dirichlet processes and a taste of Bayesian nonparametrics. 

• Week 13: Gaussian and Dirichlet process models (Chapters 21 and 23 of BDA). 

Lecture 24 (11-14-2023): ABC and Amortization 

Lecture 25 (11-16-2023): Probabilistic numerics 

• Week 14: Project Progress Reports 

Lecture 26 (11-21-2023): Presentations on progress of projects. 

Thanksgiving Break (11-23-2023): No class 

• Week 15: Causal inference (Chapter 8 of BDA). 

Lecture 28 (11-28-2023):An introduction to the Rubin Causal Model and noncompliance. 

Lecture 29 (11-30-2023): Bayesian optimization 

• Week 16: Review, and open problems in Bayesian data analysis. 
• Week 17: Finals week, no class, project reports due. 

Course Work and Requirements: 

• Homework will generally be posted on a Thursday, and due two or three weeks later. Barring excused absences, no late homework will be accepted. If you anticipate a conflict with a homework deadline due to university-sponsored activity (e.g. attending a conference), please let me know as easly as possible.

There will be 4 homework assignments accounting for 40% of your course grade. R Markdown should be used for homework submission. A short tutorial can be found at shiny.rstudio.com/articles/rmarkdown.html. You may discuss problems with other students, but you must write your own solution independently, and you must provide the names of students that you had significant discussions with. 

• Paper reviews/presentations will be due around the moddle of the semester. Specific details will be given in a separate handout.
• Group projects will be due at the end of the semester. Specific details on the project will be given in a separate handout. 

For your homeworks, paper reports and group projects, please note the Purdue Honors Pledge: “As a Boilermaker pursuing academic excellence, I pledge to be honest and true in all that I do. Accountable together - we are Purdue.” 

Course Policies 

Addressing Questions via E-mail: Please feel free to e-mail questions to the instructors, but reserve those that involve extensive computation or mathematical expressions for office hours. If your question involves programming, please be sure to e-mail a minimal working example of your code to the instructors. 

Incompletes: Incompletes will only be given under emergency circumstances, e.g., a serious auto accident, death of family member, etc. (see the grief absence policy for further information). Incompletes will not be given to students failing the course.

Grading: The grading scale is predetermined so as to eliminate competition with other students, and to ensure that you always know your grade in the class. Your grade is based upon your performance only. Grades will not be curved.

Evaluation: Student feedback is essential for any course to be successful. Feedback questionnaires will be included in each assignment. These evaluations should be taken seriously, and will be addressed directly by the instructors. 

Re-grading: All grade disputes are to be made on paper, and submitted directly to Dr. Rao. Discussions or arguments for re-grades will not be done in person. 

A student has until one week after receiving his/her grade to dispute the grade (in writing). Handling re-grades in this manner eliminates the “end of the semester” digging for points. 

When disputing a grade, you should state the question, the dispute, and the number of points you feel you should have received for the question. If you do not state the number of points you think are reasonable for the re-grade, zero points will be given as the re-grade. Please note that when you ask for a question to be re-graded, the entire assignment may be re-graded, and there is a possibility of losing points. 

Dropping the Course: The instructors reserve the right to not sign anyone out of the course once the deadline for dropping without the instructors signature has passed. Please take care to pay attention to these dates. 

Attendance and Participation: Students: You are expected to attend lectures. You are expected to arrive on time, or before. You are expected to stay until the end of lecture unless you have asked in advance to leave early. You are expected to be prepared and participate. 

Do not come to class if you are feeling ill, but DO email me at [email protected], with the subject line: STAT656 absence. I do not need details about your symptoms. Just let me know you are feeling ill and cannot come to class. Unless it falls under the University excused absence regulations (see below), any work due should be submitted on time via our course Brightspace (but see the class homework policy). 

When conflicts or absences can be anticipated, such as for many University sponsored activities and religious observations, you should inform the instructors of the situation as far in advance as possible. For unanticipated or emergency absences when advance notification is not possible, you should contact the instructors as soon as possible by e-mail, or the Department of Statistics main office. When you are unable to make direct contact with the instructors and unable to leave word with the Department of Statistics because of circumstances beyond your control, and in cases of bereavement, you or your representative should contact the Office of the Dean of Students. The instructors will try to accommodate you either by excusing you or by allowing you an extension when possible. Ultimately, you are responsible for all required coursework and bear full responsibility for any academic consequences that may result due to your absence. 

Links to the complete attendance policy and implications can be found at 

www.purdue.edu/advocacy/students/absences.html and www.purdue.edu/studentregulations/regulations_procedures/classes.html. 

Instructors: You can expect that we will attend lectures. We will arrive in the lecture room prior to the start of lecture, and will end lecture on time. You can expect that we will be prepared for lecture, try our best to convey the information for the course, and show respect for all students. 

If we are unable to attend lecture you will know in advance, and we will either cancel class or provide a guest instructor. We will be present for our office hours, and available for scheduled appointments. 

The amount of material covered in each lecture is governed by the speed with which we complete the material. Every group of students is different, and we would rather 5 teach the material well (and have you learn it) than speed through the topics for the purpose of covering a preset number of topics. Accordingly, the course outline is subject to change as the course progresses.

Grief Absence Policy for Students: Purdue University recognizes that a time of bereavement is very difficult for a student. The University therefore provides the following rights to students facing the loss of a family member through the Grief Absence Policy for Students (GAPS). Students will be excused for funeral leave and given the opportunity to earn equivalent credit and to demonstrate evidence of meeting the learning outcomes for missed assignments or assessments in the event of the death of a member of the student’s family. 

Counseling and Psychological Services Information: Purdue University is committed to advancing the mental health and well-being of its students. If you find yourself beginning to feel some stress, anxiety, and/or feeling slightly overwhelmed, try WellTrack. Sign in and find information and tools at your fingertips, available to you at any time. If you need support and information about options and resources, please see the Office of the Dean of Students for drop-in hours (Monday - Friday, 8:00 AM - 5:00 PM). If you or someone you know is feeling depressed and/or in need of mental health support, services are available. For help, such individuals should contact Counseling and Psychological Services (CAPS) at (765) 494-6995, and www.purdue.edu/caps during and after hours, on weekends and holidays, or by going to the CAPS office on the second floor of the Purdue University Student Health Center (PUSH) during business hours. 

University Emergency Information: A safety briefing will be conducted on the first day of class. In the event of a major campus emergency or temporary suspension of classes, course requirements, deadlines and grading percentages are subject to changes that may be necessitated by a revised semester calendar or other circumstances beyond the instructors’ control. You can get information about changes in this course by means of the course web page, or contacting the instructors via e-mail or phone. You are expected to read your Purdue e-mail on a frequent basis. 

Violent Behavior Policy: Purdue University is committed to providing a safe and secure campus environment for members of the university community. Purdue strives to create an educational environment for students and a work environment for employees that promote educational and career goals. Violent behavior impedes such goals. Therefore, violent behavior is prohibited in or on any University Facility or while participating in any university activity. 

Academic Dishonesty: Purdue prohibits “dishonesty in connection with any University activity. Cheating, plagiarism, or knowingly furnishing false information to the University are examples of dishonesty.” [Part 5, Section III-B-2-a, University Regulations] Furthermore, the University Senate has stipulated that “the commitment of acts of cheating, lying, and deceit in any of their diverse forms (such as the use of substitutes for taking examinations, the use of illegal cribs, plagiarism, and copying during examinations) is dishonest and must not be tolerated. Moreover, knowingly to aid and abet, directly or indirectly, other parties in committing dishonest acts is in itself dishonest.” [University Senate Document 72-18, December 15, 1972].

Academic integrity is one of the highest values that Purdue University holds. You are encouraged to alert university officials to potential breeches of this value by either e-mailing [email protected], calling 765-494-8778, or contacting the Office of the Dean of Students (www.purdue.edu/odos). While information may be submitted anonymously, the more information that is submitted provides the greatest opportunity for the university to investigate the concern. Bonus points will be given to students who report instances of cheating. 

Incidents of academic misconduct in this course will be addressed by the course instructor and referred to the Office of Student Rights and Responsibilities (OSRR, www.purdue.edu/odos/osrr) for review at the university level. Any violation of course policies as it relates to academic integrity will result minimally in a failing or zero grade for that particular assignment, and at the instructor’s discretion may result in a failing grade for the course. In addition, all incidents of academic misconduct will be forwarded to OSRR, where university penalties, including removal from the university, may be considered. Use of instructor solution manuals or related resources will not be tolerated. 

Use of Copyrighted Materials: Among the materials that may be protected by copyright law are the lectures, notes, and other material presented in class or as part of the course. Always assume the materials presented by the instructors are protected by copyright unless the instructors have stated otherwise. Students enrolled in, and authorized visitors to, Purdue University courses are permitted to take notes, which they may use for individual/group study or for other non-commercial purposes reasonably arising from enrollment in the course or the University generally. 

Notes taken in class are, however, generally considered to be “derivative works” of the instructors’ presentations and materials, and they are thus subject to the instructors’ copyright in such presentations and materials. No individual is permitted to sell or otherwise barter notes, either to other students or to any commercial concern, for a course without the express written permission of the course instructor. To obtain permission to sell or barter notes, the individual wishing to sell or barter the notes must be registered in the course or must be an approved visitor to the class. Course instructors may choose to grant or not grant such permission at their own discretion, and may require a review of the notes prior to their being sold or bartered. If they do grant such permission, they may revoke it at any time, if they so choose. 

Students with Disabilities: Purdue University is required to respond to the needs of the students with disabilities as outlined in both the Rehabilitation Act of 1973 and the Americans with Disabilities Act of 1990 through the provision of auxiliary aids and services that allow a student with a disability to fully access and participate in the programs, services, and activities at Purdue University. 

Purdue University strives to make learning experiences as accessible as possible. If you anticipate or experience physical or academic behaviors based on disability, you are welcome to let the instructors know so that they can discuss options. You are also encouraged to contact the disability resource center at [email protected] or by phone 765-494-1247. If you have a disability that requires special academic accommodation, please make an appointment to speak with the instructors within the first three (3) 7 weeks of the semester in order to discuss any adjustments. It is important to talk about this at the beginning of the semester. It is the student’s responsibility to notify the Disability Resource Center (www.purdue.edu/drc) of an impairment/condition that may require accommodations and/or classroom modifications. 

Nondiscrimination: Purdue University is committed to maintaining a community that recognizes and values the inherent worth and dignity of every person; fosters tolerance, sensitivity, understanding, and mutual respect among its members; and encourages each individual to strive to reach his or her own potential. In pursuit of its goal of academic excellence, the University seeks to develop and nurture diversity. The University believes that diversity among its many members strengthens the institution, stimulates creativity, promotes the exchange of ideas, and enriches campus life. 

Purdue University’s nondiscrimination policy can be found at www.purdue.edu/purdue/ea_eou_statement.php. Purdue University prohibits discrimination against any member of the University community on the basis of race, religion, color, sex, age, national origin or ancestry, genetic information, marital status, parental status, sexual orientation, gender identity and expression, disability, or status as a veteran. The University will conduct its programs, services and activities consistent with applicable federal, state and local laws, regulations and orders and in conformance with the procedures and limitations as set forth in Executive Memorandum No. D-1, which provides specific contractual rights and remedies. Any student who believes they have been discriminated against may visit www.purdue.edu/report-hate to submit a complaint to the Office of Institutional Equity. Information may be reported anonymously. 8