Prof. Emma Klugman

Assistant Professor of the Practice & Director of Master’s Program in Data Science

Boston College Lynch School of Education and Human Development

Biography

I am a psychometrician, statistician, and educator who works at the intersection of data science and education.

Before moving to Boston College to launch our new MS in Data Science, as its Program Director and an Assistant Professor of the Practice in Data Science, I completed my PhD at the Harvard Graduate School of Education in Education Policy & Program Evaluation with a minor in Data Science. I also hold a Masters in Education and a Bachelors in Statistics and Computer Science from Harvard.

My work has been published in Educational Measurement: Issues and Practice and presented at the annual meeting of the National Council on Measurement in Education as well as to the International Association for Statistical Education. I have also done cross-disciplinary work, including a project published in the Journal of Social Media in Society, and an article co-authored with some former students which appeared in the journal Cannabis and Cannabinoid Research.

My dissertation research, “Essays on Statistics and Data Science Education” was supported by my wonderful committee: Andrew Ho (chair), Luke Miratrix, and Sebastian Munoz-Najar Galvez.

Much of my recent and ongoing research focuses on the teaching of data science and statistics, as in the following three projects:

I employ text analysis and psychometric methods to explore what is being taught in college statistics and data science courses, using a large dataset of 32,000 course syllabi. I develop a new framework for developing measurement instruments for text corpora, and use this to create a measurement for modern vs. traditional pedagogical approaches, which I then use to explore continuity vs. change over nearly a decade of data: 2010-2018.
I use latent profile analysis and visualizations to describe and characterize the population of US students who take statistics while in high school.
I carry out an original mixed methods study measuring the interpretability and trustworthiness of machine learning models among highschoolers, as well as exploring their attitudes towards ethical questions around AI’s role in society.

During my time at Harvard, I developed and taught a new course titled “Big Data, Surveillance, Algorithmic Bias, and Ethics in Education Data Science” with my inimitable colleague Avriel Epps-Darling, offered for graduate students at Harvard in the spring of 2022. I also worked with my adviser, Professor Andrew Ho, in assessing validity evidence for state test score uses, creating recommendations and tools for an appropriate role for testing in a COVID-disrupted era, teaching psychometric methods, and updating the Stanford Education Data Archive.

I also have experience across government, industry, and academia, including work with ETS, the Behavioural Economics Team of the Australian Government, the National Center for the Improvement of Educational Assessment, the World Bank, a successful state lawsuit for education funding equity, and several years of teaching, mentoring, and course development. In my free time, I enjoy baking, tea, volleyball, and hosting board game nights.

Interests

Training Ethical and Excellent Data Scientists
Conducting Research on Statistics & Data Science Education
Evaluating Uses of Testing in Education

Education

PhD in Education Policy and Data Science, 2024

Harvard University
MA in Education, 2020

Harvard University
BA in Statistics and Computer Science, 2018

Harvard College

Previous Teaching Experience

Original Course: HGSE A218A: Big Data, Surveillance, Algorithmic Bias, and Ethics in Education Data Science

I was an Instructor at Harvard for my original course “Big Data, Surveillance, Algorithmic Bias, and Ethics in Education Data Science”. I co-designed and co-taught this course with my inimitable colleague Avriel Epps-Darling. I have since shared one of the resources we developed at a teaching conference. I am very proud of this course, and am happy to talk more about our lessons learned in teaching it. Our course summary:

Data permeates education. Multimodal learning analytics, personalized learning, and data-driven autonomous decision making have all gained widespread acceptance, but is it ethical to collect millions of data points from children? Is this data collection and data use fair? Ethical? Oppressive? Who gets to decide?

In this course, we will uncover the choices, biases, and assumptions that permeate data pipelines, from data collection through labeling, modeling, de-identifying, publishing, and model deployment. Students will develop their critical awareness of ongoing issues in this space, gain familiarity with a suite of ethical and ideological frameworks that can be used to evaluate cases, and develop skills in writing and speaking about these topics.

We will do this in the context of several case studies, including the use of machine learning algorithms for identifying “at-risk” students by merging disciplinary and welfare data, surveillance of student data on school devices and networks, the statistical risks associated with anonymizing data by suppressing sample minorities, and the challenges of assessing algorithmic fairness in the context of proprietary data and black-box algorithms. Throughout, we will discuss themes of cost-benefit analysis, the different stakeholder interests, responsibility and buck-passing, anti-racism, intersectionality, and the role of the law.

My other teaching experiences are detailed below:

Statistics: I served as a Teaching Fellow with Prof. Luke Miratrix for Stat 151/HGSE S043: Multilevel and Longitudinal Models (Fall 2021). I co-taught review sections and provided feedback on student work, including students’ original research for their final projects, one of we since published with me as a co-author. I received a Certificate of Distinction in Teaching for my work in this course, based on outstanding student evaluations.

I have also served as a Teaching Fellow with Dr. Joe McIntyre to teach HGSE S010B: Designing Surveys and Questionnaires: Principles and Methods, where I taught weekly review sessions and gave feedback on student projects. I worked with Prof. Kerrie Nelson as a Course Assistant for Harvard Stat 100: Introduction to Quantitative Methods for the Social Sciences and Humanities (Fall 2017). I planned and taught weekly sections, graded problem sets and exams, and held weekly office hours. I served as a Peer Tutor in 2017 for Harvard Stat 111: Introduction to Statistical Inference.

Mathematics: I served as a Teaching Fellow with Dr. Brendan Kelly for Math Q: Quantitative Analysis for Economics and the Social Sciences (Fall 2022 and Spring 2023). I helped to build out the course’s curriculum and resources in data analysis, including randomization-based statistical inference, R programming, and statistical modelling. I previously served as a Quantitative Reasoning Mentor at the Crimson Summer Academy (Summer 2018), a Harvard program for low-income, high-achieving secondary school students from the Cambridge/Boston area. I also served as a Teacher Aide in high school Maths, Statistics, and Calculus classes at Otumoetai College in New Zealand (2015).

Text Analysis: I worked with Prof. Sebastian Munoz-Najar Galvez to build and teach a new spring course, HGSE S59: Quantitative Text Analysis for Education Research (Spring 2021). I served as the course’s Teaching Fellow, writing and teaching weekly lab sessions, grading student work, and helping to guide final projects. We also worked together in the fall of 2020 to develop the course syllabus, assignments, and pedagogy.

Data Science: I served as a Teaching Fellow with Prof. Luke Miratrix to teach HGSE S22: Introduction to Statistical Computing and Data Science in Education (Spring 2021). I taught fortnightly lab sessions, supported students with office hours, and provided feedback on student work and final projects.

Psychometrics: I served as a Teaching Fellow with Prof. Andrew Ho for HGSE S61: Statistical and Psychometric Methods for Educational Measurement (Fall 2020). I co-taught review sections and provided feedback on student work.

Policymaking: I served as the Senior Teaching Assistant with Dr. Teddy Svoronos’ for Evidence for Decisions (Summer 2019), a Harvard Kennedy School course in the new Public Leadership Credential. I managed a team of three other teaching assistants, liaising with program staff, providing feedback on student work, and holding office hours to support learners.

Ethics: In addition to building out my original data ethics course (described above) I have twice served as a Teaching Fellow with Prof. Jim Waldo and Prof. Mike Smith for Applied Computation 221: Critical Thinking in Data Science (Spring 2019 & Spring 2022), a course required for students in Harvard’s Master of Science in Data Science program. I graded and provided feedback on student essays and problem sets, held office hours, and advised final projects.