Introduction to Data Science

Statistics and Prediction Algorithms Through Case Studies

Preface

This is the website for the Statistics and Prediction Algorithms Through Case Studies, the second part of Introduction to Data Science.

The website for the first part, Data Wrangling and Visualization with R, is here.

This book started out as part of the class notes used in the HarvardX Data Science Series.

A hardcopy version of the first edition of the book, which combined both parts, is available from CRC Press.

A free PDF of the October 24, 2019 version of the book, which combined both parts, is available from Leanpub.

The Quarto code used to generate the book is available on GitHub.

This work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International CC BY-NC-SA 4.0.

We make announcements related to the book on X. For updates follow @rafalab.

Acknowledgments

Special thanks to Jenna Landy for her careful editing and advice; Amy Gill, Jorge Cornick, Barry MacLean, and Robert Gentleman for dozens of comments, edits, and suggestions; Stephanie Hicks, who twice co-taught my data science classes; and Yihui Xie, who patiently answered my many questions about bookdown. Thanks also to Héctor Corrada-Bravo for advice on teaching machine learning, Alyssa Frazee for helping create the homework problem that became the Recommendation Systems case study, and Danilo Pérez Rivera for a discussion that improved the dimension reduction section. Thanks to Hadley Wickham, Mine Çetinkaya-Rundel, and Garrett Grolemund for making the Quarto code for their R for Data Science book open, and to Alex Nones for proofreading the manuscript.

This book grew out of applied statistics courses taught over more than fifteen years. The teaching assistants who worked with me made important contributions, as did the many students whose questions and comments improved the material. The latest iteration is a HarvardX series coordinated by Heather Sternshein, Zofia Gajdos, and Nicole Sanderson, whom we thank for their contributions. The courses were partially funded by NIH grant R25GM114818, and we are grateful to the National Institutes of Health for its support.

Three books particularly influenced our approach. Statistics, by David Freedman, Robert Pisani, and Roger Purves, influenced our emphasis on statistical reasoning and intuition. Stat Labs: Mathematical Statistics Through Applications, by Deborah Nolan and Terry Speed, inspired our use of real data and case studies to motivate statistical concepts. The Elements of Statistical Learning, by Trevor Hastie, Robert Tibshirani, and Jerome Friedman, provided a foundation for the machine learning and high-dimensional parts. We are grateful to these authors for shaping how we teach this material.

Thanks also to those who contributed corrections and suggestions via GitHub or emails: hbmaclean (Barry MacLean), nickyfoto (Huang Qiang), desautm (Marc-André Désautels), michaschwab (Michail Schwab), alvarolarreategui (Alvaro Larreategui), jakevc (Jake VanCampen), omerta (Guillermo Lengemann), espinielli (Enrico Spinielli), asimumba (Aaron Simumba), braunschweig (Maldewar), gwierzchowski (Grzegorz Wierzchowski), technocrat (Richard Careaga), atzakas, defeit (David Emerson Feit), shiraamitchell (Shira Mitchell), Nathalie-S, andreashandel (Andreas Handel), berkowitze (Elias Berkowitz), Dean-Webb (Dean Webber), mohayusuf, jimrothstein, mPloenzke (Matthew Ploenzke), NicholasDowand (Nicholas Dow), kant (Darío Hereñú), debbieyuster (Debbie Yuster), tuanchauict (Tuan Chau), phzeller, BTJ01 (BradJ), glsnow (Greg Snow), mberlanda (Mauro Berlanda), wfan9, larswestvang (Lars Westvang), jj999 (Jan Andrejkovic), Kriegslustig (Luca Nils Schmid), odahhani, aidanhorn (Aidan Horn), atraxler (Adrienne Traxler), alvegorova, wycheong (Won Young Cheong), med-hat (Medhat Khalil), biscotty666 (Brian Carey), kengustafson, Yowza63, ryan-heslin (Ryan Heslin), SydneyUni-Jim (Jim Nicholls), raffaem, BruciiZ (Huiyuan (Bruce) Zhou), tim8west, aosaf-e-c (Aosaf Ershad Chowdhury), annlia, jonkiparsky (Jon Kiparsky), ang101 (Angela), Daniel-Yonkov (Daniel Yonkov), lilyclements (Lily Clements), David D. Kane, El Mustapha El Abbassi, Vadim Zipunnikov, Anna Quaglieri, Chris Dong, Rick Schoenberg, Isabella Grabski, Doug Snyder, and JT Harton.