Madhu Sudan: Harvard CS 121, Fall 2020: Introduction to Theoretical Computer Science

General Info

  • Lectures: TuTh 10:30am-12pm. Access via canvas → zoom → Lectures. Attending lectures is highly recommended!
  • The Teaching Team.
  • Discussion Site: CS 121 Piazza
  • Gradescope: Needed for submitting your problem sets.
Staff

Teaching team: Harvard CS 121: Introduction to Theoretical Computer Science (Fall 2020):

  • Instructors:
    • Madhu Sudan. madhu@cs.harvard.edu
    • Adam Hesterberg. ahesterberg@g.harvard.edu

We’re very excited to have an amazing cohort of TFs and Patel Fellows associated with the course!

  • Head TF: William Burke. wburke@g.harvard.edu
  • Extension school TF: Amy Wang.
  • Patel Fellows:
    • Serena Davis. serenadavis@college.harvard.edu.
    • Nari Johnson. njohnson@college.harvard.edu.
  • TFs:
    • Joanna Boyland. jboyland@college.harvard.edu.
    • Diego Gutierrez. diegogutierrez@college.harvard.edu.
    • Ife Omidiran. omidiran@college.harvard.edu.
    • Zuzanna Skoczylas. zuzanna_skoczylas@college.harvard.edu.
    • Noah Singer. noahsinger@college.harvard.edu
    • Sahana Srinivasan. sahanasrinivasan@college.harvard.edu
    • Alec Sun. suna@college.harvard.edu
    • Richard Xu. raxu@college.harvard.edu.
Policies and Expectations (aka “syllabus”). Note that Tablets + Styluses are required for active participation in sections and office hours.

Grades will be based on the following

  1. Exams: 65% (Midterm 1 – 15%, Midterm 2 – 20%, Final – 30%).
  2. 6 Problem Sets. (30%)
  3. Class Participation. (5%)

Expectations

  1. Attendance in lecture is strongly recommended but not required. Students unable to attend lecture are expected to watch the videos within 24 hours of posting, and follow up on this in sections and office hours.
  2. Attending one section per week is also strongly recommended.
  3. Office hours are available for students to engage further in the material and to get help on material. It will also be another avenue for getting more “live” interaction in a remote semester.
  4. The class discussion board is another way to participate. Please use the forum not only to ask questions, but also answer others’ questions! Course announcements will be made on Piazza, which students are expected to check.
  5. Participation points can be earned through any of the above. What we seek in “participation” is evidence of effort to learn or teach the material, and not evidence of mastery of the material. So do not hesitate to ask what may seem like a dumb question to you in any setting where you are able to participate.
  6. Problem sets: There are detailed instructions on problem sets on what collaboration is allowed and what is not. In a nutshell, you are encouraged to collaborate when thinking about solutions. But no collaboration is allowed when writing solutions up!

Websites and Materials

  • The course website has information about the course.
  • Assignments and exams will be submitted on Gradescope.
  • Some course materials will be on Canvas.
  • Course discussion and some announcements will be on Piazza.
  • The course uses Barak’s free textbook. Some students also use Sipser’s book, which isn’t required. We strongly recommend that you read at least one.
  • On the course calendar we list the topics of each lecture and sections of the former and, where applicable, the latter book corresponding to each lecture.
  • Lectures, Sections, and Office Hours use Zoom, and have components that use shared whiteboards.
  • An internet connection good enough to access the above is required. A tablet and stylus is required for maximally effective participation in both sections and office hours.

We’re committed to supporting diversity, inclusion and belonging, and accessibility. The University Disability Office and Extension School Accessibility Office offer a variety of accommodations and services to college and extension school students, respectively, with documented disabilities. We seek to create a learning environment that supports a diversity of thoughts, perspectives and experiences: if you feel uncomfortable sharing your thoughts with the class, please tell us. We include and support students and staff of many identities (including race, gender, class, sexuality, socioeconomic status, religion, and ability), and ask that you similarly support your fellow students. If something was said in class (by us or anyone else) that made you feel uncomfortable, please talk to us about it. If you feel like your performance in the class is being impacted by your experiences outside of class, please don’t hesitate to come and talk with us. (Credit: based on the statements of Professor Boaz Barak and Dr. Monica Linden at Brown University.).

Prerequisites and mathematical background

Formally, the prerequisites for this course are “experience in formal mathematics at the level of CS 20”. Read chapter 0 of Barak’s textbook and see this page to understand what this means and how to prepare yourself for the course. Homework 0 (texpdf), which is due on the first day of class (but not part of your problem set grade) is intended to help you diagnose your mathematical background; we’ll grade it and talk with you about it if appropriate.

Problem sets

Theoretical problem sets require thinking and take substantial and unpredictable amounts of time, so you should make sure to allow for sufficient time to work on them.

Problem sets are due at 11:59pm ET on the due date posted on the course calendar. All problem sets must be submitted as PDF files through Gradescope. We recommend that the PDF be produced by LaTeX or Markdown. You may extend your deadline for a problem set by one day up to eight times, at most twice per problem set, by submitting late; problem sets late beyond this will not earn credit. You are responsible for keeping track of your own late days.

You are allowed to collaborate with other students currently taking the course, individually or by the discussion board, in discussing and brainstorming the solutions, but when you are done talking, you must write up your solutions independently and may not check them against each other. There may be no passing of homework papers between collaborators; nor is it permissible for one person simply to tell another the answer. You should list the names of all the students you collaborated with in the first page.

Each problem set has some “bonus problems” which don’t count for your pset grade, although we might use them when recommending departmental honors and writing letters of recommendation.

All regrade requests should be made via Gradescope no later than midnight on Sunday after the pset is returned. Handling regrade requests takes a lot of time from the teaching staff, time that could be spent on answering student questions, preparing course material, etc.. I encourage you to be judicious in such requests, and only reserve it for the case of clear mistakes by the grader (which can happen – we’re not perfect!). Needless to say, a regrade request triggers re-examining the problem set, which may result in a lower grade as well. Also, we will not accept a regrade request of the form “Johnny made a similar mistake and got fewer points deducted” unless Johnny is willing to have us regrade his problem set as well.

Midterms: The two midterms will be in two of the normal lecture time slots: you’ll download them at the start of lecture, spend lecture time working on them, and submit your solutions at the end. They’re unproctored; we’ll have you sign an honor statement that you didn’t use unallowed resources (extra time, other humans, computers, or the internet). Even if you don’t regularly attend lecture, you’ll need to attend those to take them. If you cannot attend the midterm for a valid reason, contact us and we will schedule a makeup. The midterms account for 15% and 20%, respectively, of your final grade.

Final: You can take the final in any three-hour block from 12-10 1:21pm to 12-12 1:21pm, by downloading it and sending us your answers 3 hours later. In the first four hours of that block we guarantee that we’ll be monitoring Piazza for questions and trying to respond in real time; you’re welcome to take it outside that time, but responses to any questions you have for staff about the exam may be too slow to be useful. The final exam will count for 30% of the grade.

Participation: Each student is expected to participate in at least two of lectures, sections, office hours, and the class discussion board; participation is worth 5% of your grade. (Anonymous or private postings don’t count for class discussion board participation credit. We encourage public posting for the benefit of other students who share your questions but posts containing your solutions to assignments should be private. You’re welcome to post anonymously, not for participation credit, if you’d like.)

CS 121.5

While this course has enough content and work to satisfy most students, some students might be interested in deeper explorations of theoretical computer science, and to learn more on advanced topics such as proof systems, derandomization, quantum computing, circuit complexity, communication complexity, error correcting codes, algorithms for learning and online optimization, and more. For these students we will hold an “advanced section” (“CS 121.5”), whose goal would not be to review material taught in class but rather to explore every week an advanced topic related to this week’s lectures.

Collaboration and academic integrity

Collaboration is highly encouraged in this course. We welcome you to form study groups, share relevant resources you found online, and talk about the concepts in this course. Several parts of the course are designed to help you form study groups: the second and fourth fifteen-minute blocks of each lecture, group work in sections, office hours, and the class discussion board.

However, there is a certain component of learning that is only achieved through the process of thinking deeply on a problem and (figuratively!) “banging your head against the wall”. So, for the problem sets themselves, you should follow the collaboration policy closely and make sure you “own your solutions”. If you get stuck on a problem set question, ask questions on the discussion board or come to office hours, and TFs will help you get “unstuck”. They will not give you hints, but rather review material from lecture and work through more examples until you are in a better place to continue your explorations. However, keep in mind that we do not expect you to solve every single pset question, and as long as you give it a good try, you will learn from the process, which is ultimately the most important thing.

We expect all students to adhere to the Harvard honor code. Some examples of activities that violate academic integrity include:

  1. Copying another student’s answer or sharing your answers for a quiz, problem set, midterm, or final. Brainstorming on problem set questions with other students is OK, sharing written solutions is not.
  2. Discussing problem sets with anyone except a current student in the course or a member of the teaching team.
  3. Not citing a person or resource that helped you in the problem set.
  4. Posting problem set questions or answers online, or sharing them with students that are not currently enrolled in this course.
  5. Using problem set solutions from previous offerings of this course.
  6. Searching online for answers for particular problem set questions. It is completely fine to find and use general resources online on the course material. You should however cite any resource you used in your answer.
  7. Searching online for answers for the online quizzes. You should only attempt the quiz after you have read the text fully, and the text itself contain all information you need to answer it.
  8. Understating the amount of late days you used in the problem set declaration.

Note that if we suspect an honor code violation, it could take us time to investigate and verify it, and so you may only find out about the issue very late in the term, or even after it ends. If you are worried that you might have inadvertently violated the honor code, it’s always best to come to us and discuss this.

If you are not sure if something is an honor code violation or not, please ask us!

Falling behind

If you are starting to have difficulties in this course we recommend that you come talk to us (the instructor or the TF’s) before you are so far behind that it is impossible to catch up. We want you to succeed in this course and are here to help you do so! We recommend that students also reach out to the Patel Fellows associated with CS 121 for help.

Patel Fellows

Nari Johnson and Serena Davis will be the Patel Fellows for CS121. The aim of the Patel fellowship is to provide extra help for Harvard College students who are having trouble in the class, typically as evidenced by performance below the class average. If you think that you are someone who would benefit from engaging with a Patel fellow (typically in the form of 1-on-1 or small group tutoring), you should reach out to the Patel fellows using this form. Please feel free to direct any additional questions to Nari and Serena at cs121.fall2020.patel.fellows@gmail.com. Please be aware that our Patel Fellows have limited capacity and are only able to work 12 hours per week, and so we will determine eligibility by helping students who are struggling the most.  We also encourage anyone who would like additional tutoring support to sign up for tutoring at the Academic Resource Center.

Student well being

Your physical and mental well being is very important to us. If you have any concerns or issues in this course, please reach out to us (Madhu and Adam), the teaching fellows, your resident dean, or Harvard services such as Counseling and Mental Health Services (phone: 617-495-2042 (on business hours), 617-495-5711 (all other times), see also Let’s Talk), the confidential peer counselling program Room 13 (phone: 617-495-4969), and the Beuro of Study Council (phone 617-495-4969).

If you have a serious medical or other emergency, please have your resident dean contact the course instructor. We will try to accommodate you as much as possible. For extension students, please contact the instructor and/or the extension TFs directly.

If you have a health condition that affects your learning or classroom experience, please let me know as soon as possible. Please also share with me any letter you have from the AEO so we can provide the appropriate accommodations.

Announcements

  • 11/19/2020: Homework 6 out. Due 12/03/2020.
  • 11/18/2020: Advanced section Thursday at 4:30pm: Roei Tell on Randomness, Pseudorandomness and Derandomization. Details here.
  • 11/08/2020: Midterm 2 review material here.
  • 10/28/2020: Homework 5 out . Due 11/12/2020. Solutions.
  • 10/21/2020: Midterm 1 Solutions: (texpdf).
  • 10/15/2020: Please take some time to respond to the Midterm Feedback Survey. Its mandatory. Its anonymous. Due Sunday (10/19).
  • 10/15/2020: Homework 4 out. Due 10/29/2020. Solutions.
  • 10/6/2020: (updated 10/8): Midterm 1 in one week (10/13/2020, “in class” = 10:30-12noon as preferred time).
  • Logistical announcement to follow here.
  • Prep Material here.
  • Sample midterm here – do look at it to familiarize yourselves with format of exam, and to practice latex’ing your answers.
  • Review material (questionssolutions).
  • 10/1/2020: Homework 3 available now. Due 10/08/2020. Solutions.
  • 9/24/2020: Reading for today’s and next two lectures include Sections 6.2, 6.4 and 6.5 from Barak’s IntroTCS. Despite what the book says, they are not optional for us!
  • 9/17/2020: Homework 2 is available now. Due 10/01/2020 (corrected). Solutions.
  • 9/10/2020: Advanced sections start today at 4:30pm! Boaz Barak is our speaker today. Here are future plans (more talks will be added).
  • 9/10/2020: Homework 1 is available now. Due 9/17/2020. Solutions.
  • 9/2/2020: First lecture is tomorrow (9/3) at 10:30am. All lectures/sections/office hours are accessible from canvas → zoom. Here is a full calendar with section times and office hours included.
  • 9/2/2020: Homework0 is due tomorrow by 11:59pm. Please submit via gradescope. You should have received a link by now. If not send a private post on Piazza.
  • 9/2/2020: Please sign up for active participation. See this post for details.
  • 8/15/2020: Here‘s a sample video (with slide deck) based roughly on the contents of lecture 1. It would be a good to quickly watch the video (skipping parts you don’t care about) to get a sense of topics, style etc. before coming to the Q&A session below. We will *not* watch the video in the session.
  • 7/20/2020: Homework Zero is available now. You can start working on it right away! Use latex to write up your solutions. Hold on to your solutions for now while we figure out the next steps. It is due by September 3, 2020. Solutions.

Topics (Tentative), Calendar and Handouts

  • Lecture 01 (Th. 09/03): Introduction. (Reading: Chapter 0. (All readings refer to the text by Barak, unless specifically indicated.)) Slides. (Slides after edits in class.) Video.
  • 09/03: Section 0 week starts. Video here. Boards: 1234Solutions.
  • 09/03: Problem Set 0 Due.
  • Lecture 02 (Tu. 09/08): Math. Background. (Reading: Chapter 1.) SlidesSlides after lecture (with solutions to exercises). Video.
  • Lecture 03 (Th. 09/10): Representing Information: Objects as Strings. (Reading: Chapter 2.) SlidesSlides after lecture. Video.
  • 09/10: Section 1 week starts. VideoSlides. Boards: 123456Solutions.
  • 09/10: Problem Set 1 Out.
  • Lecture 04 (Tu. 09/15): Defining Computation: Circuits. (Reading: Chapter 3.) Slides (pptxpdf). Slides after lecture. Video.
  • Lecture 05 (Th. 09/17): Completeness: Computing every (finite) function. Size of circuits. Upper and Lower bounds. (Reading: Chapter 4.) Slides (pptxpdf). Slides after lecture. Video.
  • 09/17: Problem Set 1 Due. Problem Set 2 Out.
  • 09/17/2020: Section 2 week starts. VideoSlidesSolutions.
  • Lecture 06 (Tu. 09/22): Code as data. Circuit lower bounds. Circuit-interpreter. (Reading: Chapter 5.) Slides (pptxpdf). Slides after lecture. Video.
  • Lecture 07 (Th. 09/24): Infinite functions and algorithms: Finite automata. (Reading: Chapter 6, especially Section 6.2.) Slides (pptxpdf). Slides after lecture. Video.
  • 09/24/2020: Section 3 week starts. VideoSlidesAnnotated SlidesSolutions.
  • Lecture 08 (Tu. 09/29): Finite automata and Regular Functions: Languages. (Reading: Sipser – Chapters 1.2, 1.3. Also Barak Sections 6.3 and 6.4). Slides (pptxpdf). Slides after lecture. Video.
  • Lecture 09 (Th. 10/01): Limits of Finite Automata: What can they not compute? (Reading: Sipser – Chapter 1.4. Also Barak Section 6.5). Slides (pptxpdf). Slides after lecture. Video.
  • 10/01/2020: Section 4 week starts. VideoSlidesSolutions.
  • 10/01: Problem Set 2 Due. Problem Set 3 out.
  • Lecture 10 (Tu. 10/06): Turing Machines:Introduction. (Reading: Chapter 7.1.) Slides (pptxpdf). Slides after lecture. Video.
  • Lecture 11 (Th. 10/08): More Turing Machines: Examples. (Reading: Chapter 7.2.). Slides (pptxpdf). Slides after lecture. Video.
  • 10/08/2020: Section 5 week (Midterm reviews). No Video. Prep material here.
  • 10/08: Problem Set 3 Due.
  • Lecture 12 (Tu. 10/13): Midterm 1.
  • Lecture 13 (Th. 10/15): Turing Equivalence + Universality. (Reading: Chapter 8, Chapters 9.1, 9.2.) Slides (pptxpdf). Slides after lecture. Video.
  • 10/15/2020: Section 6 week. VideoSlidesSolutions.
  • 10/15: Problem Set 4 Out.
  • Lecture 14 (Tu. 10/20): Uncountability and Uncomputability (Reading: Chapters 2.3, 2.4, 9.3.) Slides (pptxpdf). Slides after lecture. Video.
  • Lecture 15 (Th. 10/22): More Uncomputability. Reductions. (Reading: Chapter 9.3.) Slides (pptxpdf). Slides after lecture. Video.
  • 10/22/2020: Section 7 week. VideoSlidesSolutions.
  • Lecture 16 (Tu. 10/27): Rice’s Theorem. More Reductions. (Reading: Chapter 9.4.) Slides (pptxpdf). Slides after lecture. Video.
  • Lecture 17 (Th. 10/29): Efficient Computation: The class P. The Strong Turing-Church Thesis. (Reading: Chapters 12, 13.) Slides (pptxpdf). Slides after lecture. Video.
  • 10/29: Problem Set 4 Due. Problem Set 5 Out
  • 10/29/2020: Section 8 week. VideoSlidesSolutions.
  • *Election Day* Lecture 18 (Tu. 11/03): Polynomial Time Reductions. (Reading: Chapter 14.) Slides (pptxpdf). Slides after lecture. Video.
  • Lecture 19 (Th. 11/05): NP and NP-completeness. (Reading: Chapters 15.1, 15.2.). Slides (pptxpdf). Slides after lecture. Video.
  • 11/05/2020: No video. Midterm 2 Prep material here.
  • Lecture 20 (Tu. 11/10): Cook Levin Theorem. (Reading: Chapters 15.3-15.5.). Slides (pptxpdf). Slides after lecture. Video.
  • Lecture 21 (Th. 11/12): More NP-completeness by reductions. (Reading: Review Chapters 14, 15.) Slides (pptxpdf). Slides after lecture. Video.
  • 11/12: Problem Set 5 Due.
  • 11/12/2020: Section 10 week. VideoSlidesSolutions.
  • Lecture 22 (Tu. 11/17): Midterm 2.
  • Lecture 23 (Th. 11/19): Randomness and Computation: Probability Review. (Reading: Chapter 18.) Slides (pptxpdf). Slides after lecture. Video.
  • 11/19: Problem Set 6 Out.
  • 11/19/2020: Section 11 week. VideoSlides.
  • Lecture 24 (Tu. 11/24): Randomized algorithms: Examples. (Reading: Chapter 19.) Slides (pptxpdf). Slides after lecture. Video.
  • (11/25-11/29): Thanksgiving break.
  • 11/30/2020: Section 12 week. Final review – no videos. Prep material here.
  • Lecture 25 (Tu. 12/01): Modelling Randomized Algorithms. The classes RP and BPP. (Reading: Chapter 20.) Slide (pptxpdf). Slides after lecture. Video.
  • Lecture 26 (Th. 12/03): Course wrap-up: Quantum, Crypto. (Reading: Skim chapters 21, 22, 23.). Slides (pptxpdf). Slides after lecture. Video.
  • 12/03: Problem Set 6 Due.

Reference Materials

  • The textbook for the course is Introduction to Theoretical Computer Science by Boaz BarakHere is the pdf of the book we will be following for the term. (The book sees minor revisions frequently but the pdf version, frozen in August 2020, is our definitive version.)
  • The textbook Introduction to the Theory of Computation by Michael Sipser will cover roughly the same material though in a different order and with quite different notation.  This book is a good source of additional problems and exercises.
  • There are many additional resources and texts mentioned by Boaz Barak at his website for (the Fall 2019 version of) this course. (Scroll down to the bottom on the landing page.) Depending on your time and interest you can sample from these resources. The more perspectives you see, the more you will appreciate the course material!