EECS-343 Operating Systems, Fall 2017

Announcements

  • The final exam will be held at the time and location specified by the Office of the Registrar - December 6, from 4-6PM (Annenberg Hall G21).
  • Here is our Canvas page where we will post assignments and grades.
  • Go to our Piazza course page for the most up-to-date announcements.
  • Links to papers on why you should consider closing your laptop in class in Resources

Staff Information

Professor

Fabián E. Bustamante
Technological Institute, L465
+1 847 491-2745
email

TAs

Michael Leonard (email)
Danqi Liao (email)
Frank Wang (email)
Alexander Weiss (email)
Jennie Werner (email)

Location and Time

Lectures: Mondays and Wednesdays 8:00AM-9:20AM
Annenberg Hall G21

Discussion Sessions:Wed. 6:00-6:50
Tech Institute L361

Professor Office Hours: by appointment

TA Office Hours:TBD

Final Exam:Wed. Dec. 6, 7-9PM

Catalog Description

An introduction to basic concepts in modern operating systems. Topics include operating systems structures, processes, threads, synchronization, deadlocks, CPU scheduling, and memory management.

Prerequisites

Special accomodations

In compliance with Section 504 of the 1973 Rehabilitation Act and the Americans with Disabilities Act, Northwestern University is committed to providing equal access to all programming. Students with disabilities seeking accommodations are encouraged to contact the office of Services for Students with Disabilities (SSD) at +1 847 467-5530 or This email address is being protected from spambots. You need JavaScript enabled to view it. . SSD is located in the basement of Scott Hall. Additionally, I am available to discuss disability-related needs during office hours or by appointment.

Operating systems control all of a computer's resources and present users with the equivalent of virtual machines that are easier to program than their underlying hardware. This course provides an overview of fundamental operating system principles, complemented with discussions of concrete modern systems to help you understand how these principles are applied in real OSs. Topics covered include an overview of the components of an operating system, mutual exclusion and synchronization, implementation of processes, threads, scheduling algorithms, memory management and file systems.

Although the main learning objective of the course is to understand the requirements, design and implementation of modern operating systems, at a higher level the course aims to provide you with a good grasp of basic abstractions employed in system-level software (such as processes, threads, virtual memory, caching, etc.), while uncovering the "magic" that happens inside the box.

The course has a strong project component intended to provide essential experience in designing and implementing complex systems and working as part of a team. In designing the projects and estimating their required effort/hours, I am assuming you are (1) familiar with basic computer organization and data structures and (2) capable of programming in C in UNIX (or UNIX-like) systems (experience with pointers, explicit dynamic memory allocation, multi-file projects, etc.).

Course Organization

The course is organized as a series of lectures, TA sessions, reading, quizzes/homeworks, projects and exams:

  • Lectures - A set of lectures through which I present the core of the material.
  • TA Sessions - Discussion sessions held by the TAs to answer questions about the lecture, readings, homework assignments, and projects.
  • Readings - Textbook reading in preparation for (not substitution of) the lecture and additional reading for those interested in delving further into some topics. The outline includes the reading assignments. Each exam, midterm and final, will include two extra credit questions based on the reading list.
  • Homework/Quizzes - A series of homework-type questions from (or similar to those in) your textbook, aimed at reinforcing the material covered in the reading and the lectures. This year we will try short in-class quizzes instead, with exactly the same goal as homework assignments. A last homework assignment on the reading material for our last lecture. Collectively, these assignments serve as a sample final.
  • Projects - Four programming projects to give you a better understanding of the subject matter and some experience with system level programming including thread-level programming.
  • Exams - A midterm and a final exam covering the material presented in lectures, homework assignments and projects. Each exam, midterm and final, will include two extra credit questions based on the reading list. You'll be able to bring one page of notes to assist you during the exam.

Communication Channels

There are a number of communication channels set up for this class:

  • We use the course web site and Canvas to announce and post homework assignments, projects, etc. You should check this regularly.
  • We use Piazza for class discussion and announcements. The system is highly catered to getting you help fast and efficiently from classmates, the TA, and myself. Rather than emailing questions to the teaching staff, I encourage you to post your questions on our Piazza. To register go here.
  • Finally, there is always email for questions that would be inappropriate to post on the newsgroup (source code being a good example).When using email to contact the staff, please start your subject line with "eecs343: helpful-comment" to ensure a prompt response.

Grading

I use a criterion-referenced method to assign your grade; in other words, your grade will be based on how well you do relative to predetermined performance levels, instead of in comparison with the rest of the class.

Total scores (between 0 and 100) will be determined, roughly, as follows:

  • Homework/Quizzes 20%
  • Projects 40%
  • Exams (20% each) 40%

A note about class participation: while not explicitly included as an item in the previous list, your participation in class will be taken into consideration throughout the quarter and when granting partial and final scores/grades.

To check your grades you can use this form.

Policies

Late policy

Unless otherwise indicated, homework assignments and projects are due by midnight on their due date. If you hand in an assignment late, we will take off 10% for each day (or portion thereof) it is late. Assignments that are three or more days late receive no credit.

Cheating vs. Collaboration

Collaboration is a really good thing and we encourage it. On the other hand, cheating is considered a very serious offense. When in doubt, remember that it's OK to meet with colleagues, study for exams together, and discuss assignments with them. However, what you turn in must be your own (or for group projects, your group's own) work. Copying code, solution sets, etc. from other people or any other sources (yes, that includes the Web!) is strictly prohibited. All work on the homework assignments and exams must be done individually without help of any kind. You may ask clarifying questions of the TA's and the instructor.

Please read the McCormick's Academic Integrity policy.

Remember Ignorantia juris non excusat!

Course Outline and Approximate Dates

Because one has to be an optimist to begin an ambitious project, it is not surprising that underestimation of completion time is the norm.
-- Fernando J. Corbató, ``On Building Systems that Will Fail'', 1990 Turing Award Lecture.
Lampson and Redell, 1980
Date Topic (slides) Reading
09/20 Introduction [PDF] OSC 1; MOS 1.1-1.2
09/25 Architecture [PDF] OSC 1, 2; MOS 1.5-1.7
Redell et al., 1980
09/27 OS Concepts & Structure [PDF] OSC 2; MOS 1.5-1.7
10/02 Processes [PDF] OSC 3; MOS 2.1
10/04 Scheduling I [PDF] OSC 6; MOS 2.4
10/09 Scheduling II [PDF] OSC 6; MOS 2.4
C. Waldspurger et al., 1994
10/11 Memory Management [PDF] OSC 8; MOS 3.1-3.2
10/16 Virtual Memory [PDF] | [PDF] OSC 9; MOS 3.3-3.7
H. Levy and P. Lipman, 1982
10/18 Virtual Memory [PDF] OSC 9; MOS 3.4-3.6
10/23 Midterm
10/25 Virtual Machines [PDF] OSC 9; MOS 8.3
10/30 Threads [PDF] OSC 4; MOS 2.2
Andereson et al., 1991
11/01 Synchronization [PDF] OSC 5; MOS 2.3, 2.5
11/06 Synchronization [PDF] OSC 5; MOS 2.3, 2.5
11/09 Concurrency Bugs [PDF] OSC 7; MOS 6
11/13 I/O [PDF] OSC 13, 10; MOS 5.1-5.4
11/15 File Systems [PDF] OSC 11; MOS 4.1-2
Rosenblum and Ousterhout, 1991
11/20 File Systems [PDF] OSC 12; MOS 4.3-5
11/22 Distributed Systems [PDF] OSC 12; MOS 4.3-5
11/27 Research [PDF] Litton et al., OSDI 2016
11/29 Review [PDF] -
12/06 Final - 7-9PM -

Calendar

This calendar shows the dates for the course, including lectures, assignments, exams, etc.

Homework

There will be two kinds of homework assignments given throughout the class: reading assignments and textbook-style questions. This year we will try a new model for homework assignments, we will do 1-2 homework questions, in class, once a week. You will have 5-10' to answer and I will go over the question/answer after that. The goals is to ensure you keep up with the assigned reading.

Projects

As you can deduce from the allocation of weights for grading, programming projects make up a major portion of this class. There will be four (4) projects. Except for project 1, all projects are to be done by teams of two to three (2-3) people. Team members should work cooperatively on the design, implementation, and testing of their solution.

This quarter you will have the chance to work on a real operating system - xv6. This is a port of a classic Unix to a modern architecture (x86) inspired by the famous Lyon's Commentary on Unix 6th edition. Here is a good page we put together to get you started.

The following table of deadlines should serve you as a guideline for planning your quarter. You are given about two weeks per project. This should be plenty of time if managed carefully. Keep in mind that projects cannot be handed in more than three days late (check the course late policy for details).

You can get a copy of the handout and skeleton for any of your projects here. The skeleton for any project would be available in the format blah.tar.gz. You should save (transfer) this file to your Unix box and run % gunzip blah.tar.gz and % tar xvf blah.tar, in that order. This will create a skeleton directory that will include the source code and makefile for the skeleton as well as the associated regression testing framework.

Info on the EECS Wilkinson and T labs.

ProjectOutInGrade
Project 1: System calls and processes 09/21 10/02 TBD
Project 2: Memory 10/10 10/19 TBD
Project 3: Threads 10/31 11/09 TBD
Project 4: File System 11/14 11/23 TBD

Code Walkthrough

As part of every project grading a subset of 2-8 teams will be randomly selected to conduct a walkthrough of their code for the TAs and instructor. A walkthrough is a form of software peer review in which a designer or programmer leads participants through the software product, while the participants ask questions and make comments on code functionality, style, compliance to standards, etc.

Note: To be granted full credit for a project you must be able to carry the walkthrough and unquestionably show a full understanding of your own code.

A few points on the topic

  • For each of the four projects during the quarter, between 2-8 teams will be selected to conduct a walkthrough
  • The selection will be random with replacement; in other words, your team could be selected multiple times over the quarter
  • The walkthrough must be done in the first week immediately following submission
  • All team members must be present; one of the members will conduct the walkthrough while the other(s) take note of the questions and comments
  • Every participant (TAs, instructor and members of other teams) can ask question about and comment on the code
  • All members of the team conducting the code-review must drive the walkthrough at different times during the meeting
  • Members of other teams present during a code review session are require to provide comments on the code being presented (besides their own). They can submit their comments to the TA after the session for a total of 2.5% extra (from total) points.

Materials

Textbook

There is no "official" book for the course. A few books we have used in the past, in no particular order:

Reading List

A set of papers providing a deeper historical perspective of operating systems, in-depth treatment of some of the topics I can only briefly cover in class, and some useful practical advice in designing and implementing complex systems.

  1. The IBM 701 Computer at the General Motor Research Laboratories, George F. Ryckman, IEEE Annals, Apr-Jun 1983.
  2. An Experimental Time-Sharing System, Fernando J. Corbato, Marjorie M, Daggett and Robert C. Daley, Spring Joint Computer Conference (AIEE-IRE), May 1962.
  3. Pilot: an operating system for a personal computer, D. Redell, Y.K. Dalal, T.R. Horsley, H.C. Lauer, W.C. Lynch, P.R. McJones, H.G. Murraly and S.C. Purcell, Communication of the ACM, February 1980. [Exam question]
  4. Lottery scheduling: Flexible proportional-share resource management, Carl Waldspurger and William Weihl, Proc. of First USENIX Symoium on Operating Systems Design and Implementation, November 1994.[Exam question]
  5. Virtual Memory management in the VAX/VMS Operating System, Henry M. Levy and Peter H. Lipman, IEEE Computer, March 1982. [Exam question]
  6. The Slab Allocator: An Object-Caching Kernell Memory Allocator, Jeff Bonwick, Proc. of USENIX Summer, 1994.li>
  7. The Working Set Model for Program Behavior, Peter J. Denning, Proc. of the First ACM Symposium on Operating Systems Principles, October 1967.
  8. Scheduler Activations: Effective Kernel Support for the User-Level Management of Parallelism, Thomas E. Anderson, Brian N. Bershad, Edward D. Lazowska and Henry M. Levy, Proc. of the ACM Symposium on Operating Systems Principles, October 1991. [Exam question]
  9. Experiences with Processes and Monitors in Mesa, Butler W. Lampson and David D. Redell, Communications of the ACM, 23(2):105-117, February 1980. [Exam question]
  10. Design and implementation of the log-structured file system, Mendel Rosenblum and John Ousterhout, Proc. of the ACM Symposium on Operating Systems Principles, October 1991. [Exam question]
  11. Light-Weight Contexts: An OS Abstraction for Safety and Performance, James Litton, Anjo Vahldiek-Oberwagner, Eslam Elnikety, Deepak Garg, Bobby Bhattacharjee, and Peter Druschel, Proc. of the USENIX Symposium on Operating Systems Design and Implementation (OSDI), October 2016.[Required]

And here is a list of our readings (and links) over the years ...

Reading papers

When reading papers it is normally useful to write down a summary of about a page. Your summary should include at least:

  1. Paper title and its author(s).
  2. Brief one-line summary.
  3. A paragraph of the most important ideas: perhaps a combination of their motivations, observations, interesting parts of the design, or clever parts of their implementation.
  4. A paragraph of the largest flaws; maybe an experiment was poorly designed or the main idea had a narrow scope or applicability. Being able to assess weaknesses as well as strengths is an important skill for this course and beyond.
  5. A last paragraph where you state the relevance of the ideas today, potential future research suggested by the article, etc.

Reference Material

  • Advanced Programming in the Unix Environment, R. Stevens, Addison-Wesley, 1992. A fantastic reference book for anyone writing programs that run under Unix (Highly recommended). There is also a nicely updated second edition out there.
  • The Practice of Programming, Brian W. Kernighan and Rob Pike. Addison-Wesley, 1999. There is more to writing a program than getting the syntax right, fixing the bugs you have noticed, and making it run fast enough. Programs are read not only by computers but also by programmers. A well-written program is easier to understand, grade, and modify than a poorly-written one. This book is packed with great practical advice on style, design, interfaces, testing and debugging, maintenance, ...; all issues that programmers must deal with in the real world.
  • The C Programming Language, 2nd Ed., B. W. Kernighan and D. M. Ritchie, Prentice Hall, 1988. ``The'' reference book.
  • Chapter 12 Kernel Memory Allocation, In UNIX Internals: The new frontier, Uresh Vahalia, 1996. A nice presentation of KMA, algorithms and their evaluation.
  • Concurrent Data Structures, Mark Moir and Nir Shavit, In Handbook of Data Structures and Applications, CRC Press, 2004. A useful reference, if a bit out-of-date, on concurrent data structures.
  • Xv6: a simple Unix-like teaching OS - source code and book (another link with all version)

Why you should consider closing your laptop:

Some interesting videos:

  1. Timesharing: A Solution to Computer Bottlenecks, an interview to Fernando J. Corbato by MIT Science Reporter John Fitch (Recorded January 7, 1963).
  2. The Final Demonstration of the Xerox 'Star' Computer, David Liddle (Recorded June 17, 1998).

Potentially useful links and tools

Here is a list of potentially useful links: