• Combine Systems, Tools, Theory

    We connect material from Systems area courses (such as PL, OS, and HW) with Tools and Theory related to performance measurement and experimentation.

  • Navigate Across System Layers

    Performance is a property that is best studied from a holistic perspective. For example, memory performance straddles garbage collector, virtual memory subsystem, and cache.

  • Become an Experimenter

    We look at systems as objects of study and explore the factors affecting their performance. We consider workloads, measurement contexts, performance metrics, and data analysis.

Teaching Style How you get most out of this course

Why should you take this class?

This class prepares you to engineer efficient software systems, to detect performance problems and improve the performance of existing systems, and to identify possible new research problems.

How do you learn this?

You learn by trying, finding your own insights. This means the assignments can take a variable amount of time! We want you to formulate your own variations of problems and solve them.

How is the course structured?

  • Readings and videos to prepare before class
  • Interactive lectures
  • Practical assignments after class
  • Course: Midterm and Final exams
  • Lab: Project

Logistics When, where, what and how

Lectures Wednesdays, 10:30 - 12:15 and Fridays, 13:30 - 15:15 in SI-006
Labs Fridays, 15:30 - 17:15 in SI-006
Prerequisites Java programming language experience
Textbook David Lilja, Measuring Computer Performance, Cambridge University Press
Further Resources Check our References page.
Midterm Exam TBA
Final Exam TBA
SP Course Grade Homework: 40%, Midterm: 25%, Final: 35%
SP Lab Grade Homework: 30%, Project: 70%
Where do I ask questions?

Please use our Discussions Forum on Moodle.

Do I need to follow a coding style?

Read our Java Coding Style Guidelines.

Team Instructor and teaching assistants

We are a professor and two PhD students who are members of the Software and Programmer Efficiency research group of the Faculty of Informatics. Besides teaching this course, we spend our days (and sometimes nights) on research in the areas of programming languages, systems, and software engineering.

  • Matthias
    Hauswirth Instructor

    Office SI-207
    Contact Homepage
  • Luis
    Mastrangelo Course TA

    Office SI, 2nd floor, B.1
    Contact Homepage
  • Dmitri
    Makarov Lab TA

    Office SI, 2nd floor
    Contact Homepage

Schedule A one-semester, 14-week course

We have three weekly lecture slots, two for the course (white), and one for the lab (blue). Here is the tentative schedule. We will update it as we progress through the course.

Date Topic Preparation In-Class Activities Assignment out Assignment due
Wed, Sep. 18 Introduction Refresh your Java, C, and Intel Assembly language knowledge Introduction, Informa Session. Lecture 1 Slides. A01. Java & C -
Fri, Sep. 20 PL Program Representation Watch videos on program representation
(~45 minutes of videos, plus some breaks):
Lecture 2 Slides. A02. Simple Bytecode Metrics A01. Java & C
Fri, Sep. 20 Building the Jikes RVM Read the Care and Feeding section of the Jikes RVM User's Guide Install and build the Jikes RVM L01. Getting Started with the Jikes RVM
Wed, Sep. 25 PL Control-Flow Graph Control-flow graphs and dot. Lecture 3 Slides. A03. Control-Flow Graphs with ASM A02. Simple Bytecode Metrics
Fri, Sep. 27 PL Loops Finding loops. Lecture 4 Slides. A04. Dominator Analysis A03. Control-Flow Graphs with ASM
Fri, Sep. 27 The Jikes RVM and Adaptive Optimization Watch introductory videos on Jikes RDB
(~40 minutes of videos, plus some breaks):
Jikes RVM, AOS. Lab 2 Slides. L02. Jikes RVM Performance L01. Getting Started with the Jikes RVM
Wed, Oct. 2 PL Compiler Optimization Compiler optimization. Lecture 5 Slides. A05. Add Exception Edges to Control-Flow Graph A04. Dominator Analysis
Fri, Oct. 4 PL Program Analysis, PL Exceptions Program analysis. Lecture 6 Slides. Exception handling. - -
Fri, Oct. 4 Getting Comfortable with the Jikes RVM Jikes RVM in Eclipse, Lab 3 Slides. L03. Look and Tickle L02. Jikes RVM Performance
Wed, Oct. 9 PL Class Hierarchy, PL Call Graph Pacman project, Lecture 7 Slides. A06. Call Graph with ASM A05. Add Exception Edges to Control-Flow Graph
Fri, Oct. 11 PL Method Call Optimization Read Fast Static Analysis of C++ Virtual Function Calls by Bacon and Sweeney (Section 1 of and most of Section 2) and Optimization of Object-Oriented Programs using Static Class Hierarchy Analysis (CHA) by Dean, Grove, and Chambers (Section 1 and the first part of Section 2) Lecture 8 Slides. Feedback on A02. - -
Fri, Oct. 11 Getting Ready for the Project Feedback on L02. Questionnaire about L03. L04. Project kick-off L03. Look and Tickle
Wed, Oct. 16 PL Memory Management, Part 1 Read An Introduction to Garbage Collection: Part I - The Real Costs of C++ Memory Management Stack, heap, globals, explicit memory management, reference counting. Lecture 9 Slides. A07. Memory Management A06. Call Graph with ASM
Fri, Oct. 18 PL Memory Management, Part 2 Read An Introduction to Garbage Collection: Part II - A Look Under the Hood Heap analysis, NEW, tracing GC. Lecture 10 Slides. - -
Fri, Oct. 18 Project starts. From now on, we'll only hold lab sessions on request. L04. Project kick-off
Wed, Oct. 23 PL Memory Management, Part 3 Memory Leaks. Leak Detection. Lecture 11 Slides. Paper: Low-Overhead Memory Leak Detection Using Adaptive Statistical Profiling Optional A04 Resubmission. Dominator Analysis A07. Memory Management
Fri, Oct. 25 HW Cache Memory hierarchy. Access latency. Direct-mapped vs. set-associative vs. fully-associative caches. Lecture 12 Slides. A08. Cache Simulator Optional A04 Resubmission. Dominator Analysis
Fri, Oct. 25 Prepare demo of seed prototype Demo of seed prototypes, meetings with mentor
Wed, Oct. 30 OS Dynamic Linking Read The Inside Story on Shared Libraries and Dynamic Loading and Optional Program Library HOWTO (Linux article but the concepts are the same) Shared libraries, dlopen/dlsym, library interposition, tracing dynamic linker activity. Lecture 13 Slides. - -
Wed, Nov. 6 Tools PL DTrace Read Dynamic Instrumentation of Production Systems (all 14 pages) DTrace. Lecture 14 Slides. A09. Dynamic Instrumentation with DTrace A08. Cache Simulator
Fri, Nov. 8 OS HW Virtual Memory Virtual memory management. From malloc to mmap. Lecture 15 Slides. - A09. Dynamic Instrumentation with DTrace
Fri, Nov. 8 Prepare demo of improved prototype Demo of improved prototypes, meetings with mentor
Wed, Nov. 13 Midterm exam - -
Fri, Nov. 15 PL Dynamic Class Loading Read Dynamic Class Loading in the Java Virtual Machine (everything except Section 4 "Maintaining Type-safe Linkage") Lazy loading, type-safe linking, multiple namespaces, class unloading, user-defined class loaders (SP-Compiler Eclipse project). Lecture 16 Slides. A10. Practice with Dynamic Class Loading in the JVM -
Fri, Nov. 15
Wed, Nov. 20 PL Dynamic Binary Instrumentation Java instrumentation agent. ASM class rewriting. Lecture 17 Slides. A11. Dynamic Binary Instrumentation A10. Practice with Dynamic Class Loading in the JVM
Fri, Nov. 22 Tools PL R Optionally read Evaluating the design of the R language Variables, functions, vectors, factors, lists, data frames. Lecture 18 Slides. - -
Fri, Nov. 22 Prepare demo of improved prototype Demo of improved prototypes, meetings with mentor - -
Wed, Nov. 27 Tools Theory Descriptive Statistics & Visualization Read Chapters 1 "Introduction", 2 "Metrics of performance" and 3 "Average performance and variability" of the textbook Central tendency & dispersion, PDF, CDF, jitter plot, box plot, histogram, ECDF, quantile/quantile plot. Lecture 19 Slides. A12. Array Size Analysis A11. Dynamic Binary Instrumentation
Fri, Nov. 29 Tools PL ggplot2 Read the ggplot2 Quick Reference Grammar of Graphics, ggplot2. Lecture 20 Slides. - -
Fri, Nov. 29 - -
Wed, Dec. 4 Discussion of Midterm - A12. Array Size Analysis
Fri, Dec. 6 Theory Systematic Error Omitted Variable Bias, Profiler Bias. Lecture 21 Slides. - -
Fri, Dec. 6 Prepare demo of improved prototype Demo of improved prototypes, meetings with mentor - -
Wed, Dec. 11 Theory Random Error Read Chapter 4 "Errors in experimental measurements" of the textbook Accuracy and random error vs. precision and systematic error, confidence intervals. Lecture 22 Slides. - -
Fri, Dec. 13 HW Branch Prediction Read A study of branch prediction strategies Why? Static vs. Dynamic. Bimodal. Two-Level Adaptive. Branch Target Prediction: BTB, RAS. Lecture 23 Slides. - -
Fri, Dec. 13 Prepare demo of improved prototype Demo of improved prototypes, meetings with mentor L05. Project presentation -
Wed, Dec. 18 PL Call Profiling Read gprof: A call graph execution profiler Call graph. Calling context tree. Profiling with gprof, DTrace, hprof. DYI Java stack sampling. Lecture 24 Slides. - -
Fri, Dec. 20 Theory PL Complexity Read Algorithmic Profiling Cost Function, Asymptotic Complexity, Algorithmic Essence, Algorithmic Profiling. Lecture 25 Slides. - -
Fri, Dec. 20 Prepare presentation and demo of final system Presentation and demo of final system L05. Project presentation

Assignment Policy

Group Work

Homework consists of both written and programming assignments. You are encouraged to collaborate on the assignments. We recommend that you work in groups of two. However, you must produce the complete final writing and implementation on your own, and you must fully understand the solution you submit. Moreover, in your submission you must acknowledge all your collaborators.

Late Policy

In general, no late assignments are accepted.

Submission

The handouts of each assignment includes specific submission instructions. Please follow the submission instructions exactly as written!

Where are the assignments?

All assignments are posted on the schedule.