• Combine Systems, Tools, Theory

    We connect material from Systems area courses (such as PL, OS, and HW) with Tools and Theory related to performance measurement and experimentation.

  • Navigate Across System Layers

    Performance is a property that is best studied from a holistic perspective. For example, memory performance straddles garbage collector, virtual memory subsystem, and cache.

  • Become an Experimenter

    We look at systems as objects of study and explore the factors affecting their performance. We consider workloads, measurement contexts, performance metrics, and data analysis.

Teaching Style How you get most out of this course

Why should you take this class?

This class prepares you to engineer efficient software systems, to detect performance problems and improve the performance of existing systems, to validate performance models or theories, and to demonstrate or identify the potential of possible new research problems.

How do you learn this?

You learn by trying, finding your own insights. This means the assignments can take a variable amount of time! We want you to formulate your own variations of problems and solve them.

How is the course structured?

  • Readings and videos to prepare before class
  • Interactive lectures
  • Practical assignments after class
  • Project, Midterm and Final exams

Logistics When, where, what and how

Lectures Mondays and Thursdays, 15:30 - 17:15 in SI-013
Labs Fridays, 8:30 - 10:15 in SI-004
Prerequisites Java programming language experience
Textbook David Lilja, Measuring Computer Performance, Cambridge University Press
Midterm Exam November, 12 at 15:30
Final Exam TBA
SP Course Grade Homework: 40%, Midterm: 25%, Final: 35%
SP Lab Grade Homework: 30%, Project: 70%
Where do I ask questions?

Please use our Discussions Forum on Moodle.

Use the references page to search online resource for information.

Read the Java Coding Style Guidelines.

Team Instructor and teaching assistants

We are a professor and three PhD students who are members of the Software and Programmer Efficiency research group of the Faculty of Informatics. Besides teaching this course, we spend our days (and sometimes nights) on research in the areas of programming languages, systems, and software engineering.

  • Matthias
    Hauswirth Instructor

    Office SI-207
    Contact Homepage
  • Dmitri
    Makarov Lab TA

    Office SI-103
    Contact Homepage
  • Matej
    Vitasek Course TA

    Office Lab, 2nd floor, B.1
    Contact Homepage
  • Mohammad
    Azadmanesh Course TA

    Office Lab, 2nd floor, B.1
    Contact Homepage

Schedule A one-semester, 14-week course

We have three weekly lecture slots, two for the course (white), and one for the lab (blue).

Date Topic Preparation In-Class Activities Assignment out Assignment due
Mon, Sep. 17 Introduction Refresh your Java, C, and Intel Assembly language knowledge Lecture & Background Knowledge Probe A01. Java & C -
Thu, Sep. 20 PL Program Representation Watch videos on program representation
(~45 minutes of videos, plus some breaks):
TA introductions (Dmitri and Mohammad). Provide feedback on videos. Discussion, triggered by your questions about the videos A02. Simple Bytecode Metrics A01. Java & C
Fri, Sep. 21 Building the Jikes RVM Read the Care and Feeding section of the Jikes RVM User's Guide Install and build the Jikes RVM L01. Getting Started with the Jikes RVM
Mon, Sep. 24 PL Control-Flow Graph Control-flow graphs and dot. Lecture 4 Slides. A03. Control-Flow Graphs with ASM A02. Simple Bytecode Metrics
Thu, Sep. 27 PL Loops TA introduction (Matej). Finding loops. Lecture 5 Slides. A04. Dominator Analysis A03. Control-Flow Graphs with ASM
Fri, Sep. 28 The Jikes RVM and Adaptive Optimization Jikes RVM, AOS. Lecture 6 Slides. L02. Jikes RVM Performance L01. Getting Started with the Jikes RVM
Mon, Oct. 1 PL Compiler Optimization Compiler optimizations. Exercise measuring performance impact of loop interchange. Lecture 7 Slides. A05. Add Exception Edges to Control-Flow Graph A04. Dominator Analysis
Thu, Oct. 4 PL Exceptions, PL Analysis, PL Class Hierarchy Think about this: Could one "emulate" GOTO in Java using exception handling constructs? Discuss exception handling issues. Lecture 8 Slides. Draw call graph of pacman-src.jar. - -
Fri, Oct. 5 Tickling the Jikes RVM Lecture 9 Slides. L03. Look and Tickle L02. Jikes RVM Performance
Mon, Oct. 8 PL Call Graph Read Fast Static Analysis of C++ Virtual Function Calls by Bacon and Sweeney (Section 1 of and most of Section 2) and Optimization of Object-Oriented Programs using Static Class Hierarchy Analysis (CHA) by Dean, Grove, and Chambers (Section 1 and the first part of Section 2) Lecture 10 Slides. A06. Call Graph with ASM A05. Add Exception Edges to Control-Flow Graph
Thu, Oct. 11 PL Memory Management, Part 1 Read An Introduction to Garbage Collection: Part I - The Real Costs of C++ Memory Management Explicit memory management, reference counting. Lecture 11 Slides. - -
Fri, Oct. 12 Hacking the Jikes RVM Jikes RVM in Eclipse
In-class workshop
L04. Heap dump L03. Look and Tickle
Mon, Oct. 15 PL Memory Management, Part 2 Read An Introduction to Garbage Collection: Part II - A Look Under the Hood Garbage collection, leak detection. Lecture 13 Slides. A07. Memory Management A06. Call Graph with ASM
Thu, Oct. 18 PL Memory Management, Part 3 Soft, Weak, Phantom References. Lecture 14 Slides. - -
Fri, Oct. 19 Debugging the Jikes RVM L05. Project kick-off L04. Heap dump
Mon, Oct. 22 OS HW Virtual Memory Virtual memory management. From malloc to mmap. Lecture 16 Slides. A08. Dynamic Instrumentation with DTrace A07. Memory Management
Thu, Oct. 25 Tools DTrace Read Dynamic Instrumentation of Production Systems (all 14 pages) Mmap, generating page faults, DTrace. Lecture 17 Slides. - -
Fri, Oct. 26 From now on, we'll only hold lab sessions on request - L05. Project kick-off
Mon, Oct. 29 HW Cache Memory hierarchy. Access latency. Direct-mapped vs. set-associative vs. fully-associative caches. Lecture 18 Slides. A09. Cache Simulator A08. Dynamic Instrumentation with DTrace
Fri, Nov. 2 - -
Mon, Nov. 5 PL Dynamic Class Loading Read Dynamic Class Loading in the Java Virtual Machine (everything except Section 4 "Maintaining Type-safe Linkage") Lazy loading, type-safe linking, multiple namespaces, class unloading, user-defined class loaders (SP-Compiler Eclipse project). Lecture 19 Slides. - A09. Cache Simulator
Thu, Nov. 8 OS Dynamic Linking Read The Inside Story on Shared Libraries and Dynamic Loading Shared libraries, dlopen/dlsym, library interposition, tracing dynamic linker activity. Lecture 20 Slides. - -
Fri, Nov. 9 - -
Mon, Nov. 12 Midterm exam - -
Thu, Nov. 15 HW Branch Prediction Read A study of branch prediction strategies Why? Static vs. Dynamic. Bimodal. Two-Level Adaptive. Branch Target Prediction: BTB, RAS. Lecture 21 Slides. - -
Fri, Nov. 16 - -
Mon, Nov. 19 Tools PL R Read Evaluating the design of the R language Variables, functions, vectors, factors, lists, data frames. Lecture 22 Slides. A10. Impact of Exceptions on Basic Block Size -
Thu, Nov. 22 Tools Theory Descriptive Statistics & Visualization Read Chapters 1 "Introduction", 2 "Metrics of performance" and 3 "Average performance and variability" of the textbook Central tendency & dispersion, PDF, CDF, jitter plot, box plot, histogram, ECDF, quantile/quantile plot. Lecture 23 Slides. - -
Fri, Nov. 23 - -
Mon, Nov. 26 Theory Random Error Read Chapter 4 "Errors in experimental measurements" of the textbook Accuracy and random error vs. precision and systematic error, confidence intervals. Lecture 24 Slides. - A10. Impact of Exceptions on Basic Block Size
Thu, Nov. 29 Theory Systematic Error Omitted Variable Bias, Profiler Bias. Lecture 25 Slides. - -
Fri, Nov. 30 - -
Mon, Dec. 3 Datacenter Performance Read the Evaluate Collaboratory Technical Report #1: Can you trust your experimental results? Virtual guest lecture by Amer Diwan (Performance Detective at Google)
Matthias at ISPASS PC/MICRO
- -
Thu, Dec. 6 Matej and Mohammad Presentation Matthias at ISPASS PC/MICRO
Lecture 27 Slides.
- -
Fri, Dec. 7 - -
Mon, Dec. 10 PL Dynamic Binary Instrumentation Java instrumentation agent. ASM class rewriting. Lecture 28 Slides. - -
Thu, Dec. 13 PL Call Profiling Read gprof: A call graph execution profiler Call graph. Calling context tree. Profiling with gprof, DTrace, hprof. DYI Java stack sampling. Lecture 29 Slides. A11. Array Size Analysis -
Fri, Dec. 14 - -
Mon, Dec. 17 Theory PL Complexity Read Algorithmic Profiling Cost Function, Asymptotic Complexity, Algorithmic Essence, Algorithmic Profiling. Lecture 30 Slides. - -
Thu, Dec. 20 Tools ggplot2 Read the ggplot2 Quick Reference - A11. Array Size Analysis
Fri, Dec. 21 - -

Assignment Policy

Group Work

Homework will consist of both written and programming assignments. You are encouraged to work on the programming assignments in groups of two, but you must do the written assignments by yourself.

Late Policy

In general, no late assignments are accepted.

Submission

Programming assignments will have specific submission instructions included with the handouts. Please, follow the submission instructions exactly as written!

Where are the assignments?

All assignments are posted on the schedule

Honor Code Cheating and plagiarism are not tolerated

You are free to discuss the assignment and solutions with others. However, you must write your own assignment, and must not represent any portion of others' work as your own.