This document outlines the requirements and schedule for students to participate as part of a program committee to review papers for the Deep Learning Hardware Conference 2023. Students will be assigned papers from various subject areas and must write reviews, present strengths and weaknesses, and participate in paper discussion sessions. For the paper reviews, students must analyze and critique papers based on concepts learned in class, evaluate the technical merits and experiments, and provide feedback to help improve the work. The goal is for students to gain experience reviewing research papers and understanding the process for how papers are evaluated and discussed at academic conferences.

1. 6.5930/1
Hardware Architectures for Deep Learning
DLH-23 Program Committee
Joel Emer and Vivienne Sze
Massachusetts Institute of Technology
Electrical Engineering & Computer Science
March 20, 2023

2. Sze and Emer
Paper Review Schedule and Deliverables
• Paper Review (20% of grade)
• We will be forming a program committee of the Deep Learning Hardware
(DLH) Conference 2023
– Learn how to read papers (critique and distill class principles)
– Enable wider coverage of papers
– Gain insight of how paper decisions are made
• Schedule
– March 20 – Papers assigned (check for undetected conflicts)
– Example paper reviews (links to be shared on piazza)
– Reviews (and slides) due at 11:59 pm the day before the review session
– April 03, 19, 26, May 03, 08 – Paper review sessions
March 20, 2023

3. Sze and Emer
Paper Subject Areas & Session Dates
• During Lecture
– A. Dataflows (April 03)
– B. Sparse Accelerators 1 (April 19)
– C. Sparse Accelerators 2 (April 26)
– D. Mapping + Bit Precision (May 03)
– E. Analog & PIM (SRAM, Near Memory) (May 08)
• During Recitations
– April 21, April 28, May 05
March 20, 2023

4. Sze and Emer
Paper Reviews
• Students in 6.5931 (5 reviews)
– Write a review for one paper in each of the five subject areas
– Participate in paper discussion during session and vote on papers
• Students in 6.5930 (7 reviews)
– Write review and present one paper in front of class (lead reviewer)
– Write review and present strengths/weaknesses for three papers
– Write a review for one paper in each of the other four subject areas
– Participate in paper discussion during session and vote on papers
• By 11:59pm the day before the review session
– Submit reviews (6.5930/1) on HotCRP (https://mit-dlh23.hotcrp.com/)
– Submit slides (6.5930) for your lead paper and strength/weakness for three papers on
Google drive
March 20, 2023

5. Sze and Emer
HotCRP Website
March 20, 2023
https://mit-dlh23.hotcrp.com/

6. Sze and Emer
Program Committee Meeting
• Each paper will be assigned three core reviewers (one of which will be the lead)
– The lead will give overview and strength/weakness of paper
• Unlimited slides (10 min)
– Two reviewers will give strength/weakness
• 1 slide each (2 x 1 min)
– Discuss paper and questions from other members of committee
• No slides (2 min)
– All reviewers vote on paper (other committee members may also vote if they have read
paper or followed discussion) – score
– At end of session, rank order of paper
March 20, 2023
Timing is very tight – please come to class on time!

7. Sze and Emer
Program Committee Meeting
• Students in 6.5930
– Submit written reviews & slides for three papers (75% of grade)
• Will be assigned as lead for one of the papers
– Submit reviews for four additional papers and participate in class
discussion and vote on paper (25% of grade)
• Students in 6.5931
– Submit reviews for five papers and participate in class discussion and
vote on paper (100% of grade)
March 20, 2023

8. Sze and Emer
Real World versus DLH23 PC
Real World DLH22
Papers per PC member 20 – 110 papers 3 + 4
Review Time 2 – 5 weeks 2 – 7 weeks
Length of PC meeting 1 to 2 days 5 classes x 1.5 hours +
3 recitations x 1 hours =
10.5 hours
Number of papers
discussed
60 – 120 ~40
Discussion time per paper 5 – 15 min 12 - 15 min
February 6, 2023

9. Sze and Emer
Submitted Reviews Should Capture
• Paper Summary (In your own words!)
– Motivation of Paper: What is the problem? Why is it important? What are the
current solutions (i.e., previous work)?
– Give insight on contributions of proposed techniques in paper. Describe in terms of
concept discussed in class (e.g., stationarity of the dataflow)
– Discuss key results in paper and how do they support claimed contributions
– What are the main takeaways from this paper
• Strengths of paper
• Weakness of paper
• Innovation
– Describe key insights based on concepts from class
March 20, 2023

10. Sze and Emer
Submitted Reviews Should Capture
• Quality of Evaluation
– Do the results backup the claimed benefits? What baseline was used? Are the experiments
unbiased? Are tradeoffs and overheads evaluated?
– See https://gernot-heiser.org/benchmarking-crimes.html
• Possible extensions to the paper
– Not just a list of additional techniques (e.g., combine with X, Y, Z). Explain how various
approaches could address specific short comings of existing work.
• Questions for Authors
– What parts of the paper was not clear? What would you ask authors?
• Comments for professors
March 20, 2023

11. Sze and Emer
Other Considerations
• Graded based on ability to give intuition on technique, interpretations of graphs and
data, etc.
– Use of insights from class to analyze papers
• e.g., even if authors don’t explicitly state dataflow, can you identify which dataflow is used and why? What are the
strengths and weaknesses of this?
– Refer to ”Evaluating DNN” lecture on performing a comprehensive evaluation
– Critique not about writing and figures but about quality of technology
• Depth better than breadth (particularly summary)
– Extensions not just a list of other work to add on top, but insight in terms of how to improve
weakness; why approach would be a better on top of this
• We encourage you to draw on your expertise from outside class
– Diverse background: Devices, Algorithms, Architecture – we want to learn from your
expertise
March 20, 2023