This document provides guidance on how to effectively read and evaluate research papers. It recommends beginning by scanning the abstract, introduction, and conclusion to determine if the paper contains interesting information. If so, the next step is to identify which sections contain the key ideas. Finally, carefully read the entire paper while considering questions about how the ideas could be applied, evaluated, and extended. The document also provides tips on where to search for papers, managing references, and common paper components to examine like related work, new contributions, evaluations and assumptions.

1. How to Read Research Papers? Xiao Qin Department of Computer Science and Software Engineering Auburn Universityhttp://www.eng.auburn.edu/~xqin xqin@auburn.edu Some slides are adapted from notes by Nitin H. Vaidya at the University of Illinois at Urbana-Champaign 1

2. Three Steps in a Research Project Identify a Problem Solve the Problem: a solution Present the solution

3. Why read papers? To know what’s happening Avoid reinventing the wheel does happen commonly, too many wheels already Find interesting research topics Write a good “related work” section

7. Now…‹#›

10. Where to search papers? (3) Top Conference Proceedings AREA: System Technology Premium: SOSP: ACM SIGOPS Symp on OS Principles OSDI: UsenixSymp. OS Design and Implementation Leading: ICPP: Intl Conf on Parallel Processing ICDCS: IEEE Intl Conf. Distr. Comp Systems Reputable: GLOBECOM: Global Comm ICCC: Intl Conf on Comp Communication

11. Where to search papers? (4) Top Journals AREA: System Technology Premium: Jnl of Parallel and Distributed Computing IEEE Trans on Parallel and Distributed Systems Distributed Computing Leading: Jnl of Network and Computer Applications Parallel Computing Reputable: ACM Mobile Computing and Communicatons Review Intl Jnl of High Speed Computing

12. What to read? Major conferences Journals are a few years behind, but still can be useful Tech reports from active research groups need to know which groups to look up Survey / overview papers ACM Computing Surveys CACM, IEEE Computer, Spectrum more technical - IEEE Personal Communications, … newsletters - ACM SIGCOMM, ACM SIGMOBILE, ...

13. Manage References

14. References Management Software BiblioExpress

15. References Management Software EndNote EndNote Web Zotero RefWorks Reference Manager Mendeley See Comparison at: http://workspace.imperial.ac.uk/library/Public/Reference_management_software_comparison.pdf

16. What’s in a paper [see also how to write papers: http://www.auburn.edu/~xzq0001/WritePapers-Part 1.ppt] Abstract Introduction Motivation Problem description Solution ... Performance Analysis Conclusions Future Work

17. How to read a paper? Three levels Know why you want to read the paper To know what’s going on (e.g., scanning proceedings) title, authors, abstract Papers in your broad research area introduction, motivation, solution description, summary, conclusions sometimes reading more details useful, but not always Papers you may want to improve on read entire paper carefully

18. What to note? Authors and research group Need to know where to look for a paper on particular topic Theme of the solution Should be able to go back to the paper if you need more info Approach to performance evaluation Note any shortcomings

19. So this paper is in print ... Be skeptical If it sounds too good to be true, it often is ……

20. More Advice …

21. Three phases to reading Determine if there is anything interesting at all in the paper. Determine which portion of the paper contains the interesting stuff. Should I read the whole paper? Read the whole paper if necessary.

22. Is there anything interesting? Ideally, the abstract should tell you this, but frequently it does not. Need to jump about Read conclusion Read introduction Look at the bibliography Glance at the TOC – Table of Content (if any)

23. Which portion contains interesting stuff? A paper typically outlines its organization at the end of the introduction. Use this to determine which portion contains the exciting stuff. Quickly scan the titles of each sections in the paper.

24. Read the whole paper Read with the following questions in mind How can I use this stuff? Does this really do what the author claims to do? What if the assumptions and choices that the author made are discarded (or made invalid)? Is source code available? More questions ……

25. Context and problem statement What problems are the author trying to solve or trying to convince you of? Are they important problems? Why? Why not? What is the author’s thesis?

26. Related work evaluation Does the author describe other work in the field? If so, how does this research differ from the other work?

27. New idea What new idea is the author proposing? Architecture Algorithm Mechanism Methodology Perspective Is the idea useful and practical? 26

28. What to evaluate? What need to be evaluated to confirm the worthiness of the new idea? Runtime Throughput Resource utilization Model validation 27

29. How to evaluate? How did the author go about conducting the evaluation? Formalize and prove theorems Run simulations Artifact design and construction Collect traces from existing systems

30. Was the evaluation correct and adequate? How was the data collection done? Do you agree with the analysis of data? Do you agree with the conclusions about the data? Do you have any new interpretation of the data? Can you suggest new ways to evaluate the data?

31. Assumptions, drawbacks and extensions Can you think of other aspects of the idea that need to be evaluated? Can you think of extensions or modifications to the idea to improve it? How would you evaluate your improvement?

32. Assumptions, drawbacks and extensions Can you apply the idea or method of evaluation to your own project? Do the authors make any assumptions that are not valid or realistic? Can you come up with a more general solution that does not rely on one or more of the assumptions?

33. Future work Does the author indicate how the work should be followed up on? Does the paper generate new ideas? Does the paper implicitly or explicitly provide a new way of doing other things or of thinking about problems?

34. Summary If you remember nothing else: Where to search papers Scan papers Read papers with questions in mind

35. Download the presentation slideshttp://www.slideshare.net/xqin74 Google: slideshare Xiao Qin

36. Further Researchhttp://www.eng.auburn.edu/~xqin

37. My webpagehttp://www.eng.auburn.edu/~xqin

38. Download Slides at slidesharehttp://www.slideshare.net/xqin74

39. Questions