The document outlines 18 "meta techniques" commonly used across computer science disciplines, including caching, pipelining, speculation, parallelization, and transactions. It provides examples and brief descriptions of each technique. The author concludes by advising that these techniques should be considered as general approaches to engineering problems, with an awareness that effectively applying them requires carefully weighing trade-offs in project-specific details.
Introduction to IEEE STANDARDS and its different types.pptx
18 Meta Techniques in Computer Science
1. Kanazawa Institute of Technology
18 Meta Techniques
in
Computer Science
May 1st, 2015
Jun Nakano
Kanazawa Institute of Technology, Japan
http://www.nakanolab.net/
2. K.I.T.18 Meta Techniques in Computer Science
1. Caching
2. Blocking vs. Non-blocking
3. Profiling
4. Pipelining
5. Speculation (Prediction)
6. Relaxation
7. Eager vs. Lazy
8. Filtering
9. Reuse
10. Parallelization
11. Coupling vs. Decoupling
12. Multilevel / Clustering
13. Lookahead / Prefetching
14. Dynamic (Adaptive)
15. Replication
16. Virtualization
17. Checkpoints / Snapshots
18. Transactions
I have been seeing some common techniques used across various
disciplines of computer science. Above is the list of 18 such "meta
techniques" in no particular order. In the following slides, I will describe
ideas behind them and show some examples (somewhat biased toward
computer architecture).
218 Meta Techniques in Computer Science
4. K.I.T.#1 Caching
Idea
Exploit
» Spatial locality
» Temporal locality
» 80-20 rule
Examples
Data cache
Instruction cache: Static & Dynamic (trace cache of Pentium 4)
Buffer cache of file system and DBMS
Web browser's cache
Proxy server
Memoization
LZW compression (caching frequent patterns in a dictionary)
418 Meta Techniques in Computer Science
5. K.I.T.#2 Blocking vs. Non-blocking
Idea
Do useful stuff while waiting for something to complete
Examples
Multi-tasking (OS scheduler)
Multi-threading
Asynchronous I/O
Asynchronous communication in MPI (Message Passing
Interface): Overlapping computation and communication
Out-of-order vs. in-order execution in processor
Allow cache reads while there are outstanding cache misses
(MSHR: Miss Status Holding Register)
518 Meta Techniques in Computer Science
6. K.I.T.#3 Profiling
Idea
Use runtime information to optimize execution
Examples
When optimizing code, start with "hot spots" (via gprof, etc.)
Lots of work by compiler folks (e.g., code layout optimization
by Spike)
618 Meta Techniques in Computer Science
7. K.I.T.#4 Pipelining
Idea
For better throughput
Examples
Processor pipeline (fetch, decode, execute, memory, and write
back)
A
B
C
D
A
B
C
D
Tasks
Time Time
Wash
Dry
Fold
Sequential Laundry Pipelined Laundry
Source: http://www.cs.berkeley.edu/~pattrsn/252S01/
718 Meta Techniques in Computer Science
8. K.I.T.#5 Speculation (Prediction)
Idea
Make an educated guess
and believe you are lucky
Examples
Branch prediction
Prefetching
Value prediction
Speculative helper thread to hide cache miss latency of main
thread
818 Meta Techniques in Computer Science
9. K.I.T.#6 Relaxation
Idea
What if you remove a constraint?
Examples
Relaxed memory consistency in multiprocessor systems
» Sequential consistency: Intuitive for programmers but little room
for hardware optimization
» Relaxed consistency: Pursue hardware optimization (e.g., write
buffer) at the cost of some inconveniences to programmers
Eventual consistency for distributed hash tables
918 Meta Techniques in Computer Science
11. K.I.T.#7 Eager vs. Lazy
Idea
Eager
» Do it now if you are sure you will have to do it later, or if it is safe
and cheap
Lazy
» Otherwise, do nothing until you really have to
Examples
Demand paging (lazy)
Copy on write (lazy)
Some data structures such as Fibonacci heap that aim at
reducing amortized cost (lazy)
Just-in-time compiler (eager / lazy)
Protocols in MPI (Message Passing Interface)
» Eager: Send small data eagerly to avoid negotiation cost
» Rendezvous: Negotiate with receiver before sending large data
1118 Meta Techniques in Computer Science
12. K.I.T.#8 Filtering
Idea
Avoid going down to the lower level if possible
Examples
Cache hierarchy
» L1 cache → L2 cache → main memory
Bloom filter
1218 Meta Techniques in Computer Science
13. K.I.T.#9 Reuse
Idea
Do not repeat the same thing
» If the computation of f(x) is expensive and occurs frequently,
remember the answer
Examples
Memoization
Dynamic Instruction Reuse (Sodani and Sohi, ISCA 1997)
1318 Meta Techniques in Computer Science
14. K.I.T.#10 Parallelization
Idea
Double the computational resources and cut the time in half
» In practice, be aware of Amdahl's law
Examples
SMP
Multicore
Cluster of computers
MapReduce
1418 Meta Techniques in Computer Science
15. #11 Coupling vs. Decoupling
Idea
Less dependency is easier to handle but choose design point
wisely considering trade-offs
Examples
Cache
» L1 cache: Data / Instruction (Harvard architecture)
» L2 cache: Unified
Decoupling is often preferred in software engineering
» Interface and Implementation (object-oriented programming)
» Policy and Mechanism (computer networks)
» Control plane and data plane in SDN (Software Defined
Networking)
» Protocol stack of TCP/IP (physical, data link, network, transport,
and application)
» Database normalization
1518 Meta Techniques in Computer Science
16. K.I.T.#12 Multilevel / Clustering
Idea
Organize things neatly
Examples
Cache hierarchy
Directory structure of file systems
Grouping instructions in processor pipeline
TCP/IP
Anything that looks like a tree
1618 Meta Techniques in Computer Science
18. #13 Lookahead / Prefetching
Idea
Right thing at the right time
Examples
Hardware/software prefetching from memory
Cache line
1818 Meta Techniques in Computer Science
19. K.I.T.#14 Dynamic (Adaptive)
Idea
Effective use of runtime information while running
Examples
Just-in-time compilation of Java
Dynamic optimization in LLVM
Load balancer for Web server farm
1918 Meta Techniques in Computer Science
20. K.I.T.#15 Replication
Idea
Better reliability and/or locality
Examples
Disk mirroring
Replication in distributed databases
Replication in distributed hash tables
CDN (Content Delivery Network): Akamai, etc.
GFS (Google File System)
2018 Meta Techniques in Computer Science
21. K.I.T.#16 Virtualization
Idea
Beyond physical limitations
Redirection
Examples
Virtual memory
Virtual devices
Virtual machines
Virtual file systems
View in database
Shortcuts in Windows and symbolic links in UNIX
Virtual
Illusion
Physical
Entity
2118 Meta Techniques in Computer Science
22. K.I.T.#17 Checkpoints / Snapshots
Idea
Avoid losing data in case of failures / human errors
» Be aware of output commit problem
Examples
Database
Copy on write
2218 Meta Techniques in Computer Science
23. K.I.T.#18 Transactions
Idea
Impose atomicity for group of operations
Examples
Database
Transactional memory for parallel programming
» Trade-off between correctness and performance
• Possible to write fast and correct programs using locks, etc.,
but error-prone
• No worry about locking and unlocking in transactional
memory at the cost of some performance overhead
2318 Meta Techniques in Computer Science
24. K.I.T.Summary
1. Caching
2. Blocking vs. Non-blocking
3. Profiling
4. Pipelining
5. Speculation (Prediction)
6. Relaxation
7. Eager vs. Lazy
8. Filtering
9. Reuse
10. Parallelization
11. Coupling vs. Decoupling
12. Multilevel / Clustering
13. Lookahead / Prefetching
14. Dynamic (Adaptive)
15. Replication
16. Virtualization
17. Checkpoints / Snapshots
18. Transactions
Some advice for you if you are to apply these meta techniques to your project:
Be aware of trade-offs that exist in almost any engineering project
As they say, "Devil is in the details."
It is most appropriate to consider these meta techniques as just a food for thought.
2418 Meta Techniques in Computer Science