18 Meta Techniques in Computer Science

Kanazawa Institute of Technology
18 Meta Techniques
in
Computer Science
May 1st, 2015
Jun Nakano
Kanazawa Institute of Technology, Japan
http://www.nakanolab.net/

K.I.T.18 Meta Techniques in Computer Science
1. Caching
2. Blocking vs. Non-blocking
3. Profiling
4. Pipelining
5. Speculation (Prediction)
6. Relaxation
7. Eager vs. Lazy
8. Filtering
9. Reuse
10. Parallelization
11. Coupling vs. Decoupling
12. Multilevel / Clustering
13. Lookahead / Prefetching
14. Dynamic (Adaptive)
15. Replication
16. Virtualization
17. Checkpoints / Snapshots
18. Transactions
I have been seeing some common techniques used across various
disciplines of computer science. Above is the list of 18 such "meta
techniques" in no particular order. In the following slides, I will describe
ideas behind them and show some examples (somewhat biased toward
computer architecture).
218 Meta Techniques in Computer Science

K.I.T.
#1 Caching
#2 Blocking vs. Non-blocking
#3 Profiling
#4 Pipelining
#5 Speculation (Prediction)
#6 Relaxation
#7 Eager vs. Lazy
#8 Filtering
#9 Reuse
#10 Parallelization
#11 Coupling vs. Decoupling
#12 Multilevel / Clustering
#13 Lookahead / Prefetching
#14 Dynamic (Adaptive)
#15 Replication
#16 Virtualization
#17 Checkpoints / Snapshots
#18 Transactions

K.I.T.#1 Caching
 Idea
 Exploit
» Spatial locality
» Temporal locality
» 80-20 rule
 Examples
 Data cache
 Instruction cache: Static & Dynamic (trace cache of Pentium 4)
 Buffer cache of file system and DBMS
 Web browser's cache
 Proxy server
 Memoization
 LZW compression (caching frequent patterns in a dictionary)

K.I.T.#2 Blocking vs. Non-blocking
 Idea
 Do useful stuff while waiting for something to complete
 Examples
 Multi-tasking (OS scheduler)
 Multi-threading
 Asynchronous I/O
 Asynchronous communication in MPI (Message Passing
Interface): Overlapping computation and communication
 Out-of-order vs. in-order execution in processor
 Allow cache reads while there are outstanding cache misses
(MSHR: Miss Status Holding Register)

K.I.T.#3 Profiling
 Idea
 Use runtime information to optimize execution
 Examples
 When optimizing code, start with "hot spots" (via gprof, etc.)
 Lots of work by compiler folks (e.g., code layout optimization
by Spike)

K.I.T.#4 Pipelining
 Idea
 For better throughput
 Examples
 Processor pipeline (fetch, decode, execute, memory, and write
back)
A
B
C
D
A
B
C
D
Tasks
Time Time
Wash
Dry
Fold
Sequential Laundry Pipelined Laundry
Source: http://www.cs.berkeley.edu/~pattrsn/252S01/

K.I.T.#5 Speculation (Prediction)
 Idea
 Make an educated guess
and believe you are lucky 
 Examples
 Branch prediction
 Prefetching
 Value prediction
 Speculative helper thread to hide cache miss latency of main
thread

K.I.T.#6 Relaxation
 Idea
 What if you remove a constraint?
 Examples
 Relaxed memory consistency in multiprocessor systems
» Sequential consistency: Intuitive for programmers but little room
for hardware optimization
» Relaxed consistency: Pursue hardware optimization (e.g., write
buffer) at the cost of some inconveniences to programmers
 Eventual consistency for distributed hash tables

K.I.T.
#1 Caching
#3 Profiling
#4 Pipelining
#6 Relaxation
#7 Eager vs. Lazy
#8 Filtering
#9 Reuse
#10 Parallelization
#14 Dynamic (Adaptive)
#15 Replication
#16 Virtualization
#18 Transactions

K.I.T.#7 Eager vs. Lazy
 Idea
 Eager
» Do it now if you are sure you will have to do it later, or if it is safe
and cheap
 Lazy
» Otherwise, do nothing until you really have to
 Examples
 Demand paging (lazy)
 Copy on write (lazy)
 Some data structures such as Fibonacci heap that aim at
reducing amortized cost (lazy)
 Just-in-time compiler (eager / lazy)
 Protocols in MPI (Message Passing Interface)
» Eager: Send small data eagerly to avoid negotiation cost
» Rendezvous: Negotiate with receiver before sending large data

K.I.T.#8 Filtering
 Idea
 Avoid going down to the lower level if possible
 Examples
 Cache hierarchy
» L1 cache → L2 cache → main memory
 Bloom filter

K.I.T.#9 Reuse
 Idea
 Do not repeat the same thing
» If the computation of f(x) is expensive and occurs frequently,
remember the answer
 Examples
 Memoization
 Dynamic Instruction Reuse (Sodani and Sohi, ISCA 1997)

K.I.T.#10 Parallelization
 Idea
 Double the computational resources and cut the time in half
» In practice, be aware of Amdahl's law
 Examples
 SMP
 Multicore
 Cluster of computers
 MapReduce

 Idea
 Less dependency is easier to handle but choose design point
wisely considering trade-offs
 Examples
 Cache
» L1 cache: Data / Instruction (Harvard architecture)
» L2 cache: Unified
 Decoupling is often preferred in software engineering
» Interface and Implementation (object-oriented programming)
» Policy and Mechanism (computer networks)
» Control plane and data plane in SDN (Software Defined
Networking)
» Protocol stack of TCP/IP (physical, data link, network, transport,
and application)
» Database normalization

K.I.T.#12 Multilevel / Clustering
 Idea
 Organize things neatly
 Examples
 Cache hierarchy
 Directory structure of file systems
 Grouping instructions in processor pipeline
 TCP/IP
 Anything that looks like a tree

K.I.T.
#1 Caching
#3 Profiling
#4 Pipelining
#6 Relaxation
#7 Eager vs. Lazy
#8 Filtering
#9 Reuse
#10 Parallelization
#14 Dynamic (Adaptative)
#15 Replication
#16 Virtualization
#18 Transactions

 Idea
 Right thing at the right time
 Examples
 Hardware/software prefetching from memory
 Cache line

K.I.T.#14 Dynamic (Adaptive)
 Idea
 Effective use of runtime information while running
 Examples
 Just-in-time compilation of Java
 Dynamic optimization in LLVM
 Load balancer for Web server farm

K.I.T.#15 Replication
 Idea
 Better reliability and/or locality
 Examples
 Disk mirroring
 Replication in distributed databases
 Replication in distributed hash tables
 CDN (Content Delivery Network): Akamai, etc.
 GFS (Google File System)

K.I.T.#16 Virtualization
 Idea
 Beyond physical limitations
 Redirection
 Examples
 Virtual memory
 Virtual devices
 Virtual machines
 Virtual file systems
 View in database
 Shortcuts in Windows and symbolic links in UNIX
Virtual
Illusion
Physical
Entity

K.I.T.#17 Checkpoints / Snapshots
 Idea
 Avoid losing data in case of failures / human errors
» Be aware of output commit problem
 Examples
 Database
 Copy on write

K.I.T.#18 Transactions
 Idea
 Impose atomicity for group of operations
 Examples
 Database
 Transactional memory for parallel programming
» Trade-off between correctness and performance
• Possible to write fast and correct programs using locks, etc.,
but error-prone
• No worry about locking and unlocking in transactional
memory at the cost of some performance overhead

K.I.T.Summary
1. Caching
2. Blocking vs. Non-blocking
3. Profiling
4. Pipelining
5. Speculation (Prediction)
6. Relaxation
7. Eager vs. Lazy
8. Filtering
9. Reuse
10. Parallelization
11. Coupling vs. Decoupling
12. Multilevel / Clustering
13. Lookahead / Prefetching
14. Dynamic (Adaptive)
15. Replication
16. Virtualization
17. Checkpoints / Snapshots
18. Transactions
Some advice for you if you are to apply these meta techniques to your project:
 Be aware of trade-offs that exist in almost any engineering project
 As they say, "Devil is in the details."
It is most appropriate to consider these meta techniques as just a food for thought.

18 Meta Techniques in Computer Science

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (7)

Similar to 18 Meta Techniques in Computer Science

Similar to 18 Meta Techniques in Computer Science (20)

Recently uploaded

Recently uploaded (20)

18 Meta Techniques in Computer Science