Identifying Hotspots in the
PostgreSQL Build Processes

Shane
McIntosh

Bram
Adams

Meiyappan
Nagappan

Ahmed E.
Hassan

@...
What is a build system?
Source
code

!2
What is a build system?
Source
code

Deliverable

!2
Build systems describe how sources are
translated into deliverables
.c

.o

.cc

.o

.exe

.tex

.dvi

.pdf

.a
.deb

!3
All builds are equal…
(1)
Think

(2)
Edit
The
developer’s
work cycle

(4)
Test

(3)
Build
!4
…But some builds are more
equal than others

“

http://xkcd.com/303/

Why is the gtk build so abysmally slow?
This hinders...
Refactoring can improve
build performance
deliverable1
del1.o
del1.c

deliverable2
library.a

util1.o

util2.o

util1.c

d...
Refactoring can improve
build performance
deliverable1
del1.o
del1.c

deliverable2
library.a

util1.o

util2.o

util1.c

d...
Refactoring can improve
build performance
deliverable1
del1.o
del1.c

deliverable2
library.a

util1.o

util2.o

util1.c

d...
Refactoring can improve
build performance
deliverable1
del1.o
del1.c

deliverable2
library.a

util1.o

util2.o

util1.c

d...
Refactoring can improve
build performance
deliverable1
del1.o
del1.c

deliverable2
library.a

util1.o

util2.o

util1.c

d...
Refactoring can improve
build performance
deliverable1
deliverable2
Before refactoring:!
4 commands! del2.o
del1.o
library...
Refactoring can improve
build performance
deliverable1
del1.o
del1.c

deliverable2
library.a

util1.o

util2.o

util1.c

d...
Refactoring can improve
build performance
deliverable1

deliverable2

del1.o

util1.o

util2.o

del2.o

del1.c

util1.c

u...
Refactoring can improve
build performance
deliverable1

deliverable2

del1.o

util1.o

util2.o

del2.o

del1.c

util1.c

u...
Refactoring can improve
build performance
deliverable1

deliverable2

del1.o

util1.o

util2.o

del2.o

del1.c

util1.c

u...
Refactoring can improve
build performance
deliverable1

deliverable2

del1.o

util1.o

util2.o

del2.o

del1.c

util1.c

u...
Refactoring can improve
build performance
deliverable1
deliverable2
After refactoring:!
2 commands! del2.o
del1.o
util1.o
...
…But where should we focus
refactoring effort?
Files that rebuild
the slowest?

!8
(1)
Dependency
Graph
Construction

Dep. Graph

(2)
Dependency
Graph
Analysis

Graph
analysis
results

Version
Control
Syst...
MAKAO tool extracts build
dependency graphs
(1)
Dependency
Graph
Construction
(2)
Dependency
Graph
Analysis

Design recove...
The cost of traversing an edge is
derived by timing its build command
(1)
Dependency
Graph
Construction
(2)
Dependency
Gra...
Open source case studies!
!
!
!

!12
Build dependency graph
properties
# Nodes
# Edges

38,102

2,752,225

0

750,000

1,500,000

2,250,000

3,000,000

3,375
1...
100
50

Build Time (s)

8
6
4

0

2
0

Build Time (s)

10

12

150

Incremental build
performance

0

100

200

300

400

...
150
50

100

Build Time (s)

1.5
1.0
0.5

0

0.0

Build Time (s)

2.0

Incremental build
performance

0

200

400

600

80...
6000
0

2000

4000

Build Time (s)

100
50
0

Build Time (s)

150

8000

Incremental build
performance

0

2000

4000

600...
100
50
0

Build Time (s)

150

So, refactoring
should target the slow files!

0

100

200

300
File ID

!17

400

500

600
100
50
0

Build Time (s)

150

So, refactoring
should target the slow files!

0

100

200

300
File ID

!17

400

500

600
100
50
0

Build Time (s)

150

So, refactoring
should target the slow files!

0

100

200

300
File ID

!17

400

500

600
100
50
0

Build Time (s)

150

So, refactoring
should target the slow files!

0

100

200

300
File ID

!17

400

500

600
Let’s see what
developers think!
> Please answer the following questions for each
source file listed above:!
> (1) Is this ...
Let’s see what
developers think!
> Please answer the following questions for each
source file listed above:!
> (1) Is this ...
…But where should we focus
refactoring effort?
Files that rebuild
the slowest?

!20
…But where should we focus
refactoring effort?
Files that rebuild
the slowest?

…but they may
only rarely change!
!20
…But where should we focus
refactoring effort?
Files that rebuild
the slowest?

Files that change
the most often?

…but th...
Mining frequently changing files
from version control
Terminal

$ git clone …
…
$ git checkout <some_tag>
…
$ git log --one...
Mining frequently changing files
from version control

But many of these files
$ git clone …
…
already build quickly!
Termin...
…But where should we focus
refactoring effort?
Files that rebuild
the slowest?

Files that change
the most often?

…but th...
…But where should we focus
refactoring effort?
Files that rebuild
the slowest?

Files that change
the most often?

…but th...
…But where should we focus
refactoring effort?
Files that rebuild
the slowest?

Files that change
the most:often?
ots

…bu...
(1)
Dependency
Graph
Construction

Dep. Graph

(2)
Dependency
Graph
Analysis

Graph
analysis
results

Version
Control
Syst...
(1)
Dependency
Graph
Construction

Dep. Graph

(2)
Dependency
Graph
Analysis

Graph
analysis
results

Version
Control
Syst...
Quadrant plots highlight build
hotspots
Number of changes

(1)
Dependency
Graph
Construction
(2)
Dependency
Graph
Analysis...
Quadrant plots highlight build
hotspots
Number of changes

(1)
Dependency
Graph
Construction
(2)
Dependency
Graph
Analysis...
Quadrant plots highlight build
hotspots
Number of changes

(1)
Dependency
Graph
Construction
(2)
Dependency
Graph
Analysis...
Number of changes

Quadrant plots highlight build
hotspots
factor
Re
e first!
thes

(1)
Dependency
Graph
Construction
(2)
D...
Open source case studies!
Thresholds
!

Rebuild cost
90 seconds

Number of changes
!
Median
!

!26
1.00

●
●
●

●
● ●
●
●

Number of changes
(Normalized)
Normalized File Churn

●
●

●

●
●

0.25

●
●
●
●
●
●
●
●

●
●
●
●
...
1.00

●

●

●

●
●
●
●

Number of changes
(Normalized)
Normalized File Churn

●

0.75

0.50

0.25

●
●
●
●
●
●
●
●
●
●
●
●...
1.00

●

●

●
●
●
●●

●

●
●

Number of changes
(Normalized)
Normalized File Churn

●
●
●
●
●
●

0.75

0.50

0.25

●
●
●
●...
General trends
Transitive property of build hotspots
1

2

...

n-1

n

hot.o

hot.c

!30
General trends
Transitive property of build hotspots
1

2

...

n-1

n

hot.o

hot.c

!30
General trends
Transitive property of build hotspots
1

2

...

n-1

n

hot.o

hot.c

x.h
!30
General trends
Transitive property of build hotspots
1

2

...

n-1

n

hot.o

hot.c

x.h
!30
1

2

...

n-1

n

General trends
hot.o

Transitive property of build hotspots
hot.c

x.h

!31
1

2

...

n-1

n

General trends
hot.o

Transitive property of build hotspots
hot.c

x.h

!31
1

2

...

n-1

n

General trends
hot.o

Transitive property of build hotspots
hot.c

x.h

hub.h

!31
1

2

...

n-1

n

General trends
hot.o

Transitive property of build hotspots
hot.c

a.h

b.h

x.h

y.h

z.h

hub.h

!31
1

2

...

n-1

n

General trends
hot.o

Transitive property of build hotspots
hot.c

a.h

b.h

x.h

y.h

z.h

hub.h

!31
1

2

...

n-1

n

General trends
hot.o

Transitive property of build hotspots
hot.c

a.h

b.h

x.h

y.h

z.h

hub.h

!31
1

2

...

n-1

n

General trends
hot.o

Transitive property of build hotspots
hot.c

a.h

b.h

x.h

y.h

z.h

hub.h

Limi...
2nd International Workshop on Release Engineering
RELENG 2014

?
?
http://releng.polymtl.ca
2nd International Workshop on Release Engineering
RELENG 2014

keynotes

?
?
http://releng.polymtl.ca

talks on
research
a...
2nd International Workshop on Release Engineering
RELENG 2014

keynotes

http://releng.polymtl.ca

?
?

hosted in	

Mounta...
2nd International Workshop on Release Engineering
RELENG 2014

keynotes
McNutt
Dinah le)
(Goog

hosted in	

Mountainview (...
@shane_mcintosh
shanemcintosh@acm.org
Bonus!

!34
postgres.h
src/include/c.h
src/include/port.h
src/include/utils/elog.h
src/include/utils/palloc.h
src/include/pg_config_ma...
htup_details.h
src/include/access/htup.h
src/include/storage/itemptr.h
src/include/storage/itemid.h
src/include/storage/bu...
genam.h
src/include/access/skey.h
src/include/storage/lock.h
src/include/storage/shmem.h
src/include/utils/hsearch.h
src/i...
xact.h
src/include/access/xlog.h
src/include/access/xlogdefs.h
src/include/nodes/pg_list.h
src/include/lib/stringinfo.h

!...
builtins.h
src/include/fmgr.h
src/include/nodes/parsenodes.h
src/include/nodes/primnodes.h
src/include/nodes/pg_list.h
src...
Identifying Hotspots in the PostgreSQL Build Process
Identifying Hotspots in the PostgreSQL Build Process
Identifying Hotspots in the PostgreSQL Build Process
Identifying Hotspots in the PostgreSQL Build Process
Identifying Hotspots in the PostgreSQL Build Process
Identifying Hotspots in the PostgreSQL Build Process
Upcoming SlideShare
Loading in …5
×

Identifying Hotspots in the PostgreSQL Build Process

591 views

Published on

Software developers rely on a fast and correct build system to compile their source code changes and produce modified deliverables for testing and deployment. The scale and complexity of the PostgreSQL build process makes build performance an important topic to discuss and address.

In this talk, we will introduce a new build performance analysis technique that identifies "build hotspots", i.e., files that are slow to rebuild (by analyzing a build dependency graph), yet change often (by analyzing version control history). We will discuss the identified hotspots in the 9.2.4 release of PostgreSQL.

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
591
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
10
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Identifying Hotspots in the PostgreSQL Build Process

  1. 1. Identifying Hotspots in the PostgreSQL Build Processes Shane McIntosh Bram Adams Meiyappan Nagappan Ahmed E. Hassan @shane_mcintosh shanemcintosh@acm.org
  2. 2. What is a build system? Source code !2
  3. 3. What is a build system? Source code Deliverable !2
  4. 4. Build systems describe how sources are translated into deliverables .c .o .cc .o .exe .tex .dvi .pdf .a .deb !3
  5. 5. All builds are equal… (1) Think (2) Edit The developer’s work cycle (4) Test (3) Build !4
  6. 6. …But some builds are more equal than others “ http://xkcd.com/303/ Why is the gtk build so abysmally slow? This hinders our ability to run gtk bots and keep the gtk build green. :( ” !5
  7. 7. Refactoring can improve build performance deliverable1 del1.o del1.c deliverable2 library.a util1.o util2.o util1.c del2.o del2.c util2.c !6
  8. 8. Refactoring can improve build performance deliverable1 del1.o del1.c deliverable2 library.a util1.o util2.o util1.c del2.o del2.c util2.c !6
  9. 9. Refactoring can improve build performance deliverable1 del1.o del1.c deliverable2 library.a util1.o util2.o util1.c del2.o del2.c util2.c !6
  10. 10. Refactoring can improve build performance deliverable1 del1.o del1.c deliverable2 library.a util1.o util2.o util1.c del2.o del2.c util2.c !6
  11. 11. Refactoring can improve build performance deliverable1 del1.o del1.c deliverable2 library.a util1.o util2.o util1.c del2.o del2.c util2.c !6
  12. 12. Refactoring can improve build performance deliverable1 deliverable2 Before refactoring:! 4 commands! del2.o del1.o library.a triggered del1.c del2.c util1.o util2.o util1.c util2.c !6
  13. 13. Refactoring can improve build performance deliverable1 del1.o del1.c deliverable2 library.a util1.o util2.o util1.c del2.o del2.c util2.c !6
  14. 14. Refactoring can improve build performance deliverable1 deliverable2 del1.o util1.o util2.o del2.o del1.c util1.c util2.c del2.c !7
  15. 15. Refactoring can improve build performance deliverable1 deliverable2 del1.o util1.o util2.o del2.o del1.c util1.c util2.c del2.c !7
  16. 16. Refactoring can improve build performance deliverable1 deliverable2 del1.o util1.o util2.o del2.o del1.c util1.c util2.c del2.c !7
  17. 17. Refactoring can improve build performance deliverable1 deliverable2 del1.o util1.o util2.o del2.o del1.c util1.c util2.c del2.c !7
  18. 18. Refactoring can improve build performance deliverable1 deliverable2 After refactoring:! 2 commands! del2.o del1.o util1.o util2.o triggered del1.c del2.c util1.c util2.c !7
  19. 19. …But where should we focus refactoring effort? Files that rebuild the slowest? !8
  20. 20. (1) Dependency Graph Construction Dep. Graph (2) Dependency Graph Analysis Graph analysis results Version Control System (3) Build Hotspot Detection File Churn An approach to detect slowly rebuilding files B Qua !9
  21. 21. MAKAO tool extracts build dependency graphs (1) Dependency Graph Construction (2) Dependency Graph Analysis Design recovery and maintenance of build systems B. Adams, Herman Tromp, Kris De Schutter, Wolfgang De Meuter [ICSM 2007] !10
  22. 22. The cost of traversing an edge is derived by timing its build command (1) Dependency Graph Construction (2) Dependency Graph Analysis Median of 10 repetitions !11
  23. 23. Open source case studies! ! ! ! !12
  24. 24. Build dependency graph properties # Nodes # Edges 38,102 2,752,225 0 750,000 1,500,000 2,250,000 3,000,000 3,375 121,710 5,131 60,170 0 35,000 70,000 105,000 140,000 !13
  25. 25. 100 50 Build Time (s) 8 6 4 0 2 0 Build Time (s) 10 12 150 Incremental build performance 0 100 200 300 400 500 600 0 File ID 100 200 File ID .c files .h files !14 300 400
  26. 26. 150 50 100 Build Time (s) 1.5 1.0 0.5 0 0.0 Build Time (s) 2.0 Incremental build performance 0 200 400 600 800 1000 File ID 0 100 200 300 File ID .c files .h files !15 400 500 600
  27. 27. 6000 0 2000 4000 Build Time (s) 100 50 0 Build Time (s) 150 8000 Incremental build performance 0 2000 4000 6000 8000 10000 0 File ID 2000 4000 6000 File ID .c files .h files !16 8000 10000
  28. 28. 100 50 0 Build Time (s) 150 So, refactoring should target the slow files! 0 100 200 300 File ID !17 400 500 600
  29. 29. 100 50 0 Build Time (s) 150 So, refactoring should target the slow files! 0 100 200 300 File ID !17 400 500 600
  30. 30. 100 50 0 Build Time (s) 150 So, refactoring should target the slow files! 0 100 200 300 File ID !17 400 500 600
  31. 31. 100 50 0 Build Time (s) 150 So, refactoring should target the slow files! 0 100 200 300 File ID !17 400 500 600
  32. 32. Let’s see what developers think! > Please answer the following questions for each source file listed above:! > (1) Is this source code file a performance bottleneck? Y/N! ! For all of them no... !18
  33. 33. Let’s see what developers think! > Please answer the following questions for each source file listed above:! > (1) Is this source code file a performance bottleneck? Y/N! ! For all of them no...! ! > (2) Why do you believe this file is or is not a performance bottleneck?! ! ...because none of these headers change often. !19
  34. 34. …But where should we focus refactoring effort? Files that rebuild the slowest? !20
  35. 35. …But where should we focus refactoring effort? Files that rebuild the slowest? …but they may only rarely change! !20
  36. 36. …But where should we focus refactoring effort? Files that rebuild the slowest? Files that change the most often? …but they may only rarely change! !20
  37. 37. Mining frequently changing files from version control Terminal $ git clone … … $ git checkout <some_tag> … $ git log --oneline <some_file> !21
  38. 38. Mining frequently changing files from version control But many of these files $ git clone … … already build quickly! Terminal $ git checkout <some_tag> … $ git log --oneline <some_file> !21
  39. 39. …But where should we focus refactoring effort? Files that rebuild the slowest? Files that change the most often? …but they may only rarely change! !22
  40. 40. …But where should we focus refactoring effort? Files that rebuild the slowest? Files that change the most often? …but they may only rarely change! …but they may already be optimal! !22
  41. 41. …But where should we focus refactoring effort? Files that rebuild the slowest? Files that change the most:often? ots …but they may only rarely change! …but they may already be optimal! otsp h ft e n uild ge o on b han cus nd c Fo ly a slow build at re es th il F !22
  42. 42. (1) Dependency Graph Construction Dep. Graph (2) Dependency Graph Analysis Graph analysis results Version Control System (3) Build Hotspot Detection File Churn An approach to detect slowly rebuilding files B Qua !23
  43. 43. (1) Dependency Graph Construction Dep. Graph (2) Dependency Graph Analysis Graph analysis results Version Control System (3) Build Hotspot Detection File Churn Hotspot detection approach Build Activity Quadrant Plot !24
  44. 44. Quadrant plots highlight build hotspots Number of changes (1) Dependency Graph Construction (2) Dependency Graph Analysis (3) Build Hotspot Detection Rebuild cost !25
  45. 45. Quadrant plots highlight build hotspots Number of changes (1) Dependency Graph Construction (2) Dependency Graph Analysis (3) Build Hotspot Detection Rebuild cost !25
  46. 46. Quadrant plots highlight build hotspots Number of changes (1) Dependency Graph Construction (2) Dependency Graph Analysis (3) Build Hotspot Detection Rebuild cost !25
  47. 47. Number of changes Quadrant plots highlight build hotspots factor Re e first! thes (1) Dependency Graph Construction (2) Dependency Graph Analysis (3) Build Hotspot Detection Rebuild cost !25
  48. 48. Open source case studies! Thresholds ! Rebuild cost 90 seconds Number of changes ! Median ! !26
  49. 49. 1.00 ● ● ● ● ● ● ● ● Number of changes (Normalized) Normalized File Churn ● ● ● ● ● 0.25 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 65 hotspots! (7% of source files) ● ● ● ● 0.75 0.50 ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●●● ● ● ● ●● ● ● ●● ●● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ●●●●● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● 0 ●● ●● ● ● glib/glib.h glib/glib-object.h ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● Main culprits ● ●● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ●●● ●● ● ● ● ● ● ● ● ● 50 ● ● ●● ● Build Time ● ● ● ● ● 100 150 Rebuild cost (seconds) !27
  50. 50. 1.00 ● ● ● ● ● ● ● Number of changes (Normalized) Normalized File Churn ● 0.75 0.50 0.25 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ●● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ●● ●● ●● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ●●● ● ● ●● ● ●● ● ● ●● ● ● ● ● ● ●●●● ●● ● ●●●● ●● ●● ●● ● ● ●●● ●●● ● ●●● ● ●● ● ● ● ●●●● ●● ● ● ●●●● ● ●● ● ●● ● ●● ● ● ● ●●●●●●●● ● ●●● ●●●● ●●● ● ● ● ●● ● ●● ●● ● 732 hotspots! (8% of source files) ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ●● ● ●● ●● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ●●● ● ● ●● ●● ● ● ●● ●● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ●● ● ●● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●●●●●● ● ● ● ● ●● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● Main culprits tmtm ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ●● ● ●●● ● ● ●● ●● ●● ● ● ●● ● ● ● ●●● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ●● ● ● ●● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ●●●● ●● ● ● ●● ●● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●●●● ●● ● ● ● ●●●●●●●●●● ● ● ●●● ● ●●●● ● ● ● ●●● ●●●● ● ●● ● ● ●● ● ●●● ● ● ●● ●●● ● ● ● ●●● ● ●●● ● ● ● ● ● ●● ● ● ● ● ● ● ●●●●●●●● ● ● ● ●●●●●● ●● ● ●●● ●● ●● ●●●● ● ● ●●● ● ●● ●●● ●●● ● ●● ● ●● ● ●● ●● ● ● ● ● ● ● ● High hotspot! concentration qtbase/…/corelib qtxmlpatterns ●●●●●● ● ● ●●●● ● ●●●● ● ●●● ●●● ● 0 2000 4000 Build Time (s) 6000 8000 Rebuild cost (seconds) !28
  51. 51. 1.00 ● ● ● ● ● ●● ● ● ● Number of changes (Normalized) Normalized File Churn ● ● ● ● ● ● 0.75 0.50 0.25 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●●● ● ●● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ●● ● ●●●● ● ● ● ● ● ● ● ● ●●● ● ● ●● ● ●● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ●● ● ● ●● ● ● ●● ●● ● ● ● ● ● ●●● ●● ● ● ● ● ●● ● ● ● ● ●● ● ●●● ● ● ● ● ● ● ●● ● ●● ● ● ●● ● ● ●● ● ● ●● ● ● ● ● ●● ●● ● ● ●● ● ● ●● ● ● ●● ● ●● ● ● ●● ● ● ● ● ● ●● ● ● ●● ● ●● ●● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ●● ●● ● ●● ● ● ● ● ● ● ● ●●● ●● ● ● ● ●● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●●● ● ●● ● ● ● ●● ●● ● ● ●● ● ●● ● ●● ●● ●● ● ● ● ● ● ● ● ●● ●● ●● ● ● ●● ● ●● ● ● ● ●● ● ● ● ●● ● ● ●● ●● ●●● ●● ● ●●● ●● ● ● ● ● ●● ●● ●● ● ● ● ●● ● ●● ●● ● ● 27 hotspots! (2% of source files) ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● Main culprits ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● postgres.h access/htup.h access/genam.h access/xact.h utils/builtins.h ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ●● ● ● ●●● ● ●●● ● ● ● ● ●● ● ● ● ● ● ● ●● ●● ● ● ●● ●● ●● ● ● ● ● ● ● ● ●● ● ● 0 ● 50 100 Build Time (s) 150 200 Rebuild cost (seconds) !29
  52. 52. General trends Transitive property of build hotspots 1 2 ... n-1 n hot.o hot.c !30
  53. 53. General trends Transitive property of build hotspots 1 2 ... n-1 n hot.o hot.c !30
  54. 54. General trends Transitive property of build hotspots 1 2 ... n-1 n hot.o hot.c x.h !30
  55. 55. General trends Transitive property of build hotspots 1 2 ... n-1 n hot.o hot.c x.h !30
  56. 56. 1 2 ... n-1 n General trends hot.o Transitive property of build hotspots hot.c x.h !31
  57. 57. 1 2 ... n-1 n General trends hot.o Transitive property of build hotspots hot.c x.h !31
  58. 58. 1 2 ... n-1 n General trends hot.o Transitive property of build hotspots hot.c x.h hub.h !31
  59. 59. 1 2 ... n-1 n General trends hot.o Transitive property of build hotspots hot.c a.h b.h x.h y.h z.h hub.h !31
  60. 60. 1 2 ... n-1 n General trends hot.o Transitive property of build hotspots hot.c a.h b.h x.h y.h z.h hub.h !31
  61. 61. 1 2 ... n-1 n General trends hot.o Transitive property of build hotspots hot.c a.h b.h x.h y.h z.h hub.h !31
  62. 62. 1 2 ... n-1 n General trends hot.o Transitive property of build hotspots hot.c a.h b.h x.h y.h z.h hub.h Limit internal use of “header file hubs” as much as possible! !31
  63. 63. 2nd International Workshop on Release Engineering RELENG 2014 ? ? http://releng.polymtl.ca
  64. 64. 2nd International Workshop on Release Engineering RELENG 2014 keynotes ? ? http://releng.polymtl.ca talks on research and practice poster session and panel Deadline: February 28, 2014 Event: April 11, 2014
  65. 65. 2nd International Workshop on Release Engineering RELENG 2014 keynotes http://releng.polymtl.ca ? ? hosted in Mountainview (CA, US) by talks on research and practice poster session and panel Deadline: February 28, 2014 Event: April 11, 2014
  66. 66. 2nd International Workshop on Release Engineering RELENG 2014 keynotes McNutt Dinah le) (Goog hosted in Mountainview (CA, US) by ? http://releng.polymtl.ca talks on research and practice poster session and panel Deadline: February 28, 2014 Event: April 11, 2014
  67. 67. @shane_mcintosh shanemcintosh@acm.org
  68. 68. Bonus! !34
  69. 69. postgres.h src/include/c.h src/include/port.h src/include/utils/elog.h src/include/utils/palloc.h src/include/pg_config_manual.h !35
  70. 70. htup_details.h src/include/access/htup.h src/include/storage/itemptr.h src/include/storage/itemid.h src/include/storage/bufpage.h src/include/access/tupdesc.h src/include/catalog/pg_attribute.h src/include/access/tupmacs.h !36
  71. 71. genam.h src/include/access/skey.h src/include/storage/lock.h src/include/storage/shmem.h src/include/utils/hsearch.h src/include/storage/lwlock.h src/include/utils/relcache.h !37
  72. 72. xact.h src/include/access/xlog.h src/include/access/xlogdefs.h src/include/nodes/pg_list.h src/include/lib/stringinfo.h !38
  73. 73. builtins.h src/include/fmgr.h src/include/nodes/parsenodes.h src/include/nodes/primnodes.h src/include/nodes/pg_list.h src/include/nodes/nodes.h !39

×