This case study gives an inside look at optimization of the MongoDB Perl driver, including custom benchmarking tools, step-by-step changes and results that will surprise and amaze. If you ever needed to optimize some Perl and wondered how people go about it, this talk is for you.

1. Real World
Optimization
David Golden • @xdg
Lead Engineer, MongoDB
The Perl Conference 2017

2. Real world...
Real code...

3. MongoDB Perl Driver
v0.x — legacy
v1.x — next gen

4. Benchmarking

5. Isolate one thing
at a time

6. Don't trust averages

7. Distributions,
not averages

8. Profile just like you
benchmark

9. Wrote my own tools

10. Load test data

11. Load test data
Setup for all runs

12. Load test data
Setup for all runs
Prep before each run

13. Load test data
Setup for all runs
Prep before each run
Run the benchmark once

14. Load test data
Setup for all runs
Prep before each run
Run the benchmark once
Cleanup after each run

15. Load test data
Setup for all runs
Prep before each run
Run the benchmark once
Cleanup after each run
Teardown after all runs

16. Load test data
Setup for all runs
Prep before each run
Run the benchmark once
Cleanup after each run
Teardown after all runs
Vary scale here

17. Load test data
Setup for all runs
Prep before each run
Run the benchmark once
Cleanup after each run
Teardown after all runs
Vary scale hereVary scale here
N

18. Load test data
Setup for all runs
Prep before each run
Run the benchmark once
Cleanup after each run
Teardown after all runs
Vary scale hereVary scale here
N
Only time this!

19. Load test data
Setup for all runs
Prep before each run
Run the benchmark once
Cleanup after each run
Teardown after all runs
Vary scale hereVary scale here
N
Only profile this!

20. Load test data
Setup for all runs
Prep before each run
Run the benchmark once
Cleanup after each run
Teardown after all runs
Vary scale hereVary scale here
N
Collect ALL of these timings

21. $ mpd-bench -v v0.708.2.0
-d twitter -n 100 -m insert_one

22. $ mpd-bench -v v0.708.2.0
-d twitter -n 100 -m insert_one

23. $ mpd-bench -v v0.708.2.0
-d twitter -n 100 -m insert_one

24. $ mpd-bench -v v0.708.2.0
-d twitter -n 100 -m insert_one

25. $ mpd-bench -v v0.708.2.0
-d twitter -n 100 -m insert_one
$ mpd-bench -v $(git describe)
-d twitter -n 100 -m insert_one
--blib

26. $ mpd-bench -v v0.708.2.0
-d twitter -n 100 -m insert_one
$ mpd-bench -v $(git describe)
-d twitter -n 100 -m insert_one
--blib
$ ./chart-metric -s twitter
--verbose --index

27. “Candlestick Graph”

28. $ mpd-bench -v $(git describe)
-d twitter -n 100 -m insert_one
--blib --profile

29. $ mpd-bench -v $(git describe)
-d twitter -n 100 -m insert_one
--blib --profile
$ nytprofhtml -d -f
nyt-twitter-100-insert_one-v0.999.999.3-9-g710bf6e.out
$ open nytprof/index.html

30. “Flame Graph”

31. Call stack
Execution time
Left-to-right order is meaningless
Color is meaningless

32. Moose::Object::new

33. coercion to Tie::IxHash

34. “real work”

37. __PACKAGE__->meta->make_immutable;

39. Stop coercing to
Tie::IxHash

40. Better?

41. Mostly “real work” now

44. Heavy constructors?

46. 4x, 3x, 2x
number of inserts?

47. Drill into the top
function

50. Line Subs
Stmt Calls

52. Why three
can_read
calls per write?

53. 1. Check if server is connected by polling if
able to read and if so, whether it's EOF

54. 1. Check if server is connected by polling if
able to read and if so, whether it's EOF
RACE
CONDITION
DELETE IT!

55. 2. Used for timeout while reading response
from the server

56. Why two reads
per response?!?

57. Read algorithm
Read 4 bytes to get document length
Read length - 4 to get rest of the document

58. D'oh!

59. Read algorithm
Read 4 bytes to get document length
Read length - 4 to get rest of the document
Better
v
Read as many bytes as we can
Read more only if that's not enough

60. Oops!

61. Tie::IxHash change
broke bulk insertion

62. Wasn't running full
tests after each commit!

63. Always run tests!

67. What's next?

69. One-by-one option overrides

70. $self is a hash, so let Perl merge them

72. Let's talk OO

73. Let's talk OO
Moose
v

75. isa => Str

76. isa => Str
Do we really need these at runtime?

77. isa => Str
Do we really need these at runtime?
WITH_ASSERT ? ( isa => Str ) : ()

79. Not enough!

80. ‘Moose showing tongue’
https://www.flickr.com/photos/31176607@N05/11934293913

81. Moo
+
Class::XSAccessors

82. Constructors still
doing a lot of work

83. Private constructors
for internal classes

84. package MongoDB::Role::_PrivateConstructor;
use Moo::Role;
use MongoDB::_Constants;
BEGIN {
WITH_ASSERTS
? eval 'sub _new { my $c = shift; $c->new(@_) }'
: eval 'sub _new { my $c = shift; bless {@_}, $c }';
}

85. package MongoDB::Role::_PrivateConstructor;
use Moo::Role;
use MongoDB::_Constants;
BEGIN {
WITH_ASSERTS
? eval 'sub _new { my $c = shift; $c->new(@_) }'
: eval 'sub _new { my $c = shift; bless {@_}, $c }';
}

86. package MongoDB::Role::_PrivateConstructor;
use Moo::Role;
use MongoDB::_Constants;
BEGIN {
WITH_ASSERTS
? eval 'sub _new { my $c = shift; $c->new(@_) }'
: eval 'sub _new { my $c = shift; bless {@_}, $c }';
}

87. package MongoDB::Role::_PrivateConstructor;
use Moo::Role;
use MongoDB::_Constants;
BEGIN {
WITH_ASSERTS
? eval 'sub _new { my $c = shift; $c->new(@_) }'
: eval 'sub _new { my $c = shift; bless {@_}, $c }';
}

88. package MongoDB::Role::_PrivateConstructor;
use Moo::Role;
use MongoDB::_Constants;
BEGIN {
WITH_ASSERTS
? eval 'sub _new { my $c = shift; $c->new(@_) }'
: eval 'sub _new { my $c = shift; bless {@_}, $c }';
}
Can restore all the isa checks, too.

89. Other tidbits...

90. No moreTry::Tiny

91. Eliminate most
Type::Tiny coercions

92. Time::HiRes
Avoid tv_interval()
Use decimal time()

93. Cache
boolean::true()
and
boolean::false()

94. Cache ‘primary’ server
between topology scans

95. Cache invariants

96. CACHE!

97. Stop checking for
fork/spawn

98. Ignore rare errors that
will be caught later

99. Really marginal benefits,
but...

100. Avoid scopes

101. if ( bad_condition() ) {
Throw->("ouch!")
}
Throw->("ouch")
if bad_condition();

102. if ( $server_version > 2 )
do_current( $stuff );
}
else {
do_legacy( $stuff );
}
$server_version > 2
? do_current( $stuff )
: do_legacy( $stuff );

103. Avoid intermediate
variables

104. Avoid statements

105. $hash{foo} = $arg{foo} || '0' ;
$hash{bar} = $arg{bar} || '1' ;
$self->check( $one, $two ) ;
($hash{foo} = $arg{foo} || '0') ,
($hash{bar} = $arg{bar} || '1') ,
($self->check( $one, $two ) ) ;

106. Is all this working?

107. Winning by hitting singles!

110. Benchmark::Lab
(a work in progress*)

111. Questions?