This document discusses aggregations in Druid. It explains that aggregations involve intermediate aggregator functions (IAFs) that aggregate input rows during indexing and combining aggregator functions (CAFs) that merge the outputs of IAFs during querying. It notes that IAFs operate on rows while CAFs operate on the outputs of IAFs. It also discusses the use of serializers and deserializers when reading from and writing to disk during indexing and querying, and mentions that intermediate and indexing types can be different for complex metrics.

8. • M1 : [1,3,5] —> 1+3+5 —> [9]
• M2 : [2,4,6] —> 2+4+6 —> [12]
• R1 : [9,12] —> 9+12 —> [21]

20. Guice
Jackson

26. note : UDAF get values from selectors not as parameters
String or String[] : DimensionSelector
float : FloatColumnSelector
long : LongColumnSelector
others : ObjectColumnSelector

28. ???

29. ???

30. ???

31. ???

32. ???

33. ???

36. Yep, apparent

37. Makes cache key (query cache)
: “name” is not to be included in cache key

38. Deserialize complex intermediate object
: WTF intermediate object? : see this later (in types)
: It’s for handling jackson serialized (see @JsonValue)
: Input can be a String, byte[] or ByteBuffer

39. Not used anywhere

40. For sorting on intermediate object
: ordered limiting in group-by or top-n queries
: Ordering.natural() for primitive types
To merge AFs of different versions
For Nested Group-By Query
: resolves alias

41. output-column (alias)
output-type
input-columns

42. iterate/merge
merge
finalize
buffer size to be allocated
???

43. ??

46. getCombiningFactory()

47. Ok, we know this

51. H1 : [1,4,7] —> 1+4+7 —> [12] —>
H1 : [2,5,8] —> 2+5+8 —> [15] —>
H2 : [3,6,9] —> 3+6+9 —> [18] —>
—> H1:[12]
—> H1:[15]
—> H2:[18]
IAF.getAggregator()
Aggregator.aggregate()
Aggregator.get() ( SerDe.toBytes() )
CAF.getAggregator()
—>12+15 —>[27] [27]
—————————>[18] [18]
QUERY (Historical) : uses CAF
Aggregator.aggregate()
—> 27+18 —>[45]
IAF.combine()
IAF.finalize()
INDEXING (Realtime) : uses IAF
Aggregator.get()
( SerDe.fromByteByffer() )
JSON
QUERY (Broker) : why IAF?
( Extractor.extractValue() )

54. Should be the same
note : technically, you can use different types but don’t
Optional
(Accepts intermediate type)
Druid don’t care on this
(Just be sure it can be a JSON)

55. Intermediate Type
Indexing type
Let’s call…

60. ???

61. Intermediate-Type
Indexing-Type ?
: implementation dependent

62. Only For Complex types
Indexing-Type
Intermediate-Type

63. IAF
CMExtractor
ComplexMetricSerDe1
ObjectStrategy
getTypeName()
getTypeName()
Aggregator
Parser
COLUMN_DATA
COLUMN_TYPE
INDEX-FILE
CMExtractor
ObjectStrategy
getTypeName()
COLUMN_TYPE
COLUMN_DATA
INDEX-FILE CAF
Aggregator
ComplexMetricSerDe2
getTypeName()
• INDEXING
• QUERY
InputRows
toBytes()
fromByteBuffer()
extractValue()
ByteBuffer.map()
parse()
NOT_USEDNOT_USED
NOT_USED

68. Sorry, let’s do this in next meetup
…or not