"Introduction to Kx Technology", James Corcoran, Head of Engineering EMEA at First Derivatives About the Author: James is Senior Vice President, Fast Data Solutions at Kx where he has worked as a developer since 2009. In his career to date, he has worked in the algorithmic trading space at many of the world’s top financial institutions using Kx - a low latency technology for analysing time series data. He is a certified Professional Risk Manager and holds a masters in Quantitative Finance from University College Dublin. In recent years he has built systems for clients ranging from start-ups to blue chip companies in data intensive industries such as pharma, utilities and telco.

1. Kx Introduction
November 2016

2. 2 | KX SYSTEMS
Agenda
• About
• Kx Technology
• Solutions
• Technical Overview
• Resources
• Q & A

3. 3 | KX SYSTEMS
About
First Derivatives plc
People
1,600 FTEs
Locations
Ireland, London, New York,
Singapore,
Sydney, Tokyo,
Toronto, Zurich
History
Founded in 1996
Public listed Company
Always Profitable
Divisions
Kx
Big Fast Data
Capital Markets
Consulting
Domain experts
Technology experts
MRP/Prelytix
Digital marketing
resources
Our Software
Kx
Streaming Analytics, In
Memory Computing, -
the market standard
Tools for Kx
Develop, Deploy,
Visualize
Industry Solutions
Powered by Kx
Implemented and
supported by us

4. 4 | KX SYSTEMS
Kx Technology Overview
• Integrated columnar database &
programming system
• Streaming, real time and historical data
• Built for massive data volumes
• In-database analytics
• Parallelism
• Compression
• More
The kdb+ database Features
• Interpreted
• Event-driven
• Functional
• Array / Vector
• Query
• Time-series
q Programming Language

5. 5 | KX SYSTEMS
Tools for Kx Solutions
Solutions Overview
• Dashboards for Kx
• Analyst for Kx
• Control for Kx
• Stream for Kx
• Kx for Algos
• Kx for Flow
• Kx for Monitoring
• Kx for Pharma
• Kx for Surveillance
• Kx for Sensors
• Kx for Telcos
Visualise
&
Discover
Architect
&
Manage
Enterprise Concepts

6. 6 | KX SYSTEMS
kdb+
Technical Overview

7. 7 | KX SYSTEMS
kdb+
• Integrated columnar database & programming system
• Streaming, real time and historical data
• 32 bit free version
• 64 bit licensed version

8. 8 | KX SYSTEMS
kdb+ Features
• Standard OS & hardware
• In-database analytics
• Compression
• Parallelism
• Interfaces

9. 9 | KX SYSTEMS
q programming language
• Functional
• Array/Vector
• Query
• Interpreted
• Time-series

10. 10
q Language
Seamless database and data manipulation language
• We are fast not only due to data architecture, our native
programming language, q, runs inside the database not
in separate processes with costly data passing.
• We are one of the few fully 64 bit databases and unique
in having time as a native type, with nanosecond
resolution and a full set of operations over time.
Quick to Deliver
• Unlike many compile-link and run approaches q is
dynamic allowing much shorter development and
deployment cycles
Allows you to get more from less code
• q is terse - this means a smaller code base to maintain
and debug (one line in q versus many of Java).
Other Key Features
• Array/Vector Language
• Parallelism
• Native Compression
Java
Project Euler Problem 1* in Java
Project Euler Problem 1* in q
*Find the Sum of all multiples of 3 or 5 below 1000. For more information on the Euler Programming Problems see - https://projecteuler.net

11. 11
Sample q query
select open: first price, high: max price, low:
min price, close: last price from trade where
date = 2013.05.01, sym = `AAPL

12. 12
q vs. standard SQL
• q supports order for faster execution
• q eliminates many joins
• Standard SQL does not enforce row order
• Built-in support for CSV, JSON & XML
• higher productivity & less maintenance

13. 13
q vs. standard SQL
q:
select rev wavg s.n=`BRAZIL by o.d.year from l where
o.c.n.r=`AMERICA, o.d.year in 1995 1996, p.t=`$"ECONOMY ANODIZED STEEL"
SQL:
select year,sum(case when name='BRAZIL' then rev else 0 end)/sum(rev) from(
select extract(year from o.d)as year,l.x*(1-l.xd) as rev,n2.name
from p,s,l,o,c,n n1,n n2,r
where p.p=l.p and s.s=l.s and l.o=o.o and o.c=c.c and c.n=n1.n and
n1.r=r.r and r.name='AMERICA'
and s.n=n2.n and o.d between date'1995-01-01' and date'1996-12-31' and
p.t='ECONOMY ANODIZED STEEL')t
group by year order by year

14. 14
Parallel queries in q
Serial:
calcEvent each eventTypes
Parallel:
calcEvent peach eventTypes

15. 15
Time-Series Analysis
• Native temporal data types
• high-precision (nanosecond)
• temporal arithmetic
• asof join/bi-temporal joins
• join time-series on-the-fly

16. 16
Time-Series Analysis cont’d
select count i by userAgent,5 xbar time.minute from
events where date within (.z.D - 7;.z.D),
eventType=`login
aj[`time; select time,price from trade where
date=last date,sym=`MSFT; select time,bid,ask from
quote where date=last date,sym=`MSFT]

17. 17
q v Hive ETL comparison
q:
`product_dtl set (“ISFS”; enlist “,”) 0: `input.txt;
Hive:
create table product_dtl ( product_id: int, product-name: string, product_price:
float, product_category: string) rows format delimited fields terminated by ',' ;
describe product_dtl;
load data local inpath ’input.txt' into table product_dtl;
Hive –f /home/cloudera/sample.sql

18. 18
Streaming, Real-Time and Historical Data

19. 19 | KX SYSTEMS
Sample Architecture

20. 20 | KX SYSTEMS
Kx Solutions
Technical Overview

21. 21 | KX SYSTEMS
Dashboards for Kx

22. 22 | KX SYSTEMS
Analyst for Kx

23. 23 | KX SYSTEMS
Streams for Kx
Fast Data Management
• Clustered services
• Load-balancing
• Failover
• Automated recovery
• Query Manager
• Messaging Server
• Alert Framework
• File Watchers & Loaders
• Scheduling

24. 24 | KX SYSTEMS
Kx
Resources

25. 25 | KX SYSTEMS
Resources – Wiki Page

26. 26 | KX SYSTEMS
Resources – Kx Community

27. 27 | KX SYSTEMS
Resources – Forum & Google Group

28. 28 | KX SYSTEMS
Resources – Github

29. 29 | KX SYSTEMS
Resources – Q For Mortals

30. 30 | KX SYSTEMS
Resources – Q Tips

31. 31 | KX SYSTEMS
Resources – Free 32-bit download

32. 32 | KX SYSTEMS
Resources – Kx Community Meetups

33. 33 | KX SYSTEMS
Resources – Lecture Series

34. 34 | KX SYSTEMS
Thank You.
kx.com
@kxsystems

35. 35 | KX SYSTEMS
Appendix
• Kx Wiki
• Kx Community
• Google Group
• Kx Github
• Q For Mortals
• Q Tips
• STAC benchmarks
• Free version
• Kx Meetups
• FD Lecture Series