Nokta Digital Content Solutions by Hakan Kocakulak

1. Nokta Digital Content
Solutions
Technical Competency Presentation
Hakan Kocakulak
CTO of Noktacom Medya İnternet
Teknolojileri

3. Nokta in Nutshell

4. In 2013, according to comScore,
we have reached to %69 of
Turkish Internet Users
Nokta in Nutshell
Source; , Media Metrix, Jan-June 2013, Total Number of Internet Users

5. Source; , Video Metrix, July 2013

6. More than
40.000 trailers
2009
2010
2011
2012
2013
1.
Source; , Media Metrix, July 2013

7. comScore
Entertainment-
Humor
%42
Reach and ranks
1st
Source; , Media Metrix, July 2013

8. 37M monthly pageviews
Source; , Media Metrix, July 2013

9. comScore
Business/Financi
al
Information
category
%46
Reach and ranks
1st
Source; , Media Metrix, July 2013

10. Source; , Media Metrix, July 2013

11. Source; , Media Metrix, July 2013

12. Proven Track Record in Live
Digital Events
• Vodafone high school music
contest was broadcast live on
Izlesene and Twitter at the
same time through Izlesene
portal. (Izlesene the first
player in Turkey to broadcast
live on Twittter)
• Can Bonomo concert was
broadcast live on Izlesene
channel and got in Worldwide
& Turkish TT list on Twitter on
a soccer night.

13. NOKTA MOBILE ECOSYSTEM / EMPOWER – GROWING MOBILE AUDIENCES
1.4 M+ 12 M+
10 M+ 25:00 min

14. LOREM IPSUM
15 M+ Monthly
Visitors
350M+ Monthly
Video Views
60M+ Monthly
Visits
Time Spent
16:00 min
NOKTA WEB ECOSYSTEM

15. CDN & Private CDN
• Why Content Delivery Network? (CDN)
• CDN caches contents to the nearest POP Server
• Pulls popular content from origin, caches temporarily & saves
bandwidth, saves cost
• Reduces network latency & improves user experience
• Highly Available, when the nearest pop is dead, streams from other
pops
• Why Adaptive Stream?
• Stream quality is selected dynamically by player
• When the bandwidth to the client on any network segment is
saturated, the client reduces the quality and vice versa
• Less buffer emptying occurs and improves user experience
• Segmented, Less VOD (video on demand) start & seek time
• Less storage area for mpeg dash packing

16. CDN Services & Features
• Video & Audio Content Streaming (Both VOD and Live)
• Adaptive Stream, Mpeg Dash (Desktop, Android)
• Adaptive Stream, HLS (IOS)
• Progressive Stream, mp4 (Legacy Platforms)
• RTMP Streaming
• Static Content Caching
• Images, css, js and any static content
• Real time image resize
• Dynamic web site accelerator
• Traffic Management
• DC & Geography Awareness
• Carrier sensitivity
• Rule based
• Low cost
• Secure Links
• Transfer Rate Limits
• Monitoring
• Highly Available
• Player
• Flash player, html5 player, ios app player, android player
• Can integrate with big data analysis service

17. Video Transcoding Services (VTS)
• Fast parallel content download from source
• Parallel mpeg dash, hls and mp4 transcoding
• Configurable resolutions and bitrates, H264 and AAC encodings
• Up to 4k videos transcoded
• Image capture from video content within configurable intervals
• Storyboard image creation
• Optional three replica, highly available object storage service for
transcoded videos and images
• Optional audio fingerprint service
• Fully integrated with CDN service

18. Big Data Analyzer (BDA)
• Real Time and Batch Analysis
• Video views with several breakdowns (platform, category, ownership, etc)
• Video completion rate
• Player buffer rate
• Player connection error rate
• Last mile analysis
• Page view, visit, ctr, etc.
• Any definable metric
• Real time comparison with previous day and previous week day
• A/B Testing
• Data driven solutions (Recommendations, Personalization, Search)

19. Some Highlights
• 65 Gb/s peak traffic
• 150k visit/s at whole Nokta products
• 20k video start/min
• 1+ billion video start/month
• 150k ad view/s
• image load time < 100 ms , ad load time < 20 ms
• 25 MB/s log, 150 GB/day log, 10k msgs/s
• 25 k req/s at load balancer
• 5000 Video/day transcoded to 6 different formats
• 300+ VM on private cloud
• 150+ server
• Layer 1, 2, 3 network administration, monitoring and maintenance
• 200+ metrics on BDA
• 400+ hosts, 10000+ items monitored with 2000+ triggers
• Tubitak Research Projects
• Kullanıcı Video İzleme Davranışlarından Profil Tanıma Sistemi
• Eşle İndirge ile Bulut Üzerinde Yığın Video İşleme Sistemi

20. Behind the Scenes, Storage
• Open Source Distributed Object & Block Storage Ceph
• Highly available, infinitely scalable, DC & rack aware, data redundancy with
replication
• Current infra works on 45 servers
• Each server has 12 disk slot and each slot has 3.5 inch 4 TB NL sas disk and
2x2.5 inch 300 GB OS disk
• No RAID configuration for data disks, RAID 1 for OS disks
• Each server has 1x10 Gb/s, 2x 1 Gb/s, 1x 100 Mb/s interface
• Each data has three replica
• S3/Swift API for Object Storage, qemu end kernel driver for block storage
• Works as object storage for videos, images, statics etc
• Works as networked block storage for os & data volumes of virtual servers
working on Private Cloud

21. Behind the Scenes, Private Cloud
• Based on Open Stack
• Current infra works on 20 servers
• 2-slot 16-core Xeon servers 64-128G RAM
• 1x10 Gb/s, 2x 1 Gb/s, 1x 100 Mb/s network interface
• Multi-tenant networking
• KVM based OS virtualization
• Fast VM provisioning
• Cobbler and Puppet for bare-metal provisioning and Configuration
Management

22. Behind the Scenes, CDN
• 12-14 Gb/s attainable throughput per server for high-cardinality static
content. More for viral content or live streams
• Based on nginx with lua (openresty suite), managed in-house
• Fast SAS disks for caching
• 2x 10Gb/s, 2x1 Gb/s, 2x100 Mb/s network interface
• Highly Available
• Traffic Management
• DC & Geography Awareness
• Carrier sensitivity

23. Behind the Scenes, Transcoding
• Resource scheduling with Apache Zookeeper
• Services have been developed in Python
• Whole extensive video & audio formats supported
• Content is downloaded and transcoded in parallel.
• Deployed on 15 nodes
• Configurable resolutions & bitrates & formats, H264 and AAC encodings
• Up to 4k videos are transcoded
• Images are extracted from video at configurable interval

24. Behind the Scenes, Big Data Analysis
• 2x 20 server cluster
• One cluster for sensitive ads data
• One cluster for analytics purpose
• HDFS, Hadoop, Hbase, zookeeper, flume, storm, spark, impala, kafka are
used
• Services are coded using Scala and Java

25. Behind the Scenes, Monitoring
• BDA product is used for business metrics
• Grafana is used for Visualizing Time Series data with back end Hbase
• Zabbix is used for Network and Hosts
• OMSA is used for physical host monitoring

26. Q & A
26
Hakan Kocakulak, CTO of Noktacom Medya İnternet Teknolojileri
hakankocakulak@nokta.com
http://tr.linkedin.com/in/hakankocakulak