• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
WHAT / WHY / HOW WE’RE ENGINEERING AT SMARTSTUDY (English)
 

WHAT / WHY / HOW WE’RE ENGINEERING AT SMARTSTUDY (English)

on

  • 1,308 views

 

Statistics

Views

Total Views
1,308
Views on SlideShare
1,264
Embed Views
44

Actions

Likes
5
Downloads
7
Comments
0

2 Embeds 44

https://twitter.com 41
http://woworiya.wordpress.com 3

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    WHAT / WHY / HOW WE’RE ENGINEERING AT SMARTSTUDY (English) WHAT / WHY / HOW WE’RE ENGINEERING AT SMARTSTUDY (English) Presentation Transcript

    • WHAT / WHY / HOW WE’RE ENGINEERING AT SMARTSTUDY Park Hyunwoo / @lqez 2013-05
    • • We started at Jun, 2010. • We do engineering for products and services, not for playing with breeding-edge technologies. • 26 freaks in this ship. And 14 of them are hackers. (May 2013) We are “Service Freak”
    • This slide describes: What we do? and Why we do it? and How we did it and will do it?
    • So WHAT
    • WHAT What we’ve created. } • Mobile Apps • Log Crawler • Log Analyzer • Push Solution • Web-based tools • Open Source
    • But WHY
    • WHY •Mobile Apps • Log Crawler • Log Analyzer • Push Solution • Web-based tools • Open Source Mobile application markets are fair.Anybody can upload their apps and deploy globally. : We’ve developed and deployed 300+ apps in mobile era.
    • WHY • Mobile Apps •Log Crawler • Log Analyzer • Push Solution • Web-based tools • Open Source We knew that the success of business is up to keep eyes on the behavior of users. : We adapt technologies and experiences what we learned from 10y+ online game services.
    • WHY • Mobile Apps • Log Crawler •Log Analyzer • Push Solution • Web-based tools • Open Source a Bunch of logs has no meaning. We have to examine them in various ways via visualization. : We polish logs into Data, and again to Information, at last, we’ll get Insight from it.
    • WHY • Mobile Apps • Log Crawler • Log Analyzer •Push Solution • Web-based tools • Open Source In mobile market, push message works as same as email. Even more powerful. : We build redundant storing & massive reservation system.
    • WHY • Mobile Apps • Log Crawler • Log Analyzer • Push Solution •Web-based tools • Open Source Web is the best platform to serve tools without a headache of updating, and runs on every machines. : We stand on HTML5 with high-productivity framework like Django.
    • WHY • Mobile Apps • Log Crawler • Log Analyzer • Push Solution • Web-based tools •Open Source Open source is not only codes, but also culture. : We uses open sources and open our codes to the world.
    • Then HOW
    • HOW/ Mobile apps
    • HOW/ Mobile apps OpenGLNativeWeb
    • HOW/ Mobile apps OpenGLNativeWeb
    • OpenGLNativeWeb HOW/ Mobile apps
    • HOW/ Mobile apps OpenGLNativeWebThere are no boundary or no fixed rules. We just choose the best fit architecture.
    • HOW/ Mobile apps iOS / Android Native API OpenGL SMARTSTUDY Library Cocos2d-x C/C++ NDKPython HTML Javascript CSS Mobile Apps and Services
    • HOW/ Mobile apps Developer Z iOS Android Developer M iOS Android Web Developer B Android Windows Developer H iOS Android Bada
    • HOW/ Mobile apps Developer Z iOS Android Developer M iOS Android Web Developer B Android Windows Developer H iOS Android Bada We develop on various platforms, so Every developer are needed to work on two or more platforms.
    • HOW/ Mobile apps • We’ve focused on each app and content, not on making platform. • Release apps on various mobile platforms even on Windows Store and Smart TV. • Each developer has their own ‘Main role’ but also ‘Sub role’ to expand experiences. GOOD
    • HOW/ Mobile apps • We have own platform-independent data format and service models, but do not have own rich middleware or library. • Too many apps to be serviced. • Pressure of multi-platform experience. BAD
    • HOW/ Mobile apps • We have lots of codebase on each apps. • If we can refine them into loose-tied libraries, it’ll be good to prevent making a ‘Wheels’ again. • Document it and Share it. TO-BE
    • HOW/ Log Crawler & Analyzer
    • HOW/ Log Crawler & Analyzer • Traditional RDB + NoSQL solution • We uses MySQL InnoDB / MyISAM with MongoDB to store & analysis logs.
    • HOW/ Log Crawler & Analyzer 230logs/sec 20,000,000logs/day about 10GB/day
    • HOW/ Log Crawler & Analyzer over 3 billion logs till now.
    • Server Apache httpd mod_php Server Redis Apache httpd mod_php Predis Server Apache httpd mod_php PredisPredis
    • ReplicaSet Secondary Secondary Primary Redis
    • PV / UV DAU / MAU PU / ARPU NRU / RR ReplicaSet
    • See also: Redis, MongoDB and MySQL Crawl and Analysis log in Mobile Application Service Link : http://www.slideshare.net/lqez/redis-mongodb-mysql Video : http://youtu.be/NYtRr2URAkI
    • HOW/ Log Crawler & Analyzer GOOD • Fault tolerance log crawler. • Client stores logs in own local storage when it failed to send logs by any reason. (i.e. server problem, network unreachable) • Redis can hold logs while MongoDB have problems while upgrading or replicating.
    • HOW/ Log Crawler & Analyzer GOOD • MongoDB can hold various types of log without creating tables for each log type. • And good bye long ‘ALTER TABLE’. • MySQL is very familiar for us.
    • HOW/ Log Crawler & Analyzer BAD • Storing entire logs in only one collection is not a good idea. • Lock strangle us every each second. • Copying logs from MongoDB to MySQL needs logs of time.
    • HOW/ Log Crawler & Analyzer • Divide logs into small pieces in MongoDB. • Use aggregation framework added in 2.1. • Use shorter key name. • Add more memory and SSD. • There are (still) many global locks. Use more physical instances for each purpose. TO-BE
    • HOW/ Push Solution
    • HOW/ Push Solution • Support APNS, GCM and C2DM. • Collect ~200 tokens per every second. • Storing 20M+ tokens in MongoDB. • Adapt a similar architecture of the Log crawling system.
    • HOW/ Push Solution • Stores tokens with version of app, language, timezone and other meaningful device information without personal’s. • With reservation system, we can send messages to the targeted users. • We sent ~30,000,000 msgs per month. • via Urban Airship = $3,000 per month.
    • HOW/ Push Solution TO-BE • Current token database is isolated from other application database. • To make better user experience, combination of two databases is needed. • Timezone awareness.
    • HOW/ Web-based tools
    • HOW/ Web-based tools • Web - actually, web browser is a good platform to provide rich toolkits rapidly. • We use Python and PHP as backend and mainly use Django and Flask. • And also use jQuery, Bootstrap and Google Chart library for frontend. (as like as you)
    • See also: Creating a comic service Link : http://www.slideshare.net/lqez/ss-14820081 Video : http://youtu.be/-TEyIAqAB_E
    • See also: Do not use Django as like as SMARTSTUDY Link : http://www.slideshare.net/lqez/do-not-use-django- as-like-as-smartstudy
    • HOW/ Web-based tools TO-BE • Currently, there is no rule for Authentication and Authorization. (some uses just LDAP) • So,We’re building standard A-A library via OAuth2.
    • HOW/ Open Source
    • HOW/ Open Source • Open source is not only code or license. • It is a culture for hackers. • We’re using open source projects for our service widely. • We also recommend to make in-house library or middleware as open source.
    • HOW/ Open Source • Can’t enumerate all in here, • Linux, OpenLDAP, Redmine, Cacti, Nginx, MySQL, MongoDB, watch, ImageMagicK, ffmpeg, libx264, OpenSSH, awk, sed, find, Python, wget, Celery, PHP, tar, gzip, curl, zlib, iptables, Django, Flask,Werkzeug, vim, fabric, eventlet, Sentry, django-debug-toolbar, Bootstrap, jQuery, Postgres, Redis, uWSGI...
    • HOW/ Open Source • And open our sources, even they’re tiny. • npk : Neat package library • https://github.com/lqez/npk • django-project-layout • https://github.com/smartstudy/django-project-layout • FormEncode-Jinja2 • https://github.com/Kroisse/FormEncode-Jinja2
    • HOW/ Open Source • And we gave back to Open source by another way - Open a hands-on session. • Django In Action,Aug 2012 at SMARTSTUDY Seoul HQ. • 40 developers learned from the installation of Python to Django in 4H. • http://blog.naver.com/ez_/140166283139
    • HOW/ Open Source TO-BE • Open more codes to community. • Open more sessions, especially hands-on sessions. It’s very rare in Korea. • Be more sponsors for developer conferences. • Contribute to open source projects.
    • AND • Using AWS CloudFront with S3 • http://www.slideshare.net/lqez/using-cloud-front-with-s3-at-smartstudy • Post-mortem of Making a comic service • http://blog.naver.com/ez_/140157300770 (in Korean) • Using Sentry for Django applications • http://blog.naver.com/ez_/140177157684 (in Korean) • Test environment for mobile devices • http://www.slideshare.net/lqez/ss-20901287 (in Korean)
    • MOVING TOWARD
    • Good Product not great or elegance platforms.
    • Content rules platform, Time rules talent.
    • So,
    • Our most important value is
    • YOU, AND I
    • We, Human do make it works. not code or turing machines.
    • WHAT / WHY / HOW WE’RE ENGINEERING AT SMARTSTUDY Park Hyunwoo / @lqez 2013-05 AND WE’RE STILL WALKING ON !