Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Kotlin @ Coupang Backed - JetBrains Day seoul 2018

1,270 views

Published on

Coupang 에서 Kotlin 을 적용하여 개발한 사례

Published in: Software
  • Be the first to comment

Kotlin @ Coupang Backed - JetBrains Day seoul 2018

  1. 1. Kotlin Backend @ Coupang debop@coupang.com
  2. 2. Who is @debop • Since 1993 • Robotics, BPM Solution, Healthcare • C/C++, Object Pascal, C#, Java, Scala, Kotlin • Use Kotlin since 2016.08 • Sr. Principle Software Engineer in Coupang (2017~) • 남의 코드 뒤집기 / 빨간펜 선생 • Common Library 제작 • Architecture 참견
  3. 3. Agenda • Kotlin 도입 동기 • Kotlin 도입 사례 • Kotlinx (Common Library) • 공용 Library 개발 • spring-data-requery • Lesson & Learn
  4. 4. Kotlin 도입 동기
  5. 5. What is our problem? • 개발자 수준 편차가 크다 • 조직 수준은 최하위 개발자 수준의 의해 결정된다
  6. 6. Our status • Lack of Safety code • Not Readable code • Lack of Testing (Unit & Integration) • Heavy oriented to JPA • Lack of experience to Async/Non-Blocking Programming
  7. 7. Our environments • 1일 데이터 처리량 : 천만건(2016) -> 10억건(2018) • MySQL (Aurora) : Replication delay120 sec (max) • Backend Stack in 2016 • MySQL, RabbitMQ, Hive • Spring 3, JPA
  8. 8. Our Challenges • Akka Cluster with Lagom Framework (CQRS) • Move to NoSQL (Cassandra, HBase) • Migrate RabbitMQ to Kafka • Adopting Scala (Pilot) • Akka, Spark …
  9. 9. Failure • Adopting Scala • Learning Curve (Almost Junior) • Lack of Tech Leaders • Try to develop too complex system • Adopting Kafka • Simultaneous development (duplicate same error) • Offset managements
  10. 10. To pursue change • 객관적 시각 유지 • History 및 현실 상황 대한 이해 (기술부채) • 현 조직의 개발 역량 객관적 평가 • 동기부여 - 필요성 설득보다는 자각할 수 있도록 자극 • 충분한 학습 시간 • 변화 경험 공유 • ASP -> C#, Java -> Scala
  11. 11. Prepare • Code Quality 중요성 자각 -> Code Review • Enterprise Application Commons • Patterns 교육, Library 제공 • Enterprise Architecture 학습 • Asynchronous Programming 학습 • Kotlin Language 학습 • Kotlin is language for Android ?
  12. 12. Adopting Kotlin • Safety Code / Readable Code • Simple grammar (Easy to learn) • Interoperability between Java and Kotlin • Extension methods • Asynchronous Programming by Coroutines
  13. 13. Environments 준비 • 개발 Tool 지원 - IntelliJ IDEA • Static code analysis - ktlint, detekt • Test code coverage - jacoco (sample) • Sonarcube - sonar-kotlin plugin
  14. 14. Kotlin Backend 도입 사례
  15. 15. Use cases 2017 • Common Library (2017. 6 ~) • Kotlin extension library • Components (2017.08 ~ 2017.12) • Aho-corasick, Korean Tokenizer, Kafka Client • Standalone Web Application (2017.10~~2017.12) • Audit Tool • Large scale system (2017.09~2018.01) • Catalog creation pipeline system
  16. 16. 1. Kotlin Extension library • Coupang Standard Framework 대체 • Kotlin Best practices 제공 • Object / IO / Utils / Cache / JDBC / Spring … • Kotlin 학습 예제 제공 (Test cases / Examples) • Based debop4k (personal project)
  17. 17. Kotlinx-Units • 도량형(度量衡)(Standard Unit) • Represent Meter Units • Weight, Length, Area, Volume, Storage Volume • Yard-Pound Units • Pound, Yard/Miles, Volume, Fluid Volume … • Others • Temperature, Electlic powers, Frequency, Light, Radiation, Pressure …
  18. 18. Our usages enum class Unit(val description: String, val unitConversion: BigDecimal) { NONE("없음", BigDecimal.ZERO), // Length MM("mm", 0.1.toBigDecimal()), CM("cm", BigDecimal.ONE), M("m", 100.toBigDecimal()), KM("km", 100_000.toBigDecimal()),
  19. 19. Our usages // Length (Inch) IN("in", BigDecimal.ONE), FT("ft", 12.toBigDecimal()), YD("yd", 36.toBigDecimal()), // Weight (gram) MG("mg", 0.001.toBigDecimal()), G("g", BigDecimal.ONE), KG("kg", 1000.toBigDecimal()), T("t", 1_000_000.toBigDecimal()), OZ("oz", 28.349523.toBigDecimal()), LB("lb", 453.59237.toBigDecimal()), •Define Units by Enum •Not support conversions •Not support various units •Not support operations •Not readable code
  20. 20. Features for units • Readable code like this • 5.kg() + 400.gram() => 5.4.kg() • 100.cm() + 2.m() => 2.1.cm() • Convert various units • 5.kg().inGram() -> 5000.gram • 2.5.Kb().inBytes() -> 2500.bytes() • Convert SI Units <-> US Units
  21. 21. Use Kotlin extension methods fun Double.millimeter(): Length = lengthOf(this, MILLIMETER) fun Double.centimeter(): Length = lengthOf(this, CENTIMETER) fun Double.meter(): Length = lengthOf(this) fun Double.kilometer(): Length = lengthOf(this, KILOMETER)
  22. 22. 1. Define Unit factors enum class LengthUnit(val unitName: String, val factor: Double) { MILLIMETER("mm", 1.0), CENTIMETER("cm", 10.0), METER("m", 1e3), KILOMETER("km", 1e6); }
  23. 23. Define Unit class data class Length(val value: Double = 0.0): Comparable<Length>, Serializable { operator fun plus(other: Length): Length = Length(value + other.value) operator fun minus(other: Length): Length = Length(value - other.value) operator fun times(scalar: Int): Length = Length(value * scalar) operator fun times(scalar: Long): Length = Length(value * scalar) operator fun times(scalar: Double): Length = Length(value * scalar) operator fun times(other: Length): Area = Area(value * other.value) operator fun div(scalar: Int): Length = Length(value / scalar) operator fun div(scalar: Long): Length = Length(value / scalar) operator fun div(scalar: Double): Length = Length(value / scalar) operator fun unaryMinus(): Length = Length(-value) Override Operators
  24. 24. 2. Korean Tokenizer • 중복상품 Merge 위한 Tokenizer 필요 (명사 위주) • Twitter 에서 개발한 open-korean-text 를 Customizing • Scala vs Kotlin 성능 비교 • Kotlin version is 1.5 ~ 3X faster with Coroutines • 효과 • Full Indexing per Day 부하 감소 : 30% • Elastic Search 질의 부하 : 80%
  25. 25. “제테스 MSW-3928PR 남성 비치트렁크, 95(L), 블랙블랙계열” 은전한닢 Twitter 블랙 비치 비치트렁크 계열 제테스 3928PR 블랙블랙계열 PR MSW , ( )- 트렁크 3928 95 제테스 남성
  26. 26. “제테스 MSW-3928PR 남성 비치트렁크, 95(L), 블랙블랙계열” 은전한닢 Twitter 블랙 비치 비치트렁크 계열 제테스 3928PR 블랙블랙계열 PR MSW , ( )- 트렁크 3928 95 제테스 남성
  27. 27. “제테스 MSW-3928PR 남성 비치트렁크, 95(L), 블랙블랙계열” 은전한닢 Twitter 블랙 비치 비치트렁크 계열 제테스 3928PR 블랙블랙계열 PR MSW , ( ) L - 트렁크 3928 95 제테스 남성
  28. 28. Tokenizer Benchmark 은전한닢 Twitter RATIO RPS 73.2 429.5 5.87 X Avg Latency 626 ms 106 ms 5.91 X Avg CPU Load 90% 55% 35 % Down Twitter Tokenizer 는 한음절을 분석하지 못하는 단점이 있다
  29. 29. Scala Kotlin RATIO Tokenize 668.620 ms 197.632 ms 3.38 X Phrase extractor 644.902 ms 212.500 ms 3.13 X 구문 : “동해물과 백두산이 마르고 닳도록“ Benchmark by jmh Linux Mint 18.2 Intel I7, 32GB, SSD
  30. 30. Why Kotlin is faster? • Scala 의 loop는 느리다 - 아주 느릴 수 있다 • eclipse-collections 사용 • 메모리 절약 • Primitive type collection 지원 • Kotlin Coroutines • 비동기 / Non-Blocking
  31. 31. 3. Kafka Client - Wilson • 동기 • 안정된 Kafka 사용을 위해 Offset 관리가 필수 • 각 팀별 중복된 Client 구현 및 삽질 • 효과 • Message 중복 / 유실 없음 (Latest strategy) • Retry / Circuit breaker 지원 • Metrics 를 활용한 Ack 지원 • 전사 확대 적용 중
  32. 32. Wilson message flows Producer ConsumerKafka Retry Circuit Breaker Metrics Retry Circuit Breaker Metrics Dead letters Sending box Redis MySQL Couchbase Dead letters Received box Last sent timestamp Kafka Offset Manager Message Managements • Metrics • Recovery / Retry • Deadletter handling
  33. 33. Wilson Dashboard
  34. 34. 4. Audit Tool • 상품 정보 Audit System • developers : 1 senior, 2 junior developer • Software stack • React • Spring Boot 1.5 on Vitamin Framework • jOOQ (향후 requery로 변환 예정) • Pilot 로 시작, 개발자들의 노력으로 정식 시스템으로 승격
  35. 35. 4. Audit Tool Kotlin Coroutines + Spring MVC
  36. 36. 5. Creation Pipeline System • 상품 정보 생성 프로세스 관리 시스템 • Features • Workflow (Heavy use Kafka) • Asynchronous / Non-blocking System • Need High throughput
  37. 37. Seller Retail Kafka Deduplication Refining Processing Creation Logging Message Dispatcher Creation Pipeline flow Kafka
  38. 38. 5. Creation Pipeline System Spring Boot 1.x Kafka 0.10.x Kotlin 1.2.x on JVM 8 Couchbase 4.x Zookeeper 3.x
  39. 39. %
  40. 40. %
  41. 41. Use cases 2018 • Tagging system (2018.1 ~ 2018.3) • Spring Data Requery (alternative to JPA) (2018.4 ~) • Rule engine (Thanos rule engine) (2018.6 ~) • Wakanda system (Reactive + Coroutines) (2018.9 ~) • Creation Pipeline Upgrade (2018.10 ~)
  42. 42. Aurora Cassandra ElasticSearch HBase JVM 8 Spring DataRequery JPA/Hibernate Virtualization kubernetes Docker Spring Framework 5.xKodein Kotlin Coroutines with Reactor / RxJava Services Kafka Redis Spring Boot 2.x Webflux with Netty / gRPC Common Backend Stack (2018)
  43. 43. Boo1vsBoo2Performance
  44. 44. Spring MVC + Cassandra Spring WebFlux + Cassandra Reactive 출처: Reactive Java Performance Comparison
  45. 45. 6. Tagging System • 상품 정보에서 원하는 특성을 추출하는 시스템 • Matching 과 Refinement 에서 따로 관리 • 범용화 필요 (Configuration, Plugin) • 기존 Java & Scala 혼용 • 장애 대응에 문제 • High Throughput
  46. 46. Tagging for Matching 스톤 9.0cm 여성하이힐 MZ-135 블랙 245 Brand Gender Shoes Type Color Size
  47. 47. 7. spring-data-requery (2018) • RequeryOperations • Wrap EntityDataStore • RequeryTransactionManager for TransactionManager • Support Spring @Transactional • Better performance than spring-data-jpa • when exists, paging, not load all entities https://www.slideshare.net/debop/alternatives-of-jpahibernate https://www.slideshare.net/debop/requery-overview https://www.slideshare.net/debop/spring-data-requery
  48. 48. spring-data-requery • Repository built in SQL • ByPropertyName Auto generation methods • @Query for Native SQL Query • Query By Example • Not Supported • Association Path (not specified join method) • Named parameter in @Query (just use `?`)
  49. 49. Define Entity - Java @Getter @Entity(name = "BasicUser", copyable = true) @Table(name = "basic_user") public abstract class AbstractBasicUser extends AuditableLongEntity { @Key @Generated protected Long id; protected String name; protected String email; protected LocalDate birthday; protected Integer age; @ForeignKey @OneToOne protected AbstractBasicLocation address; @ManyToMany(mappedBy = "members") protected Set<AbstractBasicGroup> groups; @Column(unique = true) protected UUID uuid;
  50. 50. Use @Query in Repository interface DeclaredQueryRepository extends RequeryRepository<BasicUser, Long> { @Query("select * from basic_user u where u.email = ?") BasicUser findByAnnotatedQuery(String email); @Query("select * from basic_user u where u.email like ?") List<BasicUser> findAllByEmailMatches(String email); @Query("select * from basic_user u limit ?") List<BasicUser> findWithLimits(int limit); @Query("select * from basic_user u where u.name=? and u.email=? limit 1") BasicUser findAllBy(String name, String email); @Query("select u.id, u.name from basic_user u where u.email=?") List<Tuple> findAllIds(String email); @Query("select * from basic_user u where u.birthday = ?") List<BasicUser> findByBirthday(LocalDate birthday); }
  51. 51. Query By Example BasicUser user = RandomData.randomUser(); user.setName("example"); requeryTemplate.insert(user); BasicUser exampleUser = new BasicUser(); exampleUser.setName("EXA"); ExampleMatcher matcher = matching() .withMatcher("name", startsWith().ignoreCase()) .withIgnoreNullValues(); Example<BasicUser> example = Example.of(exampleUser, matcher); Return<? extends Result<BasicUser>> query = buildQueryByExample(example); BasicUser foundUser = query.get().firstOrNull(); assertThat(foundUser).isNotNull().isEqualTo(user);
  52. 52. Query by Property List<User> findByFirstnameOrLastname(String firstname, String lastname); List<User> findByLastnameLikeOrderByFirstnameDesc(String lastname); List<User> findByLastnameNotLike(String lastname); List<User> findByLastnameNot(String lastname); List<User> findByManagerLastname(String name); List<User> findByColleaguesLastname(String lastname); List<User> findByLastnameNotNull(); @Query("select u.lastname from SD_User u group by u.lastname") Page<String> findByLastnameGrouped(Pageable pageable); long countByLastname(String lastname); int countUsersByFirstname(String firstname); boolean existsByLastname(String lastname); Note: Association Path is not supported Note: Association Path is not supported
  53. 53. Future works • Coroutines for Async JDBC Operations (working) • Support Named parameter • Support `@Param` in spring data • Requery for SQL on Hadoop • Apache Phoenix (HBase) (Done) • Apache Hive, Apache Drill …
  54. 54. 8. Rule Engine (2018) • Embedded • Support Language • MVEL (almost like Java spec) • KotlinScript • Javascript • Support Multi Rule combination • Rule Definition DSL (Kotlin) • Rule Editor in WEB (in construction)
  55. 55. Rule Engine A rule engine can be viewed as a sophisticated interpreter for if/then statements, where the statements themselves are known as rules.
  56. 56. Rule Definition val rule = rule { name = "myRule" description = "myDescription" priority = 3 condition { condition1.evaluate(it) } action { action1.execute(it) } action { action2.execute(it) } } val rule = RuleBuilder() .name("myRule") .description("myDescription") .priority(3) .whenever(condition) .then(action1) .then(action2) .build() @Rule(name = "weather rule", description = "if it rains then take an umbrella") class WeatherRule { @Condition fun itRains(@Fact("rain") rain: Boolean): Boolean = rain @Action fun takeAnUmbrella() { println("It rains, take an umbrella") } }
  57. 57. 9. Wakanda • Catalog 정보 관리 시스템의 핵심 • Read, Write Side 분리 제공 • 전사 통합에 의해 Write side만 존치 • Fully Reactive / Non-Blocking
  58. 58. Editable Catalog Process
  59. 59. Cassandra ElasticSearch JVM 8 Cassandra Vanilla Driver Virtualization kubernetes Docker Spring Framework 5.x Kotlin Coroutines with Reactor Services Kafka Redis Spring Boot 2.x Webflux Wakanda Software stack
  60. 60. Lesson & Learns
  61. 61. 반성해야 하는 점 • 기본기 학습 - 닥치고 코딩 (X) • Java와 차이점 및 Kotlin best practices 검토 필요 • 첫 술에 배부를 수 없다 • 실망하지 말라, Refactoring 은 필수다 • Coroutines 에 대한 학습 및 테스트 • 어차피 비동기는 어렵다. • Coroutines는 하나의 방법일 뿐이다
  62. 62. 안정적 도입을 위한 Tips • 충분한 학습 기회 & 실습 • Code Quality 중요성 인식 • Upsource 전사 활용 중 (문화가 중요) • 강력한 동기 부여 • Tech Leader의 지속적인 Leading & Coach • 성공 사례 만들기 (작은 것부터)
  63. 63. 효과 • Safety Code • Readable Code • 성능 향상 • Latency 감소, High Throughput 달성 • Asynchronous/Non-Blocking 적용 용이 • 유지보수성 향상 (장애 감소)
  64. 64. 언어별 Catalog Tribe 개발자 비율 (2017) 5% Kotlin 10% Scala 15% Java 70% Java Scala Kotlin Python 언어별 Catalog Tribe 개발자 비율 (2018) 17% Kotlin 40% Scala 13% Java 30% Java Scala Kotlin Python
  65. 65. Resources • Kotlin Resources • kotlinx.coroutines • 20 Excellent Resources for learning Kotlin • Books • Kotlin in action, Joy of Kotlin • Try Kotlin
  66. 66. Q&A
  67. 67. Thank you!

×