Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Kotlin @ Coupang Backend 2017

1,195 views

Published on

쿠팡 카탈로그 부서에서 Kotlin 을 Backend 에 도입한 사례와 Kotlin 기본 문법과 Coroutines 에 대해 설명합니다.

Published in: Software
  • Girls for sex in your area are there: tinyurl.com/areahotsex
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Sex in your area is here: www.bit.ly/sexinarea
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Dating for everyone is here: www.bit.ly/2AJerkH
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Sex in your area for one night is there tinyurl.com/hotsexinarea Copy and paste link in your browser to visit a site)
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Girls for sex are waiting for you https://bit.ly/2TQ8UAY
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Kotlin @ Coupang Backend 2017

  1. 1. Kotlin Backend @ Coupang Spring Camp 2018 debop@coupang.com
  2. 2. Who is @debop • Since 1993 • Robotics, BPM Solution, Healthcare • C/C++, Object Pascal (Delphi), C#, Java, Scala, Kotlin • Use Kotlin since 2016.08 • Sr. Principle Software Engineer in Coupang (2017~)
  3. 3. Agenda • Kotlin - The Good, The Bad, The Ugly • Kotlin Coroutines • Kotlin Backend 도입 전략 / 사례 • Lesson & Learn • Resources
  4. 4. Kotlin The Good, The Bad and The Ugly
  5. 5. Kotlin - The Good (1/3) • Java Interoperability (@JvmXXXX) • Familiar Syntax (like Scala, C#, Groovy, Swift …) • Type Inference (like Scala, C#) • Smart Casts (`is`, `as`) • Data Class (Scala case class, POJO with @Data)
  6. 6. Kotlin - The Good (1/3) when (x) { is Int -> print(x + 1) is String -> print(x.length + 1) is IntArray -> print(x.sum()) !is BigDecimal -> print(x) 1L..10L -> print(x) } val x: String? = y as String (Unsafe) val x: String? = y as String? (Unsafe) val x: String? = y as? String (Safe) is / !is as / as?
  7. 7. Kotlin - The Good (1/3) public class User { private String name; private Integer age; public User(String name) { this(name, 0); } public User(String name, Integer age) { this.name = name; this.age = age; } public String getName() {return this.name;} public Integer getAge() {return this.age;} public void setName(String name) {this.name = name; } public void setAge(Integer age) {this.age = age; } public boolean equals(Object o) { /* 생략 */ } public int hashCode() { return Objects.hash(name, age); } protected boolean canEqual(Object other) {return other instanceof User;} public String toString() {return "User(name=" + this.getName() + ", age=" + this.getAge() + ")";} }
  8. 8. Kotlin - The Good (1/3) public class User { private String name; private Integer age; public User(String name) { this(name, 0); } public User(String name, Integer age) { this.name = name; this.age = age; } public String getName() {return this.name;} public Integer getAge() {return this.age;} public void setName(String name) {this.name = name; } public void setAge(Integer age) {this.age = age; } public boolean equals(Object o) { /* 생략 */ } public int hashCode() { return Objects.hash(name, age); } protected boolean canEqual(Object other) {return other instanceof User;} public String toString() {return "User(name=" + this.getName() + ", age=" + this.getAge() + ")";} } data class User(var name:String) { var age:Int = 0 }
  9. 9. Kotlin - The Good (2/3) • Default Arguments • String Templates (Scala - string interpolation) • Properties (C#, No Getter, Setter)
  10. 10. val tomAge = map["tom"] ?: 0 map["jane"] = 29 val query = """ | CREATE KEYSPACE IF NOT EXISTS $keyspaceName | WITH replication = { 'class': '$replicationStrategy', | 'replication_factor':$replicationFactor } """.trimMargin() val s = "abc" val str = "$s.length is ${s.length}" // prints "abc.length is 3" Kotlin - The Good (2/3) fun read(b:ByteArray, offset:Int = 0, length:Int = b.size) { TODO(”Not Implemented”) }
  11. 11. val tomAge = map["tom"] ?: 0 map["jane"] = 29 val query = """ | CREATE KEYSPACE IF NOT EXISTS $keyspaceName | WITH replication = { 'class': '$replicationStrategy', | 'replication_factor':$replicationFactor } """.trimMargin() val s = "abc" val str = "$s.length is ${s.length}" // prints "abc.length is 3" Kotlin - The Good (2/3) fun read(b:ByteArray, offset:Int = 0, length:Int = b.size) { TODO(”Not Implemented”) }
  12. 12. val tomAge = map["tom"] ?: 0 map["jane"] = 29 val query = """ | CREATE KEYSPACE IF NOT EXISTS $keyspaceName | WITH replication = { 'class': '$replicationStrategy', | 'replication_factor':$replicationFactor } """.trimMargin() val s = "abc" val str = "$s.length is ${s.length}" // prints "abc.length is 3" Kotlin - The Good (2/3) fun read(b:ByteArray, offset:Int = 0, length:Int = b.size) { TODO(”Not Implemented”) }
  13. 13. val tomAge = map["tom"] ?: 0 map["jane"] = 29 val query = """ | CREATE KEYSPACE IF NOT EXISTS $keyspaceName | WITH replication = { 'class': '$replicationStrategy', | 'replication_factor':$replicationFactor } """.trimMargin() val s = "abc" val str = "$s.length is ${s.length}" // prints "abc.length is 3" Kotlin - The Good (2/3) fun read(b:ByteArray, offset:Int = 0, length:Int = b.size) { TODO(”Not Implemented”) }
  14. 14. Kotlin - The Good (3/3) • Destructuring Declarations • Class, Property Delegates • Extension Functions (C#) • Null Safety (C#) • Better Lambdas than Java (same Scala) • Scala functional features (use arrow) • Coroutines (kotlinx-coroutines)
  15. 15. Kotlin - The Good (3/3) fun Int.millis(): Duration = Duration.ofMillis(this.toLong()) fun Int.seconds(): Duration = Duration.ofSeconds(this.toLong()) // 5.seconds() + 400.millis() —> 5400 msec if (text != null) text.length else -1 text?.length ?: -1 suspend fun Period.daySequence(): Sequence<Int> = buildSequence { var day = 0 val days = this@daySequence.days while (day < days) { yield(day++) } } }
  16. 16. Kotlin - The Good (3/3) if(command!=null) { if(command.getThreadPoolProperties()!=null) { if(command.getThreadPoolProperties().maxQueueSize()!=null) { val x = command.getThreadPoolProperties().maxQueueSize().get() } } }
  17. 17. val x = command?.getThreadPoolProperties()?.maxQueueSize()?.get() ?: 0 Kotlin - The Good (3/3) if(command!=null) { if(command.getThreadPoolProperties()!=null) { if(command.getThreadPoolProperties().maxQueueSize()!=null) { val x = command.getThreadPoolProperties().maxQueueSize().get() } } }
  18. 18. Compile time
  19. 19. Compile time Scala >>>>> Kotlin > Java
  20. 20. the bad
  21. 21. Kotlin - The Bad • No namespace (package variable, methods) • No static modifier (use companion object)
  22. 22. Kotlin - The Bad class OffHeapOutputStream(private var ohm: OffHeapMemory) : OutputStream() { companion object { private val bax = ByteArrayTool.Factory.create() @JvmField val EMPTY = OffHeapOutputStream(0) } } @file:JvmName("Localex") /** 지정한 Locale 이 null 이면 기본 Locale 을 반환합니다 */ fun Locale?.orDefault(): Locale = this ?: Locale.getDefault()
  23. 23. the ugly
  24. 24. Kotlin - The Ugly • SAM conversion • Unit returning lambdas (Use in java) • Final by default (안정성? 성능? A bit about picking defaults) • Open: 50%, Final : 50% registerCallback(() -> { /** do stuff */ return Unit.INSTANCE; })
  25. 25. Killer feature - Coroutines • Readable, clean code • Escape Callback hell • Imperative programming • Not exclusive, Cooperative • Lightweight thread (fiber, goroutines)
 • Note: Experimental !!!
  26. 26. @GetMapping("/composeasync2") DeferredResult<String> asyncCompose2() {     DeferredResult dr = new DeferredResult(); ListenableFuture<String> f1 = myService.async(); f1.addCallback(res1 -> { ListenableFuture<String> f2 = myService.async2(res1); f2.addCallback(res2 -> {     ListenableFuture<String> f3 = myService.async3(res2);      f3.addCallback(res3 -> {          dr.setResult(res3);         }, e -> {          dr.setErrorResult(e);         });     }, e -> {      dr.setErrorResult(e);     }); }, e -> { dr.setErrorResult(e); }); return dr; }
  27. 27. @GetMapping("/composeasync2") DeferredResult<String> asyncCompose2() {     DeferredResult dr = new DeferredResult(); ListenableFuture<String> f1 = myService.async(); f1.addCallback(res1 -> { ListenableFuture<String> f2 = myService.async2(res1); f2.addCallback(res2 -> {     ListenableFuture<String> f3 = myService.async3(res2);      f3.addCallback(res3 -> {          dr.setResult(res3);         }, e -> {          dr.setErrorResult(e);         });     }, e -> {      dr.setErrorResult(e);     }); }, e -> { dr.setErrorResult(e); }); return dr; } Callback Hell
  28. 28. lock.lockAsync(3000, TimeUnit.MILLISECONDS) .thenComposeAsync(rl -> updateProductItemMapping(productId, itemId)) .thenComposeAsync(updated -> adultInverseIndex.fastPutAsync(itemId, productId)) .thenComposeAsync(added -> addItemIdToList(productId, itemId)) .whenCompleteAsync((result, error) -> lock.forceUnlock()); CompletableFuture RxJava Observable .defer { Observable.just(timer.time()) } .flatMap { timerCtx -> external.doOnCompleted { timerCtx.stop() } } .subscribe { println(it) }
  29. 29. C# Way async Task<String> work() {     Thread.sleep(200);     return “done”; } async Task moreWork() {     Console.WriteLine(“Work started”);     var str = await work();     Console.WriteLine($“Work completed: {str}”); }
  30. 30. C# Way async Task<String> work() {     Thread.sleep(200);     return “done”; } async Task moreWork() {     Console.WriteLine(“Work started”);     var str = await work();     Console.WriteLine($“Work completed: {str}”); }
  31. 31. C# Way async Task<String> work() {     Thread.sleep(200);     return “done”; } async Task moreWork() {     Console.WriteLine(“Work started”);     var str = await work();     Console.WriteLine($“Work completed: {str}”); }
  32. 32. C# Way async Task<String> work() {     Thread.sleep(200);     return “done”; } async Task moreWork() {     Console.WriteLine(“Work started”);     var str = await work();     Console.WriteLine($“Work completed: {str}”); }
  33. 33. Kotlin Way suspend fun work(): String { delay(200L) return "done" } fun moreWork(): Deferred<Unit> = async { println("Work started") val str = work() println("Work completed. $str") } runBlocking { moreWork().await() }
  34. 34. Kotlin Way suspend fun work(): String { delay(200L) return "done" } fun moreWork(): Deferred<Unit> = async { println("Work started") val str = work() println("Work completed. $str") } runBlocking { moreWork().await() }
  35. 35. Kotlin Way suspend fun work(): String { delay(200L) return "done" } fun moreWork(): Deferred<Unit> = async { println("Work started") val str = work() println("Work completed. $str") } runBlocking { moreWork().await() }
  36. 36. Kotlin Way suspend fun work(): String { delay(200L) return "done" } fun moreWork(): Deferred<Unit> = async { println("Work started") val str = work() println("Work completed. $str") } runBlocking { moreWork().await() }
  37. 37. Go Way runtime.GOMAXPROCS(runtime.NumCPU()) wg:= new(sync.WaitGroup) for i:=0; i < 100000; i++ { wg.Add(1) go func() { defer wg.Done() time.Sleep(1000 * time.Millisecond) print(".") }() } wg.Wait()
  38. 38. Go Way runtime.GOMAXPROCS(runtime.NumCPU()) wg:= new(sync.WaitGroup) for i:=0; i < 100000; i++ { wg.Add(1) go func() { defer wg.Done() time.Sleep(1000 * time.Millisecond) print(".") }() } wg.Wait()
  39. 39. Go Way runtime.GOMAXPROCS(runtime.NumCPU()) wg:= new(sync.WaitGroup) for i:=0; i < 100000; i++ { wg.Add(1) go func() { defer wg.Done() time.Sleep(1000 * time.Millisecond) print(".") }() } wg.Wait()
  40. 40. Go Way runtime.GOMAXPROCS(runtime.NumCPU()) wg:= new(sync.WaitGroup) for i:=0; i < 100000; i++ { wg.Add(1) go func() { defer wg.Done() time.Sleep(1000 * time.Millisecond) print(".") }() } wg.Wait()
  41. 41. Quiz - How much time takes? runBlocking { val jobs = List(100_000) { launch(CommonPool) { delay(1000L) print(".") } } jobs.forEach { it.join() } }
  42. 42. Quiz - How much time takes? runBlocking { val jobs = List(100_000) { launch(CommonPool) { delay(1000L) print(".") } } jobs.forEach { it.join() } } 100,000 Seconds?
  43. 43. Quiz - How much time takes? runBlocking { val jobs = List(100_000) { launch(CommonPool) { delay(1000L) print(".") } } jobs.forEach { it.join() } } 100,000 Seconds? Or 1~3 seconds?
  44. 44. Quiz - How much time takes? runBlocking { val jobs = List(100_000) { launch(CommonPool) { delay(1000L) print(".") } } jobs.forEach { it.join() } } 100,000 Seconds? Or 1~3 seconds?
  45. 45. Kotlin 도입 전략
  46. 46. What is our problem? • 개발자 수준 편차가 크다 • 조직 수준은 최하위 개발자 수준의 의해 결정된다
  47. 47. 변화를 추구하려면? • 객관적 시각 유지 • History 및 현실 상황 대한 이해 (기술부채) • 현 조직의 개발 역량 평가 • 동기부여 - 필요성 설득보다는 자각할 수 있도록 자극 • 충분한 학습 시간 • 변화 경험 공유 • ASP -> C#, Java -> Scala
  48. 48. Environments 준비 • 개발 Tool 지원 - IntelliJ IDEA • Static code analysis - ktlint, detekt • Test code coverage - jacoco (sample) • Sonarcube - sonar-kotlin plugin
  49. 49. Kotlin Backend 도입 사례
  50. 50. Use cases • Common Library (2017. 6 ~) • Kotlin extension library • Components (2017.08 ~ 2017.12) • Aho-corasick, Korean Tokenizer, Kafka Client • Standalone Web Application (2017.10~~2017.12) • Audit Tool • Large scale system (2017.09~2018.01) • Catalog creation pipeline system
  51. 51. 1. Kotlin Extension library • Coupang Standard Framework 보조 • Kotlin Best practices 제공 • Object / IO / Utils / Cache / JDBC / Spring … • Kotlin 학습 예제 제공 (Test cases / Examples) • Based debop4k (personal project)
  52. 52. 2. Korean Tokenizer • 중복상품 Merge 위한 Tokenizer 필요 (명사 위주) • Twitter 에서 개발한 open-korean-text 를 Customizing • Scala vs Kotlin 성능 비교 • Kotlin version is 1.5 ~ 3X faster with Coroutines • 효과 • Full Indexing per Day 부하 감소 : 30% • Elastic Search 질의 부하 : 80%
  53. 53. “제테스 MSW-3928PR 남성 비치트렁크, 95(L), 블랙블랙계열” 은전한닢 Twitter 블랙 비치 비치트렁크 계열 제테스 3928PR 블랙블랙계열 PR MSW , ( )- 트렁크 3928 95 제테스 남성
  54. 54. “제테스 MSW-3928PR 남성 비치트렁크, 95(L), 블랙블랙계열” 은전한닢 Twitter 블랙 비치 비치트렁크 계열 제테스 3928PR 블랙블랙계열 PR MSW , ( )- 트렁크 3928 95 제테스 남성
  55. 55. “제테스 MSW-3928PR 남성 비치트렁크, 95(L), 블랙블랙계열” 은전한닢 Twitter 블랙 비치 비치트렁크 계열 제테스 3928PR 블랙블랙계열 PR MSW , ( ) L - 트렁크 3928 95 제테스 남성
  56. 56. Tokenizer Benchmark 은전한닢 Twitter RATIO RPS 73.2 429.5 5.87 X Avg Latency 626 ms 106 ms 5.91 X Avg CPU Load 90% 55% 35 % Down Twitter Tokenizer 는 한음절을 분석하지 못하는 단점이 있다
  57. 57. Scala Kotlin RATIO Tokenize 668.620 ms 197.632 ms 3.38 X Phrase extractor 644.902 ms 212.500 ms 3.13 X 구문 : “동해물과 백두산이 마르고 닳도록“ Benchmark by jmh Linux Mint 18.2 Intel I7, 32GB, SSD
  58. 58. Why Kotlin is faster? • Scala 의 loop는 느리다 - 아주 느릴 수 있다 • eclipse-collections 사용 • 메모리 절약 • Primitive type collection 지원 • Kotlin Coroutines • 비동기 / Non-Blocking
  59. 59. 3. Kafka Client - Wilson • 동기 • 안정된 Kafka 사용을 위해 Offset 관리가 필수 • 각 팀별 중복된 Client 구현 및 삽질 • 효과 • Message 중복 / 유실 없음 (Latest strategy) • Retry / Circuit breaker 지원 • Metrics 를 활용한 Ack 지원 • 전사 확대 적용 중
  60. 60. Wilson message flows Producer ConsumerKafka Retry Circuit Breaker Metrics Retry Circuit Breaker Metrics Dead letters Sending box Redis MySQL Couchbase Dead letters Received box Last sent timestamp Kafka Offset Manager Message Managements • Metrics • Recovery / Retry • Deadletter handling
  61. 61. Wilson Dashboard
  62. 62. 4. Audit Tool • 상품 정보 Audit System • developers : 1 senior, 2 junior developer • Software stack • React • Spring Boot 1.5 on Vitamin Framework • jOOQ (향후 requery로 변환 예정) • Pilot 로 시작, 개발자들의 노력으로 정식 시스템으로 승격
  63. 63. 4. Audit Tool Kotlin Coroutines + Spring MVC
  64. 64. 5. Creation Pipeline System • 상품 정보 생성 프로세스 관리 시스템 • Features • Workflow (Heavy use Kafka) • Asynchronous / Non-blocking System • Need High throughput
  65. 65. Seller Retail Kafka Deduplication Refining Processing Creation Logging Message Dispatcher Creation Pipeline flow
  66. 66. 5. Creation Pipeline System Spring Boot 1.x Kafka 0.10.x Kotlin 1.2.x on JVM 8 Couchbase 4.x / Aurora Zookeeper 3.x
  67. 67. %
  68. 68. %
  69. 69. Lesson & Learns
  70. 70. 반성해야 하는 점 • 기본기 학습 - 닥치고 코딩 (X) • Java와 차이점 및 Kotlin best practices 검토 필요 • 첫 술에 배부를 수 없다 • 실망하지 말라, Refactoring 은 필수다 • Coroutines 에 대한 학습 및 테스트 • 어차피 비동기는 어렵다. • Coroutines는 하나의 방법일 뿐이다
  71. 71. 안정적 도입을 위한 Tips • 충분한 학습 기회 & 실습 • Code Quality 중요성 인식 • Upsource 전사 활용 중 • 강력한 동기 부여 • Tech Leader의 지속적인 Leading & Coach • 성공 사례 만들기 (작은 것부터)
  72. 72. 효과 • Safety Code • Readable Code • 성능 향상 • Latency 감소, High Throughput 달성 • Asynchronous/Non-Blocking 적용 용이 • 유지보수성 향상 (장애 감소)
  73. 73. 언어별 Catalog Tribe 개발자 비율 (2017) 5% Kotlin 10% Scala 15% Java 70% Java Scala Kotlin Python 언어별 Catalog Tribe 개발자 비율 (2018 예상) 7% Kotlin 25% Scala 13% Java 55% Java Scala Kotlin Python
  74. 74. Aurora Cassandra ElasticSearch ArangoDB JVM 8 Spring DataRequery JPA/Hibernate Virtualization kubernetes Docker Spring Framework 5.xKodein Kotlin Coroutines with Reactor / RxJava Services Kafka Redis Spring Boot 2.x Webflux with Netty Common Backend Stack (2018)
  75. 75. Boo1vsBoo2Performance
  76. 76. Spring MVC + Cassandra Spring WebFlux + Cassandra Reactive 출처: Reactive Java Performance Comparison
  77. 77. Resources • Kotlin Resources • kotlinx.coroutines • spring-kotlin-coroutine • 20 Excellent Resources for learning Kotlin • Books • Kotlin in action • Try Kotlin
  78. 78. Q&A
  79. 79. Thank you!

×