@DebskiChris
#AtmosphereConf
Krzysztof Dębski
Allegro Group
Let’s build a solid base for a scale
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
Agenda
The Allegro Situation
How to build a new World...
Allegro The New World Improvement WIIFY
#AtmosphereConf @DebskiChris
The Allegro Situation
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
Allegro
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
Allegro
6 million LOC
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
400 developers
Allegro
Allegro The New World Improvement WIIFY
#AtmosphereConf @DebskiChris
A New Hope
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
A New Hope
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
A New Hope
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
A New Hope?
Service
Oriented
Ambiguity
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
Domain Driven Design
Allegro The New World Improvement WIIFY
#AtmosphereConf @DebskiChris
(Micro)Services
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
(Micro)Services
Business needs
Offer
User
Transaction
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
(Micro)Services
Offer
User
Transaction
Independent
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
(Micro)Services
API
Offer
User
Transaction
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
(Micro)Services
Polyglot
Cassandra
MongoDB
Oracle
Offe...
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
(Micro)Services
Smart Endpoints
Cassandra
MongoDB
Ora...
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
The first approach
Service
Auto deployable
Monitored
A...
Allegro The New World Improvement WIIFY
#AtmosphereConf @DebskiChris
How to build a new World?
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
The first project
Gradle
Spring
External Jetty server
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
Let’s REST
JAX-RS / JSR Compliant
@Path("/users")
@Co...
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
Let’s expose our methods
Swagger
@Path("/users")
@Api...
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
Let’s expose our methods
Swagger
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
And make them discoverable
Offer
User
Discovery
ZooKee...
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
And make them discoverable
Offer
User
Discovery
ZooKee...
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
And make them discoverable
Offer
User
Discovery
ZooKee...
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
And make them discoverable
Offer
User
Discovery
ZooKee...
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
Multiple API versions
Header support
public class Use...
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
Multiple API versions
Content negotiation
@Path("/use...
Allegro The New World Improvement WIIFY
#AtmosphereConf @DebskiChris
How to configure it?
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
How to configure it?
Zookeper
Environment properties
C...
Allegro The New World Improvement WIIFY
#AtmosphereConf @DebskiChris
And it became slow
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
Lower startup time
External Jetty
Embedded Jetty
Embe...
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
Easier deployment
Cargo deployment
Built to zip packa...
Allegro The New World Improvement WIIFY
#AtmosphereConf @DebskiChris
Tests
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
Do you test your tests?
Pitest
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
Integration tests
Test without mocks
Run from IDE
pub...
Allegro The New World Improvement WIIFY
#AtmosphereConf @DebskiChris
Monitoring
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
Let’s see what happens
LogStash
Kibana
NxLog
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
Let’s see what happens
Graphite
# Metrics
metrics.rep...
Allegro The New World Improvement WIIFY
#AtmosphereConf @DebskiChris
Monolith Alert
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
Monolith Alert
Multi module project support
Allegro The New World Improvement WIIFY
#AtmosphereConf @DebskiChris
The way to improve
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
Authentication
OAuth2
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
Patch Support
Standardized by IETF
https://tools.ietf...
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
Simplify annotations
Swagger?
@Path("/users")
@Api(va...
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
Deployment
Docker support
Allegro The New World Improvement WIIFY
#AtmosphereConf @DebskiChris
What’s in it for you?
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
Use the latest tools
GradleW issue in Continuous Inte...
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
Change is the only constant
HTTP Server
Service API p...
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
Focus on right metrics
import org.junit.Test;
import ...
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
Tools also lie
Metrics tend to lie
– Default PHP metr...
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
Tools also lie
How many violations are there?
4
At le...
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
Learn to REST
REST is not so obvious
/api/bi/XYZ123/1...
#AtmosphereConf @DebskiChris
Allegro The New World Improvement WIIFY
Community
Involve all developers in building the
Boot...
Allegro The New World Improvement WIIFY
#AtmosphereConf @DebskiChris
Q & A
Upcoming SlideShare
Loading in …5
×

Atmosphere 2014: Let's build a solid base for a scale. - Krzysztof Debski

394 views

Published on

Creating fast, reliable and easy to develop system is not an easiest task.
If you add thousands of companies using your system as a core business platform on a daily basis, it becomes challenging.
Add a flavour of a scalability and millions of users and it will be exciting.
How to prepare a system that might be affected by scalability issues everywhere? Even in a simplest component.
The answer is to dive deep into the world of services.
We decided to prepare a strong base on top of which we can build our system. Our choice was to start with the Java stack.
We've gathered all cool stuff from open source project to create bootstrap to let our developers create scalable services really fast.
In a talk I would like to focus with you about all the choices we made, we will discuss about all the ecosystem we built over services.
It is not only about the code we write, but all the tools that make our life easier.
We will look at frameworks to provide REST services, see how to make them auto discoverable.
We will also discuss tools to let us monitor and manage all the instances, etc.
If you want to create or upgrade your service to scale it is the best place to start.

Krzysztof Debski - Technical enthusiast, software engineer and product owner of Service Bootstrap developed in Allegro .
Gained software experience in the biggest polish web companies – Onet.pl and Allegro.pl.
He has a lot of experience in designing and building modern, scalable web systems.
He is open to all new technologies an programming languages and he has no fear to work on legacy systems and rewrite it in modern way.

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
394
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Atmosphere 2014: Let's build a solid base for a scale. - Krzysztof Debski

  1. 1. @DebskiChris #AtmosphereConf Krzysztof Dębski Allegro Group Let’s build a solid base for a scale
  2. 2. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY Agenda The Allegro Situation How to build a new World? The way to improve What’s in it for you?
  3. 3. Allegro The New World Improvement WIIFY #AtmosphereConf @DebskiChris The Allegro Situation
  4. 4. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY Allegro
  5. 5. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY Allegro 6 million LOC
  6. 6. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY 400 developers Allegro
  7. 7. Allegro The New World Improvement WIIFY #AtmosphereConf @DebskiChris A New Hope
  8. 8. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY A New Hope
  9. 9. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY A New Hope
  10. 10. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY A New Hope? Service Oriented Ambiguity
  11. 11. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY Domain Driven Design
  12. 12. Allegro The New World Improvement WIIFY #AtmosphereConf @DebskiChris (Micro)Services
  13. 13. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY (Micro)Services Business needs Offer User Transaction
  14. 14. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY (Micro)Services Offer User Transaction Independent
  15. 15. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY (Micro)Services API Offer User Transaction
  16. 16. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY (Micro)Services Polyglot Cassandra MongoDB Oracle Offer User Transaction
  17. 17. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY (Micro)Services Smart Endpoints Cassandra MongoDB Oracle Offer User Transaction
  18. 18. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY The first approach Service Auto deployable Monitored Auto scalable Auto healable Auto discoverable
  19. 19. Allegro The New World Improvement WIIFY #AtmosphereConf @DebskiChris How to build a new World?
  20. 20. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY The first project Gradle Spring External Jetty server
  21. 21. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY Let’s REST JAX-RS / JSR Compliant @Path("/users") @Consumes(CONTENT_TYPE_JSON) @Produces(CONTENT_TYPE_JSON) public class UsersEndpoint { // [...] @GET public UserCollectionResponse findAllUsers() { //[...] } }
  22. 22. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY Let’s expose our methods Swagger @Path("/users") @Api(value = "/users") @Consumes(CONTENT_TYPE_JSON) @Produces(CONTENT_TYPE_JSON) public class UsersEndpoint { // [...] @ApiOperation(value=“Get all users”, response=UserCollectionResponse.class) @GET public UserCollectionResponse findAllUsers() { //[...] } }
  23. 23. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY Let’s expose our methods Swagger
  24. 24. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY And make them discoverable Offer User Discovery ZooKeeper   Register  
  25. 25. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY And make them discoverable Offer User Discovery ZooKeeper   Get  User  
  26. 26. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY And make them discoverable Offer User Discovery ZooKeeper   Get  User  
  27. 27. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY And make them discoverable Offer User Discovery ZooKeeper   Get  User   Get  offer  for  user  
  28. 28. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY Multiple API versions Header support public class UserMediaType { public static final String V1_JSON = "application/vnd.allegro.user.v1+json” public static final String V2_JSON = "application/vnd.allegro.user.v2+json” } curl --dump-header - -H ”Accept: application/vnd.allegro.user.v2+json" -X GET http://localhost:8080/users
  29. 29. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY Multiple API versions Content negotiation @Path("/users") @Consumes(AllegroUserMediaType.V1_JSON) @Produces(AllegroUserMediaType.V1_JSON) public class UsersEndpoint { // [...] @GET @Produces(AllegroUserMediaType.V2_JSON) public UserCollectionResponse findAllUsers() { //[...] } }
  30. 30. Allegro The New World Improvement WIIFY #AtmosphereConf @DebskiChris How to configure it?
  31. 31. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY How to configure it? Zookeper Environment properties Command-line options
  32. 32. Allegro The New World Improvement WIIFY #AtmosphereConf @DebskiChris And it became slow
  33. 33. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY Lower startup time External Jetty Embedded Jetty Embedded UnderTow
  34. 34. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY Easier deployment Cargo deployment Built to zip package On immutable images
  35. 35. Allegro The New World Improvement WIIFY #AtmosphereConf @DebskiChris Tests
  36. 36. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY Do you test your tests? Pitest
  37. 37. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY Integration tests Test without mocks Run from IDE public class UsersIntegrationTest { private static Map<String, String> overrideConfiguration = Maps.newHashMap(); static { overrideConfiguration.put("property.name", "This was overwritten value"); } @ClassRule public static final RestServiceStarted DEPLOYED_SERVICE = new RestServiceStarted(overrideConfiguration); private WebTarget getUsersResourceWebTarget() { return DEPLOYED_SERVICE.getWebTarget().path("users"); } }
  38. 38. Allegro The New World Improvement WIIFY #AtmosphereConf @DebskiChris Monitoring
  39. 39. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY Let’s see what happens LogStash Kibana NxLog
  40. 40. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY Let’s see what happens Graphite # Metrics metrics.reporters.graphite.enabled=true metrics.reporters.graphite.host=graphite.service metrics.reporters.graphite.port=2003 metrics.reporters.graphite.prefix=stats.Prod.service metrics.reporters.interval=30
  41. 41. Allegro The New World Improvement WIIFY #AtmosphereConf @DebskiChris Monolith Alert
  42. 42. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY Monolith Alert Multi module project support
  43. 43. Allegro The New World Improvement WIIFY #AtmosphereConf @DebskiChris The way to improve
  44. 44. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY Authentication OAuth2
  45. 45. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY Patch Support Standardized by IETF https://tools.ietf.org/html/rfc6902 PATCH /user/1 HTTP/1.1 Host: user.service.local Content-Length: 312 Content-Type: application/json-patch+json [ {”op”: ”test”, ”path”: ”/firstname”, ”value”: ”Jane”}, {”op”: ”add”, ”path”: ”/maidenname”, ”value”: ”Smith”}, {”op”: ”replace”, ”path”: ”/lastname”, ”value”: ”Doe”}, {”op”: ”remove", ”path”: ”/meetings”}, {”op”: ”move”, ”from”: ”/balance”, ”path”: ”sharedbalance”}, {”op”: ”copy”, ”from”: ”/a”, ”path”: ”/b”} ]
  46. 46. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY Simplify annotations Swagger? @Path("/users") @Api(value = "/users") @Consumes(CONTENT_TYPE_JSON) @Produces(CONTENT_TYPE_JSON) public class UsersEndpoint { // [...] @ApiOperation(value=“Get all users”, response=UserCollectionResponse.class) @GET public UserCollectionResponse findAllUsers() { //[...] } }
  47. 47. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY Deployment Docker support
  48. 48. Allegro The New World Improvement WIIFY #AtmosphereConf @DebskiChris What’s in it for you?
  49. 49. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY Use the latest tools GradleW issue in Continuous Integration Environment and parent POM issues. Don’t do DDOS yourself and your partners’ sites.
  50. 50. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY Change is the only constant HTTP Server Service API provider Dependency injection engine Deployment tools
  51. 51. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY Focus on right metrics import org.junit.Test; import static net.trajano... public class MediaTypesTest { @Test public void mediaTypesShouldBeValidUtilityClasses() throws Throwable { assertUtilityClassWellDefined(UserMediaType.class); } }
  52. 52. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY Tools also lie Metrics tend to lie – Default PHP metrics in SonarQube – Tested file •  4535 CLOC •  Whole code written using imperative programming How many violations are there?
  53. 53. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY Tools also lie How many violations are there? 4 At least according to Sonar
  54. 54. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY Learn to REST REST is not so obvious /api/bi/XYZ123/1 /recommendations/1/items /offer/100?output=ESI/JSON
  55. 55. #AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY Community Involve all developers in building the Bootstrap. Or they will build their own tools. Łukasz Drumiński Mateusz Gajewski @wendigo
  56. 56. Allegro The New World Improvement WIIFY #AtmosphereConf @DebskiChris Q & A

×