EclipseCon 2010 API Design and Evolution (Tutorial)

3,054 views

Published on

In this tutorial, we explain how to design and evolve APIs without breaking client code: API Design Patterns and Idioms, Guidelines for writing Specifications, and Process for good API Design.

0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,054
On SlideShare
0
From Embeds
0
Number of Embeds
352
Actions
Shares
0
Downloads
20
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

EclipseCon 2010 API Design and Evolution (Tutorial)

  1. 1. API Design and Evolution Boris Bokowski, IBM Rational Martin Oberhuber, Wind River Michael Scharf, Wind River Based on presentation material co-authored with John Arthorne and Jim des Rivieres, IBM Canada © Copyright 2007, 2010 IBM Corp. and Wind River; made available under the EPL v1.0 March 24, 2010
  2. 2. Authors Dr. Boris Bokowski, IBM Rational, Ottawa Canada Full time committer on the Eclipse Platform and e4, Technical lead of the Eclipse Platform UI team, Member of the Eclipse Architecture Council, Committer Representative on the Eclipse Board of Directors. Dipl.-Ing. Martin Oberhuber, Wind River, Salzburg Austria Senior Member of Technical Staff at Wind River, Certified ScrumMaster, Chair of the Eclipse Architecture Council, Eclipse PMC Member, Target Management Project Lead, Eclipse Platform Core and e4 Committer. Dipl.-Phys. Michael Scharf, Wind River, Heidelberg Germany Principal Technologist and Architectural Overseer at Wind River, Member of the Eclipse Architecture Council, Model Enthusiast, Committer and Officially Approved Eclipse Troublemaker. API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 2
  3. 3. A Simple Example  Requirement: As a Tutorial Presenter, I want to know how many committers vs non-committers are in my talk, such that I can present material that matters to the audience.  API, 1st cut: public interface Course { public int countCommitters(); public int countNonCommitters(); } API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 3
  4. 4. Specification Example  Adding Specifications: /** * Represents a course at a conference. */ public interface Course { /** * Returns the number of committers who attended the course. * @return the number of committers who attended the course. */ public int countCommitters(); /** * Returns the number of non-committers who attended the course. * @return the number of non-committers who attended the course. */ public int countNonCommitters(); }  What's broken with this? API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 4
  5. 5. What's broken with this?  ...committers who attended the course is not what I want! - I want a live count (current snapshot)  I'm also interested in the total number of attendees! - Ok that was not in the requirement, but I found it when testing the API  I want the total number to be exact - Approximate percentage is OK for attendee role, but percentages should sum up as expected - Another new requirement  Actually, I may be interested in more roles in the future - E.g. contributor, commercial vs private - Yet another requirement API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 5
  6. 6. What can we learn from this?  Specification: API Documentation is important (“live count” vs “attendees”) to make crystal clear what the API does.  Process: Testing the API against an actual client is important from the earliest stages, to verify that it does what was expected. - Client and implementer learn from each other when discussing API Good API needs time and multiple iterations against real clients. API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 6
  7. 7. Requirements, revisited  As a Tutorial Presenter, I want to know after the talk how many people attended, such that I can boast how successful it was. - What about people who came and left again? - What about knowing whether they liked the talk? ➔ Eliminate waste: Avoid a new API iteration by first discussing requirements! ➔ Remember, API design is a process involving (at the very least) 2 parties. - In the ideal case, for making API rock solid, there's 3 distinct clients - Yet working with 1 primary client closely saves time, use others for validation of concepts. API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 7
  8. 8. Requirements, revisited (2)  As a Tutorial Presenter, I want to know after the talk how many distinct people heard the talk, and how many left a “+1” card when leaving, such that I get feedback about audience satisfaction.  As a Tutorial Presenter, I want to get live approximate statistics about committers vs non-committers in my talk, such that I can present material that matters to the audience. I may be interested in other attendee roles in the future, such as contributors, private or commercial attendees.  Contest: There will be a prize for the best API for these reqs - Use the patterns you'll learn in this course, and your ingenuity - E-Mail Solutions until today 7pm to martin.oberhuber@windriver.com - Will discuss solutions and give out the prize at the Unconference, 7:15pm in this room API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 8
  9. 9. Contest  Launch Eclipse SDK or any package (JDT preferred)  File > Import > General : Existing Projects into Workspace  Archive: org.eclipsecon.apitutorial.zip  Hand off: File > Export > General : Archive File API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 9
  10. 10. Goals of this Course  Pick up some usable advice for designing good API - Understand why good API is important - Understand the key aspects of good API - Best practices, patterns and idioms for process, code and spec We'll talk mostly about style – this is not a course on SW Architecture  Know how to safely evolve existing API - Learn what's new in terms of tooling and policy around API  Have fun hearing some API Stories  Get some good references to read on or read up on this talk API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 10
  11. 11. Agenda  Introduction  Why API?  API Design - What is Good API? - Design – Guidelines, Patterns and Idioms - Specification – Language, Do's and Dont's  API Evolution - Binary, Source and Contract Compatibility - API Deprecation  Tools for Crafting API - API Process - API Tooling  References, Q&A API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 11
  12. 12. Why APIs?  APIs are interfaces with specified and supported behavior. - Specification: Guideline for implementations, judge what's a bug - Supported: Bugs will be fixed  Key for successful componentization and scaling up - Develop, test and evolve components individually - API: Provide a stable Platform to build on - SPI: Allow multiple implementations, e.g. improve performance  Good APIs capture and bind clients - Clients invest into API (by code, and by learning the API)  Good APIs minimize waste - Encourage code re-use instead of re-writing - Reduce code and complexity - Bad APIs lead to bugs, maintenance and documentation needs API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 12
  13. 13. Different styles of API evolution Product development without compatibility Platform development open-ended promises set of Clients Clients will continue to work Clients must adapt to without being recompiled changes, cost is unknown Internal product Internal product development family development clients known Clients will adapt to Clients should continue to changes, cost is known work when recompiled lock-step development not in lock-step (“screw the client”) (“maintain compatibility”) API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 13
  14. 14. API Design API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 14
  15. 15. Characteristics of good API API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 15
  16. 16. Characteristics of good API ? API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 16
  17. 17. Characteristics of Good API  Easy to Learn  Leads to Readable Code  Hard to Misuse  Complete  Stable API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 17
  18. 18. Make your API Easy to Learn  Implies simplicity - But not too simple  Minimal surprise (consistency) - Meets the expectations of a user - Depend on the knowledge of the users - Mimic the environment  Avoids boilerplate code  Client Centric API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
  19. 19. Make your API Client Code Easy to Read  Implies the right level of abstraction  Think in terms of clients  Naming matters! API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
  20. 20. Easy to Read vs. Easy to Write (Learn) API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
  21. 21. Easy to Read vs. Easy to Write (Learn) new Label(parent, SWT.SEPARATOR | SWT.HORIZONTAL); API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
  22. 22. Easy to Read vs. Easy to Write (Learn) new Label(parent, SWT.SEPARATOR | SWT.HORIZONTAL);  Easy to Read  Hard to Write API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
  23. 23. Easy to Read vs. Easy to Write (Learn) new Label(parent, SWT.SEPARATOR | SWT.HORIZONTAL); new Label(parent, true, true); API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
  24. 24. Easy to Read vs. Easy to Write (Learn) new Label(parent, SWT.SEPARATOR | SWT.HORIZONTAL); new Label(parent, true, true);  Hard to Read  Easy to Write (code completion) API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
  25. 25. Easy to Read vs. Easy to Write (Learn) new Label(parent, SWT.SEPARATOR | SWT.HORIZONTAL); new Label(parent, true, true); Label label=new Label(parent); label.setSeparator(true); label.setHorizontal(true); API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
  26. 26. Easy to Read vs. Easy to Write (Learn) new Label(parent, SWT.SEPARATOR | SWT.HORIZONTAL); new Label(parent, true, true); Label label=new Label(parent); label.setSeparator(true); label.setHorizontal(true);  Easy to Read (but verbose)  Easy to Write (but verbose) API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
  27. 27. Make your API Hard to Misuse  Use java (JDT) to guide the user  Avoid hidden contracts API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
  28. 28. The “ultimate” API API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
  29. 29. The “ultimate” API class U { static Object do(Object o) {} } API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 29
  30. 30. The “ultimate” API class U { static Object do(Object... o) {} } A little Java 5 Convenience API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 30
  31. 31. Hidden Contracts class U { static Object do(Object... o) {} } Object w=U.do("createWindow"); U.do(w,"setSize",500,500); U.do(w,"show"); API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 31
  32. 32. Hidden Contracts class U { static Object do(Object... o) {} } API is more than the signature API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 32
  33. 33. Is this Problematic? void setup(Map map); Collection elements(); API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 33
  34. 34. Is this Problematic? void setup(Map map); Collection elements();  Easy to Misuse  Hard to Write  Needs Lots of API Documentation API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 34
  35. 35. Is this Better? void setup(Map<String,Object> map); Collection<IElement> elements(); API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 35
  36. 36. Is this Better? void setup(Map<String,Object> map); Collection<IElement> elements();  It still needs lots of API documentation API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 36
  37. 37. Make your API Stable  Easy to extend and evolve  Do not over-specify  Do not break clients API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
  38. 38. Make your API Complete  Lets the users do everything they want  Fits the requirements  Do mot miss important concepts API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
  39. 39. Recipes for Good API  Heuristics  Deviation should be with a reason API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 39
  40. 40. API is a Cover Story  Hide implementation  Hide information  Hide internal data formats  Think about concepts rather than code API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
  41. 41. Names Matter  Good names drive development - Easy to Learn, Easy to Read  If it's hard to name, it's probably broken - take a step back  Avoid abbreviations - Hard to remember where abbreviation is used API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
  42. 42. One API should do One Thing and do it Well  Less is better than too much API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
  43. 43. As Small as Possible (but not Smaller)  Minimal to satisfy requirements - can always add later but not remove  Avoid the need for boilerplate code in the client API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
  44. 44. Consider non-functional aspects  Performance  Threading  State Ownership  Cleanup API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
  45. 45. Mimic the base Platform  Naming  Patterns and Language - makes it easy to learn new API API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
  46. 46. Java API Guidelines  Class Guidelines  Method Guidelines  Exception Guidelines API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 46
  47. 47. Class Design Guidelines API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 47
  48. 48. Try to keep Classes Immutable  Immutable objects are simple, thread-safe and reusable  If there must be state, keep it small and known, and document it API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
  49. 49. Prohibit Subclassing – or clearly document it  Subclasses are sensitive to implementation change in the base API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
  50. 50. Subclass only “IsA” relations  A Set is a Collection  Properties is not a HashMap API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
  51. 51. Avoid interfaces that clients may implement  Use abstract classes instead to allow evolving API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
  52. 52. Separate API from SPI (Service Provider Interface)  Evolve API and SPI separately  SPI may have less strict rules  Example: org.eclipse.core.filesystem API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
  53. 53. Method Design Guidelines API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 53
  54. 54. Prefer factory method over public constructor  Allows object caching and sharing  Life-cycle control  Dependency Injection is even better! API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
  55. 55. Make simple things simple, make hard things possible  Don't force the client write boilerplate code  Don't violate the principle of least astonishment  Don't clients into exception processing API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
  56. 56. Use consistent naming and ordering of parameters  Platform: IProgressMonitor always comes last  Avoid parameter lists longer than 4 - impossible to remember API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
  57. 57. Use proper parameter and return types  Input types as specific as possible - as generic as makes sense  Avoid String if something better exists - static type checking  Avoid Collections API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
  58. 58. Exception Design Guidelines API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 58
  59. 59. Exceptions are for the exceptional  Don't use exceptions for control flow  Don't use return values for exceptional behavior - Java is not C API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
  60. 60. Use Checked Exceptions only if client must be alerted  Don't force the client handle situations it doesn't care about  Allow exceptions fall through to proper handler API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
  61. 61. Fail fast  Report errors soon – compile time is best  Include failure cause in exceptions API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0
  62. 62. Summary: API Design  Make your API easy to read, write and evolve  Be client centric  API is a Cover Story  Names Matter! Repeat until happy  Keep API Small: If in doubt, leave it out  Make simple things simple, make hard things possible  Consider non-functional aspects: No Spec - No API.  Emulate the Platform API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 62
  63. 63. Eclipse Design Idioms  Use int option flags to allow for future extension - Same effort but more flexible than boolean - Common Pattern used in the Platform, e.g. IProject.open(IResource.NONE, IProgressMonitor mon)  Separate concerns: Do one thing at a time - Adapter pattern to mix-and-match functionality - IAddonInterface addon = (IAddonInterface)Platform.getAdapterManager().getAdapter( baseObject, IaddonInterface.class)  e4: Use Dependency Injection API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 63
  64. 64. API problems that make life difficult for clients  Spec too vague, no single consistent Story  Completeness (e.g., add but no remove)  Names (avoid overly long names for elements that are used a lot)  All interfaces, but no constructor  Check assumptions about defaults  Internationalization: specify if strings are visible to the end-user  Lack of considering non-functional requirements - thread safety - progress reporting - being able to cancel long-running operations - nesting, composeability API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 64
  65. 65. API Specifications API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 65
  66. 66. API specs There's many things an API User needs to know, which are not expressed in the Java signatures...  Specification Design Goals by Role A. Tell client what they need to know to use it B. Tell an implementor how to implement it properly C. Tell tester about key behaviors to test D. Determines blame in event of failure ➔ Document religiously, but only as far as the cover story is concerned (don't let implementation leak) API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 66
  67. 67. Question: What do we spec for methods? /* © Copyright 2007 IBM Corp. All rights reserved. This source code is made available under the terms of the Eclipse Public License, v1.0. */ /** * * ??? * * * */ public Object pop(); API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 67
  68. 68. Question: What do we spec for methods? /* © Copyright 2007 IBM Corp. All rights reserved. This source code is made available under the terms of the Eclipse Public License, v1.0. */ /** * Removes and returns the object on the top of this stack. * The stack must not be empty. * * @return the object popped off the stack; may * be <code>null</code> */ public Object pop();  Method specs include - Preconditions - Results API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 68
  69. 69. Lessons learned  API is not just public methods  Document religiously, and carefully No specs. No API. API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 69
  70. 70. Appropriate level of specification detail  Is the specification too specific or detailed, making it difficult to evolve or implement? - Gain flexibility by hiding implementation. - API is a cover story.  Is the spec too vague, making it difficult for clients to know the correct usage? - Always assume clients are unfamiliar with your code. - Tell a compelling and seamless story. - If incomplete, clients will look up your implementation :(  Is the API designed to be implemented or extended by clients? - Be aware of different kinds of clients. - Keep Client API separate from Service Provider API (SPI). API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 70
  71. 71. Specification Checklist  What do YOU want to read in an API Spec? API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 71
  72. 72. Over-specification makes Evolving hard  Exposing unnecessary implementation details  Factory methods: specifying that it "returns a new instance" rather than "returns an instance" - prevents future caching/pooling  Specifying the whole truth, rather than just the truth - Specifying all failure cases (instead of "reasons for failure include...") - Specifying that an enum or similar pool of types is a closed set - prevents adding entries later (eg: resource types) - Specifying precise data structures of return types (HashSet, rather than just Set or Collection) API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 72
  73. 73. Over-specification makes Implementation hard Making promises that are hard to keep:  Over-specifying precision of results (returns the number of nanoseconds since the file was changed)  Returning a data structure that promises to remain up to date indefinitely (leaking live objects)  Promises about the order of operations - prevents future concurrency  Real-time promises API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 73
  74. 74. API Contract language  The language used in an API contract is very important  Changing a single word can completely alter the meaning of an API  It is important for APIs to use consistent terminology so clients learn what to expect API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 74
  75. 75. API Contract language  RFC on specification language: http://www.ietf.org/rfc/rfc2119.txt  Must, must not, required, shall: it is a programmer error for callers not to honor these conditions. If you don’t follow them, you’ll get a runtime exception (or worse)  Should, should not, recommended: Implications of not following these conditions need to be specified, and clients need to understand the trade-offs from not following them  May, can: A condition or behavior that is completely optional API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 75
  76. 76. API Contract language Some Eclipse project conventions:  Not intended: indicates that you won’t be prohibited from doing something, but you do so at your own risk and without promise of compatibility. Example: “This class is not intended to be subclassed”  Fail, failure: A condition where a method will throw a checked exception  Long-running: A method that can take a long time, and should never be called in the UI thread  Internal use only: An API that exists for a special caller. If you’re not that special caller, don’t touch it Emulate the Platform language since your clients will know it. Use API Tooling tags. API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 76
  77. 77. Summary: API Specification review checklist  Purpose: Summarized in 1 introductory sentence  Contract: Pre-conditions, Post-conditions  Data: Argument and result types, ownership, special cases  Failure: Any cases worth specifying? Don't overdo!  Non-functional aspects: as relevant for Cover Story - Side effects - Concurrency - Event ordering - Callbacks  Is the story complete, compelling, and seamless?  Does your spec help your clients without limiting your freedom? API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 77
  78. 78. Summary: API Design  Know your Clients and Iterate over Requirements with them  Make simple things simple, make hard things possible  Keep API Small: If in doubt, leave it out.  Names Matter! Repeat until happy.  API is a Cover Story  Specs are important! No Spec - No API.  Emulate the Platform Know where to look for details: http://wiki.eclipse.org/API_Central API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 78
  79. 79. Evolving API API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 79
  80. 80. Evolving APIs  Different situations when evolving an API  Compatibility  Techniques for evolving APIs  Techniques for writing APIs that are evolvable API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 80
  81. 81. But remember  Try to get it right the first time.  Failing that, do not break clients. APIs should be stable. API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 81
  82. 82. Compatibility  Contract – Are existing contracts still tenable?  Binary – Do existing binaries still run?  Source – Does existing source code still compile? API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 82
  83. 83. Contract compatibility After: Before: /* © Copyright 2007 IBM Corp. All rights reserved. This source code is made available under the terms of the Eclipse Public License, v1.0. */ /** /** * Returns the current display. * Returns the current display, if any. * @return the display; never null * @return the display, or null if none */ */ public Display getDisplay(); public Display getDisplay(); • Not contract compatible for callers of getDisplay • Contract compatible for getDisplay implementors API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 83
  84. 84. Contract compatibility  Weaken method preconditions – expect less of callers - Compatible for callers; breaks implementors  Strengthen method postconditions – promise more to callers - Compatible for callers; breaks implementors  Strenghten method preconditions – expect more of callers - Breaks callers; compatible for implementors  Weaken method postconditions – promise less to callers - Breaks callers; compatible for implementors API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 84
  85. 85. Compatibility quiz  Is the code snippet a binary compatible change?  Is it source compatible?  For clients?  For implementers? API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 85
  86. 86. Compatibility quiz #1 Before: After: /* © Copyright 2007 IBM Corp. All rights reserved. This source code is made available under the terms of the Eclipse Public License, v1.0. */ public class Test { public class Test { public void foo() { public void foo() { } } System.out.print(“Yes”); } } System.out.print(“Oui”);  • Binary compatible • Method bodies do not affect binary compatibility API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 86
  87. 87. Compatibility quiz #2 Before: After: /* © Copyright 2007 IBM Corp. All rights reserved. This source code is made available under the terms of the Eclipse Public License, v1.0. */ public class Test { public class Test { public void foo() {} public void foo() {} x public void bar() {} } } • Not binary compatible • Not source compatible • Deleting methods is a breaking change API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 87
  88. 88. Compatibility quiz #3 Before: After: /* © Copyright 2007 IBM Corp. All rights reserved. This source code is made available under the terms of the Eclipse Public License, v1.0. */ x public class Test { public class Test { public void foo() {} public void foo(int flags) {} } • Not binary compatible • Parameters are part of the method signature API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 88
  89. 89. Compatibility quiz #4 Before: After: /* © Copyright 2007 IBM Corp. All rights reserved. This source code is made available under the terms of the Eclipse Public License, v1.0. */ x public class Test { public class Test { public void foo(String s) {} public void foo(Object o) {} } } • Not binary compatible • Source compatible API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 89
  90. 90. Compatibility quiz #5 Before: After: /* © Copyright 2007 IBM Corp. All rights reserved. This source code is made available under the terms of the Eclipse Public License, v1.0. */ public class Test { public class Test { } public void foo(Object o) {} } public void foo(Object o) {} public void foo(String s) {}  • Binary compatible • When source references are recompiled they may be bound to the new method • Will cause errors in source references with a null argument, such as test.foo(null) API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 90
  91. 91. Compatibility quiz #6 Before: After: /* © Copyright 2007 IBM Corp. All rights reserved. This source code is made available under the terms of the Eclipse Public License, v1.0. */ public class Super { public class Super { } public void foo(String s) {} } public void foo(String s) {}  public class Sub extends Super { public class Sub extends Super { public void foo(String s) {} } } • Binary compatible • A different method will be called at runtime when method “void foo(String) is invoked on an object of type “Sub” API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 91
  92. 92. Compatibility quiz #7 Before: After: /* © Copyright 2007 IBM Corp. All rights reserved. This source code is made available under the terms of the Eclipse Public License, v1.0. */ x public class Test { public class Test { public static final int x = 5; public static final int x = 6; } } • Not binary compatible • Constant values that can be computed by the compiler are in-lined at compile time. Referring code that is not recompiled will still have the value “5” in-lined in their code API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 92
  93. 93. Compatibility quiz #8 Before: After: /* © Copyright 2007 IBM Corp. All rights reserved. This source code is made available under the terms of the Eclipse Public License, v1.0. */ public class Test { public class Test { public static final public static final  String s = “foo”.toString(); String s = “bar”. toString(); } • Binary compatible • Constant value cannot be computed at compile-time API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 93
  94. 94. Compatibility quiz #9 Before: After: x /* © Copyright 2007 IBM Corp. All rights reserved. This source code is made available under the terms of the Eclipse Public License, v1.0. */ package org.eclipse.internal.p1; package org.eclipse.internal.p1; public class Super { public class Super { protected void foo(String s) {} } } package org.eclipse.p1; package org.eclipse.p1; public class Sub extends Super { public class Sub extends Super { } } • Not binary compatible • Protected members accessible from an API type are API • Is this valid if class Sub is final or says clients must not subclass? API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 94
  95. 95. Compatibility quiz #10 Before: After: /* © Copyright 2007 IBM Corp. All rights reserved. This source code is made available under the terms of the Eclipse Public License, v1.0. */ public class E extends Exception {} public class E extends Exception {}  public class Test { public class Test { protected void foo() throws E {} protected void foo() {} } } • Binary compatible • There is no distinction between checked and unchecked exceptions at runtime • Not source compatible because catch blocks in referring methods may become unreachable API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 95
  96. 96. Compatibility quiz #11 Before: After: /* © Copyright 2007 IBM Corp. All rights reserved. This source code is made available under the terms of the Eclipse Public License, v1.0. */ public class A { public class A { } public void foo(String s) {} public class C extends A { } public void foo(String s) {} public class B extends A {}  public void foo(String s) { public class C extends B { super.foo(s); public void foo(String s) { } super.foo(s); } } } • Binary compatible • The super-type structure can be changed as long as the available methods and fields don’t change API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 96
  97. 97. Compatibility quiz #12 Before: After: x /* © Copyright 2007 IBM Corp. All rights reserved. This source code is made available under the terms of the Eclipse Public License, v1.0. */ public class Test { public class Test { } public Test(String s) { } } • Not binary or source compatible • When a constructor is added, the default constructor is no longer generated by the compiler. References to the default constructor are now invalid • You should always specify at least one constructor for every API class to prevent the default constructor from coming into play (even if it is private) • A constructor generated by the compiler also won’t appear in javadoc API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 97
  98. 98. Compatibility quiz #13 Before: After: x /* © Copyright 2007 IBM Corp. All rights reserved. This source code is made available under the terms of the Eclipse Public License, v1.0. */ public class Test { public class Test { public void foo() {} public boolean foo() { } return true; } } • Not binary compatible because the return type is part of the method signature • Source compatible only if the return type was previously void API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 98
  99. 99. Lessons learned  It is very difficult to determine if a change is binary compatible  Binary compatibility and source compatibility can be very different  You can’t trust the compiler to flag non-binary compatible changes Evolve once, check twice.  If in doubt, check “The Bible”: Evolving Java-based APIs, Jim des Rivieres et al available from http://wiki.eclipse.org/API_Central API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 99
  100. 100. Binary compatibility DONT's for API elements  Rename a package, class, method, or field  Delete a package, class, method, or field  Decrease visibility (change public to non-public)  Add or delete method parameters  Change type of a method parameter  Add or delete checked exceptions to a method  Change return type of a method  Change type of a field  Change value of a compile-time constant field  Change an instance method to/from a static method  Change an instance field to/from a static field  Change a class to/from an interface  Make a class final (if clients may subclass)  Make a class abstract (if clients may subclass) ... API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 100
  101. 101. Binary compatibility DO's for API elements  Add packages, classes, and interfaces  Change body of a method  Do anything you want with non-API elements  Add fields and type members to classes and interfaces  Add methods to classes (if clients cannot subclass)  Add methods to interfaces (if clients cannot implement)  Add non-abstract methods to classes (if clients may implement)  Reorder class and interface member declarations  Change value of a field (if not compile-time constant)  Move a method up to a superclass  Make a final class non-final  Make an abstract class non-abstract  Change name of method formal parameter ... API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 101
  102. 102. Evolving Generified API Add type parameter Breaks compatibility (unless type was not generic) Delete type parameter Breaks compatibility Re-order type parameters Breaks compatibility Rename type parameters Binary compatible Add, delete, or change type Breaks compatibility bounds of type parameters API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 102
  103. 103. Techniques for evolving APIs  Create extension interfaces, use naming convention (ITurtleExtension, ITurtle2)  Wiegand’s device  Deprecate and try again  Proxy that implements old API by calling new API API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 103
  104. 104. Techniques for enabling API evolution  Use abstract classes instead of interfaces for non-trivial types if clients are allowed to implement/specialize  Separate service provider interfaces from client interfaces  Separate concerns for different service providers  Hook methods  Mechanisms for plugging in generic behavior (IAdaptable) or generic state, such as getProperty() and setProperty() methods API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 104
  105. 105. Non-Java API's  File Formats - Workspace Compatibility section in the Eclipse Project's Project Plan - Provide Migration code when changing, e.g. Preferences  Extension Points - Expected Parameters - Can be evolved like Java APIs: Version Numbering - No Tooling available yet  ID's - Even Extensions are API for consumers! - Example: ID's of Property Testers expected to be in the Platform - No Tooling, no good automatic documentation!  Data Protocols, Wire Formats, …  Functionality of commandline args, config files, … - Example: eclipse -configuration /tmp API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 105
  106. 106. API Deprecation  Reasons for deprecating API - A new story fixed issues (e.g. concurrency, performance) - Small API – Less to Learn, easy to understand - Planning a major implementation rewrite (think e4)  Reasons for removing API - At first, deprecation is just a recommendation  But nobody will act if there is no axe in the window  May remain deprecated forever if just for ease-of-learning - Schedule API Removal for serious issues - Removal breaks clients and must be clearly announced (Policies) - Announce removal date. Use API Tooling to find parties to discuss.  Eclipse Project Policy for Removal: 2 years lead time http://wiki.eclipse.org/API_Central API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 106
  107. 107. Who can remember all this? API Design is hard... … but there is rescue! API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 107
  108. 108. API Process API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 108
  109. 109. Before you begin  Have and agree on common API guidelines, e.g.: - Eclipse Naming conventions http://wiki.eclipse.org/Naming_Conventions - How to Use the Eclipse API http://www.eclipse.org/articles/Article-API%20use/eclipse-api-usage-rules.html  Have someone in charge of the API early on  Have well-defined component boundaries and dependencies - There's no good API for bad Architecture - E.g., Core vs. UI API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 109
  110. 110. Work with a client  APIs exist to serve the needs of clients - Where are those clients? What do they need?  Important to work with actual clients when designing API - Designing APIs requires feedback from real clients who will use it - Otherwise risks crummy API that real clients cannot use  Find a primary client - Ideally: adjacent component, different team, same release schedule - E.g., JDT UI is primary client of JDT Core  Work closely with primary client - Listen to their problems with using API - Watch out for lots of utility classes in client code symptomatic of mismatch between API and what client really needs - Work together to find solutions API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 110
  111. 111. API Design Process  Gather Requirements – but with healthy skepticism - Some “Requirements” are really implementation suggestions - API should be the Cover Story but allow different implementations!  Start the spec with 1 page - Iterate, rinse, return - take time to evolve  Code early against the API - Tests, Examples to see how it “feels” in real life - This is not throw-away code but may become regression tests or examples as part of the spec later on - Even more important for SPI! - If there is just one implementation of a service that's meant to be changeable, you'll surely mess up.  Have realistic expectations: expect to evolve API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 111
  112. 112. Development cycle, releasing an API  Important to distinguish between - On-going development - Released API  In each development cycle, opportunity to work with well-known clients on new API  Release cycles can be short or long  Release cycles of API, implementation, and Client may be: - The same (lock step development) - Different (more likely!)  What kind of compatibility promise do you make? API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 112
  113. 113. Across components, use API, not internals  Component provides API for arbitrary clients - API exists to tame inter-component coupling  Client components are expected to use API “to spec” - Not depend on behavior not covered in API spec - Not depend on internals - Foolish to make exceptions for close friends  Close friends don’t point out flaws  Sets bad example for others API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 113
  114. 114. Eclipse API Process Guidelines  Provisional API Guidelines - Mark x-internal:=”true” in the Manifest until solidified - Allows for wider exposure to clients, even across releases - NEW: Allowed to have provisional API in its final namespace  No more forced breaking of clients when refactoring from “internal” package to “public” package namespace  X-internal is the premier markup of provisional API  Must have a client - Each extension point or API must have a client in Open Source - Ensure that example code and test code is there  Freeze by M6 - With PMC Escalation process into M7 Reference: http://wiki.eclipse.org/API_Central API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 114
  115. 115. API Tooling API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 115
  116. 116. API Tooling for Developers  Has become indispensable for - Adding proper Javadoc Tags - Reporting mis-use according to tags - Detecting Binary Compatibility Breakage - Detecting Leakage of non-API Types into API - Updating Version Numbers correctly  Super easy setup - Window > Preferences > PDE > API Baselines : Register Baseline - Single directory with plugins to compare against - Problem Markers; use Ctrl+1 Quickfix to add problem filters  Documentation: http://www.eclipse.org/pde/pde-api-tools/ API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 116
  117. 117. Version numbers  major.minor.service.qualifier  From release to release, the version number changes as follows: - When you break the API, increment the major number - When you change the API in an (binary) upwards compatible way, increment the minor number - When you make other changes, increment the service number - The qualifier is changed for each build  Proposal: Less strict versioning for Service Provides (SPI) - Breaking changes in SPI are allowed with minor rev - A variant of the “I know my clients so I can break them” story - Better not make such API public, if I know you, use internals  Guidelines, see http://wiki.eclipse.org/API_Central API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 117
  118. 118. API Tooling for Build Engineers  .api_description file needed in binary bundles - Conveys information from API tags like @noimplement - No problem reporting without these in the binary baseline  Setup: Just add this to your build.properties: - generateAPIDescription=true API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 118
  119. 119. API Tooling for Reports  API (mis) Usage Reporting - Who uses my internals? - New with 3.6M5: Run > External Tools...  API Comparison - Generate an XML Snapshot of API - Compare two snapshots, generate XML / HTML Report - Useful for New&Noteworthy, Release Reviews API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 119
  120. 120. Summary: What you should take home  Why API – 4 quadrants  Design Guidelines - Make Simple things Simple, make complex things possible  No Specs – No API - Non-functional aspects: Performance, Threading, State, ... - Cover Story: Avoid Overspecification  API Evolution: binary – source – contract compatibility - API Tooling  Process for API: Requirements – Rinse – Repeat - No API without Clients  Where to look for more - References API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 120
  121. 121. References  API Central on Eclipse Wiki http://wiki.eclipse.org/API_Central - API Process: This Presentation, linked on API Central  Eclipse API Process and Policies, all linked from API Central - API Design: Joshua Bloch, Effective API Design (1 Hr Video) http://www.infoq.com/presentations/effective-api-design - API Specifications:  EclipseCon 2006 API Tutorial, see API Central  Sun, Requirements for Writing Java API Specifications  Sun, How to Write Doc Comments for the Javadoc Tool - API Evolution  Evolving Java-Based API's, see API Central  EclipseCon 2007: Eclipse APIs and Java 5, see API Central - API Tooling, Documentation and Wiki all linked from API Central  Joshua Bloch, Effective Java Programming Language Guide API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 121
  122. 122. And remember the Contest...  Create your API for counting Course attendees and Committer / User Statistics  E-Mail solutions to martin.oberhuber@windriver.com  Come to the Winner Selection at the Unconference - Today 19:15 in this room (Lafayette) - We'll all learn a lot by looking at solutions - … and the winner can take their prize away  If you have any more questions about API... - Find us at the conference, or file a bug with the Architecture Council - http://wiki.eclipse.org/Architecture_Council API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 122
  123. 123. Legal Notices  IBM and Rational are registered trademarks of International Business Corp. in the United States and other countries  Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both  Other company, product, or service names may be trademarks or service marks of others API Design and Evolution © 2007, 2010 IBM Corp and Wind River; made available under the EPL v1.0 123

×