Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Edc2013 compliance conundrum-alperin


Published on

  • Be the first to comment

  • Be the first to like this

Edc2013 compliance conundrum-alperin

  1. 1. For the Evans Data Corp 2013 Developer Relations Conference Jon Alperin, Managing Director, Avaya DevConnect Program The Compliance C Th C li Conundrum: D i i points f E d Decision i t for Ecosystem I t t Interoperability P bilit Programs You’ve enabled your developer and created a value-added ecosystem of 3rd party and homegrown solutions. Whether you are a mobile platform, a cloud-based solution, or typical enterprise or consumer applications, extracting value from this ecosystem often requires a tighter go-to-market plan, and that, in turn, means ensuring that these third party solutions actually work correctly with your products and services. In this session, we’ll explore k d i i points i ’ll l key decision i t involved i scoping, b ildi and executing l d in i building d ti interoperability or compliance testing programs for your ecosystem. You’ll learn where there are tradeoffs to be made in various approaches, and see how you can effectively bridge from pure developer programs to value-added partner programs with direct revenue impacts.© 2009 Avaya Inc. All rights reserved. 1
  2. 2. The Avaya DevConnect Program is one of the leading B2B developer programs in the communications industry, supporting customers, Avaya channel partners, technology partners and competitors alike in the development of value-added and interoperable solutions that leverage Avaya products and technologies. The DevConnect Program takes a 3 phase approach towards growing our ecosystem, spanning developer enablement, business development and Go to Market programs. Within this third phase, DevConnect Compliance Testing programs are used to validate interoperability for the benefit of our customers, channels and sales teams, and Avaya support teams, as well as those technology companies, both friendly and competitive, that participate in the program.© 2011 Avaya Inc. All rights reserved. 2
  3. 3. Interoperability programs sit in the overlap between pure developer programs andpartner programs. Each type of program will generally have their own sets ofstakeholders, and it is in the intersection between them that you can see where theinfluence and benefits of formalizing an interoperability or compliance programbenefits both audiences. 3
  4. 4. Interoperability programs can be found under many names. And in some cases, theterminology used to describe a program can have a specific and direct bearing on thescope of a program, or even a legal implication to consumers, partners and your owncompany.When considering launching a formal program, consider which term really describesthe business goals best. 4
  5. 5. For the remainder of this session, I’ll be focusing on the “Big W” questions – thewhy’s, who’s, when’s, where’s and hoW’s (yeah, I know… but it still has a ‘W’ in theword).In considering and designing a program, these are a number of decision points to bemade along the way. 5
  6. 6. First and foremost, of course, you need to clearly understand why you are building aprogram at all.Is it primarily for your own benefit, to protect your reputation with your customers andchannels? I it t protect your revenue stream, to ensure that 3rd party solutions which h l ? Is to t t t t th t t l ti hi hmay plug in to your platform or cloud-based solution do not negatively impact your cashflow?Do you used it to open new routes to market, filling gaps in your own portfolio? Or doesyour product or service offering actually require 3rd party components to be the driver forsales, in which case you are beholden to the quality of those offers in order to make yourown money? ?You may also need to consider competitive factors – do your solutions generallyinteroperate with other systems and solutions in your customers’ environments? Does thatmake it a barrier to your own sales if you cannot prove interoperability? Do you need thatinteroperability to be mutually acknowledged by those other companies?Or i hi i lO is this simply a reaction to pain your organization currently experiences i terms of i i i i l i in fcustomer dissatisfaction or increased support costs, caused by the quality or issuesothers have introduced, and for which your reputation may also suffer? 6
  7. 7. Once you’ve determine your underlying reasons for a program, you can begin to look atthe target audience. Is this a service that is available to any and all of your developers? Orsomething you make available only to specific partners that are part of your GTM motion?This d i iThi decision point will also play i t th question of scale vs. d th D i t ill l l in to the ti f l depth. Depending upon th di thesize of your community, making testing programs available to anyone in your communitycan increase costs, or create resource constraints further down the line in terms of howmuch high-touch vs. low-touch efforts you are directly providing towards the oversight andexecution of this effort. It also speaks to the overall timeframes you target for testing –trying to serve a large potential audience almost implies needing to automate and driverapid test cycles, vs. longer, more complex and thorough test efforts.In addition, depending upon the rationale for having an interop program in the first place,you need to consider if this is just pair-wise testing with a single individual partner, orwhether the solutions under test are complex, multi-vendor environments that need towork in a cohesive and comprehensive manner.And finally, you may also need to consider how competitive solutions fit in the scope ofthe effort – A you allowing competitors to participate i testing, or are there other h ff Are ll i i i i in i h hcheckpoints in place to determine rationale for participation? 7
  8. 8. Another point to consider is the overall scope of testing, which will drive further decisions regarding the type of test bed and test tools needed, as well as the overall timeframes involved in testing.ForF many solutions, basic i t f l ti b i interface validation will generally suffice. B t th d lid ti ill ll ffi But the deeper and d more critical the GTM elements are, the more you need to look at functional validation of a solution, possibly even going as far as testing operational issues, scalability, reliability, performance and more.
  9. 9. When designing a test plan, one must consider the scope of the test effort, and howyou position to customers, end users and your ecosystem where your responsibilities iti t t d d t h ibilitifor ensuring completeness and accuracy end.Is this just focusing on proper API usage or conformance to a specific industrystandard? Will you provide test tools that enforce a minimum set of functionalcapabilities? Will these tools test boundary conditions and error conditions that areunlikely to occur in normal operations, but for which 3rd parties should be able tohandle without issue?It’s also important to consider inbound vs. outbound testing. In some cases, it is the3rd party application driving and invoking functionality offered by yourplatform/service/product. In other cases, you may be expecting those applications todeliver specific functionality or react in very specific ways to support your own valueproposition. Do your test plans reflect those cases?And,And in some cases (particularly for B2C markets) will you be making non-technical markets), non technicaljudgements as to suitability of an application for your stamp of approval (the so-calledMorality judgement)?You’ll also want to consider the timing of testing – Will you be testing alpha or betareleases from your developers, or only GA-candidate products? Will you be doingtesting on beta releases of your own products? And what does this mean downstreamfor support purposes if these tests are completed on pre-GA products?Also, consider how public your test plans are. How much does a potential developerneed to know prior to committing to test about the specifics of the test plan or theminimum criteria? How much do channels and customers need to know about thespecifics of test scope? 9
  10. 10. When it comes to actually executing tests, there’s a lot of flexibility and different ways to do this. But you also need to consider what level of control and insight you may be giving up especially as it relates to up, your GTM objectives. First, you may have the ability to outsource testing to a third party firm like TekVizion or AppLabs. But you may also want to maintain the expertise with these 3rd party applications and solutions in-house. Then again, keeping one or more full time employees sitting around to do testing means having a pretty good handle on what sort of demand you’ll have for testing, and the timeframes and investments in human capital necessary to support those tests. Then you need to decide where to do the tests. Not everything can ( should) be done in the cloud. In y y g (or ) fact, for some solutions, it may be impossible to do testing anywhere but in a controlled lab environment. And while that lab could be your, it might need to be your partners. Which means one or both of you will need to send people and products to spots all around the world. And never underestimate the legal side, from customs to simply protecting the intellectual property of others (especially if you are testing multiple independent companies who might otherwise compete with one another at the same time and in the same facilities). When you consider test execution, you also need to give consideration to the extent by which is can be automated – and the tools necessary to do this Does a one size fits all test plan really work for you and this. one-size your ecosystem? Or does the actual application under test require a human being to ‘drive’ the test plan, and adjust to real-world differences and fine tune test efforts in real time? Would you support a more informal self-test if it were possible to provide a standard tool and test plan? Does that change your ability to provide support? And what sort of proof points would you look for – a simple “say so” by the vendor, or a specific output from your own tools?© 2011 Avaya Inc. All rights reserved. 10
  11. 11. It’s important to realize that just like your own products, testing is unlikely to beexhaustive. There will always be cases and conditions that simply would not be costjustified to put through the program. Moreover, every feature of your product may notbe used the same way (or at all) by your ecosystem. So forcing tests that simply donot match the functionality being exploited could lead to a high, erroneous failure rate.And don’t assume your own products are bulletproof. Testing will undoubtedlyuncover a problem in your own product – the question is whether you consider that afailure on the part of the 3rd party.And never underestimate the politics here – he who controls the checkbook is goingto have a big say, especially if the testing costs are high. Even the fact that a solutionis under test may be something that a company does not wish to have made public,until they have positive results to report. And even sales opportunities can putpressure on an organization to declare a test “good enough” even though there are g g g gknown bugs or failed test cases, much like products ship out the door with their ownshortcomings noted in release notes (or buried deep in documentation). 11
  12. 12. No matter what, someone has to pay for all this effort. You’re going to need people tomanage the program, develop the test plans and tools, and execute or review testresults. Depending upon your solution, you may need lab facilities, specialized testbeds and other tooling, etc.If you’re lucky enough to have deep pockets, you may be able to fund it completelyout of your pocket, which gives you great control over how things operate, and lowersbarriers to participation by others.On the other hand, you may want to artificially raise those barriers, and set a pricepoint that generally discourages only the most serious developers from pursuing aformal program with you. This can help quite a bit with managing the cost, andallowing for prioritization and focus of your resources. These costs can be break-even, or even profit-making efforts if the end result of having your seal of approval ona 3rd party solution helps the vendor drive their own revenue. yOr you can share the costs, perhaps building recovery of costs incurred from testingin to a GTM model.You may also want to consider a tiered approach, with different types of tests anddifferent price points (and benefits). 12
  13. 13. Finally, you need to answer the “What’s in it for me?” for your developers. Even at nocost for the testing, developers are still investing their own time and energy in to thisprocess, and they want to understand what they get out of this program.Is it just a statement of conformity of compliance, posted to your website? Achecksheet of test result coverages? Or are they getting some level of additionaldocumentation that offers value to them, their sales channels and their customers?Does this include marketing elements, from basic logo’s and marks that they can use,to certificates and plaques that they can display? Does it give them PR and othermarketing benefits, or link them to GTM Programs like app stores?And what is lifecycle expectations? Is the testing recognized as valid forever, or justfor the current major release? Or even more limited to a specific minor release? Anddoes it matter if the vendor changes their own products in ways that have no obvious g yor direct bearing on the point of interoperability?What about when there is an API change? Or, more interestingly, when the APIdoesn’t change, but the data exchanged across it does due to new functionality?All of these are considerations that your developers will chew on as they determinewhether the investment in your program is as g y p g good for them as it is for y you. 13
  14. 14. In summary, it’s all about understanding the tradeoffs. Having a very open testingprogram based on a specific, well-defined set of compliance criteria, intended to drivehigh volumes with little hands-on expertise and even less long-term responsibilities isa very different program scope, cost and investment model that one that serves adifferent GTM need.Tradeoffs abound, from how you handle competitive products within the scope of yourtest activities, to the implications of making “moral” decisions to put a stamp ofapproval on a certain type of application. Even test case failures aren’t necessarily asdefinitive as one might think, depending upon the end goals. 14
  15. 15. Thank you. 15