Download to read offline
Uploaded byAlban Gérôme
Reshaping the Hype Cycle
The document discusses reshaping the Gartner Hype Cycle model based on lessons learned from 10 years of experience in web analytics. The Gartner Hype Cycle is presented as a composite of two curves: a hype curve and an acceptance curve. Several techniques are proposed to "give hype a haircut" by improving the model with expert guidance and focusing on value, and to "lift up the acceptance curve" with approaches like early proof of value, addressing implementation challenges, and testing. This reshaped model aims to better represent the realities of new technology adoption over time based on data and experience.
More Related Content
![A Cheat Sheet For Marketing With Pinterest [Infographic]](https://cdn.slidesharecdn.com/ss_thumbnails/pinterest-infographic-140806095014-phpapp01-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Reshaping the Hype Cycle
More from Alban Gérôme
![[DSC Europe 25] Hans Kleinsman - The Compliance Gearbox: How Tax Tech Mediate...](https://cdn.slidesharecdn.com/ss_thumbnails/dxdytie1toel0hr90bjs-2-251212103250-174fdbe7-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Miodrag Pesovic & Vladislav Radonjic - Federated Data Archite...](https://cdn.slidesharecdn.com/ss_thumbnails/gsbe3y5it5uhndi4e08e-1-251212103249-f1008e0c-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Uros Pesic - The Reality of AI in Marketing.pdf](https://cdn.slidesharecdn.com/ss_thumbnails/rtkodnmtycovsllvzsyn-9-251215095918-b0c6bfe3-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Nikolay Burlutskiy - Best Practices for Building Enterprise M...](https://cdn.slidesharecdn.com/ss_thumbnails/uirvaiuvq8y1w8hzd9tx-7-251212103249-2619edb4-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Marko Krstic - Understanding the AI Threat Landscape - Risks,...](https://cdn.slidesharecdn.com/ss_thumbnails/tiyim1ins5jvbrvzpzla-2-251209104645-c69d3553-thumbnail.jpg?width=640&height=640&fit=bounds)
Reshaping the Hype Cycle
- 1. Reshaping the HypeCycle Lessons learnt and opinions from ten years spent in web analytics Alban Gérôme @albangerome MEE London 17-18 October 2018
- 2. The Gartner HypeCycle @albangerome Innovation Trigger Peak of Inflated Expectations Trough of Disillusionment Slope of Enlightenment Plateau of Productivity
- 3. A composite of2 curves? @albangerome Hype curve
- 4. A composite of2 curves? @albangerome Hype curve Acceptance curve
- 5. A composite of2 curves? @albangerome Gartner hype cycle
- 7. Giving hype ahaircut
- 8.
- 9. Snip #1 @albangerome 1. Expertstop to bottom
- 10. Snip #2 @albangerome 1. Expertstop to bottom 2. No noise, all signal
- 11. Snip #3 @albangerome 1. Expertstop to bottom 2. No noise, all signal 3. Huge uplifts
- 12. Snip #4 @albangerome 1. Expertstop to bottom 2. No noise, all signal 3. Huge uplifts 4. Very supportive culture
- 13. Snip #5 @albangerome 1. Expertstop to bottom 2. No noise, all signal 3. Huge uplifts 4. Very supportive culture 5. “No coding necessary”
- 15.
- 16.
- 17. Acceptance lift #1 @albangerome 1.Prove value early
- 18. Acceptance lift #2 @albangerome 1.Prove value early 2. IT can’t implement
- 19. Acceptance lift #3 @albangerome 1.Prove value early 2. IT can’t implement 3. Scarcity & Social Proof
- 20. Acceptance lift #4 @albangerome 1.Prove value early 2. IT can’t implement 3. Scarcity & Social Proof 4. Hub & Spoke Model
- 21. Acceptance lift #5 @albangerome 1.Prove value early 2. IT can’t implement 3. Scarcity & Social Proof 4. Hub & Spoke Model 5. Canaries
- 22. Acceptance lift #6 @albangerome 1.Prove value early 2. IT can’t implement 3. Scarcity & Social Proof 4. Hub & Spoke Model 5. Canaries 6. Automated tests
- 24.
- 25. Hype curve posthaircut… @albangerome Hype curve
- 26. … and iterativeimplementation @albangerome Hype curve Acceptance curve
- 27.
- 28. Before and after OriginalGartner Hype Cycle Reshaped Hype Cycle
- 30. A few closingthoughts @albangerome
- 31. Escalation of commitment @albangerome Construction proposal Modeland blueprints revealed Extra budget unlocked Building inauguration day What a nice building!
- 32.
- 33. Be data-informed notdata-driven! 51% of C-suite executives fully support their organisation’s digital and analytics strategy 58% of the executives say employee engagement and adoption of data requires leadership by example The Economist Intelligence Unit 2015KPMG 2017
- 34.
Editor's Notes
- #3 The Black Eyed Peas were singing ‘I Gotta A Feeling”. “Inglourious Basterds”, “The Hangover” and “Avatar” came onto the screens. TMZ just published rumours that Michael Jackson had been found dead in Beverly Hills. I lost my job as a senior front-end developer. About a year earlier, I was working at a London start-up called Touch Clarity, which was soon acquired by Omniture but I had not used SiteCatalyst at that point, only our Touch Clarity tool. Omniture SiteCatalyst became Adobe Analytics and Touch Clarity was part of what is now known as became Adobe Target. Then, a job ad about web analytics caught my attention, I applied, got the role and I never looked back. It certainly was no walk in the park and here are lessons I have learnt and opinions I have formed over my first 10 years in web analytics I would like to share with you arranged around 3 main themes: implementation, data analysis and the culture
- #19 The organisation had easy access to not one but possibly a team of implementation experts. It also had a web analyst, also an expert trained by the vendor. All may well even be certified if the vendor provides these and the web analyst has rehearsed his demo. Getting experts of that calibre to work on your websites will not come cheaply and your organisation will not have budgeted for this. You do not have to depend on the vendor’s consultants but you will need implementation experts and people experienced in using the tool. In fact, you will need a analytics department.
- #20 The organisation had easy access to not one but possibly a team of implementation experts. It also had a web analyst, also an expert trained by the vendor. All may well even be certified if the vendor provides these and the web analyst has rehearsed his demo. Getting experts of that calibre to work on your websites will not come cheaply and your organisation will not have budgeted for this. You do not have to depend on the vendor’s consultants but you will need implementation experts and people experienced in using the tool. In fact, you will need a analytics department.
- #21 The data used in the demo shows an obvious spike that took no time to find. As you track more and more metrics, add more dimensions, the signal to noise ratio increases and it quickly becomes harder to know where to start. Attention bias can prevent us from spotting something interesting. Then we might go round the PDCA loop more than once before zeroing in on the correct hypothesis instead of being spot on with the very first one. Leverage anomaly detection if your tool allows this. Managing analysis paralysis requires an analysis method such as REAN, AAARR or something similar and only analysts with prior experience know this.
- #22 The return on analytics spend rarely grows in spurts but slowly and incrementally. Your nuggets, if you ever find them, are probably someone else’s massive mistake. What if this nugget gets the person responsible in trouble or even lose their job? Possible. Has your competition all made the same error? Unlikely. Correcting these mistakes will probably just make you catch up the competition on a certain dimension rather than giving you the edge and lead. What gives you that lead is a sound foundation topped with the cumulative effect of many small incremental changes.
- #23 A supportive culture that gives credit to where it’s due. In the demo, recommendations get implemented, no questions asked. Then the analyst can claim credit and leads to a virtuous spiral toward digital transformation and data-based decision-making. In many organisations, the analyst or analytics team has been demoted to a pure support function leveraging the valid point that they lack domain knowledge and come up with recommendations that are not aligned with the business. The organisations consider them as the guardians of data quality, ask them to produce reports, to extract huge amounts of data for the incumbent teams who will try to extract the actionable insight themselves and take all the credit. The incumbent teams do not want to feel like they are placed under conservatorship and been told what to do by a support team.
- #24 The minimal tagging effort will give you some reports out of the box to be sure. But that’s just an ad for a flat in a new exclusive housing project. The pictures you see in the ad is for one of the most expensive flats in the building and then the ad says “Starting at £400,000”. £400,000 will not buy the kind of flat shown in the ad. No coding necessary will probably will not give the kind of report displayed in the demo. Sooner or later you will need to go through the motions. Collecting the business questions, prioritising them, getting them shaped into a tagging guide, IT to implement, QA. That’s probably how far down we can go for now. Over time, as organisations mature, they will have a more realistic approach to a launching an analytics programme. We can reshape the acceptance curve too and lift it up.
- #26 IT will do the implementation will lead to a disaster, do not do a big bang implementation, go agile Do not succumb to premature democratisation Implement a hub and spoke model where the hub should eventually disappear or dissipate into the organisation once mature enough Data extracts should be verified by a colleague and/or have the data extraction steps documented Keep the resulting curve above the threshold below which you could lose key supporters
- #28 Aim for a MVP implementation to demonstrate the value faster. John Gall, a system theory expert of the 1970’s wrote: “Some complex systems actually work but building a complex one from scratch never works. You have to start over, beginning with a working simple system.” That becomes “Some complex data capture implementations actually work but building a complex one from scratch never works. You have to start over, beginning with a working simple implementation.” So if you try to track everything 2 things happen to your acceptance curve, you reach the plateau much later and getting there takes longer
- #29 IT can get you started on your implementation pretty fast but in the long run, you will have stunted analytics capabilities Analytics may be just Javascript, IT has Javascript experts and is used to capturing the requirements to be sure. But since when analytics is taught in Computer Science? Analytics implementations leverage proprietary Javascript functions. Some vendors might only give you an implementation manual if you paid for implementation training! Other vendors might give public access to their implementation documentation but in any case, that’s a lot to study and in the meantime the clock is ticking. Some of the promised implementation features may well end-up descoped or deferred to a later release. That will not help an analytics implementation MVP grow because the resources earmarked for the phase 2 are used to tackle deferred phase 1 features and so on. An implementation consultant could start after IT can and yet still deliver more implementation features for phase 1 because their learning days are behind them and they can collect the requirements too. Remember that your implementation is never “done” once and for all, you will need that consultant back regularly
- #30 Your implementation MVP is not the only thing that will grow with each iteration. You should only grant access to happy few to analytics to begin with and then grow that small group by only a few new people. In the 18th century Antoine-Augustin Parmentier was an advocate for the introduction of potatoes to the French diet and throughout Europe. Potatoes were introduced in dinners where prominent guests, offering to royals bouquets of potato blossoms. But his master trick was have a very visible potato field watched by armed guards during the day and then letting them go home at night, prompting the crowd to steal the potatoes. That was the real purpose of that field. Scarcity and Social Proof, 2 concepts of Robert Cialdini more commonly leveraged by Apple today, will help drive the appetite across the organisation for analytics access. This Social Proof from highly prominent people translate at organisational level by C-suite support. They need to lead by example
- #31 2 people believe they have extracted the same data and find huge discrepancies. This is largely a reproducibility issue, both believe they have extracted the same data the same way but that is not possible or the figures would match. Earlier in my career, I worked in pharmaceutical industry, double-blind medical trials to be precise and I was on the database team for a little bit. When the nurses could not get a bottle number from the automated voice response system to resupply a tester, they would call the support line who would use an graphical interface and get a suitable bottle number for the titration the tester was randomised into. It that also failed, they contacted people like me who would go into the Oracle database, write a short SQL script to get a bottle number. If that script was wrong, it could kill someone, literally because it might have taken someone who was randomised to placebo to suddenly getting the highest possible titration they could have been randomised into instead of placebo. So, we ran the SQL script once, rollback, i.e. cancel the execution of the code and print it out and then a colleague would do the same thing and both resulted in the same bottle number, one of the two of us ran the same script again but for real this time. Both print outs were dated, signed by 2 database developers and then filed. If the 2 aspiring web analysts could do something similar there would be a single version of the truth but that may not be very practical. Luckily, there’s an alternative and it’s called a hub and spoke model. You create a new dedicated analytics team made of analytics experts and analytics enthusiasts, junior analysts in the incumbent teams. That new team as not budgeted for either, I know. The hub should focus on finding actionable insights. A rivalry for the data-driven crown might emerge between this new analytics hub, often called the analytics centre of excellence, and the incumbent teams. The C-suite has asked these incumbent teams to become data-driven and they might want to treat this new team as a pure support team who will do nothing more than extracting the data, do all the reporting and ensuring the data quality. Do not expect to find and kind analytics experts very long if the incumbent teams get their way. Expect these experts to leave and tell all their friends about what a “fine” organisation you are operating. But they lack “domain knowledge”, “business acumen“, I get that. So instead provide them with a steer by asking them the business questions you would like them to answer instead of bombarding them with data extracts only to cherry-pick the 1% that confirms prior beliefs. That’s not data-driven, that’s what I call “data-justified”! The credit for making the organisation data-driven must translate into different things in each team involved: The incumbent teams can claim credit for giving a steer to the analytics team and helping them becoming better aligned with the business, helping them develop their domain knowledge and even let these few data experts spend some time in their team to gain operational knowledge as part of a rotation between departments. These incumbent teams can also claim credit for operating as spokes and developing their analytics skills The analytics hub can claim credit for shielding the other teams from making data extracting errors that would occur if they were thrown at the deep end without much training but also against many congnitive biases such as “belief persistence”, “confirmation bias”, “cognitive dissonance”, not afraid to show data that reveals issues and ensure a high level of objectivity
- #32 The spokes are people who have been with the business for some time and have some data literacy, people from the finance department for example. They will do all the reporting and monitoring for the data that is of immediate concern for their team. It will be much easier for them to spot issues because the remit is much smaller than what the analytics centre of excellence would have to contend with. Also they have skin in the game because any anomalies in that data impacts them directly. The spokes will be your “canaries in the coalmine”, spotting issues much more reliably and faster. When a massive issue is impacting your analytics, the analytics hub will get one and then a few calls from the various spokes. By call #2, the hub could call all the other spokes and know the whole scope of the issue, in parallel rather than by in serial manner. Mistakes are part of the learning process but they will be spotted fast and addressed fast too.
- #33 Ask your implementation contractors to speak to IT as well about how they run automated tests before going live. The implementation experts will know how good looks like and help IT include additional tests scripts that will check the integrity of the analytics implementation. Expect this process to gain in sophistication after a few iterations there too. Eventually, any potential analytics reporting outages should be captured even before going live, before your canaries can spot them. This double layer of checks breeds high trust in the data across the organisation. If that data goes against the beliefs held prior, the business will find playing the “what if this data were the result of a bad implementation” card much more difficult.
- #42 Construction projects are often good examples of this escalation of commitment. At first people are seduced by the blueprints, early models and 3d computer simulations. The budget is signed-off, construction starts and people are giddy with excitement. Then the first unexpected issues appear, adjustments are made but soon a bigger budget is required or the project grinds to a halt. Nobody wants the eyesore or a half-completed construction site, demolishing would require money anyway. If that demolition budget was spent as extra budget for the completion of the project, it might just get completed. The Eurotunnel is a good example of a project that totally exceeded its initial estimates but is widely considered a success. Letting go of a dream is painful so people will overcommit to see it becoming true. If they knew of the total commitment required upfront the project would never get off the ground. It seems that many construction projects will fail without escalation of commitment. Most technologies go through the Gartner hype cycle rather than following this ideal adoption curve and analytics is no exception. An HBR article of June 2016 reported that: "More than half of senior executives experienced a backlog of at least two years on critical new analytics applications.“ Is escalation of commitment really the only way to deliver a successful analytics programme? Every client is different but the analytics vendors are well placed to know that there will be surprises along the way and even the best budget and delivery estimates will fly out of the window. They will focus on the common denominator: regardless of the type of organisation you are, our tool is awesome, easy to use and can deliver return on your investment. This focus maximises sales of product licences, implementation consulting time, training courses including on-site. In 10 to 20 years from now the adoption curve for analytics may look like a sigma curve with a slowly increasing plateau. But for now, the need for fast results will make organisations make unwise decisions and the adoption curve will look like the Gartner hype cycle.
- #43 Most of the people sitting at these demos do not have significant experience in our field. They are the people who will hold the purse strings, however. Bertrand Russell said: “The whole problem with the world is that fools and fanatics are always so certain of themselves, and wiser people so full of doubts”. He was not the first to come to this conclusion, we can trace this idea all the way to Confucius. These decision-makers will confuse the Pollyanna world of these demos with reality. But the external experts you have recruited for your new analytics department are just as prone to this effect. They might believe that actionable insight that is not aligned to the business strategy can have a bigger return than one that is aligned. Inexperienced people tend to feel more confident about their abilities and underestimate the difficulties they might face regardless of the side of the fence they are on, strategic or tactical. But quickly reality brings them down earth. With more experience, they will start doubting themselves and reach a real low. With even more experience, their initial foolish assumptions are slowly replaced by tried and tested practices. Eventually, we reach mastery but we retain a healthy dose of self-doubt, more open to learning something new and seeing our assumptions challenged. We are looking at the Dunning-Kruger Effect and the fact that this curve bears a similarity to Gartner hype cycle. In fact, I believe that this is the key driver of the hype curve. There are many aspects of these demos that should be met with more healthy scepticism:
- #45 You need support from the C-suite, that will send the message loud and clear that if the senior managers want a shot at the C-suite, they will have to learn how to combine their years of experience and domain knowledge with increased data literacy to become data-informed. No support from the C-suite sends the clear message that it is not required once the senior managers reach that stage because it’s all but another fad. People see themselves as very rational in their decision-making but in fact, we take decisions on more emotional level. The data and the facts are useful but only when we have to defend and justify the decisions we took based on emotions. So the resistance is understandable but taking decisions on a pure data-driven basis is not desirable either. In 2016, the Indonesian government offered a bounty to fight against the proliferation of rats, $1.50 per rat. 40% of the population lives on less than $2 a day. This may have worked for a while but then people starting running rat farms to ensure bigger and more regular earnings. The decision was data-driven in my view but people gamed it and this backfired. Data-informed is in my view the ideal compromise of the experience and a data-based approach. The C-suite and the senior managers should find this prospect more palatable.