Portfolio Oversight With eazyBI
•Download as PPTX, PDF•
2 likes•479 views
How to leverage eazyBI for combined planning and finance reporting across the Tempo product suite. - Kris Siwiec, Lead Consultant - New Verve Consulting, United Kingdom
Recommended
Recommended
More Related Content
What's hot
What's hot (18)
Similar to Portfolio Oversight With eazyBI
Similar to Portfolio Oversight With eazyBI (20)
More from eazyBI
More from eazyBI (15)
Recently uploaded
Recently uploaded (20)
Portfolio Oversight With eazyBI
- 1. Portfolio Oversight with eazyBI Kris Siwiec • Lead Consultant • New Verve Consulting • new_verve kris@newverveconsulting.com
- 3. The who Who are these people?
- 6. The what What is this all about?
- 7. Integration is the path to riches and peace. E C PRESCOTT
- 8. Why eazyBI?
- 9. Love We love: the tool, the community, the ecosystem… Why eazyBI?
- 10. Love Freedom Why eazyBI? For the tool, the community, the ecosystem. To structure, query and present data.
- 11. Why eazyBI? Freedom IntegrationLove To structure, query and present data. Bridging gaps in functionality and understanding. For the tool, the community, the ecosystem.
- 12. Why NOT Tempo?
- 13. “He that breaks a thing to find out what it is has left the path of wisdom” Tempo “tantrums” Rigidness Built-in reports & gadgets not designed for easy customization • Pre-defined views of data • Limited number of - presets - units - scales
- 14. “You shall not pass”Tempo “tantrums” Obstructiveness Built-in reports & gadgets respect user permissions • Jira permissions - Browse Projects - Filter Owner - … • Tempo permissions - Administrators - Team Administrators - Account Administrators - Rate Administrators - Folio Owner
- 15. “A wizard is never late…nor is he early” Tempo “tantrums” Laziness Built-in reports & gadgets computed on- the-fly
- 16. “My preciousss…”Tempo “tantrums” Possessiveness • Built-in reports not available through dashboard gadgets • Limited choice of built-in gadgets • Gadgets respect user permissions i.e. not suitable for unprivileged users (e.g. JSD customers)
- 17. • Rigidness • Obstructivenes s • Laziness • Possessivenes s • Love • Freedom • Integration = riches & peace WHY EAZYBI WHY NOT TEMPO
- 18. Agenda The eazy
- 19. The eazy Getting our hands on the data
- 20. Database API Different points of view Bottom-up view of data Top-down view of data
- 22. select portfolio_name, expired, actual, account_id, account_key, account_name, rate_table_id, rate_table_name, cost_name, id, category, workload_scheme, team_id, team_name, team_role_id, role_id, role_name, start_date_p, end_date_p, amount from ( select concat('ALOC', aloc.`ID`) id, fo.`NAME` portfolio_name, if(fo.`END_DATE`>=CURDATE(),'OPEN','EXPIRED') expired, if(po.`ACTUAL`=1,'Actual','Planned') actual, # filter.`VALUE` query, # only used with AO_2D3BEA_FILTER and AO_2D3BEA_USER_FILTER # filter.`reqcontent` query, # returns the full JQL filter query substring_index(substring_index(filter.`reqcontent`, ''')', 1), '(''', -1) account_key_jql, # extracts the account key from JQL query # po.`ACCOUNT_KEY` account_key, # comes up NULL #account.`ID` account_id, fo.`WORKLOAD_SCHEME_ID` workload_scheme, fo.`RATE_TABLE_ID` rate_table_id, po.`TEAM_ROLE_ID` team_role_id, # WARN: does not line up with role_id below user.`ROLE_ID` role_id, user.`ROLE_NAME` role_name, # for comparison with ID user.`TEAM_ID` team_id, user.`TEAM_NAME` team_name, # for comparison with ID po.`CATEGORY` category, # 'Salaries' type, # optional constant po.`NAME` cost_name, wage.`WAGE`*aloc.`PERCENTAGE`/100*8*(5 * (DATEDIFF(aloc.`END_DATE`, aloc.`START_DATE`) DIV 7) + MID('0123455501234445012333450122234501101234000123450', 7 * WEEKDAY(aloc.`START_DATE`) + WEEKDAY(aloc.`END_DATE`) + 1, 1)) amount, # needs reviewing aloc.`START_DATE` start_date, aloc.`START_DATE` start_date_p, aloc.`END_DATE` end_date_p from AO_2D3BEA_POSITION as po join AO_2D3BEA_FOLIO as fo on po.`FOLIO_ID`= fo.`ID` join AO_2D3BEA_ALLOCATION as aloc on aloc.`POSITION_ID`=po.`ID` join AO_2D3BEA_WAGE as wage on wage.`POSITION_ID`=po.`ID` join AO_2D3BEA_USER_INDEX as user on user.`USER_KEY`=po.`MEMBER` # join AO_2D3BEA_USER_FILTER as filter on filter.`ID`=fo.`SAVED_FILTER_ID` # both AO_2D3BEA_FILTER and AO_2D3BEA_USER_FILTER are useless join searchrequest as filter on filter.`ID`=fo.`SAVED_FILTER_ID` ) t1 join ( select * from ( select ID account_id, NAME account_name, `KEY` account_key from AO_C3C6E8_ACCOUNT_V1 ) t3 join ( select NAME rate_table_name, SCOPE_ID scope_id, SCOPE_TYPE scope_type, PARENT_ID parent_id from AO_C3C6E8_RATE_TABLE where SCOPE_TYPE='ACCOUNT' ) t4 on t3.`account_id`=t4.`scope_id` ) t2 #on t1.`account_key_jql`=t2.`account_key` and t1.`rate_table_id`=t2.`rate_table_id` on t1.`account_key_jql`=t2.`account_key` and t1.`rate_table_id`=t2.`parent_id` # cannot join on rate_table_id as we need the parent rate table
- 23. • indirect data access • high access time • low access cost • balance of granularity and control • direct data access • low access time • high access cost • maximum granularity and control DATABASE API
- 24. Point taken
- 26. The not-so-eazy Getting our hands on MORE data
- 27. Request timeouts WHEN MORE DATA = BIGGER DATA Request proxying • Response time • Response size • Page size • Cache response • Format/paginate • Publish response
- 28. 1h → 1m → 1s / account Data processing time reduction on a 500-account portfolio
- 29. API coverage WHEN MORE DATA = BETTER DATA API stability • Not all data officially exposed • Sometimes in the response you get more than you bargained on • Public endpoints often experimental • Private endpoints here be dragons
- 30. Agenda Wee demo
- 31. Wee demo How does this all fit together?
- 32. Data processing workflow Data cache Source application Data source Data proxy • User properties • Active Objects • AWS S3 • …Database • REST API data connector in eazyBI • Pagination • Authentication • Public endpoints • Private endpoints • SQL queries • …Screen scrapes • Microservice • Scripted service / CRON job • Custom endpoint
- 34. Agenda Wrap-up
- 35. Wrap-up DO • Proxy your data • Cache your data • Use public endpoints DON'T • Presume performance • Use DB queries • Use private endpoints DOs & DON'Ts
- 36. Questions? Kris Siwiec • Lead Consultant • New Verve Consulting • new_verve kris@newverveconsulting.com
- 37. Cheers! Kris Siwiec • Lead Consultant • New Verve Consulting • new_verve kris@newverveconsulting.com
Editor's Notes
- This is the boring slide. Here’s New Verve picking up a recent business award nomination, with our glamorous founder to your left. We’re an Atlassian solution partner and a micro-business, proudly embracing the start-up culture. We like a personal approach and like to think that small is powerful and agile. Being a start-up, in practice we don’t really have fixed roles: we work collectively and our roles emerge naturally from our experience. We’re going through an exciting growth period, with a couple more team members already having joined since this picture was taken.
- Now that that’s out of the way, let’s talk about why we’re here.
- A brainy quote to set us off. Rather than showing off swanky reports, this talk will be about the technical challenges of implementing eazyBI reports as part of a wider ecosystem. Integration carries unique logistical considerations that are worth exploring. In this scenario, we’re integrating with the Tempo product suite, and we’re going to look at why we could or should make particular architectural decisions.
- 3 why use eazyBI reports? * love <3 (we’re besties) * freedom! (to structure & query data) * bridging gaps * in functionality (migration) * in understanding (transition)
- 3 why use eazyBI reports? * love <3 (we’re besties) * freedom! (to structure & query data) * bridging gaps * in functionality (migration) * in understanding (transition) We think it’s a real gift and we want to give back!
- 3 why use eazyBI reports? * love <3 (we’re besties) * freedom! (to structure & query data) * bridging gaps * in functionality (migration) * in understanding (transition)
- 3 why use eazyBI reports? * love <3 (we’re besties) * freedom! (to structure & query data) * bridging gaps * in functionality (migration) * in understanding (transition) We feel that integration is a very interesting use case for eazyBI and a healthy attitude to boot. You don’t build a wall between you & Mexico, or try and leave the continent you’re part of: you build a bridge across the border.
- Short description of Tempo: Tempo is a cloud-first software company that helps teams at more than 10,000 companies—SMBs and large-scale enterprises—collaborate, plan and schedule resources, manage budgets, and track time directly from their daily workflow. Longer description of Tempo: Tempo products are designed to enhance the efficiency of Atlassian's JIRA helping teams and businesses collaborate, plan, budget, track, and work smarter seamlessly in one central software platform. More than 10,000 customers in over 100 countries trust Tempo to be successful. Our customers include small startups to large-scale distributed enterprises such as NASA, Amazon, Paypal, Toshiba, Disney, BMW, and Princeton University. Tempo is one of the largest, award-winning, and bestselling add-on vendors in the Atlassian Ecosystem, and has an established network of more than 100 partners worldwide. Don’t get us wrong, we’re not knocking Tempo at all! We’re Tempo partners as well, after all. Tempo produces an excellent product suite that we think is by itself best-suited for manager-level oversight, rather than customer-level reporting. Here’s why…
- 4 why not use Tempo built-in reports? * pre-defined view of data * permission considerations (try exposing an Account Budget Burn-up Chart or an Account Time and Expenses Report to unprivileged customers) * performance challenges with large amounts of data (try running the Steering Committee Report over a portfolio of 450 projects) * embedding options We’ve called these “Tempo tantrums” – they’re not really tantrums, but it has a nice ring to it. They are based on some quotes from everyone’s favorite wizard. Dumbledore applied to be on the slides, but he wasn’t Jira-certified, so.
- 4 why not use Tempo built-in reports? * pre-defined view of data * permission considerations (try exposing an Account Budget Burn-up Chart or an Account Time and Expenses Report to unprivileged customers) * performance challenges with large amounts of data (try running the Steering Committee Report over a portfolio of 450 projects) * embedding options As a Tempo user, you’re interacting with Jira as a Jira user.
- 4 why not use Tempo built-in reports? * pre-defined view of data * permission considerations (try exposing an Account Budget Burn-up Chart or an Account Time and Expenses Report to unprivileged customers) * performance challenges with large amounts of data (try running the Steering Committee Report over a portfolio of 450 projects) * embedding options
- 4 why not use Tempo built-in reports? * pre-defined view of data * permission considerations (try exposing an Account Budget Burn-up Chart or an Account Time and Expenses Report to unprivileged customers) * performance challenges with large amounts of data (try running the Steering Committee Report over a portfolio of 450 projects) * embedding options
- So, to summarize, we have hopefully made a very strong case ;-) for using eazyBI to deliver planning/financial reports above & beyond what Tempo comes stock with…
- Fortunately, Tempo does expose its data via Active Objects database storage as well as REST API calls: now we will discuss which of these options is in our opinion better-suited for everyday integration with eazyBI.
- 5 getting your hands on the data * database query import approach * pros & CONS + direct data access + maximum granularity & control - need to cross-reference data from multiple database tables - need to reverse-engineer all calculations done by Tempo (e.g. costs accrued) * REST API import approach * PROS & cons + indirect data access (various measures precomputed upfront by the API) + good balance of granularity & control - indirect data access (must obey API limitations) - need to pre-process data prior to publishing eazyBI allows us to use various data source connectors: among these are a SQL query connector and a REST call connector. Let consider the pros & cons of each.
- 5 getting your hands on the data * database query import approach * pros & CONS + direct data access + maximum granularity & control - need to cross-reference data from multiple database tables - need to reverse-engineer all calculations done by Tempo (e.g. costs accrued) * REST API import approach * PROS & cons + indirect data access (various measures precomputed upfront by the API) + good balance of granularity & control - indirect data access (must obey API limitations) - need to pre-process data prior to publishing The DBA is naïve & idealistic: he looks up to the sky and hopes for the best. The hacker looks down from the sky: she’s already there and she knows what she wants.
- Theory – a simple query will suffice, right!? It turns out that the more data you would like to pull in, the harder you have to look, especially if you want to cross-reference various data points. Imagine you’re trying to cross-reference some Tempo folios with their parent portfolios, related accounts, team allocations, issue worklogs…
- Reality – by the time we’re finished with the query, we need to be a SQL grand-master, we’re at least middle-aged already, and this is just the beginning… There is a lot of SQL in that query: you can’t see it, because it was never meant to see the light of day. The SQL was crap, BTW: both the movie and the code. There’s a key differentiator between DB and REST data access, and that is the technical debt/cost carried by the data access operation. Let’s consider this in more detail.
- Database Pros: * direct data access * maximum granularity & control Cons: * need to cross-reference data from multiple database tables * need to reverse-engineer all calculations done by Tempo (e.g. costs accrued) API Pros: * indirect data access (various measures precomputed upfront by the API) * good balance of granularity & control Cons * indirect data access (must obey API limitations) * need to pre-process data prior to publishing (to avoid timeouts)
- 5 getting your hands on the data * database query import approach * pros & CONS + direct data access + maximum granularity & control - need to cross-reference data from multiple database tables - need to reverse-engineer all calculations done by Tempo (e.g. costs accrued) * REST API import approach * PROS & cons + indirect data access (various measures precomputed upfront by the API) + good balance of granularity & control - indirect data access (must obey API limitations) - need to pre-process data prior to publishing
- 6 getting our hands on MORE data * public API endpoints (experimental) * private API endpoints (enter at your own risk)
- 6 getting our hands on MORE data * public API endpoints (experimental) * private API endpoints (enter at your own risk)
- Request timeouts – when the API call takes too long or doesn’t return * Too much pre-processing – weak point of native APIs, linked to below * Too much data in response – some calls return more than you’d expect! * Response size too large * Page size too large Direct vs proxied calls
- A naïve implementation importing data directly from public endpoints can take upwards of 1h/account, depending on the combination of data required A slightly smarter implementation which pre-processes some data upfront can take 1m/account An optimized implementation which pre-processes and caches all data at the proxy can be as efficient as 1s/account
- 6 getting our hands on MORE data * public API endpoints (experimental) * private API endpoints (enter at your own risk)
- Emphasis on wee
- 6 getting our hands on MORE data * public API endpoints (experimental) * private API endpoints (enter at your own risk)
- Here are the basic building blocks for our data processing workflow. There are many options for choosing your data sources: it may be a combination of any number of data providers, from official endpoints to screen scrapes… A convenient option for implementing a quick & dirty microservice is ScriptRunner for Jira. Data can be cached anywhere convenient: from basic user properties to remote data stores in S3.
- Here’s an example anonymized snapshot of a live eazyBI report pulling in a combined planning/financial dataset based on a cube of multiple Tempo sources.
- We’ve put private endpoints under DON’Ts, as they’re a health & safety hazard, so don’t use private endpoints…but really, DO! :-) Just be aware of the risks…