1
© 2021  InfluxData Inc. All Rights Reserved.
© 2021  InfluxData Inc. All Rights Reserved.
ML-Based Data-Driven Software
Development with InfluxDB 2.0
Hari Prasad Sudharshan
Ezhilmathy Shanmugam Veeramani
2
© 2021  InfluxData Inc. All Rights Reserved.
© 2021  InfluxData Inc. All Rights Reserved.
Every city has its own problems to solve; ours had its own.
3
© 2021  InfluxData Inc. All Rights Reserved.
© 2021  InfluxData Inc. All Rights Reserved.
Everything (development) has a cost.
4
© 2021  InfluxData Inc. All Rights Reserved.
© 2021  InfluxData Inc. All Rights Reserved.
City Problem- 1:4 times more rain (inches) than LA. On average, 1 inch more than Seattle annually.
: 40 inches in 60 days is the problem.
5
© 2021  InfluxData Inc. All Rights Reserved.
© 2021  InfluxData Inc. All Rights Reserved.
6
© 2021  InfluxData Inc. All Rights Reserved.
© 2021  InfluxData Inc. All Rights Reserved.
Motivation : As a beneficiary of the Development in the city (home). I
was part of the problem too. I felt compelled to give back.
• Constants -- Lake capacity, expected Water from Telugu Ganga Canal.
Variables -- Daily water level, daily water consumption, evaporation
co-efficient, surface temperature, rainfall data, water received from Telugu
Ganga Canal.
Started with Excel sheet, moved to SQL. I was on the lookout for a DB that
would make my job easier. Thanks to Google algorithms & SEO of InfluxDB, I
was frequently seeing InfluxDB.
• Requirement: I had to map every variable with respect to time, Influx made
it absolutely simple.
• Status: The open data on variables was blocked in 2019 as we approached
zero day. Collecting data since 1960s, working on anomaly detection, early
detection of drought with available data.
7
© 2021  InfluxData Inc. All Rights Reserved.
© 2021  InfluxData Inc. All Rights Reserved.
Pitchdarkdata : An attempt to create visibility to our blind spots.
Who is it for?
Anyone who uses Gerrit: Free, web-based team code collaboration tool developed by Google.
Android OSS development is a notable project on Gerrit platform.
What’s it for?
Provide Engineering analytics and deeper SDLC insights (blind spots) that get buried within
Jira (business cases). Jira/Project Management insights are important but they have a top
view foot above.
How is it done today? What are the limits of current practice?
Emails, gut feeling, Jira data is what’s driving most Org today. There are tools for Engineering
Analytics, but they are expensive & not tailor-made.
Most organizations do not know what is happening under the hood (blind spot).
What are we trying to do?
Democratize data in favor of those who really get the work done. Developers are the heavy
lifters.
What difference will it make ?
Give recognition where it is due, know your bottleneck with data not with emails (biased), PPT
(biased). Provide accurate actionable feedback to developers, Scrum Masters towards
Organizational goals. A great Engineer need not be left behind because of lack of
communication/marketing skills. The system will complement the lack of insight (blind spot).
8
© 2021  InfluxData Inc. All Rights Reserved.
© 2021  InfluxData Inc. All Rights Reserved.
Code
Gerrit (Code
collaborator)
Jenkins-Maven
(Build, Unit
Test)
SonarQube
(Static
analyser)
Master
Pitchdarkdata Design: InfluxDB + Flux queries + Custom API
to scrape data from all sources to provide Engineering Intelligence
InfluxDB +
Custom API
Visualize
Infer
Take action
9
© 2021  InfluxData Inc. All Rights Reserved.
© 2021  InfluxData Inc. All Rights Reserved.
How do developers commit w.r.t Time. Inferences
Merged
Commits
New
Commits
New Feature
Bug Fix
Enhancement
Build Passes
(UT, Static
Analysis)
Code Reviewed
Functionality
Available
10
© 2021  InfluxData Inc. All Rights Reserved.
© 2021  InfluxData Inc. All Rights Reserved.
1 Month Window to infer patterns
New (today)
Backlog (New +
Old)
Merge
11
© 2021  InfluxData Inc. All Rights Reserved.
© 2021  InfluxData Inc. All Rights Reserved.
Growth Trajectory of Repo (LOC)
Linear Projection : Half a million more lines to manage in 10 months
New features
Code Optimization
Bug fix
Obsolete features
Dead code (unused)
Modularize
(common code)
Identify duplication
12
© 2021  InfluxData Inc. All Rights Reserved.
© 2021  InfluxData Inc. All Rights Reserved.
Velocity of SDLC, Feedback KPI, Response Time.
Analyse your key bottleneck.
Delight
customers
Quick
& quality
feedback
Develop
13
© 2021  InfluxData Inc. All Rights Reserved.
© 2021  InfluxData Inc. All Rights Reserved.
References
Code Sources :
1. Scraper code
2. Flux queries + Flux logic
3. Dashboard screenshots
3. Export Templates
GitHub : https:/
/github.com/pitchdarkdata/InfluxDays2021_Demo
Water sources:
https:/
/chennaimetrowater.tn.gov.in/watersupplysystem.html
http:/
/123.63.203.150/public/lake.htm
Picture credits:
https:/
/www.dtnext.in/News/City/2016/02/02002439/PWD-plans-link-canals-between-major-water-bodies-serving.vpf
https:/
/www.weather-forecast.com/locations/Madras/photos/5621
https:/
/theculturetrip.com/asia/india/articles/7-things-to-do-on-a-rainy-day-in-chennai/
https:/
/www.viator.com/Chennai-attractions/Marina-Beach/overview/d4624-a14736
https:/
/mobile.twitter.com/i/web/status/1427284727073890305
Chennai SaaS Capital of India :
https:/
/www.crayondata.com/heres-why-chennai-is-the-saas-capital-of-india/
https:/
/timesofindia.indiatimes.com/business/india-business/a-silent-saas-revolution-is-brewing-in-chennai/articleshow/67583586.cms
https:/
/www.deccanchronicle.com/nation/politics/111018/chennai-is-saas-capital-of-india-edappadi-k-palaniswami.html
https:/
/www.forbes.com/sites/aayushipratap/2021/09/22/inside-freshworks-rise-from-india-enterprise-tech-startup-to-10-billion-ipo/?sh=6f15afa235b1
14
© 2021  InfluxData Inc. All Rights Reserved.
Questions?

Hari-Prasad Sudharshan [Fujitsu Network Communications] | ML-Based Data-Driven Software Development with InfluxDB 2.0 | InfluxDays NA 2021

  • 1.
    1 © 2021  InfluxDataInc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. ML-Based Data-Driven Software Development with InfluxDB 2.0 Hari Prasad Sudharshan Ezhilmathy Shanmugam Veeramani
  • 2.
    2 © 2021  InfluxDataInc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Every city has its own problems to solve; ours had its own.
  • 3.
    3 © 2021  InfluxDataInc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Everything (development) has a cost.
  • 4.
    4 © 2021  InfluxDataInc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. City Problem- 1:4 times more rain (inches) than LA. On average, 1 inch more than Seattle annually. : 40 inches in 60 days is the problem.
  • 5.
    5 © 2021  InfluxDataInc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved.
  • 6.
    6 © 2021  InfluxDataInc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Motivation : As a beneficiary of the Development in the city (home). I was part of the problem too. I felt compelled to give back. • Constants -- Lake capacity, expected Water from Telugu Ganga Canal. Variables -- Daily water level, daily water consumption, evaporation co-efficient, surface temperature, rainfall data, water received from Telugu Ganga Canal. Started with Excel sheet, moved to SQL. I was on the lookout for a DB that would make my job easier. Thanks to Google algorithms & SEO of InfluxDB, I was frequently seeing InfluxDB. • Requirement: I had to map every variable with respect to time, Influx made it absolutely simple. • Status: The open data on variables was blocked in 2019 as we approached zero day. Collecting data since 1960s, working on anomaly detection, early detection of drought with available data.
  • 7.
    7 © 2021  InfluxDataInc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Pitchdarkdata : An attempt to create visibility to our blind spots. Who is it for? Anyone who uses Gerrit: Free, web-based team code collaboration tool developed by Google. Android OSS development is a notable project on Gerrit platform. What’s it for? Provide Engineering analytics and deeper SDLC insights (blind spots) that get buried within Jira (business cases). Jira/Project Management insights are important but they have a top view foot above. How is it done today? What are the limits of current practice? Emails, gut feeling, Jira data is what’s driving most Org today. There are tools for Engineering Analytics, but they are expensive & not tailor-made. Most organizations do not know what is happening under the hood (blind spot). What are we trying to do? Democratize data in favor of those who really get the work done. Developers are the heavy lifters. What difference will it make ? Give recognition where it is due, know your bottleneck with data not with emails (biased), PPT (biased). Provide accurate actionable feedback to developers, Scrum Masters towards Organizational goals. A great Engineer need not be left behind because of lack of communication/marketing skills. The system will complement the lack of insight (blind spot).
  • 8.
    8 © 2021  InfluxDataInc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Code Gerrit (Code collaborator) Jenkins-Maven (Build, Unit Test) SonarQube (Static analyser) Master Pitchdarkdata Design: InfluxDB + Flux queries + Custom API to scrape data from all sources to provide Engineering Intelligence InfluxDB + Custom API Visualize Infer Take action
  • 9.
    9 © 2021  InfluxDataInc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. How do developers commit w.r.t Time. Inferences Merged Commits New Commits New Feature Bug Fix Enhancement Build Passes (UT, Static Analysis) Code Reviewed Functionality Available
  • 10.
    10 © 2021  InfluxDataInc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. 1 Month Window to infer patterns New (today) Backlog (New + Old) Merge
  • 11.
    11 © 2021  InfluxDataInc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Growth Trajectory of Repo (LOC) Linear Projection : Half a million more lines to manage in 10 months New features Code Optimization Bug fix Obsolete features Dead code (unused) Modularize (common code) Identify duplication
  • 12.
    12 © 2021  InfluxDataInc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Velocity of SDLC, Feedback KPI, Response Time. Analyse your key bottleneck. Delight customers Quick & quality feedback Develop
  • 13.
    13 © 2021  InfluxDataInc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. References Code Sources : 1. Scraper code 2. Flux queries + Flux logic 3. Dashboard screenshots 3. Export Templates GitHub : https:/ /github.com/pitchdarkdata/InfluxDays2021_Demo Water sources: https:/ /chennaimetrowater.tn.gov.in/watersupplysystem.html http:/ /123.63.203.150/public/lake.htm Picture credits: https:/ /www.dtnext.in/News/City/2016/02/02002439/PWD-plans-link-canals-between-major-water-bodies-serving.vpf https:/ /www.weather-forecast.com/locations/Madras/photos/5621 https:/ /theculturetrip.com/asia/india/articles/7-things-to-do-on-a-rainy-day-in-chennai/ https:/ /www.viator.com/Chennai-attractions/Marina-Beach/overview/d4624-a14736 https:/ /mobile.twitter.com/i/web/status/1427284727073890305 Chennai SaaS Capital of India : https:/ /www.crayondata.com/heres-why-chennai-is-the-saas-capital-of-india/ https:/ /timesofindia.indiatimes.com/business/india-business/a-silent-saas-revolution-is-brewing-in-chennai/articleshow/67583586.cms https:/ /www.deccanchronicle.com/nation/politics/111018/chennai-is-saas-capital-of-india-edappadi-k-palaniswami.html https:/ /www.forbes.com/sites/aayushipratap/2021/09/22/inside-freshworks-rise-from-india-enterprise-tech-startup-to-10-billion-ipo/?sh=6f15afa235b1
  • 14.
    14 © 2021  InfluxDataInc. All Rights Reserved. Questions?