Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
What to Upload to SlideShare
Next

0

Share

PAC 2020 Santorin - Gopalkrishnan Yadav

Azure Multiple Pipeline Performance Monitor

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all
  • Be the first to like this

PAC 2020 Santorin - Gopalkrishnan Yadav

  1. 1. PERFORMANCE IS NOT A MYTH P E R F O R M A N C E A D V I S O R Y C O U N C I L SANTORINI GREECE FEBRUARY 26 - 27 2020 Azure Multiple Pipeline Performance Monitor Gopalkrishnan Yadav
  2. 2. P E R F O R M A N C E A D V I S O R Y C O U N C I L byP E R F O R M A N C E A D V I S O R Y C O U N C I L Service Current Team Size: 550+ Customer Base: 50+ active clients Global presence in 15+ countries Multiple Engagement and delivery models Capability across technology domains :Legacy, ERP, Web, Cloud, Big Data, Mobile Alliance and Expertise Portfolio of Testing Services across business domains Ranked #1 by Ovum, ‘leading position’ by Nelson Hall, IDC Testing Leader 2015 by Gartner • Performance Testing and Engineering Methodology • Performance 360 Framework • PerfNEXT- Perftrack, Perf Analytics, Log Analyzer, LRAC and SPARK Utilities Innovation Leadership • End to End Performance Testing (Baseline/Load/Stress/Scalability/Endurance) • Performance Engineering • Setting up of Performance testing practice • Performance Testing CoE setup • Managed Performance Test Center (MPTC) • PTaaS • Performance Consulting • QBP & Maturity Assessment • Transformation Consulting • WAN, Mobile and Cloud Performance Testing • Performance Lab setup • HPE LoadRunner provisioning Service Offerings Manufacturing RetailTelecomInsurance Media & EntertainmentAviationBanking Energy & Util.
  3. 3. P E R F O R M A N C E A D V I S O R Y C O U N C I L byP E R F O R M A N C E A D V I S O R Y C O U N C I L NFT Hub – IP Accelerator Key Features Automatic executions with CI/CD facilities for regression projects Process driven approach standardized across the enterprise using ‘Perf Track’ as orchestration engine Interwoven set of tools and accelerators providing complete life cycle support for performance projects execution ‘WLM’ tool for realistic work load modeling and test scenario generation using ‘Log Analyzer’ Reduced scripting time and increased quality using ‘LRAC’ for multiple scripts. Automated script validation. ‘Perf Analytics’ providing deep insights into the past test runs and at the same time providing predictive views for future runs through machine learning “Script less Automation” “Performance Test Management” “Realistic Load Testing” “CI/CD Ready” “Analytics” “End to End Platform” “Electronic Documents” “Validation” Automation Benefits/Savings Script Design Effort Management Effort Report Prep Effort Enables “Realistic” Load Testing Helps in Requirement Gathering
  4. 4. 4© 2019 Capgemini – Internal use only. All rights reserved.Cloud with AWS | May 2019 OneShare OneShare is Sogeti’s Cloud Platform solution consisting of Self-Service portal, Templates and Services for cloud environment provisioning and management.  Helps speed up the provisioning of Dev and Test environments with Self/Managed service and resource templates  Easy provision of environments in Azure, AWS, Google Cloud Platform & IBM Cloud and self manage by Dev and Test teams  Control Cloud usage costs through usage monitoring and resource scheduling  Standardize on Environments throughout the enterprise in a unified, robust and repeatable way  Multi Cloud provisioning, Speed up provisioning of Dev/Test environments, Standardized Environments; Start, Stop and Schedule VMs & Environments  Gain insights on environments and subscription costs, Control cloud usage costs, ‘Pay as you use’ pricing model, , VM Utilization Report  Microfocus Test tooling and Managed Services  Dev/Test Template Management Services, Infra Management, Role Based access and Quota based resource creation  Azure DevOps Integration for CI/CD configuration and execution  Solution leads: Santanu De Overview Benefits References More information  Fiskars, Posti Group, Boots, Bob Evans, Primark, SignPost, Neste, Outokumpu. Velmet,  Smiths
  5. 5. P E R F O R M A N C E A D V I S O R Y C O U N C I L byP E R F O R M A N C E A D V I S O R Y C O U N C I L Overview • The Azure performance monitoring cannot be done using traditional approach of record replay model • The standard performance testing tools like JMeter, LoadRunner does not support the log analytics • The Best recommended model to assess the performance of Pipelines is using Azure Log analytics feature • The near real time data is monitored and in advance configuration alert mechanism can be implemented
  6. 6. P E R F O R M A N C E A D V I S O R Y C O U N C I L byP E R F O R M A N C E A D V I S O R Y C O U N C I L ADF Architecture ADFv2-Ingest Date Lake Store Create QS tables (Landed/Processed) Flat Files Data bases UDL BDL Azure Analysis ServicesPDS-SQLDW 1 2 3 4 5 6 ADF Performance monitoring Curate UDL data to BDL Data bricks Automation testing Power BI ADF code commit to VSTS Data Bricks– VSTS Git Integration ADF – Git Integration
  7. 7. P E R F O R M A N C E A D V I S O R Y C O U N C I L byP E R F O R M A N C E A D V I S O R Y C O U N C I L Pipeline Execution
  8. 8. P E R F O R M A N C E A D V I S O R Y C O U N C I L byP E R F O R M A N C E A D V I S O R Y C O U N C I L Performance Monitoring Strategy
  9. 9. P E R F O R M A N C E A D V I S O R Y C O U N C I L byP E R F O R M A N C E A D V I S O R Y C O U N C I L Performance Monitoring Approach
  10. 10. P E R F O R M A N C E A D V I S O R Y C O U N C I L byP E R F O R M A N C E A D V I S O R Y C O U N C I L Performance Metrics 01 02 03 05 06 07 Successful Pipeline count Failed Pipeline count Output data written Vs output data Read Activity Duration Successful Activity count Failed Activity count 04 Integration runtime CPU utilization Integration Runtime available memory08
  11. 11. P E R F O R M A N C E A D V I S O R Y C O U N C I L byP E R F O R M A N C E A D V I S O R Y C O U N C I L Workspace Configuration In the Azure portal, click All services. In the list of resources, type Log Analytics.. Select Log Analytics workspaces. •Click Add, and then select choices for the following items: Provide a name for the new Log Analytics workspace, such as DefaultLAWorkspace. •Select a Subscription to link to by selecting from the drop-down list if the default selected is not appropriate. •For Resource Group, choose to use an existing resource group already setup or create a new one. •Select an available Location. •After providing the required information on the Log Analytics Workspace pane, click OK
  12. 12. P E R F O R M A N C E A D V I S O R Y C O U N C I L byP E R F O R M A N C E A D V I S O R Y C O U N C I L Diagnostic Log Enablement Settings • In the portal, navigate to Azure Monitor and click on Diagnostic Settings • Optionally filter the list by resource group or resource type, then click on the resource for which you would like to set a diagnostic setting. • If no settings exist on the resource you have selected, you are prompted to create a setting. Click "Turn on diagnostics." • If there are existing settings on the resource, you will see a list of settings already configured on this resource. Click "Add diagnostic setting."
  13. 13. P E R F O R M A N C E A D V I S O R Y C O U N C I L byP E R F O R M A N C E A D V I S O R Y C O U N C I L • Give your setting a name and check the box for Send to Log Analytics, then select a Log Analytics workspace • Click Save Diagnostic Log Enablement Settings
  14. 14. P E R F O R M A N C E A D V I S O R Y C O U N C I L byP E R F O R M A N C E A D V I S O R Y C O U N C I L Configure Pipeline Monitoring Settings Microsoft Azure use Kusto Language to develop the query. In the monitoring section there are many default KPI is available which is very easy to configure. Following steps needs to be followed to configure the dashboard • Login to the Microsoft Azure Portal • Click on Monitor tab • Click on Explore Metrics • Select the Resource group • Select the metrics and choose the right aggregation • Click on Pin to dashboard
  15. 15. P E R F O R M A N C E A D V I S O R Y C O U N C I L byP E R F O R M A N C E A D V I S O R Y C O U N C I L Azure Analytics • SAMPLE QUERY METRICS • Output data written Vs Output data read Vs Pipeline Name • Activity start time Vs Activity End Time Vs Output copy duration • Activity total duration Vs Succeeded • Output data written Vs Output data read Vs Pipeline Name: • AzureDiagnostics • | where TimeGenerated > ago(7d) • | project Output_dataWritten_d, activityName_s , Output_dataRead_d , pipelineName_s • Activity start time Vs Activity End Time Vs Output copy duration • AzureDiagnostics • | where ResourceGroup contains "80011" • | project start_t , end_t, pipelineName_s , activityName_s , Output_copyDuration_d • | render timechart
  16. 16. P E R F O R M A N C E A D V I S O R Y C O U N C I L byP E R F O R M A N C E A D V I S O R Y C O U N C I L • Activity total duration Vs Succeeded AzureDiagnostics | project activityName_s , start_t , end_t , status_s | extend duration = datetime_diff('second',end_t, start_t) | extend duration = duration/60 | where status_s != "Succeeded" and activityName_s != "" and durationm > 5 Azure Analytics
  17. 17. P E R F O R M A N C E A D V I S O R Y C O U N C I L byP E R F O R M A N C E A D V I S O R Y C O U N C I L Sample Report
  18. 18. P E R F O R M A N C E A D V I S O R Y C O U N C I L byP E R F O R M A N C E A D V I S O R Y C O U N C I L • Succeeded Pipeline VS Duration • Succeeded Activity VS Duration
  19. 19. P E R F O R M A N C E A D V I S O R Y C O U N C I L byP E R F O R M A N C E A D V I S O R Y C O U N C I L Query to Generate Graph
  20. 20. P E R F O R M A N C E A D V I S O R Y C O U N C I L byP E R F O R M A N C E A D V I S O R Y C O U N C I L Sample Report
  21. 21. P E R F O R M A N C E A D V I S O R Y C O U N C I L byP E R F O R M A N C E A D V I S O R Y C O U N C I L Benefits • Help customer to assess the performance of pipeline jobs • Give confidence on the data processing technique • Stability on the jobs can be achieved • The Microsoft Azure analytics has many feature to be explored for designing the feature. The package comes free with full license entitlement.
  22. 22. PERFORMANCE IS NOT A MYTH P E R F O R M A N C E A D V I S O R Y C O U N C I L SANTORINI GREECE FEBRUARY 26 - 27 2020 Thanks

Azure Multiple Pipeline Performance Monitor

Views

Total views

110

On Slideshare

0

From embeds

0

Number of embeds

0

Actions

Downloads

0

Shares

0

Comments

0

Likes

0

×