Using database statistics and queries, this document discusses how to identify courses that meet criteria for archiving based on the number of new or modified course entities since the last archive. It provides an example SQL query to return the course ID and count of new/modified entities for each course, in order to determine which courses should be archived during the next run of the archival process. Administrative planning topics like archive frequency and impact on users are also covered. Strategic questions are posed around using database statistics to understand online testing capacity and potential database impacts.
So Your Boss Wants You to Performance Test BlackboardSteve Feldman
This document provides guidance for performance testing the Blackboard Learn platform. It discusses setting clear expectations for the testing, including goals, objectives, schedules and metrics. It recommends planning testing scenarios based on functional analysis and log/data mining. Two recommended transaction types are HTTP drivers for load and browser rendering for end-to-end tests. Partial and full payloads as well as arrival rates should be considered. A variety of test types like endurance, adoption and regression are outlined. External performance engineering teams can assist with planning, execution and reporting if needed.
SQL Server 2017 - Adaptive Query Processing and Automatic Query TuningJavier Villegas
The document discusses SQL Server 2017 features including Query Store, Automatic Query Tuning, and Adaptive Query Processing. Query Store captures query execution plans and statistics over time to help troubleshoot performance issues. Automatic Query Tuning identifies and fixes performance regressions by automatically selecting better query plans. Adaptive Query Processing allows the query optimizer to modify the execution plan while a query is running based on actual runtime statistics, leading to more efficient plans.
Harshal Tarambale is an SAP ABAP Lead with over 5.5 years of experience in SAP ABAP development. He has experience in areas like RICEFW, Webdynpro, debugging, and managing development teams. Currently working as an ABAP Lead at Accenture, his previous roles include SSE at Accenture and Consultant at Capgemini. He holds an SAP ABAP certification and ITIL Foundation certification. He is seeking new assignments to further develop his technical and leadership skills.
According to Google, SRE is what you get when you treat operations as if it’s a software problem. In this video, I briefly explain key SRE processes. Video: https://youtu.be/BdFmRJAnB6A
According to Google, SRE is what you get when you treat operations as if it’s a software problem. In this video, I briefly explain what is and isn't toil, how to identify, measure and eliminate them.
Youtube channel here: https://youtu.be/EgpCw15fIK8
Cookbook for Administrating Blackboard LearnSteve Feldman
This document provides a summary of a presentation about administering Learn, Blackboard's learning management system. It discusses how past experiences may not always apply to present situations due to changing data and conditions. Two historical examples are presented: the French and American efforts to build the Panama Canal, and London's 1854 cholera epidemic. Both examples illustrate how data was key to understanding failures and successes. The presentation emphasizes looking at time-series data, outliers, and statistical variability to best understand user experiences and problems. Ignoring data or only looking at averages can lead to inaccurate conclusions.
This document summarizes a three day outage in September that affected an organization's most important asset. The outage was caused by human error when a Unix machine was rebooted. Key lessons learned include the importance of always having monitoring systems on, keeping all stakeholders informed during a crisis, admitting mistakes, and practicing failure scenarios to improve response routines. The problems that led to the outage started long before and highlight the need for organizations to practice responding to failures.
This document introduces a reference architecture from Sun Microsystems and Blackboard for optimizing eLearning environments. It describes a performance maturity model with 5 levels for eLearning solutions and notes that most institutions are currently at the lowest, most reactive level. The reference architecture is designed to help institutions advance to higher, more proactive levels through proven, scalable components that can deliver consistently high performance in a cost-effective way. It includes guidance on optimizing the application, database, storage, and management tiers to meet the demanding requirements of mission-critical eLearning systems.
So Your Boss Wants You to Performance Test BlackboardSteve Feldman
This document provides guidance for performance testing the Blackboard Learn platform. It discusses setting clear expectations for the testing, including goals, objectives, schedules and metrics. It recommends planning testing scenarios based on functional analysis and log/data mining. Two recommended transaction types are HTTP drivers for load and browser rendering for end-to-end tests. Partial and full payloads as well as arrival rates should be considered. A variety of test types like endurance, adoption and regression are outlined. External performance engineering teams can assist with planning, execution and reporting if needed.
SQL Server 2017 - Adaptive Query Processing and Automatic Query TuningJavier Villegas
The document discusses SQL Server 2017 features including Query Store, Automatic Query Tuning, and Adaptive Query Processing. Query Store captures query execution plans and statistics over time to help troubleshoot performance issues. Automatic Query Tuning identifies and fixes performance regressions by automatically selecting better query plans. Adaptive Query Processing allows the query optimizer to modify the execution plan while a query is running based on actual runtime statistics, leading to more efficient plans.
Harshal Tarambale is an SAP ABAP Lead with over 5.5 years of experience in SAP ABAP development. He has experience in areas like RICEFW, Webdynpro, debugging, and managing development teams. Currently working as an ABAP Lead at Accenture, his previous roles include SSE at Accenture and Consultant at Capgemini. He holds an SAP ABAP certification and ITIL Foundation certification. He is seeking new assignments to further develop his technical and leadership skills.
According to Google, SRE is what you get when you treat operations as if it’s a software problem. In this video, I briefly explain key SRE processes. Video: https://youtu.be/BdFmRJAnB6A
According to Google, SRE is what you get when you treat operations as if it’s a software problem. In this video, I briefly explain what is and isn't toil, how to identify, measure and eliminate them.
Youtube channel here: https://youtu.be/EgpCw15fIK8
Cookbook for Administrating Blackboard LearnSteve Feldman
This document provides a summary of a presentation about administering Learn, Blackboard's learning management system. It discusses how past experiences may not always apply to present situations due to changing data and conditions. Two historical examples are presented: the French and American efforts to build the Panama Canal, and London's 1854 cholera epidemic. Both examples illustrate how data was key to understanding failures and successes. The presentation emphasizes looking at time-series data, outliers, and statistical variability to best understand user experiences and problems. Ignoring data or only looking at averages can lead to inaccurate conclusions.
This document summarizes a three day outage in September that affected an organization's most important asset. The outage was caused by human error when a Unix machine was rebooted. Key lessons learned include the importance of always having monitoring systems on, keeping all stakeholders informed during a crisis, admitting mistakes, and practicing failure scenarios to improve response routines. The problems that led to the outage started long before and highlight the need for organizations to practice responding to failures.
This document introduces a reference architecture from Sun Microsystems and Blackboard for optimizing eLearning environments. It describes a performance maturity model with 5 levels for eLearning solutions and notes that most institutions are currently at the lowest, most reactive level. The reference architecture is designed to help institutions advance to higher, more proactive levels through proven, scalable components that can deliver consistently high performance in a cost-effective way. It includes guidance on optimizing the application, database, storage, and management tiers to meet the demanding requirements of mission-critical eLearning systems.
The document discusses the importance of logs and log data analysis. It notes that most people do not properly utilize or manage log data from their systems. Logs can provide valuable insights into trends, system health, capacity planning, security threats and more. The document promotes log centralization and normalization using tools like Logstash and Kibana to facilitate log management and analytics. It provides examples of log sources like Apache access logs, Tomcat logs and challenges readers to setup a basic Logstash pipeline to process and visualize log data.
This document discusses emerging technology challenges and forecasts future trends. It examines the importance of forecasting technology adoption curves and preparing populations for transitions. Examples are given of past technology misses like the Apple Newton and Pluto being demoted from planet status. The document then focuses on distributed caching systems and elastic Java virtual machines that can adjust memory allocation on demand. It argues these technologies allow applications to better handle usage spikes and periods of high demand. Finally, it discusses strategies for cloud computing and third-party analytics and monitoring services.
Scaling Blackboard Learn™ for High Performance and DeliverySteve Feldman
This document discusses scaling Blackboard Learn for high performance and availability. It provides a brief history of Blackboard's technology shift from Perl to Java. It describes the evolution of Blackboard's reference architectures (RefArch) over time to incorporate more features like logging, monitoring, cloud services and virtualization. It defines concepts like performance, scalability and availability, and discusses strategies used by Blackboard to achieve high performance, such as automated provisioning, complete monitoring, application lifecycle management and web optimization services.
PTOn...Finding the Time to Dedicate Individual Projects of Passion and Role Steve Feldman
A culture presentation about how my teams make use of a dedicated time model called PTOn for working on individual projects of passion and/or role on our team. It's similar to Google's 20% model, but a little different.
Database administrators are responsible for maintaining high performance of databases like Oracle, SQL Server, and Red Hat that support enterprise applications like Blackboard. Quest Software's Spotlight tool helps administrators monitor and diagnose performance issues across various database platforms in real-time and historically. It provides visualizations of process flows, metrics, and alerts to identify bottlenecks and root causes of poor performance. Spotlight supports various database environments including Oracle, Oracle RAC, and SQL Server databases.
The document provides an introduction to load testing, including definitions of key terms, reasons for performing load testing, tools used, steps to prepare for load testing such as defining performance objectives and scenarios, and how to conduct load tests through initial configuration, calibration, establishing a baseline, collecting samples/measurements, and optimization.
This document provides an overview of Dell's reference architecture for optimizing Blackboard Academic Suite software deployments. The architecture is designed to support large-scale implementations with up to 30,000 active users through a virtualized and scalable infrastructure. It utilizes Dell PowerEdge servers, EqualLogic storage, and management tools from Quest Software. The reference architecture aims to help customers achieve high performance, scalability, simplified management, and lower TCO. It provides sizing guidelines and best practices for implementing the application, database, and storage tiers in a way that is optimized for Blackboard Academic Suite.
The document discusses how Blackboard sizes its Academic Suite software based on benchmarking. It provides details on the benchmarking methodology, including modeling user behavior, data growth, and performance objectives. The results showed how the software performed under different workload levels on various hardware configurations. The last part discusses using the benchmark results and sizing guide to determine an institution's adoption profile and appropriate hardware configuration based on factors like sessions per hour and page loads.
This document discusses an introduction to software performance engineering (SPE) presented by Steve Feldman, Tim Moore, and Ryan O'Neil from Blackboard's Performance Center of Excellence. It provides an overview of SPE methodology, which takes a systematic, quantitative approach to constructing software to meet performance objectives. The 7-step SPE process involves assessing performance risk, identifying critical use cases, selecting key performance scenarios, establishing performance objectives, constructing performance models, determining software resource requirements, and adding system resource requirements. The presentation also demonstrates applying SPE to develop a sample Blackboard building block called "Santa's Little Helper" for monitoring performance metrics.
Allison Sheridan of the NosillaCast Podcast (hosted at podfeet.com) presented this at Blogworld Expo in October 2010 in Las Vegas. It focused on how to make both your podcast and your blog more accessibile to try and capture some of the 600,000 people worldwide who are visually or hearing impaired.
This document provides numerous tips for improving Mac productivity and customizing the Mac interface. It describes shortcuts for quickly switching between apps and windows, moving and copying files, customizing the toolbar and sidebar, using Spotlight to find files, adding signatures in Preview, tweaking the Dock, accessing more file info, finding menu items and preferences, using accent keys, and various shortcuts using modifier keys like Option and Command. The tips are intended to help users work more efficiently on a Mac.
Bb world 2012 using database statistics to make capacity planning decisions...Geoff Mower
This document discusses using database statistics and metrics to inform capacity planning decisions at Blackboard. It provides examples of the types of data that can be collected from the Blackboard database, such as table sizes, growth rates, and resource usage during testing. Collecting and analyzing this data can help with administrative planning, budgeting, and strategic decisions around supporting more online testing. The document recommends setting up automated data collection and logging to track database metrics over time for capacity planning purposes.
The document provides an overview of SAP BI training. It discusses that SAP stands for Systems Applications and Products in Data Processing and was founded in 1972 in Germany. It is the world's fourth largest software provider and largest provider of ERP software. The training covers topics such as the 3-tier architecture of SAP, data warehousing, ETL, the SAP BI architecture and key components, OLTP vs OLAP, business intelligence definition, and the ASAP methodology for SAP implementations.
The document discusses building a data warehouse in SQL Server. It provides an agenda that covers topics like an overview of data warehousing, data warehouse design, dimension and fact tables, and physical design. It also discusses components of a data warehousing solution like the data warehouse database, ETL processes, and security considerations.
The document outlines a multi-month implementation plan for a BI project with the following key stages:
1) Preparation and Planning in Month 1 involving prioritization, hardware installation, staffing, and software procurement.
2) ETL development from Month 1-3 involving requirement analysis, design, development and testing of the ETL processes.
3) Initial deployment from Month 2-3 setting up the metadata framework and data governance with report reductions.
4) Ongoing development from Month 4-10 involving further report reductions, incremental deployments, building the data library and dashboards. Headcount savings also take effect during this stage.
5) Long term operations starting from Month 11 involving targeting
ADV Slides: Platforming Your Data for Success – Databases, Hadoop, Managed Ha...DATAVERSITY
Thirty years is a long time for a technology foundation to be as active as relational databases. Are their replacements here? In this webinar, we say no.
Databases have not sat around while Hadoop emerged. The Hadoop era generated a ton of interest and confusion, but is it still relevant as organizations are deploying cloud storage like a kid in a candy store? We’ll discuss what platforms to use for what data. This is a critical decision that can dictate two to five times additional work effort if it’s a bad fit.
Drop the herd mentality. In reality, there is no “one size fits all” right now. We need to make our platform decisions amidst this backdrop.
This webinar will distinguish these analytic deployment options and help you platform 2020 and beyond for success.
Develop a Custom Data Solution Architecture with NorthBayAmazon Web Services
Organizations that have vast amounts of data in legacy applications often experience difficulties delivering that data to business unit end-users. Register to learn how Eliza Corporation and Scholastic overcame this challenge by leveraging a Data Lake solution from NorthBay on AWS to optimize data analytics and provide greater visibility. AWS and NorthBay will give you an in-depth overview of how you can use a Data Lake in conjunction with your existing on-premises or cloud-based Data Warehouse. NorthBay helps organizations scale their ETL and data warehousing workloads using Amazon EMR and Amazon Redshift. Join us to learn: • Best practices for using a Data Lake in conjunction with your existing data warehouse • The key aspects of introducing agile and scrum methodologies into an enterprise • The most impactful cost-savings levers that are addressed via a cloud data warehouse migration
Who should attend: Heads of Analytics, Heads of BI, Analytics Managers, BI Teams, Senior Analysts
This document provides tips for optimizing performance in Power BI by focusing on different areas like data sources, the data model, visuals, dashboards, and using trace and log files. Some key recommendations include filtering data early, keeping the data model and queries simple, limiting visual complexity, monitoring resource usage, and leveraging log files to identify specific waits and bottlenecks. An overall approach of focusing on time-based optimization by identifying and addressing the areas contributing most to latency is advocated.
The document discusses the topics that will be covered in the MKTG 555 Marketing Analytics course. It will provide an overview of business intelligence processes and technologies. Students will learn about data warehousing, descriptive and predictive analytics using tools like SQL Server, SAS, and Tableau. The course aims to provide hands-on experience with these technologies. Likely projects include creating pivot tables and charts in Excel using data from SQL Server, building tabular data models in PowerPivot, and developing KPI dashboards in PerformancePoint and SharePoint.
Riyas Mohamed is an IT professional with over 7 years of experience in roles such as Oracle DBA, business analyst, and software tester. He has extensive experience with database administration, installation, backup/recovery, performance tuning, and reconciliation of asset data across multiple databases. Currently he works as an IT service delivery professional at Hewlett Packard Enterprise in Chennai, India.
Database administrators use Quest Software's Toad products to efficiently manage Blackboard Academic Suite databases. Toad allows administrators to automate daily, weekly, and monthly tasks like user management, backup scheduling, performance reporting, and SQL optimization. This helps control database environments that are constantly changing due to software upgrades, adoption growth, and other factors. Toad provides workflow flexibility and automation to simplify common administration processes.
The document discusses the importance of logs and log data analysis. It notes that most people do not properly utilize or manage log data from their systems. Logs can provide valuable insights into trends, system health, capacity planning, security threats and more. The document promotes log centralization and normalization using tools like Logstash and Kibana to facilitate log management and analytics. It provides examples of log sources like Apache access logs, Tomcat logs and challenges readers to setup a basic Logstash pipeline to process and visualize log data.
This document discusses emerging technology challenges and forecasts future trends. It examines the importance of forecasting technology adoption curves and preparing populations for transitions. Examples are given of past technology misses like the Apple Newton and Pluto being demoted from planet status. The document then focuses on distributed caching systems and elastic Java virtual machines that can adjust memory allocation on demand. It argues these technologies allow applications to better handle usage spikes and periods of high demand. Finally, it discusses strategies for cloud computing and third-party analytics and monitoring services.
Scaling Blackboard Learn™ for High Performance and DeliverySteve Feldman
This document discusses scaling Blackboard Learn for high performance and availability. It provides a brief history of Blackboard's technology shift from Perl to Java. It describes the evolution of Blackboard's reference architectures (RefArch) over time to incorporate more features like logging, monitoring, cloud services and virtualization. It defines concepts like performance, scalability and availability, and discusses strategies used by Blackboard to achieve high performance, such as automated provisioning, complete monitoring, application lifecycle management and web optimization services.
PTOn...Finding the Time to Dedicate Individual Projects of Passion and Role Steve Feldman
A culture presentation about how my teams make use of a dedicated time model called PTOn for working on individual projects of passion and/or role on our team. It's similar to Google's 20% model, but a little different.
Database administrators are responsible for maintaining high performance of databases like Oracle, SQL Server, and Red Hat that support enterprise applications like Blackboard. Quest Software's Spotlight tool helps administrators monitor and diagnose performance issues across various database platforms in real-time and historically. It provides visualizations of process flows, metrics, and alerts to identify bottlenecks and root causes of poor performance. Spotlight supports various database environments including Oracle, Oracle RAC, and SQL Server databases.
The document provides an introduction to load testing, including definitions of key terms, reasons for performing load testing, tools used, steps to prepare for load testing such as defining performance objectives and scenarios, and how to conduct load tests through initial configuration, calibration, establishing a baseline, collecting samples/measurements, and optimization.
This document provides an overview of Dell's reference architecture for optimizing Blackboard Academic Suite software deployments. The architecture is designed to support large-scale implementations with up to 30,000 active users through a virtualized and scalable infrastructure. It utilizes Dell PowerEdge servers, EqualLogic storage, and management tools from Quest Software. The reference architecture aims to help customers achieve high performance, scalability, simplified management, and lower TCO. It provides sizing guidelines and best practices for implementing the application, database, and storage tiers in a way that is optimized for Blackboard Academic Suite.
The document discusses how Blackboard sizes its Academic Suite software based on benchmarking. It provides details on the benchmarking methodology, including modeling user behavior, data growth, and performance objectives. The results showed how the software performed under different workload levels on various hardware configurations. The last part discusses using the benchmark results and sizing guide to determine an institution's adoption profile and appropriate hardware configuration based on factors like sessions per hour and page loads.
This document discusses an introduction to software performance engineering (SPE) presented by Steve Feldman, Tim Moore, and Ryan O'Neil from Blackboard's Performance Center of Excellence. It provides an overview of SPE methodology, which takes a systematic, quantitative approach to constructing software to meet performance objectives. The 7-step SPE process involves assessing performance risk, identifying critical use cases, selecting key performance scenarios, establishing performance objectives, constructing performance models, determining software resource requirements, and adding system resource requirements. The presentation also demonstrates applying SPE to develop a sample Blackboard building block called "Santa's Little Helper" for monitoring performance metrics.
Allison Sheridan of the NosillaCast Podcast (hosted at podfeet.com) presented this at Blogworld Expo in October 2010 in Las Vegas. It focused on how to make both your podcast and your blog more accessibile to try and capture some of the 600,000 people worldwide who are visually or hearing impaired.
This document provides numerous tips for improving Mac productivity and customizing the Mac interface. It describes shortcuts for quickly switching between apps and windows, moving and copying files, customizing the toolbar and sidebar, using Spotlight to find files, adding signatures in Preview, tweaking the Dock, accessing more file info, finding menu items and preferences, using accent keys, and various shortcuts using modifier keys like Option and Command. The tips are intended to help users work more efficiently on a Mac.
Bb world 2012 using database statistics to make capacity planning decisions...Geoff Mower
This document discusses using database statistics and metrics to inform capacity planning decisions at Blackboard. It provides examples of the types of data that can be collected from the Blackboard database, such as table sizes, growth rates, and resource usage during testing. Collecting and analyzing this data can help with administrative planning, budgeting, and strategic decisions around supporting more online testing. The document recommends setting up automated data collection and logging to track database metrics over time for capacity planning purposes.
The document provides an overview of SAP BI training. It discusses that SAP stands for Systems Applications and Products in Data Processing and was founded in 1972 in Germany. It is the world's fourth largest software provider and largest provider of ERP software. The training covers topics such as the 3-tier architecture of SAP, data warehousing, ETL, the SAP BI architecture and key components, OLTP vs OLAP, business intelligence definition, and the ASAP methodology for SAP implementations.
The document discusses building a data warehouse in SQL Server. It provides an agenda that covers topics like an overview of data warehousing, data warehouse design, dimension and fact tables, and physical design. It also discusses components of a data warehousing solution like the data warehouse database, ETL processes, and security considerations.
The document outlines a multi-month implementation plan for a BI project with the following key stages:
1) Preparation and Planning in Month 1 involving prioritization, hardware installation, staffing, and software procurement.
2) ETL development from Month 1-3 involving requirement analysis, design, development and testing of the ETL processes.
3) Initial deployment from Month 2-3 setting up the metadata framework and data governance with report reductions.
4) Ongoing development from Month 4-10 involving further report reductions, incremental deployments, building the data library and dashboards. Headcount savings also take effect during this stage.
5) Long term operations starting from Month 11 involving targeting
ADV Slides: Platforming Your Data for Success – Databases, Hadoop, Managed Ha...DATAVERSITY
Thirty years is a long time for a technology foundation to be as active as relational databases. Are their replacements here? In this webinar, we say no.
Databases have not sat around while Hadoop emerged. The Hadoop era generated a ton of interest and confusion, but is it still relevant as organizations are deploying cloud storage like a kid in a candy store? We’ll discuss what platforms to use for what data. This is a critical decision that can dictate two to five times additional work effort if it’s a bad fit.
Drop the herd mentality. In reality, there is no “one size fits all” right now. We need to make our platform decisions amidst this backdrop.
This webinar will distinguish these analytic deployment options and help you platform 2020 and beyond for success.
Develop a Custom Data Solution Architecture with NorthBayAmazon Web Services
Organizations that have vast amounts of data in legacy applications often experience difficulties delivering that data to business unit end-users. Register to learn how Eliza Corporation and Scholastic overcame this challenge by leveraging a Data Lake solution from NorthBay on AWS to optimize data analytics and provide greater visibility. AWS and NorthBay will give you an in-depth overview of how you can use a Data Lake in conjunction with your existing on-premises or cloud-based Data Warehouse. NorthBay helps organizations scale their ETL and data warehousing workloads using Amazon EMR and Amazon Redshift. Join us to learn: • Best practices for using a Data Lake in conjunction with your existing data warehouse • The key aspects of introducing agile and scrum methodologies into an enterprise • The most impactful cost-savings levers that are addressed via a cloud data warehouse migration
Who should attend: Heads of Analytics, Heads of BI, Analytics Managers, BI Teams, Senior Analysts
This document provides tips for optimizing performance in Power BI by focusing on different areas like data sources, the data model, visuals, dashboards, and using trace and log files. Some key recommendations include filtering data early, keeping the data model and queries simple, limiting visual complexity, monitoring resource usage, and leveraging log files to identify specific waits and bottlenecks. An overall approach of focusing on time-based optimization by identifying and addressing the areas contributing most to latency is advocated.
The document discusses the topics that will be covered in the MKTG 555 Marketing Analytics course. It will provide an overview of business intelligence processes and technologies. Students will learn about data warehousing, descriptive and predictive analytics using tools like SQL Server, SAS, and Tableau. The course aims to provide hands-on experience with these technologies. Likely projects include creating pivot tables and charts in Excel using data from SQL Server, building tabular data models in PowerPivot, and developing KPI dashboards in PerformancePoint and SharePoint.
Riyas Mohamed is an IT professional with over 7 years of experience in roles such as Oracle DBA, business analyst, and software tester. He has extensive experience with database administration, installation, backup/recovery, performance tuning, and reconciliation of asset data across multiple databases. Currently he works as an IT service delivery professional at Hewlett Packard Enterprise in Chennai, India.
Database administrators use Quest Software's Toad products to efficiently manage Blackboard Academic Suite databases. Toad allows administrators to automate daily, weekly, and monthly tasks like user management, backup scheduling, performance reporting, and SQL optimization. This helps control database environments that are constantly changing due to software upgrades, adoption growth, and other factors. Toad provides workflow flexibility and automation to simplify common administration processes.
Data Management - Full Stack Deep LearningSergey Karayev
This document discusses data management for deep learning projects. It covers five main topics: sources of data, labeling data, data storage, data versioning, and data processing. For data sources, it describes obtaining publicly available datasets, collecting and labeling proprietary data, and techniques for data augmentation. For labeling data, it discusses interfaces for annotators, sources of labor like outsourcing, and labeling software. For storage, it outlines options for files, objects, databases, and data lakes. It describes different levels of data versioning from unversioned to specialized solutions. And it proposes using workflows and schedulers like Airflow to automate multi-step data processing tasks.
ADV Slides: When and How Data Lakes Fit into a Modern Data ArchitectureDATAVERSITY
Whether to take data ingestion cycles off the ETL tool and the data warehouse or to facilitate competitive Data Science and building algorithms in the organization, the data lake – a place for unmodeled and vast data – will be provisioned widely in 2020.
Though it doesn’t have to be complicated, the data lake has a few key design points that are critical, and it does need to follow some principles for success. Avoid building the data swamp, but not the data lake! The tool ecosystem is building up around the data lake and soon many will have a robust lake and data warehouse. We will discuss policy to keep them straight, send data to its best platform, and keep users’ confidence up in their data platforms.
Data lakes will be built in cloud object storage. We’ll discuss the options there as well.
Get this data point for your data lake journey.
This document is a resume for S. Jayachandran seeking an IT role. It summarizes his professional experience including 9.8 years as an Oracle Certified Professional DBA and SAP BASIS administrator. Currently an Assistant Manager at DELPHI-TVS Diesel Systems, his responsibilities include managing IT teams, implementing SAP systems, and administering Oracle databases. He is proficient in technologies like SAP, Oracle, Linux and Windows servers.
The document discusses the database development life cycle (DBLC), which follows a similar process to the systems development life cycle (SDLC). The DBLC involves gathering requirements, database analysis, design, implementation, testing and evaluation, and maintenance. It describes each stage in detail, including conceptual, logical, and physical data modeling during the design stage. The goal is to systematically plan and develop a database to meet requirements while ensuring completeness, integrity, flexibility, and usability.
Collaborate 2018: Optimizing Your Robust Oracle EBS Footprint for Radical Eff...Datavail
In this session, we'll present real case studies of Datavail's proven optimization methodology and performance tuning techniques that have enabled Fortune 500 companies to scale Oracle EBS for radical efficiency, meet their most crucial business needs, and increase the lifespan of their investment.
The document provides an overview of the speaker's experience and qualifications. It lists the speaker's 7+ years of experience working with Microsoft's BI stack, current role as a MS BI Solution Architect, consulting experience, certifications including MCSA and MCTS in SQL Server, education including a BE in Computer Science and an MBA. It also provides links to the speaker's blogs and details on various SQL and BI articles authored.
ADV Slides: Building and Growing Organizational Analytics with Data LakesDATAVERSITY
Data lakes are providing immense value to organizations embracing data science.
In this webinar, William will discuss the value of having broad, detailed, and seemingly obscure data available in cloud storage for purposes of expanding Data Science in the organization.
SQL Shot is a unique highly graphic oriented performance and tuning for Microsoft SQL Server, Sybase ASE and Oracle Database isolating any performance issue in seconds.
Day 2 05 - steve feldman - logging mattersSteve Feldman
Steve Feldman gave a presentation at DevOps Days Washington, DC about the importance of log frequency and the hidden costs of excessive logging. He discussed how log frequency impacts resource usage and storage footprint, and how the costs of log management software are tied to log volume. Feldman also covered how to leverage log frequency metrics to detect anomalies and establish baselines, and emphasized that logging requirements should be defined upfront for features, with the same attention given to logging as unit tests.
The document discusses the Blackboard Reference Architecture, which provides a blueprint for optimizing enterprise application performance. It describes key components of the reference architecture, including platform infrastructure, web/application delivery and management, storage architectures, monitoring and management, and user performance management. The reference architecture aims to help institutions achieve a high level of performance maturity and optimize their systems to support user experiences.
This document provides an introduction to enterprise application management for Blackboard. It discusses the importance of understanding the various architectural components, having the right tools to manage the application, and developing plans for operations, capacity, disasters, and quality assurance. Going enterprise requires understanding user needs, technical foundations, and planning for growth and issues.
The document discusses performance engineering at Blackboard, including defining key concepts like performance, scalability, and the application performance index (Apdex). It outlines Blackboard's performance engineering process and methodology, including using tools like LoadRunner for testing and establishing performance archetype ratios to measure scalability. Planned performance engineering projects for 2007 are also mentioned, such as virtualization testing and monitoring initiatives.
The document describes a reference architecture for deploying Blackboard Learn online learning software on a scalable virtualized infrastructure. Key components of the architecture include:
1) An application tier with Blackboard Learn instances running on virtual machines for flexibility and high utilization. This tier is optimized to support over 100,000 concurrent users.
2) A storage tier using Dell EqualLogic storage arrays to provide pooled resources for virtual machine storage.
3) An optimized Oracle database configuration on powerful Dell servers to handle the database requirements.
4) A network environment and best practices to ensure high performance as the solution scales. Management and monitoring tools help maintain performance and availability.
This document discusses a scalable solution for Blackboard education software using Dell hardware and Oracle database software. It provides an overview of testing done to measure the scalability of Blackboard applications running on Dell servers with an Oracle Real Application Cluster database configuration. Tests showed the solution could sustain over 30,000 active user sessions per hour, demonstrating strong scalability. The document provides guidance on sizing, performance tuning, and implementing a reference deployment of the Blackboard/Dell solution.
This document discusses the Dell/Blackboard solution for education, which provides a scalable and reliable infrastructure for deploying the Blackboard Academic Suite. Key components of the solution include Blackboard application software, Dell server and storage hardware, and Oracle Database software. Testing showed the solution could support over 30,000 concurrent user sessions per hour. The document provides details on system sizing, performance tuning, and a reference deployment configuration using Dell and Blackboard components.
This document summarizes a presentation about deploying Tomcat clusters in an advanced Blackboard environment. The presentation introduces clustering concepts and techniques for planning and deploying Tomcat application clusters. It defines what clusters and nodes are, and explains the differences between clustered and load balanced nodes. The presentation reviews configuration requirements for setting up a Tomcat cluster and load distribution, and provides examples of cluster code. It also discusses session replication, guidelines for setting up clusters, and includes benchmark statistics comparing clustered and non-clustered configurations.
The document summarizes the results of benchmarking tests performed on the Blackboard Academic Suite to determine system sizing requirements. Key findings include:
- Tests showed a Unicode conversion taking minutes for small datasets, hours for moderate, and under 3 days for large datasets, meeting objectives.
- Regression performance from version 6.3 to 7.X met the objective of no more than a 5% degradation and potential for a 5% improvement.
- Benchmarking of different hardware platforms like Sun, Dell, and Windows showed performance varied based on configuration.
The document discusses performance forensics tools that can be used at different layers of the architectural stack and client layer to perform root cause analysis of performance or scalability issues. It introduces tools for analyzing the user experience layer using HTTP data, JVM instrumentation tools, database instrumentation tools focused on sessions and wait events, and log analyzers. Specific tools discussed include Fiddler2, Coradiant Truesight, JSTAT, -VerboseGC and -Xloggc, IBM Pattern Modeling Tool for Java GC, JHAT, JMAP, SAP Memory Analyzer, ASH, and the SQL Server Performance Dashboard.
This document provides best practices for optimizing Blackboard Learn performance. It recommends deploying for performance from the start, optimizing platform components continuously through measurements, using scalable deployments like 64-bit architectures and virtualization, improving page responsiveness through techniques like gzip compression and image optimization, optimizing the web server, Java Virtual Machine, and database through configuration and tools. It emphasizes the importance of understanding resource utilization, wait events, execution plans, and statistics/histograms for database optimization.
The document discusses deploying Blackboard learning management systems in a highly performant, scalable, and available manner. It covers key concepts around performance, scalability, and availability. It then discusses the drivers behind online learning growth and increasing user expectations. The rest of the document provides recommendations around deployment strategies that focus on flexibility, efficiency, adaptability, and reliability to ensure the continuous and optimal performance of Blackboard systems at any scale through virtualization, large memory allocation, distributed architectures, automation, and high-availability techniques.
071510 sun b_1515_feldman_stephen_forpublicSteve Feldman
This document discusses scaling Blackboard to support large online learning communities. It notes that online enrollments are growing significantly faster than overall higher education. Communities and stakes are getting larger as competition increases for students and funding. Blackboard must support larger class sizes, richer content, extended user sessions, and near 24/7 availability. This requires scalable, high-performance, and highly available systems through techniques like virtualization, fast provisioning, emphasis on asynchronous tools and databases, and advanced monitoring.
Microsoft Power Point Best Practices For Scaling Heavily Adopted And Concur...Steve Feldman
The document discusses various tools used for monitoring and optimizing performance in a Blackboard environment. It recommends using Toad as the primary database access tool for tasks like script execution and basic DBA work. It also recommends using PAO (Performance Analysis for Oracle) as the primary monitoring tool for its warehousing capabilities and ability to compare workloads over time. Foglight is highlighted as a favorite tool for its lightweight instrumentation, ability to define rule-based traces to identify slow method calls and SQL statements, and other features. Configuration examples are provided for optimizing Java settings like heap size and garbage collection with aggressive -XX options.
The document provides a performance report for a website that summarizes response times, HTTP response codes, and makes recommendations. It found redundant requests that increased load times, including the same XML file requested 26 times. Predicted response times were 400ms originally but could be improved to 190ms by enabling GZIP compression and removing excess files. The report recommends fixing redundant requests and other design changes to improve performance.
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfMalak Abu Hammad
Discover how MongoDB Atlas and vector search technology can revolutionize your application's search capabilities. This comprehensive presentation covers:
* What is Vector Search?
* Importance and benefits of vector search
* Practical use cases across various industries
* Step-by-step implementation guide
* Live demos with code snippets
* Enhancing LLM capabilities with vector search
* Best practices and optimization strategies
Perfect for developers, AI enthusiasts, and tech leaders. Learn how to leverage MongoDB Atlas to deliver highly relevant, context-aware search results, transforming your data retrieval process. Stay ahead in tech innovation and maximize the potential of your applications.
#MongoDB #VectorSearch #AI #SemanticSearch #TechInnovation #DataScience #LLM #MachineLearning #SearchTechnology
Essentials of Automations: The Art of Triggers and Actions in FMESafe Software
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
UiPath Test Automation using UiPath Test Suite series, part 6DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 6. In this session, we will cover Test Automation with generative AI and Open AI.
UiPath Test Automation with generative AI and Open AI webinar offers an in-depth exploration of leveraging cutting-edge technologies for test automation within the UiPath platform. Attendees will delve into the integration of generative AI, a test automation solution, with Open AI advanced natural language processing capabilities.
Throughout the session, participants will discover how this synergy empowers testers to automate repetitive tasks, enhance testing accuracy, and expedite the software testing life cycle. Topics covered include the seamless integration process, practical use cases, and the benefits of harnessing AI-driven automation for UiPath testing initiatives. By attending this webinar, testers, and automation professionals can gain valuable insights into harnessing the power of AI to optimize their test automation workflows within the UiPath ecosystem, ultimately driving efficiency and quality in software development processes.
What will you get from this session?
1. Insights into integrating generative AI.
2. Understanding how this integration enhances test automation within the UiPath platform
3. Practical demonstrations
4. Exploration of real-world use cases illustrating the benefits of AI-driven test automation for UiPath
Topics covered:
What is generative AI
Test Automation with generative AI and Open AI.
UiPath integration with generative AI
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Dr. Sean Tan, Head of Data Science, Changi Airport Group
Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Maruthi Prithivirajan, Head of ASEAN & IN Solution Architecture, Neo4j
Get an inside look at the latest Neo4j innovations that enable relationship-driven intelligence at scale. Learn more about the newest cloud integrations and product enhancements that make Neo4j an essential choice for developers building apps with interconnected data and generative AI.
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionAggregage
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!SOFTTECHHUB
As the digital landscape continually evolves, operating systems play a critical role in shaping user experiences and productivity. The launch of Nitrux Linux 3.5.0 marks a significant milestone, offering a robust alternative to traditional systems such as Windows 11. This article delves into the essence of Nitrux Linux 3.5.0, exploring its unique features, advantages, and how it stands as a compelling choice for both casual users and tech enthusiasts.
Sudheer Mechineni, Head of Application Frameworks, Standard Chartered Bank
Discover how Standard Chartered Bank harnessed the power of Neo4j to transform complex data access challenges into a dynamic, scalable graph database solution. This keynote will cover their journey from initial adoption to deploying a fully automated, enterprise-grade causal cluster, highlighting key strategies for modelling organisational changes and ensuring robust disaster recovery. Learn how these innovations have not only enhanced Standard Chartered Bank’s data infrastructure but also positioned them as pioneers in the banking sector’s adoption of graph technology.
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex ProofsAlex Pruden
This paper presents Reef, a system for generating publicly verifiable succinct non-interactive zero-knowledge proofs that a committed document matches or does not match a regular expression. We describe applications such as proving the strength of passwords, the provenance of email despite redactions, the validity of oblivious DNS queries, and the existence of mutations in DNA. Reef supports the Perl Compatible Regular Expression syntax, including wildcards, alternation, ranges, capture groups, Kleene star, negations, and lookarounds. Reef introduces a new type of automata, Skipping Alternating Finite Automata (SAFA), that skips irrelevant parts of a document when producing proofs without undermining soundness, and instantiates SAFA with a lookup argument. Our experimental evaluation confirms that Reef can generate proofs for documents with 32M characters; the proofs are small and cheap to verify (under a second).
Paper: https://eprint.iacr.org/2023/1886
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex Proofs
Bb world 2011 capacity planning
1. Using Database
Statistics to Make
Capacity Planning
Decisions
Geoff Mower
Senior Software Performance Engineer, Blackboard
Steve Feldman
Senior Director Software Performance Engineering and Architecture, Blackboard
2. Who Are We?
Geoff Mower- Senior Software Performance Engineer
5 years on the frontlines as a DBA
2 hour interview on a plane
Performance and Scalability for
Outcomes & Community
3. Who Are We?
Steve Feldman
• Blackboard since 2003
• Performance Engineering
from the start
• Platform Architecture in
2005
• Security Engineering in
2010
@seven_seconds
“Love my job…love my
team. If you email me, I will
http://goo.gl/Z4Rq5
respond.”
4. What We Want You to Do?
ask questions Mine your data Ask follow-up questions
about your
data
5. What are our obstacles?
• Certain data points are transient
– Logging will need to be created in certain situations
to fill in missing data points.
• See slide #45 & #46 for an example
6. What You Need
• Permissions to run SQL scripts on your
Blackboard Learn environment's database
• Basic SQL fundamentals
• Analysis of results
• Database Tools
– Aquadata client & Server or Quest’s Toad
7. Skill and Knowledge Sets
Open DB Documentation
• Blackboard Learn™, Release 9.1 Open
Database Document
– http://bit.ly/loW9hi
8.
9. Asking the Right Questions
Collecting the Right Data
Organizing and Analyzing the Data Making Decisions
Ask Follow-up Questions
10. Types of Data
Data Types Table Example
ACTIVITY_ACCUMULATOR User navigation
Event
COURSE_USERS snapshot
Process
COURSE_CONTENT User created
User entities
LEARN_admin.bb_instance installation
Static
SESSIONS Tracks who has
Transient an active session
11. Areas of Capacity Planning
• How can the Blackboard Learn schema help you
capacity plan for your business requirements in
regards to?
Administrative Planning Strategic Functional Decisions
Expense and Budget Management
13. Administrative Planning
To be able to restore individual courses, can we use the
course archive tool as a backup function?
14. Administrative Planning
Asking the Right Questions
How will we know which courses need archiving?
Frequency of course archive task?
Will the archiving process impact user experience?
Does the course archive tool have limitation?
Location of course archives?
15. Administrative Planning
Collecting the Right Data
How will we know which courses need archiving?
– Course entity creation and modification
Creation Tables Modification Tables
COURSE_CONTENTS COURSE_CONTENTS
GRADEBOOK_MAIN GRADEBOOK_MAIN
GRADEBOOK_GRADES GRADEBOOK_GRADES
QTI_ASI_DATA QTI_ASI_DATA
QTI_RESULT_DATA QTI_RESULT_DATA
FORUM_MAIN FORUM_MAIN
MSG_MAIN MSG_MAIN
CALENDAR
STAFFINFORMATION
16. Administrative Planning
Organizing and Analyzing the Data
• Data Points: Course/Organizations to be
archived SQL/Oracle
COURSE_ID Number of New/Modified Course Entities
Hist_101 42
Eng_252 42
Sci_139 38
Eng_429 37
Art_202 35
Eco_323 35
17. Administrative Planning
Making Decisions from the Data
• How many new or modified course/organization
entities are required for a course/organization to
be archived?
• How many courses/organizations should we
archive each time the archive task is execution?
• What is the maximum time between archive
tasks?
18. Administrative Planning
Acting on Your Descion
SQL/Oracle
Hist_101, c:Course_Archives2011-07-09
Eng_252, c:Course_Archives2011-07-09
Sci_139, c:Course_Archives2011-07-09
Eng_429, c:Course_Archives2011-07-09
…
• Create a JOB that would run the SQL script to generate
a txt file containing the query results
• Create a schedule task that would run the
batch_ImportExport.bat tool and direct it to the txt file
created by the JOB in the previous step.
20. Asking the Right Questions
Expense and Budget Management
• How much do we need to budget for in order to fund our
database requirements?
21. Expense and Budget Management
• When are we going to hit our 95% disk spaced used
threshold.
22. Expense and Budget Management
• Available disk space issue
– Talk to Network Security Manager (owner of the SAN)
• We need more disk space
23. Expense and Budget Management
Asking the Right Questions
• What did I miss?
– When will we run out of space?
– How long do we expect the requested disk space to
last us?
• Facts vs. guess work
24. Expense and Budget Management
Collecting the Right Data
• Data Points: Database file size
• Binary
• Data Files (.mdf)
• TempDB
• Transaction Logs (.ldf)
• Back-up/restore process
25. Expense and Budget Management
Organizing and Analyzing the Data
• Data Points: Database file size SQL/Oracle
Database Name Physical Name Size in MB
BBLEARN_stats F:dataBBLEARN_stats_data.mdf 70338
BBLEARN F:dataBBLEARN_data.mdf 45520
cms_files_inst F:datacms_files_inst_data.mdf 5704
BBLEARN G:logsBBLEARN_log.ldf 3083
cms_files_inst G:logscms_files_inst_log.ldf 1361
cms_files_courses F:datacms_files_courses_data.mdf 1072
cms_files_users F:datacms_files_users_data.mdf 926
msdb F:dataMSSQL.1MSSQLDATAMSDBData.mdf 792
cms_files_library F:datacms_files_library_data.mdf 720
cms_files_orgs F:datacms_files_orgs_data.mdf 384
msdb F:dataMSSQL.1MSSQLDATAMSDBLog.ldf 259
cms_files_users G:logscms_files_users_log.ldf 214
cms_files_courses G:logscms_files_courses_log.ldf 207
26. Expense and Budget Management
Organizing and Analyzing the Data
• Data Points: Table Row Count and Sizes SQL/Oracle
Row
Table Name Count Reserved Data Index Size Unused
ACTIVITY_ACCUMULATOR 377266616 84097056 46905888 37183888 7280
COURSE_USERS 3837931 496024 258272 237496 256
USERS 303687 139480 96848 42440 192
COURSE_MAIN 121623 86024 47760 37984 280
NAVIGATION_ITEM 1876 2032 608 1128 296
SYSTEM_TRACKING 3550 1936 1672 160 104
USER_ROLES 4114 544 160 288 96
27. Expense and Budget Management
Organizing and Analyzing the Data
Data Points: Table growth rate (by year) of the
Activity_Accumulator SQL/Oracle
12000000
10000000
80000000
60000000
40000000
20000000
0
2003 2004 2005 2006 2007 2008 2009 2010
28.
29. Expense and Budget Management
Organizing and Analyzing the Data
Data Points: ACTIVITY_ACCUMULATOR growth rate (by
month) SQL/Oracle
12000000
10000000
8000000
6000000
4000000
2000000
0
30. Expense and Budget Management
• Data Points: COURSE_USERS table growth rate (by
month)
900000
800000
700000
600000
500000
400000
300000
200000
100000
0
August
August
August
31. Expense and Budget Management
Making Decisions from the Data
• Conclusion
• Due to opening of school tasks we will see our
available disk space decrease below 5% during the
month of August.
• In order to make it 3 years we will need a minimum
120 GB.
34. Strategic Functional Decisions
Asking the Right Questions
• Study…
• Online & offline test taking populations
• The tests themselves
• Test taking behavior
• How testing impacts the database
• Locating when you will see resource contention
35. Strategic Functional Decisions
Historical Use of Online Testing
What is the percent of students from the previous term that have
completed a test online? SQL/Oracle
Online
Offline
36.
37.
38.
39. Strategic Functional Decisions
Collecting the Right Data
Previous Use Full Adoption
Courses with Tests 11,274 55,504
Enrollments with Tests 225,480 1,110,080
Number of questions 4,219,032 25,694,266
submitted in a term
Peak Question Submission 60,346 297,094
by hour
40. Strategic Functional Decisions
Collecting the Right Data
• Determine CPU and I/O cost of testing (SQL server)
– Track the Stored Procedures performing the works
SQL/Oracle
» qti_result_data_cr
» update_gradebook_grade
» attempt_cr
» update_qti_res_bbmd_dtmod
» gb2_is_log_enabled
» gb2_attempt_logger
41. Strategic Functional Decisions
Organizing and Analyzing the Data
• Determine CPU and I/O cost of testing (SQL server)
– Track the Stored Procedures performing the works SQL/Oracle
42. Strategic Functional Decisions
Organizing and Analyzing the Data
• Determine CPU and I/O cost of testing
– Track the queries performing the work SQL/Oracle
43. Strategic Functional Decisions
Organizing and Analyzing the Data
Determine IO cost of all cached stored procedures SQL/Oracle
Average IO
700
600
500
400
300
200
100
0
44. Strategic Functional Decisions
Making Decisions from the Data
• A reasonable understanding…
• How tests are currently used
• How tests will be used
• How much each question costs the database upon
submission
• Lack information on…
• Baseline of database’s IO and CPU resources
• Non-testing database transactions
45. Strategic Functional Decisions
Collecting the Right Data
• Determine the breaking point of CPU and I/O of testing
– Create a CPU logging script SQL/Oracle
• SQL server logging limitation
– 4 hours of CPU tracking
– Lacks IO counters
– Create a IO logging script SQL/Oracle
• Snapshot in time
– Need to log the delta between the last logged IO metric and current
query results
– Create a Query logging script SQL/Oracle
• Compares test question submission to all other transactions
46. Strategic Functional Decisions
Collecting the Right Data
• Create a separate database from the Blackboard
Learn’s databases (you can use the database server
Blackboard Learn is running on)
• Create tables with in the new database that match
the query results you wish to store
• Create a JOB that will run each query and insert the
results into the corresponding table and database.
47. Capacity Planning
is an art if done
properly…
or a disaster if not
done at all.
50. Please provide feedback for this session by emailing
DevConFeedback@blackboard.com.
The title of this session is:
Using Database Statistics to Make Capacity Planning
Decisions
52. Administrative Planning
What courses meet the course archive criteria
declare @Last_Archive datetime
declare @Datediff int
declare @Number_of_New_Entities int
set @Last_Archive = '2011-6-7' --Enter the last datetime the course archival process ran
set @Datediff = 24 --Enter the time duration between course creation and course entities you would like to track. For example, to avoid archive courses that were created since the last archive process enter the time of your longest running snapshot or SIS process in terms of hours.
set @Number_of_New_Entities = 1 --Enter the number of new course entities required for a course to be archived
select course_id, CC.CC + GM.GM + GG.GG + QA.QA + A.A + ConM.CONFERENCE_MAIN + FM.FORUM_MAIN + MM.MSG_MAIN + C.C + S.S AS [Number of New/Modified Course Entities]
from course_main cm
left join (select crsmain_pk1, COUNT(cc.pk1) CC
from course_contents cc
left join course_main cm on cm.pk1 = cc.crsmain_pk1
where (cc.dtcreated > @Last_Archive
or cc.dtmodified > @Last_Archive)
and datediff(hour, cm.dtcreated, cc.dtcreated) > @Datediff
group by crsmain_pk1) CC on CC.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(gm.pk1) GM
from gradebook_main gm
left join course_main cm on cm.pk1 = gm.crsmain_pk1
where (date_added > @Last_Archive
or date_modified > @Last_Archive)
and datediff(hour, cm.dtcreated, gm.date_added) > @Datediff
group by crsmain_pk1) GM on GM.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(gg.pk1) GG
from gradebook_main gm
left join gradebook_grade gg on gm.pk1 = gg.gradebook_main_pk1
left join course_main cm on gm.crsmain_pk1 = cm.pk1
where (gg.date_added > @Last_Archive
OR gg.date_modified > @Last_Archive)
and datediff(hour, cm.dtcreated, gg.date_added) > @Datediff
group by crsmain_pk1) GG on GG.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(qa.pk1) QA
from qti_asi_data qa
left join course_main cm on cm.pk1 = qa.crsmain_pk1
where (bbmd_date_added > @Last_Archive
OR bbmd_date_modified > @Last_Archive)
and datediff(hour, cm.dtcreated, qa.bbmd_date_added) > @Datediff
group by crsmain_pk1) QA on QA.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(a.pk1) A
from attempt a
left join gradebook_grade gg on a.pk1 = gg.last_attempt_pk1
left join gradebook_main gm on gm.pk1 = gg.gradebook_main_pk1
left join course_main cm on cm.pk1 = gm.crsmain_pk1
where (a.attempt_date > @Last_Archive
or a.date_modified > @Last_Archive)
and datediff(hour, cm.dtcreated, a.attempt_date) > @Datediff
group by crsmain_pk1) A on A.crsmain_pk1 = cm.pk1
left join (select co.owner_pk1, COUNT(CON.pk1) CONFERENCE_MAIN
from conference_owner CO
left join conference_main CON on CON.conference_owner_pk1 = CO.pk1
left join course_main cm on cm.pk1 = co.owner_pk1
where co.owner_table = 'COURSE_MAIN'
and (CON.dtcreated > @Last_Archive
OR CON.dtmodified > @Last_Archive)
and datediff(hour, cm.dtcreated, con.dtcreated) > @Datediff
group by owner_pk1) ConM on ConM.OWNER_PK1 = cm.pk1
left join (select co.owner_pk1, COUNT(fr.pk1) FORUM_MAIN
from conference_owner CO
left join conference_main CON on CON.conference_owner_pk1 = CO.pk1
left join forum_main fr on fr.confmain_pk1 = CON.pk1
left join course_main cm on cm.pk1 = co.owner_pk1
where owner_table = 'COURSE_MAIN'
and (fr.dtcreated > @Last_Archive
OR fr.dtmodified > @Last_Archive)
and datediff(hour, cm.dtcreated, fr.dtcreated) > @Datediff
group by owner_pk1) FM on FM.OWNER_PK1 = cm.pk1
left join (select co.owner_pk1, COUNT(mm.pk1) MSG_MAIN
from conference_owner CO
left join conference_main CON on CON.conference_owner_pk1 = CO.pk1
left join forum_main fr on fr.confmain_pk1 = CON.pk1
left join msg_main mm on mm.forummain_pk1 = fr.pk1
left join msg_main mm_reply on mm_reply.msgmain_pk1 = mm.pk1
left join course_main cm on cm.pk1 = co.owner_pk1
where owner_table = 'COURSE_MAIN'
and (mm.dtcreated > @Last_Archive
OR mm.dtmodified > @Last_Archive)
and datediff(hour, cm.dtcreated, mm.dtcreated) > @Datediff
group by owner_pk1) MM on MM.OWNER_PK1 = cm.pk1
left join (select crsmain_pk1, COUNT(c.pk1) C
from calendar c
left join course_main cm on cm.pk1 = c.crsmain_pk1
where c.dtmodified > @Last_Archive
and datediff(hour, cm.dtcreated, c.dtmodified) > @Datediff
group by crsmain_pk1) C on C.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(s.pk1) S
from staffinformation S
left join course_main cm on cm.pk1 = s.crsmain_pk1
where s.dtmodified > @Last_Archive
and datediff(hour, cm.dtcreated, s.dtmodified) > @Datediff
group by crsmain_pk1) S on S.crsmain_pk1 = cm.pk1
53. Administrative Planning
What courses meet the course archive criteria
set serveroutput on size 100000
---Administrative Planning
What courses meet the course archive criteria
declare
v_Last_Archive date;
v_Datediff int;
v_Number_of_New_Entities int;
v_Directory varchar(256);
v_File_Name varchar(256);
Begin
v_Last_Archive := TO_DATE('7-JUN-2011'); --Enter the last datetime the course archival process ran
v_Datediff := 1 ; --this is the day
--Enter the time duration between course creation and course entities you would like to track. For example, to avoid archive courses that were created ----since the last archive process enter the time of your longest running snapshot or SIS process in terms of hours.
v_Number_of_New_Enti ties := 1;
--Enter the number of new course entities required for a course to be archived
select 'usrlocalblackboardCourse_Archi ves ' | | TO_CHAR(SYSDATE, 'DD-Mon-YYYY') | | '' into v_Directory FROM Dual;
--Alter the string to build the directory you wish to store your archives
dbms_output.enable(9999999);
dbms_output.new_line( );
FOR rec IN (
select course_id | | v_Directory as output
from course_main cm
left join (select crsmain_pk1, COUNT(cc.pk1) CC
from course_contents cc
left join course_main cm on cm.pk1 = cc.crsmain_pk1
where (cc.dtcreated > v_Last_Archive
or cc.dtmodified > v_Last_Archive)
and cm.dtcreated- cc.dtcreated> v_Datediff
group by crsmain_pk1) CC on CC.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(gm.pk1) GM
from gradebook_main gm
left join course_main cm on cm.pk1 = gm.crsmain_pk1
where (date_added > v_Last_Archive
or date_modified > v_Last_Archive)
and cm.dtcreated- gm.date_added> v_Datediff
group by crsmain_pk1) GM on GM.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(gg.pk1) GG
from gradebook_main gm
left join gradebook_grade gg on gm.pk1 = gg.gradebook_main_pk1
left join course_main cm on gm.crsmain_pk1 = cm.pk1
where (gg.date_added > v_Last_Archive
OR gg.date_modified > v_Last_Archive)
and cm.dtcreated- gg.date_added> v_Datediff
group by crsmain_pk1) GG on GG.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(qa.pk1) QA
from qti_asi_data qa
left join course_main cm on cm.pk1 = qa.crsmain_pk1
where (bbmd_date_added > v_Last_Archive
OR bbmd_date_modified > v_Last_Archive)
and cm.dtcreated- qa.bbmd_date_added> v_Datediff
group by crsmain_pk1) QA on QA.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(a.pk1) A
from attempt a
left join gradebook_grade gg on a.pk1 = gg.last_attempt_pk1
left join gradebook_main gm on gm.pk1 = gg.gradebook_main_pk1
left join course_main cm on cm.pk1 = gm.crsmain_pk1
where (a.attempt_date > v_Last_Archive
or a.date_modified > v_Last_Archive)
and cm.dtcreated- a.attempt_date > v_Datediff
group by crsmain_pk1) A on A.crsmain_pk1 = cm.pk1
left join (select co.owner_pk1, COUNT(CON.pk1) CONFERENCE_MAIN
from conference_owner CO
left join conference_main CON on CON.conference_owner_pk1 = CO.pk1
left join course_main cm on cm.pk1 = co.owner_pk1
where co.owner_table = 'COURSE_MAIN'
and (CON.dtcreated > v_Last_Archive
OR CON.dtmodified > v_Last_Archive)
and cm.dtcreated-con.dtcreated > v_Datediff
group by owner_pk1) ConM on ConM.OWNER_PK1 = cm.pk1
left join (select co.owner_pk1, COUNT(fr.pk1) FORUM_MAIN
from conference_owner CO
left join conference_main CON on CON.conference_owner_pk1 = CO.pk1
left join forum_main fr on fr.confmain_pk1 = CON.pk1
left join course_main cm on cm.pk1 = co.owner_pk1
where owner_table = 'COURSE_MAIN'
and (fr.dtcreated > v_Last_Archive
OR fr.dtmodified > v_Last_Archive)
and cm.dtcreated- fr.dtcreated > v_Datediff
group by owner_pk1) FM on FM.OWNER_PK1 = cm.pk1
left join (select co.owner_pk1, COUNT(mm.pk1) MSG_MAIN
from conference_owner CO
left join conference_main CON on CON.conference_owner_pk1 = CO.pk1
left join forum_main fr on fr.confmain_pk1 = CON.pk1
left join msg_main mm on mm.forummain_pk1 = fr.pk1
left join msg_main mm_reply on mm_reply.msgmain_pk1 = mm.pk1
left join course_main cm on cm.pk1 = co.owner_pk1
where owner_table = 'COURSE_MAIN'
and (mm.dtcreated > v_Last_Archive
OR mm.dtmodifi ed > v_Last_Archive)
and cm.dtcreated- mm.dtcreated > v_Datediff
group by owner_pk1) MM on MM.OWNER_PK1 = cm.pk1
left join (select crsmain_pk1, COUNT(c.pk1) C
from calendar c
left join course_main cm on cm.pk1 = c.crsmain_pk1
where c.dtmodified > v_Last_Archive
and cm.dtcreated- c.dtmodified > v_Datediff
group by crsmain_pk1) C on C.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(s.pk1) S
from staffinformation S
left join course_main cm on cm.pk1 = s.crsmain_pk1
where s.dtmodified > v_Last_Archive
and cm.dtcreated- s.dtmodified > v_Datediff
group by crsmain_pk1) S on S.crsmain_pk1 = cm.pk1
where nvl(CC.CC,0) + nvl(GM.GM,0) + nvl(GG.GG,0) + nvl(QA.QA,0) + nvl(A.A,0) + nvl(ConM.CONFERENCE_MAIN,0) + nvl(FM.FORUM_MAIN,0) + nvl(MM.MSG_MAIN,0) + nvl(S.S,0) + nvl(C.C,0) >= v_Number_of_New_Entities )
LOOP
dbms_output.put_line (rec.output);
END LOOP;
dbms_output.new_line( );
end;
54. Administrative Planning
What courses meet the course archive criteria
declare @Last_Archive datetime
declare @Datediff int
declare @Number_of_New_Entities int
declare @Directory varchar(256)
declare @File_Name varchar(256)
set @Last_Archive = '2011-6-7' --Enter the last datetime the course archival process ran
set @Datediff = 24 --Enter the time duration between course creation and course entities you would like to track. For example, to avoid archive courses that were created since the last archive process enter the time of your longest running snapshot or SIS process in terms of hours.
set @Number_of_New_Entities = 1 --Enter the number of new course entities required for a course to be archived
set @Directory = (SELECT ', c:Course_Archives' + CAST(CONVERT(date, GETDATE()) as varchar(10)) + '') --Alter the string to build the directory you wish to store your archives
select course_id + @Directory
from course_main cm
left join (select crsmain_pk1, COUNT(cc.pk1) CC
from course_contents cc
left join course_main cm on cm.pk1 = cc.crsmain_pk1
where (cc.dtcreated > @Last_Archive
or cc.dtmodified > @Last_Archive)
and datediff(hour, cm.dtcreated, cc.dtcreated) > @Datediff
group by crsmain_pk1) CC on CC.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(gm.pk1) GM
from gradebook_main gm
left join course_main cm on cm.pk1 = gm.crsmain_pk1
where (date_added > @Last_Archive
or date_modified > @Last_Archive)
and datediff(hour, cm.dtcreated, gm.date_added) > @Datediff
group by crsmain_pk1) GM on GM.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(gg.pk1) GG
from gradebook_main gm
left join gradebook_grade gg on gm.pk1 = gg.gradebook_main_pk1
left join course_main cm on gm.crsmain_pk1 = cm.pk1
where (gg.date_added > @Last_Archive
OR gg.date_modified > @Last_Archive)
and datediff(hour, cm.dtcreated, gg.date_added) > @Datediff
group by crsmain_pk1) GG on GG.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(qa.pk1) QA
from qti_asi_data qa
left join course_main cm on cm.pk1 = qa.crsmain_pk1
where (bbmd_date_added > @Last_Archive
OR bbmd_date_modified > @Last_Archive)
and datediff(hour, cm.dtcreated, qa.bbmd_date_added) > @Datediff
group by crsmain_pk1) QA on QA.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(a.pk1) A
from attempt a
left join gradebook_grade gg on a.pk1 = gg.last_attempt_pk1
left join gradebook_main gm on gm.pk1 = gg.gradebook_main_pk1
left join course_main cm on cm.pk1 = gm.crsmain_pk1
where (a.attempt_date > @Last_Archive
or a.date_modified > @Last_Archive)
and datediff(hour, cm.dtcreated, a.attempt_date) > @Datediff
group by crsmain_pk1) A on A.crsmain_pk1 = cm.pk1
left join (select co.owner_pk1, COUNT(CON.pk1) CONFERENCE_MAIN
from conference_owner CO
left join conference_main CON on CON.conference_owner_pk1 = CO.pk1
left join course_main cm on cm.pk1 = co.owner_pk1
where co.owner_table = 'COURSE_MAIN'
and (CON.dtcreated > @Last_Archive
OR CON.dtmodified > @Last_Archive)
and datediff(hour, cm.dtcreated, con.dtcreated) > @Datediff
group by owner_pk1) ConM on ConM.OWNER_PK1 = cm.pk1
left join (select co.owner_pk1, COUNT(fr.pk1) FORUM_MAIN
from conference_owner CO
left join conference_main CON on CON.conference_owner_pk1 = CO.pk1
left join forum_main fr on fr.confmain_pk1 = CON.pk1
left join course_main cm on cm.pk1 = co.owner_pk1
where owner_table = 'COURSE_MAIN'
and (fr.dtcreated > @Last_Archive
OR fr.dtmodified > @Last_Archive)
and datediff(hour, cm.dtcreated, fr.dtcreated) > @Datediff
group by owner_pk1) FM on FM.OWNER_PK1 = cm.pk1
left join (select co.owner_pk1, COUNT(mm.pk1) MSG_MAIN
from conference_owner CO
left join conference_main CON on CON.conference_owner_pk1 = CO.pk1
left join forum_main fr on fr.confmain_pk1 = CON.pk1
left join msg_main mm on mm.forummain_pk1 = fr.pk1
left join msg_main mm_reply on mm_reply.msgmain_pk1 = mm.pk1
left join course_main cm on cm.pk1 = co.owner_pk1
where owner_table = 'COURSE_MAIN'
and (mm.dtcreated > @Last_Archive
OR mm.dtmodified > @Last_Archive)
and datediff(hour, cm.dtcreated, mm.dtcreated) > @Datediff
group by owner_pk1) MM on MM.OWNER_PK1 = cm.pk1
left join (select crsmain_pk1, COUNT(c.pk1) C
from calendar c
left join course_main cm on cm.pk1 = c.crsmain_pk1
where c.dtmodified > @Last_Archive
and datediff(hour, cm.dtcreated, c.dtmodified) > @Datediff
group by crsmain_pk1) C on C.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(s.pk1) S
from staffinformation S
left join course_main cm on cm.pk1 = s.crsmain_pk1
where s.dtmodified > @Last_Archive
and datediff(hour, cm.dtcreated, s.dtmodified) > @Datediff
group by crsmain_pk1) S on S.crsmain_pk1 = cm.pk1
where ISNULL(CC.CC,0) + ISNULL(GM.GM,0) + ISNULL(GG.GG,0) + ISNULL(QA.QA,0) + ISNULL(A.A,0) + ISNULL(ConM.CONFERENCE_MAIN,0) + ISNULL(FM.FORUM_MAIN,0) + ISNULL(MM.MSG_MAIN,0) + ISNULL(S.S,0) + ISNULL(C.C,0) >= @Number_of_New_Entities
55. Administrative Planning
What courses meet the course archive criteria
set serveroutput on size 100000
---Administrative Planning
What courses meet the course archive criteria
declare
v_Last_Archive date;
v_Datediff int;
v_Number_of_New_Entities int;
v_Directory varchar(256);
v_File_Name varchar(256);
Begin
v_Last_Archive := TO_DATE('7-JUN-2011'); --Enter the last datetime the course archival process ran
v_Datediff := 1 ; --this is the day
--Enter the time duration between course creation and course entities you would like to track. For example, to avoid archive courses that were created ----since the last archive process enter the time of your longest running snapshot or SIS process in terms of hours.
v_Number_of_New_Enti ties := 1;
--Enter the number of new course entities required for a course to be archived
select 'usrlocalblackboardCourse_Archi ves ' | | TO_CHAR(SYSDATE, 'DD-Mon-YYYY') | | '' into v_Directory FROM Dual;
--Alter the string to build the directory you wish to store your archives
dbms_output.enable(9999999);
dbms_output.new_line( );
FOR rec IN (
select course_id | | v_Directory as output
from course_main cm
left join (select crsmain_pk1, COUNT(cc.pk1) CC
from course_contents cc
left join course_main cm on cm.pk1 = cc.crsmain_pk1
where (cc.dtcreated > v_Last_Archive
or cc.dtmodified > v_Last_Archive)
and cm.dtcreated- cc.dtcreated> v_Datediff
group by crsmain_pk1) CC on CC.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(gm.pk1) GM
from gradebook_main gm
left join course_main cm on cm.pk1 = gm.crsmain_pk1
where (date_added > v_Last_Archive
or date_modified > v_Last_Archive)
and cm.dtcreated- gm.date_added> v_Datediff
group by crsmain_pk1) GM on GM.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(gg.pk1) GG
from gradebook_main gm
left join gradebook_grade gg on gm.pk1 = gg.gradebook_main_pk1
left join course_main cm on gm.crsmain_pk1 = cm.pk1
where (gg.date_added > v_Last_Archive
OR gg.date_modified > v_Last_Archive)
and cm.dtcreated- gg.date_added> v_Datediff
group by crsmain_pk1) GG on GG.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(qa.pk1) QA
from qti_asi_data qa
left join course_main cm on cm.pk1 = qa.crsmain_pk1
where (bbmd_date_added > v_Last_Archive
OR bbmd_date_modified > v_Last_Archive)
and cm.dtcreated- qa.bbmd_date_added> v_Datediff
group by crsmain_pk1) QA on QA.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(a.pk1) A
from attempt a
left join gradebook_grade gg on a.pk1 = gg.last_attempt_pk1
left join gradebook_main gm on gm.pk1 = gg.gradebook_main_pk1
left join course_main cm on cm.pk1 = gm.crsmain_pk1
where (a.attempt_date > v_Last_Archive
or a.date_modified > v_Last_Archive)
and cm.dtcreated- a.attempt_date > v_Datediff
group by crsmain_pk1) A on A.crsmain_pk1 = cm.pk1
left join (select co.owner_pk1, COUNT(CON.pk1) CONFERENCE_MAIN
from conference_owner CO
left join conference_main CON on CON.conference_owner_pk1 = CO.pk1
left join course_main cm on cm.pk1 = co.owner_pk1
where co.owner_table = 'COURSE_MAIN'
and (CON.dtcreated > v_Last_Archive
OR CON.dtmodified > v_Last_Archive)
and cm.dtcreated-con.dtcreated > v_Datediff
group by owner_pk1) ConM on ConM.OWNER_PK1 = cm.pk1
left join (select co.owner_pk1, COUNT(fr.pk1) FORUM_MAIN
from conference_owner CO
left join conference_main CON on CON.conference_owner_pk1 = CO.pk1
left join forum_main fr on fr.confmain_pk1 = CON.pk1
left join course_main cm on cm.pk1 = co.owner_pk1
where owner_table = 'COURSE_MAIN'
and (fr.dtcreated > v_Last_Archive
OR fr.dtmodified > v_Last_Archive)
and cm.dtcreated- fr.dtcreated > v_Datediff
group by owner_pk1) FM on FM.OWNER_PK1 = cm.pk1
left join (select co.owner_pk1, COUNT(mm.pk1) MSG_MAIN
from conference_owner CO
left join conference_main CON on CON.conference_owner_pk1 = CO.pk1
left join forum_main fr on fr.confmain_pk1 = CON.pk1
left join msg_main mm on mm.forummain_pk1 = fr.pk1
left join msg_main mm_reply on mm_reply.msgmain_pk1 = mm.pk1
left join course_main cm on cm.pk1 = co.owner_pk1
where owner_table = 'COURSE_MAIN'
and (mm.dtcreated > v_Last_Archive
OR mm.dtmodifi ed > v_Last_Archive)
and cm.dtcreated- mm.dtcreated > v_Datediff
group by owner_pk1) MM on MM.OWNER_PK1 = cm.pk1
left join (select crsmain_pk1, COUNT(c.pk1) C
from calendar c
left join course_main cm on cm.pk1 = c.crsmain_pk1
where c.dtmodified > v_Last_Archive
and cm.dtcreated- c.dtmodified > v_Datediff
group by crsmain_pk1) C on C.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(s.pk1) S
from staffinformation S
left join course_main cm on cm.pk1 = s.crsmain_pk1
where s.dtmodified > v_Last_Archive
and cm.dtcreated- s.dtmodified > v_Datediff
group by crsmain_pk1) S on S.crsmain_pk1 = cm.pk1
where nvl(CC.CC,0) + nvl(GM.GM,0) + nvl(GG.GG,0) + nvl(QA.QA,0) + nvl(A.A,0) + nvl(ConM.CONFERENCE_MAIN,0) + nvl(FM.FORUM_MAIN,0) + nvl(MM.MSG_MAIN,0) + nvl(S.S,0) + nvl(C.C,0) >= v_Number_of_New_Entities )
LOOP
dbms_output.put_line (rec.output);
END LOOP;
dbms_output.new_line( );
end;
56. Expense and Budget Management
Database File Size
SELECT
DB_NAME(database_id) AS DatabaseName,
Name AS Logical_Name,
Physical_Name,
(size*8)/1024 SizeMB
FROM sys.master_files
order by SizeMB desc
57. Expense and Budget Management
Database File Size
select (select sum(bytes)/1024/1024 from dba_data_files)+
(select sum(bytes)/1024/1024 from dba_temp_files) "Size in MB" from dual;
58. Expense and Budget Management
Table Row Count and Sizes
select IDENTITY( int ) AS pk1, name into Table_List from sys.tables
CREATE TABLE Table_Results
(name nvarchar(256),
rows int,
reserved nvarchar(256),
data nvarchar(256),
index_size nvarchar(256),
unused nvarchar(256))
declare @pk1 int
declare @table nvarchar(256)
declare @end int
set @pk1 = 0
set @end = (select top 1 PK1 from Table_List order by pk1 Desc)
WHILE @pk1 < @end
begin
set @pk1 = @pk1 + 1
set @table = (select name from Table_List where pk1 = @pk1)
insert into Table_Results
exec sp_spaceused @table
end
select *
from table_results
drop table table_list
drop table table_results
59. Expense and Budget Management
Table Row Count and Sizes
Select OWNER, TABLE_NAME, TABLESPACE_NAME, NUM_ROWS, PCT_FREE , PCT_USED
, (blocks*8/1024,2) as MB
from dba_tables
order by OWNER, TABLESPACE_NAME, TABLE_NAME;
60. Expense and Budget Management
Table growth rate (by year)
/* depending on the table, the dtcreated could need to be changed to timestamp */
select YEAR(dtcreated), COUNT(*)
from course_contents --Enter the table name here
group by YEAR(dtcreated)
61. Expense and Budget Management
Table growth rate (by year)
select TO_char(dtcreated,'YYYY') , COUNT(*)
from course_contents --Enter the table name here
group by TO_char(dtcreated,'YYYY') ;
62. Expense and Budget Management
Table growth rate (by month)
/* depending on the table, the dtcreated could need to be changed to timestamp */
select YEAR(dtcreated), MONTH(dtcreated), COUNT(*)
from course_main --Enter the table name here
group by YEAR(dtcreated), MONTH(dtcreated)
63. Expense and Budget Management
Table growth rate (by month)
select TO_char(dtcreated,'YYYY'), TO_char(dtcreated,'MM') , COUNT(*)
from course_contents --Enter the table name here
group by TO_char(dtcreated,'YYYY'), TO_char(dtcreated,'MM')
order by TO_char(dtcreated,'YYYY'), TO_char(dtcreated,'MM') ;
64. Strategic Functional Decisions
What is the percent of enrollments from the previous term that have
completed a test online?
declare @start_date datetime
declare @end_date datetime
declare @TotalPopulation int
declare @TestingPopulation int
set @start_date = '2010-9-1 00:00:00:000'
set @end_date = '2011-9-1 00:00:00:000'
set @TotalPopulation = (select Count(distinct users_pk1)
from course_users cu
left join course_main cm on cm.pk1 = cu.crsmain_pk1
where role = 'S'
and dtcreated > @start_date
and dtcreated < @end_date)
set @TestingPopulation = (select Count(Distinct cu.users_pk1)
from course_users cu
left join gradebook_grade gg on gg.course_users_pk1 = cu.pk1
left join gradebook_main gm on gg.gradebook_main_pk1 = gm.pk1
left join course_main cm on cm.pk1 = cu.crsmain_pk1
where gm.qti_asi_data_pk1 > 0
and gg.first_attempt_pk1 > 0
and dtcreated > @start_date
and dtcreated < @end_date)
select @TestingPopulation as Online_Testing, @TotalPopulation - @TestingPopulation as Offline_Testing
65. Strategic Functional Decisions
What is the percent of enrollments from the previous term that have
completed a test online?
set serveroutput on size 100000
declare v_start_date date;
v_end_date date;
v_TotalPopulation int;
v_TestingPopulation int;
Begin
dbms_output.enable(9999999);
dbms_output.new_line();
v_start_date := to_date('1-JAN-2010');
v_end_date := to_date('1-JAN-2012');
select Count(distinct users_pk1) into v_TotalPopulation
from course_users cu
left join course_main cm on cm.pk1 = cu.crsmain_pk1
where role = 'S'
and dtcreated > v_start_date
and dtcreated < v_end_date;
select Count(Distinct cu.users_pk1) into v_TestingPopulation
from course_users cu
left join gradebook_grade gg on gg.course_users_pk1 = cu.pk1
left join gradebook_main gm on gg.gradebook_main_pk1 = gm.pk1
left join course_main cm on cm.pk1 = cu.crsmain_pk1
where gm.qti_asi_data_pk1 > 0
and gg.first_attempt_pk1 > 0
and dtcreated > v_start_date
and dtcreated < v_end_date;
dbms_output.put_line (v_TestingPopulation || ',' || to_char(v_TotalPopulation - v_TestingPopulation));
--select v_TestingPopulation as Online_Testing, v_TotalPopulation - v_TestingPopulation as Offline_Testing;
end;
66. Strategic Functional Decisions
How many tests have been completed per enrollment?
declare @enrollment_start datetime
declare @enrollment_end datetime
set @enrollment_start = '2010-9-1 00:00:00:000'
set @enrollment_end = '2011-9-1 00:00:00:000'
select NumberOfTests, COUNT(NumberOfTests) NumberOfEnrollments
into #BinTests
from (
select AllStudents.users_pk1, NumberOfTests = Case
When TestingStudents.TheCount IS NULL THEN 0
When TestingStudents.TheCount > 0 THEN TestingStudents.TheCount
END
FROM (
select distinct users_pk1
from course_users
where role = 'S'
and enrollment_date > @enrollment_start
and enrollment_date < @enrollment_end) AllStudents
left join(
select cu.users_pk1, COUNT(gg.pk1) as TheCount
from course_users cu
left join gradebook_grade gg on gg.course_users_pk1 = cu.pk1
left join gradebook_main gm on gg.gradebook_main_pk1 = gm.pk1
where gm.qti_asi_data_pk1 > 0
and gg.first_attempt_pk1 > 0
and cu.enrollment_date > @enrollment_start
and cu.enrollment_date < @enrollment_end
group by cu.users_pk1) TestingStudents
on AllStudents.users_pk1 = TestingStudents.users_pk1) TestBin
group by NumberOfTests
declare @start int
declare @end int
declare @enrollments int
set @start = 0
set @end = (Select top 1 NumberOfTests from #BinTests order by NumberOfTests desc)
create table #Results(
Number_Of_Tests int,
Number_Of_Enrollments int)
While @start-1 < @end
Begin
set @enrollments = (select NumberOfEnrollments from #BinTests where NumberOfTests = @start)
insert into #Results
(Number_of_Tests, Number_Of_Enrollments)
values
(@start, @enrollments)
set @start = @start + 1
end
select Number_Of_Tests, Number_Of_Enrollments = Case
When Number_Of_Enrollments IS NULL THEN 0
When Number_Of_Enrollments >= 0 THEN Number_Of_Enrollments
End
from #Results
drop table #BinTests
drop table #Results
Editor's Notes
What do you think about a slide that has a picture of one of us looking down and one of us looking up at each other? The way it’s presented right now is kind of boring. It’s like going in for a colonoscopy. Next to our pictures, we would have a quick some nice looking bullets (visuals) that say you are the SPE for Outcomes and Community system, as well as a point about you being one of them.We could have a bubble image coming out of our pictures as though we are thinking some funny thought in our head.These sets of slides have to get our audience comfortable with us. We need to project a wacky or fun persona to get them comfortable. Think 2 Wacky and Crazy guys from SNL.
You have 4 goals right? So why not say this with 4 images and no words? We can keep this as is for the submission to legal.
You might even want to suggest a tool to use to capture the data. A good open source tool is Aquadata Studio. A great commercial tool is Quest Software’s Toad. We kind of need to mention Toad because Quest sponsors our team. We might even want to put a picture of Toad and Aquafold. You might even want to go so far as to reference the Aqua Data Server which is a repository for storing a lot of this information and doing reporting.http://www.aquafold.com
http://library.blackboard.com/ref/fea8bb86-1720-4b14-bb81-542d973b5338/index.htmAny time we use a link, put the smaller version using a tool like bit.ly or goo.gl or even tiny.urlWe might even want to capture a quick snapshot image of the Documentation intro or something…
capacity planning*The act of studying historical data with the intent of identifying repeating patterns and applying the repeating patterns to the decisions that impact the future of the studied system.
(Geoff) Going to make the clip are semi-transparent. Don’t now Geoff, I think this section is missing some slides. We had discussed presenting a process, which you kind of start with the next slides, but there’s not a lot of focus. I suggest that we add a visual slide next. The visual should present a recommendation from us about how to go through the Capacity Planning process:Asking the Right QuestionsCollecting the Right DataOrganizing and Analyzing the DataMaking Decisions from the DataThis process should be very circular in nature in which we always come back to “Asking the right questions!”
We need to change up this slide in some way. It makes it seem like the slide says there are only 3 focus areas of Capacity Planning. How can we say that these are the 3 areas we will focus on, but that there are many other areas of focus?I suggest we do this in a visual. Since it’s 3 areas of focus, we could use a triangle image. (Geoff) I think a triangle image would suggest that the 3 areas are related in some way other than the database and the capacity planning topic.
Downtime (Cold)Traditional MaintenancePatchesFirmwareCold backupMajor upgradesBb 9.1Hardware or software purchasesNo Downtime (Hot)IntegrationSnapshotSISBatch FileMajority of backup processesSQL JobsEach section should have a slide for each of these 4 points. We can obviously have more than 4, but these 4 points need to make the slides.
Frequency of course archive task?Archive all courses or only courses that have been modified?How do we define a modified course?Will the archiving process impact user experience?Does the course archive tool have limitation?Number of entities with in a course (grade book, discussion board replies, etc)Types of content?Location of course archives?Size limitations?Retrieval process?Naming convention?
(Geoff) Graph Slide with query link
This section needs 4 slides about these points…Asking the Right QuestionsCollecting the Right DataOrganizing and Analyzing the DataMaking Decisions from the Data
OperationalBackup & RecoveryCapitalHardwareStorageCPUMemoryI/OSoftwareLicensesMultiple CPUsUnsupported systems
This section needs 4 slides about these points…Asking the Right QuestionsCollecting the Right DataOrganizing and Analyzing the DataMaking Decisions from the Data