Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Bb world 2011 capacity planning
1. Using Database
Statistics to Make
Capacity Planning
Decisions
Geoff Mower
Senior Software Performance Engineer, Blackboard
Steve Feldman
Senior Director Software Performance Engineering and Architecture, Blackboard
2. Who Are We?
Geoff Mower- Senior Software Performance Engineer
5 years on the frontlines as a DBA
2 hour interview on a plane
Performance and Scalability for
Outcomes & Community
3. Who Are We?
Steve Feldman
• Blackboard since 2003
• Performance Engineering
from the start
• Platform Architecture in
2005
• Security Engineering in
2010
@seven_seconds
“Love my job…love my
team. If you email me, I will
http://goo.gl/Z4Rq5
respond.”
4. What We Want You to Do?
ask questions Mine your data Ask follow-up questions
about your
data
5. What are our obstacles?
• Certain data points are transient
– Logging will need to be created in certain situations
to fill in missing data points.
• See slide #45 & #46 for an example
6. What You Need
• Permissions to run SQL scripts on your
Blackboard Learn environment's database
• Basic SQL fundamentals
• Analysis of results
• Database Tools
– Aquadata client & Server or Quest’s Toad
7. Skill and Knowledge Sets
Open DB Documentation
• Blackboard Learn™, Release 9.1 Open
Database Document
– http://bit.ly/loW9hi
8.
9. Asking the Right Questions
Collecting the Right Data
Organizing and Analyzing the Data Making Decisions
Ask Follow-up Questions
10. Types of Data
Data Types Table Example
ACTIVITY_ACCUMULATOR User navigation
Event
COURSE_USERS snapshot
Process
COURSE_CONTENT User created
User entities
LEARN_admin.bb_instance installation
Static
SESSIONS Tracks who has
Transient an active session
11. Areas of Capacity Planning
• How can the Blackboard Learn schema help you
capacity plan for your business requirements in
regards to?
Administrative Planning Strategic Functional Decisions
Expense and Budget Management
13. Administrative Planning
To be able to restore individual courses, can we use the
course archive tool as a backup function?
14. Administrative Planning
Asking the Right Questions
How will we know which courses need archiving?
Frequency of course archive task?
Will the archiving process impact user experience?
Does the course archive tool have limitation?
Location of course archives?
15. Administrative Planning
Collecting the Right Data
How will we know which courses need archiving?
– Course entity creation and modification
Creation Tables Modification Tables
COURSE_CONTENTS COURSE_CONTENTS
GRADEBOOK_MAIN GRADEBOOK_MAIN
GRADEBOOK_GRADES GRADEBOOK_GRADES
QTI_ASI_DATA QTI_ASI_DATA
QTI_RESULT_DATA QTI_RESULT_DATA
FORUM_MAIN FORUM_MAIN
MSG_MAIN MSG_MAIN
CALENDAR
STAFFINFORMATION
16. Administrative Planning
Organizing and Analyzing the Data
• Data Points: Course/Organizations to be
archived SQL/Oracle
COURSE_ID Number of New/Modified Course Entities
Hist_101 42
Eng_252 42
Sci_139 38
Eng_429 37
Art_202 35
Eco_323 35
17. Administrative Planning
Making Decisions from the Data
• How many new or modified course/organization
entities are required for a course/organization to
be archived?
• How many courses/organizations should we
archive each time the archive task is execution?
• What is the maximum time between archive
tasks?
18. Administrative Planning
Acting on Your Descion
SQL/Oracle
Hist_101, c:Course_Archives2011-07-09
Eng_252, c:Course_Archives2011-07-09
Sci_139, c:Course_Archives2011-07-09
Eng_429, c:Course_Archives2011-07-09
…
• Create a JOB that would run the SQL script to generate
a txt file containing the query results
• Create a schedule task that would run the
batch_ImportExport.bat tool and direct it to the txt file
created by the JOB in the previous step.
20. Asking the Right Questions
Expense and Budget Management
• How much do we need to budget for in order to fund our
database requirements?
21. Expense and Budget Management
• When are we going to hit our 95% disk spaced used
threshold.
22. Expense and Budget Management
• Available disk space issue
– Talk to Network Security Manager (owner of the SAN)
• We need more disk space
23. Expense and Budget Management
Asking the Right Questions
• What did I miss?
– When will we run out of space?
– How long do we expect the requested disk space to
last us?
• Facts vs. guess work
24. Expense and Budget Management
Collecting the Right Data
• Data Points: Database file size
• Binary
• Data Files (.mdf)
• TempDB
• Transaction Logs (.ldf)
• Back-up/restore process
25. Expense and Budget Management
Organizing and Analyzing the Data
• Data Points: Database file size SQL/Oracle
Database Name Physical Name Size in MB
BBLEARN_stats F:dataBBLEARN_stats_data.mdf 70338
BBLEARN F:dataBBLEARN_data.mdf 45520
cms_files_inst F:datacms_files_inst_data.mdf 5704
BBLEARN G:logsBBLEARN_log.ldf 3083
cms_files_inst G:logscms_files_inst_log.ldf 1361
cms_files_courses F:datacms_files_courses_data.mdf 1072
cms_files_users F:datacms_files_users_data.mdf 926
msdb F:dataMSSQL.1MSSQLDATAMSDBData.mdf 792
cms_files_library F:datacms_files_library_data.mdf 720
cms_files_orgs F:datacms_files_orgs_data.mdf 384
msdb F:dataMSSQL.1MSSQLDATAMSDBLog.ldf 259
cms_files_users G:logscms_files_users_log.ldf 214
cms_files_courses G:logscms_files_courses_log.ldf 207
26. Expense and Budget Management
Organizing and Analyzing the Data
• Data Points: Table Row Count and Sizes SQL/Oracle
Row
Table Name Count Reserved Data Index Size Unused
ACTIVITY_ACCUMULATOR 377266616 84097056 46905888 37183888 7280
COURSE_USERS 3837931 496024 258272 237496 256
USERS 303687 139480 96848 42440 192
COURSE_MAIN 121623 86024 47760 37984 280
NAVIGATION_ITEM 1876 2032 608 1128 296
SYSTEM_TRACKING 3550 1936 1672 160 104
USER_ROLES 4114 544 160 288 96
27. Expense and Budget Management
Organizing and Analyzing the Data
Data Points: Table growth rate (by year) of the
Activity_Accumulator SQL/Oracle
12000000
10000000
80000000
60000000
40000000
20000000
0
2003 2004 2005 2006 2007 2008 2009 2010
28.
29. Expense and Budget Management
Organizing and Analyzing the Data
Data Points: ACTIVITY_ACCUMULATOR growth rate (by
month) SQL/Oracle
12000000
10000000
8000000
6000000
4000000
2000000
0
30. Expense and Budget Management
• Data Points: COURSE_USERS table growth rate (by
month)
900000
800000
700000
600000
500000
400000
300000
200000
100000
0
August
August
August
31. Expense and Budget Management
Making Decisions from the Data
• Conclusion
• Due to opening of school tasks we will see our
available disk space decrease below 5% during the
month of August.
• In order to make it 3 years we will need a minimum
120 GB.
34. Strategic Functional Decisions
Asking the Right Questions
• Study…
• Online & offline test taking populations
• The tests themselves
• Test taking behavior
• How testing impacts the database
• Locating when you will see resource contention
35. Strategic Functional Decisions
Historical Use of Online Testing
What is the percent of students from the previous term that have
completed a test online? SQL/Oracle
Online
Offline
36.
37.
38.
39. Strategic Functional Decisions
Collecting the Right Data
Previous Use Full Adoption
Courses with Tests 11,274 55,504
Enrollments with Tests 225,480 1,110,080
Number of questions 4,219,032 25,694,266
submitted in a term
Peak Question Submission 60,346 297,094
by hour
40. Strategic Functional Decisions
Collecting the Right Data
• Determine CPU and I/O cost of testing (SQL server)
– Track the Stored Procedures performing the works
SQL/Oracle
» qti_result_data_cr
» update_gradebook_grade
» attempt_cr
» update_qti_res_bbmd_dtmod
» gb2_is_log_enabled
» gb2_attempt_logger
41. Strategic Functional Decisions
Organizing and Analyzing the Data
• Determine CPU and I/O cost of testing (SQL server)
– Track the Stored Procedures performing the works SQL/Oracle
42. Strategic Functional Decisions
Organizing and Analyzing the Data
• Determine CPU and I/O cost of testing
– Track the queries performing the work SQL/Oracle
43. Strategic Functional Decisions
Organizing and Analyzing the Data
Determine IO cost of all cached stored procedures SQL/Oracle
Average IO
700
600
500
400
300
200
100
0
44. Strategic Functional Decisions
Making Decisions from the Data
• A reasonable understanding…
• How tests are currently used
• How tests will be used
• How much each question costs the database upon
submission
• Lack information on…
• Baseline of database’s IO and CPU resources
• Non-testing database transactions
45. Strategic Functional Decisions
Collecting the Right Data
• Determine the breaking point of CPU and I/O of testing
– Create a CPU logging script SQL/Oracle
• SQL server logging limitation
– 4 hours of CPU tracking
– Lacks IO counters
– Create a IO logging script SQL/Oracle
• Snapshot in time
– Need to log the delta between the last logged IO metric and current
query results
– Create a Query logging script SQL/Oracle
• Compares test question submission to all other transactions
46. Strategic Functional Decisions
Collecting the Right Data
• Create a separate database from the Blackboard
Learn’s databases (you can use the database server
Blackboard Learn is running on)
• Create tables with in the new database that match
the query results you wish to store
• Create a JOB that will run each query and insert the
results into the corresponding table and database.
47. Capacity Planning
is an art if done
properly…
or a disaster if not
done at all.
50. Please provide feedback for this session by emailing
DevConFeedback@blackboard.com.
The title of this session is:
Using Database Statistics to Make Capacity Planning
Decisions
52. Administrative Planning
What courses meet the course archive criteria
declare @Last_Archive datetime
declare @Datediff int
declare @Number_of_New_Entities int
set @Last_Archive = '2011-6-7' --Enter the last datetime the course archival process ran
set @Datediff = 24 --Enter the time duration between course creation and course entities you would like to track. For example, to avoid archive courses that were created since the last archive process enter the time of your longest running snapshot or SIS process in terms of hours.
set @Number_of_New_Entities = 1 --Enter the number of new course entities required for a course to be archived
select course_id, CC.CC + GM.GM + GG.GG + QA.QA + A.A + ConM.CONFERENCE_MAIN + FM.FORUM_MAIN + MM.MSG_MAIN + C.C + S.S AS [Number of New/Modified Course Entities]
from course_main cm
left join (select crsmain_pk1, COUNT(cc.pk1) CC
from course_contents cc
left join course_main cm on cm.pk1 = cc.crsmain_pk1
where (cc.dtcreated > @Last_Archive
or cc.dtmodified > @Last_Archive)
and datediff(hour, cm.dtcreated, cc.dtcreated) > @Datediff
group by crsmain_pk1) CC on CC.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(gm.pk1) GM
from gradebook_main gm
left join course_main cm on cm.pk1 = gm.crsmain_pk1
where (date_added > @Last_Archive
or date_modified > @Last_Archive)
and datediff(hour, cm.dtcreated, gm.date_added) > @Datediff
group by crsmain_pk1) GM on GM.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(gg.pk1) GG
from gradebook_main gm
left join gradebook_grade gg on gm.pk1 = gg.gradebook_main_pk1
left join course_main cm on gm.crsmain_pk1 = cm.pk1
where (gg.date_added > @Last_Archive
OR gg.date_modified > @Last_Archive)
and datediff(hour, cm.dtcreated, gg.date_added) > @Datediff
group by crsmain_pk1) GG on GG.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(qa.pk1) QA
from qti_asi_data qa
left join course_main cm on cm.pk1 = qa.crsmain_pk1
where (bbmd_date_added > @Last_Archive
OR bbmd_date_modified > @Last_Archive)
and datediff(hour, cm.dtcreated, qa.bbmd_date_added) > @Datediff
group by crsmain_pk1) QA on QA.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(a.pk1) A
from attempt a
left join gradebook_grade gg on a.pk1 = gg.last_attempt_pk1
left join gradebook_main gm on gm.pk1 = gg.gradebook_main_pk1
left join course_main cm on cm.pk1 = gm.crsmain_pk1
where (a.attempt_date > @Last_Archive
or a.date_modified > @Last_Archive)
and datediff(hour, cm.dtcreated, a.attempt_date) > @Datediff
group by crsmain_pk1) A on A.crsmain_pk1 = cm.pk1
left join (select co.owner_pk1, COUNT(CON.pk1) CONFERENCE_MAIN
from conference_owner CO
left join conference_main CON on CON.conference_owner_pk1 = CO.pk1
left join course_main cm on cm.pk1 = co.owner_pk1
where co.owner_table = 'COURSE_MAIN'
and (CON.dtcreated > @Last_Archive
OR CON.dtmodified > @Last_Archive)
and datediff(hour, cm.dtcreated, con.dtcreated) > @Datediff
group by owner_pk1) ConM on ConM.OWNER_PK1 = cm.pk1
left join (select co.owner_pk1, COUNT(fr.pk1) FORUM_MAIN
from conference_owner CO
left join conference_main CON on CON.conference_owner_pk1 = CO.pk1
left join forum_main fr on fr.confmain_pk1 = CON.pk1
left join course_main cm on cm.pk1 = co.owner_pk1
where owner_table = 'COURSE_MAIN'
and (fr.dtcreated > @Last_Archive
OR fr.dtmodified > @Last_Archive)
and datediff(hour, cm.dtcreated, fr.dtcreated) > @Datediff
group by owner_pk1) FM on FM.OWNER_PK1 = cm.pk1
left join (select co.owner_pk1, COUNT(mm.pk1) MSG_MAIN
from conference_owner CO
left join conference_main CON on CON.conference_owner_pk1 = CO.pk1
left join forum_main fr on fr.confmain_pk1 = CON.pk1
left join msg_main mm on mm.forummain_pk1 = fr.pk1
left join msg_main mm_reply on mm_reply.msgmain_pk1 = mm.pk1
left join course_main cm on cm.pk1 = co.owner_pk1
where owner_table = 'COURSE_MAIN'
and (mm.dtcreated > @Last_Archive
OR mm.dtmodified > @Last_Archive)
and datediff(hour, cm.dtcreated, mm.dtcreated) > @Datediff
group by owner_pk1) MM on MM.OWNER_PK1 = cm.pk1
left join (select crsmain_pk1, COUNT(c.pk1) C
from calendar c
left join course_main cm on cm.pk1 = c.crsmain_pk1
where c.dtmodified > @Last_Archive
and datediff(hour, cm.dtcreated, c.dtmodified) > @Datediff
group by crsmain_pk1) C on C.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(s.pk1) S
from staffinformation S
left join course_main cm on cm.pk1 = s.crsmain_pk1
where s.dtmodified > @Last_Archive
and datediff(hour, cm.dtcreated, s.dtmodified) > @Datediff
group by crsmain_pk1) S on S.crsmain_pk1 = cm.pk1
53. Administrative Planning
What courses meet the course archive criteria
set serveroutput on size 100000
---Administrative Planning
What courses meet the course archive criteria
declare
v_Last_Archive date;
v_Datediff int;
v_Number_of_New_Entities int;
v_Directory varchar(256);
v_File_Name varchar(256);
Begin
v_Last_Archive := TO_DATE('7-JUN-2011'); --Enter the last datetime the course archival process ran
v_Datediff := 1 ; --this is the day
--Enter the time duration between course creation and course entities you would like to track. For example, to avoid archive courses that were created ----since the last archive process enter the time of your longest running snapshot or SIS process in terms of hours.
v_Number_of_New_Enti ties := 1;
--Enter the number of new course entities required for a course to be archived
select 'usrlocalblackboardCourse_Archi ves ' | | TO_CHAR(SYSDATE, 'DD-Mon-YYYY') | | '' into v_Directory FROM Dual;
--Alter the string to build the directory you wish to store your archives
dbms_output.enable(9999999);
dbms_output.new_line( );
FOR rec IN (
select course_id | | v_Directory as output
from course_main cm
left join (select crsmain_pk1, COUNT(cc.pk1) CC
from course_contents cc
left join course_main cm on cm.pk1 = cc.crsmain_pk1
where (cc.dtcreated > v_Last_Archive
or cc.dtmodified > v_Last_Archive)
and cm.dtcreated- cc.dtcreated> v_Datediff
group by crsmain_pk1) CC on CC.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(gm.pk1) GM
from gradebook_main gm
left join course_main cm on cm.pk1 = gm.crsmain_pk1
where (date_added > v_Last_Archive
or date_modified > v_Last_Archive)
and cm.dtcreated- gm.date_added> v_Datediff
group by crsmain_pk1) GM on GM.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(gg.pk1) GG
from gradebook_main gm
left join gradebook_grade gg on gm.pk1 = gg.gradebook_main_pk1
left join course_main cm on gm.crsmain_pk1 = cm.pk1
where (gg.date_added > v_Last_Archive
OR gg.date_modified > v_Last_Archive)
and cm.dtcreated- gg.date_added> v_Datediff
group by crsmain_pk1) GG on GG.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(qa.pk1) QA
from qti_asi_data qa
left join course_main cm on cm.pk1 = qa.crsmain_pk1
where (bbmd_date_added > v_Last_Archive
OR bbmd_date_modified > v_Last_Archive)
and cm.dtcreated- qa.bbmd_date_added> v_Datediff
group by crsmain_pk1) QA on QA.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(a.pk1) A
from attempt a
left join gradebook_grade gg on a.pk1 = gg.last_attempt_pk1
left join gradebook_main gm on gm.pk1 = gg.gradebook_main_pk1
left join course_main cm on cm.pk1 = gm.crsmain_pk1
where (a.attempt_date > v_Last_Archive
or a.date_modified > v_Last_Archive)
and cm.dtcreated- a.attempt_date > v_Datediff
group by crsmain_pk1) A on A.crsmain_pk1 = cm.pk1
left join (select co.owner_pk1, COUNT(CON.pk1) CONFERENCE_MAIN
from conference_owner CO
left join conference_main CON on CON.conference_owner_pk1 = CO.pk1
left join course_main cm on cm.pk1 = co.owner_pk1
where co.owner_table = 'COURSE_MAIN'
and (CON.dtcreated > v_Last_Archive
OR CON.dtmodified > v_Last_Archive)
and cm.dtcreated-con.dtcreated > v_Datediff
group by owner_pk1) ConM on ConM.OWNER_PK1 = cm.pk1
left join (select co.owner_pk1, COUNT(fr.pk1) FORUM_MAIN
from conference_owner CO
left join conference_main CON on CON.conference_owner_pk1 = CO.pk1
left join forum_main fr on fr.confmain_pk1 = CON.pk1
left join course_main cm on cm.pk1 = co.owner_pk1
where owner_table = 'COURSE_MAIN'
and (fr.dtcreated > v_Last_Archive
OR fr.dtmodified > v_Last_Archive)
and cm.dtcreated- fr.dtcreated > v_Datediff
group by owner_pk1) FM on FM.OWNER_PK1 = cm.pk1
left join (select co.owner_pk1, COUNT(mm.pk1) MSG_MAIN
from conference_owner CO
left join conference_main CON on CON.conference_owner_pk1 = CO.pk1
left join forum_main fr on fr.confmain_pk1 = CON.pk1
left join msg_main mm on mm.forummain_pk1 = fr.pk1
left join msg_main mm_reply on mm_reply.msgmain_pk1 = mm.pk1
left join course_main cm on cm.pk1 = co.owner_pk1
where owner_table = 'COURSE_MAIN'
and (mm.dtcreated > v_Last_Archive
OR mm.dtmodifi ed > v_Last_Archive)
and cm.dtcreated- mm.dtcreated > v_Datediff
group by owner_pk1) MM on MM.OWNER_PK1 = cm.pk1
left join (select crsmain_pk1, COUNT(c.pk1) C
from calendar c
left join course_main cm on cm.pk1 = c.crsmain_pk1
where c.dtmodified > v_Last_Archive
and cm.dtcreated- c.dtmodified > v_Datediff
group by crsmain_pk1) C on C.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(s.pk1) S
from staffinformation S
left join course_main cm on cm.pk1 = s.crsmain_pk1
where s.dtmodified > v_Last_Archive
and cm.dtcreated- s.dtmodified > v_Datediff
group by crsmain_pk1) S on S.crsmain_pk1 = cm.pk1
where nvl(CC.CC,0) + nvl(GM.GM,0) + nvl(GG.GG,0) + nvl(QA.QA,0) + nvl(A.A,0) + nvl(ConM.CONFERENCE_MAIN,0) + nvl(FM.FORUM_MAIN,0) + nvl(MM.MSG_MAIN,0) + nvl(S.S,0) + nvl(C.C,0) >= v_Number_of_New_Entities )
LOOP
dbms_output.put_line (rec.output);
END LOOP;
dbms_output.new_line( );
end;
54. Administrative Planning
What courses meet the course archive criteria
declare @Last_Archive datetime
declare @Datediff int
declare @Number_of_New_Entities int
declare @Directory varchar(256)
declare @File_Name varchar(256)
set @Last_Archive = '2011-6-7' --Enter the last datetime the course archival process ran
set @Datediff = 24 --Enter the time duration between course creation and course entities you would like to track. For example, to avoid archive courses that were created since the last archive process enter the time of your longest running snapshot or SIS process in terms of hours.
set @Number_of_New_Entities = 1 --Enter the number of new course entities required for a course to be archived
set @Directory = (SELECT ', c:Course_Archives' + CAST(CONVERT(date, GETDATE()) as varchar(10)) + '') --Alter the string to build the directory you wish to store your archives
select course_id + @Directory
from course_main cm
left join (select crsmain_pk1, COUNT(cc.pk1) CC
from course_contents cc
left join course_main cm on cm.pk1 = cc.crsmain_pk1
where (cc.dtcreated > @Last_Archive
or cc.dtmodified > @Last_Archive)
and datediff(hour, cm.dtcreated, cc.dtcreated) > @Datediff
group by crsmain_pk1) CC on CC.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(gm.pk1) GM
from gradebook_main gm
left join course_main cm on cm.pk1 = gm.crsmain_pk1
where (date_added > @Last_Archive
or date_modified > @Last_Archive)
and datediff(hour, cm.dtcreated, gm.date_added) > @Datediff
group by crsmain_pk1) GM on GM.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(gg.pk1) GG
from gradebook_main gm
left join gradebook_grade gg on gm.pk1 = gg.gradebook_main_pk1
left join course_main cm on gm.crsmain_pk1 = cm.pk1
where (gg.date_added > @Last_Archive
OR gg.date_modified > @Last_Archive)
and datediff(hour, cm.dtcreated, gg.date_added) > @Datediff
group by crsmain_pk1) GG on GG.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(qa.pk1) QA
from qti_asi_data qa
left join course_main cm on cm.pk1 = qa.crsmain_pk1
where (bbmd_date_added > @Last_Archive
OR bbmd_date_modified > @Last_Archive)
and datediff(hour, cm.dtcreated, qa.bbmd_date_added) > @Datediff
group by crsmain_pk1) QA on QA.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(a.pk1) A
from attempt a
left join gradebook_grade gg on a.pk1 = gg.last_attempt_pk1
left join gradebook_main gm on gm.pk1 = gg.gradebook_main_pk1
left join course_main cm on cm.pk1 = gm.crsmain_pk1
where (a.attempt_date > @Last_Archive
or a.date_modified > @Last_Archive)
and datediff(hour, cm.dtcreated, a.attempt_date) > @Datediff
group by crsmain_pk1) A on A.crsmain_pk1 = cm.pk1
left join (select co.owner_pk1, COUNT(CON.pk1) CONFERENCE_MAIN
from conference_owner CO
left join conference_main CON on CON.conference_owner_pk1 = CO.pk1
left join course_main cm on cm.pk1 = co.owner_pk1
where co.owner_table = 'COURSE_MAIN'
and (CON.dtcreated > @Last_Archive
OR CON.dtmodified > @Last_Archive)
and datediff(hour, cm.dtcreated, con.dtcreated) > @Datediff
group by owner_pk1) ConM on ConM.OWNER_PK1 = cm.pk1
left join (select co.owner_pk1, COUNT(fr.pk1) FORUM_MAIN
from conference_owner CO
left join conference_main CON on CON.conference_owner_pk1 = CO.pk1
left join forum_main fr on fr.confmain_pk1 = CON.pk1
left join course_main cm on cm.pk1 = co.owner_pk1
where owner_table = 'COURSE_MAIN'
and (fr.dtcreated > @Last_Archive
OR fr.dtmodified > @Last_Archive)
and datediff(hour, cm.dtcreated, fr.dtcreated) > @Datediff
group by owner_pk1) FM on FM.OWNER_PK1 = cm.pk1
left join (select co.owner_pk1, COUNT(mm.pk1) MSG_MAIN
from conference_owner CO
left join conference_main CON on CON.conference_owner_pk1 = CO.pk1
left join forum_main fr on fr.confmain_pk1 = CON.pk1
left join msg_main mm on mm.forummain_pk1 = fr.pk1
left join msg_main mm_reply on mm_reply.msgmain_pk1 = mm.pk1
left join course_main cm on cm.pk1 = co.owner_pk1
where owner_table = 'COURSE_MAIN'
and (mm.dtcreated > @Last_Archive
OR mm.dtmodified > @Last_Archive)
and datediff(hour, cm.dtcreated, mm.dtcreated) > @Datediff
group by owner_pk1) MM on MM.OWNER_PK1 = cm.pk1
left join (select crsmain_pk1, COUNT(c.pk1) C
from calendar c
left join course_main cm on cm.pk1 = c.crsmain_pk1
where c.dtmodified > @Last_Archive
and datediff(hour, cm.dtcreated, c.dtmodified) > @Datediff
group by crsmain_pk1) C on C.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(s.pk1) S
from staffinformation S
left join course_main cm on cm.pk1 = s.crsmain_pk1
where s.dtmodified > @Last_Archive
and datediff(hour, cm.dtcreated, s.dtmodified) > @Datediff
group by crsmain_pk1) S on S.crsmain_pk1 = cm.pk1
where ISNULL(CC.CC,0) + ISNULL(GM.GM,0) + ISNULL(GG.GG,0) + ISNULL(QA.QA,0) + ISNULL(A.A,0) + ISNULL(ConM.CONFERENCE_MAIN,0) + ISNULL(FM.FORUM_MAIN,0) + ISNULL(MM.MSG_MAIN,0) + ISNULL(S.S,0) + ISNULL(C.C,0) >= @Number_of_New_Entities
55. Administrative Planning
What courses meet the course archive criteria
set serveroutput on size 100000
---Administrative Planning
What courses meet the course archive criteria
declare
v_Last_Archive date;
v_Datediff int;
v_Number_of_New_Entities int;
v_Directory varchar(256);
v_File_Name varchar(256);
Begin
v_Last_Archive := TO_DATE('7-JUN-2011'); --Enter the last datetime the course archival process ran
v_Datediff := 1 ; --this is the day
--Enter the time duration between course creation and course entities you would like to track. For example, to avoid archive courses that were created ----since the last archive process enter the time of your longest running snapshot or SIS process in terms of hours.
v_Number_of_New_Enti ties := 1;
--Enter the number of new course entities required for a course to be archived
select 'usrlocalblackboardCourse_Archi ves ' | | TO_CHAR(SYSDATE, 'DD-Mon-YYYY') | | '' into v_Directory FROM Dual;
--Alter the string to build the directory you wish to store your archives
dbms_output.enable(9999999);
dbms_output.new_line( );
FOR rec IN (
select course_id | | v_Directory as output
from course_main cm
left join (select crsmain_pk1, COUNT(cc.pk1) CC
from course_contents cc
left join course_main cm on cm.pk1 = cc.crsmain_pk1
where (cc.dtcreated > v_Last_Archive
or cc.dtmodified > v_Last_Archive)
and cm.dtcreated- cc.dtcreated> v_Datediff
group by crsmain_pk1) CC on CC.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(gm.pk1) GM
from gradebook_main gm
left join course_main cm on cm.pk1 = gm.crsmain_pk1
where (date_added > v_Last_Archive
or date_modified > v_Last_Archive)
and cm.dtcreated- gm.date_added> v_Datediff
group by crsmain_pk1) GM on GM.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(gg.pk1) GG
from gradebook_main gm
left join gradebook_grade gg on gm.pk1 = gg.gradebook_main_pk1
left join course_main cm on gm.crsmain_pk1 = cm.pk1
where (gg.date_added > v_Last_Archive
OR gg.date_modified > v_Last_Archive)
and cm.dtcreated- gg.date_added> v_Datediff
group by crsmain_pk1) GG on GG.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(qa.pk1) QA
from qti_asi_data qa
left join course_main cm on cm.pk1 = qa.crsmain_pk1
where (bbmd_date_added > v_Last_Archive
OR bbmd_date_modified > v_Last_Archive)
and cm.dtcreated- qa.bbmd_date_added> v_Datediff
group by crsmain_pk1) QA on QA.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(a.pk1) A
from attempt a
left join gradebook_grade gg on a.pk1 = gg.last_attempt_pk1
left join gradebook_main gm on gm.pk1 = gg.gradebook_main_pk1
left join course_main cm on cm.pk1 = gm.crsmain_pk1
where (a.attempt_date > v_Last_Archive
or a.date_modified > v_Last_Archive)
and cm.dtcreated- a.attempt_date > v_Datediff
group by crsmain_pk1) A on A.crsmain_pk1 = cm.pk1
left join (select co.owner_pk1, COUNT(CON.pk1) CONFERENCE_MAIN
from conference_owner CO
left join conference_main CON on CON.conference_owner_pk1 = CO.pk1
left join course_main cm on cm.pk1 = co.owner_pk1
where co.owner_table = 'COURSE_MAIN'
and (CON.dtcreated > v_Last_Archive
OR CON.dtmodified > v_Last_Archive)
and cm.dtcreated-con.dtcreated > v_Datediff
group by owner_pk1) ConM on ConM.OWNER_PK1 = cm.pk1
left join (select co.owner_pk1, COUNT(fr.pk1) FORUM_MAIN
from conference_owner CO
left join conference_main CON on CON.conference_owner_pk1 = CO.pk1
left join forum_main fr on fr.confmain_pk1 = CON.pk1
left join course_main cm on cm.pk1 = co.owner_pk1
where owner_table = 'COURSE_MAIN'
and (fr.dtcreated > v_Last_Archive
OR fr.dtmodified > v_Last_Archive)
and cm.dtcreated- fr.dtcreated > v_Datediff
group by owner_pk1) FM on FM.OWNER_PK1 = cm.pk1
left join (select co.owner_pk1, COUNT(mm.pk1) MSG_MAIN
from conference_owner CO
left join conference_main CON on CON.conference_owner_pk1 = CO.pk1
left join forum_main fr on fr.confmain_pk1 = CON.pk1
left join msg_main mm on mm.forummain_pk1 = fr.pk1
left join msg_main mm_reply on mm_reply.msgmain_pk1 = mm.pk1
left join course_main cm on cm.pk1 = co.owner_pk1
where owner_table = 'COURSE_MAIN'
and (mm.dtcreated > v_Last_Archive
OR mm.dtmodifi ed > v_Last_Archive)
and cm.dtcreated- mm.dtcreated > v_Datediff
group by owner_pk1) MM on MM.OWNER_PK1 = cm.pk1
left join (select crsmain_pk1, COUNT(c.pk1) C
from calendar c
left join course_main cm on cm.pk1 = c.crsmain_pk1
where c.dtmodified > v_Last_Archive
and cm.dtcreated- c.dtmodified > v_Datediff
group by crsmain_pk1) C on C.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(s.pk1) S
from staffinformation S
left join course_main cm on cm.pk1 = s.crsmain_pk1
where s.dtmodified > v_Last_Archive
and cm.dtcreated- s.dtmodified > v_Datediff
group by crsmain_pk1) S on S.crsmain_pk1 = cm.pk1
where nvl(CC.CC,0) + nvl(GM.GM,0) + nvl(GG.GG,0) + nvl(QA.QA,0) + nvl(A.A,0) + nvl(ConM.CONFERENCE_MAIN,0) + nvl(FM.FORUM_MAIN,0) + nvl(MM.MSG_MAIN,0) + nvl(S.S,0) + nvl(C.C,0) >= v_Number_of_New_Entities )
LOOP
dbms_output.put_line (rec.output);
END LOOP;
dbms_output.new_line( );
end;
56. Expense and Budget Management
Database File Size
SELECT
DB_NAME(database_id) AS DatabaseName,
Name AS Logical_Name,
Physical_Name,
(size*8)/1024 SizeMB
FROM sys.master_files
order by SizeMB desc
57. Expense and Budget Management
Database File Size
select (select sum(bytes)/1024/1024 from dba_data_files)+
(select sum(bytes)/1024/1024 from dba_temp_files) "Size in MB" from dual;
58. Expense and Budget Management
Table Row Count and Sizes
select IDENTITY( int ) AS pk1, name into Table_List from sys.tables
CREATE TABLE Table_Results
(name nvarchar(256),
rows int,
reserved nvarchar(256),
data nvarchar(256),
index_size nvarchar(256),
unused nvarchar(256))
declare @pk1 int
declare @table nvarchar(256)
declare @end int
set @pk1 = 0
set @end = (select top 1 PK1 from Table_List order by pk1 Desc)
WHILE @pk1 < @end
begin
set @pk1 = @pk1 + 1
set @table = (select name from Table_List where pk1 = @pk1)
insert into Table_Results
exec sp_spaceused @table
end
select *
from table_results
drop table table_list
drop table table_results
59. Expense and Budget Management
Table Row Count and Sizes
Select OWNER, TABLE_NAME, TABLESPACE_NAME, NUM_ROWS, PCT_FREE , PCT_USED
, (blocks*8/1024,2) as MB
from dba_tables
order by OWNER, TABLESPACE_NAME, TABLE_NAME;
60. Expense and Budget Management
Table growth rate (by year)
/* depending on the table, the dtcreated could need to be changed to timestamp */
select YEAR(dtcreated), COUNT(*)
from course_contents --Enter the table name here
group by YEAR(dtcreated)
61. Expense and Budget Management
Table growth rate (by year)
select TO_char(dtcreated,'YYYY') , COUNT(*)
from course_contents --Enter the table name here
group by TO_char(dtcreated,'YYYY') ;
62. Expense and Budget Management
Table growth rate (by month)
/* depending on the table, the dtcreated could need to be changed to timestamp */
select YEAR(dtcreated), MONTH(dtcreated), COUNT(*)
from course_main --Enter the table name here
group by YEAR(dtcreated), MONTH(dtcreated)
63. Expense and Budget Management
Table growth rate (by month)
select TO_char(dtcreated,'YYYY'), TO_char(dtcreated,'MM') , COUNT(*)
from course_contents --Enter the table name here
group by TO_char(dtcreated,'YYYY'), TO_char(dtcreated,'MM')
order by TO_char(dtcreated,'YYYY'), TO_char(dtcreated,'MM') ;
64. Strategic Functional Decisions
What is the percent of enrollments from the previous term that have
completed a test online?
declare @start_date datetime
declare @end_date datetime
declare @TotalPopulation int
declare @TestingPopulation int
set @start_date = '2010-9-1 00:00:00:000'
set @end_date = '2011-9-1 00:00:00:000'
set @TotalPopulation = (select Count(distinct users_pk1)
from course_users cu
left join course_main cm on cm.pk1 = cu.crsmain_pk1
where role = 'S'
and dtcreated > @start_date
and dtcreated < @end_date)
set @TestingPopulation = (select Count(Distinct cu.users_pk1)
from course_users cu
left join gradebook_grade gg on gg.course_users_pk1 = cu.pk1
left join gradebook_main gm on gg.gradebook_main_pk1 = gm.pk1
left join course_main cm on cm.pk1 = cu.crsmain_pk1
where gm.qti_asi_data_pk1 > 0
and gg.first_attempt_pk1 > 0
and dtcreated > @start_date
and dtcreated < @end_date)
select @TestingPopulation as Online_Testing, @TotalPopulation - @TestingPopulation as Offline_Testing
65. Strategic Functional Decisions
What is the percent of enrollments from the previous term that have
completed a test online?
set serveroutput on size 100000
declare v_start_date date;
v_end_date date;
v_TotalPopulation int;
v_TestingPopulation int;
Begin
dbms_output.enable(9999999);
dbms_output.new_line();
v_start_date := to_date('1-JAN-2010');
v_end_date := to_date('1-JAN-2012');
select Count(distinct users_pk1) into v_TotalPopulation
from course_users cu
left join course_main cm on cm.pk1 = cu.crsmain_pk1
where role = 'S'
and dtcreated > v_start_date
and dtcreated < v_end_date;
select Count(Distinct cu.users_pk1) into v_TestingPopulation
from course_users cu
left join gradebook_grade gg on gg.course_users_pk1 = cu.pk1
left join gradebook_main gm on gg.gradebook_main_pk1 = gm.pk1
left join course_main cm on cm.pk1 = cu.crsmain_pk1
where gm.qti_asi_data_pk1 > 0
and gg.first_attempt_pk1 > 0
and dtcreated > v_start_date
and dtcreated < v_end_date;
dbms_output.put_line (v_TestingPopulation || ',' || to_char(v_TotalPopulation - v_TestingPopulation));
--select v_TestingPopulation as Online_Testing, v_TotalPopulation - v_TestingPopulation as Offline_Testing;
end;
66. Strategic Functional Decisions
How many tests have been completed per enrollment?
declare @enrollment_start datetime
declare @enrollment_end datetime
set @enrollment_start = '2010-9-1 00:00:00:000'
set @enrollment_end = '2011-9-1 00:00:00:000'
select NumberOfTests, COUNT(NumberOfTests) NumberOfEnrollments
into #BinTests
from (
select AllStudents.users_pk1, NumberOfTests = Case
When TestingStudents.TheCount IS NULL THEN 0
When TestingStudents.TheCount > 0 THEN TestingStudents.TheCount
END
FROM (
select distinct users_pk1
from course_users
where role = 'S'
and enrollment_date > @enrollment_start
and enrollment_date < @enrollment_end) AllStudents
left join(
select cu.users_pk1, COUNT(gg.pk1) as TheCount
from course_users cu
left join gradebook_grade gg on gg.course_users_pk1 = cu.pk1
left join gradebook_main gm on gg.gradebook_main_pk1 = gm.pk1
where gm.qti_asi_data_pk1 > 0
and gg.first_attempt_pk1 > 0
and cu.enrollment_date > @enrollment_start
and cu.enrollment_date < @enrollment_end
group by cu.users_pk1) TestingStudents
on AllStudents.users_pk1 = TestingStudents.users_pk1) TestBin
group by NumberOfTests
declare @start int
declare @end int
declare @enrollments int
set @start = 0
set @end = (Select top 1 NumberOfTests from #BinTests order by NumberOfTests desc)
create table #Results(
Number_Of_Tests int,
Number_Of_Enrollments int)
While @start-1 < @end
Begin
set @enrollments = (select NumberOfEnrollments from #BinTests where NumberOfTests = @start)
insert into #Results
(Number_of_Tests, Number_Of_Enrollments)
values
(@start, @enrollments)
set @start = @start + 1
end
select Number_Of_Tests, Number_Of_Enrollments = Case
When Number_Of_Enrollments IS NULL THEN 0
When Number_Of_Enrollments >= 0 THEN Number_Of_Enrollments
End
from #Results
drop table #BinTests
drop table #Results
Editor's Notes
What do you think about a slide that has a picture of one of us looking down and one of us looking up at each other? The way it’s presented right now is kind of boring. It’s like going in for a colonoscopy. Next to our pictures, we would have a quick some nice looking bullets (visuals) that say you are the SPE for Outcomes and Community system, as well as a point about you being one of them.We could have a bubble image coming out of our pictures as though we are thinking some funny thought in our head.These sets of slides have to get our audience comfortable with us. We need to project a wacky or fun persona to get them comfortable. Think 2 Wacky and Crazy guys from SNL.
You have 4 goals right? So why not say this with 4 images and no words? We can keep this as is for the submission to legal.
You might even want to suggest a tool to use to capture the data. A good open source tool is Aquadata Studio. A great commercial tool is Quest Software’s Toad. We kind of need to mention Toad because Quest sponsors our team. We might even want to put a picture of Toad and Aquafold. You might even want to go so far as to reference the Aqua Data Server which is a repository for storing a lot of this information and doing reporting.http://www.aquafold.com
http://library.blackboard.com/ref/fea8bb86-1720-4b14-bb81-542d973b5338/index.htmAny time we use a link, put the smaller version using a tool like bit.ly or goo.gl or even tiny.urlWe might even want to capture a quick snapshot image of the Documentation intro or something…
capacity planning*The act of studying historical data with the intent of identifying repeating patterns and applying the repeating patterns to the decisions that impact the future of the studied system.
(Geoff) Going to make the clip are semi-transparent. Don’t now Geoff, I think this section is missing some slides. We had discussed presenting a process, which you kind of start with the next slides, but there’s not a lot of focus. I suggest that we add a visual slide next. The visual should present a recommendation from us about how to go through the Capacity Planning process:Asking the Right QuestionsCollecting the Right DataOrganizing and Analyzing the DataMaking Decisions from the DataThis process should be very circular in nature in which we always come back to “Asking the right questions!”
We need to change up this slide in some way. It makes it seem like the slide says there are only 3 focus areas of Capacity Planning. How can we say that these are the 3 areas we will focus on, but that there are many other areas of focus?I suggest we do this in a visual. Since it’s 3 areas of focus, we could use a triangle image. (Geoff) I think a triangle image would suggest that the 3 areas are related in some way other than the database and the capacity planning topic.
Downtime (Cold)Traditional MaintenancePatchesFirmwareCold backupMajor upgradesBb 9.1Hardware or software purchasesNo Downtime (Hot)IntegrationSnapshotSISBatch FileMajority of backup processesSQL JobsEach section should have a slide for each of these 4 points. We can obviously have more than 4, but these 4 points need to make the slides.
Frequency of course archive task?Archive all courses or only courses that have been modified?How do we define a modified course?Will the archiving process impact user experience?Does the course archive tool have limitation?Number of entities with in a course (grade book, discussion board replies, etc)Types of content?Location of course archives?Size limitations?Retrieval process?Naming convention?
(Geoff) Graph Slide with query link
This section needs 4 slides about these points…Asking the Right QuestionsCollecting the Right DataOrganizing and Analyzing the DataMaking Decisions from the Data
OperationalBackup & RecoveryCapitalHardwareStorageCPUMemoryI/OSoftwareLicensesMultiple CPUsUnsupported systems
This section needs 4 slides about these points…Asking the Right QuestionsCollecting the Right DataOrganizing and Analyzing the DataMaking Decisions from the Data