A showcase of open source portfolio implementations


Published on

Intro to ePortfolios and Folio Thinking; The OSP Tools in Sakai; An OSP User Scenario; Successful ePortfolio Projects; The Indiana University Personal Development Plan and Presentation Maker; The Kumamoto OSP Implementation;
The Uniformed Services University of the Health Sciences OSP Implementation; The OSP Reports Package on OpenEdPractices.org

Published in: Education, Technology
  • Be the first to comment

A showcase of open source portfolio implementations

  1. 1. A Showcase of Open Source Portfolio Implementations Janice A. Smith, Three Canoes LLC Lynn Ward, Indiana University Reba-Anna Lee, Marist College Ryuichi Matsuba, Kumamoto University June 10-15, 2012Growing Community; Jasig-Sakai 2012i ConferenceGrowing Possibilities 1
  2. 2. Workshop AgendaPart One Part Two, Continued Intro to ePortfolios and  The Marist OPS Folio Thinking Implementation The OSP Tools in Sakai  The Kumamoto OSP An OSP User Scenario Implementation Successful ePortfolio  The Uniformed Services Projects University of the Health Sciences OSP ImplementationPart Two The Indiana University Part Three Personal Development Plan and Presentation  The OSP Reports Package Maker on OpenEdPractices.org Jasig-Sakai 2012i Conference 2
  3. 3. Introduction to Portfolios and Folio Thinking Janice A. Smith, Three Canoes LLC Jasig-Sakai 2012i Conference 3
  4. 4. Two Faces of ePortfoliosPersonal Learning Environment Assessment Management System Helen Barrett - http://electronicportfolios.org/balance/ Jasig-Sakai 2012i Conference 4
  5. 5. Portfolios as Personal Learning Environments Guide students in collecting information about Resume Portfolio themselves Assist students in managing their virtual identity Examples: ◦ Resumes ◦ Professional Portfolios ◦ Showcase Portfolios Jasig-Sakai 2012i Conference 5
  6. 6. Portfolios as Assessment Management Systems  Guide students in the creation of portfolio-worthy evidence  Link evidence to standards, outcomes, or objectives  Evaluate evidence using rubrics  Use reports to: ◦ Aggregate and analyze data ◦ Identify representative artifacts  Examples include: ◦ General Education Portfolios ◦ Disciplinary Portfolios ◦ Co-Curricular Transcripts ◦ Graduation Portfolios Take a look at my learning! Jasig-Sakai 2012i Conference 6
  7. 7. Folio Thinking Folio thinking starts with what we know about learning by identifying:  Measurable outcomes  Processes that work toward these outcomes  Software that enables the processesSection 1 Jasig-Sakai 2012i Conference 7
  8. 8. From Science to Software  Learning science defines dimensions of deep learning  Folio thinking defines behaviors linked to these dimensions  The “Portfolio Process” promotes these behaviorsSection 1 Jasig-Sakai 2012i Conference 8
  9. 9. Dimensions of Deep Learning  Reflective Learning  Integrative Learning  Social LearningSection 1 Jasig-Sakai 2012i Conference 9
  10. 10. Reflective Learning Professionals use reflection in the workplace –  Learning and performance is a cycle of reflection and action ◦ (Schon, The Reflective Practitioner)  Information becomes knowledge when situated in the cycle by a knower ◦ (Brown and Duguid, Social Life of Information)  The cycle is hard-wired into our brains ◦ (Zull, The Art of Changing the Brain)Section 1 Jasig-Sakai 2012i Conference 10
  11. 11. Dimensions of Reflection Reflection-in-action Constructive reflection “reviewing, “developing a cumulative, projecting, multi-selved, revising” multi-vocal identity” Reflection as conversation Reflection-in-presentation with artifacts, “articulating the relationships with self, between and among” creation, with others creator, and context of creation” (— Kathleen Yancey, Reflection in the Writing Classroom)Section 1 Jasig-Sakai 2012i Conference 11
  12. 12. Integrative Learning as Expert Thinking Experts –  Develop a conceptual framework for information  Notice features and patterns  Organize their knowledge to reflect deep understanding  Apply their knowledge to new situations  Monitor their own understanding as they work by ◦ modifying concepts ◦ identifying information gaps ◦ taking control of their learning ◦ (Donovan, et. al., How People Learn)Section 1 Jasig-Sakai 2012i Conference 12
  13. 13. Integrative Learning for Careers Scholarship of integration – an important component of research ◦ (Boyer, Scholarship Reconsidered) Systems thinking – crucial for professional excellence ◦ (Partnership for 21st Century Skills) Multiple careers require continuous learning ◦ (Chen and Mazow, Stanford Center for Innovation in Learning) Increasingly, we have career trajectories, not careers ◦ (Brown 2004 AAHE keynote) In the USA –  75% of students are older, independent, work full time, attend part time (NCES 2002)  58% attend multiple institutions (NCES 2002) Jasig-Sakai 2012i Conference 13
  14. 14. Social Learning  Our identities are formed through participation in communities of practice ◦ (Wenger, Communities of Practice)  Learning to be a member of a disciplinary or professional community is as important or more important than learning “content” ◦ (Brown and Duguid)  Multiple identities ◦ Are formed in the context of multiple communities ◦ Are integrated into learning career trajectories  The social dimension is the most difficult of the three to accomplish via portfoliosSection 1 Jasig-Sakai 2012i Conference 14
  15. 15. Summarizing Folio Thinking  Academia values the creation, organization, and refinement of knowledge  Folio thinking gets to the heart of these academic values by asking students to define and refine their relationship to knowledge  Folio thinking promotes deep learning through ◦ Reflection on learning ◦ Integration of learning ◦ Sharing of learning  Folio thinking leads to the outcome of students explaining and predicting their development as learners  The results of folio thinking are available for the assessment and evaluation of learningSection 1 Jasig-Sakai 2012i Conference 15
  16. 16. The Open Source Portfolio Tools in Sakai Janice A. Smith, Three Canoes LLC Jasig-Sakai 2012i Conference 16
  17. 17. In Sakai, an ePortfolio can be . . . The complete collection of an individual’s multimedia artifacts and reflections. ◦ This process takes place in the Resources tool. A specific set of artifacts and reflections to share with others ◦ This process takes place in the Portfolios tool. ◦ Combined with Resources, the Portfolios tool offers a Personal Learning Environment. A specific collection of artifacts and reflections that represent an individual’s learning and accomplishments. ◦ This process takes place in the Matrices tool. ◦ Combined with Resources and Assignments, the Matrices tool offers an Assessment Management System. Jasig-Sakai 2012i Conference 17
  18. 18. The Portfolio Tools in Sakai Interoperable tool suite for flexible portfolio workflows Assignments Resources Customized through Showcase ◦ User interface or XML coding Learning Portfolios Tool Matrices Portfolios ◦ Primary showcase tool ◦ Support from Resources, Forms, Portfolio Glossary Forms and Portfolio templates Templates Matrices Tool ◦ Primary assessment tool Assessment ◦ Support from Resources, Reports Assignments, Forms, Glossary, and Evaluations Reports Tool ◦ Analysis, display, and export of data Jasig-Sakai 2012i Conference 18
  19. 19. The Resources Tool Available in all sites Interacts with most tools Stores all file types Enter or update data once to use it for multiple purposes My Workspace folder for artifact collection Folders for site owners to distribute content Portfolio interaction folder to collect completed forms from matrices Jasig-Sakai 2012i Conference 19
  20. 20. The Portfolios Tool Users create portfolios with Resume portfolio templates designed by programs and/or institutions Portfolio templates specify content and style and provide users with options for self- expression Portfolios can be shared with ◦ One or more users ◦ Guest users, via email notification and password Co-Curricular Transcript ◦ As public URLs ◦ As a download to the desktop Presentation Maker from Indiana University provides a simple portfolio template for multiple purposes Assessment data from a matrix can be added to a portfolio Jasig-Sakai 2012i Conference 20
  21. 21. The Matrices Tool Used to assess student learning Scaffolded workflows ◦ Rows for criteria ◦ Columns for levels Click matrix cells to open workflow Cells can be linked to ◦ Course assignments ◦ Matrix cells in same site or other sites Cells can be set to ◦ “Unlocked” (ready for use) ◦ “Locked” (unavailable) Cells can be set to unlock as owner submits previous cell for evaluation ◦ Top to bottom ◦ Left to right ◦ As instructor deems appropriate Jasig-Sakai 2012i Conference 21
  22. 22. The Evaluations Tool Centralized receipt of matrix cells submitted for evaluation. Allows instructors to evaluate cells without accessing matrix. Instructors may also access individual student matrices via drop-down menu. Evaluation possible with clickable, scorable evaluation rubrics. Aggregated evaluations across sites available in My Workspace. Jasig-Sakai 2012i Conference 22
  23. 23. The Assignments Tool Link assignments to matrix cells within or across sites ◦ Map course activities to programmatic learning outcomes Link cells in one matrix to cells in another matrix ◦ Map program outcomes to institutional outcomes Assignments submitted for evaluation automatically populate linked matrix cells Participants may or may not be required to visit the matrix for reflection and feedback. Jasig-Sakai 2012i Conference 23
  24. 24. The Forms and Glossary ToolsCreators of matrices: Add forms to prompt: ◦ Artifact collection ◦ Documentation of learning ◦ Reflection ◦ Feedback ◦ Evaluation Add glossary items for: ◦ Mouse-over short definitions ◦ Pop-up longer definitions Jasig-Sakai 2012i Conference 24
  25. 25. Using Forms in Portfolios Creators of portfolio Resume templates use forms to: ◦ Provide user options for Required Settings (formerly called Outline Options)  Title and description  Browser title  Themes (including do-it- yourself) ◦ Structure pages in the Co-Curricular portfolio Transcript  Open ended pages using rich text fields  Structured pages using a variety of field types  Combination ◦ Provide structure for a single portfolio page Jasig-Sakai 2012i Conference 25
  26. 26. Using Forms in Matrices Creators of matrices add forms to matrix cells to prompt: ◦ Artifact collection ◦ Documentation of learning ◦ Reflection ◦ Feedback ◦ Evaluation A form that is filled out and saved by a user is called a form instance or a saved form Jasig-Sakai 2012i Conference 26
  27. 27. Form Fields Text Text Area Rich Text Editor Attachment URL Selection: Radio button (up to four choices) or drop-down menu Boolean (checkbox) Sub forms Additional customized options such as those used in Three Canoes forms for the AAC&U Evaluation RubricsForm Builder is a useful tool for initial work with form fields. Jasig-Sakai 2012i Conference 27
  28. 28. Forms for Reporting Report definitions can be customized to aggregate data from: ◦ Evaluation and feedback forms ◦ Reflection forms ◦ Artifact collection and documentation forms (including attachments) Reports required structured data. Using forms to structure data is the most effective OSP reporting strategy. Rating Statistics for Written Communication Skills Jasig-Sakai 2012i Conference 28
  29. 29. The Reports Tool Allows administrators to: ◦ Load SQL queries via report definitions ◦ Define displays of report queries ◦ Schedule reports to run periodically ◦ Store results for later analysis Customizable report definitions shared by Indiana University for ◦ Evaluation status ◦ Evaluation ratings ◦ Artifact attachments Rating Statistics for Written Communication Skills Jasig-Sakai 2012i Conference 29
  30. 30. An OSP User Scenario Janice A. Smith, Three Canoes LLC Jasig-Sakai 2012i Conference 30
  31. 31. Open Source Portfolio (OSP) Scenario Four interacting personas: Clipper College  Brian Jeffreys ◦ Visual Arts undergraduate  Grace Connolly ◦ Visual Arts professor Undergraduate institution with a ◦ Brian’s major advisor ◦ General education program ◦ Capstone course instructor ◦ Visual Arts degree program  Sharon Westerly An institution dedicated to the ◦ General Education professor improvement of teaching and ◦ Skilled in using portfolios learning to teach institutionally Using OSP for portfolio assessment valued skills and showcasing  Bruce McAllisterPhoto credit 1: Downloaded from the internet on 3-7-11 from stock.xchng, http://www.sxc.hu. ◦ Chair of Visual Arts Department“Sepia Portrait” uploaded by Vikash Sharma in 2010.Photo credit 2: Photo downloaded from the internet on 3-7-11 from stock.xchng, http://www.sxc.hu.“Melody 2,” Uploaded by Jesse Therrien ◦ Encourages faculty to assemblePhoto credit 3: Photo downloaded from the internet on 3-7-11 from stock.xchng, http://www.sxc.hu.“Spontaneous” uploaded by Benjamin Earwicker evidence of student learning forPhoto Credit 4: Photo downloaded from the internet on 3-7-11 from stock.xchng,http://www.sxc.hu. “Beto Book 4” Ensaio Fotografico Beto 2008, uploaded by Beto Lima in 2010 institutional and program assessment Jasig-Sakai 2012i Conference 31
  32. 32. Introduction to the Portfolio During his first year experience, Brian: ◦ Collects learning artifacts in an OSP Matrix according to institutional learning outcomes. ◦ Uses OSP Forms in the matrix to:  Reflect on how artifacts meet learning outcomes.  Evaluate his work with rubrics for each outcome.  Receive formative feedback about his learning. ◦ Submits artifacts, reflections, and self-evaluation for summative evaluation in relation to each outcome. Professor Westerly, General Education: ◦ Uses the portfolio tools in Sakai to:  Provide portfolio training.  Teach a variety of institutionally valued skills.  Offer feedback and evaluation in relation to institutional outcomes. ◦ Prepares Brian to continually develop his portfolio throughout four years of college. Jasig-Sakai 2012i Conference 32
  33. 33. Institutional Outcomes  Inquiry and analysis  Critical thinking  Creative thinking  Written communication  Oral communication  Reading  Quantitative literacy  Information literacy  Teamwork  Problem solving  Personal and Social Responsibility  Civic knowledge and engagement—local and global  Intercultural knowledge and competence  Ethical reasoning  Integrative and applied learning - Adopted from the AAC&U VALUE Rubrics at http://www.aacu.org/value/ Jasig-Sakai 2012i Conference 33
  34. 34. Major Course of Study Professor Connolly helps Brian create an assessment portfolio representing his accomplishments in Visual Arts according to disciplinary outcomes adopted by the department. 1. Creating, Performing and Participating in the Arts: Actively engage in the processes that constitute creation and performance and participate in various roles in the visual arts. 2. Knowing and Using Arts Materials and Resources: Become knowledgeable about and make use of the materials and resources available for participation in visual arts in various roles. 3. Responding to and Analyzing Works of Visual Art: Respond critically to a variety of visual arts, connecting the individual work to other works and to other aspects of human endeavor and thought. 4. Understanding the Cultural Dimensions and Contributions of the Visual Arts: Develop an understanding of the personal and cultural forces that shape artistic communication and how the visual arts in turn shape the diverse cultures of past and present society. Adapted from NYState Learning Standards for the Arts at Three Levels, downloaded fromhttp://www.p12.nysed.gov/ciai/arts/pub/artlearn.pdf on 3-18-2011. Jasig-Sakai 2012i Conference 34
  35. 35. Showcase and Assessment Portfolios  In the capstone course in Visual Arts, Professor Connolly helps Brian use a portfolio template in Sakai to: ◦ Create and refine versions of a showcase portfolio to:  Pursue an internship.  Apply to graduate school. ◦ Present his assessment portfolio to a faculty panel as a final graduation requirement.  By the time he graduates, Brian has: ◦ Has thought deeply about his professional abilities and career goals. ◦ Is well prepared to present his skills and accomplishments to future employers and graduate institutions. Jasig-Sakai 2012i Conference 35
  36. 36. Program Assessment Professor Bruce McAllister, Chair of Visual Arts Organizes faculty to use Assignments and the Matrix to: ◦ Assemble evidence of student learning. ◦ Show how the curriculum addresses institutional and disciplinary learning outcomes. ◦ Continually assess student progress in mastering these outcomes. Generates customized Reports to: ◦ Demonstrate how course assignments address learning outcomes. ◦ Aggregate faculty evaluation of assignments in relation to learning outcomes. ◦ Assemble a random sample of learning artifacts from different courses, outcomes, and levels of student mastery. Jasig-Sakai 2012i Conference 36
  37. 37. Benefits of the Sakai Portfolio Tools Seamless integration with Sakai Flexibility for diverse portfolio processes Integration with registration system Guidance for portfolio activities Emphasis on assessment process, as well as showcasing achievement Showcase workflow via portfolios tool and IU’s Presentation Maker Customized reports available Open source community ready to share Jasig-Sakai 2012i Conference 37
  38. 38. Three Canoes OSP Demo http://demo.threecanoes.com/portal Instance resets nightlyPlay the role of student and instructor: Student username and password: port_student/port_student Instructor username and password: port_coord/port_coord Jasig-Sakai 2012i Conference 38
  39. 39. Characteristics ofSuccessful Portfolio Projects Janice A. Smith, Three Canoes LLC Jasig-Sakai 2012i Conference 39
  40. 40. Successful ePortfolio ProjectsChecklist for Success: Institutional Purpose Leadership Resources Portfolio Use Functional Considerations Technical Considerations Communities of Practice Jasig-Sakai 2012i Conference 40
  41. 41. Successful Projects - Purpose Long term plan involves multiple stakeholders Solution fits institutional and disciplinary processes Student life and variety of disciplines included Cross fertilization of ideas across stakeholder groups Planning for scalability and adoption by new users Planning for sustainability Options for showcasing learning as well as assessing learning Jasig-Sakai 2012i Conference 41
  42. 42. Successful Projects - Leadership Identified portfolio leader with sufficient ◦ Time to develop and maintain project ◦ Resources to develop and maintain project ◦ Administrative back-up Identified portfolio champions ready to promote portfolio use Stakeholder group meets regularly and works toward consensus Planning for succession of portfolio leadership Proactive thinking about transition of leadership at higher levels Jasig-Sakai 2012i Conference 42
  43. 43. Successful Projects - Resources Centralized support system with attention to functional and technical needs Sufficient resources available for ◦ Functional support  Planning for new groups  Train the trainer training  End user training  Peer tutoring ◦ Technical support  Help Desk  System maintenance  System enhancement Jasig-Sakai 2012i Conference 43
  44. 44. Successful Projects - Use Attention to student and faculty concerns Provision for lifelong learning Available as advising tool Involvement of ◦ Career services and student life ◦ Multiple disciplines ◦ General education Attention to transfer students Jasig-Sakai 2012i Conference 44
  45. 45. Successful Projects - Functional Faculty understand how to use portfolios effectively ◦ In the classroom, a hybrid course, and online ◦ In an academic program Assessment activities compliment ◦ Teaching and learning activities ◦ Career development activities Folio thinking transforms our ideas about learning. Portfolios support meta-cognitive processes. There is a growing portfolio culture on campus. Jasig-Sakai 2012i Conference 45
  46. 46. Successful Projects - Technical Single sign-on via portal Minimizing the number of clicks Integration with ◦ LMS for use of data from courses ◦ SIS for system-reported data ◦ Other relevant applications Planning for migration to new versions Contributions to open source community ◦ New code ◦ Community activities Jasig-Sakai 2012i Conference 46
  47. 47. Successful Projects - Community Peer support for using portfolios among students and faculty Participation in the open source community, http://www.sakaiproject.org Participation in portfolio organizations like AAEEBL, http://www.aaeebl.org/ Participation in portfolio research activities like the Inter/National Coalition for Electronic Portfolio Research, http://ncepr.org/ Interaction with national and international ePortfolio experts On-campus adoption encouraged by faculty and students presentations to their peers Jasig-Sakai 2012i Conference 47
  48. 48. OSP Community Resources Janice A. Smith, Three Canoes LLC Jasig-Sakai 2012i Conference 48
  49. 49. OSP Community Resources  OSP Community on Sakai Confluence ◦ http://confluence.sakaiproject.org/display/OSP/Proj ect+--+Portfolio ◦ Weekly phone conference, listserv, OSP documentation, and resources  OSP Community Library ◦ http://openedpractices.org ◦ Use cases and data structures for export  Supporting vendors offering services for Sakai/OSP implementation ◦ The Longsight Group LLC, http://longsight.com ◦ rSmart, http://rsmart.com ◦ Three Canoes LLC, http://threecanoes.com ◦ Unicon, http://unicon.net Jasig-Sakai 2012i Conference 49
  50. 50. Lynn Ward, Indiana University Jasig-Sakai 2012i Conference 50
  51. 51. Urban public research university Founded 1969 2 universities, 21 schools 118 UG, 138 Grad/Prof degrees30,300 students (22,100 UG)97% non-residential55% first-generation students72% UG full time57% of all work off campus Jasig-Sakai 2012i Conference 51
  52. 52. Persistence to Second Year (All Graduation Rates of 2004 FullBeginners and Transfers)1 Time Beginners 21from Undergraduate Retention Report, Persistence to Second Year-2009 Cohort,All Beginners, Vol. 20, No. 1.12 from Official Graduation Rates Report, 2004 Full-time Beginners, Vol.19, No. 1 Jasig-Sakai 2012i Conference 52
  53. 53. Personal development planning is a process whichwill enable first year students at IUPUI tounderstand, implement, and mark progress toward adegree and career goal by creating and following apersonalized plan that is open to revision andreevaluation every semester in collaboration with anacademic advisor, faculty member, or mentor. Buyarski, 2011 Jasig-Sakai 2012i Conference 53
  54. 54. The personal development plan is designed tofoster:1. Goal commitment (student commitment toearning a degree)2. Academic achievement (through goal settingand planning)3. Curricular coherence and meaning in the first-year seminar4. Each of these goals is a way to foster studentdevelopment Buyarski, 2011 Jasig-Sakai 2012i Conference 54
  55. 55. Jasig-Sakai 2012i Conference 55
  56. 56. • Fall 2008 pilot: 1250 student complete a paper PDP• Fall 2009-Spring 2010: reconceptualization as an electronic portfolio that could be revisited and revised• Fall 2010 pilot: 16 sections, ~350 students, 1 portfolio site• Fall 2011 pilot: 50 sections, ~1200 students, 50 portfolio sites• Fall 2012: 50 sections, ~1200 students, portfolio tools provisioned in freshman seminar course sites Jasig-Sakai 2012i Conference 56
  57. 57.  Portfolio matrix (each cell represents one section of PDP) Custom forms and renderers for creating each section Rubric-based evaluation form for each cell/section Custom version of IU’s Presentation Maker portfolio template Jasig-Sakai 2012i Conference 57
  58. 58. Jasig-Sakai 2012i Conference 58
  59. 59. Jasig-Sakai 2012i Conference 59
  60. 60. Jasig-Sakai 2012i Conference 2012 Jasig Sakai 60
  61. 61. Jasig-Sakai 2012i Conference 61
  62. 62. 2012 Jasig Sakai Conference Jasig-Sakai 2012i Conference 62
  63. 63.  Marginally significantly higher fall adjusted semester GPAs (2.89) compared to nonparticipants (2.79) One-Year Fall to Fall Retention rate for students who completed an ePDP (80%) was significantly higher than for students who did not complete an ePDP (72%). Students who completed all parts of the PDP, whether paper or on-line, had significantly better outcomes in the following areas compared to students who only completed some parts of the PDP: succeed academically, make a successful transition to IUPUI, adjust to college life, understand IUPUI’s Principles of Undergraduate Learning (PULs), feel connected to IUPUI, and feel able to meet the demands and expectations of college.* Buyarski, C., Trejullo, D., & Hansen, M. (2011). PDP program evaluation report. Unpublished manuscript, University College, IUPUI, Indianapolis, IN. Jasig-Sakai 2012i Conference 63
  64. 64. Jasig-Sakai 2012i Conference 64
  65. 65. An activity of Japan Sakai Community Ryuichi Matsuba, Makoto Miyazaki, Kumamoto Univ., Japan Ja-Sakaiger 1 & 2 Jasig-Sakai 2012i Conference 2012 Jasig Sakai 65
  66. 66. For students’ learning with Portfolio Processes: collection, reflection, presentation,…We employ OSP, LMS, uPortal, Web-site, etc SakaiCLE/OSP is used in ◦ Graduate School of Instructional Systems ◦ University-wide (under development) Jasig-Sakai 2012i Conference 2012 Jasig Sakai 66
  67. 67. 12th Sakai Conference – Los Angeles, California – June 14-16 Jasig-Sakai 2012i Conference 67 67
  68. 68. • MSc & Mphil, PhD in Instructional Systems• 100% online Program started in April 2006• Training e-Learning Professionals in business and academic fields with 4I’s: ID, IT, IM (Management) & IP (Intellectual Property) The principle motivation of our students is to obtain the competencies they want. ◦ 12 Core and 7 Optional competencies acquired by taking Required and Selective courses respectively. The competencies are directly connected with assignments of each courses.  Students obtain a competency when they pass an assignment in a coueses 11th Sakai Conference - June 15-17, 2010 68 Jasig-Sakai 2012i Conference 68
  69. 69. Assignment (e-Learning fundamental course) • Report your own eLearning experience in BBS • Write your comments on the opinions of classmates Report for the assignment Report Comment Question Critique Reflection Jasig-Sakai 2012i Conference 69
  70. 70. Automaticallydata migrationsand mapping Jasig-Sakai 2012i Conference 70
  71. 71. Self Evaluation of a competency 3: Excellent 2: Satisfactory 1: Poor, unsatisfactoryThe rationaleforthe evaluationof the evidence The radar chart showing achievement of program Self- competencies with four- introductio Evidence items are automatically linked files grade evaluation n that relate to an appropriate competency Jasig-Sakai 2012i Conference 71
  72. 72. Jasig-Sakai 2012i Conference 2012 Jasig Sakai 72
  73. 73. Peer-AssessmentSelf-Assessment A feedback commentsSubmitting reports,a reflection Learners Classmates Outcomes automatically Submission Evaluation registing Click! LMS Teachers Jasig-Sakai 2012i Conference 73
  74. 74. 12th Sakai Conference – Los Angeles, California – June 14-16 Jasig-Sakai 2012i Conference 74 74
  75. 75.  Ongoing effort in Systemically Aligning outcome-based Bachelor’s Degree Programs All of outcomes in LMS are collected in Sakai/OSP (same as GSIS) Display formats can be selected by students and teachers 12th Sakai Conference – Los Angeles, California – June 14-16 Jasig-Sakai 2012i Conference 75 75
  76. 76. Segmentation of LOs detailed items of LOs Basic skills and knowledge for logic, concepts,Subject-Specific Outcomes methodology, and imaginationLiberal Arts Content knowledge Comprehension for culture, society, nature, and lifeForeign Language and ICT skills Talents for use of foreign language and ICTGeneric Skills (Cognitive) Numerical processing, reading, logical thinking, writing, presentation, debateCompetencies Communication, imagination, teamwork, social(Affective and Behavioral) participation, career formation, self-regulated learning 12th Sakai Conference – Los Angeles, California – June 14-16 Jasig-Sakai 2012i Conference 76 76
  77. 77. There are variety requests from Stakeholders:Students, Faculties, Administrative staffs,…Use for • accumulate learning outcomes • student advising • encourage deep learning, Career planning,… • show grades with evidences(outcomes) • … Jasig-Sakai 2012i Conference 77
  78. 78. Educational philosophy of Kumamoto university.Preview: Educational philosophy of Kumamoto university. Terms Bachelor degree program outcomes Jasig-Sakai 2012i Conference 78 78
  79. 79. • Course data, Grades,… from SIS • Evidence (outcomes) from LMS This Portfolio is automatically tailored to the individual! Subject portfolio Years Subject name 1st Year/semester 1st 2nd 1st 2nd credit Subjecttaxonomy Subject Grade Outcomes s Jasig-Sakai 2012i Conference 79 79
  80. 80.  About 1,800 fresh persons Objective for ◦ Orientation in University learning ◦ Learn of academic skills  Writing  Presentation Basic  Improving communication  Use of library Basic ◦ Independence Basic IT seminar literacy ◦ Carrier planning 3 courses are combined in FYE FYE Jasig-Sakai 2012i Conference 2012 Jasig Sakai 80
  81. 81. Pre-Self checking about skills/knowledge Learn of the course materials (blended style)Weeklyassignment•Submit a reportIncluding notes Final assignmentand reflections•Online self- (summative reflection report)testing Post-Self checking about skills/knowledge Jasig-Sakai 2012i Conference 81
  82. 82.  Now LMS is used as an ePortfolio ◦ It means all achievements and reflection in LMS But students cannot conduct reflective works across the courses Consider •Matrix •Presentation •Others • ?? Jasig-Sakai 2012i Conference 82
  83. 83. •Offer personalized learning environments (a kind of portal)•Build a support community for lifelong learning WORKSPACE SHOWCASE Input ePortfolio Input •Learning outcomes Student-centered •curriculum vitae •Reports for social work •career trajectory •Deep learning •Assessment date •Clarify •credential •Reflections •Reports workshop Competencies •make an appeal •Carrier support Enter Graduate alumni association Jasig-Sakai 2012i Conference 83
  84. 84. The Uniformed ServicesUniversity of the Health Sciences Implementation of the Open Source Portfolio Janice A. Smith, Ph.D. Three Canoes LLC Jasig-Sakai 2012i Conference 84
  85. 85. USUHS OSP ImplementationUSUHS Is a long term implementer of the rSmart CLE Began OSP with an rSmart/Serensoft (assisted by Three Canoes) implementation of a Faculty CV portfolio template Created matrices linked to assignments for School of Medicine Clerkships Continued with a larger rSmart/Three Canoes project that includes: ◦ An self-scoring evaluation form with clickable descriptors ◦ Three reports aggregating data from:  The evaluation form  Profile data including photos and biographical statements  PDF files and dates for certificates obtained outside of USUHS ◦ A matrix, forms, and a portfolio template for the School of Nursing ◦ Additional forms to collect School of Medicine departmental assessment data Jasig-Sakai 2012i Conference 85
  86. 86. Faculty CV (Portfolio Template) Jasig-Sakai 2012i Conference 86
  87. 87. School of Medicine Matrices Jasig-Sakai 2012i Conference 87
  88. 88. Evaluation Rubrics Jasig-Sakai 2012i Conference 88
  89. 89. Reports Jasig-Sakai 2012i Conference 89
  90. 90. Proposed School of Nursing Matrix Jasig-Sakai 2012i Conference 90
  91. 91. Proposed Departmental Assessment Forms From Paper To Digital Jasig-Sakai 2012i Conference 91
  92. 92. For Further Information Eric Hanson, USUHS Chief Knowledge Officer ◦ eric.hanson@usuhs.edu Mike Song, USUHS OSP Developer ◦ michael.song.ctr@usuhs.mil Lori Owens, rSmart Project Manager ◦ lowens@rsmart.com Janice Smith, Three Canoes Consultant ◦ janice.smith@threecanoes.com Sean Keesler, Three Canoes Consultant ◦ sean.keesler@threecanoes.com Jasig-Sakai 2012i Conference 92
  93. 93. Lynn Ward, Indiana University Jasig-Sakai 2012i Conference 93
  94. 94.  > 25 active portfolio projects underway, many in professional schools Majority focus on outcomes assessment at the program level Highly individualized approach to assessment Individual differences accommodated via custom scaffoldings and forms Jasig-Sakai 2012i Conference 94
  95. 95.  .25 FTE developers .10-20 FTE designer 1.25 FTE combined consultant, analyst, tech writer, trainer, QA specialist Faculty support through teaching centers Outsource some QA and development work Growing knowledge among consultants in campus teaching centers Jasig-Sakai 2012i Conference 95
  96. 96.  Project coordinators need to aggregate, analyze, and summarize evaluation (and other types of) data Reflection, feedback, and evaluations forms are customized for each project. ◦ Evaluation forms differ widely in terms of  # of rating fields  # rating scale  # labels assigned to each point on scale Most schools, including IU, do not have programmers available to develop custom reports for each project Jasig-Sakai 2012i Conference 96
  97. 97.  Define local form standards ◦ Flexible ◦ Predictable Publish and promote standards Define report definitions based on those standards Reports look for specific form elements and renders them in a standard way if they exist. Jasig-Sakai 2012i Conference 97
  98. 98.  Naming convention for rating fields (elements) ◦ Rating fields (rating, rating 1, rating2, rating3 …rating20) ◦ Comments (comments) Rating fields must be represented as a selection field (drop-down menu) All rating fields in the form (except “rating”) must use the same rating scale. Values in rating field must be a whole or decimal number, “n/a”, or null Rating fields may not be repeatable Jasig-Sakai 2012i Conference 98
  99. 99.  Rating fields and comments fields can use any display name Rating scales can have any number of values The display label for each value in the scale can be any valid string Evaluation forms can have any valid name Each matrix cell can use a different evaluation form. Jasig-Sakai 2012i Conference 99
  100. 100. 10Jasig-Sakai 2012i Conference 0
  101. 101. 10Jasig-Sakai 2012i Conference 1
  102. 102. 10Jasig-Sakai 2012i Conference 2
  103. 103. Click to see summaryreport for cell 10 Jasig-Sakai 2012i Conference 3
  104. 104. Click to return Click to see to main (matrix) detailed report view. for cellJuly 2009 Jasig-Sakai 2012i Conference 10 4
  105. 105. Click to return to main (matrix) view. Click to return to summaryClick NetID to view open cell innew window. 10 Jasig-Sakai 2012i Conference 5
  106. 106. 10Jasig-Sakai 2012i Conference 6
  107. 107. 10Jasig-Sakai 2012i Conference 7
  108. 108. 10Jasig-Sakai 2012i Conference 8
  109. 109. 10Jasig-Sakai 2012i Conference 9
  110. 110. 11Jasig-Sakai 2012i Conference 0
  111. 111.  Download definitions and instructions from: http://openedpractices.org/resource/osp- report-definitions Merge Reports tool code on a local test instance Install report definitions in Reports tool Test and tweak as needed Evaluation forms for rating reports must conform to IU standards. Template included in package. 11 Jasig-Sakai 2012i Conference 1
  112. 112. Janice A. Smith, Three Canoes LLC janice.smith@threecanoes.com Lynn Ward, Indiana University leward@iupui.edu Reba-Anna Lee, Marist College rebaanna.lee@marist.eduRyuichi Matsuba, Kumamoto University matsuba@kumamoto-u.ac.jp 11 Jasig-Sakai 2012i Conference 2