This document summarizes a summer internship project conducted by 7 students at IIT Bombay to modify the edX online learning platform. The project was approved from May 10th to July 6th 2014 under the supervision of Prof. Deepak Phatak. The interns developed a system to synchronize courses between different instances of the edX platform, allowing a parent institution to offer its courses through remote centers. They implemented features like user registration and transfer of course content, files, questions, videos and images between edX and Moodle instances. Tests showed the content could be transferred with low latency.
This master's thesis proposes a framework to share group information across heterogeneous group management systems using Shibboleth federated identity management. The framework would allow group administrators to authorize group members to access particular services based on their group memberships. It analyzes requirements and provides an architecture with core elements like an attribute synchronizer and group manager. The implementation details connecting group information systems like Mailman, LDAP, and Active Directory to Shibboleth are also described.
This document describes a senior project submitted by Wongsarun Chatamornwong and Ronnakrit Kunaviriyasiri to Mahidol University International College in partial fulfillment of a Bachelor of Science degree in Computer Science. The project, called Meka Code, aims to develop an online integrated development environment (IDE) that allows instructors and students to have a shared coding environment and tools. Key features of Meka Code include providing Linux containers to users, a graphical user interface within containers, and functionality for instructors to create courses and assign work and for students to enroll in courses and submit assignments.
This document outlines the steps to build a blog system using the Yii PHP framework. It begins with installing Yii and creating a skeleton application. It then discusses requirements analysis and overall design. Subsequent chapters cover initial prototyping including setting up the database, scaffolding models and CRUD operations, and user authentication. Later chapters customize the blog models, controllers and views for posts and comments. It also covers creating reusable portlets and finalizing the application.
This master's thesis investigates using the OSLC standard to target the documentation requirements of ISO 26262 for automotive safety-critical systems. The thesis focuses on part 6 clause 8 of ISO 26262 regarding software unit design and implementation. It proposes creating an OSLC tool-chain infrastructure for sharing safety information fragments to generate a safety case. The thesis models regulatory and company requirements, defines an OSLC domain specification, and evaluates RDF constraint languages for shaping safety resources. The goal is to provide a methodological approach for semantically representing ISO 26262 work products to enable traceability and documentation for a safety case. A case study models a software unit as a proof of concept.
This document provides guidance on planning and deploying IBM Tivoli Composite Application Manager for Web Resources V6.2 (ITCAM) to monitor Web application server performance. It discusses the ITCAM architecture and how it interconnects with J2EE and WebSphere data collectors. It also covers hardware and software prerequisites, typical deployment environments, and provides a sample project plan for setting up ITCAM with tasks such as environment preparation, software installation, and customizing the product.
This document provides a summary of implementing LDAP authentication for the IBM DS8000 storage system. Key points include:
- Configuring LDAP servers and Tivoli Storage Productivity Center for LDAP authentication.
- Creating certificates and a truststore file to enable secure communication between components.
- Configuring the DS8000 for LDAP authentication through the GUI or CLI and mapping LDAP groups to DS8000 roles.
- Administering users and groups in Tivoli Storage Productivity Center and mapping roles to LDAP groups for single sign-on.
The document provides guidance on setting up the necessary infrastructure and components to implement LDAP authentication for the DS8000. This allows centralized management of users through an LDAP directory and single
This document provides an overview of IBM Tivoli Dynamic Workload Broker version 1.1. It discusses the product's architecture and components, business scenarios and solutions, technical overview, installation process, and how to work with the broker. The document is intended to help users get started with Tivoli Dynamic Workload Broker.
This document provides an overview of the Java language environment. It discusses the origins and design goals of Java, including being simple, object-oriented, robust, secure, architecture neutral, portable, interpreted, and dynamically loaded. It describes the Java language itself and how it is similar to and differs from C/C++. Key aspects covered include the object model, bytecode execution, security features, and support for multithreading. Performance comparisons with other languages are also presented.
This master's thesis proposes a framework to share group information across heterogeneous group management systems using Shibboleth federated identity management. The framework would allow group administrators to authorize group members to access particular services based on their group memberships. It analyzes requirements and provides an architecture with core elements like an attribute synchronizer and group manager. The implementation details connecting group information systems like Mailman, LDAP, and Active Directory to Shibboleth are also described.
This document describes a senior project submitted by Wongsarun Chatamornwong and Ronnakrit Kunaviriyasiri to Mahidol University International College in partial fulfillment of a Bachelor of Science degree in Computer Science. The project, called Meka Code, aims to develop an online integrated development environment (IDE) that allows instructors and students to have a shared coding environment and tools. Key features of Meka Code include providing Linux containers to users, a graphical user interface within containers, and functionality for instructors to create courses and assign work and for students to enroll in courses and submit assignments.
This document outlines the steps to build a blog system using the Yii PHP framework. It begins with installing Yii and creating a skeleton application. It then discusses requirements analysis and overall design. Subsequent chapters cover initial prototyping including setting up the database, scaffolding models and CRUD operations, and user authentication. Later chapters customize the blog models, controllers and views for posts and comments. It also covers creating reusable portlets and finalizing the application.
This master's thesis investigates using the OSLC standard to target the documentation requirements of ISO 26262 for automotive safety-critical systems. The thesis focuses on part 6 clause 8 of ISO 26262 regarding software unit design and implementation. It proposes creating an OSLC tool-chain infrastructure for sharing safety information fragments to generate a safety case. The thesis models regulatory and company requirements, defines an OSLC domain specification, and evaluates RDF constraint languages for shaping safety resources. The goal is to provide a methodological approach for semantically representing ISO 26262 work products to enable traceability and documentation for a safety case. A case study models a software unit as a proof of concept.
This document provides guidance on planning and deploying IBM Tivoli Composite Application Manager for Web Resources V6.2 (ITCAM) to monitor Web application server performance. It discusses the ITCAM architecture and how it interconnects with J2EE and WebSphere data collectors. It also covers hardware and software prerequisites, typical deployment environments, and provides a sample project plan for setting up ITCAM with tasks such as environment preparation, software installation, and customizing the product.
This document provides a summary of implementing LDAP authentication for the IBM DS8000 storage system. Key points include:
- Configuring LDAP servers and Tivoli Storage Productivity Center for LDAP authentication.
- Creating certificates and a truststore file to enable secure communication between components.
- Configuring the DS8000 for LDAP authentication through the GUI or CLI and mapping LDAP groups to DS8000 roles.
- Administering users and groups in Tivoli Storage Productivity Center and mapping roles to LDAP groups for single sign-on.
The document provides guidance on setting up the necessary infrastructure and components to implement LDAP authentication for the DS8000. This allows centralized management of users through an LDAP directory and single
This document provides an overview of IBM Tivoli Dynamic Workload Broker version 1.1. It discusses the product's architecture and components, business scenarios and solutions, technical overview, installation process, and how to work with the broker. The document is intended to help users get started with Tivoli Dynamic Workload Broker.
This document provides an overview of the Java language environment. It discusses the origins and design goals of Java, including being simple, object-oriented, robust, secure, architecture neutral, portable, interpreted, and dynamically loaded. It describes the Java language itself and how it is similar to and differs from C/C++. Key aspects covered include the object model, bytecode execution, security features, and support for multithreading. Performance comparisons with other languages are also presented.
This document is a feasibility study report submitted by Benjamin Kremer for the MSc Computer Science degree at University College London. The report examines the feasibility of constructing a system to verify and quantify collaborative work using blockchain architecture. The project aimed to address the problem of student disengagement by developing an API and mobile application to interact with a blockchain that records collaborative task and team data. While the project did not fully establish a way to verify and quantify collaboration, it demonstrated the concept is feasible with more time and blockchain expertise. The report describes the background, requirements, design, implementation, and testing of the prototype system developed as a proof of concept.
This document provides a deployment guide for IBM Tivoli Usage and Accounting Manager V7.1. It discusses planning the solution environment including hardware and software prerequisites. It also covers installing and configuring the product, including the database, server components, and data collection packs. Finally it demonstrates basic product usage through setting up accounting resources, running a data collection job, and generating reports. The document aims to help deploy and demonstrate the key capabilities of the IBM financial management solution.
Sap co stepbystep config & user manual part 2PallaviChawla8
This document provides an overview of SAP CO (Controlling) integrated training. It outlines various configuration settings and end user steps for valuation and costing in CO, including:
1) Defining valuation strategies and assigning costing keys
2) Actual costing using material ledger
3) Product costing by order using work in process, target costs, and variance analysis
The document also covers cost object controlling, CO-PA profitability analysis, and integration with FI, CO, MM, and PP functional areas.
This document is the user manual for Scrivener 1.0 for Microsoft Windows. It provides instructions for installing and using the application. The manual contains 5 sections that cover introduction and preparation, writing, final phases, appendices with menus and keyboard shortcuts, and options. It guides the user through creating and organizing projects, outlining drafts, compiling written work, and using tools for writing, editing, annotating, and researching within Scrivener.
This document provides a deployment guide for IBM Tivoli Compliance Insight Manager. It begins with an overview of the product architecture and components, including the Tivoli Compliance Insight Manager cluster, Enterprise Server, Standard Server, actuators, Management Console, iView Web portal, databases, and component architecture. It then discusses the product processes of collection, mapping and loading, data aggregation and consolidation, and reporting and presentation. The document also provides guidance on planning for customer engagement, including defining solution tasks and scope. It includes a case study of implementing the solution for a fictional company called Gym and Health Incorporation.
This document presents a graduation project submitted by eight authors to fulfill the requirements of a B.Sc. degree in computer and systems engineering from Alexandria University. The project introduces GenieApp, a cloud computing application that aims to centralize software and resources to make maintenance and upgrades easier for users while allowing pay-per-use payment. The document includes an acknowledgment, abstract, table of contents, and several chapters that describe cloud computing concepts, GenieApp features, the architecture and design of GenieApp, and the development process.
This document provides guidance on deploying IBM Tivoli Composite Application Manager for WebSphere (ITCAM for WebSphere). It includes sample code, installation instructions, and assistance with scope development for a services engagement with ITCAM for WebSphere. The document covers planning the engagement, demonstrating the key capabilities of ITCAM for WebSphere through a sample implementation, and implementing the full ITCAM for WebSphere solution. It also discusses complementary solutions that can be bundled with an ITCAM for WebSphere engagement.
This document specifies the Linked Media Layer architecture and describes its key components. The architecture includes a repository layer for media storage and metadata, an integration layer, and a service layer. It also describes modules for unstructured search using Apache Nutch/Solr, media collection from social networks, searching media resources with latent semantic indexing, and participation in the MediaEval 2013 benchmarking initiative for video search and hyperlinking tasks.
The document is a manual for Tivoli Business Systems Manager Version 2.1. It provides an overview of the product, which allows for end-to-end business impact management through integrated systems management. The manual details the product structure, components, functions, database structure, user interface, and planning requirements for implementation. It is intended to help users understand and implement the key capabilities of Tivoli Business Systems Manager.
The document is a thesis submitted by Maliththa S. S. Bulathwela for the degree of Master of Science in Computational Statistics and Machine Learning at University College London. The thesis explores building a self-adaptive topic engine to extract insights from customer feedback data. Initial work uses supervised support vector machines for topic classification and adapts trust modeling techniques to enhance the reliability of crowd-sourced labeled data. Latent Dirichlet allocation is then used to detect emerging topics from unlabeled data. The results were promising, suggesting further work could build self-adapting topic engines using techniques from the thesis.
This document provides an overview and guide for using Business Objects reporting tools with Tivoli Data Warehouse 1.2. It covers Business Objects products and platform, installing Business Objects desktop components, configuring Business Objects for Tivoli Data Warehouse, creating reports, advanced reporting and security features, and deploying reports. The document contains examples and step-by-step instructions for setting up Business Objects and generating simple to advanced reports on Tivoli Data Warehouse data.
This document provides instructions for installing and setting up IBM TotalStorage Productivity Center V2.3. It discusses the required prerequisites, including software, ports, and user accounts. It also describes planning considerations like the lab environment setup. The document then covers the step-by-step installation process and important post-installation configuration tasks to ensure proper functionality.
This document provides an overview and introduction to using JDBC (Java Database Connectivity) to connect Java applications to databases. It discusses establishing a connection via a JDBC URL, interacting with databases using statements and result sets, and mapping between SQL and Java data types. The document is a guide for developers to get started using JDBC.
This document provides recommendations for implementing IBM Tivoli Change and Configuration Management Database (CCMDB). It discusses gathering requirements such as the need for governance, change management, and configuration management processes. It also covers customizing the CCMDB data model, promoting configuration items, and implementing federation between multiple CCMDB instances. The recommendations are based on best practices for setting up and using CCMDB effectively in organizations.
This document provides implementation details for writing a compiler in C using Lex and Yacc. It is meant to accompany a textbook on compiler design that describes compiler implementation using Java. This document contains equivalent information for building a compiler in C, with chapters that correspond to the textbook chapters. It describes how to implement lexical analysis with Lex, parsing with Yacc, semantic analysis, code generation, and more. The document is not intended to be self-contained and should be used along with the textbook.
This masters report describes the COAcHMAN project which aims to simplify user interactions with smart homes through context awareness. The report conducts background research on context awareness and home automation technologies. As a result, a software solution called COAcHMAN is proposed which enables homes to react based on the user's context rather than requiring direct user interaction. COAcHMAN integrates with the openHAB home automation platform and uses online user profiles to provide familiar interfaces for users. The implementation of COAcHMAN is described along with further development areas like authentication and using internal sensor data.
This research report presents a methodology for developing families of business information systems. The methodology aims to improve the design of business families for service-oriented architectures. The report introduces the concept of business family engineering and describes a software process for developing business families. It defines how to model variability in business families using feature models and how to design the domain and choreographies of business families. The goal is to provide a holistic approach for systematically developing multiple business information systems that share common processes and functionalities.
This document provides a tutorial for using SAP Data Services. It begins with an introduction that outlines the objectives, product overview, and preparation steps. It then covers topics like defining source and target metadata, importing metadata, and populating dimensions. Each section provides instructions and explanations for completing tasks in SAP Data Services like creating projects, adding jobs and data flows, defining transforms, validating, and executing jobs. The goal is to teach users how to extract, transform, and load data using the product.
Thesis - Nora Szepes - Design and Implementation of an Educational Support Sy...Nóra Szepes
This document describes the design and implementation of a new educational support system portal and thin client. It discusses the specification phase where user requirements were gathered. The Mithril JavaScript framework was chosen for implementing the student client module. The design follows a Model-View-Controller pattern. Testing was done using Cucumber, Zombie and Istanbul to validate the design and implementation.
This document introduces design patterns and discusses their application to a program called Oodle. It begins by outlining the goals of object-oriented programming like reusability and maintainability. It then describes tools like encapsulation, intelligent data, and inheritance. The document introduces design patterns as reusable solutions to common programming problems. It uses a notation system to diagram class relationships. Finally, it establishes the original requirements of the Oodle program to provide a running example for exploring design pattern implementations.
This document is a feasibility study report submitted by Benjamin Kremer for the MSc Computer Science degree at University College London. The report examines the feasibility of constructing a system to verify and quantify collaborative work using blockchain architecture. The project aimed to address the problem of student disengagement by developing an API and mobile application to interact with a blockchain that records collaborative task and team data. While the project did not fully establish a way to verify and quantify collaboration, it demonstrated the concept is feasible with more time and blockchain expertise. The report describes the background, requirements, design, implementation, and testing of the prototype system developed as a proof of concept.
This document provides a deployment guide for IBM Tivoli Usage and Accounting Manager V7.1. It discusses planning the solution environment including hardware and software prerequisites. It also covers installing and configuring the product, including the database, server components, and data collection packs. Finally it demonstrates basic product usage through setting up accounting resources, running a data collection job, and generating reports. The document aims to help deploy and demonstrate the key capabilities of the IBM financial management solution.
Sap co stepbystep config & user manual part 2PallaviChawla8
This document provides an overview of SAP CO (Controlling) integrated training. It outlines various configuration settings and end user steps for valuation and costing in CO, including:
1) Defining valuation strategies and assigning costing keys
2) Actual costing using material ledger
3) Product costing by order using work in process, target costs, and variance analysis
The document also covers cost object controlling, CO-PA profitability analysis, and integration with FI, CO, MM, and PP functional areas.
This document is the user manual for Scrivener 1.0 for Microsoft Windows. It provides instructions for installing and using the application. The manual contains 5 sections that cover introduction and preparation, writing, final phases, appendices with menus and keyboard shortcuts, and options. It guides the user through creating and organizing projects, outlining drafts, compiling written work, and using tools for writing, editing, annotating, and researching within Scrivener.
This document provides a deployment guide for IBM Tivoli Compliance Insight Manager. It begins with an overview of the product architecture and components, including the Tivoli Compliance Insight Manager cluster, Enterprise Server, Standard Server, actuators, Management Console, iView Web portal, databases, and component architecture. It then discusses the product processes of collection, mapping and loading, data aggregation and consolidation, and reporting and presentation. The document also provides guidance on planning for customer engagement, including defining solution tasks and scope. It includes a case study of implementing the solution for a fictional company called Gym and Health Incorporation.
This document presents a graduation project submitted by eight authors to fulfill the requirements of a B.Sc. degree in computer and systems engineering from Alexandria University. The project introduces GenieApp, a cloud computing application that aims to centralize software and resources to make maintenance and upgrades easier for users while allowing pay-per-use payment. The document includes an acknowledgment, abstract, table of contents, and several chapters that describe cloud computing concepts, GenieApp features, the architecture and design of GenieApp, and the development process.
This document provides guidance on deploying IBM Tivoli Composite Application Manager for WebSphere (ITCAM for WebSphere). It includes sample code, installation instructions, and assistance with scope development for a services engagement with ITCAM for WebSphere. The document covers planning the engagement, demonstrating the key capabilities of ITCAM for WebSphere through a sample implementation, and implementing the full ITCAM for WebSphere solution. It also discusses complementary solutions that can be bundled with an ITCAM for WebSphere engagement.
This document specifies the Linked Media Layer architecture and describes its key components. The architecture includes a repository layer for media storage and metadata, an integration layer, and a service layer. It also describes modules for unstructured search using Apache Nutch/Solr, media collection from social networks, searching media resources with latent semantic indexing, and participation in the MediaEval 2013 benchmarking initiative for video search and hyperlinking tasks.
The document is a manual for Tivoli Business Systems Manager Version 2.1. It provides an overview of the product, which allows for end-to-end business impact management through integrated systems management. The manual details the product structure, components, functions, database structure, user interface, and planning requirements for implementation. It is intended to help users understand and implement the key capabilities of Tivoli Business Systems Manager.
The document is a thesis submitted by Maliththa S. S. Bulathwela for the degree of Master of Science in Computational Statistics and Machine Learning at University College London. The thesis explores building a self-adaptive topic engine to extract insights from customer feedback data. Initial work uses supervised support vector machines for topic classification and adapts trust modeling techniques to enhance the reliability of crowd-sourced labeled data. Latent Dirichlet allocation is then used to detect emerging topics from unlabeled data. The results were promising, suggesting further work could build self-adapting topic engines using techniques from the thesis.
This document provides an overview and guide for using Business Objects reporting tools with Tivoli Data Warehouse 1.2. It covers Business Objects products and platform, installing Business Objects desktop components, configuring Business Objects for Tivoli Data Warehouse, creating reports, advanced reporting and security features, and deploying reports. The document contains examples and step-by-step instructions for setting up Business Objects and generating simple to advanced reports on Tivoli Data Warehouse data.
This document provides instructions for installing and setting up IBM TotalStorage Productivity Center V2.3. It discusses the required prerequisites, including software, ports, and user accounts. It also describes planning considerations like the lab environment setup. The document then covers the step-by-step installation process and important post-installation configuration tasks to ensure proper functionality.
This document provides an overview and introduction to using JDBC (Java Database Connectivity) to connect Java applications to databases. It discusses establishing a connection via a JDBC URL, interacting with databases using statements and result sets, and mapping between SQL and Java data types. The document is a guide for developers to get started using JDBC.
This document provides recommendations for implementing IBM Tivoli Change and Configuration Management Database (CCMDB). It discusses gathering requirements such as the need for governance, change management, and configuration management processes. It also covers customizing the CCMDB data model, promoting configuration items, and implementing federation between multiple CCMDB instances. The recommendations are based on best practices for setting up and using CCMDB effectively in organizations.
This document provides implementation details for writing a compiler in C using Lex and Yacc. It is meant to accompany a textbook on compiler design that describes compiler implementation using Java. This document contains equivalent information for building a compiler in C, with chapters that correspond to the textbook chapters. It describes how to implement lexical analysis with Lex, parsing with Yacc, semantic analysis, code generation, and more. The document is not intended to be self-contained and should be used along with the textbook.
This masters report describes the COAcHMAN project which aims to simplify user interactions with smart homes through context awareness. The report conducts background research on context awareness and home automation technologies. As a result, a software solution called COAcHMAN is proposed which enables homes to react based on the user's context rather than requiring direct user interaction. COAcHMAN integrates with the openHAB home automation platform and uses online user profiles to provide familiar interfaces for users. The implementation of COAcHMAN is described along with further development areas like authentication and using internal sensor data.
This research report presents a methodology for developing families of business information systems. The methodology aims to improve the design of business families for service-oriented architectures. The report introduces the concept of business family engineering and describes a software process for developing business families. It defines how to model variability in business families using feature models and how to design the domain and choreographies of business families. The goal is to provide a holistic approach for systematically developing multiple business information systems that share common processes and functionalities.
This document provides a tutorial for using SAP Data Services. It begins with an introduction that outlines the objectives, product overview, and preparation steps. It then covers topics like defining source and target metadata, importing metadata, and populating dimensions. Each section provides instructions and explanations for completing tasks in SAP Data Services like creating projects, adding jobs and data flows, defining transforms, validating, and executing jobs. The goal is to teach users how to extract, transform, and load data using the product.
Thesis - Nora Szepes - Design and Implementation of an Educational Support Sy...Nóra Szepes
This document describes the design and implementation of a new educational support system portal and thin client. It discusses the specification phase where user requirements were gathered. The Mithril JavaScript framework was chosen for implementing the student client module. The design follows a Model-View-Controller pattern. Testing was done using Cucumber, Zombie and Istanbul to validate the design and implementation.
This document introduces design patterns and discusses their application to a program called Oodle. It begins by outlining the goals of object-oriented programming like reusability and maintainability. It then describes tools like encapsulation, intelligent data, and inheritance. The document introduces design patterns as reusable solutions to common programming problems. It uses a notation system to diagram class relationships. Finally, it establishes the original requirements of the Oodle program to provide a running example for exploring design pattern implementations.
This thesis examines using machine learning methods to extract cyber threat intelligence from hacker forums. It proposes a two-phase process using supervised and unsupervised learning. In phase one, classifiers like support vector machines are used to classify forum posts as relevant or not to security. Phase two applies topic modeling to identified relevant posts to discover discussion themes. Experiments on a real hacker forum show these methods effectively identify security-related information including zero-days, credentials, and malware. The study demonstrates hacker forums can provide useful threat intelligence and machine learning helps analyze large amounts of forum data.
A Mobile and Web application for time measurement intended to get an accurate picture of the productive time in a production environment in order to reveal the root causes behind ineffective/idle time and to eliminate non-added activities/tasks .
Technical Key-words : Ionic 2, Angular 2, PouchDB, CouchDB ,
DB Replication Protocol, Django, Python NvD3 charts .
This document provides instructions for using the ArchiCAD 10 Interactive Training Guide. It includes an e-book, ArchiCAD project files, and instructional video clips. The guide walks users through building a sample architectural project in ArchiCAD while teaching core design, documentation, and presentation skills. It is recommended to install ArchiCAD 10 and the training guide, then open the provided chapter project files in ArchiCAD to follow along with the corresponding video clips. This allows the user to learn ArchiCAD skills interactively by watching instructions and repeating the steps in the software.
This document describes a project to develop an expert search system that mines academic expertise from funded research in Scottish universities. The system aims to integrate data on funded projects from external sources with an existing academic search engine to improve its search results. It will extract expertise information from publications and funded projects to generate expert profiles. Learning to rank algorithms will then be used to rank experts based on their profiles for specific queries. The goal is to enhance the current search engine that identifies experts based on publications by incorporating additional evidence of expertise from funded research projects.
The document provides a software requirements specification for a Distributed Lecturing and Examination System (DLES) being developed by a student project team. The DLES will allow for real-time lecture conducting and viewing, downloading of materials, viewing of whiteboards and slides, recording and storing of videos, and online examinations with monitoring. It aims to bridge the gap between lecturers and students and create a virtual classroom environment. The successful system is expected to enable natural communication, seamlessly integrate with course management systems, and help ensure student comprehension.
This thesis examines machine learning approaches using Hadoop in the cloud. It implements a distributed machine learning infrastructure in the cloud without dependence on distributed file systems or shared memory. This infrastructure learns and configures a distributed network of learners. The results are then filtered, fused and visualized. The thesis also develops a machine learning infrastructure using Python and compares the two approaches. It uses real-world immigration and GDP datasets from a government database to test the frameworks. The cloud-based approach is able to scale to petabytes of data with minimal configuration.
This document describes a student project to develop a prototype file transfer application called Chuck that uses QR codes. The project aims to address the need for easy file transfers between multiple devices. The document outlines the design and development process, including interaction design, technical design of the transmission schema and application, prototype assessment through user testing, iteration of the prototype based on feedback, and evaluation of the effectiveness and future work. Key aspects of the project include creating mockups, building an Android prototype, evaluating it with participants, and improving the prototype based on results.
eclipse is an open source programming tool.
s an open-source software system
whose aim is to serve as a platform for integrating various Logic Programming extensions
This thesis evaluates different template systems for requirements specification. It first conducts a theoretical evaluation of MASTeR, EARS, and Planguage template systems to analyze their strengths and weaknesses. It then implements a user interface for template systems to assist with requirements documentation. Finally, an experiment is performed with industry professionals and students to compare MASTeR and EARS in reading and writing requirements. The results suggest that template systems can improve requirements specification and that MASTeR may be more suitable.
This document is the Software Guide for version 3.20 of the ORFEO Toolbox (OTB). OTB is a set of algorithms encapsulated in a software library developed by CNES to efficiently exploit results from methodological remote sensing research and development studies. It is implemented in C++ and based on the Insight Toolkit (ITK). The guide provides an introduction to OTB, instructions for downloading and installing it, and overviews of the system organization and essential concepts like the data processing pipeline and spatial objects.
An Optical Character Recognition Engine For Graphical Processing UnitsKelly Lipiec
This dissertation investigates building an optical character recognition (OCR) engine for graphical processing units (GPUs). It describes Jeremy Reed's doctoral dissertation from the University of Kentucky in 2016. The dissertation introduces basic OCR and GPU concepts. It then describes in detail the SegRec algorithm developed by the author for segmenting and recognizing characters on a GPU. Evaluation results comparing SegRec to other OCR engines are provided. The dissertation concludes by discussing limitations and opportunities for improving SegRec and developing it into a full-featured OCR system.
Oracle ADF Tutorial- Hands-on Lab Practices. This document is part of Oracle Application Development Framework Training materials and technical resources, used in development, implementation and customization of Oracle Fusion cloud, SaaS and on-premise application. Prepared by Deepak Bhagat as part of Oracle ADF Corporate Training.
This document provides an architectural design for a collaborative problem solving software called ProjectPlace. It describes the modules, data structures, databases and interfaces that will be used to implement the project. The design uses a three-tier architecture pattern with modules for the client applet, server, logger, common room, project room, and plugins. It also describes the data dependencies and use cases like login, chatting, project creation, and more.
This document describes the development of a web application called "WebFlow RH" for managing human resources at company BGI. It was developed using the Java EE platform and frameworks like Hibernate, JSF and PrimeFaces. The application allows managing employee data, requesting absences, handling the cash register, and updating competency grids.
Similar to FRG_P4_Final_Project_Report_Modifications_to_edX_platform (20)
1. IITB Summer Internship 2014
Project Report
Modifications to edX Platform
Principal Investigator
Prof. D.B. Phatak
Project In-Charge
Mr. Nagesh Karmali
Project Mentor Project Team Members
Srinu Kolukuluri Anand Solanki
Divyanshu Goyal
Grishma Jena
Harsh Sharma
Kripal Gaurav
Pushkar Narayan
Rajarshi Sarkar
Last Updated: July 2, 2014
2. Summer Internship 2014
Project Approval Certificate
Department of Computer Science and Engineering
Indian Institute of Technology Bombay
The project entitled “Modifications to edX Platform” submitted by Anand Solanki,
Divyanshu Goyal, Grishma Jena, Harsh Sharma, Kripal Gaurav, Pushkar Narayan and
Rajarshi Sarkar is approved for Summer Internship 2014 programme from 10th May 2014
to 6th July 2014, at Department of Computer Science and Engineering, IIT Bombay.
Prof. Deepak B. Phatak
Dept of CSE, IITB
Principal Investigator
Mr. Nagesh Karmali
Dept of CSE, IITB
Project In-charge
Place: IIT Bombay, Mumbai
Date: July 2, 2014
3. Declaration
I declare that this written submission represents my ideas in my own words and where
others’ ideas or words have been included, I have adequately cited and referenced the orig-
inal sources. I also declare that I have adhered to all principles of academic honesty and
integrity and have not misrepresented or fabricated or falsified any idea/data/fact/source
in my submission. I understand that any violation of the above will be cause for disci-
plinary action by the Institute and can also evoke penal action from the sources which
have thus not been properly cited or from whom proper permission has not been taken
when needed.
Anand Solanki
NIT, Agartala
Divyanshu Goyal
BITS, Pilani
Grishma Jena
Symbiosis Institute of Technology
Harsh Sharma
MNIT, Jaipur
Kripal Gaurav
NIT, Agartala
Pushkar Narayan
NIT, Agartala
Rajarshi Sarkar
BIT, Mesra
Date: July 2, 2014
4. Acknowledgement
We, the summer interns of the team Modifications to edX Platform, are overwhelmed
in all humbleness and gratefulness to acknowledge our deep gratitude to all those who
have helped us put our ideas to perfection and have assigned tasks, well above the level
of simplicity and into something concrete and unique We, whole heartedly thank Prof.
D.B. Phatak for having faith in us, selecting us to be a part of his valuable project and
for constantly motivating us to do better. We are very thankful to our project incharge,
Mr. Nagesh Karmali, Mrs. Sukla Nag and our mentor Mr. Srinu Kolukuluri for their
valuable suggestions. They were and are always there to show us the right track when
needed help. With help of their brilliant guidance and encouragement, we all were able
to complete our tasks properly and were up to the mark in all the tasks assigned. During
the process, we got a chance to see the stronger side of our technical and non-technical
aspects and also strengthen our concepts. Here by, we gladly consider ourselves to be
the most fortunate batch of interns. Last but not the least, we whole heartedly thank all
our other colleagues working in different projects under Prof. D.B Phatak for helping us
evolve better with their critical advice.
5. Abstract
MOOCs have brought distance learning into the public discourse like nothing ever be-
fore. Two renowned examples of MOOCs are edX and moodle. The edX platform uses
online learning software that uses interactive experiences and moodle has several features
considered typical of an e-learning platform, in addition to some original innovations like
its filtering system. Under this project, we have transferred content from edX to moodle
and also from edX to edX to enhance the process of e-learning. EdX to moodle trans-
fer is benificial when these two platforms are hosted by the same institute/university so
overhead of registering new users or establishing a course for the different domain of users
of both the platforms will be reduced significantly while edX to edX transfer is benificial
when a parent institution will like to offer its courses to the remote centres affiliated with
it.
12. Chapter 1
Introduction to MOOC
MOOC stands for Massive Open Online Course. It is a format to organize learning/teach-
ing/training on a specific topic in a more informal collaborative way. It is a gathering
of participants, of people willing to jointly exchange knowledge and experiences for each
of them to build upon. As such it is within the hands of the participants and organizers
of a MOOC to change it to their needs. This allows them to use the information and
to construct their own ideas or projects. Moodle and edX are two common examples of
MOOC. [4]
1.1 Benefits of MOOC
• It can be organized in any setting that has connectivity (which can include the
Web, but also local connections via Wi-Fi e.g.)
• It can be organized in in any language you like (taking into account the main
language of target audience)
• It can be organized as quickly as you can inform the participants (which makes it
a powerful format for priority learning in e.g. aid relief)
• Contextualized content can be shared by all.
• It can be used beyond time zones and physical boundaries.
• Learning happens in a more informal setting, at a place of your convenience and
often around your own schedule.
• There is no need of having a degree to follow the course, only the willingness to
learn (at high speed).
• Own personal learning environment and/or network can be added by participating
in a MOOC.
1.2 Possible challenges of MOOC
• It feels chaotic as participants create their own content
• It demands digital literacy
1
13. CHAPTER 1. INTRODUCTION TO MOOC
• It demands time and effort from the participants
• It is organic, which means the course will take on its own trajectory (you have got
to let go).
• As a participant you need to be able to self-regulate your learning and possibly give
yourself a learning goal to achieve.
2
14. Chapter 2
edX Installation [1] [2]
2.1 Pre-Requisites
1. Fresh ubuntu 12.04 LTS and execute sudo apt-get update and sudo apt-get upgrade
2. In a fresh ubuntu system root doesnt actually have a password set. So, set the root
password:
sudo passwd
3. Install paver. You must install this being root. Commands for this are given below:
su
pip install -U Paver
4. Django 1.4.12 (Download from https://pypi.python.org/pypi/Django/1.4.12). Please
don’t install any other version of django as the edX platform supports django 1.4.12.
Add the following line in the ∼/.bashrc file. To edit the ∼/.bashrc file use the fol-
lowing command:
nano ∼/.bashrc
Then add the following line in the end:
where /path/to/Django-1.4.12 is the path to your django directory.
5. Install git using the following command:
sudo apt-get install git
3
15. CHAPTER 2. EDX INSTALLATION [?] [?]
2.2 Setting Proxy Information
1. Download ubproxy and set the proxy settings applicable.
cd /path_to/ubproxy
chmod 777 ubproxy
sudo ./ubproxy
Set the host (proxy server url), port (proxy port number), proxy authentication
username and proxy authentication password.
2. Login as root (su)
In the following steps change proxyusername by your proxy username, proxypass-
word by your proxy password., proxy.server.com by your proxy server url, proxy-
portno by proxy port.
3. Set proxy information in /etc/apt/apt.conf (nano /etc/apt/apt.conf)
Acquire::http::proxy
"http://proxyusername:proxypassword@proxy.server.com:proxyportno/";
Acquire::ftp::proxy
"ftp://proxyusername:proxypassword@proxy.server.com:proxyportno/";
Acquire::https::proxy
"https://proxyusername:proxypassword@proxy.server.com:proxyportno/";
save the file and exit (ctrl + x)
4. Add the lines in /etc/sudoers (nano /etc/sudoers)
Defaults env_reset
Defaults
secure_path="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
Defaults env_keep += "http_proxy https_proxy ftp_proxy socks_proxy no_proxy"
save the file and exit (ctrl + x)
5. Add the lines in /etc/environment (nano /etc/environment)
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games"
socks_proxy="socks://proxy.server.com:proxyportno/"
no_proxy="localhost"
http_proxy="http://proxyusername:proxypassword@proxy.server.com:proxyportno"
ftp_proxy="ftp://proxyusername:proxypassword@proxy.server.com:proxyportno"
https_proxy="https://proxyusername:proxypassword@proxy.server.com:proxyportno"
save the file and exit (ctrl + x)
6. Set git proxy.
git config --global http.proxy
https://proxyusername:proxypassword@proxy.server.com:proxyportno
git config --global https.proxy
https://proxyusername:proxypassword@proxy.server.com:proxyportno
4
17. CHAPTER 2. EDX INSTALLATION [?] [?]
cd numpy-1.6.2
python setup.py install
cd ..
cd scipy-0.10.1
python setup.py install
cd ..
rm -rf numpy-1.6.2 scipy-0.10.1
cd $HOME/.virtualenvs/edx-platform/lib/python2.7/site-packages
wget
http://pypi.python.org/packages/source/d/distribute/distribute-0.6.28.tar.gz
tar -xzvf distribute-0.6.28.tar.gz
cd distribute-0.6.28/
python setup.py install
cd ..
rm distribute-0.6.28.tar.gz
pip install -r
/home/your_username_folder/edx_all/edx-platform/requirements/edx/pre.txt
cd /home/your_username_folder/edx_all/edx-platform/
sudo apt-get install phantomjs
npm config set strict-ssl false
npm config set registry "http://registry.npmjs.org/"
rake install_prereqs
cd /home/your_username_folder/edx_all
pip install argcomplete
cd /home/your_username_folder/edx_all/edx-platform
bundle install
rake install_prereqs
cd /home/your_username_folder/edx_all
mkdir db log data
cd /home/your_username_folder/edx_all/edx-platform
./manage.py lms syncdb --noinput migrate
./manage.py cms syncdb --noinput
./manage.py cms syncdb --noinput migrate
./manage.py cms createsuperuser
The installation is complete here!! Now reboot your system! (sudo
reboot)
2.4 To run the Servers:
1. Download the run lms.sh and run cms.sh file with this documentation. Open
the files and edit the following line: cd /home/your username folder/edx all/edx-
platform
2. Login as root and go the directory where run lms.sh file is present. Run the
run lms.sh file in the terminal.
su
6
18. CHAPTER 2. EDX INSTALLATION [?] [?]
bash run lms.sh
This will run LMS (check at localhost:8000).
3. After the run lms.sh script starts the LMS server open a new terminal tab. Login
as root and go the directory where run cms.sh file is present. Run the run cms.sh
file in the terminal.
su
bash run cms.sh
This will run CMS (check at localhost:8001).
4. Now install Robomongo (to access MongoDB data) and SQLite database browser
or Sqliteman (to access SQLite Database).
5. If decide to use the edX platform without any email configuration then to activate a
registered user fetch the activation key of the user from the auth registration table
in SQLite database and paste the following link in your browser:
localhost:8000/activate/activation key fetched from auth registration tabl e
where activation key fetched from auth registration table is the activation key of
the user from the auth registration table in SQLite database.
Now you can login into LMS and CMS successfully.
7
19. Chapter 3
Moodle Installation[3]
1. Install Apache/MySQL/PHP
Open up Terminal and install the following;
sudo apt-get update
sudo apt-get install apache2 mysql-client mysql-server php5
’It will prompt you to set the root password for mysql - take note of it,
you will need it in step 5’
2. Install Additional Software
sudo apt-get install aspell graphviz php5-pspell php5-curl php5-gd php5-intl php5-
mysql php5-xmlrpc php5-ldap
Restart Apache so that the modules are loaded correctly
sudo service apache2 restart
We will be using Git to install/update the Moodle Core Application
sudo apt-get install git-core
3. Download Moodle
Setup your local repository and download Moodle, We will use /opt for the local
repository.
cd /opt
sudo git clone git://github.com/moodle/moodle.git
cd moodle
sudo git branch -a
sudo git branch –track MOODLE 25 STABLE origin/MOODLE 25 STABLE
sudo git checkout MOODLE 25 STABLE
4. Copy local repository to /var/www
sudo cp -R /opt/moodle /var/www/
sudo mkdir /var/moodledata
sudo chown -R www-data /var/moodledata
sudo chmod -R 777 /var/moodledata
8
20. CHAPTER 3. MOODLE INSTALLATION[?]
sudo chmod -R 0755 /var/www/moodle
5. Setup MySQL Server
First we need to change the default storage engine to innodb
sudo vi /etc/mysql/my.cnf
Scroll down to the [mysqld] section and under Basic Settings add the following line
under the last statement
default-storage-engine = innodb
In order to save my.cnf using the editor, type the following in sequence which will
save then close the editor:
:w
:q
Restart MySQL Server for changes to take affect
sudo service mysql restart
Now we need to create the Moodle MySQL User with the correct permissions
Use the password you created in step 1
mysql -u root -p
mysql> CREATE DATABASE moodle DEFAULT CHARACTER SET utf8 COL-
LATE utf8 unicode ci;
Where it says ”moodledude” and ”passwordformoodledude” you should change to
the username and password of your choosing.
mysql> GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,CREATE TEM-
PORARY TABLES,DROP,INDEX,ALTER ON moodle.* TO moodledude@localhost
IDENTIFIED BY ’passwordformoodledude’;
mysql> quit;
6. Complete Setup
Note - If you are not comfortable using terminal to create the config.php file that
needs to be created when going through the installer, you should temporarily make
the webroot writable by doing the following:
sudo chmod -R 777 /var/www/moodle
After you have ran the installer and you have moodle setup, you NEED to revert
permissions so that it is no longer writable using the below command.
sudo chmod -R 0755 /var/www/moodle
7. Open your browser and go to http://IP.ADDRESS.OF.SERVER/moodle
Follow the prompts selecting: Database Type
Choose: mysqli Database Settings
Host server: localhost
Database: moodle
User: moodledude (the user you created when setting up the database)
9
21. CHAPTER 3. MOODLE INSTALLATION[?]
Password: passwordformoodledude (the password for the user you created)
Tables Prefix: mdl Environment Checks
This will indicate if any elements required to run moodle haven’t been installed.
Next next next...
follow prompts and confirm installation Create a Site Administrator Account
Create your moodle user account which will have site administrator permissions.
The password you select has to meet certain security requirements. Installation
Complete
Congrats! You can now start using Moodle!
10
22. Chapter 4
Database Management and the Tools
4.1 Database management system used by moodle
The database that is used by moodle platform is:
• MySQL : MySQL is a freely available open source Relational Database Manage-
ment System (RDBMS) that uses Structured Query Language (SQL).
The tool used for interacting with MySQL database is MySQL workbench.
MySQL Workbench is a visual database design tool that integrates SQL devel-
opment, administration, database design, creation and maintenance into a single
integrated development environment for the MySQL database system.
4.2 Database management system used by edX
The database that is used by edX platform is:
• SQLite : SQLite is a relational database management system contained in a C
programming library. In contrast to other database management systems, SQLite
is not a separate process that is accessed from the client application, but an integral
part of it.
The tool used for interacting with SQLite database is SQLite browser. SQLite
Database Browser is a high quality, visual, open source tool to create, design, and
edit database files compatible with SQLite.
• MongoDB :[5] MongoDB is an open-source document database, and the leading
NoSQL database.It is written in C++.
The tool used for interacting with MongoDB database is Robomongo. Robomongo
is a shell-centric cross-platform open source MongoDB management tool.
11
23. Chapter 5
edX Distributed Platform for
Course Synchronisation
5.1 Introduction
edX Distributed Platform for Course Synchronisation is an application developed for edx
using Django to facilitate selective transfer of course data across remote edX platforms
and with the motive of improving online education. RCMS is developed to reduce load of
thousands of users on a single server by distributing courses among various remote servers
and to facilitate independent working of all the remote servers. RCMS provides func-
tionalities such as course data transfer, course data removal, automatic/manual update,
offering courses, removing courses from offered courses list.
Figure 5.1: edX Distributed Platform for Course Synchronisation Architecture
12
24. CHAPTER 5. EDX DISTRIBUTED PLATFORM FOR COURSE
SYNCHRONISATION
5.2 edX Distributed Platform for Course Synchroni-
sation Installation
For github README.md of edX Distributed Platform for Course Synchronisation, please
visit https://github.com/fresearchgroup/Modifications-to-edX-Platform/
Up and Running with RCMS
Your edX Installation folder should have this structure after copying RCMS files.
* cms
* envs
* common.py
* urls.py
* common
* djangoapps
* rcms
* dump
* logs
* rcms.log
* update
* admin.py
* __init__.py
* models.py
* rcms.cfg
* tasks.py
* views.py
* static
* rcms
* dibu_server.css
* templates
* remote_sync
* not_authorized.html
* server_interface.html
* user_interface.html
Change the dump path, update path, log path in rcms.cfg to rcms folder in
your edX Installation directory. See the rcms.cfg file for more information. Copy all the
files present in Remote Course Management System folder to your edx-Installation
directory according to the folder structure. Create a Virtual Environment using these
commands:
export WORKON_HOME=$HOME/.virtualenvs
source /etc/bash_completion.d/virtualenvwrapper
workon edx-platform
Change your current directory to edx-platform/ having manage.py file. Make man-
age.py executable by:
sudo chmod +x manage.py
13
25. CHAPTER 5. EDX DISTRIBUTED PLATFORM FOR COURSE
SYNCHRONISATION
Then sync db using:
./manage.py cms syncdb
./manage.py cms syncdb --migrate
Access RCMS-Admin Panel using http://localhost:8001/rcms and signin using
admin/superuser account which you created earlier during installation process.
Access University Panel on Remote-Machine: http://(Your Public IP):8001/rcms
and signin with University Account.
The University Account is a user-account verified by Django-admin of edX. Just go to
Django-admin panel by visiting http://localhost:8001/admin log in with your su-
peruser credentials, then go to rcms panel, then Universities, add the user from the
dropdown list and enter other details manually.
Automatic Update has been implemented using celery beat, which is pre-installed in
edX. For automatic update, you need to install rabbitmq-server for ubuntu: Download
Link: http://www.rabbitmq.com/download.html
sudo dpkg -i install rabbitmq-server*.deb
sudo apt-get -f install
Now to enqueue the Update Task,
First make a periodic task by going to django-admin panel, then djcelery tab, then
periodic task, add periodic task Select rcms.tasks.update from registered tasks. select
an interval, and save the task. Now create a Virtual Environment by these commands:
export WORKON_HOME=$HOME/.virtualenvs
source /etc/bash_completion.d/virtualenvwrapper
workon edx-platform
Change your current directory to edx-platform/ directory having a manage.py file.
Make manage.py executable by:
sudo chmod +x manage.py
Now Run celery beat using manage.py:
./manage.py cms celery beat
5.3 Database Models
edX Distributed Platform for Course Synchronisation is included under CMS model of
edX. It consists of two tables i.e. Offered Courses and Universities.
Figure 5.2: edX Distributed Platform for Course Synchronisation in Django-admin Panel
14
26. CHAPTER 5. EDX DISTRIBUTED PLATFORM FOR COURSE
SYNCHRONISATION
The Schema of the tables [6] are as follows:
Sl No. Field Name Field Description
1 name CharField(max length=30)
2 course id CharField(max length=10, unique= True)
3 org CharField(max length=50)
4 start date DateField(blank = True)
5 course url URLField(blank = True)
Table 5.1: Offered Courses Table
Figure 5.3: Offered Courses in Django-admin Panel
Sl No. Field Name Field Description
1 user OneToOneField(User, unique=True)
2 name CharField(max length = 100,unique= True)
3 city CharField(max length = 30)
4 state CharField(max length = 30)
5 courses taken CharField(max length = 1000, blank = True)
6 server ip CharField(max length=30, blank = True)
Table 5.2: Universities Table
15
27. CHAPTER 5. EDX DISTRIBUTED PLATFORM FOR COURSE
SYNCHRONISATION
Figure 5.4: Universities in Django-admin Panel
Here Universities has a user field which has foreign key to the User model of
django.contrib.auth.models. Please refer to our code in Appendix B.
5.4 Admin Interface
The admin interface is available for the admin of IIT Bombay which is the university
offering the various online courses. The interface provides functionalities needed for the
admin to execute his responsibilities.
The IITB admin can add universities and admins for the respectives institutes which
will be offered, the online courses given by IIT Bombay. This is done by accessing the
RCMS admin interface via http://localhost:8001/rcms .
The RCMS admin module has 3 panels. The interface connects with the edX on the
fly. The courses available that can be offered to the institutes is shown in the first panel.
The admin can select any number of those courses and choose to offer them to subscribed
universities. The second panel displays a list of all the courses that are being currently
offered. It also has a button Unoffer Courses that will removed the selected course(s) and
place them in the first panel. The last panel shows a list of the universities that have
subscribed to IIT Bombays edX courses.
16
28. CHAPTER 5. EDX DISTRIBUTED PLATFORM FOR COURSE
SYNCHRONISATION
Figure 5.5: Admin Panel for offering Courses
Figure 5.6: Admin Panel for offering Courses
17
29. CHAPTER 5. EDX DISTRIBUTED PLATFORM FOR COURSE
SYNCHRONISATION
5.5 University Interface
The user of this interface is an admin of a university that has subscribed to IIT Bombay’s
edX courses. It is divided into two panels. The first one shows a list of available courses
offered by IIT Bombay that can be committed to. The user can click on the ‘Com-
mit courses’button to subscribe to further courses. The second panel displays all the
courses of IIT Bombay that are currently subscribed to. The user may select courses and
click on ‘Uncommit courses’to unsubscribe from them. All the contents of the selected
course(s) are deleted from the MongoDB, effectively deleting the course from the user’s
(institute’s)edX. The user may also choose to manually update the courses whenever he
wishes to.
Figure 5.7: University Panel for Remote Institute
5.6 Course Management in edX using mongoDB
In case of edX, courseware is managed with the help of mongoDB. To create rich, engaging
online courses, course authors must be able to combine components from a variety of
sources. XBlock is edXs component architecture that makes this possible. Courses
are built hierarchically of pieces called XBlocks. Like an HTML <div>, XBlocks can
represent pieces as small as a paragraph of text, a video, or a multiple-choice input field,
or as large as a section, a chapter, or an entire course. XBlocks are not limited to just
delivering courses. A complete educational ecosystem will make use of a number of web
applications, all of which will need access to course content and data.
XBlocks provide the structure and APIs needed to build components for use in all of
these applications.
18
30. CHAPTER 5. EDX DISTRIBUTED PLATFORM FOR COURSE
SYNCHRONISATION
GridFS is a specification for storing and retrieving files that exceed the BSON-
document size limit of 16MB. Instead of storing a file in a single document, GridFS
divides a file into parts, or chunks, and stores each of those chunks as a separate document.
GridFS uses two collections to store files. One collection stores the file chunks, and the
other stores file metadata.
GridFS places the collections in a common bucket by prefixing each with the bucket
name. By default, GridFS uses two collections with names prefixed by fs bucket:
• fs.files: stores the files metadata.
• fs.chunks: stores the binary chunks.
Modulestores: These implement the ModuleStoreRead and ModuleStoreWrite to
provide access to XBlock content.
modulestore. id: The unique ID for this BSON document.
fs.files. id: The unique ID for this BSON document.
fs.chunks.files id: The id of the “parent” document, as specified in the files collection.
A course can have as many documents in mongoDB as needed. A sample document
for each collection used in storing courseware is given below.
A sample xmodule.modulestore document.
{
"_id" : {
"tag" : "i4x",
"org" : "IITB",
"course" : "CS201",
"category" : "course",
"name" : "2014_T1",
"revision" : null
},
"definition" : {
"children" : [
"i4x://IITB/CS201/chapter/a614191c11bd4ccdb4f2677801b7cf27"
],
"data" : {
"wiki_slug" : "IITB.CS201.2014_T1"
}
},
"metadata" : {
"end" : "2014-06-24T18:30:00Z",
"tabs" : [
{
"type" : "courseware",
"name" : "Courseware"
},
{
"type" : "course_info",
19
32. CHAPTER 5. EDX DISTRIBUTED PLATFORM FOR COURSE
SYNCHRONISATION
"locked" : false,
"md5" : "891a9675e5d2222b5fa281fd5c63a666"
}
A sample xcontent.fs.chunks document.
{
"_id" : ObjectId("5396ac168ae8421d025e73ee"),
"n" : 0,
"data" : { "$binary" : "/9j/4AA.....kB//9k=", "$type" : "00" },
"files_id" : {
"tag" : "c4x",
"org" : "Narnia",
"course" : "PA101",
"category" : "thumbnail",
"name" : "123.jpg",
"revision" : null
}
}
5.7 Course Transfer
As soon as a University commits to a course listed on University Panel, it gets transferred
to their mongoDB with the help of mongodump and mongorestore shell commands.
mongo shell commands such as ‘mongodump’and ‘mongorestore’can be used to connect
to remote mongo instances and make our queries. mongodump is a utility for creating
a binary export of the contents of a database. mongodump is used in conjunction with
mongorestore to restore databases. The mongorestore program writes data from a binary
database dump created by mongodump to a MongoDB instance. mongorestore can create
a new database or add data to an existing database.
xmodule.modulestore: ‘{“ id.course”:“CS101”}’
xcontent.fs.files: ‘{“ id.course”:“CS101”}’
xcontent.fs.chunks: ‘{“files id.course”:“CS101”}’
If you restore to an existing database, mongorestore will only insert into the existing
database, and does not perform updates of any kind. If existing documents have the same
value id field in the target database and collection, mongorestore will not overwrite those
documents.
mongodump --collection <source collection> --db <source database> --query q
--out dump_path
mongorestore --collection <target collection> --db <target database> dump_path +
/<target database>/<target collection>.bson --host ip --port 27017
21
33. CHAPTER 5. EDX DISTRIBUTED PLATFORM FOR COURSE
SYNCHRONISATION
• db <source database> - to specify the name of the database that contains the
collection to be dumped/restored.
• collection <source collection> - to specify the collection to be dumped/re-
stored.
• query <q > - allows only those documents to be exported which satisfy the given
query
• db <target database > - to specify the name of the database where the content
is to be imported
• collection <target collection> - to specify the name of the collection where the
content is to be imported
• out <filename >- specify a file to write the export to. If nothing is specified, the
mongoexport writes data to standard output (e.g. stdout).
• host <ip> - Specifies a resolvable hostname for the mongod to which to connect.
• port <27017> - Specifies the TCP port on which the MongoDB instance listens
for client connections.
• username <username> - Specifies a username to authenticate to the MongoDB
instance.
• password <password> - Specifies a password to authenticate to the MongoDB
instance
mongodump --collection modulestore --db xmodule --query ’{"_id.course":
"course_selected"}’ --out dump_path
mongorestore --collection modulestore --db xmodule dump_path +
/xmodule/modulestore.bson --host ip --port 27017
Please refer to our code in Appendix B.
5.8 Remove Course from mongoDB
A University may also want to remove the committed courses from their courseware
mongoDB. ‘mongo’shell command with ‘host’and ‘–eval’arguments can be used to make
a query to a remote instance of mongoDB. Here variable ‘ id.course’refers to course ID of
the course which University wants to remove from its database and ‘ id.org’refers to the
organisation which has offered this course. Two arguments have been used to remove the
course to prevent removal of courses with similiar course IDs, eg. ‘CS101’of ‘IITBX’and
‘CS101’of ‘IITMX’.
Thus each and every content which corresponds to the selected course ID and the
Institute offering the course is removed from ‘fs.chunks’, ‘fs.files’collections of ‘xcon-
tent’database and ‘modulestore’collection of ‘xmodule’database.
22
34. CHAPTER 5. EDX DISTRIBUTED PLATFORM FOR COURSE
SYNCHRONISATION
mongo ip:27017/xmodule --eval ’db.modulestore.remove({$and:[{"_id.course":
"c_id"}, {"_id.org":"org"}]})’
5.9 Logs
Every activity related to edX Distributed Platform for Course Synchronisation such as
‘Transfer of Course’, ‘Removal of Course from mongoDB’, ‘Updating Course’, ‘Offering
Course’, ‘Removing Courses from Offered Courses List’, etc is logged in a file ‘rcms.log’,
which can be specified in ‘rcms.cfg’config file. Logging has been implemented using Shell
Output of all the mongo commands, and ‘ConfigParser’module of Python.
Figure 5.8: edX Distributed Platform for Course Synchronisation Logs
5.10 Update Courses
An update feature has been provided for each of the courses subscribed to by an institute.
The course contents on the main server of IIT Bombay are compared with those of the
institute. Changes are incorporated in the institute’s edX. This helps in maintaining
consistency of the course data between IIT Bombay and the institute. Any given course
in edX stores the content in MongoDB in three collections: modulestore, fs.files and
fs.chunks. All the three need to be checked for updates. The update process in MongoDB
takes place with the help of the mongoimport and mongoexport commands.
sudo mongoexport --db <source database> --collection <source collection>
-q query --out <filename>
23
35. CHAPTER 5. EDX DISTRIBUTED PLATFORM FOR COURSE
SYNCHRONISATION
sudo mongoimport --db <target database> --collection <target collection>
--upsert <filename> --host ip --port 27017
• sudo - to grant user root privileges
• mongoexport - produces a JSON or CSV export of data stored in a MongoDB
instance
• db <source database> - to specify the name of the database that contains the
collection to be exported
• collection <source collection> - to specify the collection to be exported
• q <query> - allows only those documents to be exported which satisfy the given
query
• out <filename> - specify a file to write the export to. If nothing is specified, the
mongoexport writes data to standard output (e.g. stdout).
• mongoimport - provides a route to import content from a JSON, CSV, or TSV
export created by mongoexport
• db <target database>- to specify the name of the database where the content is
to be imported
• collection <target collection>- to specify the name of the collection where the
content is to be imported
• upsert <filename>- to modify the import process to update existing objects in
the database if they match an imported object, while inserting all other objects.
If fields are not specified using the –upsertFields, mongoimport will upsert on the
basis of the id field.
• host <ip> - Specifies a resolvable hostname for the mongod to which to connect.
• port <27017> - Specifies the TCP port on which the MongoDB instance listens
for client connections.
An example of commands used to update the modulestore collection in the xmodule
database for the course CPE433 is as follows:
sudo mongoexport --db xmodule --collection modulestore -q
’{"_id.course":"CPE433"}’ --out /home/test/modulestore.bson
sudo mongoimport --db test --collection modulestore --upsert
/home/test/modulestore.bson --host 192.168.150.1 --port 27017
In the RCMS module, we have provided functionality for automatic as well as manual
update.
24
36. CHAPTER 5. EDX DISTRIBUTED PLATFORM FOR COURSE
SYNCHRONISATION
5.10.1 Automatic Update
Automatic updates can be scheduled to occur periodically. EdX uses Celery in con-
junction with Django. Celery is a Python library which is a job/task queue based on
distributed message passing. It focuses on real-time operation but supports scheduling as
well. The Django admin can choose the checking for updates to happen in given time in-
tervals or at specific times. The Django admin interface provides two options for periodic
tasks - intervals and cron tabs. In the Djcelery panel, the admin can choose an interval
or a crontab for the update task. The interval can be specified in terms of microseconds,
seconds, minutes, hours or days. This ensures that the update mechanism will be imple-
mented after the given interval of time. An alternative is to provide a crontab for the
update. A specific hour, day of the week, day of the month and month of the year need
to be given.
The automated process allows for regular checking of updates to take place without
any human intervention.
5.10.2 Manual Update
At the user end which is operated by the admin of the institute, a feature is available to
update a particular course or all the courses. The admin may log into the user interface
of the RCMS module and click on the update button next to course subscribed to.
Additionally, the admin can choose to update all the courses committed to using the
‘Update All’button. All the updates take place in real-time. Please refer to our code in
Appendix B.
5.11 Future Scope
Additional features can be added to the RCMS module. Some of them are as follows:
1. Performance report upload: Many a times, the course content creators or teach-
ers in the university offering edX courses (in this case, IIT Bombay) want insights
into performances of the enrolled students of the different institutes. EdX supports
partial functionality in this aspect. The progress report and gradesheets can be
viewed by a student and by a course instructor for all the students of the particular
course. Future work can involve creation of a portal for teachers to upload the
performance reports for a course, which can be used for evaluation by IIT Bombay.
2. Statistics of past offerings: Data collected from the past offerings of a particular
course offered by IIT Bombay can be shown to potential students. The information
may help the students decide on whether they should enrol for a course and help
them prepare a strategy or to successfully complete the course.
25
37. Chapter 6
Transfer of Data from edX to
Moodle
6.1 User Content Transfer from edX to Moodle
6.1.1 Introduction
Synchronisation of user related tables which contains the user data of edX and Moodle
is done through a Java Database Connection application developed. The JDBC [7] [8]
[9] application (please refer to our code in Appendix A) extracts the data from edX and
inserts it into Moodle.
6.1.2 User tables in edX and Moodle
• auth user table:
The auth user table is built into the edX Django Web framework. It holds generic
information necessary for user login and permissions.
• mdl user table [10]:
The mdl user table follws MySQL database sequence and holds the holds the user
information necessary for user login and permissions in Moodle platform.
6.1.3 Creating Java Database Connectivity Application
• Register the JDBC driver:
Requires that you initialize a driver so you can open a communications channel
with the database.
• Open a connection:
Requires using the DriverManager.getConnection() method to create a Connection
ob- ject, which represents a physical connection with the database.
1. For mysql
conn = DriverManager.getConnection(database url,username,password)
2. For sqlite
conn = DriverManager.getConnection(database url,port)
26
38. CHAPTER 6. TRANSFER OF DATA FROM EDX TO MOODLE
Creating JDBC Application for transferring data from edX auth user sqlite database
table to Moodle mdl user mysql table.
• Execute a query to retrieve data from edX sqlite table:
Requires using an object of type Statement for building and submitting an SQL
statement to the database.
ResultSet object name = statement.executeQuery(write the sql query here)
• Extract data from result set and insert it into moodle mysql table:
Execute the sql query to write the extracted sqlite user related data into the moodles
mdl user table.
• Trigger:
A trigger needs to be added in the render to response() function of the
/home/sogeking/edx all/edx-platform/common/djangoapps/edxmako/shortcuts.py
file of edX.
27
39. CHAPTER 6. TRANSFER OF DATA FROM EDX TO MOODLE
6.1.4 Password
The moodle accepts password in form of no less than 8 characters which includes atleast 1
special character ,atleast 1 capital and small alphabet and atleast one numeral while there
is no such case as this in edX. So we have created a code which contains many passwords
and their hashes.When a new user is created this program is executed and when this
program is executed a random password is selected and that is sent through mail to the
newly registered user while the corresponding hash is inserted into the mdl user table.This
password can be then changed by the user after first login.
6.1.5 Implementation
Figure 6.1: User Registration page
28
40. CHAPTER 6. TRANSFER OF DATA FROM EDX TO MOODLE
Figure 6.2: Registration information of User
29
41. CHAPTER 6. TRANSFER OF DATA FROM EDX TO MOODLE
Figure 6.3: Registration information of User
Figure 6.4: User registration confirmation
30
42. CHAPTER 6. TRANSFER OF DATA FROM EDX TO MOODLE
Figure 6.5: Moodle admin contains newly added user
Figure 6.6: User login in Moodle
31
43. CHAPTER 6. TRANSFER OF DATA FROM EDX TO MOODLE
Figure 6.7: User Home Page in Moodle
6.2 Course Transfer from edX to moodle
6.2.1 Introduction
To synchronize the course related tables that contains the basic informations of course
of the moodle and edX alongwith the mongodb database of the edX a ,JDBC (Java
Database Connection)application is created. The JDBC application (please refer to our
code in Appendix A) will extract data from edX and insert it into moodle. The user
defined functions that are used in the JDBC application is to extract the information
of course id written in string1/string2/... format in edx course accessrole table and to
extract full name of the course from mongodb database.
6.2.2 Creating JDBC Application
Creating JDBC Application for transferring data from edX student accessrole sqlite table
and mongoDB database to Moodle mdl course mysql table.
• Import the packages:
Requires that you include the packages containing the JDBC classes needed for
database programming.
• Register the JDBC driver:
Requires that you initialize a driver so you can open a communications channel
with the database.
32
44. CHAPTER 6. TRANSFER OF DATA FROM EDX TO MOODLE
• Open a connection:
Requires using the DriverManager.getConnection() method to create a Connection
ob- ject, which represents a physical connection with the database.
– For mysql
conn = DriverManager.getConnection(database url,username,password)
– For sqlite
conn = DriverManager.getConnection(jdbc:sqlite:path to edX database)
– For mongodb
conn = DriverManager.getConnection(database url,port)
• Execute a query to retrieve data from edX sqlite table:
Requires using an object of type Statement for building and submitting an SQL
statement to the database.
ResultSet object name = statement.executeQuery(write the sql query here)
• Used function for extracting mongoDB data:
To extract the course full name from the mongodb database.
• Used generateinitial() function:
To generate the short name from the extracted mongodb database.
• Extract data from result set and insert it into moodle mysql table:
Execute the sql query to write the extracted sqlite and mongodb database course
related data into the moodles mdl course table.
• Clean up the environment:
Requires explicitly closing all database resources versus relying on the JVMs garbage
collection.
• Trigger:
A trigger was added in the course handler() function of the
edx all/edx-platform/cms/djangoapps/contentstore/views file of edX.
33
45. CHAPTER 6. TRANSFER OF DATA FROM EDX TO MOODLE
Figure 6.8: Trigger in course.py file for transferring Course
34
46. CHAPTER 6. TRANSFER OF DATA FROM EDX TO MOODLE
6.2.3 Implementation
Figure 6.9: Courses in edX
Figure 6.10: Courses in moodle
35
47. CHAPTER 6. TRANSFER OF DATA FROM EDX TO MOODLE
Figure 6.11: Course creation in edX
Figure 6.12: Created course in edX
36
48. CHAPTER 6. TRANSFER OF DATA FROM EDX TO MOODLE
Figure 6.13: Transferred course in moodle
Figure 6.14: Transferred course content in moodle
37
49. CHAPTER 6. TRANSFER OF DATA FROM EDX TO MOODLE
6.3 Section Transfer from edX to moodle
6.3.1 Introduction
• The transfer of sections from edX to moodle involves fetching the recent section
from the MongoDB database of edx first.
• Then the section is transferred to moodle by populating moodle’s database using a
JDBC program. (please refer to our code in Appendix A)
• Finally, the rebuild course cache plugin [11] [12] is used to make the section appear
within a moodle course.
6.3.2 Steps in transferring a section from edX to moodle
• First of all, edX follows a heirarchy (A section can have multiple subsections. A
subsection can have multiple units. A unit can have multiple problems. A problem
can have questions, full screen images, videos, etc).
• MongoDB stores the link of all courses as category: course. Links of all the sections
in a course is stored in the metadata of the course and also as category: chapter in
MongoDB.
• Moodle stores its course content in topics in topic format. There can be any number
of topics in a moodle course, however upto 30 topics can be set to appear in course
by default. set
• So, whenever a section is created in edX, simultaneously a topic in moodle gets
created.
• The tables referred/populated to make a topic in moodle are: mdl course sections,
mdl course modules and mdl course.
• A trigger to do all these things has been added in course handler() function of
edx all/edx-platform/cms/djangoapps/contentstore/views/course.py.
38
50. CHAPTER 6. TRANSFER OF DATA FROM EDX TO MOODLE
6.3.3 Implementation
Figure 6.15: Before creating a section in edX
Figure 6.16: Before creating a topic in moodle
39
51. CHAPTER 6. TRANSFER OF DATA FROM EDX TO MOODLE
Figure 6.17: Creating a section in edX
Figure 6.18: After creating a section in edX
40
52. CHAPTER 6. TRANSFER OF DATA FROM EDX TO MOODLE
Figure 6.19: Transferred edX section in moodle as a topic
6.4 File Transfer from edX to moodle
6.4.1 Introduction
• The transfer of course content from edX to moodle involves fetching the content
(PDF here) from the MongoDB database of edx first. Then by using a script,
written in python, the given content (PDF here) is dumped (stored) into a dump
folder that stores the integrated content which was earlier stored as chunks in the
MongoDB database.
• Then the content (PDF here) is copied to the moodledata directory where moo-
dle stores its course contents. After that moodle database is populated with the
appropiate field information related to the file that is earlier copied from edx to
moodle. It is done by using a JDBC program. (please refer to our code in Ap-
pendix A)
• Finally, turn on the editing option of the course and run the rebuild course cache
plugin [11] [12] to make the content (PDF here) appear within a moodle course.
6.4.2 Fetching the chunks of data from MongoDB database
MongoDB stores the edX files in fs.chunks collection of xcontent database as chunks. The
information about the file is stored in the fs.files collection of the xcontent database. To
retrieve the latest file from MongoDB we used the following command:
41
53. CHAPTER 6. TRANSFER OF DATA FROM EDX TO MOODLE
Figure 6.20: Retrieve File from MongoDB
6.4.3 Copy the fetched data into moodledata directory
In moodle filenames are given according to the SHA1 hash of their content. This means
each file with particular contents is stored once, irrespective of how many times it is
included in different places, even if it is referred to by different names. To relate a file on
disc to a user-comprehensible path or filename, you need to use the files database table.
Suppose a file has SHA1 contenthash as: 081371cb102fa559e81993fddc230c79205232ce.
Then it will be stored as:
moodledata/filedir/08/13/081371cb102fa559e81993fddc230c79205232ce.
The directory under the filedir directory gets the name from the first and second character
of the file’s contenthash. The subdirectory under the filedir directory gets the name from
the third and fourth character of the file’s contenthash. This means that moodle cannot
store two files with the same SHA1 hash. We did this using the following command:
Figure 6.21: Get the Contenthash and Copy the fetched data into moodledata directory
6.4.4 Change permissions of the new directory created in the
moodledata/filedir directory
We have to change permissions of all the sub-directories present in the moodledata/filedir
directory in order to allow moodle to access the files. The permissions were set using the
following commands:
Figure 6.22: Set permissions for the files in moodledata/filedir
42
54. CHAPTER 6. TRANSFER OF DATA FROM EDX TO MOODLE
6.4.5 Populate the moodle tables in the moodle database
We then populated the following tables:
mdl files, mdl context, mdl resource, mdl course sections, mdl course modules
6.4.6 Run the rebuild course cache plugin
After transferring the file to the moodledata/filedir directory and populating the tables
we need to rebuild the course cache so that the transferred content would appear on the
required course. We downloaded and installed the course rebuild cache plugin in order
to rebuild the course caches. The plugin has to be copied to the moodle/admin/tools
directory and then can be installed by accessing the following address on the browser:
http://127.0.0.1/moodle/admin/tool/rebuildcoursecache. After the plugin has been in-
stalled it can then be run to rebuild course caches using command line. The command
’php index.php courseid’ builds the cache for only the course having a specific courseid.
6.4.7 Implementation
Figure 6.23: Uploading a file in edX 1
43
55. CHAPTER 6. TRANSFER OF DATA FROM EDX TO MOODLE
Figure 6.24: Uploading a file in edX 2
Figure 6.25: Uploading a file in edX 3
44
56. CHAPTER 6. TRANSFER OF DATA FROM EDX TO MOODLE
Figure 6.26: File in edX transferred to moodle
6.5 Question Transfer from edX to moodle
6.5.1 Introduction
• edX has the following question types: Blank Common Problem, Checkboxes, Drop-
down, Mutiple Choice, Numerical Input, Text Input, Blank Advanced Problem, Cir-
cuit Schematic Builder, Custom Javascript Display and Grading, Custom Python-
Evaluated Input, Drag and Drop, Image Mapped Input, Math Expression Input
and Problem with Adaptive Hint.
• Moodle has the following question types: Calculated, Calculated Multichoice, Cal-
culated Simple, Embedded Answers, Essay, Matching, Multiple Choice, Numerical,
Random Short Answer Matching, Short Answer, True/False.
• The transfer of questions (we have considered only multiple choice questions with
single correct option) from edX to moodle involves fetching the recent question from
the MongoDB database of edx first.
• Then the question is transferred to moodle by populating moodle’s database using
a JDBC program. (please refer to our code in Appendix A)
• Finally, the rebuild course cache plugin [11] [12] is used to make the question appear
within a moodle course.
45
57. CHAPTER 6. TRANSFER OF DATA FROM EDX TO MOODLE
6.5.2 Steps in transferring a question from edX to moodle
• First of all, edX follows a heirarchy (A section can have multiple subsections. A
subsection can have multiple units. A unit can have multiple problems. A problem
can have multiple questions).
• MongoDB stores the link of all courses as category: course. Links of all the sections
in a course is stored in the metadata of the course and also as category: chapter
in MongoDB. Links of all the subsections in a section is stored in the metadata of
the section and also as category: sequential in MongoDB. Links of all the units in a
subsection is stored in the metadata of the subsection and also as category: vertical
in MongoDB. Links of all the problems in a unit is stored in the metadata of the
unit and also as category: problem in MongoDB.
• For moodle each question is added in a quiz and for edX each problem is added in a
unit (which lies inside a subsection). So, whenever a unit is created in a subsection
of edX then simultaneously a quiz is created in moodle bearing the subsection name
of the edX course. This is done by a JDBC program.
• Now we have the required quiz in moodle. So, all the questions in the problems un-
der a subsection in edX are transferred to the moodle’s quiz (bearing the subsection
name of edX) one by one. This is done by a JDBC program.
• So, all the problems of edX under a subsection is transferred to a quiz in moodle
under a topic. The moodle topic has the edX section name and the moodle quiz
has the edX subsection name.
• The tables referred/populated to make a quiz in moodle are: mdl quiz,
mdl course sections, mdl course modules, mdl course.
• The tables referred/populated to make a question in moodle are: mdl course,
mdl question, mdl quiz question instance, mdl quiz, mdl question answers,
mdl question multichoice, mdl question categories.
• A trigger to do all these things has been added in compute publish state() function
of edx all/edx-platform/cms/djangoapps/contentstore/views/utils.py.
46
58. CHAPTER 6. TRANSFER OF DATA FROM EDX TO MOODLE
6.5.3 Implementation
Figure 6.27: Creating a Question in edX 1
Figure 6.28: Creating a Question in edX 2
47
59. CHAPTER 6. TRANSFER OF DATA FROM EDX TO MOODLE
Figure 6.29: Checking the created Question in edX
Figure 6.30: Transferred Quiz and Question in moodle 1
48
60. CHAPTER 6. TRANSFER OF DATA FROM EDX TO MOODLE
Figure 6.31: Transferred Quiz and Question in moodle 2
Figure 6.32: Attempting the transferred question in moodle
49
61. CHAPTER 6. TRANSFER OF DATA FROM EDX TO MOODLE
Figure 6.33: Submitting the transferred question in moodle
Figure 6.34: Reviewing the transferred question in moodle
50
62. CHAPTER 6. TRANSFER OF DATA FROM EDX TO MOODLE
6.6 Video Transfer from edX to moodle
6.6.1 Introduction
Transfer of Embedded Video from edX to Moodle involves fetching of URL information
from edX and populate corresponding tables in moodle by that information. To create a
video content in an edX course, youtube link of that video content is required. So that
link along with some course informations are m from edx to populate the moodle tables.
6.6.2 Steps in transferring a video from edX to moodle
• First of all URL information, Course Id, Section name and Sub-section name is
fetched from MongoDB using a JDBC program. (please refer to our code in Ap-
pendix A)
• Then corresponding moodle tables are populated by these informations using same
JDBC program.
• The tables referred or populated to make a question in moodle are: mdl context,
mdl course sections, mdl course modules and mdl url table.
• A trigger is used to directly call the JDBC program using the os.system() command
of python.
• Finally, the Rebuild Course Cache Plugin is used to build the course caches and to
make the video content appear on moodle platform.
• A trigger to do all these things has been added in compute publish state() function
of edx all/edx-platform/cms/djangoapps/contentstore/views/utils.py.
51
63. CHAPTER 6. TRANSFER OF DATA FROM EDX TO MOODLE
6.6.3 Implementation
Figure 6.35: Creating a video in edX
Figure 6.36: A video content created in edX
52
64. CHAPTER 6. TRANSFER OF DATA FROM EDX TO MOODLE
Figure 6.37: Video transferred in moodle
Figure 6.38: An embedded video in moodle
53
65. CHAPTER 6. TRANSFER OF DATA FROM EDX TO MOODLE
6.7 Transfer of Image File from edX to moodle
6.7.1 Introduction
• The transfer of image file from edX to moodle involves fetching the content image
file from the MongoDB database of edx first. Then by using a script, written
in python, the given file is dumped (stored) into a dump folder that stores the
integrated content which was earlier stored as chunks in the MongoDB database.
• Then the image file is copied to the moodledata directory where moo- dle stores its
course contents. After that moodle database is populated with the appropiate field
information related to the file that is earlier copied from edx to moodle. It is done
by using a JDBC program. (please refer to our code in Appendix A)
• The rebuild course cache plugin is run with the help of the script written in python
to make the image appear within a moodle course.
• A trigger to do all these things has been added in compute publish state() function
of edx all/edx-platform/cms/djangoapps/contentstore/views/utils.py.
6.7.2 Fetching the chunks of data from MongoDB database
MongoDB stores the edX files present in a unit in modulestore collection of xmodule
database as a single document. To retrieve the latest file in a unit from MongoDB we
used the following command:
Figure 6.39: Python script to dump image file
6.7.3 Copy the fetched data into moodledata directory
In moodle filenames are given according to the SHA1 hash of their content. This means
each file with particular contents is stored once, irrespective of how many times it is in-
cluded in different places, even if it is referred to by different names. To relate a file on disc
to a user-comprehensible path or filename, you need to use the files database table. Sup-
pose a file has SHA1 contenthash as: 081371cb102fa559e81993fddc230c79205232ce. Then
it will be stored as: moodledata/filedir/08/13/081371cb102fa559e81993fddc230c79205232ce.
The directory under the filedir directory gets the name from the first and second character
of the files contenthash. The subdirectory under the filedir directory gets the name from
the third and fourth character of the files contenthash. This means that moodle cannot
store two files with the same SHA1 hash. We did this using the following command:
54
66. CHAPTER 6. TRANSFER OF DATA FROM EDX TO MOODLE
Figure 6.40: Get the Contenthash and Copy the fetched data into moodledata directory
6.7.4 Change permissions of the new directory created in the
moodledata/filedir directory
We have to change permissions of all the sub-directories present in the moodledata/filedir
directory in order to allow moodle to access the files. The permissions were set using the
following commands
Figure 6.41: Python code to change permission
6.7.5 Populate the moodle tables in the moodle database
We then populated the following tables:
mdl files, mdl context, mdl resource, mdl course, mdl sections, mdl course modules
6.7.6 Naming convention for the transferred image file
As the edX MongoDB database doesnot store the file name of the image so in order to
name the image in the moodle section we use the naming convention as:
“name of the subsection image N”(where N denotes the number of the Nth image added)
6.7.7 Run the rebuild course cache plugin
After transferring the file to the moodledata/filedir directory and populating the tables
we need to rebuild the course cache so that the transferred content would appear on the
required course. We downloaded and installed the course rebuild cache plugin in order
to rebuild the course caches. The plugin has to be copied to the moodle/admin/tools
directory and then can be installed by accessing the following address on the browser:
http://127.0.0.1/moodle/admin/tool/rebuildcoursecache. After the plugin has been in-
stalled it can then be run to rebuild course caches using the following script written in
python.
55
67. CHAPTER 6. TRANSFER OF DATA FROM EDX TO MOODLE
6.7.8 Implementation
Figure 6.42: Uploading an image in edX
Figure 6.43: Image uploaded in edX
56
68. CHAPTER 6. TRANSFER OF DATA FROM EDX TO MOODLE
Figure 6.44: Transferred image in moodle
Figure 6.45: Image appeared as embedded image in moodle
57
69. CHAPTER 6. TRANSFER OF DATA FROM EDX TO MOODLE
6.8 Results
The latency of all the triggers were measured in seconds (across four consequent transfers,
i.e., Transfer 1, 2, 3 and 4) using time() method of python’s time module. All the transfers
were conducted on a 2 GHz Intel Core i7-2630QM processor having 4 GB 1333 MHz DDR3
RAM where 90 courses were live on edX and moodle. The latencies are shown below:
Transfer 1 Transfer 2 Transfer 3 Transfer 4
User Transfer 1.272 s 1.220 s 1.258 s 1.281 s
Course Transfer 1.916 s 1.865 s 1.611 s 1.981 s
Section Transfer 2.665 s 3.113 s 2.016 s 2.443 s
File (PDF) Transfer 3.687 s 3.642 s 3.615 s 3.476 s
Question Transfer 2.277 s 2.275 s 2.204 s 2.244 s
Video Transfer 2.044 s 2.057 s 2.165 s 2.056 s
Table 6.1: Latency for Content Transfer
Latency of PDF Transfer was above 3 seconds in each case. This was because the
PDF was integrated from the chunks of MongoDB. Size of the PDF uploaded was also
varied in each transfer. Details about the size is shown below:
Transfer 1 Transfer 2 Transfer 3 Transfer 4
PDF Size 65 KB 483 KB 614 KB 1.6 MB
Table 6.2: PDF Sizes for each transfer
Load Averages for Transfer 2, 3 and 4 were more or less similar to that of Transfer 1
(as Transfer 1, 2, 3 and 4 were consequent). Also, the load averages for last 5 minutes in
case of PDF is fairly high (1.42). This was infact a contributing factor for high latencies
of PDF Transfer. Load Averages immediately before Transfer 1 are shown below:
Load Averages
for the last
1 minute
Load Averages
for the last
5 minutes
Load Averages
for the last
15 minutes
User Transfer 0.77 0.71 0.63
Course Transfer 0.89 0.99 0.83
Section Transfer 1.47 1.27 0.97
File (PDF) Transfer 1.28 1.42 1.19
Question Transfer 1.18 1.30 1.39
Video Transfer 0.72 0.91 1.19
Table 6.3: Load Averages immediately before Transfer 1
58
70. CHAPTER 6. TRANSFER OF DATA FROM EDX TO MOODLE
6.9 Future Work
The data that was transferred was only the basic user information and content required
for a course. The transfer of file was only considered for .pdf format but files of other
formats (like docx, odt, txt, png, jpg and many more) could also be transferred which
are provided as the course content.
The transfer of quiz was only for multiple choice questions but as edX also has other
type of problems like true/false, check box type, descriptive type, etc so transfer of these
can also be considered in future.
The current work was only for data transferring from edX to moodle but lacks in data
synchronization, i.e., if a course content is modified after creation on the edX then the
modification should be reflected on moodle also. Also, the same can be done vice-versa
too, i.e., transferring the and synchronizing data from edX to moodle.
6.10 Discussion and Conclusion
The transfer of data from edX to moodle was beneficial in the condition when these two
platforms platforms, edX and moodle, were hosted by the same institute/university so
overhead of establishing a course for the different domain of users of both the platforms
will be reduced significantly.
The mdl user table of moodle contains country field which was mandatory to be
populated in order to create a user while auth user table of edX does not contain any
such field. So, the country field was populated by a default country abbreviation (in this
case IN for India) which can be later altered in the profile edit options after user login.
The password in moodle must have at least 8 characters, at least 1 digit(s), at least
1 lower case letter(s), at least 1 upper case letter(s), at least 1 non-alphanumeric char-
acter(s) while there were no such constraints while setting a password in edX. So, to
preserve the integrity of moodle a mail was sent containing the random system generated
password following the integrity constraints which can be altered later in account editing
options.
Also, the option of adding description to files and videos was not available in edX
while it was compulsory in moodle. So, the files and videos from edX are transferred to
moodle with a ”No description available” string with them.
59
71. Appendix A
edX to moodle transfer Code
Github Repository: https://github.com/fresearchgroup/Modifications-to-edX-Platform
A.1 edX to moodle Course transfer
1 import com.mongodb.MongoClient;
2 import com.mongodb.DB;
3 import com.mongodb.DBCollection;
4 import com.mongodb.BasicDBObject;
5 import com.mongodb.DBObject;
6 import com.mongodb.DBCursor;
7 import java.net.UnknownHostException;
8 import java.sql .∗;
9
10 public class edx to moodle course synchronisation {
11 static final String JDBC DRIVER = ”com.mysql.jdbc.Driver”;
12 static final String DB URL = ”jdbc:mysql://localhost/moodle”;
13 static final String USER = ”root”;
14 static final String PASS = ”root”;
15 public static void main(String[] args) throws ClassNotFoundException, UnknownHostException
{
16 // load the sqlite −JDBC driver using the current class loader
17 Class.forName(”org.sqlite.JDBC”);
18 Class.forName(”com.mysql.jdbc.Driver”);
19 String resd = ””, sql = ””;
20 Connection conn = null; //mysql connection
21 Connection connection = null; //sqlite connection
22 try {
23 conn = DriverManager.getConnection(DB URL, USER, PASS);
24 // create a database connection
25 connection = DriverManager.getConnection(”jdbc:sqlite:/home/rajarshi/edx all/db/edx.
db”);
26 Statement stmt = conn.createStatement(); //mysql statement
27 Statement statement = connection.createStatement();
28 statement.setQueryTimeout(30); // set timeout to 30 sec.
29 //read from edx
30 ResultSet rs = statement.executeQuery(”select course id from student courseaccessrole
where id = (select max(id) from student courseaccessrole)”);
31 String course id = rs.getString(”course id”);
32 String [] temp;
33 String delimiter = ”/”;
60
72. APPENDIX A. EDX TO MOODLE TRANSFER CODE
34 temp = course id.split(delimiter);
35 //System.out.println(temp[0]+ ” ” + temp[1] + ” ” +temp[2]);
36 //To connect to mongodb server
37 MongoClient mongoClient = new MongoClient(”localhost”, 27017);
38 //Now connect to your databases
39 DB db = mongoClient.getDB(”xmodule”);
40 //System.out.println(”Connection to Database Successfull”);
41 DBCollection coll = db.getCollection(”modulestore”);
42 //System.out.println(”Collection Selected Successfull ”);
43 BasicDBObject query = new BasicDBObject(” id.category”, ”course”).append(” id.course
”, temp[1]);
44 //BasicDBObject query = new BasicDBObject(” id.category”, ”course”).append(” id.
course”, ”CS201”);
45 DBCursor cursor = coll.find(query);
46 String result = ””;
47 while (cursor.hasNext()) {
48 DBObject tobj = cursor.next();
49 result = tobj.get(”metadata”).toString();
50 }
51 String [] temp1 = result.split(”:”);
52 String resa = temp1[14];
53 String [] resb = resa. split (”,”);
54 //System.out.print(resb[0]) ;
55 String resc = resb[0];
56 resd = resc.substring(2, resc.length() − 2);
57 //System.out.print(”Course Name: ” +resd);
58 cursor. close ();
59 String coursename = resd;
60 String shortname = generateInitials(coursename);
61 String idtomatch = ””;
62 //read from moodle
63 rs = stmt.executeQuery(”select idnumber from mdl course where id = (select max(id)
from mdl course)”);
64 while (rs.next()) {
65 idtomatch = (String) rs.getString(”idnumber”).toString();
66 }
67 if (temp[1].compareTo(idtomatch) != 0) {
68 //write into moodle(mdl course)
69 sql = ”insert into mdl course (category,fullname,shortname,sortorder,idnumber,
summary,summaryformat,showgrades,newsitems,startdate,marker,maxbytes,
legacyfiles,showreports,visible,visibleold,groupmode,groupmodeforce,
defaultgroupingid,timecreated,timemodified,requested,enablecompletion,
completionnotify)” + ”values (1,’” + coursename + ”’,’” + shortname + ”
’,10000,’” + temp[1] + ”’,’Course Summary has not been entered
’,1,1,5,1609353000,0,0,0,0,1,1,0,0,0,0,0,0,0,0)”;
70 // startdate (31/12/2020)= 1609353000
71 stmt.executeUpdate(sql);
72 stmt.executeUpdate(”update mdl course set sortorder=sortorder+1”);
73 stmt.executeUpdate(”update mdl course set sortorder=1 where id=1;”);
74 }
75 } catch (SQLException e) {
76 // if the error message is ”out of memory”, it probably means no database file is found
77 System.err.println(e.getMessage());
78 } finally {
79 try {
80 if (connection != null) connection.close ();
81 } catch (SQLException e) {
82 // connection close failed .
61
77. APPENDIX A. EDX TO MOODLE TRANSFER CODE
198 result = tobj.get(” id”).toString();
199 resultcopy = tobj.get(” definition ”).toString
();
200 //System.out.println(” ”+result);
201 String [] temp1vr = result.split(”:”);
202 String resavr = temp1vr[5];
203 String [] resbvr = resavr. split (”,”);
204 String rescvr = resbvr[0];
205 String courseidvr = rescvr.substring(2,
rescvr.length() − 2);
206 //System.out.println(courseidvr);
207 if ( verticalids [m].compareTo(courseidvr)
== 0) {
208 //System.out.println(resultcopy);
209 String [] tempaavr = resultcopy.split(”]”
);
210 //System.out.println(tempaach[0]);
211 String resiivr = tempaavr[0].substring
(17, tempaavr[0].length());
212 String resiiivr = resiivr .replace(” ”, ”
”).replace(”””, ””);
213 //System.out.println( resiiivr );
214 String [] temp9 = resiiivr. split (”,”);
215 String [] problemids = new String[temp9.
length];
216 for (int n = 0; n < temp9.length; n++)
{
217 problemids[n] = temp9[n].substring(
temp9[n].length() − 32, temp9[n
].length());
218 //System.out.println(” ”+
problemids[n]+”: Problem/
Video/Image”); // ids of all
problems
219 if (problemids[n].compareTo(
problemid edx) == 0) {
220 //System.out.println(”
nProblem is in Section ”+(i
+1));
221 //System.out.println(”
nProblem is in Section ”+(i
+1));
222 section it was added = i + 1;
223 subsection it was added = l + 1;
224 subsection name to be sent =
subsectioname;
225 //System.out.println(”image:n
”+
image description to be sent
);
226 break;
227 }
228 }
229 }
230 }
231 }
232 }
233 }
66
78. APPENDIX A. EDX TO MOODLE TRANSFER CODE
234 }
235 }
236 }
237 }
238 }
239 }
240 }
241 subsection name to be sent = subsection name to be sent.substring(20,
subsection name to be sent.length() − 2);
242 System.out.print(”Image is in Section: ” + section it was added + ”, Image is in
SubSection: ” + subsection it was added);
243 System.out.println(”, Subsection Name: ” + subsection name to be sent);
244 image description to be sent = image description to be sent.substring(
image description to be sent .indexOf(”<h2>”), image description to be sent.indexOf
(”</p>n<p><img”));
245 image description to be sent = image description to be sent.replace(”<h2>”, ””);
246 image description to be sent = image description to be sent.replace(”</h2>”, ””);
247 image description to be sent = image description to be sent.replace(”Full Screen Image”,
””);
248 image description to be sent = image description to be sent.replace(”<p>”, ””);
249 image description to be sent = image description to be sent.replace(”n”, ””);
250 System.out.println(image description to be sent);
251 System.out.println(”Image no. in the course: ” + image no in the course);
252 System.out.println(”Image name: ” + subsection name to be sent.concat(” Image ”).
concat(Integer.toString(image no in the course)));
253 p = Runtime.getRuntime().exec(new String[] {
254 ”/bin/bash”, ”−c”, ”file −−mime−type −b /home/rajarshi/
edx to moodle synchronisation/image transfer/image”
255 });
256 input = new BufferedReader(new InputStreamReader(p.getInputStream()));
257 String mimetype = ””;
258 while (( line = input.readLine()) != null) {
259 mimetype = line;
260 }
261 System.out.println(mimetype);
262 String filepathforsize = ”/var/moodledata/filedir/” + subdirectory1 + ”/” +
subdirectory2 + ”/” + contenthash;
263 System.out.println( filepathforsize );
264 long filesize = getFileSize( filepathforsize );
265 System.out.println( filesize );
266 //for mdl file part 1..................................
267 int contextId = 5;
268 String component = ”user”;
269 String filearea = ”draft”;
270 String [] temp = mimetype.split(”/”);
271 String filename = subsection name to be sent.concat(” Image ”).concat(Integer.toString
(image no in the course)) + ”.” + temp[1];
272 System.out.println(filename);
273 String filepath = ”/”;
274 int itemid = (int)(Math.random() ∗ 1000000000);
275 String inputforpathhash = ”/” + contextId + ”/” + component + ”/” + filearea + ”/”
+ itemid + ”/” + filename;
276 String pathnamehash = sha1(inputforpathhash);
277 String sourcename = ”O:8:”stdClass”:1:{s:6:”source”;s:38:”” + filename + ””;}”;
278 //System.out.println(sourcename);
279 sql = ”Insert into mdl files (contenthash,pathnamehash,contextid,component,filearea,
itemid,filepath,filename,userid, filesize ,mimetype,status,source,author,license ,
timecreated,timemodified,sortorder)” + ”values (’” + contenthash + ”’,’” +
67