FRG_P4_Final_Project_Report_Modifications_to_edX_platform

IITB Summer Internship 2014
Project Report
Modiﬁcations to edX Platform
Principal Investigator
Prof. D.B. Phatak
Project In-Charge
Mr. Nagesh Karmali
Project Mentor Project Team Members
Srinu Kolukuluri Anand Solanki
Divyanshu Goyal
Grishma Jena
Harsh Sharma
Kripal Gaurav
Pushkar Narayan
Rajarshi Sarkar
Last Updated: July 2, 2014

Summer Internship 2014
Project Approval Certiﬁcate
Department of Computer Science and Engineering
Indian Institute of Technology Bombay
The project entitled “Modiﬁcations to edX Platform” submitted by Anand Solanki,
Divyanshu Goyal, Grishma Jena, Harsh Sharma, Kripal Gaurav, Pushkar Narayan and
Rajarshi Sarkar is approved for Summer Internship 2014 programme from 10th May 2014
to 6th July 2014, at Department of Computer Science and Engineering, IIT Bombay.
Prof. Deepak B. Phatak
Dept of CSE, IITB
Principal Investigator
Mr. Nagesh Karmali
Dept of CSE, IITB
Project In-charge
Place: IIT Bombay, Mumbai
Date: July 2, 2014

Declaration
I declare that this written submission represents my ideas in my own words and where
others’ ideas or words have been included, I have adequately cited and referenced the orig-
inal sources. I also declare that I have adhered to all principles of academic honesty and
integrity and have not misrepresented or fabricated or falsiﬁed any idea/data/fact/source
in my submission. I understand that any violation of the above will be cause for disci-
plinary action by the Institute and can also evoke penal action from the sources which
have thus not been properly cited or from whom proper permission has not been taken
when needed.
Anand Solanki
NIT, Agartala
Divyanshu Goyal
BITS, Pilani
Grishma Jena
Symbiosis Institute of Technology
Harsh Sharma
MNIT, Jaipur
Kripal Gaurav
NIT, Agartala
Pushkar Narayan
NIT, Agartala
Rajarshi Sarkar
BIT, Mesra
Date: July 2, 2014

Acknowledgement
We, the summer interns of the team Modiﬁcations to edX Platform, are overwhelmed
in all humbleness and gratefulness to acknowledge our deep gratitude to all those who
have helped us put our ideas to perfection and have assigned tasks, well above the level
of simplicity and into something concrete and unique We, whole heartedly thank Prof.
D.B. Phatak for having faith in us, selecting us to be a part of his valuable project and
for constantly motivating us to do better. We are very thankful to our project incharge,
Mr. Nagesh Karmali, Mrs. Sukla Nag and our mentor Mr. Srinu Kolukuluri for their
valuable suggestions. They were and are always there to show us the right track when
needed help. With help of their brilliant guidance and encouragement, we all were able
to complete our tasks properly and were up to the mark in all the tasks assigned. During
the process, we got a chance to see the stronger side of our technical and non-technical
aspects and also strengthen our concepts. Here by, we gladly consider ourselves to be
the most fortunate batch of interns. Last but not the least, we whole heartedly thank all
our other colleagues working in diﬀerent projects under Prof. D.B Phatak for helping us
evolve better with their critical advice.

Abstract
MOOCs have brought distance learning into the public discourse like nothing ever be-
fore. Two renowned examples of MOOCs are edX and moodle. The edX platform uses
online learning software that uses interactive experiences and moodle has several features
considered typical of an e-learning platform, in addition to some original innovations like
its filtering system. Under this project, we have transferred content from edX to moodle
and also from edX to edX to enhance the process of e-learning. EdX to moodle trans-
fer is benificial when these two platforms are hosted by the same institute/university so
overhead of registering new users or establishing a course for the different domain of users
of both the platforms will be reduced significantly while edX to edX transfer is benificial
when a parent institution will like to offer its courses to the remote centres affiliated with
it.

List of Figures
5.1 edX Distributed Platform for Course Synchronisation Architecture . . . . 12
5.2 edX Distributed Platform for Course Synchronisation in Django-admin Panel 14
5.3 Offered Courses in Django-admin Panel . . . . . . . . . . . . . . . . . . . 15
5.4 Universities in Django-admin Panel . . . . . . . . . . . . . . . . . . . . . 16
5.5 Admin Panel for offering Courses . . . . . . . . . . . . . . . . . . . . . . 17
5.6 Admin Panel for offering Courses . . . . . . . . . . . . . . . . . . . . . . 17
5.7 University Panel for Remote Institute . . . . . . . . . . . . . . . . . . . . 18
5.8 edX Distributed Platform for Course Synchronisation Logs . . . . . . . . 23
6.1 User Registration page . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
6.2 Registration information of User . . . . . . . . . . . . . . . . . . . . . . . 29
6.3 Registration information of User . . . . . . . . . . . . . . . . . . . . . . . 30
6.4 User registration confirmation . . . . . . . . . . . . . . . . . . . . . . . . 30
6.5 Moodle admin contains newly added user . . . . . . . . . . . . . . . . . . 31
6.6 User login in Moodle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
6.7 User Home Page in Moodle . . . . . . . . . . . . . . . . . . . . . . . . . 32
6.8 Trigger in course.py file for transferring Course . . . . . . . . . . . . . . . 34
6.9 Courses in edX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
6.10 Courses in moodle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
6.11 Course creation in edX . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
6.12 Created course in edX . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
6.13 Transferred course in moodle . . . . . . . . . . . . . . . . . . . . . . . . . 37
6.14 Transferred course content in moodle . . . . . . . . . . . . . . . . . . . . 37
6.15 Before creating a section in edX . . . . . . . . . . . . . . . . . . . . . . . 39
6.16 Before creating a topic in moodle . . . . . . . . . . . . . . . . . . . . . . 39
6.17 Creating a section in edX . . . . . . . . . . . . . . . . . . . . . . . . . . 40
6.18 After creating a section in edX . . . . . . . . . . . . . . . . . . . . . . . . 40
6.19 Transferred edX section in moodle as a topic . . . . . . . . . . . . . . . . 41
6.20 Retrieve File from MongoDB . . . . . . . . . . . . . . . . . . . . . . . . . 42
6.21 Get the Contenthash and Copy the fetched data into moodledata directory 42
6.22 Set permissions for the files in moodledata/filedir . . . . . . . . . . . . . 42
6.23 Uploading a file in edX 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
6.26 File in edX transferred to moodle . . . . . . . . . . . . . . . . . . . . . . 45
6.27 Creating a Question in edX 1 . . . . . . . . . . . . . . . . . . . . . . . . 47
6.28 Creating a Question in edX 2 . . . . . . . . . . . . . . . . . . . . . . . . 47
6.29 Checking the created Question in edX . . . . . . . . . . . . . . . . . . . 48
ii

LIST OF FIGURES
6.30 Transferred Quiz and Question in moodle 1 . . . . . . . . . . . . . . . . 48
6.31 Transferred Quiz and Question in moodle 2 . . . . . . . . . . . . . . . . 49
6.32 Attempting the transferred question in moodle . . . . . . . . . . . . . . . 49
6.33 Submitting the transferred question in moodle . . . . . . . . . . . . . . . 50
6.34 Reviewing the transferred question in moodle . . . . . . . . . . . . . . . 50
6.35 Creating a video in edX . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
6.36 A video content created in edX . . . . . . . . . . . . . . . . . . . . . . . 52
6.37 Video transferred in moodle . . . . . . . . . . . . . . . . . . . . . . . . . 53
6.38 An embedded video in moodle . . . . . . . . . . . . . . . . . . . . . . . . 53
6.39 Python script to dump image ﬁle . . . . . . . . . . . . . . . . . . . . . . 54
6.40 Get the Contenthash and Copy the fetched data into moodledata directory 55
6.41 Python code to change permission . . . . . . . . . . . . . . . . . . . . . . 55
6.42 Uploading an image in edX . . . . . . . . . . . . . . . . . . . . . . . . . 56
6.43 Image uploaded in edX . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
6.44 Transferred image in moodle . . . . . . . . . . . . . . . . . . . . . . . . . 57
6.45 Image appeared as embedded image in moodle . . . . . . . . . . . . . . . 57
iii

List of Tables
5.1 Oﬀered Courses Table . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
5.2 Universities Table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
6.1 Latency for Content Transfer . . . . . . . . . . . . . . . . . . . . . . . . 58
6.2 PDF Sizes for each transfer . . . . . . . . . . . . . . . . . . . . . . . . . 58
6.3 Load Averages immediately before Transfer 1 . . . . . . . . . . . . . . . 58
iv

Contents
1 Introduction to MOOC 1
1.1 Beneﬁts of MOOC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Possible challenges of MOOC . . . . . . . . . . . . . . . . . . . . . . . . 1
2 edX Installation [1] [2] 3
2.1 Pre-Requisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.2 Setting Proxy Information . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.3 Installation (Do everything being root) . . . . . . . . . . . . . . . . . . . 5
2.4 To run the Servers: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3 Moodle Installation[3] 8
4 Database Management and the Tools 11
4.1 Database management system used by moodle . . . . . . . . . . . . . . . 11
4.2 Database management system used by edX . . . . . . . . . . . . . . . . . 11
5 edX Distributed Platform for Course Synchronisation 12
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
5.2 edX Distributed Platform for Course Synchronisation Installation . . . . 13
5.3 Database Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
5.4 Admin Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
5.5 University Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
5.6 Course Management in edX using mongoDB . . . . . . . . . . . . . . . . 18
5.7 Course Transfer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
5.8 Remove Course from mongoDB . . . . . . . . . . . . . . . . . . . . . . . 22
5.9 Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
5.10 Update Courses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
5.10.1 Automatic Update . . . . . . . . . . . . . . . . . . . . . . . . . . 25
5.10.2 Manual Update . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
5.11 Future Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
6 Transfer of Data from edX to Moodle 26
6.1 User Content Transfer from edX to Moodle . . . . . . . . . . . . . . . . . 26
6.1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
6.1.2 User tables in edX and Moodle . . . . . . . . . . . . . . . . . . . 26
6.1.3 Creating Java Database Connectivity Application . . . . . . . . . 26
6.1.4 Password . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
6.1.5 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
6.2 Course Transfer from edX to moodle . . . . . . . . . . . . . . . . . . . . 32
v

CONTENTS
6.2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
6.2.2 Creating JDBC Application . . . . . . . . . . . . . . . . . . . . . 32
6.2.3 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
6.3 Section Transfer from edX to moodle . . . . . . . . . . . . . . . . . . . . 38
6.3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
6.3.2 Steps in transferring a section from edX to moodle . . . . . . . . 38
6.3.3 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
6.4 File Transfer from edX to moodle . . . . . . . . . . . . . . . . . . . . . . 41
6.4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
6.4.2 Fetching the chunks of data from MongoDB database . . . . . . . 41
6.4.3 Copy the fetched data into moodledata directory . . . . . . . . . 42
6.4.4 Change permissions of the new directory created in the moodle-
data/filedir directory . . . . . . . . . . . . . . . . . . . . . . . . . 42
6.4.5 Populate the moodle tables in the moodle database . . . . . . . . 43
6.4.6 Run the rebuild course cache plugin . . . . . . . . . . . . . . . . . 43
6.4.7 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
6.5 Question Transfer from edX to moodle . . . . . . . . . . . . . . . . . . . 45
6.5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
6.5.2 Steps in transferring a question from edX to moodle . . . . . . . . 46
6.5.3 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
6.6 Video Transfer from edX to moodle . . . . . . . . . . . . . . . . . . . . . 51
6.6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
6.6.2 Steps in transferring a video from edX to moodle . . . . . . . . . 51
6.6.3 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
6.7 Transfer of Image File from edX to moodle . . . . . . . . . . . . . . . . . 54
6.7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
6.7.2 Fetching the chunks of data from MongoDB database . . . . . . . 54
6.7.3 Copy the fetched data into moodledata directory . . . . . . . . . 54
6.7.4 Change permissions of the new directory created in the moodle-
data/filedir directory . . . . . . . . . . . . . . . . . . . . . . . . . 55
6.7.5 Populate the moodle tables in the moodle database . . . . . . . . 55
6.7.6 Naming convention for the transferred image file . . . . . . . . . . 55
6.7.7 Run the rebuild course cache plugin . . . . . . . . . . . . . . . . . 55
6.7.8 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
6.8 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
6.9 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
6.10 Discussion and Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . 59
A edX to moodle transfer Code 60
A.1 edX to moodle Course transfer . . . . . . . . . . . . . . . . . . . . . . . . 60
A.2 edX to moodle Image transfer . . . . . . . . . . . . . . . . . . . . . . . . 62
A.3 edX to moodle PDF transfer . . . . . . . . . . . . . . . . . . . . . . . . . 71
A.4 edX to moodle Quiz transfer . . . . . . . . . . . . . . . . . . . . . . . . . 76
A.5 edX to moodle Section transfer . . . . . . . . . . . . . . . . . . . . . . . 86
A.6 edX to moodle User transfer . . . . . . . . . . . . . . . . . . . . . . . . . 88
A.7 edX to moodle Video transfer . . . . . . . . . . . . . . . . . . . . . . . . 90
A.8 Password generation and emailing to user . . . . . . . . . . . . . . . . . . 96
vi

CONTENTS
B edX Distributed Platform for Course Synchronisation 98
B.1 models.py . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
B.2 tasks.py . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
B.3 views.py . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
vii

Chapter 1
Introduction to MOOC
MOOC stands for Massive Open Online Course. It is a format to organize learning/teach-
ing/training on a speciﬁc topic in a more informal collaborative way. It is a gathering
of participants, of people willing to jointly exchange knowledge and experiences for each
of them to build upon. As such it is within the hands of the participants and organizers
of a MOOC to change it to their needs. This allows them to use the information and
to construct their own ideas or projects. Moodle and edX are two common examples of
MOOC. [4]
1.1 Beneﬁts of MOOC
• It can be organized in any setting that has connectivity (which can include the
Web, but also local connections via Wi-Fi e.g.)
• It can be organized in in any language you like (taking into account the main
language of target audience)
• It can be organized as quickly as you can inform the participants (which makes it
a powerful format for priority learning in e.g. aid relief)
• Contextualized content can be shared by all.
• It can be used beyond time zones and physical boundaries.
• Learning happens in a more informal setting, at a place of your convenience and
often around your own schedule.
• There is no need of having a degree to follow the course, only the willingness to
learn (at high speed).
• Own personal learning environment and/or network can be added by participating
in a MOOC.
1.2 Possible challenges of MOOC
• It feels chaotic as participants create their own content
• It demands digital literacy
1

CHAPTER 1. INTRODUCTION TO MOOC
• It demands time and eﬀort from the participants
• It is organic, which means the course will take on its own trajectory (you have got
to let go).
• As a participant you need to be able to self-regulate your learning and possibly give
yourself a learning goal to achieve.
2

Chapter 2
edX Installation [1] [2]
2.1 Pre-Requisites
1. Fresh ubuntu 12.04 LTS and execute sudo apt-get update and sudo apt-get upgrade
2. In a fresh ubuntu system root doesnt actually have a password set. So, set the root
password:
sudo passwd
3. Install paver. You must install this being root. Commands for this are given below:
su
pip install -U Paver
4. Django 1.4.12 (Download from https://pypi.python.org/pypi/Django/1.4.12). Please
don’t install any other version of django as the edX platform supports django 1.4.12.
Add the following line in the ∼/.bashrc ﬁle. To edit the ∼/.bashrc ﬁle use the fol-
lowing command:
nano ∼/.bashrc
Then add the following line in the end:
where /path/to/Django-1.4.12 is the path to your django directory.
5. Install git using the following command:
sudo apt-get install git
3

CHAPTER 2. EDX INSTALLATION [?] [?]
2.2 Setting Proxy Information
1. Download ubproxy and set the proxy settings applicable.
cd /path_to/ubproxy
chmod 777 ubproxy
sudo ./ubproxy
Set the host (proxy server url), port (proxy port number), proxy authentication
username and proxy authentication password.
2. Login as root (su)
In the following steps change proxyusername by your proxy username, proxypass-
word by your proxy password., proxy.server.com by your proxy server url, proxy-
portno by proxy port.
3. Set proxy information in /etc/apt/apt.conf (nano /etc/apt/apt.conf)
Acquire::http::proxy
"http://proxyusername:proxypassword@proxy.server.com:proxyportno/";
Acquire::ftp::proxy
"ftp://proxyusername:proxypassword@proxy.server.com:proxyportno/";
Acquire::https::proxy
"https://proxyusername:proxypassword@proxy.server.com:proxyportno/";
save the ﬁle and exit (ctrl + x)
4. Add the lines in /etc/sudoers (nano /etc/sudoers)
Defaults env_reset
Defaults
secure_path="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
Defaults env_keep += "http_proxy https_proxy ftp_proxy socks_proxy no_proxy"
5. Add the lines in /etc/environment (nano /etc/environment)
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games"
socks_proxy="socks://proxy.server.com:proxyportno/"
no_proxy="localhost"
http_proxy="http://proxyusername:proxypassword@proxy.server.com:proxyportno"
ftp_proxy="ftp://proxyusername:proxypassword@proxy.server.com:proxyportno"
https_proxy="https://proxyusername:proxypassword@proxy.server.com:proxyportno"
6. Set git proxy.
git config --global http.proxy
https://proxyusername:proxypassword@proxy.server.com:proxyportno
git config --global https.proxy
https://proxyusername:proxypassword@proxy.server.com:proxyportno
4

2.3 Installation (Do everything being root)
1. Clone the edx-platform repository
sudo apt-get update
sudo apt-get -y install git vim
mkdir /home/your_username_folder/edx_all
cd /home/your_username_folder/edx_all
git clone https://github.com/edx/edx-platform.git
cd edx-platform
2. Install EDX Requirements
sudo apt-get install python-software-properties
sudo add-apt-repository -y ppa:chris-lea/node.js
sudo add-apt-repository -y ppa:chris-lea/node.js-libs
sudo add-apt-repository -y ppa:chris-lea/libjs-underscore
sudo apt-get -y update
DEBIAN FRONTEND=noninteractive
sudo apt-get -yq install gfortran graphviz libgraphviz-dev graphviz-dev
libatlas-dev libblas-dev python-software-properties pkg-config gfortran
libatlas-dev libblas-dev liblapack-dev liblapack3gf curl git python-virtualenv
python-scipy python-numpy build-essential python-dev gfortran libfreetype6-dev
libpng12-dev libjpeg-dev libtiff4-dev zlib1g-dev libxml2-dev libxslt-dev
yui-compressor graphviz libgraphviz-dev graphviz-dev mysql-server
libmysqlclient-dev libgeos-dev libreadline6 libreadline6-dev mongodb nodejs
coffeescript mysql-client virtualenvwrapper libgeos-ruby1.8 lynx-cur
git clone https://github.com/sstephenson/rbenv.git ~/.rbenv
git clone https://github.com/sstephenson/ruby-build.git
~/.rbenv/plugins/ruby-build
echo "export PATH="$HOME/.rbenv/bin:$PATH"" >> $HOME/.${shelltype}rc
echo ’eval "$(rbenv init -)"’ >> $HOME/.${shelltype}rc
export PATH="$HOME/.rbenv/bin:$PATH
eval "$(rbenv init -)"
rbenv install 1.9.3-p374
rbenv global 1.9.3-p374
gem install bundler
rbenv rehash
bundle install --gemfile Gemfile
export WORKON_HOME=$HOME/.virtualenvs
source /etc/bash_completion.d/virtualenvwrapper
mkvirtualenv -a "$HOME/.virtualenvs" --system-site-packages edx-platform
curl -sL -o numpy.tar.gz
http://downloads.sourceforge.net/project/numpy/NumPy/1.6.2/numpy-1.6.2.tar.gz
curl -sL -o scipy.tar.gz
http://downloads.sourceforge.net/project/scipy/scipy/0.10.1/scipy-0.10.1.tar.gz
tar xf numpy.tar.gz
tar xf scipy.tar.gz
rm -f numpy.tar.gz scipy.tar.gz
5

cd numpy-1.6.2
python setup.py install
cd ..
cd scipy-0.10.1
cd ..
rm -rf numpy-1.6.2 scipy-0.10.1
cd $HOME/.virtualenvs/edx-platform/lib/python2.7/site-packages
wget
http://pypi.python.org/packages/source/d/distribute/distribute-0.6.28.tar.gz
tar -xzvf distribute-0.6.28.tar.gz
cd distribute-0.6.28/
cd ..
rm distribute-0.6.28.tar.gz
pip install -r
/home/your_username_folder/edx_all/edx-platform/requirements/edx/pre.txt
cd /home/your_username_folder/edx_all/edx-platform/
sudo apt-get install phantomjs
npm config set strict-ssl false
npm config set registry "http://registry.npmjs.org/"
rake install_prereqs
pip install argcomplete
cd /home/your_username_folder/edx_all/edx-platform
bundle install
rake install_prereqs
mkdir db log data
cd /home/your_username_folder/edx_all/edx-platform
./manage.py lms syncdb --noinput migrate
./manage.py cms syncdb --noinput
./manage.py cms syncdb --noinput migrate
./manage.py cms createsuperuser
The installation is complete here!! Now reboot your system! (sudo
reboot)
2.4 To run the Servers:
1. Download the run lms.sh and run cms.sh file with this documentation. Open
the files and edit the following line: cd /home/your username folder/edx all/edx-
platform
2. Login as root and go the directory where run lms.sh file is present. Run the
run lms.sh file in the terminal.
su
6

bash run lms.sh
This will run LMS (check at localhost:8000).
3. After the run lms.sh script starts the LMS server open a new terminal tab. Login
as root and go the directory where run cms.sh file is present. Run the run cms.sh
file in the terminal.
su
bash run cms.sh
This will run CMS (check at localhost:8001).
4. Now install Robomongo (to access MongoDB data) and SQLite database browser
or Sqliteman (to access SQLite Database).
5. If decide to use the edX platform without any email configuration then to activate a
registered user fetch the activation key of the user from the auth registration table
in SQLite database and paste the following link in your browser:
localhost:8000/activate/activation key fetched from auth registration tabl e
where activation key fetched from auth registration table is the activation key of
the user from the auth registration table in SQLite database.
Now you can login into LMS and CMS successfully.
7

Chapter 3
Moodle Installation[3]
1. Install Apache/MySQL/PHP
Open up Terminal and install the following;
sudo apt-get update
sudo apt-get install apache2 mysql-client mysql-server php5
’It will prompt you to set the root password for mysql - take note of it,
you will need it in step 5’
2. Install Additional Software
sudo apt-get install aspell graphviz php5-pspell php5-curl php5-gd php5-intl php5-
mysql php5-xmlrpc php5-ldap
Restart Apache so that the modules are loaded correctly
sudo service apache2 restart
We will be using Git to install/update the Moodle Core Application
sudo apt-get install git-core
3. Download Moodle
Setup your local repository and download Moodle, We will use /opt for the local
repository.
cd /opt
sudo git clone git://github.com/moodle/moodle.git
cd moodle
sudo git branch -a
sudo git branch –track MOODLE 25 STABLE origin/MOODLE 25 STABLE
sudo git checkout MOODLE 25 STABLE
4. Copy local repository to /var/www
sudo cp -R /opt/moodle /var/www/
sudo mkdir /var/moodledata
sudo chown -R www-data /var/moodledata
sudo chmod -R 777 /var/moodledata
8

CHAPTER 3. MOODLE INSTALLATION[?]
sudo chmod -R 0755 /var/www/moodle
5. Setup MySQL Server
First we need to change the default storage engine to innodb
sudo vi /etc/mysql/my.cnf
Scroll down to the [mysqld] section and under Basic Settings add the following line
under the last statement
default-storage-engine = innodb
In order to save my.cnf using the editor, type the following in sequence which will
save then close the editor:
:w
:q
Restart MySQL Server for changes to take affect
sudo service mysql restart
Now we need to create the Moodle MySQL User with the correct permissions
Use the password you created in step 1
mysql -u root -p
mysql> CREATE DATABASE moodle DEFAULT CHARACTER SET utf8 COL-
LATE utf8 unicode ci;
Where it says ”moodledude” and ”passwordformoodledude” you should change to
the username and password of your choosing.
mysql> GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,CREATE TEM-
PORARY TABLES,DROP,INDEX,ALTER ON moodle.* TO moodledude@localhost
IDENTIFIED BY ’passwordformoodledude’;
mysql> quit;
6. Complete Setup
Note - If you are not comfortable using terminal to create the config.php file that
needs to be created when going through the installer, you should temporarily make
the webroot writable by doing the following:
After you have ran the installer and you have moodle setup, you NEED to revert
permissions so that it is no longer writable using the below command.
7. Open your browser and go to http://IP.ADDRESS.OF.SERVER/moodle
Follow the prompts selecting: Database Type
Choose: mysqli Database Settings
Host server: localhost
Database: moodle
User: moodledude (the user you created when setting up the database)
9

CHAPTER 3. MOODLE INSTALLATION[?]
Password: passwordformoodledude (the password for the user you created)
Tables Preﬁx: mdl Environment Checks
This will indicate if any elements required to run moodle haven’t been installed.
Next next next...
follow prompts and conﬁrm installation Create a Site Administrator Account
Create your moodle user account which will have site administrator permissions.
The password you select has to meet certain security requirements. Installation
Complete
Congrats! You can now start using Moodle!
10

Chapter 4
Database Management and the Tools
4.1 Database management system used by moodle
The database that is used by moodle platform is:
• MySQL : MySQL is a freely available open source Relational Database Manage-
ment System (RDBMS) that uses Structured Query Language (SQL).
The tool used for interacting with MySQL database is MySQL workbench.
MySQL Workbench is a visual database design tool that integrates SQL devel-
opment, administration, database design, creation and maintenance into a single
integrated development environment for the MySQL database system.
4.2 Database management system used by edX
The database that is used by edX platform is:
• SQLite : SQLite is a relational database management system contained in a C
programming library. In contrast to other database management systems, SQLite
is not a separate process that is accessed from the client application, but an integral
part of it.
The tool used for interacting with SQLite database is SQLite browser. SQLite
Database Browser is a high quality, visual, open source tool to create, design, and
edit database ﬁles compatible with SQLite.
• MongoDB :[5] MongoDB is an open-source document database, and the leading
NoSQL database.It is written in C++.
The tool used for interacting with MongoDB database is Robomongo. Robomongo
is a shell-centric cross-platform open source MongoDB management tool.
11

Chapter 5
edX Distributed Platform for
Course Synchronisation
5.1 Introduction
edX Distributed Platform for Course Synchronisation is an application developed for edx
using Django to facilitate selective transfer of course data across remote edX platforms
and with the motive of improving online education. RCMS is developed to reduce load of
thousands of users on a single server by distributing courses among various remote servers
and to facilitate independent working of all the remote servers. RCMS provides func-
tionalities such as course data transfer, course data removal, automatic/manual update,
oﬀering courses, removing courses from oﬀered courses list.
Figure 5.1: edX Distributed Platform for Course Synchronisation Architecture
12

CHAPTER 5. EDX DISTRIBUTED PLATFORM FOR COURSE
SYNCHRONISATION
5.2 edX Distributed Platform for Course Synchroni-
sation Installation
For github README.md of edX Distributed Platform for Course Synchronisation, please
visit https://github.com/fresearchgroup/Modifications-to-edX-Platform/
Up and Running with RCMS
Your edX Installation folder should have this structure after copying RCMS files.
* cms
* envs
* common.py
* urls.py
* common
* djangoapps
* rcms
* dump
* logs
* rcms.log
* update
* admin.py
* __init__.py
* models.py
* rcms.cfg
* tasks.py
* views.py
* static
* rcms
* dibu_server.css
* templates
* remote_sync
* not_authorized.html
* server_interface.html
* user_interface.html
Change the dump path, update path, log path in rcms.cfg to rcms folder in
your edX Installation directory. See the rcms.cfg file for more information. Copy all the
files present in Remote Course Management System folder to your edx-Installation
directory according to the folder structure. Create a Virtual Environment using these
commands:
workon edx-platform
Change your current directory to edx-platform/ having manage.py file. Make man-
age.py executable by:
sudo chmod +x manage.py
13

SYNCHRONISATION
Then sync db using:
./manage.py cms syncdb
./manage.py cms syncdb --migrate
Access RCMS-Admin Panel using http://localhost:8001/rcms and signin using
admin/superuser account which you created earlier during installation process.
Access University Panel on Remote-Machine: http://(Your Public IP):8001/rcms
and signin with University Account.
The University Account is a user-account verified by Django-admin of edX. Just go to
Django-admin panel by visiting http://localhost:8001/admin log in with your su-
peruser credentials, then go to rcms panel, then Universities, add the user from the
dropdown list and enter other details manually.
Automatic Update has been implemented using celery beat, which is pre-installed in
edX. For automatic update, you need to install rabbitmq-server for ubuntu: Download
Link: http://www.rabbitmq.com/download.html
sudo dpkg -i install rabbitmq-server*.deb
sudo apt-get -f install
Now to enqueue the Update Task,
First make a periodic task by going to django-admin panel, then djcelery tab, then
periodic task, add periodic task Select rcms.tasks.update from registered tasks. select
an interval, and save the task. Now create a Virtual Environment by these commands:
workon edx-platform
Change your current directory to edx-platform/ directory having a manage.py file.
Make manage.py executable by:
sudo chmod +x manage.py
Now Run celery beat using manage.py:
./manage.py cms celery beat
5.3 Database Models
edX Distributed Platform for Course Synchronisation is included under CMS model of
edX. It consists of two tables i.e. Offered Courses and Universities.
Figure 5.2: edX Distributed Platform for Course Synchronisation in Django-admin Panel
14

SYNCHRONISATION
The Schema of the tables [6] are as follows:
Sl No. Field Name Field Description
1 name CharField(max length=30)
2 course id CharField(max length=10, unique= True)
3 org CharField(max length=50)
4 start date DateField(blank = True)
5 course url URLField(blank = True)
Table 5.1: Oﬀered Courses Table
Figure 5.3: Oﬀered Courses in Django-admin Panel
Sl No. Field Name Field Description
1 user OneToOneField(User, unique=True)
2 name CharField(max length = 100,unique= True)
3 city CharField(max length = 30)
4 state CharField(max length = 30)
5 courses taken CharField(max length = 1000, blank = True)
6 server ip CharField(max length=30, blank = True)
Table 5.2: Universities Table
15

SYNCHRONISATION
Figure 5.4: Universities in Django-admin Panel
Here Universities has a user field which has foreign key to the User model of
django.contrib.auth.models. Please refer to our code in Appendix B.
5.4 Admin Interface
The admin interface is available for the admin of IIT Bombay which is the university
offering the various online courses. The interface provides functionalities needed for the
admin to execute his responsibilities.
The IITB admin can add universities and admins for the respectives institutes which
will be offered, the online courses given by IIT Bombay. This is done by accessing the
RCMS admin interface via http://localhost:8001/rcms .
The RCMS admin module has 3 panels. The interface connects with the edX on the
fly. The courses available that can be offered to the institutes is shown in the first panel.
The admin can select any number of those courses and choose to offer them to subscribed
universities. The second panel displays a list of all the courses that are being currently
offered. It also has a button Unoffer Courses that will removed the selected course(s) and
place them in the first panel. The last panel shows a list of the universities that have
subscribed to IIT Bombays edX courses.
16

SYNCHRONISATION
Figure 5.5: Admin Panel for oﬀering Courses
Figure 5.6: Admin Panel for oﬀering Courses
17

SYNCHRONISATION
5.5 University Interface
The user of this interface is an admin of a university that has subscribed to IIT Bombay’s
edX courses. It is divided into two panels. The first one shows a list of available courses
offered by IIT Bombay that can be committed to. The user can click on the ‘Com-
mit courses’button to subscribe to further courses. The second panel displays all the
courses of IIT Bombay that are currently subscribed to. The user may select courses and
click on ‘Uncommit courses’to unsubscribe from them. All the contents of the selected
course(s) are deleted from the MongoDB, effectively deleting the course from the user’s
(institute’s)edX. The user may also choose to manually update the courses whenever he
wishes to.
Figure 5.7: University Panel for Remote Institute
5.6 Course Management in edX using mongoDB
In case of edX, courseware is managed with the help of mongoDB. To create rich, engaging
online courses, course authors must be able to combine components from a variety of
sources. XBlock is edXs component architecture that makes this possible. Courses
are built hierarchically of pieces called XBlocks. Like an HTML <div>, XBlocks can
represent pieces as small as a paragraph of text, a video, or a multiple-choice input field,
or as large as a section, a chapter, or an entire course. XBlocks are not limited to just
delivering courses. A complete educational ecosystem will make use of a number of web
applications, all of which will need access to course content and data.
XBlocks provide the structure and APIs needed to build components for use in all of
these applications.
18

SYNCHRONISATION
GridFS is a specification for storing and retrieving files that exceed the BSON-
document size limit of 16MB. Instead of storing a file in a single document, GridFS
divides a file into parts, or chunks, and stores each of those chunks as a separate document.
GridFS uses two collections to store files. One collection stores the file chunks, and the
other stores file metadata.
GridFS places the collections in a common bucket by prefixing each with the bucket
name. By default, GridFS uses two collections with names prefixed by fs bucket:
• fs.files: stores the files metadata.
• fs.chunks: stores the binary chunks.
Modulestores: These implement the ModuleStoreRead and ModuleStoreWrite to
provide access to XBlock content.
modulestore. id: The unique ID for this BSON document.
fs.files. id: The unique ID for this BSON document.
fs.chunks.files id: The id of the “parent” document, as specified in the files collection.
A course can have as many documents in mongoDB as needed. A sample document
for each collection used in storing courseware is given below.
A sample xmodule.modulestore document.
{
"_id" : {
"tag" : "i4x",
"org" : "IITB",
"course" : "CS201",
"category" : "course",
"name" : "2014_T1",
"revision" : null
},
"definition" : {
"children" : [
"i4x://IITB/CS201/chapter/a614191c11bd4ccdb4f2677801b7cf27"
],
"data" : {
"wiki_slug" : "IITB.CS201.2014_T1"
}
},
"metadata" : {
"end" : "2014-06-24T18:30:00Z",
"tabs" : [
{
"type" : "courseware",
"name" : "Courseware"
},
{
"type" : "course_info",
19

SYNCHRONISATION
"name" : "Course Info"
},
{
"type" : "textbooks",
"name" : "Textbooks"
},
{
"type" : "discussion",
"name" : "Discussion"
},
{
"type" : "wiki",
"name" : "Wiki"
},
{
"type" : "progress",
"name" : "Progress"
}
],
"discussion_topics" : {
"General" : {
"id" : "i4x-IITB-CS201-course-2014_T1"
}
},
"start" : "2014-06-24T00:00:00Z",
"display_name" : "Data Structures and Algorithms",
"course_image" : "logo11w.png"
}
}
A sample xcontent.fs.ﬁles document.
{
"_id" : {
"tag" : "c4x",
"org" : "Narnia",
"course" : "PA101",
"category" : "thumbnail",
"name" : "123.jpg",
"revision" : null
},
"contentType" : "image/jpeg",
"displayname" : "123.jpg",
"chunkSize" : 262144,
"thumbnail_location" : null,
"filename" : "/c4x/Narnia/PA101/thumbnail/123.jpg",
"length" : 1808,
"import_path" : null,
"uploadDate" : ISODate("2014-06-10T06:56:22.904Z"),
20

SYNCHRONISATION
"locked" : false,
"md5" : "891a9675e5d2222b5fa281fd5c63a666"
}
A sample xcontent.fs.chunks document.
{
"_id" : ObjectId("5396ac168ae8421d025e73ee"),
"n" : 0,
"data" : { "$binary" : "/9j/4AA.....kB//9k=", "$type" : "00" },
"files_id" : {
"tag" : "c4x",
"org" : "Narnia",
"course" : "PA101",
"category" : "thumbnail",
"name" : "123.jpg",
"revision" : null
}
}
5.7 Course Transfer
As soon as a University commits to a course listed on University Panel, it gets transferred
to their mongoDB with the help of mongodump and mongorestore shell commands.
mongo shell commands such as ‘mongodump’and ‘mongorestore’can be used to connect
to remote mongo instances and make our queries. mongodump is a utility for creating
a binary export of the contents of a database. mongodump is used in conjunction with
mongorestore to restore databases. The mongorestore program writes data from a binary
database dump created by mongodump to a MongoDB instance. mongorestore can create
a new database or add data to an existing database.
xmodule.modulestore: ‘{“ id.course”:“CS101”}’
xcontent.fs.files: ‘{“ id.course”:“CS101”}’
xcontent.fs.chunks: ‘{“files id.course”:“CS101”}’
If you restore to an existing database, mongorestore will only insert into the existing
database, and does not perform updates of any kind. If existing documents have the same
value id field in the target database and collection, mongorestore will not overwrite those
documents.
mongodump --collection <source collection> --db <source database> --query q
--out dump_path
mongorestore --collection <target collection> --db <target database> dump_path +
/<target database>/<target collection>.bson --host ip --port 27017
21

SYNCHRONISATION
• db <source database> - to specify the name of the database that contains the
collection to be dumped/restored.
• collection <source collection> - to specify the collection to be dumped/re-
stored.
• query <q > - allows only those documents to be exported which satisfy the given
query
• db <target database > - to specify the name of the database where the content
is to be imported
• collection <target collection> - to specify the name of the collection where the
content is to be imported
• out <filename >- specify a file to write the export to. If nothing is specified, the
mongoexport writes data to standard output (e.g. stdout).
• host <ip> - Specifies a resolvable hostname for the mongod to which to connect.
• port <27017> - Specifies the TCP port on which the MongoDB instance listens
for client connections.
• username <username> - Specifies a username to authenticate to the MongoDB
instance.
• password <password> - Specifies a password to authenticate to the MongoDB
instance
mongodump --collection modulestore --db xmodule --query ’{"_id.course":
"course_selected"}’ --out dump_path
mongorestore --collection modulestore --db xmodule dump_path +
/xmodule/modulestore.bson --host ip --port 27017
Please refer to our code in Appendix B.
5.8 Remove Course from mongoDB
A University may also want to remove the committed courses from their courseware
mongoDB. ‘mongo’shell command with ‘host’and ‘–eval’arguments can be used to make
a query to a remote instance of mongoDB. Here variable ‘ id.course’refers to course ID of
the course which University wants to remove from its database and ‘ id.org’refers to the
organisation which has offered this course. Two arguments have been used to remove the
course to prevent removal of courses with similiar course IDs, eg. ‘CS101’of ‘IITBX’and
‘CS101’of ‘IITMX’.
Thus each and every content which corresponds to the selected course ID and the
Institute offering the course is removed from ‘fs.chunks’, ‘fs.files’collections of ‘xcon-
tent’database and ‘modulestore’collection of ‘xmodule’database.
22

SYNCHRONISATION
mongo ip:27017/xmodule --eval ’db.modulestore.remove({$and:[{"_id.course":
"c_id"}, {"_id.org":"org"}]})’
5.9 Logs
Every activity related to edX Distributed Platform for Course Synchronisation such as
‘Transfer of Course’, ‘Removal of Course from mongoDB’, ‘Updating Course’, ‘Offering
Course’, ‘Removing Courses from Offered Courses List’, etc is logged in a file ‘rcms.log’,
which can be specified in ‘rcms.cfg’config file. Logging has been implemented using Shell
Output of all the mongo commands, and ‘ConfigParser’module of Python.
Figure 5.8: edX Distributed Platform for Course Synchronisation Logs
5.10 Update Courses
An update feature has been provided for each of the courses subscribed to by an institute.
The course contents on the main server of IIT Bombay are compared with those of the
institute. Changes are incorporated in the institute’s edX. This helps in maintaining
consistency of the course data between IIT Bombay and the institute. Any given course
in edX stores the content in MongoDB in three collections: modulestore, fs.files and
fs.chunks. All the three need to be checked for updates. The update process in MongoDB
takes place with the help of the mongoimport and mongoexport commands.
sudo mongoexport --db <source database> --collection <source collection>
-q query --out <filename>
23

SYNCHRONISATION
sudo mongoimport --db <target database> --collection <target collection>
--upsert <filename> --host ip --port 27017
• sudo - to grant user root privileges
• mongoexport - produces a JSON or CSV export of data stored in a MongoDB
instance
• db <source database> - to specify the name of the database that contains the
collection to be exported
• collection <source collection> - to specify the collection to be exported
• q <query> - allows only those documents to be exported which satisfy the given
query
• out <filename> - specify a file to write the export to. If nothing is specified, the
mongoexport writes data to standard output (e.g. stdout).
• mongoimport - provides a route to import content from a JSON, CSV, or TSV
export created by mongoexport
• db <target database>- to specify the name of the database where the content is
to be imported
• collection <target collection>- to specify the name of the collection where the
content is to be imported
• upsert <filename>- to modify the import process to update existing objects in
the database if they match an imported object, while inserting all other objects.
If fields are not specified using the –upsertFields, mongoimport will upsert on the
basis of the id field.
• host <ip> - Specifies a resolvable hostname for the mongod to which to connect.
• port <27017> - Specifies the TCP port on which the MongoDB instance listens
for client connections.
An example of commands used to update the modulestore collection in the xmodule
database for the course CPE433 is as follows:
sudo mongoexport --db xmodule --collection modulestore -q
’{"_id.course":"CPE433"}’ --out /home/test/modulestore.bson
sudo mongoimport --db test --collection modulestore --upsert
/home/test/modulestore.bson --host 192.168.150.1 --port 27017
In the RCMS module, we have provided functionality for automatic as well as manual
update.
24

SYNCHRONISATION
5.10.1 Automatic Update
Automatic updates can be scheduled to occur periodically. EdX uses Celery in con-
junction with Django. Celery is a Python library which is a job/task queue based on
distributed message passing. It focuses on real-time operation but supports scheduling as
well. The Django admin can choose the checking for updates to happen in given time in-
tervals or at specific times. The Django admin interface provides two options for periodic
tasks - intervals and cron tabs. In the Djcelery panel, the admin can choose an interval
or a crontab for the update task. The interval can be specified in terms of microseconds,
seconds, minutes, hours or days. This ensures that the update mechanism will be imple-
mented after the given interval of time. An alternative is to provide a crontab for the
update. A specific hour, day of the week, day of the month and month of the year need
to be given.
The automated process allows for regular checking of updates to take place without
any human intervention.
5.10.2 Manual Update
At the user end which is operated by the admin of the institute, a feature is available to
update a particular course or all the courses. The admin may log into the user interface
of the RCMS module and click on the update button next to course subscribed to.
Additionally, the admin can choose to update all the courses committed to using the
‘Update All’button. All the updates take place in real-time. Please refer to our code in
Appendix B.
5.11 Future Scope
Additional features can be added to the RCMS module. Some of them are as follows:
1. Performance report upload: Many a times, the course content creators or teach-
ers in the university offering edX courses (in this case, IIT Bombay) want insights
into performances of the enrolled students of the different institutes. EdX supports
partial functionality in this aspect. The progress report and gradesheets can be
viewed by a student and by a course instructor for all the students of the particular
course. Future work can involve creation of a portal for teachers to upload the
performance reports for a course, which can be used for evaluation by IIT Bombay.
2. Statistics of past offerings: Data collected from the past offerings of a particular
course offered by IIT Bombay can be shown to potential students. The information
may help the students decide on whether they should enrol for a course and help
them prepare a strategy or to successfully complete the course.
25

Chapter 6
Transfer of Data from edX to
Moodle
6.1 User Content Transfer from edX to Moodle
6.1.1 Introduction
Synchronisation of user related tables which contains the user data of edX and Moodle
is done through a Java Database Connection application developed. The JDBC [7] [8]
[9] application (please refer to our code in Appendix A) extracts the data from edX and
inserts it into Moodle.
6.1.2 User tables in edX and Moodle
• auth user table:
The auth user table is built into the edX Django Web framework. It holds generic
information necessary for user login and permissions.
• mdl user table [10]:
The mdl user table follws MySQL database sequence and holds the holds the user
information necessary for user login and permissions in Moodle platform.
6.1.3 Creating Java Database Connectivity Application
• Register the JDBC driver:
Requires that you initialize a driver so you can open a communications channel
with the database.
• Open a connection:
Requires using the DriverManager.getConnection() method to create a Connection
object, which represents a physical connection with the database.
1. For mysql
conn = DriverManager.getConnection(database url,username,password)
2. For sqlite
conn = DriverManager.getConnection(database url,port)
26

CHAPTER 6. TRANSFER OF DATA FROM EDX TO MOODLE
Creating JDBC Application for transferring data from edX auth user sqlite database
table to Moodle mdl user mysql table.
• Execute a query to retrieve data from edX sqlite table:
Requires using an object of type Statement for building and submitting an SQL
statement to the database.
ResultSet object name = statement.executeQuery(write the sql query here)
• Extract data from result set and insert it into moodle mysql table:
Execute the sql query to write the extracted sqlite user related data into the moodles
mdl user table.
• Trigger:
A trigger needs to be added in the render to response() function of the
/home/sogeking/edx all/edx-platform/common/djangoapps/edxmako/shortcuts.py
ﬁle of edX.
27

6.1.4 Password
The moodle accepts password in form of no less than 8 characters which includes atleast 1
special character ,atleast 1 capital and small alphabet and atleast one numeral while there
is no such case as this in edX. So we have created a code which contains many passwords
and their hashes.When a new user is created this program is executed and when this
program is executed a random password is selected and that is sent through mail to the
newly registered user while the corresponding hash is inserted into the mdl user table.This
password can be then changed by the user after ﬁrst login.
6.1.5 Implementation
Figure 6.1: User Registration page
28

Figure 6.2: Registration information of User
29

Figure 6.3: Registration information of User
Figure 6.4: User registration conﬁrmation
30

Figure 6.5: Moodle admin contains newly added user
Figure 6.6: User login in Moodle
31

Figure 6.7: User Home Page in Moodle
6.2 Course Transfer from edX to moodle
6.2.1 Introduction
To synchronize the course related tables that contains the basic informations of course
of the moodle and edX alongwith the mongodb database of the edX a ,JDBC (Java
Database Connection)application is created. The JDBC application (please refer to our
code in Appendix A) will extract data from edX and insert it into moodle. The user
deﬁned functions that are used in the JDBC application is to extract the information
of course id written in string1/string2/... format in edx course accessrole table and to
extract full name of the course from mongodb database.
6.2.2 Creating JDBC Application
Creating JDBC Application for transferring data from edX student accessrole sqlite table
and mongoDB database to Moodle mdl course mysql table.
• Import the packages:
Requires that you include the packages containing the JDBC classes needed for
database programming.
• Register the JDBC driver:
Requires that you initialize a driver so you can open a communications channel
with the database.
32

• Open a connection:
Requires using the DriverManager.getConnection() method to create a Connection
object, which represents a physical connection with the database.
– For mysql
conn = DriverManager.getConnection(database url,username,password)
– For sqlite
conn = DriverManager.getConnection(jdbc:sqlite:path to edX database)
– For mongodb
conn = DriverManager.getConnection(database url,port)
• Execute a query to retrieve data from edX sqlite table:
Requires using an object of type Statement for building and submitting an SQL
statement to the database.
ResultSet object name = statement.executeQuery(write the sql query here)
• Used function for extracting mongoDB data:
To extract the course full name from the mongodb database.
• Used generateinitial() function:
To generate the short name from the extracted mongodb database.
• Extract data from result set and insert it into moodle mysql table:
Execute the sql query to write the extracted sqlite and mongodb database course
related data into the moodles mdl course table.
• Clean up the environment:
Requires explicitly closing all database resources versus relying on the JVMs garbage
collection.
• Trigger:
A trigger was added in the course handler() function of the
edx all/edx-platform/cms/djangoapps/contentstore/views ﬁle of edX.
33

Figure 6.8: Trigger in course.py ﬁle for transferring Course
34

Figure 6.9: Courses in edX
Figure 6.10: Courses in moodle
35

Figure 6.11: Course creation in edX
Figure 6.12: Created course in edX
36

Figure 6.13: Transferred course in moodle
Figure 6.14: Transferred course content in moodle
37

6.3 Section Transfer from edX to moodle
6.3.1 Introduction
• The transfer of sections from edX to moodle involves fetching the recent section
from the MongoDB database of edx ﬁrst.
• Then the section is transferred to moodle by populating moodle’s database using a
JDBC program. (please refer to our code in Appendix A)
• Finally, the rebuild course cache plugin [11] [12] is used to make the section appear
within a moodle course.
6.3.2 Steps in transferring a section from edX to moodle
• First of all, edX follows a heirarchy (A section can have multiple subsections. A
subsection can have multiple units. A unit can have multiple problems. A problem
can have questions, full screen images, videos, etc).
• MongoDB stores the link of all courses as category: course. Links of all the sections
in a course is stored in the metadata of the course and also as category: chapter in
MongoDB.
• Moodle stores its course content in topics in topic format. There can be any number
of topics in a moodle course, however upto 30 topics can be set to appear in course
by default. set
• So, whenever a section is created in edX, simultaneously a topic in moodle gets
created.
• The tables referred/populated to make a topic in moodle are: mdl course sections,
mdl course modules and mdl course.
• A trigger to do all these things has been added in course handler() function of
edx all/edx-platform/cms/djangoapps/contentstore/views/course.py.
38

Figure 6.15: Before creating a section in edX
Figure 6.16: Before creating a topic in moodle
39

Figure 6.17: Creating a section in edX
Figure 6.18: After creating a section in edX
40

Figure 6.19: Transferred edX section in moodle as a topic
6.4 File Transfer from edX to moodle
6.4.1 Introduction
• The transfer of course content from edX to moodle involves fetching the content
(PDF here) from the MongoDB database of edx first. Then by using a script,
written in python, the given content (PDF here) is dumped (stored) into a dump
folder that stores the integrated content which was earlier stored as chunks in the
MongoDB database.
• Then the content (PDF here) is copied to the moodledata directory where moo-
dle stores its course contents. After that moodle database is populated with the
appropiate field information related to the file that is earlier copied from edx to
moodle. It is done by using a JDBC program. (please refer to our code in Ap-
pendix A)
• Finally, turn on the editing option of the course and run the rebuild course cache
plugin [11] [12] to make the content (PDF here) appear within a moodle course.
6.4.2 Fetching the chunks of data from MongoDB database
MongoDB stores the edX files in fs.chunks collection of xcontent database as chunks. The
information about the file is stored in the fs.files collection of the xcontent database. To
retrieve the latest file from MongoDB we used the following command:
41

Figure 6.20: Retrieve File from MongoDB
6.4.3 Copy the fetched data into moodledata directory
In moodle filenames are given according to the SHA1 hash of their content. This means
each file with particular contents is stored once, irrespective of how many times it is
included in different places, even if it is referred to by different names. To relate a file on
disc to a user-comprehensible path or filename, you need to use the files database table.
Suppose a file has SHA1 contenthash as: 081371cb102fa559e81993fddc230c79205232ce.
Then it will be stored as:
moodledata/filedir/08/13/081371cb102fa559e81993fddc230c79205232ce.
The directory under the filedir directory gets the name from the first and second character
of the file’s contenthash. The subdirectory under the filedir directory gets the name from
the third and fourth character of the file’s contenthash. This means that moodle cannot
store two files with the same SHA1 hash. We did this using the following command:
Figure 6.21: Get the Contenthash and Copy the fetched data into moodledata directory
6.4.4 Change permissions of the new directory created in the
moodledata/filedir directory
We have to change permissions of all the sub-directories present in the moodledata/filedir
directory in order to allow moodle to access the files. The permissions were set using the
following commands:
Figure 6.22: Set permissions for the files in moodledata/filedir
42

6.4.5 Populate the moodle tables in the moodle database
We then populated the following tables:
mdl files, mdl context, mdl resource, mdl course sections, mdl course modules
6.4.6 Run the rebuild course cache plugin
After transferring the file to the moodledata/filedir directory and populating the tables
we need to rebuild the course cache so that the transferred content would appear on the
required course. We downloaded and installed the course rebuild cache plugin in order
to rebuild the course caches. The plugin has to be copied to the moodle/admin/tools
directory and then can be installed by accessing the following address on the browser:
http://127.0.0.1/moodle/admin/tool/rebuildcoursecache. After the plugin has been in-
stalled it can then be run to rebuild course caches using command line. The command
’php index.php courseid’ builds the cache for only the course having a specific courseid.
Figure 6.23: Uploading a file in edX 1
43

44

Figure 6.26: File in edX transferred to moodle
6.5 Question Transfer from edX to moodle
6.5.1 Introduction
• edX has the following question types: Blank Common Problem, Checkboxes, Drop-
down, Mutiple Choice, Numerical Input, Text Input, Blank Advanced Problem, Cir-
cuit Schematic Builder, Custom Javascript Display and Grading, Custom Python-
Evaluated Input, Drag and Drop, Image Mapped Input, Math Expression Input
and Problem with Adaptive Hint.
• Moodle has the following question types: Calculated, Calculated Multichoice, Cal-
culated Simple, Embedded Answers, Essay, Matching, Multiple Choice, Numerical,
Random Short Answer Matching, Short Answer, True/False.
• The transfer of questions (we have considered only multiple choice questions with
single correct option) from edX to moodle involves fetching the recent question from
the MongoDB database of edx ﬁrst.
• Then the question is transferred to moodle by populating moodle’s database using
a JDBC program. (please refer to our code in Appendix A)
• Finally, the rebuild course cache plugin [11] [12] is used to make the question appear
within a moodle course.
45

6.5.2 Steps in transferring a question from edX to moodle
• First of all, edX follows a heirarchy (A section can have multiple subsections. A
subsection can have multiple units. A unit can have multiple problems. A problem
can have multiple questions).
• MongoDB stores the link of all courses as category: course. Links of all the sections
in a course is stored in the metadata of the course and also as category: chapter
in MongoDB. Links of all the subsections in a section is stored in the metadata of
the section and also as category: sequential in MongoDB. Links of all the units in a
subsection is stored in the metadata of the subsection and also as category: vertical
in MongoDB. Links of all the problems in a unit is stored in the metadata of the
unit and also as category: problem in MongoDB.
• For moodle each question is added in a quiz and for edX each problem is added in a
unit (which lies inside a subsection). So, whenever a unit is created in a subsection
of edX then simultaneously a quiz is created in moodle bearing the subsection name
of the edX course. This is done by a JDBC program.
• Now we have the required quiz in moodle. So, all the questions in the problems un-
der a subsection in edX are transferred to the moodle’s quiz (bearing the subsection
name of edX) one by one. This is done by a JDBC program.
• So, all the problems of edX under a subsection is transferred to a quiz in moodle
under a topic. The moodle topic has the edX section name and the moodle quiz
has the edX subsection name.
• The tables referred/populated to make a quiz in moodle are: mdl quiz,
mdl course sections, mdl course modules, mdl course.
• The tables referred/populated to make a question in moodle are: mdl course,
mdl question, mdl quiz question instance, mdl quiz, mdl question answers,
mdl question multichoice, mdl question categories.
• A trigger to do all these things has been added in compute publish state() function
of edx all/edx-platform/cms/djangoapps/contentstore/views/utils.py.
46

Figure 6.27: Creating a Question in edX 1
Figure 6.28: Creating a Question in edX 2
47

Figure 6.29: Checking the created Question in edX
Figure 6.30: Transferred Quiz and Question in moodle 1
48

Figure 6.31: Transferred Quiz and Question in moodle 2
Figure 6.32: Attempting the transferred question in moodle
49

Figure 6.33: Submitting the transferred question in moodle
Figure 6.34: Reviewing the transferred question in moodle
50

6.6 Video Transfer from edX to moodle
6.6.1 Introduction
Transfer of Embedded Video from edX to Moodle involves fetching of URL information
from edX and populate corresponding tables in moodle by that information. To create a
video content in an edX course, youtube link of that video content is required. So that
link along with some course informations are m from edx to populate the moodle tables.
6.6.2 Steps in transferring a video from edX to moodle
• First of all URL information, Course Id, Section name and Sub-section name is
fetched from MongoDB using a JDBC program. (please refer to our code in Ap-
pendix A)
• Then corresponding moodle tables are populated by these informations using same
JDBC program.
• The tables referred or populated to make a question in moodle are: mdl context,
mdl course sections, mdl course modules and mdl url table.
• A trigger is used to directly call the JDBC program using the os.system() command
of python.
• Finally, the Rebuild Course Cache Plugin is used to build the course caches and to
make the video content appear on moodle platform.
51

Figure 6.35: Creating a video in edX
Figure 6.36: A video content created in edX
52

Figure 6.37: Video transferred in moodle
Figure 6.38: An embedded video in moodle
53

6.7 Transfer of Image File from edX to moodle
6.7.1 Introduction
• The transfer of image file from edX to moodle involves fetching the content image
file from the MongoDB database of edx first. Then by using a script, written
in python, the given file is dumped (stored) into a dump folder that stores the
integrated content which was earlier stored as chunks in the MongoDB database.
• Then the image file is copied to the moodledata directory where moodle stores its
course contents. After that moodle database is populated with the appropiate field
information related to the file that is earlier copied from edx to moodle. It is done
by using a JDBC program. (please refer to our code in Appendix A)
• The rebuild course cache plugin is run with the help of the script written in python
to make the image appear within a moodle course.
6.7.2 Fetching the chunks of data from MongoDB database
MongoDB stores the edX files present in a unit in modulestore collection of xmodule
database as a single document. To retrieve the latest file in a unit from MongoDB we
used the following command:
Figure 6.39: Python script to dump image file
6.7.3 Copy the fetched data into moodledata directory
In moodle filenames are given according to the SHA1 hash of their content. This means
each file with particular contents is stored once, irrespective of how many times it is in-
cluded in different places, even if it is referred to by different names. To relate a file on disc
to a user-comprehensible path or filename, you need to use the files database table. Sup-
pose a file has SHA1 contenthash as: 081371cb102fa559e81993fddc230c79205232ce. Then
it will be stored as: moodledata/filedir/08/13/081371cb102fa559e81993fddc230c79205232ce.
The directory under the filedir directory gets the name from the first and second character
of the files contenthash. The subdirectory under the filedir directory gets the name from
the third and fourth character of the files contenthash. This means that moodle cannot
store two files with the same SHA1 hash. We did this using the following command:
54

Figure 6.40: Get the Contenthash and Copy the fetched data into moodledata directory
6.7.4 Change permissions of the new directory created in the
moodledata/filedir directory
We have to change permissions of all the sub-directories present in the moodledata/filedir
directory in order to allow moodle to access the files. The permissions were set using the
following commands
Figure 6.41: Python code to change permission
6.7.5 Populate the moodle tables in the moodle database
We then populated the following tables:
mdl files, mdl context, mdl resource, mdl course, mdl sections, mdl course modules
6.7.6 Naming convention for the transferred image file
As the edX MongoDB database doesnot store the file name of the image so in order to
name the image in the moodle section we use the naming convention as:
“name of the subsection image N”(where N denotes the number of the Nth image added)
6.7.7 Run the rebuild course cache plugin
After transferring the file to the moodledata/filedir directory and populating the tables
we need to rebuild the course cache so that the transferred content would appear on the
required course. We downloaded and installed the course rebuild cache plugin in order
to rebuild the course caches. The plugin has to be copied to the moodle/admin/tools
directory and then can be installed by accessing the following address on the browser:
http://127.0.0.1/moodle/admin/tool/rebuildcoursecache. After the plugin has been in-
stalled it can then be run to rebuild course caches using the following script written in
python.
55

Figure 6.42: Uploading an image in edX
Figure 6.43: Image uploaded in edX
56

Figure 6.44: Transferred image in moodle
Figure 6.45: Image appeared as embedded image in moodle
57

6.8 Results
The latency of all the triggers were measured in seconds (across four consequent transfers,
i.e., Transfer 1, 2, 3 and 4) using time() method of python’s time module. All the transfers
were conducted on a 2 GHz Intel Core i7-2630QM processor having 4 GB 1333 MHz DDR3
RAM where 90 courses were live on edX and moodle. The latencies are shown below:
Transfer 1 Transfer 2 Transfer 3 Transfer 4
User Transfer 1.272 s 1.220 s 1.258 s 1.281 s
Course Transfer 1.916 s 1.865 s 1.611 s 1.981 s
Section Transfer 2.665 s 3.113 s 2.016 s 2.443 s
File (PDF) Transfer 3.687 s 3.642 s 3.615 s 3.476 s
Question Transfer 2.277 s 2.275 s 2.204 s 2.244 s
Video Transfer 2.044 s 2.057 s 2.165 s 2.056 s
Table 6.1: Latency for Content Transfer
Latency of PDF Transfer was above 3 seconds in each case. This was because the
PDF was integrated from the chunks of MongoDB. Size of the PDF uploaded was also
varied in each transfer. Details about the size is shown below:
Transfer 1 Transfer 2 Transfer 3 Transfer 4
PDF Size 65 KB 483 KB 614 KB 1.6 MB
Table 6.2: PDF Sizes for each transfer
Load Averages for Transfer 2, 3 and 4 were more or less similar to that of Transfer 1
(as Transfer 1, 2, 3 and 4 were consequent). Also, the load averages for last 5 minutes in
case of PDF is fairly high (1.42). This was infact a contributing factor for high latencies
of PDF Transfer. Load Averages immediately before Transfer 1 are shown below:
Load Averages
for the last
1 minute
Load Averages
for the last
5 minutes
Load Averages
for the last
15 minutes
User Transfer 0.77 0.71 0.63
Course Transfer 0.89 0.99 0.83
Section Transfer 1.47 1.27 0.97
File (PDF) Transfer 1.28 1.42 1.19
Question Transfer 1.18 1.30 1.39
Video Transfer 0.72 0.91 1.19
Table 6.3: Load Averages immediately before Transfer 1
58

6.9 Future Work
The data that was transferred was only the basic user information and content required
for a course. The transfer of file was only considered for .pdf format but files of other
formats (like docx, odt, txt, png, jpg and many more) could also be transferred which
are provided as the course content.
The transfer of quiz was only for multiple choice questions but as edX also has other
type of problems like true/false, check box type, descriptive type, etc so transfer of these
can also be considered in future.
The current work was only for data transferring from edX to moodle but lacks in data
synchronization, i.e., if a course content is modified after creation on the edX then the
modification should be reflected on moodle also. Also, the same can be done vice-versa
too, i.e., transferring the and synchronizing data from edX to moodle.
6.10 Discussion and Conclusion
The transfer of data from edX to moodle was beneficial in the condition when these two
platforms platforms, edX and moodle, were hosted by the same institute/university so
overhead of establishing a course for the different domain of users of both the platforms
will be reduced significantly.
The mdl user table of moodle contains country field which was mandatory to be
populated in order to create a user while auth user table of edX does not contain any
such field. So, the country field was populated by a default country abbreviation (in this
case IN for India) which can be later altered in the profile edit options after user login.
The password in moodle must have at least 8 characters, at least 1 digit(s), at least
1 lower case letter(s), at least 1 upper case letter(s), at least 1 non-alphanumeric char-
acter(s) while there were no such constraints while setting a password in edX. So, to
preserve the integrity of moodle a mail was sent containing the random system generated
password following the integrity constraints which can be altered later in account editing
options.
Also, the option of adding description to files and videos was not available in edX
while it was compulsory in moodle. So, the files and videos from edX are transferred to
moodle with a ”No description available” string with them.
59

Appendix A
edX to moodle transfer Code
Github Repository: https://github.com/fresearchgroup/Modifications-to-edX-Platform
A.1 edX to moodle Course transfer
1 import com.mongodb.MongoClient;
2 import com.mongodb.DB;
3 import com.mongodb.DBCollection;
4 import com.mongodb.BasicDBObject;
5 import com.mongodb.DBObject;
6 import com.mongodb.DBCursor;
7 import java.net.UnknownHostException;
8 import java.sql .∗;
9
10 public class edx to moodle course synchronisation {
11 static final String JDBC DRIVER = ”com.mysql.jdbc.Driver”;
12 static final String DB URL = ”jdbc:mysql://localhost/moodle”;
13 static final String USER = ”root”;
14 static final String PASS = ”root”;
15 public static void main(String[] args) throws ClassNotFoundException, UnknownHostException
{
16 // load the sqlite −JDBC driver using the current class loader
17 Class.forName(”org.sqlite.JDBC”);
18 Class.forName(”com.mysql.jdbc.Driver”);
19 String resd = ””, sql = ””;
20 Connection conn = null; //mysql connection
21 Connection connection = null; //sqlite connection
22 try {
23 conn = DriverManager.getConnection(DB URL, USER, PASS);
24 // create a database connection
25 connection = DriverManager.getConnection(”jdbc:sqlite:/home/rajarshi/edx all/db/edx.
db”);
26 Statement stmt = conn.createStatement(); //mysql statement
27 Statement statement = connection.createStatement();
28 statement.setQueryTimeout(30); // set timeout to 30 sec.
29 //read from edx
30 ResultSet rs = statement.executeQuery(”select course id from student courseaccessrole
where id = (select max(id) from student courseaccessrole)”);
31 String course id = rs.getString(”course id”);
32 String [] temp;
33 String delimiter = ”/”;
60

APPENDIX A. EDX TO MOODLE TRANSFER CODE
34 temp = course id.split(delimiter);
35 //System.out.println(temp[0]+ ” ” + temp[1] + ” ” +temp[2]);
36 //To connect to mongodb server
37 MongoClient mongoClient = new MongoClient(”localhost”, 27017);
38 //Now connect to your databases
39 DB db = mongoClient.getDB(”xmodule”);
40 //System.out.println(”Connection to Database Successfull”);
41 DBCollection coll = db.getCollection(”modulestore”);
42 //System.out.println(”Collection Selected Successfull ”);
43 BasicDBObject query = new BasicDBObject(” id.category”, ”course”).append(” id.course
”, temp[1]);
44 //BasicDBObject query = new BasicDBObject(” id.category”, ”course”).append(” id.
course”, ”CS201”);
45 DBCursor cursor = coll.find(query);
46 String result = ””;
47 while (cursor.hasNext()) {
48 DBObject tobj = cursor.next();
49 result = tobj.get(”metadata”).toString();
50 }
51 String [] temp1 = result.split(”:”);
52 String resa = temp1[14];
53 String [] resb = resa. split (”,”);
54 //System.out.print(resb[0]) ;
55 String resc = resb[0];
56 resd = resc.substring(2, resc.length() − 2);
57 //System.out.print(”Course Name: ” +resd);
58 cursor. close ();
59 String coursename = resd;
60 String shortname = generateInitials(coursename);
61 String idtomatch = ””;
62 //read from moodle
63 rs = stmt.executeQuery(”select idnumber from mdl course where id = (select max(id)
from mdl course)”);
64 while (rs.next()) {
65 idtomatch = (String) rs.getString(”idnumber”).toString();
66 }
67 if (temp[1].compareTo(idtomatch) != 0) {
68 //write into moodle(mdl course)
69 sql = ”insert into mdl course (category,fullname,shortname,sortorder,idnumber,
summary,summaryformat,showgrades,newsitems,startdate,marker,maxbytes,
legacyfiles,showreports,visible,visibleold,groupmode,groupmodeforce,
defaultgroupingid,timecreated,timemodified,requested,enablecompletion,
completionnotify)” + ”values (1,’” + coursename + ”’,’” + shortname + ”
’,10000,’” + temp[1] + ”’,’Course Summary has not been entered
’,1,1,5,1609353000,0,0,0,0,1,1,0,0,0,0,0,0,0,0)”;
70 // startdate (31/12/2020)= 1609353000
71 stmt.executeUpdate(sql);
72 stmt.executeUpdate(”update mdl course set sortorder=sortorder+1”);
73 stmt.executeUpdate(”update mdl course set sortorder=1 where id=1;”);
74 }
75 } catch (SQLException e) {
76 // if the error message is ”out of memory”, it probably means no database file is found
77 System.err.println(e.getMessage());
78 } finally {
79 try {
80 if (connection != null) connection.close ();
81 } catch (SQLException e) {
82 // connection close failed .
61

83 System.err.println(e);
84 }
85 }
86 }
87 public static String generateInitials (String original ) {
88 String initial = ””;
89 String [] split = original. split (” ”);
90 for (String value: split ) {
91 initial += value.substring(0, 1);
92 }
93 return initial .toUpperCase();
94 }
95 }
A.2 edX to moodle Image transfer
1 import com.mongodb.MongoClient;
2 import com.mongodb.DB;
3 import com.mongodb.DBCollection;
4 import com.mongodb.BasicDBObject;
5 import com.mongodb.DBObject;
6 import com.mongodb.DBCursor;
7 import java.io.BufferedReader;
8 import java.io. File ;
9 import java.io.IOException;
10 import java.io.InputStreamReader;
11 import java.math.BigInteger;
12 import java.security .MessageDigest;
13 import java.security .NoSuchAlgorithmException;
14 import java.security .SecureRandom;
15 import java.sql .∗;
16 import javax.activation .MimetypesFileTypeMap;
17
18 public class edx to moodle image transfer {
19 static final String JDBC DRIVER = ”com.mysql.jdbc.Driver”;
20 static final String DB URL = ”jdbc:mysql://localhost/moodle”;
21 static final String USER = ”root”;
22 static final String PASS = ”root”;
23 public static void main(String[] args) throws ClassNotFoundException, IOException,
SQLException, NoSuchAlgorithmException {
24 Class.forName(”com.mysql.jdbc.Driver”);
25 Connection conn = null;
26 Connection conn1 = null;
27 Connection conn2 = null;
28 conn = DriverManager.getConnection(DB URL, USER, PASS);
29 conn1 = DriverManager.getConnection(DB URL, USER, PASS);
30 conn2 = DriverManager.getConnection(DB URL, USER, PASS);
31 Statement stmt = conn.createStatement(); //mysql statement
32 Statement stmt1 = conn1.createStatement(); //mysql statement
33 Statement stmt2 = conn2.createStatement(); //mysql statement
34 //To connect to mongodb server
35 MongoClient mongoClient = new MongoClient(”localhost”, 27017);
36 //Now connect to your databases
37 DB db = mongoClient.getDB(”xmodule”);
38 DBCollection coll = db.getCollection(”modulestore”);
62

39 String image description to be sent = ””;
40 BasicDBObject query = new BasicDBObject(” id.category”, ”html”).append(”metadata.
display name”, ”Full Screen Image”);
41 DBCursor cursor = coll.find(query);
42 String result = ””, resultcopy = ””;
43 int subsection it was added = 0;
44 int section it was added = 0;
47 result = tobj.get(” id”).toString();
48 resultcopy = result;
49 image description to be sent = tobj.get(” definition ”).toString();
50 }
51 String [] temp1 = result.split(”:”);
52 String resa = temp1[3];
53 String [] resb = resa. split (”,”);
54 String resc = resb[0];
55 String courseid = resc.substring(2, resc.length() − 2);
56 System.out.println(courseid);
57 String [] temp2 = resultcopy.split(”:”);
58 String rese = temp2[5];
59 String [] resf = rese. split (”,”);
60 String resg = resf [0];
61 String problemid edx = resg.substring(2, resg.length() − 2);
62 System.out.println(problemid edx);
63 //System.out.println();
64 String subsectioname = ””;
65 String subsection name to be sent = ””;
66 String line ;
67 Process p = Runtime.getRuntime().exec(new String[] {
68 ”/bin/bash”, ”−c”, ”php /home/rajarshi/edx to moodle synchronisation/image transfer/
contenthash.php”
69 });
70 BufferedReader input = new BufferedReader(new InputStreamReader(p.getInputStream()));
71 String contenthash = ””;
72 while (( line = input.readLine()) != null) {
73 contenthash = line;
74 }
75 input.close ();
76 int flag = 1;
77 String sql = ”select ∗ from mdl files where contenthash=” + ”’” + contenthash + ”’”;
78 ResultSet rs = stmt.executeQuery(sql);
79 int contxtid = 0;
81 contxtid = rs.getInt(”contextid”);
82 }
83 if (contxtid != 0) {
84 sql = ”select ∗ from mdl context where id=” + contxtid;
85 rs = stmt.executeQuery(sql);
86 String path = ””;
88 path = rs.getString(”path”);
89 }
90 System.out.println(path);
91 String [] temp = path.split(”/”);
92 int idtomatch = Integer.parseInt(temp[3]);
93 sql = ”select ∗ from mdl context where id=” + idtomatch;
63

96 idtomatch = rs.getInt(”instanceid”);
97 }
98 sql = ”select ∗ from mdl course where idnumber=” + ”’” + courseid + ”’”;
100 int id = 0;
102 id = rs.getInt(”id”);
103 }
104 if (idtomatch == id) {
105 flag = 0;
106 }
107 }
108 if ( flag == 1) {
109 String subdirectory1 = contenthash.substring(0, 2);
110 String subdirectory2 = contenthash.substring(2, 4);
111 System.out.println(contenthash);
112 System.out.println(subdirectory1);
113 System.out.println(subdirectory2);
114 /////////////////////////////////////////////////////////
115 int image no in the course = 0;
116 BasicDBObject query2 = new BasicDBObject(” id.category”, ”html”).append(”metadata.
display name”, ”Full Screen Image”).append(” id.course”, courseid);
117 //System.out.println(query2);
118 cursor = coll. find(query2);
121 image no in the course = image no in the course + 1;
122 //System.out.println(image no in the course);
123 }
124 /////////////////////////////////////////////////////////
125 // courseid and problemid fetched
126 BasicDBObject query1 = new BasicDBObject(” id.category”, ”course”);
130 result = tobj.get(” definition ”).toString();
131 resultcopy = result;
132 String [] temp3 = result.split(”]”);
133 String resd = temp3[0];
134 String resi = resd.substring(17, resd.length());
135 if ( resi .compareTo(””) != 0) {
136 String [] temp4 = result.split(”/”);
137 if (temp4[3].compareTo(courseid) == 0) {
138 String [] temp5 = resi.replace(” ”, ””).replace(”””, ””). split (”,”); // ids
of all section
139 String [] chapterids = new String[temp5.length];
140 for (int i = 0; i < temp5.length; i++) {
141 chapterids[ i ] = temp5[i].substring(temp5[i].length() − 32, temp5[i].
length());
142 //System.out.println(chapterids[ i]+”: Section”);
143 BasicDBObject query3 = new BasicDBObject(” id.category”, ”chapter”);
146 tobj = cursor.next();
148 resultcopy = tobj.get(” definition ”).toString();
149 //System.out.println(” ”+result);
64

150 String [] temp1ch = result.split(”:”);
151 String resach = temp1ch[5];
152 String [] resbch = resach. split (”,”);
153 String rescch = resbch[0];
154 String courseidch = rescch.substring(2, rescch.length() − 2);
155 if (chapterids[ i ]. compareTo(courseidch) == 0) // enter the section
156 {
157 //System.out.println(resultcopy);
158 String [] tempaach = resultcopy.split(”]”);
159 //System.out.println(tempaach[0]);
160 String resiich = tempaach[0].substring(17, tempaach[0].length())
;
161 String resiiich = resiich .replace(” ”, ””).replace(”””, ””);
162 //System.out.println( resiiich );
163 String [] temp7 = resiiich. split (”,”);
164 String [] sequenceids = new String[temp7.length];
165 for (int l = 0; l < temp7.length; l++) {
166 sequenceids[l ] = temp7[l].substring(temp7[l].length() − 32,
temp7[l].length());
167 //System.out.println(” ”+sequenceids[l]+”: SubSection”); //
ids of all subsections
168 BasicDBObject query4 = new BasicDBObject(” id.category”,
”sequential”);
173 resultcopy = tobj.get(” deﬁnition ”).toString();
175 subsectioname = tobj.get(”metadata”).toString();
176 String [] temp1sq = result.split(”:”);
177 String resasq = temp1sq[5];
178 String [] resbsq = resasq. split (”,”);
179 String rescsq = resbsq[0];
180 String courseidsq = rescsq.substring(2, rescsq.length()
− 2);
181 //System.out.println(courseidsq);
182 if (sequenceids[l ]. compareTo(courseidsq) == 0) {
184 String [] tempaasq = resultcopy.split(”]”);
186 String resiisq = tempaasq[0].substring(17, tempaasq
[0].length());
187 String resiiisq = resiisq .replace(” ”, ””).replace(”
””, ””);
188 //System.out.println( resiiisq );
189 String [] temp8 = resiiisq. split (”,”);
190 String [] verticalids = new String[temp8.length];
191 for (int m = 0; m < temp8.length; m++) {
192 verticalids [m] = temp8[m].substring(temp8[m].
length() − 32, temp8[m].length());
193 //System.out.println(” ”+verticalids[m]+”:
Unit”); // ids of all units
194 BasicDBObject query5 = new BasicDBObject(”
id.category”, ”vertical”);
65

199 resultcopy = tobj.get(” deﬁnition ”).toString
();
201 String [] temp1vr = result.split(”:”);
202 String resavr = temp1vr[5];
203 String [] resbvr = resavr. split (”,”);
204 String rescvr = resbvr[0];
205 String courseidvr = rescvr.substring(2,
rescvr.length() − 2);
206 //System.out.println(courseidvr);
207 if ( verticalids [m].compareTo(courseidvr)
== 0) {
209 String [] tempaavr = resultcopy.split(”]”
);
211 String resiivr = tempaavr[0].substring
(17, tempaavr[0].length());
212 String resiiivr = resiivr .replace(” ”, ”
”).replace(”””, ””);
213 //System.out.println( resiiivr );
214 String [] temp9 = resiiivr. split (”,”);
215 String [] problemids = new String[temp9.
length];
216 for (int n = 0; n < temp9.length; n++)
{
217 problemids[n] = temp9[n].substring(
temp9[n].length() − 32, temp9[n
].length());
218 //System.out.println(” ”+
problemids[n]+”: Problem/
Video/Image”); // ids of all
problems
219 if (problemids[n].compareTo(
problemid edx) == 0) {
220 //System.out.println(”
nProblem is in Section ”+(i
+1));
221 //System.out.println(”
nProblem is in Section ”+(i
+1));
222 section it was added = i + 1;
223 subsection it was added = l + 1;
224 subsection name to be sent =
subsectioname;
225 //System.out.println(”image:n
”+
image description to be sent
);
226 break;
227 }
228 }
229 }
230 }
231 }
232 }
233 }
66

234 }
235 }
236 }
237 }
238 }
239 }
240 }
241 subsection name to be sent = subsection name to be sent.substring(20,
subsection name to be sent.length() − 2);
242 System.out.print(”Image is in Section: ” + section it was added + ”, Image is in
SubSection: ” + subsection it was added);
243 System.out.println(”, Subsection Name: ” + subsection name to be sent);
244 image description to be sent = image description to be sent.substring(
image description to be sent .indexOf(”<h2>”), image description to be sent.indexOf
(”</p>n<p><img”));
245 image description to be sent = image description to be sent.replace(”<h2>”, ””);
246 image description to be sent = image description to be sent.replace(”</h2>”, ””);
247 image description to be sent = image description to be sent.replace(”Full Screen Image”,
””);
248 image description to be sent = image description to be sent.replace(”<p>”, ””);
249 image description to be sent = image description to be sent.replace(”n”, ””);
250 System.out.println(image description to be sent);
251 System.out.println(”Image no. in the course: ” + image no in the course);
252 System.out.println(”Image name: ” + subsection name to be sent.concat(” Image ”).
concat(Integer.toString(image no in the course)));
253 p = Runtime.getRuntime().exec(new String[] {
254 ”/bin/bash”, ”−c”, ”file −−mime−type −b /home/rajarshi/
edx to moodle synchronisation/image transfer/image”
255 });
256 input = new BufferedReader(new InputStreamReader(p.getInputStream()));
257 String mimetype = ””;
258 while (( line = input.readLine()) != null) {
259 mimetype = line;
260 }
261 System.out.println(mimetype);
262 String filepathforsize = ”/var/moodledata/filedir/” + subdirectory1 + ”/” +
subdirectory2 + ”/” + contenthash;
263 System.out.println( filepathforsize );
264 long filesize = getFileSize( filepathforsize );
265 System.out.println( filesize );
266 //for mdl file part 1..................................
267 int contextId = 5;
268 String component = ”user”;
269 String filearea = ”draft”;
270 String [] temp = mimetype.split(”/”);
271 String filename = subsection name to be sent.concat(” Image ”).concat(Integer.toString
(image no in the course)) + ”.” + temp[1];
272 System.out.println(filename);
273 String filepath = ”/”;
274 int itemid = (int)(Math.random() ∗ 1000000000);
275 String inputforpathhash = ”/” + contextId + ”/” + component + ”/” + filearea + ”/”
+ itemid + ”/” + filename;
276 String pathnamehash = sha1(inputforpathhash);
277 String sourcename = ”O:8:”stdClass”:1:{s:6:”source”;s:38:”” + filename + ””;}”;
278 //System.out.println(sourcename);
279 sql = ”Insert into mdl files (contenthash,pathnamehash,contextid,component,filearea,
itemid,filepath,filename,userid, filesize ,mimetype,status,source,author,license ,
timecreated,timemodified,sortorder)” + ”values (’” + contenthash + ”’,’” +
67

pathnamehash + ”’,’” + contextId + ”’,’” + component + ”’,’” + filearea + ”’,’” +
itemid + ”’,’” + filepath + ”’,’” + filename + ”’,2,’” + filesize + ”’,’” +
mimetype + ”’,0,’” + sourcename + ”’,’Rajarshi Sarkar’,’allrightsreserved ’,0,0,0) ”;
280 stmt.execute(sql);
281 inputforpathhash = ”/” + contextId + ”/” + component + ”/” + filearea + ”/” +
itemid + ”/” + ”.”;
282 pathnamehash = sha1(inputforpathhash);
283 sql = ”Insert into mdl files (contenthash,pathnamehash,contextid,component,filearea,
itemid,filepath,filename,userid, filesize ,status,timecreated,timemodified,sortorder)”
+ ”values (’da39a3ee5e6b4b0d3255bfef95601890afd80709’,’” + pathnamehash + ”’,’”
+ contextId + ”’,’” + component + ”’,’” + filearea + ”’,’” + itemid + ”’,’” +
filepath + ” ’,’.’,2,0,0,0,0,0) ”;
284 stmt.execute(sql);
285 //end of mdl file part1
.....................................................................
286 //for mdl context .................................
287 String test = ”select ∗ from mdl course”;
288 rs = stmt.executeQuery(test);
289 int courseid1 = 0;
290 String coursenumber = ””;
292 courseid1 = rs.getInt(”id”);
293 coursenumber = rs.getString(”idnumber”);
294 if (coursenumber.compareTo(courseid) == 0) break;
295 }
296 String test1 = ”select ∗ from mdl context where instanceid=” + courseid1 + ” and
contextlevel=50”;
297 int idagainstinstanceid = 0;
298 rs = stmt.executeQuery(test1);
300 idagainstinstanceid = rs.getInt(”id”);
301 }
302 System.out.println(idagainstinstanceid);
303 int instanceidcontent = 0;
304 test1 = ”select ∗ from mdl context where contextlevel=70”;
305 rs = stmt.executeQuery(test1);
307 instanceidcontent = rs.getInt(”instanceid”);
308 }
309 instanceidcontent++;
310 //System.out.println(instanceidcontent);
311 String path for mdl context = ”/1/3/” + Integer.toString(idagainstinstanceid) + ”/”;
312 test1 = ”insert into mdl context (contextlevel, instanceid, path, depth)” + ” values
(70,’” + instanceidcontent + ”’,’” + path for mdl context + ”’,4)”;
313 //System.out.println(test1);
314 stmt.execute(test1);
315 rs = stmt.executeQuery(”select ∗ from mdl context where id =(select max(id) from
mdl context)”);
316 int idforpath = 0;
318 idforpath = rs.getInt(”id”);
319 }
320 path for mdl context = ”’/1/3/” + Integer.toString(idagainstinstanceid) + ”/” + Integer.
toString(idforpath) + ”’”;
321 System.out.println(path for mdl context);
322 rs = stmt.executeQuery(”select ∗ from mdl context”);
323 int lastidfromcontexttable = 0;
68

FRG_P4_Final_Project_Report_Modifications_to_edX_platform

FRG_P4_Final_Project_Report_Modifications_to_edX_platform

Recommended

Recommended

More Related Content

What's hot

What's hot (18)

Viewers also liked

Viewers also liked (17)

Similar to FRG_P4_Final_Project_Report_Modifications_to_edX_platform

Similar to FRG_P4_Final_Project_Report_Modifications_to_edX_platform (20)

FRG_P4_Final_Project_Report_Modifications_to_edX_platform