F21SC Industrial Programming CW2 Data Analytics (35) 20192.docx

F21SC: Industrial Programming CW2: Data Analytics (35%)
2019/20
Assessed Coursework 2 — Data Analysis of a Document
Tracker
1 Overview
The aim of this coursework is to develop a simple, data-
intensive application in Python 3.
This is an pair project, and you will have to identify the
contributions of each group member in the
report. Each group member needs to contribute a substantial
implementation task to the project.
The learning objective of this coursework is for students to
develop proficiency in advanced program-
ming concepts, stemming from both object-oriented and
functional programming paradigms, and to apply
these programming skills to a concrete application of moderate
size. Design choices regarding languages,
tools, and libraries chosen for the implementation need to be
justified in the accompanying report.
This coursework will develop personal abilities in using modern
scripting languages as a “glueware” to
build, configure and maintain a moderately complex application
and deepen the understanding of integrating
components on a Linux system.
The report needs to critically reflect on the software used for

implementing this application, and discuss
advantages and disadvantages of this choice. The report should
also contain a discussion, contrasting soft-
ware development on Windows and Linux systems and
comparing software development in scripting vs.
systems languages (based on the experience from the two pieces
of coursework).
2 Lab Environment
Software environment: You should use Python 3 as installed on
the Linux lab machines (EM 2.50) for
the implementation. This installation also provides the pandas,
tkinter, and matplot libraries.
If you want to develop the software on your own laptop you
need to install the above software. Both
Python and the libraries are available for download at:
https://www.python.org/download.
For each of the chosen technologies, the report should discuss
why it is the most appropriate choice for
this application, and possible alternatives should be mentioned.
3 Data Analysis of a Document Tracker
In this assignment, you are required to develop a simple Python-
based application, that analyses and displays
document tracking data from a major web site.
The issuu.com platform is a web site for publishing documents.
It is widely used by many on-line
publishers and currently hosts about 15 million documents. The
web site tracks usage of the site and makes
the resulting, anonymised data available to a wider audience.
For example, it records who views a certain

document, the browser used for viewing it, the way how the
user arrived at this page etc. In this exercise,
we use one of these data sets to perform data processing and
analysis in Python.
The data format is described on this web site
http://labs.issuu.com/dataset spec.html (see also this local
page for the data spec). Familiarise yourself with the details of
the data representation before you start
implementation. The initial (small) data set used for this
exercise is available online here:
http://www.macs.hw.ac.uk/˜hwloidl/Courses/F21SC/issuu
cw2.json. For testing use this tiny sample data
file at http://www.macs.hw.ac.uk/˜hwloidl/Courses/F21SC/issuu
sample.json.
The application needs to run on an up-to-date Linux platform
(Ubuntu 18.04). The application should
be developed in Python 3, using appropriate libraries for input,
data processing and visualisation. Possible
Deadline: 3:30PM (Edi) / 12:00PM (Dubai) on 5th of December
2019; (pair project)
https://www.python.org/download
issuu.com
http://labs.issuu.com/dataset_spec.html
http://www.macs.hw.ac.uk/~hwloidl/Courses/F21SC/dataset_spe
c.html
http://www.macs.hw.ac.uk/~hwloidl/Courses/F21SC/dataset_spe
c.html
http://www.macs.hw.ac.uk/~hwloidl/Courses/F21SC/issuu_cw2.
json
http://www.macs.hw.ac.uk/~hwloidl/Courses/F21SC/issuu_samp
le.json

2019/20
choices are the json library for parsing, the pandas library for
processing the input data (optional), the
tkinter library for GUI functionality and the matplot library for
visualising the results. You need to
identify the advantages of your choice of libraries.
The application must provide the following functionality:
1. Python: The core logic of the application should be
implemented in Python 3.
2. Views by country/continent: We want to analyse, for a given
document, from which countries and
continents the document has been viewed. The data should be
displayed as a histogram of countries,
i.e. counting the number of occurrences for each country in the
input file.
(a) The application should take a string as input, which uniquely
specifies a document (a document
UUID), and return a histogram of countries of the viewers. The
histogram can be displayed
using matplotlib.
(b) Use the data you have collected in the previous task, group
the countries by continent, and
generate a histogram of the continents of the viewers. The
histogram can be displayed using
matplotlib.
3. Views by browser: In this task we want to identify the most
popular browser. To this end, the

application has to examine the visitor useragent field and count
the number of occurrences
for each value in the input file.
(a) The application should return and display a histogram of all
browser identifiers of the viewers.
(b) In the previous task, you will see that the browser strings
are very verbose, distinguishing
browser by e.g. version and OS used. Process the input of the
above task, so that only the
main browser name is used to distinguish them (e.g. Mozilla),
and again display the result as
a histogram.
4. “Also likes” functionality: Popular document-hosting web
sites, such as Amazon, provide informa-
tion about related documents based on document tracking
information. One such feature is the “also
likes” functionality: for a given document, identify, which other
documents have been read by this
document’s readers. The idea is that, without examining the
detail of either document, the informa-
tion that both documents have been read by the same reader
relates two documents with each other.
Figure 1 gives an example of this functionality. In this task, you
should write a function that generates
such an “other readers of this document also like” list, which is
parametrised over the function to
determine the order in the list of documents. Display the top 10
documents, which are “liked” by
other readers.
To achieve this task you will need to do the following:
(a) Implement a function that takes a document UUID and

returns all visitor UUIDs of readers of
that document.
(b) Implement a function that takes a visitor UUID and returns
all document UUIDs that have been
read by this visitor.
(c) Using the two functions above, implement a function to
implement the “also like” functionality,
which takes as parameters the above document UUID and
(optionally) visitor UUID, and addi-
tionally a sorting function on documents. The function should
return a list of “liked” documents,
sorted by the sorting function parameter. Note: the
implementation of this function must not fix
the way how documents are sorted, and use the sorting function
parameter instead.
2019/20
(d) Use this function to produce an “also like” list of
documents, using a sorting function, based on
the number of readers of the same document. Provide a
document UUID and visitor UUID as
input and produce a list of top 10 document UUIDs as a result.
5. “Also likes” graph: For the above “also like” functionality,
generate a graph that displays the re-
lationship between the input document and all documents that
have been found as “also like” docu-

ments (and only these documents). Highlight the input
document and user by shading in that graph,
and use arrows to capture the “has-read” relationship (i.e. arrow
from reader to document). In the
graph shorten all visitor UUIDs and document UUIDs to the last
4 hex-digits. As an example, the
graph below uses document b4fe and reader 6771 as input
(shaded green) and displays 7 “also like”
documents, together with the readers that relate these
documents with the input document (download
.dot source file and .ps output file):
Readers
Documents
Size: 1m
6771
b4fe
d38c
e16e 2bb587c4 9da6
1b2b
90c2 56e3 a2c8
Hint: Use the .dot format as graph representation. Use the
graphviz package with the dot tool to
translate the .dot into a .ps format (and then optionally in .pdf
format). For a detailed description
see this dot User Manual. You can install the graphviz package
on an Ubuntu machine by typing

(in a terminal window): sudo apt-get install graphviz
As an example of graphviz/dot usage, you can generate the
above graph from the dot source
code and view it like this:
% wget
http://www.macs.hw.ac.uk/˜hwloidl/Courses/F21SC/1m_3.dot
% dot -Tps -o 1m_3.ps 1m_3.dot
% evince 1m_3.ps
6. GUI usage: To read the required data and to display the
statistical data, develop a simple GUI based
on tkinter that reads the user inputs described above, and with
buttons to process the data as
required per task.
7. Command-line usage: The application shall provide a
command-line interface to test its functionality
in an automated way, like this:
% cw2 -u user_uuid -d doc_uuid -t task_id -f file_name
to check the results of implementing task task_id using inputs
user_uuid for the user UUID and
doc_uuid for the document UUID; file_name is the name of the
JSON file with the input data.
The task ids should be: 2a, 2b, 3a, 3b, 4d, 5, 6, matching the
tasks above (task id 6
should run Task 5 and then automatically launch a GUI showing
the resulting also-likes graph).
4 Submission
You must submit the complete project files, containing the
source code, a stand-alone executable, and

the report (in .pdf format) as one .zip file no later than 3:30 PM
on 5th of December 2019. Addition-
http://www.macs.hw.ac.uk/~hwloidl/Courses/F21SC/1m_3.dot
http://www.macs.hw.ac.uk/~hwloidl/Courses/F21SC/1m_3.ps
http://www.graphviz.org/content/dot-language
http://www.graphviz.org/
http://www.graphviz.org/Documentation/dotguide.pdf
2019/20
ReadersDocuments
Input
Also read ...
Top result (most also readers)
Other results
Figure 1: Example of identifying also-likes documents. Starting
from the current reader and document
(green), all readers are identified, who have also read the input
document (blue). From the other documents,
read by these readers, the top 10 documents, counted by number
of readers are identified and displayed.
In this example the red document is top of this list, and the two
pink documents are also on the result list.
The automatically generated graph should display all three
result documents, but doesn’t have to distinguish

between “best” and “others” by shading. The unused, black
users and documents shouldn’t be shown in that
graph.
ally, a screencast or video of running the application, with an
explanation as voice-over audio needs to be
submitted to Vision. This is mandatory, and without the
screencast or video the submission is incomplete
and may be marked as 0 points. The standard penalty of -30% of
the maximum available mark applies to
late submissions. No submissions will be accepted after 5
working days beyond the submission deadline.
The main function driving the application should be called cw2,
as discussed in “Command-line Usage”
above. Submission must be through Vision, submitting all of the
above files in one .zip file. Additionally,
an electronic submission of report, sources and application by
email is recommended. This coursework is
worth 35% of the module’s mark.
You are marked for your application, the structure, code and
comments used, your testing, your report
and the screencast/video of demonstrating the running of your
application. The marking scheme for this
project is attached.
5 Report Format
The report should have between 10–20 pages and use the
following format (if you need space for additional
screenshots, put them into an appendix, not counting against the
page limit, but don’t rely on the screenshots
in your discussion):
1. Introduction: State the purpose of the report, your remit and
any assumptions you have made during

the development process.
2. Requirements’ checklist: Here you should clearly show which
requirements you have delivered and
which you haven’t.
2019/20
3. Design Considerations: Here you should clearly state what
you have done to your application to
make it more usable and accessible.
4. User Guide: Use screen shots of the running application
along with text descriptions to help you
describe how to operate the application.
5. Developer Guide: Describe your application design and main
areas of code in order to help another
developer understand your work and how they might develop it.
You may find it useful to supplement
the text with code fragments.
6. Testing: Show the results for testing all cases and prove that
the outputs are what are expected. If
certain conditions cause erroneous results or the application to
crash then report these honestly.
7. Reflections on programming language and implementation:
Based on your experience in imple-
menting this application, reflect which language features and

technologies have been most helpful,
identify limitations of your application and suggest ways how to
overcome this limitations. Also re-
flect on the usability of the (kind of) language (either system or
scripting language) for this application
domain, and on its wider applicability.
8. What did I learn from CW1? A short discussion on lessons
learnt from the feedback given on CW1
and a discussion how you integrated this feedback into CW2.
Cover both coding and report writing,
possibly more (project management, preparing for interview
style questions etc).
9. Conclusions: Reflect on what you are most proud of in the
application and what you’d have liked to
have done differently.
10. A final section should contain the main references used in
this report and in the implementation. These
references need to be cited in the report to indicate where the
reference has been used in the report or
implementation.
Marking Scheme
Criteria Marks
Meeting system requirements 45
Report Quality 20
Content and communication (Design Considerations, Reflec-
tions, User guide & Developers guide, Testing)
The Application 15
Code quality, secure coding, sufficient comments, sensible vari-
able/object names, good code/class/method design, use of ad-
vanced lang. features.
Initiative, innovation, professional conduct, creativity and ef-

forts above & beyond the call of duty
15
Screencast/Video of running the application with explanation as
voice-over audio.
5
Total marks 100
2019/20
Professional Conduct and Plagiarism
This is a pair project, and you will have to identify the
contributions of each group member in the report.
Each group member needs to contribute a substantial
implementation task to the project. Where external
resources have been use, these need to be clearly identified and
referenced.
Plagiarism:
This project is assessed as an pair project. You will work with
your partner on the implementa-
tion and should split up tasks in a reasonable way to arrive at a
complete implementation of the
specification. You will have to identify the contributions of
each group member in the report. You
can discuss general technical issues related to this work with

other students, however, you must
not share concrete pieces of code or text. Readings, web sources
and any other material that you
use from sources other than lecture material must be
appropriately acknowledged and referenced.
Plagiarism in any part of your report will result in referral to
the disciplinary committee, which may
lead to you losing all marks for this coursework and may have
further implications on your degree.
For details see this link
Late Submission Policy
The standard penalty of -30% of the maximum available mark
applies to late submissions. No submissions
will be accepted after 5 working days beyond the submission
deadline.
https://www.hw.ac.uk/students/studies/examinations/plagiarism.
htmOverviewLab EnvironmentData Analysis of a Document
TrackerSubmissionReport FormatReserved Comments to
Instructors:Reflection on previous instances of the course:
9 cases
· SOLVE THE 8 CASES IN DETAILS. DETAILS ARE
NECESSERY IN THIS ASSIGMNET. ALSO, DON’T FORGET
TO SLOVE QUESIOTNS #9
· DO NOT MISS ANY QUESTIONS. OTHERWISE YOU WILL
GET 0

· Here are disease to think of while solving these cases: Viral
Hepatitis and Measles, Vaccines and Hospital-Acquired
Infections (HAI), Middle East Respiratory Syndrome (MERS),
Meningitis and Rabies, Vector-borne and Emerging Infections,
Hantavirus and Nipah, Foodborne and Waterborne Diseases,
Ebola, Clostridium difficile, and Necrotizing Fasciitis, Acute
Respiratory Infections and MRSA, Influenza and Avian and
Swine Flu, Tuberculosis, Polio, Candida auris, and more…
· NOTE: These diseases may and may not be the disease you’re
working on. You need to search. These just to give you an idea.
Answering the questions are the most important.
CASE #1
1. Today is December 2nd. The local hospital has reported ten
cases of a mysterious infectious disease among children. The
children have suffered from nausea, diarrhea (profuse and
watery), abdominal pain, and fever, primarily. Each of the
children has vomited at least twice in the past 24 hours. All of
the children are between the ages of 7 and 11 and reportedly
have been to Franklin Park Mall in the past three days.
Specifically, the play area and cars near the food court. Using
your infectious disease knowledge, what would you do to try to
identify the agent? What other public health actions should you
take to prevent more cases from occurring? What do you think
is the likely agent and why? What are the potential fomites in a
shopping mall and how would these be assessed? (20 points)
Some clues: All the children at the McDonald’s at the food
court on November 28th or November 29th. The same
employees worked at McDonald’s on both days, though Jenny
had been ill from November 24th – 26th, suffering similar
symptoms as the children. She rotates duties at McDonald’s and
is often on her phone while at work, responding to texts at 867-
5309. The equipment and storage units at McDonald’s have

been inspected and are clean. Bathroom is clean but some
employee’s, like Jenny, do not wash their hands. All the
employees rotate between food service, cash register operation,
and drinks replacement, table cleaning, and utensil replacement.
Nine of the ten children played at the play area on the 27th or
28th. Five of them sat on Santa’s lap and told him what they
want for Christmas. The play area includes a ball pit, bounce
house, Merry-Go-Round, and toys in the middle of the mall
(fenced off, but parents are with the kids).
CASE #2
2. Bioterrorism can occur in a variety of ways and may or may
not include Category A agents. What makes agents like Bacillus
anthracis both a “good” biological weapon and a “bad” one? Do
the same for Yersinia pestis. Are either of these transmissible
person-to-person? Why must epidemiologists be involved with
the aftermath of suicide bombing events? Explain why
infectious agents, with varying levels of potential morbidity and
mortality, could be a real problem after a bombing. Who needs
to be fully examined and treated and why? (15 points)
CASE #3
3. A 45-year old man visits the emergency department at
University of Toledo Medical Center on November 30th. He
says he has been very weak the entire weekend; had a fever of
101 degrees Fahrenheit; joint pain in his wrists, knees, and
ankles; and muscle pain (myalgia). He also has a maculopapular
rash on his back and thighs. You ask about his recent activity
history and he informs you he has been assisting in the relief

and recovery efforts in Puerto Rico following the hurricanes (he
was there from November 16th – November 28th). He works for
Doctors Without Borders and was there with seven of his
colleagues. He said one of his colleagues has similar symptoms
but although he is a doctor, is too stubborn and will not get
treatment unless it gets “really serious”.
A physical exam reveals numerous mosquito bites, which the
patient says has been itching for the past five days. What
further questions do you ask the patient as an epidemiologist as
well as a health care provider? What tests would you run to
determine the agent? What do you think is the likely agent
based on this detective work and what treatment needs to be
performed on the patient? What should you do about his
colleagues who were with him? (15 points)
CASE #4
4. Mandy comes to the Parkview Medical Center Emergency
Room in Pueblo, Colorado with her boyfriend, Kyle. Mandy is
complaining of a headache, has a fever of 100.4 Fahrenheit, and
chills. Initial physical examination reveals swollen lymph
glands and a small ulcer on her right shoulder. She does not
recall being bitten by any of their animals but does state there
are plenty of insects around. The initial headache started two
days ago, but once the fever and chills started, they came to the
hospital.
They live on a farm and raise goats, rabbits, sheep, and
chickens. They also grow vegetables (carrots, kale, lettuce,
tomatoes, peppers). Mandy is often doing the gardening and
takes care of the chicken eggs, while Kyle focuses on the other
animals and hunting deer for an additional supply of winter
food. They have a freshwater pond on their property where they
enjoy swimming and playing in. Geese have been known to
spend their mornings around the pond, nibbling on the grass and

defecating.
What additional information would you ask of Kyle and Mandy?
What is/are the potential infectious agent(s) responsible and
why? How would you treat this case? Do you do anything for
Kyle and if so, what do you do? (15 points)
CASE #5
5. Bob (age 36) is a builder and contractor. His company, Bob
the Builder, Inc., is in charge of constructing new landscape to
the outside of the Bellagio Hotel and Casino in Las Vegas,
Nevada. During this process, they have had to uproot some palm
trees, tear down a fountain, and re-construct a swimming pool.
Every day, he comes home tired and usually covered with dirt
and debris. His wife, Anne (40), is a dentist. They have two
kids, Leonard (13) and Penny (10). Leonard is a seventh-grade
student at CalTech Prep and a talented cellist. Penny is a 4-H
fair all-star, winning many awards for her goats, rabbits, and
pigs.
Bob and two of his fellow employees have been sore for days,
not surprising for such strenuous work. However, all of them
are now suffering from headaches, have fevers, and are
coughing a lot, sometimes coughing up blood with their sputum.
Additionally, Penny has has a fever of 101 Fahrenheit currently,
and was complaining of chills the night before. Her mom
noticed that Penny’s lymph glands around her arms appear
swollen and Penny says they hurt.
Based on the scenario provided, what additional questions do
you need to ask of Bob, Penny, and Anne? List all of them and
provide a rationale as to why you are asking these questions.
Is there anyone else you need to interview and if so, why? Is

this a public health emergency and if so, why?
What infectious agents appear to be the culprits here and why?
Provide a rationale for your differential diagnosis. Then, list the
steps you need to follow, in order, to treat everyone affected.
Specify which authorities must be contacted throughout the
process and indicate who is providing what
medications/vaccines/treatments. (20 points)
CASE #6
6. A 47 year-old man went to the emergency department after
ten days of pain and progressive numbness in the left hand and
arm and pain in is lower neck and upper back. He had sought
treatment from a chiropractor for the past week but only helped
his back pain, as his numbness and tingling had worsened and
he was experiencing weakness in his left hand and arm. A
neurologic examination revealed normal strength and sensation
of his lower extremities. His right arm showed normal strength,
but the left hand showed no grip, and the patient could only lift
his left arm a few inches. The patient was afebrile, and his
blood pressure was normal when he arrived at the emergency
department.
You ask about any contact with animals and pets, especially out
of the ordinary. He informs you that he had been awakened,

eight months earlier, by a bat on his left arm, but he did not
seek medical attention. Rather, he killed the bat with a tennis
racquet and disposed of it in his outdoor trash can. He is
married to his wife (Mary, age 42) and has two daughters (Jane
16, Mandy 11). His wife and kids report no symptoms but do
say they never contacted a pest control specialist to inspect
their home for any nests.
What additional questions do you ask the man? The family?
Tests that you could run? Based on what they may tell you,
what do you think may be the infectious disease and why? What
is the man’s prognosis? (20 points)
CASE #7
7. Pam (age 40) presents to the emergency department with
droopy eyelids, blurred vision, difficulty speaking, and
difficulty breathing. Her family (husband Jack (41), and son
Tyler (10) brought her to the emergency department. Neither
appears to be symptomatic.
Jack informs the healthcare providers that they are part of a
farm co-op and raise livestock (pigs, sheep, cows, goats), raise
bees for which they make honey and beeswax candles. They
make their own cheese and smoked meats, along with the honey,
candles, and even organic soap that they take to local markets to
sell. Pam does take medical marijuana for pain but Jack does
not think that has to do with her current situation.
Pam has had difficulty defecating. Jack suspects this is due to
her recent lack of appetite. She cannot seem to eat anything.
Prior to the lack of appetite, she had some nausea, vomiting,
and diarrhea. Now, she moves slowly, talks slowly, and breathes
slowly.
What questions would you ask Pam? Her family? What tests

could you run? Would you communicate anyone else regarding
the local markets (e.g. staff, customers, news stations)? What do
you think is the infectious disease and why? (15 points)
CASE #8
8. On December 1st, a 47-year-old African-American man, a US
Postal Service employee who worked in the District of
Columbia mail distribution center came to the emergency
department with mild nonproductive cough, nausea, vomiting,
and stomach cramps. A less-than-thorough examination was
provided and he was given some antibiotics and sent home.
Later in the day, he arrived back at the emergency department
complaining of vomiting and profuse sweating. His past medical
history included asthma and renal calculi. Therapy for asthma
had not recently included corticosteroids. He was afebrile and
had orthostatic hypotension.
He informed the physicians that on December 1st, one of the
envelopes had a white substance, resembling talcum powder,
leaking out it. He did not think anything of it and continued
working. He also said that some of his co-workers had been
feeling ill and they were going to other hospitals in the area for
treatment. What would you do at the post office? What samples
would you take from the post office? From the employee(s)?
What do you think is the infectious agent (and why) and what
would you do in terms of preventing additional cases and
potentially identifying new cases as soon as possible?
(20 points)

QUESTION #9
9. Let us finish off with some potpourri. List all of the Category
A agents. Why are they considered to be in this category? Of
these agents, which one is considered to be the most lethal and
why (provide evidence)? Why might some of the diseases here
be misdiagnosed at the initial healthcare encounter? Which
agent has three major types of entry/disease? Which agent is
often thought to be associated with honey and what are other
ways this agent could be acquired? (15 points)

F21SC Industrial Programming CW2 Data Analytics (35) 20192.docx

Recommended

Recommended

More Related Content

Similar to F21SC Industrial Programming CW2 Data Analytics (35) 20192.docx

Similar to F21SC Industrial Programming CW2 Data Analytics (35) 20192.docx (20)

More from lmelaine

More from lmelaine (20)

Recently uploaded

Recently uploaded (20)

F21SC Industrial Programming CW2 Data Analytics (35) 20192.docx