Apache Arrow: Open Source Standard Becomes an Enterprise Necessity
Import
1. data importing
Find this presentation and more at:
http://www.engineerica.com/workshops
Presenter: Diego Jancic
2. The basics - ADX
• Uploads your data to Accudemia
• Accepts CSV files out-of-the-box
• Custom data sources
• Schedule data uploads
http://www.engineerica.com/workshops
2
4. Preparing your data
• One or many files?
Recommended:
Students, Instructors, Tutors, Classes, Enrollment, Center Assignments, Demographics
• Full export or incremental changes
http://www.engineerica.com/workshops
4
5. Files format
• Comma-separated values (CSV) file
• Header row required
(text itself not important)
http://www.engineerica.com/workshops
5
7. One row, multiple info
• Examples:
– Student & Subject Area
– Tutor & Subject Area
– Instructor & Subject Area
– Student & Subject Area & Class
– Center & Subject Area
– Student & User Group
http://www.engineerica.com/workshops
7
11. Importing data
for the next semester
• It defaults to the current semester (if any)
• You can always specify a semester column to
import data for the upcoming semester.
http://www.engineerica.com/workshops
11
12. How is the data merged?
From the documentation:
“The ADX engine finds duplicates in only the key fields of each entity. (…)
The key fields used for finding duplicates are the following:
• Center:
– Records with the same Name are considered duplicates, therefore only one is kept.
• Student:
– Records with the same Person Number are considered duplicates, therefore only one is kept.
• Subject Area:
– Records with the same Name are considered duplicates, therefore only one is kept.
– Records with the same Code and Code Prefix are considered duplicates, therefore only one is
kept.
• Student Enrollment:
– Records with the same Student and Class are considered duplicates, therefore only one is kept.
http://www.engineerica.com/workshops
12
13. New data and assignments
• To assign two things (i.e. student to a class):
– Specify the key fields
• To create new data (i.e. students)
– Specify all the important fields
http://www.engineerica.com/workshops
13
14. The upload process
1. Source file gets processed On your
2. Data is validated for consistency Computer
3. Processed file is uploaded (ADX)
ADX: OK
4. New data is merged
Accudemia
5. Data is live!
Data available
http://www.engineerica.com/workshops
14
15. Checking the server status
Advanced -> Import -> View Execution Details
http://www.engineerica.com/workshops
15
23. Custom data sources
Your
database
Accudemia
Your
CSV file
Your ADX ADX processes
Add-On the data stream
CSV parser
http://www.engineerica.com/workshops
23
24. Questions?
Need help?
Email us: support@accudemia.com
Find this presentation and related resources at:
http://www.engineerica.com/workshops
24