SlideShare a Scribd company logo
1 of 24
data importing

Find this presentation and more at:
http://www.engineerica.com/workshops

Presenter: Diego Jancic
The basics - ADX
• Uploads your data to Accudemia

• Accepts CSV files out-of-the-box

• Custom data sources

• Schedule data uploads

                          http://www.engineerica.com/workshops
                                                            2
Let’s meet ADX




       http://www.engineerica.com/workshops
                                         3
Preparing your data

• One or many files?
  Recommended:
       Students, Instructors, Tutors, Classes, Enrollment, Center Assignments, Demographics




• Full export or incremental changes



                                             http://www.engineerica.com/workshops
                                                                                        4
Files format
• Comma-separated values (CSV) file

• Header row required
  (text itself not important)




                                http://www.engineerica.com/workshops
                                                                  5
Mapping the data




        http://www.engineerica.com/workshops
                                          6
One row, multiple info
• Examples:
   –   Student & Subject Area
   –   Tutor & Subject Area
   –   Instructor & Subject Area
   –   Student & Subject Area & Class
   –   Center & Subject Area
   –   Student & User Group




                                        http://www.engineerica.com/workshops
                                                                          7
Some examples




      http://www.engineerica.com/workshops
                                        8
Example: Student info




          http://www.engineerica.com/workshops
                                            9
Example: Student enrollment




             http://www.engineerica.com/workshops
                                              10
Importing data
          for the next semester

• It defaults to the current semester (if any)



• You can always specify a semester column to
  import data for the upcoming semester.


                          http://www.engineerica.com/workshops
                                                           11
How is the data merged?
From the documentation:
“The ADX engine finds duplicates in only the key fields of each entity. (…)

The key fields used for finding duplicates are the following:
• Center:
     – Records with the same Name are considered duplicates, therefore only one is kept.

•   Student:
     – Records with the same Person Number are considered duplicates, therefore only one is kept.

•   Subject Area:
     – Records with the same Name are considered duplicates, therefore only one is kept.
     – Records with the same Code and Code Prefix are considered duplicates, therefore only one is
       kept.

•   Student Enrollment:
     – Records with the same Student and Class are considered duplicates, therefore only one is kept.

                                                   http://www.engineerica.com/workshops
                                                                                                12
New data and assignments
• To assign two things (i.e. student to a class):
  – Specify the key fields




• To create new data (i.e. students)
  – Specify all the important fields

                             http://www.engineerica.com/workshops
                                                              13
The upload process

1. Source file gets processed                      On your
2. Data is validated for consistency              Computer
 3. Processed file is uploaded                      (ADX)

                ADX: OK

 4. New data is merged
                                                Accudemia
 5. Data is live!
              Data available
                          http://www.engineerica.com/workshops
                                                           14
Checking the server status
Advanced -> Import -> View Execution Details




                                http://www.engineerica.com/workshops
                                                                 15
Student demographics




          http://www.engineerica.com/workshops
                                           16
Student demographics

• Columns not mapped in ADX

• Specific header names

• First, create a profile questionnaire


                          http://www.engineerica.com/workshops
                                                           17
The profile questionnaire

             Page #1




                  Question #1




                  Question #2




            http://www.engineerica.com/workshops
                                             18
Header format
@Profile(Page=1, Question=1)
@Profile(Page=1, Question=2)
@Profile(Page=1, Question=3)
@Profile(Page=2, Question=1)
@Profile(Page=2, Question=2)
@Profile(Page=2, Question=3)


                       http://www.engineerica.com/workshops
                                                        19
Header format
@Profile(Page=1, Question=1)
@Profile(Page=1, Question=2)
@Profile(Page=1, Question=3)
@Profile(Page=2, Question=1)
@Profile(Page=2, Question=2)
@Profile(Page=2, Question=3)


                       http://www.engineerica.com/workshops
                                                        20
Demographics file example




            http://www.engineerica.com/workshops
                                             21
Custom data sources




         http://www.engineerica.com/workshops
                                          22
Custom data sources

  Your
database
                                                        Accudemia
 Your
CSV file


            Your ADX     ADX processes
             Add-On     the data stream


           CSV parser



                                   http://www.engineerica.com/workshops
                                                                    23
Questions?



Need help?
          Email us: support@accudemia.com

Find this presentation and related resources at:
            http://www.engineerica.com/workshops
                                                   24

More Related Content

Viewers also liked

Conceptofaccountabilityinpublicservice 120711120428-phpapp01
Conceptofaccountabilityinpublicservice 120711120428-phpapp01Conceptofaccountabilityinpublicservice 120711120428-phpapp01
Conceptofaccountabilityinpublicservice 120711120428-phpapp01Naseem Bangulzai
 
Catalogo nottepiu
Catalogo nottepiuCatalogo nottepiu
Catalogo nottepiunottepiu
 
Higiene industrial e segurança do trabalho
Higiene industrial e segurança do trabalhoHigiene industrial e segurança do trabalho
Higiene industrial e segurança do trabalhoAndrerlã Siqueira
 
click community o social media marketingu (CC 2014)
click community o social media marketingu (CC 2014)click community o social media marketingu (CC 2014)
click community o social media marketingu (CC 2014)click community
 
Tema 9 Belén
Tema 9 BelénTema 9 Belén
Tema 9 Belénbelenpm9
 
NCSTL briefs and Unauthorized Signature with cover pages
NCSTL briefs and Unauthorized Signature with cover pagesNCSTL briefs and Unauthorized Signature with cover pages
NCSTL briefs and Unauthorized Signature with cover pagesAndrew Wellman
 
Mendaftar webhost gratis, konfigurasi hosting dan database
Mendaftar webhost gratis, konfigurasi hosting dan databaseMendaftar webhost gratis, konfigurasi hosting dan database
Mendaftar webhost gratis, konfigurasi hosting dan databaseGhaz Alkho
 
Seminario: "VANTAGGI ED OPPORTUNITÀ PER LE AZIENDE NEL SETTORE ENERGETICO E ...
Seminario: "VANTAGGI ED OPPORTUNITÀ  PER LE AZIENDE NEL SETTORE ENERGETICO E ...Seminario: "VANTAGGI ED OPPORTUNITÀ  PER LE AZIENDE NEL SETTORE ENERGETICO E ...
Seminario: "VANTAGGI ED OPPORTUNITÀ PER LE AZIENDE NEL SETTORE ENERGETICO E ...eAmbiente
 
2015 unit 2 week 4
2015 unit 2 week 42015 unit 2 week 4
2015 unit 2 week 4scottsloan95
 
Traballos durante o día de San Valentín
Traballos  durante o día de San ValentínTraballos  durante o día de San Valentín
Traballos durante o día de San Valentínnicocrisfran
 

Viewers also liked (15)

Dayana cordero
Dayana corderoDayana cordero
Dayana cordero
 
Conceptofaccountabilityinpublicservice 120711120428-phpapp01
Conceptofaccountabilityinpublicservice 120711120428-phpapp01Conceptofaccountabilityinpublicservice 120711120428-phpapp01
Conceptofaccountabilityinpublicservice 120711120428-phpapp01
 
Catalogo nottepiu
Catalogo nottepiuCatalogo nottepiu
Catalogo nottepiu
 
Dsp11
Dsp11Dsp11
Dsp11
 
Higiene industrial e segurança do trabalho
Higiene industrial e segurança do trabalhoHigiene industrial e segurança do trabalho
Higiene industrial e segurança do trabalho
 
click community o social media marketingu (CC 2014)
click community o social media marketingu (CC 2014)click community o social media marketingu (CC 2014)
click community o social media marketingu (CC 2014)
 
Tema 9 Belén
Tema 9 BelénTema 9 Belén
Tema 9 Belén
 
NCSTL briefs and Unauthorized Signature with cover pages
NCSTL briefs and Unauthorized Signature with cover pagesNCSTL briefs and Unauthorized Signature with cover pages
NCSTL briefs and Unauthorized Signature with cover pages
 
Educacion fisica
Educacion fisicaEducacion fisica
Educacion fisica
 
Mendaftar webhost gratis, konfigurasi hosting dan database
Mendaftar webhost gratis, konfigurasi hosting dan databaseMendaftar webhost gratis, konfigurasi hosting dan database
Mendaftar webhost gratis, konfigurasi hosting dan database
 
Seminario: "VANTAGGI ED OPPORTUNITÀ PER LE AZIENDE NEL SETTORE ENERGETICO E ...
Seminario: "VANTAGGI ED OPPORTUNITÀ  PER LE AZIENDE NEL SETTORE ENERGETICO E ...Seminario: "VANTAGGI ED OPPORTUNITÀ  PER LE AZIENDE NEL SETTORE ENERGETICO E ...
Seminario: "VANTAGGI ED OPPORTUNITÀ PER LE AZIENDE NEL SETTORE ENERGETICO E ...
 
2015 unit 2 week 4
2015 unit 2 week 42015 unit 2 week 4
2015 unit 2 week 4
 
Dayana3.0
Dayana3.0Dayana3.0
Dayana3.0
 
Traballos durante o día de San Valentín
Traballos  durante o día de San ValentínTraballos  durante o día de San Valentín
Traballos durante o día de San Valentín
 
Document
DocumentDocument
Document
 

Similar to Import

Intro to Machine Learning with H2O and AWS
Intro to Machine Learning with H2O and AWSIntro to Machine Learning with H2O and AWS
Intro to Machine Learning with H2O and AWSSri Ambati
 
Cloud Native Analysis Platform for NGS analysis
Cloud Native Analysis Platform for NGS analysisCloud Native Analysis Platform for NGS analysis
Cloud Native Analysis Platform for NGS analysisYaoyu Wang
 
Lessons Learned from Managing Thousands of Production Apache Spark Clusters w...
Lessons Learned from Managing Thousands of Production Apache Spark Clusters w...Lessons Learned from Managing Thousands of Production Apache Spark Clusters w...
Lessons Learned from Managing Thousands of Production Apache Spark Clusters w...Databricks
 
Load Profilers Forum
Load Profilers Forum Load Profilers Forum
Load Profilers Forum Anna Enos
 
Proof of Concept for Learning Analytics Interoperability
Proof of Concept for Learning Analytics InteroperabilityProof of Concept for Learning Analytics Interoperability
Proof of Concept for Learning Analytics InteroperabilityOpen Cyber University of Korea
 
Technical Challenges and Approaches to Build an Open Ecosystem of Heterogeneo...
Technical Challenges and Approaches to Build an Open Ecosystem of Heterogeneo...Technical Challenges and Approaches to Build an Open Ecosystem of Heterogeneo...
Technical Challenges and Approaches to Build an Open Ecosystem of Heterogeneo...Ricard de la Vega
 
Building a Big Data Pipeline
Building a Big Data PipelineBuilding a Big Data Pipeline
Building a Big Data PipelineJesus Rodriguez
 
The Database Sizing Workflow
The Database Sizing WorkflowThe Database Sizing Workflow
The Database Sizing WorkflowKristofferson A
 
Workshop on php mysql by arth technology
Workshop on php mysql by arth technologyWorkshop on php mysql by arth technology
Workshop on php mysql by arth technologyAdarsh Patel
 
Building a Data Pipeline from Scratch - Joe Crobak
Building a Data Pipeline from Scratch - Joe CrobakBuilding a Data Pipeline from Scratch - Joe Crobak
Building a Data Pipeline from Scratch - Joe CrobakHakka Labs
 
Vertica 7.0 Architecture Overview
Vertica 7.0 Architecture OverviewVertica 7.0 Architecture Overview
Vertica 7.0 Architecture OverviewAndrey Karpov
 
Iochem.carles bo
Iochem.carles boIochem.carles bo
Iochem.carles bomaredata
 
The Tale of Two Deployments: Greenfield and Monolith Apps with Docker Enterpr...
The Tale of Two Deployments: Greenfield and Monolith Apps with Docker Enterpr...The Tale of Two Deployments: Greenfield and Monolith Apps with Docker Enterpr...
The Tale of Two Deployments: Greenfield and Monolith Apps with Docker Enterpr...Docker, Inc.
 
Ee 352 lab 1 (tutorial) - schwappach - 15 oct 09
Ee 352   lab 1 (tutorial) - schwappach - 15 oct 09Ee 352   lab 1 (tutorial) - schwappach - 15 oct 09
Ee 352 lab 1 (tutorial) - schwappach - 15 oct 09Loren Schwappach
 
Data Quality
Data QualityData Quality
Data Qualityjerdeb
 
Basic Application Performance Optimization Techniques (Backend)
Basic Application Performance Optimization Techniques (Backend)Basic Application Performance Optimization Techniques (Backend)
Basic Application Performance Optimization Techniques (Backend)Klas Berlič Fras
 
Apache Arrow: Open Source Standard Becomes an Enterprise Necessity
Apache Arrow: Open Source Standard Becomes an Enterprise NecessityApache Arrow: Open Source Standard Becomes an Enterprise Necessity
Apache Arrow: Open Source Standard Becomes an Enterprise NecessityWes McKinney
 

Similar to Import (20)

Intro to Machine Learning with H2O and AWS
Intro to Machine Learning with H2O and AWSIntro to Machine Learning with H2O and AWS
Intro to Machine Learning with H2O and AWS
 
Cloud Native Analysis Platform for NGS analysis
Cloud Native Analysis Platform for NGS analysisCloud Native Analysis Platform for NGS analysis
Cloud Native Analysis Platform for NGS analysis
 
Lessons Learned from Managing Thousands of Production Apache Spark Clusters w...
Lessons Learned from Managing Thousands of Production Apache Spark Clusters w...Lessons Learned from Managing Thousands of Production Apache Spark Clusters w...
Lessons Learned from Managing Thousands of Production Apache Spark Clusters w...
 
Load Profilers Forum
Load Profilers Forum Load Profilers Forum
Load Profilers Forum
 
Technical Challenges and Approaches to Build an Open Ecosystem of Heterogeneo...
Technical Challenges and Approaches to Build an Open Ecosystem of Heterogeneo...Technical Challenges and Approaches to Build an Open Ecosystem of Heterogeneo...
Technical Challenges and Approaches to Build an Open Ecosystem of Heterogeneo...
 
Technical Challenges and Approaches to Build an Open Ecosystem of Heterogeneo...
Technical Challenges and Approaches to Build an Open Ecosystem of Heterogeneo...Technical Challenges and Approaches to Build an Open Ecosystem of Heterogeneo...
Technical Challenges and Approaches to Build an Open Ecosystem of Heterogeneo...
 
Proof of Concept for Learning Analytics Interoperability
Proof of Concept for Learning Analytics InteroperabilityProof of Concept for Learning Analytics Interoperability
Proof of Concept for Learning Analytics Interoperability
 
Technical Challenges and Approaches to Build an Open Ecosystem of Heterogeneo...
Technical Challenges and Approaches to Build an Open Ecosystem of Heterogeneo...Technical Challenges and Approaches to Build an Open Ecosystem of Heterogeneo...
Technical Challenges and Approaches to Build an Open Ecosystem of Heterogeneo...
 
Building a Big Data Pipeline
Building a Big Data PipelineBuilding a Big Data Pipeline
Building a Big Data Pipeline
 
The Database Sizing Workflow
The Database Sizing WorkflowThe Database Sizing Workflow
The Database Sizing Workflow
 
Workshop on php mysql by arth technology
Workshop on php mysql by arth technologyWorkshop on php mysql by arth technology
Workshop on php mysql by arth technology
 
Building a Data Pipeline from Scratch - Joe Crobak
Building a Data Pipeline from Scratch - Joe CrobakBuilding a Data Pipeline from Scratch - Joe Crobak
Building a Data Pipeline from Scratch - Joe Crobak
 
Vertica 7.0 Architecture Overview
Vertica 7.0 Architecture OverviewVertica 7.0 Architecture Overview
Vertica 7.0 Architecture Overview
 
Iochem.carles bo
Iochem.carles boIochem.carles bo
Iochem.carles bo
 
Ado.net
Ado.netAdo.net
Ado.net
 
The Tale of Two Deployments: Greenfield and Monolith Apps with Docker Enterpr...
The Tale of Two Deployments: Greenfield and Monolith Apps with Docker Enterpr...The Tale of Two Deployments: Greenfield and Monolith Apps with Docker Enterpr...
The Tale of Two Deployments: Greenfield and Monolith Apps with Docker Enterpr...
 
Ee 352 lab 1 (tutorial) - schwappach - 15 oct 09
Ee 352   lab 1 (tutorial) - schwappach - 15 oct 09Ee 352   lab 1 (tutorial) - schwappach - 15 oct 09
Ee 352 lab 1 (tutorial) - schwappach - 15 oct 09
 
Data Quality
Data QualityData Quality
Data Quality
 
Basic Application Performance Optimization Techniques (Backend)
Basic Application Performance Optimization Techniques (Backend)Basic Application Performance Optimization Techniques (Backend)
Basic Application Performance Optimization Techniques (Backend)
 
Apache Arrow: Open Source Standard Becomes an Enterprise Necessity
Apache Arrow: Open Source Standard Becomes an Enterprise NecessityApache Arrow: Open Source Standard Becomes an Enterprise Necessity
Apache Arrow: Open Source Standard Becomes an Enterprise Necessity
 

Import

  • 1. data importing Find this presentation and more at: http://www.engineerica.com/workshops Presenter: Diego Jancic
  • 2. The basics - ADX • Uploads your data to Accudemia • Accepts CSV files out-of-the-box • Custom data sources • Schedule data uploads http://www.engineerica.com/workshops 2
  • 3. Let’s meet ADX http://www.engineerica.com/workshops 3
  • 4. Preparing your data • One or many files? Recommended: Students, Instructors, Tutors, Classes, Enrollment, Center Assignments, Demographics • Full export or incremental changes http://www.engineerica.com/workshops 4
  • 5. Files format • Comma-separated values (CSV) file • Header row required (text itself not important) http://www.engineerica.com/workshops 5
  • 6. Mapping the data http://www.engineerica.com/workshops 6
  • 7. One row, multiple info • Examples: – Student & Subject Area – Tutor & Subject Area – Instructor & Subject Area – Student & Subject Area & Class – Center & Subject Area – Student & User Group http://www.engineerica.com/workshops 7
  • 8. Some examples http://www.engineerica.com/workshops 8
  • 9. Example: Student info http://www.engineerica.com/workshops 9
  • 10. Example: Student enrollment http://www.engineerica.com/workshops 10
  • 11. Importing data for the next semester • It defaults to the current semester (if any) • You can always specify a semester column to import data for the upcoming semester. http://www.engineerica.com/workshops 11
  • 12. How is the data merged? From the documentation: “The ADX engine finds duplicates in only the key fields of each entity. (…) The key fields used for finding duplicates are the following: • Center: – Records with the same Name are considered duplicates, therefore only one is kept. • Student: – Records with the same Person Number are considered duplicates, therefore only one is kept. • Subject Area: – Records with the same Name are considered duplicates, therefore only one is kept. – Records with the same Code and Code Prefix are considered duplicates, therefore only one is kept. • Student Enrollment: – Records with the same Student and Class are considered duplicates, therefore only one is kept. http://www.engineerica.com/workshops 12
  • 13. New data and assignments • To assign two things (i.e. student to a class): – Specify the key fields • To create new data (i.e. students) – Specify all the important fields http://www.engineerica.com/workshops 13
  • 14. The upload process 1. Source file gets processed On your 2. Data is validated for consistency Computer 3. Processed file is uploaded (ADX) ADX: OK 4. New data is merged Accudemia 5. Data is live! Data available http://www.engineerica.com/workshops 14
  • 15. Checking the server status Advanced -> Import -> View Execution Details http://www.engineerica.com/workshops 15
  • 16. Student demographics http://www.engineerica.com/workshops 16
  • 17. Student demographics • Columns not mapped in ADX • Specific header names • First, create a profile questionnaire http://www.engineerica.com/workshops 17
  • 18. The profile questionnaire Page #1 Question #1 Question #2 http://www.engineerica.com/workshops 18
  • 19. Header format @Profile(Page=1, Question=1) @Profile(Page=1, Question=2) @Profile(Page=1, Question=3) @Profile(Page=2, Question=1) @Profile(Page=2, Question=2) @Profile(Page=2, Question=3) http://www.engineerica.com/workshops 19
  • 20. Header format @Profile(Page=1, Question=1) @Profile(Page=1, Question=2) @Profile(Page=1, Question=3) @Profile(Page=2, Question=1) @Profile(Page=2, Question=2) @Profile(Page=2, Question=3) http://www.engineerica.com/workshops 20
  • 21. Demographics file example http://www.engineerica.com/workshops 21
  • 22. Custom data sources http://www.engineerica.com/workshops 22
  • 23. Custom data sources Your database Accudemia Your CSV file Your ADX ADX processes Add-On the data stream CSV parser http://www.engineerica.com/workshops 23
  • 24. Questions? Need help? Email us: support@accudemia.com Find this presentation and related resources at: http://www.engineerica.com/workshops 24