GRC Integration And Management Of Diverse Environmental Data Sets

•

1 like•303 views

Cameron Kiddle Research Fellow, Grid Research Centre University of Calgary Presented at the Cybera/CANARIE National Summit 2009, as part of the session "New Frontiers in Data Integration." This session showcased a selection of leading-edge initiatives that are breaking new ground and setting new precedents around the collection and integration of data.

Education Technology

Integration and Management
of Diverse Environmental
Data Sets
Cameron Kiddle

Research Fellow
Grid Research Centre
University of Calgary

Outline
!! Data Challenges
!! GeoChronos (Spectral Libraries)
!! Spectral Library Demonstration
!! Cloud Services for Water Management
!! Cloud Services for Water Management
Demonstration

Summit 09 2
Oct. 14, 2009

Data Challenges - Acquisition
!! Many different data sources

!! Different regulations/mechanisms for
accessing data

!! Lack of automation

!! Finding the right data

Summit 09 3
Oct. 14, 2009

Data Challenges - Management
!! Scattered and unorganized data

!! Inadequate tools for recording/
maintaining metadata
"! Data without metadata is meaningless
"! Lack of suitable metadata standards
"! Validation of metadata

!! Tracking provenance of data
Summit 09 4
Oct. 14, 2009

Data Challenges – Pre-processing
!! Raw data typically cannot be directly
analyzed

!! Significant amount of time spent
preparing data for analysis

!! Lack of automation

Summit 09 5
Oct. 14, 2009

GeoChronos
!! Partners
"! CANARIE (NEP-1)
"! Center for Earth Observation Sciences,
University of Alberta
"! Cybera
"! Grid Research Centre, University of Calgary

Summit 09 6
Oct. 14, 2009

GeoChronos
!! An on-line platform (http://geochronos.org/)
"! For:
!! Earth Observation Scientists
"! Facilitating:
!! Collaboration between scientists
!! Application access, management and sharing
!! Data access, management and sharing
"! Leveraging:
!! Web 2.0 and social networking technologies
!! Cloud computing technologies
!! Semantic Web technologies

Summit 09 7
Oct. 14, 2009

GeoChronos
!! Data Solutions - Spectral Libraries
"! Store, share and browse spectral data
"! View spectral plots, metadata, ancillary data and maps
"! Manage and generate metadata for spectra
"! Create and share metadata schemas
!! Technology
"! iRODS (http://www.irods.org/) for data storage/management
"! Semantic Web technologies such as RDF (Resource
Description Framework) to link/relate data
!! Next Steps
"! Generalization of spectral library solution - acquire, store,
manage, browse and share other types of data (i.e., satellite,
flux, phenology, meteorological, etc.)
"! Automate data workflows (i.e., mosaic, reproject and subset
MODIS data) using cloud-based services

Summit 09 8
Oct. 14, 2009

Spectral Library - Demonstration

Summit 09 9
Oct. 14, 2009

Spectral Library - Browse

Summit 09 10
Oct. 14, 2009

Spectral Library - Metadata

Summit 09 11
Oct. 14, 2009

Spectral Library - Ancillary Data

Summit 09 12
Oct. 14, 2009

Cloud Services for Water Management
!! Partners
"! Alberta Advanced Education and Technology
"! Alberta WaterSMART
"! Cybera
"! Geosensor Web Lab, University of Calgary
"! Grid Research Centre, University of Calgary
"! Tesera Systems Inc

Summit 09 13
Oct. 14, 2009

Cloud Services for Water Management
!! In preliminary stages of project
!! Explore use of cloud services to store,
manipulate and expose data related to water
management
!! Link and correlate a wide variety of data
from a large number of sources
!! Cloud-based analysis and visualization tools
!! Investigate integration with existing Alberta
WaterPortal (http://www.albertawater.com/)
Summit 09 14
Oct. 14, 2009

Cloud Services for Water Management -
Demonstration

Summit 09 15
Oct. 14, 2009

Data Types and Locations Displayed on
Google Earth

Summit 09 16
Oct. 14, 2009

Water Flow Data Shown using Google
Visualization API

Summit 09 17
Oct. 14, 2009

Integrating and Visualizing Different Data Sets
From Calgary’s Flood in 2005 on Google Earth

Summit 09 18
Oct. 14, 2009

Contact Information

http://grid.ucalgary.ca/

Cameron Kiddle
kiddlec@cpsc.ucalgary.ca
http://pages.cspc.ucalgary.ca/~kiddlec/

Summit 09 19
Oct. 14, 2009

This presentation examines some of the challenges scientists face and describes various cyberinfrastructure technologies that help address these challenges. Example projects employing cyberinfrastructure technologies that we have worked on at the Grid Research Centre, including the GeoChronos project, are also presented. This presentation was given at the IAI International Wireless Sensor Networks Summer School held at the University of Alberta on July 6th, 2009.

Virtual Appliances, Cloud Computing, and Reproducible Research

University of Washington

Machine learning and multimedia information retrievalSi Krishan

Semantic Web Landscape 2009

LeeFeigenbaum

Virtual Environments for Research in Archaeology (Mark Baker)

Onroerend Erfgoed

The VERA (Virtual Environment for Research in Archaeology) project is based on a research excavation of part of the large Roman town at Silchester in north Hampshire, which aims to trace the site's development from its origins before the Roman conquest to its abandonment in the fifth century A.D. The VERA project aims to investigate how archaeologists use Information Technology (IT) in the context of a field excavation, and also for post-excavation analysis. The project is investigating the impact of e-Infrastructure and the take-up of digital devices on an archaeological community. VERA is a two-year project funded by the JISC VRE 2 programme, involving researchers from the University of Reading, University College London, and York Archaeological Trust. In this talk I will outline and discuss the core aspects of the VERA project, ranging from on-site activities and usability studies, through to collaborative work with the archaeological community, and the technical developments to support fieldwork and post excavation analysis.

Web portal (GIS)

Janak Parajuli

Béatrice Markhoff - Semantic mediation ArSol and CIDOC CRM

ariadnenetwork

Lisbon DevOps: "Seven (More) Deadly Sins of Microservices"

Daniel Bryant

All is not completely rosy in microservice-land. It’s often a sign of an architectural approach’s maturity that anti-patterns begin to be identified and classified alongside well-established principles and practices. Daniel Bryant introduces seven deadly sins from real projects, which left unchecked could easily ruin your next microservices project. Daniel offers an updated tour of some of the nastiest anti-patterns in microservices from several real-world projects he’s encountered as a consultant, providing a series of anti-pattern “smells” to watch out for and exploring the tools and techniques you need to avoid or mitigate the potential damage. Topics include: Pride: the admission of the challenges with testing in a distributed system Envy: introducing inappropriate intimacy within services by creating a shared “canonical” domain model Wrath: failing to deal with the inevitable bad things that occur when operating new technologies, both from the people and technical aspects Sloth: composing services in a lazy fashion, which ultimately leads to the creation of a "distributed monolith” Lust: embracing the latest and greatest technology without evaluating the operational impact incurred by these choices.

GIS in the Rockies Geospatial Revolution

Peter Batty

Big Data, Linked Data

Paul Miller

Collaborative Science: Technologies & Examples - Cameron Kiddle, Grid Researc...

Cybera Inc.

Cloud Programming Models: eScience, Big Data, etc.Alexandru Iosup

Parker Keio 2011: Interoperable Manuscript FrameworkRobert Sanderson

Archaeological complete 2010Jayatunga Amaraweera

Building an Outsourcing Ecosystem for ScienceEuroCloud

Towards an effective digital archiving system for archaeological complete 2010Nadeeka Rathnabahu

GeoChronos - CANARIE NEP Showcase 2010 Presentation

Cameron Kiddle

Emc 2013 Big Data in Astronomy

Fabio Porto

Cyber Summit 2016: Technology, Education, and Democracy

Cybera Inc.

What are the opportunities and the challenges offered by emerging modes of technologically-inflected communication and decision-making? What is our role and responsibility as educators and as developers of research and teaching digital infrastructures? What do students need in the 21st century? As education institutions and providers struggle to respond to the first two questions, are we abrogating our responsibility to the last? In this talk, Matt Ratto will describe some of the opportunities and the challenges we currently face, laying out a model of action for how to potentially address the questions raised above. Core to his thinking are two related points; first that we must help students develop a greater sense of how the informational world and its attendant infrastructures helps shape how and what we think, and second, that a good way to do this is to give students the space to engage in reflexive acts of technological production – what Matt has termed ‘critical making.’ He will provide concrete examples from both his research and his teaching that demonstrate the value and importance of reflexive, hands-on work with digital technologies in helping students develop the critical digital literacy skills they need to function in today’s society. Matt Ratto is an Associate Professor in the Faculty of Information at the University of Toronto and directs the Semaphore Research cluster on Inclusive Design, Mobile and Pervasive Computing and, as part of Semaphore, the Critical Making lab.

Cyber Summit 2016: Understanding Users' (In)Secure Behaviour

Cybera Inc.

There is a prevailing belief that users are the weakest link the security chain. In this presentation, Dr. Chiasson discusses how this perspective is inherently counterproductive to achieving increased cyber security and explore alternatives with a higher chance of improving security. Why do users behave insecurely even though most will readily state that security and privacy are important? This talk will cover some of our recent research exploring reasons why users' actions do not necessarily reflect their desire for security and how the configuration of security systems may actually weaken security in practice. She presents her work using eye-tracking to determine how users make phishing determinations, and how we can persuade users to behave more securely through improving their mental models of passwords and by making adjustments to the system configurations.

Similar to GRC Integration And Management Of Diverse Environmental Data Sets

GeoChronos - CANARIE NEP Showcase 2009 Presentation

Cameron Kiddle

GeoChronos - SC09 Demo Presentation

Cameron Kiddle

Cyberinfrastructure and its Role in Science

Cameron Kiddle

Virtual Appliances, Cloud Computing, and Reproducible Research

University of Washington

Machine learning and multimedia information retrievalSi Krishan

Semantic Web Landscape 2009

LeeFeigenbaum

Virtual Environments for Research in Archaeology (Mark Baker)

Onroerend Erfgoed

Web portal (GIS)

Janak Parajuli

Béatrice Markhoff - Semantic mediation ArSol and CIDOC CRM

ariadnenetwork

Lisbon DevOps: "Seven (More) Deadly Sins of Microservices"

Daniel Bryant

GIS in the Rockies Geospatial Revolution

Peter Batty

Big Data, Linked Data

Paul Miller

Collaborative Science: Technologies & Examples - Cameron Kiddle, Grid Researc...

Cybera Inc.

Cloud Programming Models: eScience, Big Data, etc.Alexandru Iosup

Parker Keio 2011: Interoperable Manuscript FrameworkRobert Sanderson

Archaeological complete 2010Jayatunga Amaraweera

Building an Outsourcing Ecosystem for ScienceEuroCloud

Towards an effective digital archiving system for archaeological complete 2010Nadeeka Rathnabahu

GeoChronos - CANARIE NEP Showcase 2010 Presentation

Cameron Kiddle

Emc 2013 Big Data in Astronomy

Fabio Porto

Similar to GRC Integration And Management Of Diverse Environmental Data Sets (20)

GeoChronos - CANARIE NEP Showcase 2009 Presentation

GeoChronos - SC09 Demo Presentation

Cyberinfrastructure and its Role in Science

Virtual Appliances, Cloud Computing, and Reproducible Research

Machine learning and multimedia information retrieval

Semantic Web Landscape 2009

Virtual Environments for Research in Archaeology (Mark Baker)

Web portal (GIS)

Béatrice Markhoff - Semantic mediation ArSol and CIDOC CRM

Lisbon DevOps: "Seven (More) Deadly Sins of Microservices"

GIS in the Rockies Geospatial Revolution

Big Data, Linked Data

Collaborative Science: Technologies & Examples - Cameron Kiddle, Grid Researc...

Cloud Programming Models: eScience, Big Data, etc.

Parker Keio 2011: Interoperable Manuscript Framework

Archaeological complete 2010

Building an Outsourcing Ecosystem for Science

Towards an effective digital archiving system for archaeological complete 2010

GeoChronos - CANARIE NEP Showcase 2010 Presentation

Emc 2013 Big Data in Astronomy

More from Cybera Inc.

Cyber Summit 2016: Technology, Education, and Democracy

Cybera Inc.

Cyber Summit 2016: Understanding Users' (In)Secure Behaviour

Cybera Inc.

Cyber Summit 2016: Insider Threat Indicators: Human Behaviour

Cybera Inc.

Serious threats to private and governmental organizations do not only come from the outside world, but also come from within. Some employees and contractors with legitimate access to buildings, networks, assets and information deliberately misuse their priviledged access to cause harm to their organization. What are the reasons behind their actions? Is it debts, greed, ideology, disgruntlement, or divided loyalty? Regardless of their motivations or vulnerabilities, traitors have very similar types of personality and display a certain pattern of behaviours before committing an insider incident. As a prevention measure, it is vital that organizations and employees understand, recognize and detect the common indicators of insider threat. Would you recognize the signs? Mario Vachon is an Insider Threat Security Specialist with the RCMP Departmental Security Branch.

Cyber Summit 2016: Research Data and the Canadian Innovation Challenge

Cybera Inc.

Canada allocates a substantial amount of public funding to research, which is a critical factor in ensuring we remain innovative and competitive. Increasingly this funding is geared to the support and development of digital research infrastructure (DRI), including the underlying networks and the associated data acquisition, storage, analysis and visualization. In order to maximize the benefits of increasingly complex DRI and the research it facilitates, it is important to make sure data is properly stewarded, accessible and reusable. By adopting appropriate approaches to research data management we are better positioned to respond to challenges, such as effectively measuring research impacts, and ensuring the reproducibility, privacy, and security of research outputs. Research Data Canada (RDC) is a member-driven organization committed to developing a sustainable approach to research data management, one based on interoperability and best practices. This session will provide an update on the efforts of RDC and partner organizations, including: CANARIE, Compute Canada, CARL Portage Network, CASRAI, the TriAgencies, and the Leadership Council for Digital Infrastructure. Intersections with international activities and projects will also be highlighted. These efforts are ultimately designed to faciliate a cohesive national approach to research data management, and one based on a clearly articulated vision for supporting innovation and discovery in Canada. Mark Leggott is the Executive Director of Research Data Canada.

Cyber Summit 2016: Knowing More and Understanding Less in the Age of Big Data

Cybera Inc.

The Internet has revolutionized how — and how much — each of us can know. Our digital tools put the knowledge of the world at our fingertips — and soon, maybe, right into our heads. But what kinds of of knowledge do our devices give us, and how are they reshaping and challenging the role that education and libraries should play in our lives? This talk was delivered by Michael Patrick Lynch, professor of philosophy at the University of Connecticut, where he directs the university’s Humanities Institute.

Cyber Summit 2016: Privacy Issues in Big Data Sharing and Reuse

Cybera Inc.

Although there is no well-established definition of big data, its main characteristic is its sheer volume. Large volumes of data are generated by people (e.g., via social media) and by technology, including sensors (e.g., cameras, microphones), trackers (e.g., RFID tags, web surfing behavior) and other devices (e.g., mobile phones, wearables for self-surveillance/quantified self), whether or not they are connected to the Internet of Things. However, the large volumes of data needed to capitalize on the benefits of big data can to some extent also be established by the reuse of existing data, a source that is sometimes overlooked. Data can be reused for purposes similar to that for which it was initially collected, but also beyond these purposes. Similarly, data can be reused in its original context, but also beyond this context. However, such repurposing and recontextualizing of data may lead to privacy issues. For instance, data reuse may lead to issues regarding informed consent and informational self-determination. When the data is used for profiling and other types of predictive analytics, also issues regarding stigmatization and discrimination may arise. This presentation by Bart Custers, Head of Research, eLaw – Center for Law and Digital Technologies at Leiden University, The Netherlands, focuses on the privacy issues of big data sharing and reuse and how these issues could be addressed.

Cyber Summit 2016: Establishing an Ethics Framework for Predictive Analytics ...

Cybera Inc.

Stephen Childs was hired by the University of Calgary to develop an individual-level predictive model mapping students' decisions to attend the University. In his experience, the higher education sector was slow to use all the data it has available, but this is now changing. As interest in making use of organizational data grows, staff must consider how these models will be used, and any problems that could arise. When individual predictions become the basis for decisions, how do we ensure our algorithms don't make existing problems worse? A framework for handling these issues now will let organizations handle these issues in a way that is consistent with their values. Given the culture of today's institutions, and the success of predictive analytics in other fields, there is no doubt that these tools will be used. These techniques can improve student success and the competitiveness of educational organizations, but the benefits should not be gained at the expense of individuals within the system. This talk will propose a set of best practices for using institutional data for predictive modelling to address equity, privacy and other concerns. We must start thinking of this now, before other practices become entrenched.

Cyber Summit 2016: The Data Tsunami vs The Network: How More Data Changes Eve...

Cybera Inc.

Canada’s National Research and Education Network, like other ultra-speed research networks, has evolved to transfer massive amounts of data at 100Gbps and beyond. But with the volume of data traffic growing at more than 50% per year, the ability to move increasing volumes of data is challenging. What are the kinds of applications in research and education that are driving this growth? What are the implications of the coming data tsunami on our communication networks? And what happens to network economics to keep up with the demand? CANARIE’s Chief Technology Officer, Mark Wolff, explores these topics and offer insights into how the NREN will evolve to continue to meet the unique needs of Canada’s research and education community.

Cyber Summit 2016: Issues and Challenges Facing Municipalities In Securing Data

Cybera Inc.

The City of Calgary is responsible for providing municipal services to 1.1 million people and 16,000 employees with more than 700 sites and critical infrastructure units. The municipal services represent a $60B asset base including water and wastewater treatment plants, light rapid transit, emergency services, roads and recreation facilities, and has revenue and procurement streams of $4.0B annually. During his tenure, Owen Key, Chief Security Officer and the Chief Information Officer for the City, has implemented enterprise systems for CCTV, access and ID control, physical security information management systems, and has responsibility for information security.

Cyber Summit 2016: Using Law Responsibly: What Happens When Law Meets Technol...

Cybera Inc.

The law has long struggled to keep pace with the rapid change that comes with the Internet and new technologies. From the cross-border challenges posed by a global network to the privacy implications of big data, law and policy simply cannot move at “Internet speed.” Yet despite the difficulties, politicians and policy makers increasingly find themselves at the heart of emerging policy issues, asked to address the balance between privacy and surveillance, the competing copyright interests of creators and users, and the market structure for network providers and disruptive competitors. This keynote talk will explore the emerging law and policy challenges, highlighting how all Internet users have the opportunity to help shape the digital policy landscape. Dr. Michael Geist is a law professor at the University of Ottawa where he holds the Canada Research Chair in Internet and E-commerce Law.

Privacy, Security & Access to Data

Cybera Inc.

Do Universities Dream of Big Data

Cybera Inc.

Historically, the University of Alberta lacked a centrally managed repository for reporting data, resulting in inconsistency and disparity in access for units across campus. Meaningful and actionable reports were limited, and only focused on the interests and goals of the few units with data analysts who could synthesize the information. Over the last couple of years, the University of Alberta has undertaken major changes in how information is managed and utilized. At the forefront of this change has been an increased interest in supporting the development of analytics and supporting tools. Beginning with the implementation of a centrally managed data warehouse with self-service capabilities, and the introduction of cloud services with business process analysis tools, the University is just starting down the road of big data. This presentation explores opportunities and challenges for the University of Alberta in utilizing big data.

Predicting the Future With Microsoft Bing

Cybera Inc.

The next generation of data scientists will be asked to build predictive models that can extract inferences from very large datasets which are unobservable at the surface, even to the best domain experts. Microsoft has access to some truly large data sets, web and search data from the Bing search engine and social data through collaborations with Twitter. In this talk, we show you how a small team of data scientists used this data to build the Bing Predicts engine — a collection of machine learnt predictive models that is beating industry experts at predicting the outcome of events like the Super Bowl, the Oscars, elections and referendums and even breakthroughs in health sciences. The talk will also give a preview of how organizations can adopt a big data mindset to generate and experiment with large data sets and to make amazing predictions using their own data.

Analytics 101: How to not fail at analytics

Cybera Inc.

Are MOOC's past their peak?

Cybera Inc.

Opening the doors of the laboratory

Cybera Inc.

While the use of online instructional technologies allows the presentation of theoretical science materials, how do we deal with the fact that such courses often include hands-on labs? Laboratory simulations can only provide a solution for online students in a limited and often artificial way. Nearly 20 years ago, Athabasca University developed a solution to the problem of students having to travel to complete their lab work. Emerging technologies at the time allowed for quantitative physics labs to be sent to students as a small kit. The physics initiative was so successful, with over 5,000 students served, that it was picked up in other fields at Athabasca University. Over the years, such material has become cheaper, easier to use, and more integrated with modern computers. Athabasca is now pioneering ways to put real labs directly onto the internet. In this session, the methods used to make real lab experiences available to online students will be discussed, and some of them demonstrated.

Open City - Edmonton

Cybera Inc.

Unlocking the power of healthcare data

Cybera Inc.

Data science and the use of big data in healthcare delivery could revolutionize the field by decreasing costs and vastly improving efficiency and outcomes. There is an abundance of healthcare data in Canada, but it is mostly siloed and difficult to access due to privacy and security challenges. This session will offer insights into best practices for healthcare analytics programs, as well as use cases that demonstrate the potential benefits that can be realized through this work.

Checking in on Healthcare Data Analytics

Cybera Inc.

Open access and open data: international trends and strategic context

Cybera Inc.

Governments around the world fund billions of dollars in research every year. Ensuring that the results of research are available to the public, other researchers and industry has become an important underlying value in order to maximize the impact of our publicly funded research. This session will discuss what’s driving the trend towards greater openness and provide an overview of international developments that will help put Canada’s activities into context.

More from Cybera Inc. (20)