1) The document discusses challenges in managing large drone datasets using big data technologies and proposes a new architecture. It highlights issues like volume, variety, scalability and the need for real-time insights.
2) Key aspects of the proposed architecture include distributed storage and computing clusters, data partitioning strategies, and frameworks like Apache Spark and RasterFrames that can handle raster data at scale.
3) The use case presented is using AI and drones for intelligent inspection of large-scale photovoltaic installations, identifying defects through semantic segmentation and other deep learning methods on RGB and thermal imagery.
Lecture on Collaborative Augmented Reality given to the COSC 426 graduate class in AR. Taught by Mark Billinghurst from the HIT Lab NZ at the University of Canterbury.
L'offre de logiciels pour bibliothèque et les conditions d'acceptation d'un c...Marc Maisonneuve
Cette intervention de Marc Maisonneuve, le 13 janvier 2017 à l'Université de Fribourg s'inscrit dans le cadre du Certificat en gestion de documentation et de bibliothèque.
Déroulé de l'intervention
Introduction : quelques repères historiques
1. L’offre de logiciels métier pour bibliothèque
1.1 La typologie des logiciels
1.2 Les évolutions des système de gestion de bibliothèque et des opacs
1.3 L’évolution des modes de commercialisation des logiciels métier pour bibliothèque
2. Les modes d’exploitation d’un logiciel : sur ses propres serveurs, sur ceux d’un prestataire, en SaaS (plateforme de services)
3. Zoom sur le mode SaaS (plateforme de services)
3.1 Les solutions principalement disponibles en SaaS
3.2 Les avantages du mode SaaS
3.3 Les points de vigilance associés au mode SaaS
3.4 L’abonnement à une plateforme de services : à quoi dois-je penser ?
4. Les exigences d’accessibilité numérique
"Delivering COBie data" presents the research work of University of Padua and Technical University of Denmark in order to undesrtand best techniques for a better management of building envelopes during time.
COMP 4010 Lecture 5 on Interaction Design for Virtual Reality. Taught by Gun Lee on August 21st 2018 at the University of South Australia. Slides by Mark Billinghurst
Performansa Dayalı Mimari Tasarım, mimari sürdürülebilirlikilkeleri ve fiziksel çevre şartlarını tasarım sürecine sayısal, ölçülebilir bir faktör olarak olarak dahil edip, bina performansını gerçek zamanlı olarak ölçen ve çözümler üreten spiral bir tasarım yaklaşımıdır. BIM tabanlı bina performans analizi (BPA) yöntemleri, getirmiş olduğu gerçek zamanlı ve etkileşimli tasarım geri dönüşü olanakları sayesinde performansa mimari tasarımın erişilebilirliğini ve nesnelliğini önemli derecede arttırmıştır. Bu sunum Mimar Sinan Güzel Sanatlar ve Beykent üniversitelerinde gerçekleştirilen bir yüksek lisans dersi çerçevesinde BIM tabanlı BPA’nın tasarım süreçlerinde nasıl kullanılabileceğini anlatmaktadır. Derste kullanılan materyalin önemli bir kısmı Autodesk’in ilgili sertfika programından adapte edilmiştir. Dersle ilgili tüm kuramsal ve uygulamalı içerik de sayisalmimar.com sistesinde mevcuttr.
Lecture on Collaborative Augmented Reality given to the COSC 426 graduate class in AR. Taught by Mark Billinghurst from the HIT Lab NZ at the University of Canterbury.
L'offre de logiciels pour bibliothèque et les conditions d'acceptation d'un c...Marc Maisonneuve
Cette intervention de Marc Maisonneuve, le 13 janvier 2017 à l'Université de Fribourg s'inscrit dans le cadre du Certificat en gestion de documentation et de bibliothèque.
Déroulé de l'intervention
Introduction : quelques repères historiques
1. L’offre de logiciels métier pour bibliothèque
1.1 La typologie des logiciels
1.2 Les évolutions des système de gestion de bibliothèque et des opacs
1.3 L’évolution des modes de commercialisation des logiciels métier pour bibliothèque
2. Les modes d’exploitation d’un logiciel : sur ses propres serveurs, sur ceux d’un prestataire, en SaaS (plateforme de services)
3. Zoom sur le mode SaaS (plateforme de services)
3.1 Les solutions principalement disponibles en SaaS
3.2 Les avantages du mode SaaS
3.3 Les points de vigilance associés au mode SaaS
3.4 L’abonnement à une plateforme de services : à quoi dois-je penser ?
4. Les exigences d’accessibilité numérique
"Delivering COBie data" presents the research work of University of Padua and Technical University of Denmark in order to undesrtand best techniques for a better management of building envelopes during time.
COMP 4010 Lecture 5 on Interaction Design for Virtual Reality. Taught by Gun Lee on August 21st 2018 at the University of South Australia. Slides by Mark Billinghurst
Performansa Dayalı Mimari Tasarım, mimari sürdürülebilirlikilkeleri ve fiziksel çevre şartlarını tasarım sürecine sayısal, ölçülebilir bir faktör olarak olarak dahil edip, bina performansını gerçek zamanlı olarak ölçen ve çözümler üreten spiral bir tasarım yaklaşımıdır. BIM tabanlı bina performans analizi (BPA) yöntemleri, getirmiş olduğu gerçek zamanlı ve etkileşimli tasarım geri dönüşü olanakları sayesinde performansa mimari tasarımın erişilebilirliğini ve nesnelliğini önemli derecede arttırmıştır. Bu sunum Mimar Sinan Güzel Sanatlar ve Beykent üniversitelerinde gerçekleştirilen bir yüksek lisans dersi çerçevesinde BIM tabanlı BPA’nın tasarım süreçlerinde nasıl kullanılabileceğini anlatmaktadır. Derste kullanılan materyalin önemli bir kısmı Autodesk’in ilgili sertfika programından adapte edilmiştir. Dersle ilgili tüm kuramsal ve uygulamalı içerik de sayisalmimar.com sistesinde mevcuttr.
Lecture 6 of the COMP 4010 course on AR/VR. This lecture is about designing AR systems. This was taught by Mark Billinghurst at the University of South Australia on September 1st 2022.
A talk from the Creator Track at AWE USA 2019 - the World's #1 XR Conference & Expo in Santa Clara, California May 29-31, 2019.
Rokid: Design a seamless solution for AR glasses
Qi Xiong | Rokid
Paris Fan | Rokid
Nan Wang | Rokid
Gonglue Jiang | Rokid
In this talk, we will discuss how Rokid creates a seamless AR experience from two aspects: ergonomic design(hardware) and systematic UI/UX design(software). The presentation will also share the insights of the entire product development cycle of Rokid Glass: from crafting product appearance with strict constraints, deploying user interfaces on a new and unique AR device, to the case studies and progressive design of real implementations. The talk offers a glimpse of the future AR design methodology by demonstrating the superpower enabled by design thinking. Rokid Glass is an AI-enabled, all-in-one AR glasses already in mass production, with seamless user experience which widely adopted by the industry.
https://awexr.com
CapellaDays2022 | Politecnico di Milano | Interplanetary Space Mission as a r...Obeo
Systems engineering is an iterative approach traditionally applied one-way, from the definition of the user needs to the implementation of a solution that satisfies certain requirements and is constrained by cost and schedule. This presentation instead aims at exploring the educational benefits of applying the opposite practice, thus retrieving system and subsystem level requirements based on a solution already implemented and taking advantage of the MBSE possibilities to realize a model of the system according to the ARCADIA method and systems engineering approach, using the Capella MBSE Tool. This reverse-engineering process has been applied to a renowned Space mission, the ESA Mars Express satellite, whose goal is to investigate all aspects of the martian environment, including the subsurface, surface and atmosphere of the planet, in order to search for evidence of extinct or extant life. The uppermost goal of this project is to demonstrate the benefits for university students at a Master's level keen on systems engineering in implementing the Capella tool to retrieve the system architecture and the operational processes in a "reversed" strategy. In this work, students have been compelled to apply systems engineering processes to justify the design choices and exploit the already well-known missions and capabilities to build the architecture and functional chains as a starting point for the reverse engineering of the identified subsystems. The results prove it is possible, and also recommendable time-wise, to teach Space engineering and Systems engineering students by using this inverse approach, rather than the canonic one in which students have to design a whole mission from scratch.
A brief course we provide to our key trade partners on how to work with Navisworks effectively and efficiently to do clash detection on a typical complex project.
VCSELs – Market and Technology Trends 2019 by Yole DéveloppementYole Developpement
New functionalities in smartphone and automotive are boosting the VCSEL market.
More information on https://www.i-micronews.com/products/vcsels-market-and-technology-trends-2019/
Towards a Framework for XR Ethics - Kent Bye, AWE, November 11, 2021Kent Bye
For all the ways that immersive technologies can be used for good, they can be used for evil. This talk will provide some conceptual frames for making sense of the landscape of XR ethical dilemmas including human rights principles, tradeoffs between contextual dimensions, and mapping relationships between techno-social, political, and economic domains. This talk will be reporting back on some of the work done by the IEEE Global Initiative on the Ethics of Extended Reality, as well as provide insights into how to integrate ethically-aligned design and responsible innovation best practices into your experiential design process.
Talk given by Mark Billinghurst at the DIGI_X conference in Auckland, New Zealand on June 21st 2018. The talk was about how Mixed Reality can be applied in the work place.
Three-dimensional, virtual representation of a design project
Adds fourth dimension of time and fifth dimension of cost
“Cloud” allows different members of cross-functional team to work on the project in one place
Building Information Modeling (BIM) is the process of generating and managing building data during its life cycle. BIM uses three-dimensional, real-time, dynamic building modeling software to increase productivity in building design and construction
To create a BIM, a modeler uses intelligent objects (Families) to build the model.
CapellaDays2022 | Saratech | Interface Control Document Generation and Linkag...Obeo
Generation of Interface Control Documents (ICDs) using a model-based method has a number of advantages over text-based approaches. This paper describes the Python-based software that was written to automatically generate different versions of an ICD from a structure model in Capella. One use case for this approach is checking parts changes captured in the Engineering Bill of Materials (EBOM) using a PLM tool. We demonstrate an automated workflow that links changes in the EBOM to a request to vet the change against the ICD. This presentation will discuss our rationale, approach, results, and lessons learned.
COMP 4010 - Lecture 1: Introduction to Virtual RealityMark Billinghurst
Lecture 1 of the VR/AR class taught by Mark Billinghurst and Bruce Thomas at the University of South Australia. This lecture provides an introduction to VR and was taught on July 26th 2016.
LiDAR for Automotive and Industrial Applications 2019 by Yole DéveloppementYole Developpement
Is rationalization happening in the LiDAR market?
More information on: https://www.i-micronews.com/produit/lidar-for-automotive-and-industrial-applications-2019/
Flutter vs react native – from developer pointBOSC Tech Labs
With the rising number of mobile internet users, there is huge traffic. When it comes to developing a mobile application using cross-platform technologies both Flutter and React Native is the best available option. So we will learn about which is best for your next project requirement.
Lecture 6 of the COMP 4010 course on AR/VR. This lecture is about designing AR systems. This was taught by Mark Billinghurst at the University of South Australia on September 1st 2022.
A talk from the Creator Track at AWE USA 2019 - the World's #1 XR Conference & Expo in Santa Clara, California May 29-31, 2019.
Rokid: Design a seamless solution for AR glasses
Qi Xiong | Rokid
Paris Fan | Rokid
Nan Wang | Rokid
Gonglue Jiang | Rokid
In this talk, we will discuss how Rokid creates a seamless AR experience from two aspects: ergonomic design(hardware) and systematic UI/UX design(software). The presentation will also share the insights of the entire product development cycle of Rokid Glass: from crafting product appearance with strict constraints, deploying user interfaces on a new and unique AR device, to the case studies and progressive design of real implementations. The talk offers a glimpse of the future AR design methodology by demonstrating the superpower enabled by design thinking. Rokid Glass is an AI-enabled, all-in-one AR glasses already in mass production, with seamless user experience which widely adopted by the industry.
https://awexr.com
CapellaDays2022 | Politecnico di Milano | Interplanetary Space Mission as a r...Obeo
Systems engineering is an iterative approach traditionally applied one-way, from the definition of the user needs to the implementation of a solution that satisfies certain requirements and is constrained by cost and schedule. This presentation instead aims at exploring the educational benefits of applying the opposite practice, thus retrieving system and subsystem level requirements based on a solution already implemented and taking advantage of the MBSE possibilities to realize a model of the system according to the ARCADIA method and systems engineering approach, using the Capella MBSE Tool. This reverse-engineering process has been applied to a renowned Space mission, the ESA Mars Express satellite, whose goal is to investigate all aspects of the martian environment, including the subsurface, surface and atmosphere of the planet, in order to search for evidence of extinct or extant life. The uppermost goal of this project is to demonstrate the benefits for university students at a Master's level keen on systems engineering in implementing the Capella tool to retrieve the system architecture and the operational processes in a "reversed" strategy. In this work, students have been compelled to apply systems engineering processes to justify the design choices and exploit the already well-known missions and capabilities to build the architecture and functional chains as a starting point for the reverse engineering of the identified subsystems. The results prove it is possible, and also recommendable time-wise, to teach Space engineering and Systems engineering students by using this inverse approach, rather than the canonic one in which students have to design a whole mission from scratch.
A brief course we provide to our key trade partners on how to work with Navisworks effectively and efficiently to do clash detection on a typical complex project.
VCSELs – Market and Technology Trends 2019 by Yole DéveloppementYole Developpement
New functionalities in smartphone and automotive are boosting the VCSEL market.
More information on https://www.i-micronews.com/products/vcsels-market-and-technology-trends-2019/
Towards a Framework for XR Ethics - Kent Bye, AWE, November 11, 2021Kent Bye
For all the ways that immersive technologies can be used for good, they can be used for evil. This talk will provide some conceptual frames for making sense of the landscape of XR ethical dilemmas including human rights principles, tradeoffs between contextual dimensions, and mapping relationships between techno-social, political, and economic domains. This talk will be reporting back on some of the work done by the IEEE Global Initiative on the Ethics of Extended Reality, as well as provide insights into how to integrate ethically-aligned design and responsible innovation best practices into your experiential design process.
Talk given by Mark Billinghurst at the DIGI_X conference in Auckland, New Zealand on June 21st 2018. The talk was about how Mixed Reality can be applied in the work place.
Three-dimensional, virtual representation of a design project
Adds fourth dimension of time and fifth dimension of cost
“Cloud” allows different members of cross-functional team to work on the project in one place
Building Information Modeling (BIM) is the process of generating and managing building data during its life cycle. BIM uses three-dimensional, real-time, dynamic building modeling software to increase productivity in building design and construction
To create a BIM, a modeler uses intelligent objects (Families) to build the model.
CapellaDays2022 | Saratech | Interface Control Document Generation and Linkag...Obeo
Generation of Interface Control Documents (ICDs) using a model-based method has a number of advantages over text-based approaches. This paper describes the Python-based software that was written to automatically generate different versions of an ICD from a structure model in Capella. One use case for this approach is checking parts changes captured in the Engineering Bill of Materials (EBOM) using a PLM tool. We demonstrate an automated workflow that links changes in the EBOM to a request to vet the change against the ICD. This presentation will discuss our rationale, approach, results, and lessons learned.
COMP 4010 - Lecture 1: Introduction to Virtual RealityMark Billinghurst
Lecture 1 of the VR/AR class taught by Mark Billinghurst and Bruce Thomas at the University of South Australia. This lecture provides an introduction to VR and was taught on July 26th 2016.
LiDAR for Automotive and Industrial Applications 2019 by Yole DéveloppementYole Developpement
Is rationalization happening in the LiDAR market?
More information on: https://www.i-micronews.com/produit/lidar-for-automotive-and-industrial-applications-2019/
Flutter vs react native – from developer pointBOSC Tech Labs
With the rising number of mobile internet users, there is huge traffic. When it comes to developing a mobile application using cross-platform technologies both Flutter and React Native is the best available option. So we will learn about which is best for your next project requirement.
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataSafe Software
Once in a while, there really is something new under the sun. The rise of cloud-hosted data has fueled innovation in spatial data storage, enabling a brand new serverless architectural approach to spatial data sharing. Join us in our upcoming webinar to learn all about these new ways to organize your data, and leverage data shared by others. Explore the potential of Cloud Native Geospatial Formats in your workflows with FME, as we introduce five new formats: COGs, COPC, FlatGeoBuf, GeoParquet, STAC and ZARR.
Learn from industry experts Michelle Roby from Radiant Earth and Chris Holmes from Planet about these cloud-native geospatial data formats and how they can make data easier to manage, share, and analyze. To get us started, they’ll explain the goals of the Cloud-Native Geospatial Foundation and provide overviews of cloud-native technologies including the Cloud-Optimized GeoTIFF (COG), SpatioTemporal Asset Catalogs (STAC), and GeoParquet.
Following this, our seasoned FME team will guide you through practical demonstrations, showcasing how to leverage each format to its fullest potential. Learn strategic approaches for seamless integration and transition, along with valuable tips to enhance performance using these formats in FME.
Discover how these formats are reshaping geospatial data handling and how you can seamlessly integrate them into your FME workflows and harness the explosion of cloud-hosted data.
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FMESafe Software
Following the popularity of “Cloud Revolution: Exploring the New Wave of Serverless Spatial Data,” we’re thrilled to announce this much-anticipated encore webinar.
In this sequel, we’ll dive deeper into the Cloud-Native realm by uncovering practical applications and FME support for these new formats, including COGs, COPC, FlatGeoBuf, GeoParquet, STAC, and ZARR.
Building on the foundation laid by industry leaders Michelle Roby of Radiant Earth and Chris Holmes of Planet in the first webinar, this second part offers an in-depth look at the real-world application and behind-the-scenes dynamics of these cutting-edge formats. We will spotlight specific use-cases and workflows, showcasing their efficiency and relevance in practical scenarios.
Discover the vast possibilities each format holds, highlighted through detailed discussions and demonstrations. Our expert speakers will dissect the key aspects and provide critical takeaways for effective use, ensuring attendees leave with a thorough understanding of how to apply these formats in their own projects.
Elevate your understanding of how FME supports these cutting-edge technologies, enhancing your ability to manage, share, and analyze spatial data. Whether you’re building on knowledge from our initial session or are new to the serverless spatial data landscape, this webinar is your gateway to mastering cloud-native formats in your workflows.
Big Data with IOT approach and trends with case studySharjeel Imtiaz
The Big Data with IOT approach and trends. It will give you complete exposure of data science process and also will give insight how the step by step data science process explore the big data of TripAdvisor case study.
Hadoop Infrastructure @Uber Past, Present and FutureDataWorks Summit
Uber’s mission is to provide transportation as reliable as running water and for fulfilling that mission data plays a critical role. In Uber, Hadoop plays a critical role in Data Infrastructure. We want to talk about the journey of Hadoop @Uber and our future plans in terms of scaling for billions of trips. We will talk about most unique use case Uber have and how Hadoop and eco system which we built, helped us in this journey. We want to talk about how we scaled from 10 -> 2000 and In future to scale up to 10’s X1000 of Nodes. We will talk about our mistakes, learning and wins and how we process billions of events per day. We will talk about the unique challenges and real world use-cases and how we will co-locate the Uber’s service architecture with batch (e.g data pipelines, machine learning and analytical workloads). Uber have done lot of improvements to current Hadoop eco system and uniquely solved some of the problems in a way which is never been solved in the past. This presentation will help audience to use this as an example and even encourage them to enhance the eco system. This will help to increase the community of these project and overall help the whole big data space. Audience is anybody who is working on Big Data and want to understand how to scale Hadoop and eco system for 10s of thousands of node. This talk will help them understand the Hadoop ecosystem and how to efficiently use that. It will also introduce them to some of the awesome technologies which Uber team is building in big data space.
Using Familiar BI Tools and Hadoop to Analyze Enterprise NetworksMapR Technologies
From the Hadoop Summit 2015 Session with Nick Amato.
This session examines practical ways you can begin leveraging network data sources in Hadoop using familiar technologies like SQL and BI tools. Using the diverse sets of sources available, such as traces, routing protocol data, and direct packet captures from critical network locations, we will examine the capabilities of BI tools in the network context and examine cases for extracting value from data collected from the network infrastructure.
Learn more about the tools, techniques and technologies for working productively with data at any scale. This presentation introduces the family of data analytics tools on AWS which you can use to collect, compute and collaborate around data, from gigabytes to petabytes. We'll discuss Amazon Elastic MapReduce, Hadoop, structured and unstructured data, and the EC2 instance types which enable high performance analytics.
Jon Einkauf, Senior Product Manager, Elastic MapReduce, AWS
Alan Priestley, Marketing Manager, Intel and Bob Harris, CTO, Channel 4
First in Class: Optimizing the Data Lake for Tighter IntegrationInside Analysis
The Briefing Room with Dr. Robin Bloor and Teradata RainStor
Live Webcast October 13, 2015
Watch the archive: https://bloorgroup.webex.com/bloorgroup/lsr.php?RCID=012bb2c290097165911872b1f241531d
Hadoop data lakes are emerging as peers to corporate data warehouses. However, successful data management solutions require a fusion of all relevant data, new and old, which has proven challenging for many companies. With a data lake that’s been optimized for fast queries, solid governance and lifecycle management, users can take data management to a whole new level.
Register for this episode of The Briefing Room to learn from veteran Analyst Dr. Robin Bloor as he discusses the relevance of data lakes in today’s information landscape. He’ll be briefed by Mark Cusack of Teradata, who will explain how his company’s archiving solution has developed into a storage point for raw data. He’ll show how the proven compression, scalability and governance of Teradata RainStor combined with Hadoop can enable an optimized data lake that serves as both reservoir for historical data and as a "system of record” for the enterprise.
Visit InsideAnalysis.com for more information.
ADV Slides: When and How Data Lakes Fit into a Modern Data ArchitectureDATAVERSITY
Whether to take data ingestion cycles off the ETL tool and the data warehouse or to facilitate competitive Data Science and building algorithms in the organization, the data lake – a place for unmodeled and vast data – will be provisioned widely in 2020.
Though it doesn’t have to be complicated, the data lake has a few key design points that are critical, and it does need to follow some principles for success. Avoid building the data swamp, but not the data lake! The tool ecosystem is building up around the data lake and soon many will have a robust lake and data warehouse. We will discuss policy to keep them straight, send data to its best platform, and keep users’ confidence up in their data platforms.
Data lakes will be built in cloud object storage. We’ll discuss the options there as well.
Get this data point for your data lake journey.
Big data analytics and machine intelligence v5.0Amr Kamel Deklel
Why big data
What is big data
When big data is big data
Big data information system layers
Hadoop echo system
What is machine learning
Why machine learning with big data
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FMESafe Software
Following the popularity of "Cloud Revolution: Exploring the New Wave of Serverless Spatial Data," we're thrilled to announce this much-anticipated encore webinar.
In this sequel, we'll dive deeper into the Cloud-Native realm by uncovering practical applications and FME support for these new formats, including COGs, COPC, FlatGeoBuf, GeoParquet, STAC, and ZARR.
Building on the foundation laid by industry leaders Michelle Roby of Radiant Earth and Chris Holmes of Planet in the first webinar, this second part offers an in-depth look at the real-world application and behind-the-scenes dynamics of these cutting-edge formats. We will spotlight specific use-cases and workflows, showcasing their efficiency and relevance in practical scenarios.
Discover the vast possibilities each format holds, highlighted through detailed discussions and demonstrations. Our expert speakers will dissect the key aspects and provide critical takeaways for effective use, ensuring attendees leave with a thorough understanding of how to apply these formats in their own projects.
Elevate your understanding of how FME supports these cutting-edge technologies, enhancing your ability to manage, share, and analyze spatial data. Whether you're building on knowledge from our initial session or are new to the serverless spatial data landscape, this webinar is your gateway to mastering cloud-native formats in your workflows.
This talk explain how Delta Lake can be used as a reference architecture for data lakehouse. It gives the main concepts and principles behind Delta lake
Overview of Interpretability Approaches in Deep learning: Focus on Convnet ar...Dr Hajji Hicham
Slides of a tutorial I've given during the AI IndabaX Morocco conference, 30 April 2019: An Overview of Interpretability Approaches in Deep learning, with a focus in convnet. I review attribution approaches such as Perturbation, Gradient, and Pertinence techniques. and many others.
Code used in this session is here: https://github.com/hajjihi/IndabaxMorocco/blob/master/Convnet_Interpretability_IndabaXMorocco.ipynb
This talk (in french) develops how users can extend Spark and Spark SQL for processing Spatial Big Data. The talk focus only on Vector Data but the same tricks can be applied to Raster Datasets.
A longer version will be posted later with more details.
Explore our comprehensive data analysis project presentation on predicting product ad campaign performance. Learn how data-driven insights can optimize your marketing strategies and enhance campaign effectiveness. Perfect for professionals and students looking to understand the power of data analysis in advertising. for more details visit: https://bostoninstituteofanalytics.org/data-science-and-artificial-intelligence/
Data Centers - Striving Within A Narrow Range - Research Report - MCG - May 2...pchutichetpong
M Capital Group (“MCG”) expects to see demand and the changing evolution of supply, facilitated through institutional investment rotation out of offices and into work from home (“WFH”), while the ever-expanding need for data storage as global internet usage expands, with experts predicting 5.3 billion users by 2023. These market factors will be underpinned by technological changes, such as progressing cloud services and edge sites, allowing the industry to see strong expected annual growth of 13% over the next 4 years.
Whilst competitive headwinds remain, represented through the recent second bankruptcy filing of Sungard, which blames “COVID-19 and other macroeconomic trends including delayed customer spending decisions, insourcing and reductions in IT spending, energy inflation and reduction in demand for certain services”, the industry has seen key adjustments, where MCG believes that engineering cost management and technological innovation will be paramount to success.
MCG reports that the more favorable market conditions expected over the next few years, helped by the winding down of pandemic restrictions and a hybrid working environment will be driving market momentum forward. The continuous injection of capital by alternative investment firms, as well as the growing infrastructural investment from cloud service providers and social media companies, whose revenues are expected to grow over 3.6x larger by value in 2026, will likely help propel center provision and innovation. These factors paint a promising picture for the industry players that offset rising input costs and adapt to new technologies.
According to M Capital Group: “Specifically, the long-term cost-saving opportunities available from the rise of remote managing will likely aid value growth for the industry. Through margin optimization and further availability of capital for reinvestment, strong players will maintain their competitive foothold, while weaker players exit the market to balance supply and demand.”
Quantitative Data AnalysisReliability Analysis (Cronbach Alpha) Common Method...2023240532
Quantitative data Analysis
Overview
Reliability Analysis (Cronbach Alpha)
Common Method Bias (Harman Single Factor Test)
Frequency Analysis (Demographic)
Descriptive Analysis
Adjusting primitives for graph : SHORT REPORT / NOTESSubhajit Sahu
Graph algorithms, like PageRank Compressed Sparse Row (CSR) is an adjacency-list based graph representation that is
Multiply with different modes (map)
1. Performance of sequential execution based vs OpenMP based vector multiply.
2. Comparing various launch configs for CUDA based vector multiply.
Sum with different storage types (reduce)
1. Performance of vector element sum using float vs bfloat16 as the storage type.
Sum with different modes (reduce)
1. Performance of sequential execution based vs OpenMP based vector element sum.
2. Performance of memcpy vs in-place based CUDA based vector element sum.
3. Comparing various launch configs for CUDA based vector element sum (memcpy).
4. Comparing various launch configs for CUDA based vector element sum (in-place).
Sum with in-place strategies of CUDA mode (reduce)
1. Comparing various launch configs for CUDA based vector element sum (in-place).
1. Spatial Data Infrastructure and Earth Observation Education and Training for North Africa
AI and Big Data for Drone processing
usecase of photovoltaic inspection
25-April 2022
2. Spatial Data Infrastructure and Earth Observation Education and Training for North Africa
Agenda
• Big Data/AI and Drone
• Opportunities
• Challenges, Why is it Hard?
• Big Data Challenges…
• Toward a new architecture for drone Big Data
• Partitioning
• Storage
• Computing
• Some existing Big Data/AI frameworks for Drone
• Use case:
• Using AI and Drone for photovoltaic inspection
3. Spatial Data Infrastructure and Earth Observation Education and Training for North Africa
Big Data / AI and Drone
5. Spatial Data Infrastructure and Earth Observation Education and Training for North Africa
Moreover
• The potential of drones data is often underestimated
• Archiving collected data
• Curretly, we are doing more archiving tasks than managing drone data efficiently
• Almost no existing Big Data infrastructure can handle drones efficiently,
• Even if Big data is almost mature for other domains: Finance, Banking…
• Often it is
• Hard to store
• Hard to manage
• Hard to process
• Hard to get insight
• How ???
6. Spatial Data Infrastructure and Earth Observation Education and Training for North Africa
Hard to store: Volume
A very small drone project can generate more than
10 GB, sometimes more than 40Gb
15 million images of drone can make up more than 175
terabytes of data.
How to Store and Compute such growing volume?
FEDS : 13,000 flights this year
7. Spatial Data Infrastructure and Earth Observation Education and Training for North Africa
Hard to store: Variety
• “Drones can now provide a wide variety
of data types, everything from a few
basic photos through to complex
measurable 3D models with annotations
and overlays.”
Visual Encylopedia of drone data
Aerial Photography and Video
Orthomosaic Map
Digital Elevation Model (DEM)
3D Pointcloud Model
Multispectral Mapping
Thermal Imagery and Mapping
8. Spatial Data Infrastructure and Earth Observation Education and Training for North Africa
Hard to process:
Computing Model and Scalability
• Currently, drone image processing are done in one server: NOT SCALABLE
• Scalability is the property of a system to handle a growing amount of work by
adding resources to the system
• In Big Data, It is mostly done by distributing storage and computing
• Distributed computing can provide Scalability, but drone data friendly is Difficult
Processing/Querying drone data can take up to a few hours
Objective : real time (few seconds)
9. Spatial Data Infrastructure and Earth Observation Education and Training for North Africa
Going beyond traditional algorithms
Why not use Neural networks that have made great success with image:
▪ Semantic segmentation
▪ Object recognition, Classification..
▪ Description Generation for Drone Images Using Attribute Attention Mechanism
But theses new algorithms require more storage capacities and computing
power
Hard to get Insights
10. Spatial Data Infrastructure and Earth Observation Education and Training for North Africa
Current approaches are obsolete
we need to reinvent everything
Storage
Access Availability
Computing
Fast Accurate
Analytics
Machine
Learning
Deep
Learning
Search
By
semantic
By Spatial
Queries
…
11. Spatial Data Infrastructure and Earth Observation Education and Training for North Africa
1. New architecture to be redefined
Analytical Queries
Structured Storage
Cluster
Computing Cluster
…
Large Scale
Time series NDVI
•Distributing both STORAGE
•AND COMPUTING
12. Spatial Data Infrastructure and Earth Observation Education and Training for North Africa
2. Need to correlate drone data with external
datasets
More Insights
Census Data
Economic Data
Weather
…
13. Spatial Data Infrastructure and Earth Observation Education and Training for North Africa
3. Toward a declarative language (SQL-Like) over drone
data
Change in NDVI over the spring and early summer of 2018
Select normalized_difference(nir, red) as ndvi
From Feds_droneDataset
Where
date between ‘10-10-2017’ and ‘10-10-2019’
Examples from
‘10-10-2017’ to ‘10-10-2019’
Best option for Data Scientists
14. Spatial Data Infrastructure and Earth Observation Education and Training for North Africa
Recall that storage should be distributed across a cluster
• Before detailing storage techniques, let’s talk about Partitioning
Structured Storage
Cluster
…
Node A
Node B
Node F
Node G
15. Spatial Data Infrastructure and Earth Observation Education and Training for North Africa
Challenge for going distributed:
Data Partitioning
Partitioning means the process of physically dividing data into separate data
stores
Data is divided into partitions that can be managed and accessed separately.
Node 1
Node 2
Node 3
Node 4
16. Spatial Data Infrastructure and Earth Observation Education and Training for North Africa
Node 1
Node 2
Node 3
By Band
RGB
Red Band
Green Band
Blue Band
First simple approach is to partition by band
17. Spatial Data Infrastructure and Earth Observation Education and Training for North Africa
Node 1
Node 2
Node 3
By Time
Spring
Summer
Autumn
Other simple approach is to partition by time
(season)
18. Spatial Data Infrastructure and Earth Observation Education and Training for North Africa
Node 1
Node 2
Node 3
Decompose into NxN regular grids
But the Most efficient approach is to combine Tiling and Distribution
Tiling allows large raster datasets to be broken-up into manageable pieces higher level raster I/O interface.
19. Spatial Data Infrastructure and Earth Observation Education and Training for North Africa
Which Partition strategy to choose?
• Not in the scope of this presentation
• Check with your main objective:
• If for Scalability,
• If for Query Performance,
• If for Availability
• Many Best practices are available
• Sometimes we make use of Global Index for Optimizing Queries
20. Spatial Data Infrastructure and Earth Observation Education and Training for North Africa
The computing part
Computing
Model
HADOOP/MapReduce Spark/Spark SQL
We have at least two interesting computing models
21. Spatial Data Infrastructure and Earth Observation Education and Training for North Africa
Spark vs Hadoop MapReduce
Source: Data Flair
We will focus Next on Apache Spark
According to benchmarks studies, Spark is much better than Hadoop
MapReduce
22. Spatial Data Infrastructure and Earth Observation Education and Training for North Africa
Frameworks for Raster Big Data
23. Spatial Data Infrastructure and Earth Observation Education and Training for North Africa
Frameworks for Raster Big Data
Apache Spark / Spark SQL
• Rasterframes (My favorite)
Earth AI (To follow)
Google Earth Engine
Rasdaman
SciDB
24. Spatial Data Infrastructure and Earth Observation Education and Training for North Africa
• Spark project for Raster Data
• Spark Dataframe like abstraction for handling Raster Data : Provides ability to work with
Raster imagery in a convenient yet scalable format
• You can use Spark ML for building ML Models
B1
B2
B3
B4
tile or tile_n (where n is a band number)
25. Spatial Data Infrastructure and Earth Observation Education and Training for North Africa
Standard Tile Operations
• Many raster operations are ready to be executed in a distributed manner : can be
executed over Spark Cluster
• Ready to use
26. Spatial Data Infrastructure and Earth Observation Education and Training for North Africa
RasterFrames: SQL Query
• Can I Use spatial predicate in my query: intersection query?
27. Spatial Data Infrastructure and Earth Observation Education and Training for North Africa
SQL query in Rasterframes
SELECT month, ndvi_stats.*
FROM ("
SELECT month, rf_agg_stats(rf_normalized_difference(nir, red)) as ndvi_stats
FROM red_nir_tiles_monthly_2017
WHERE st_intersects(st_reproject(rf_geometry(red), rf_crs(red), 'EPSG:4326'),
st_makePoint(34.870605, -4.729727))
GROUP BY month
ORDER BY month )
"")
Compute the average NDVI per month for a single tile in an Area of
Interest
28. Spatial Data Infrastructure and Earth Observation Education and Training for North Africa
Earth AI
• is a Cloud-native software that enables you to apply advanced machine
learning algorithms to EO data at scale
• Both a non-code-based visual interface and pre-built workflows
• Ready-To-Use Datasets
• data archive includes more years of historical imagery and scientific datasets
• Elastic Compute
• Designed for scalability from the beginning, Earth AI platform scales seamlessly, so
you can think more about insights than Dev Ops
30. Spatial Data Infrastructure and Earth Observation Education and Training for North Africa
Earth AI
• Classifying an ecoregion using Decision Tree Classifier
31. Spatial Data Infrastructure and Earth Observation Education and Training for North Africa
SciDB
• Array-based data management and analytical system
• Arrays are divided into equally sized chunks
• Chunks are distributed over many SciDB instances
• Size and shape of chunks are defined by users per array and
have strong effects on computation times
• Storage is nearly sparse
• Relies on shared nothing architectures
• Open-source version available, extensible by UDFs
32. Spatial Data Infrastructure and Earth Observation Education and Training for North Africa
INTELLIGENT INSPECTION OF LARGE-SCALE
PHOTOVOLTAIC INSTALLATIONS THROUGH
RGB AND THERMAL INFRARED IMAGERY
ACQUIRED BY UNMANNED AERIAL VEHICLES
Imane SEBARI (a), Yahya ZEFRI (a), Hicham
HAJJI (a), Ghassane ANIBA (b)
(a) Photogrammetry-Cartography Department, School of Geomatics and Surveying
Engineering, IAV Hassan II, Rabat, Morocco
(b) Electrical Engineering Department, Mohammadia School of Engineers, Mohammed V
University in Rabat, Morocco
www.smartdrone4pv.com
33. Spatial Data Infrastructure and Earth Observation Education and Training for North Africa
INTELLIGENT INSPECTION OF LARGE-SCALE PHOTOVOLTAIC INSTALLATIONS THROUGH RGB AND
THERMAL INFRARED IMAGERY ACQUIRED BY UNMANNED AERIAL VEHICLES
CONTEXT & PROBLEMATIC
Solar
Photovoltaic
Modules
Contrasted
Temperatures
Humidity
Intrusion
Fierce
Winds
Rain/Snow/Hail
Handling
& Installation
Multiple
Defects
that develop over
time and penalize
the electricity
production Remotely Sensed
RGB and infrared
Imagery by UAVs
Contactless characterization
Faster image acquisition
Large-scale coverage
Increased accessibility
Hotspots
Delaminations
Discolorations
Cracks
34. Spatial Data Infrastructure and Earth Observation Education and Training for North Africa
THE SMARTDRONE4PV PROJECT
1. ADVANCED UAV
PHOTOGRAMMETRY
for RGB and long-wave
thermal infrared image
acquisition
2. DEEP LEARNING
SOLUTIONS
for defect detection and
classification on the used
imagery types
3. BIG DATA
ANALYTICS
to handle the huge
datasets that are generated
by large-scale PV plants
School of Geomatics and Surveying Engineering, IAV Hassan II, Rabat, Morocco
Mohammadia School of Engineers, Mohammed V University in Rabat, Morocco
Research Institute for Solar Energy and New Energies, Rabat, Morocco.
ETAFAT, Casablanca, Morocco.
INTELLIGENT INSPECTION OF LARGE-SCALE PHOTOVOLTAIC INSTALLATIONS THROUGH RGB AND
THERMAL INFRARED IMAGERY ACQUIRED BY UNMANNED AERIAL VEHICLES
35. Spatial Data Infrastructure and Earth Observation Education and Training for North Africa
DATA ACQUISITION
INTELLIGENT INSPECTION OF LARGE-SCALE PHOTOVOLTAIC INSTALLATIONS THROUGH RGB AND
THERMAL INFRARED IMAGERY ACQUIRED BY UNMANNED AERIAL VEHICLES
RGB and thermal infrared on-
field image acquisition
SfM-MVS
photogrammetric
post-processing
36. Spatial Data Infrastructure and Earth Observation Education and Training for North Africa 4
INTELLIGENT INSPECTION OF LARGE-SCALE PHOTOVOLTAIC INSTALLATIONS THROUGH RGB AND
THERMAL INFRARED IMAGERY ACQUIRED BY UNMANNED AERIAL VEHICLES
DEEP LEARNING-BASED DEFECT DETECTION
IMAGE CLASSIFICATION
37. Spatial Data Infrastructure and Earth Observation Education and Training for North Africa 4
INTELLIGENT INSPECTION OF LARGE-SCALE PHOTOVOLTAIC INSTALLATIONS THROUGH RGB AND
THERMAL INFRARED IMAGERY ACQUIRED BY UNMANNED AERIAL VEHICLES
DEEP LEARNING-BASED DEFECT DETECTION
SEMANTIC SEGMENTATION
38. Spatial Data Infrastructure and Earth Observation Education and Training for North Africa 4
INTELLIGENT INSPECTION OF LARGE-SCALE PHOTOVOLTAIC INSTALLATIONS THROUGH RGB AND
THERMAL INFRARED IMAGERY ACQUIRED BY UNMANNED AERIAL VEHICLES
DEEP LEARNING-BASED DEFECT DETECTION
OBJECT DETECTION