Data for New Technologies to Shape The Future of Transport by David Pickeral from the IBM Industry Smarter Solutions Team. Presented at Transforming Transportation 2014 co-organized by EMBARQ and The World Bank.
Machine learning is the ability of computers to learn through experiences to improve their performance. Research On Global Markets recently published a report on the global machine learning market. To take a look at snippets from the report.
Data for New Technologies to Shape The Future of Transport by David Pickeral from the IBM Industry Smarter Solutions Team. Presented at Transforming Transportation 2014 co-organized by EMBARQ and The World Bank.
Machine learning is the ability of computers to learn through experiences to improve their performance. Research On Global Markets recently published a report on the global machine learning market. To take a look at snippets from the report.
The global automotive equipment rental and leasing market was valued at $403.9 billion in 2017. Asia Pacific was the largest geographic region accounting for $155.9 billion or 38.6% of the global market.
Business Discovery @ Delhi International Airport - GMR GroupQlikView-India
Learn how QlikView Business Discovery is enabling Delhi International Airport Ltd. (DIAL) with strategic decision making. The speaker shares his views on how Qlik is helping DIAL in their Business Intelligence & Analytics approach to fact-based decision making
Collaboration using Open Source Software has resulted in fascinating broad-industry bases to support applications in the auto industry. Might we see similar efforts in healthcare?
Presented at AI NEXTCon Seattle 1/17-20, 2018
http://aisea18.xnextcon.com
join our free online AI group with 50,000+ tech engineers to learn and practice AI technology, including: latest AI news, tech articles/blogs, tech talks, tutorial videos, and hands-on workshop/codelabs, on machine learning, deep learning, data science, etc..
(New) Business models shaping future mobilityJosep Laborda
The "as a Service" paradigm coming to mobility, MaaS, insights on the market and drivers for MaaS and some hints on factors that will drive MaaS, such as cooperation among stakeholders, electrification, autonomous driving and the sharing economy.
Smart Mobility at Eindhoven University of Technology
Eindhoven University of Technology (TU/e) focuses on the societal challenges in Smart Mobility. Good mobility is of great importance for individuals, as well as for businesses and the economy. For this reason around 230 researchers from dozens of TU/e research groups are working in the Smart Mobility areas on clean, efficient and intelligent vehicle technology, and on logistics and traffic systems. The aims include reducing emissions and congestion, and increasing safety. Examples of recent developments at TU/e are intelligent cars that communicate with each other to prevent congestion, lighter batteries for electric cars, cleaner and more economical diesel engines, and optimized planning models for goods transport.
Focus areas:
• Automotive Technology
• Transport and Logistics
• Intelligent Transport Systems
• Mobility and Traffic
• ICT / Embedded Systems
Introductory deck describing Parkofon's unique infrastructure-free technology and value proposition across multiple modes as a provider of Mobility-as-a-Service (MaaS)
Global Challenge Porjct Report -Coursework of University of Bristol ssusera0a3b6
Group work in MSc Engineering Management, University of Bristol. This report proposes optimized solutions to the challenges of commuter transport in cities in developing countries to promote a low-carbon transformation.
Autonomous Vehicles: Technologies, Economics, and OpportunitiesJeffrey Funk
These slides use concepts from my (Jeff Funk) course entitled analyzing hi-tech opportunities to show how the cost and performance of autonomous vehicles are improving rapidly. LIDAR, other sensors, ICs, and wireless are experiencing rapid improvements that are enabling the overall cost of AVs to fall. For example, the latency of wireless systems is improving rapidly thus enabling vehicles to be controlled with wireless systems. This is also creating many new opportunities in the vehicle industry in the Internet of Things, data analytics, and logistics. The slides include a detailed discussion of AVs in Singapore, a likely early adopter.
The global automotive equipment rental and leasing market was valued at $403.9 billion in 2017. Asia Pacific was the largest geographic region accounting for $155.9 billion or 38.6% of the global market.
Business Discovery @ Delhi International Airport - GMR GroupQlikView-India
Learn how QlikView Business Discovery is enabling Delhi International Airport Ltd. (DIAL) with strategic decision making. The speaker shares his views on how Qlik is helping DIAL in their Business Intelligence & Analytics approach to fact-based decision making
Collaboration using Open Source Software has resulted in fascinating broad-industry bases to support applications in the auto industry. Might we see similar efforts in healthcare?
Presented at AI NEXTCon Seattle 1/17-20, 2018
http://aisea18.xnextcon.com
join our free online AI group with 50,000+ tech engineers to learn and practice AI technology, including: latest AI news, tech articles/blogs, tech talks, tutorial videos, and hands-on workshop/codelabs, on machine learning, deep learning, data science, etc..
(New) Business models shaping future mobilityJosep Laborda
The "as a Service" paradigm coming to mobility, MaaS, insights on the market and drivers for MaaS and some hints on factors that will drive MaaS, such as cooperation among stakeholders, electrification, autonomous driving and the sharing economy.
Smart Mobility at Eindhoven University of Technology
Eindhoven University of Technology (TU/e) focuses on the societal challenges in Smart Mobility. Good mobility is of great importance for individuals, as well as for businesses and the economy. For this reason around 230 researchers from dozens of TU/e research groups are working in the Smart Mobility areas on clean, efficient and intelligent vehicle technology, and on logistics and traffic systems. The aims include reducing emissions and congestion, and increasing safety. Examples of recent developments at TU/e are intelligent cars that communicate with each other to prevent congestion, lighter batteries for electric cars, cleaner and more economical diesel engines, and optimized planning models for goods transport.
Focus areas:
• Automotive Technology
• Transport and Logistics
• Intelligent Transport Systems
• Mobility and Traffic
• ICT / Embedded Systems
Introductory deck describing Parkofon's unique infrastructure-free technology and value proposition across multiple modes as a provider of Mobility-as-a-Service (MaaS)
Global Challenge Porjct Report -Coursework of University of Bristol ssusera0a3b6
Group work in MSc Engineering Management, University of Bristol. This report proposes optimized solutions to the challenges of commuter transport in cities in developing countries to promote a low-carbon transformation.
Autonomous Vehicles: Technologies, Economics, and OpportunitiesJeffrey Funk
These slides use concepts from my (Jeff Funk) course entitled analyzing hi-tech opportunities to show how the cost and performance of autonomous vehicles are improving rapidly. LIDAR, other sensors, ICs, and wireless are experiencing rapid improvements that are enabling the overall cost of AVs to fall. For example, the latency of wireless systems is improving rapidly thus enabling vehicles to be controlled with wireless systems. This is also creating many new opportunities in the vehicle industry in the Internet of Things, data analytics, and logistics. The slides include a detailed discussion of AVs in Singapore, a likely early adopter.
Winning with City and Video Intelligence - AWS Public Sector Summit Singapore...Amazon Web Services
Video is an effective tool for keeping people and property safe, but it still remains an underutilized source for operational, business, and public safety data. Hitachi Video Analytics is an intelligent solution for public sector organizations to extract rich operational and security insights from video data, while helping to safeguard the privacy of their customers, employees, and the citizens they protect. In this session, find out how you can make better use of your video data by providing critical intelligence that can help to improve operations and better protect people and assets.
3D Accelerometer and Acoustic Sensor Market Size, Share, & Trends Estimation ...subishsam
Sensors like 3D accelerometers and acoustic sensors can measure acceleration and sound. It can be used to measure how fast something is moving, how loud something is, or both at the same time. The sensor can measure acceleration and sound in three dimensions because it has three axes for acceleration and three axes for sound.
3D sensors are used in many different industries, including consumer electronics, healthcare, industrial robotics, security, automotive, surveillance, and many more. 3D sensors are made up of devices that respond to their surroundings in three dimensions by making 3D maps of the area around the user. The sensor is made up of different sensing technologies, such as ultrasound, time-of-flight (TOF), and structured light. As the need for gesture analysis applications grows, 3D sensors play an important role in improving the performance and efficiency of large, complex systems in industries like electronics and automotive.
Flight Information Display Systems (FIDS) Market Size, Share, & Trends Estima...subishsam
The Flight Information Display System (FIDS) is a computer system that airports use to tell passengers about their flights. FIDS can tell you the status of flights coming in and going out, as well as where to go and what to do. FIDS is important because it gives travelers the information they need to make good decisions about their trips.
Flight information systems are mostly electronic displays at airports that show passengers information about their flights. The system is run by a computer system, and it shows information like when different flights arrive or leave in real time to make things easier for passengers.
Solutions for ADAS and AI data engineering using OpenPOWER/POWER systemsGanesan Narayanasamy
The ultimate goal of ADAS feature development is to make our roads safer and better suited for fully autonomous vehicles in the long run. Still, manufacturers and buyers shouldn’t underestimate the importance of ADAS for meeting current automotive challenges. The most significant impact of advanced driver assistance systems is in providing drivers with essential information and automating difficult and repetitive tasks. This increases safety for everyone on the road
Things to Learn About:
*Customer success stories from Navistar and other leading auto manufacturers and insurers
*Key use cases and data architecture strategies for managing data from connected vehicles
For the full video of this presentation, please visit:
https://www.embedded-vision.com/industry-analysis/video-interviews-demos/path-adas-autonomy-presentation-strategy-analytics
For more information about embedded vision, please visit:
http://www.embedded-vision.com
Roger Lanctot, Director of Automotive Connected Mobility at Strategy Analytics, delivers the presentation "The Path from ADAS to Autonomy" at the Embedded Vision Alliance's December 2017 Vision Industry and Technology Forum. Lanctot shares his unique perspective on what the industry can realistically expect to achieve with ADAS and autonomous vehicles, using computer vision and other technologies.
Hitch-hikers guide to AI for Connected and Autonomous VehiclesBill Harpley
These are the slides from a talk given to the 'Self-driving and Autonomous Vehicles' meetup group, in Brighton on 12/02/2017. It provides an overview of how Artificial Intelligence (AI), Machine Learning and Deep Learning are shaping the future of the automotive industry.
Integrated Facility Management Ifm Market Survey Report 2023 Along with Stati...subishsam
The Integrated Facility Management Ifm Market research reports 2023-2030. A detailed study accumulated to offer the Latest insights about acute features of the Global Integrated Facility Management Ifm market. This report provides a detailed overview of key factors in the Integrated Facility Management Ifm Market and factors such as driver, restraint, past, and current trends, regulatory scenarios, and technology development. This report elaborates the market size, revenue, and growth of the Integrated Facility Management Ifm industry, and breaks it down according to the type, application, and consumption area of Integrated Facility Management Ifm. The report also conducted a PESTEL analysis of the industry to study the industry’s main influencing factors and entry barriers.
Safety Check is an IoT solution to prevent increasing number of road accidents due to Drink driving, rash driving & fatigue.
It is a pocket-friendly solution that every responsible driver and car manufacturer would like to own in their cars.
The field of DL has matured a lot in the last decade and changed a lot in the last few years. New architectures scaled to be larger/deeper, take advantage of a large number of datasets and parallel computing power.
Supervised DL methods, namely, CNNs and RNNs, are the natural choice for researchers in the automotive domain.
Important aspects such as compute power requirements, model transparency, and interpretability, model compliance with vehicle safety standards, all of which are expected to appreciably impact the adoption rate of DL in the automotive industry.
How to Successfully Apply Data & AI in the Marketing Value Chain. In this session Artefact will be starting with the role of data in the current world and what it has lead to currently: Where are we with Data & Artificial Intelligence? the future is definitely here. We will make it concrete and explore where to apply Data & AI in digital marketing? What can AI do and what can't it do (yet)?Possible areas are automation, optimization and more. Artefact will make it practical to conclude and explain what using Data & AI means practically for Digital Marketing? What are the actionable next steps in Planning, setup/workflow, getting control and creative.
Similar to Big Data LDN 2017: Deep Learning, DeepAd Car Recognition Project (20)
Blueprint Series: Banking In The Cloud – Ultra-high Reliability ArchitecturesMatt Stubbs
Data architecture for a challenger bank.Speaker: Jason Maude, Head of Technology Advocacy, Starling BankSpeaker Bio: Jason Maude is a coder, coach, and public speaker. He has over a decade of experience working in the financial sector, primarily in creating and delivering software. He is passionate about explaining complex technical concepts to those who are convinced that they won't be able to understand them. He currently works at Starling Bank as their Head of Technology Advocacy and host of the Starling podcast.Filmed at Skills Matter/Code Node London on 9th May 2019 as part of the Big Data LDN Meetup Blueprint Series.Meetup sponsored by DataStax.
Speed Up Your Apache Cassandra™ Applications: A Practical Guide to Reactive P...Matt Stubbs
Speaker: Cedrick Lunven, Developer Advocate, DataStax
Speaker Bio: Cedrick is a Developer Advocate at DataStax where he finds opportunities to share his passions by speaking about developing distributed architectures and implementing reference applications for developers. In 2013, he created FF4j, an open source framework for Feature Toggle which he still actively maintains. He is now contributor in JHipster team.
Talk Synopsis: We have all introduced more or less functional programming and asynchronous operations into our applications in order to speed up and distribute treatments (e.g., multi-threading, future, completableFuture, etc.). To build truly non-blocking components, optimize resource usage, and avoid "callback hell" you have to think reactive—everything is an event.
From the frontend UI to database communications, it’s now possible to develop Java applications as fully reactive with frameworks like Spring WebFlux and Reactor. With high throughput and tunable consistency, applications built on top of Apache Cassandra™ fit perfectly within this pattern.
DataStax has been developing Apache Cassandra drivers for years, and in the latest version of the enterprise driver we introduced reactive programming.
During this session we will migrate, step by step, a vanilla CRUD Java service (SpringBoot / SpringMVC) into reactive with both code review and live coding. Bring home a working project!
Filmed at Skills Matter/Code Node London on 9th May 2019 as part of the Big Data LDN Meetup Blueprint Series.
Meetup sponsored by DataStax.
Blueprint Series: Expedia Partner Solutions, Data PlatformMatt Stubbs
Join Anselmo for an engaging overview of the new end-to-end data architecture at Expedia Group, taking a journey through cloud and on-prem data lakes, real-time and batch processes and streamlined access for data producers and consumers. Find out how the new architecture unifies a complex mix of data sources and feeds the data science development cycle. Expedia might appear to be a market-leading travel company – in reality, it’s a highly successful technology and data science company.
Blueprint Series: Architecture Patterns for Implementing Serverless Microserv...Matt Stubbs
Richard Freeman talks about how the data science team at JustGiving built KOALA, a fully serverless stack for real-time web analytics capture, stream processing, metrics API, and storage service, supporting live data at scale from over 26M users. He discusses recent advances in serverless computing, and how you can implement traditionally container-based microservice patterns using serverless-based architectures instead. Deploying Serverless in your organisation can dramatically increase the delivery speed, productivity and flexibility of the development team, while reducing the overall running, DevOps and maintenance costs.
Big Data LDN 2018: DATABASE FOR THE INSTANT EXPERIENCEMatt Stubbs
Date: 14th November 2018
Location: Customer Experience Theatre
Time: 12:30 - 13:00
Speaker: David Maitland
Organisation: Redis Labs
About: This session will cover the technology underpinning at the software infrastructure level required to deliver the instant experience to the end user and enterprises alike. Use cases and value derived by major brands will be shared in this insightful session based the world's most loved database REDIS.
Big Data LDN 2018: BIG DATA TOO SLOW? SPRINKLE IN SOME NOSQLMatt Stubbs
Date: 14th November 2018
Location: Customer Experience Theatre
Time: 11:50 - 12:20
Speaker: Perry Krug
Organisation: Couchbase
About: Who wants to see an ad today for the shoes they bought last week? Everyone knows that customer experience is driven by data: don't waste an opportunity to get them the right data at the right time. Real-time results are critical, but raw speed isn't everything: you need power and flexibility to react to changes on the fly. Come learn how market-leading enterprises are using Couchbase as their speed layer for ingestion, incremental view and presentation layers alongside Kafka, Spark and Hadoop to liberate their data lakes.
Big Data LDN 2018: ENABLING DATA-DRIVEN DECISIONS WITH AUTOMATED INSIGHTSMatt Stubbs
Date: 13th November 2018
Location: Customer Experience Theatre
Time: 11:50 - 12:20
Speaker: Charlotte Emms
Organisation: seenit
About: How do you get your colleagues interested in the power of data? Taking you through Seenit’s journey using Couchbase's NoSQL database to create a regular, fully automated update in an easily digestible format.
Big Data LDN 2018: DATA MANAGEMENT AUTOMATION AND THE INFORMATION SUPPLY CHAI...Matt Stubbs
Date: 14th November 2018
Location: Governance and MDM Theatre
Time: 10:30 - 11:00
Speaker: Mike Ferguson
Organisation: IBS
About: For most organisations today, data complexity has increased rapidly. In the area of operations, we now have cloud and on-premises OLTP systems with customers, partners and suppliers accessing these applications via APIs and mobile apps. In the area of analytics, we now have data warehouse, data marts, big data Hadoop systems, NoSQL databases, streaming data platforms, cloud storage, cloud data warehouses, and IoT-generated data being created at the edge. Also, the number of data sources is exploding as companies ingest more and more external data such as weather and open government data. Silos have also appeared everywhere as business users are buying in self-service data preparation tools without consideration for how these tools integrate with what IT is using to integrate data. Yet new regulations are demanding that we do a better job of governing data, and business executives are demanding more agility to remain competitive in a digital economy. So how can companies remain agile, reduce cost and reduce the time-to-value when data complexity is on the up?
In this session, Mike will discuss how companies can create an information supply chain to manufacture business-ready data and analytics to reduce time to value and improve agility while also getting data under control.
Date: 13th November 2018
Location: Governance and MDM Theatre
Time: 12:30 - 13:00
Organisation: Immuta
About: Artificial intelligence is rising in importance, but it’s also increasingly at loggerheads with data protection regimes like the GDPR—or so it seems. In this talk, Sophie will explain where and how AI and GDPR conflict with one another, and how to resolve these tensions.
Big Data LDN 2018: REALISING THE PROMISE OF SELF-SERVICE ANALYTICS WITH DATA ...Matt Stubbs
Date: 13th November 2018
Location: Governance and MDM Theatre
Time: 11:50 - 12:20
Speaker: Mark Pritchard
Organisation: Denodo
About: Self-service analytics promises to liberate business users to perform analytics without the assistance of IT, and this in turn promises to free IT to focus on enhancing the infrastructure.
Join us to learn how data virtualization will allow you to gain real-time access to enterprise-wide data and deliver self-service analytics. We will explore how you can seamlessly unify fragmented data, replace your high-maintenance and high cost data integrations with a single, low-maintenance data virtualization layer; and how you can preserve your data integrity and ensure data lineage is fully traceable.
Big Data LDN 2018: TURNING MULTIPLE DATA LAKES INTO A UNIFIED ANALYTIC DATA L...Matt Stubbs
Date: 13th November 2018
Location: Governance and MDM Theatre
Time: 11:10 - 11:40
Organisation: TIBCO
About: The big data phenomenon continues to accelerate, resulting in multiple data lakes at most organisations. However, according to Gartner, “Through 2019, 90% of the information assets from big data analytic efforts will be siloed and unusable across multiple business processes.”
Are you ready to unleash this data from these silos and deliver the insights your organisation needs to drive compelling customer experiences, innovative new products and optimized operations? In this session you will learn how to apply data virtualisation to: - Access, transform and deliver data from across your lakes, clouds and other data sources - Empower a range of analytic users and tools with all the data they need - Move rapidly to a modern and flexible data architecture for the long run In addition, you will see a demonstration of data virtualisation in action.
Big Data LDN 2018: CONSISTENT SECURITY, GOVERNANCE AND FLEXIBILITY FOR ALL WO...Matt Stubbs
Date: 14th November 2018
Location: Data-Driven Ldn Theatre
Time: 12:30 - 13:00
Organisation: Cloudera
About: The growth of public cloud is reinforcing the need to think more carefully about taking a consistent approach to data governance as technology teams build out a flexible and agile infrastructure to meet the demands of the business.
Join this session to learn more about Cloudera's recommended approach for enterprise-grade security and governance and how to ensure a consistent framework across private, public and on-premises environments.
Big Data LDN 2018: MICROLISE: USING BIG DATA AND AI IN TRANSPORT AND LOGISTICSMatt Stubbs
Date: 14th November 2018
Location: Data-Driven Ldn Theatre
Time: 11:10 - 11:40
Organisation: Microlise
About: Microlise are a leading provider of technology solutions to the transport and logistics industry worldwide. Discover how, with over 400,000 connected assets generating billions of messages a day, Microlise is evolving its platform to bring real-time analytics to its customers to improve safety, security and efficiency outcomes.
Big Data LDN 2018: EXPERIAN: MAXIMISE EVERY OPPORTUNITY IN THE BIG DATA UNIVERSEMatt Stubbs
Date: 14th November 2018
Location: Data-Driven Ldn Theatre
Time: 10:30 - 11:00
Speaker: Anna Matty
Organisation: Experian
About: Today there is a widespread focus on the 'how' in relation to problem solving. How can we gain better knowledge of what consumers want, or need? How can we be more efficient, reduce the cost to serve, or grow the lifetime value of a customer? But, how do you move to a place where you are not only solving a problem, you are redesigning the entire strategic potential of that problem? You are being armed with insight on what the problem is.
Data and innovation offer huge potential to revolutionise all markets. There is an opportunity to be one step ahead of the need, to redesign journeys and enhance enterprise strategies. To do this you need access to the most advanced analytics but also the best quality, including variations and types of data, and then the technology that can act on this insight. Data science can present a unique opportunity for uncovered growth and accelerate your business through strategic innovation – fast. In this session you will hear more about how today's analytics can move from a single task, to an ongoing strategic opportunity. An opportunity that helps you move at the speed of the market and helps you maximise every opportunity.
Big Data LDN 2018: A LOOK INSIDE APPLIED MACHINE LEARNINGMatt Stubbs
Date: 13th November 2018
Location: Data-Driven Ldn Theatre
Time: 13:10 - 13:40
Speaker: Brian Goral
Organisation: Cloudera
About: The field of machine learning (ML) ranges from the very practical and pragmatic to the highly theoretical and abstract. This talk describes several of the challenges facing organisations that want to leverage more of their data through ML, including some examples of the applied algorithms that are already delivering value in business contexts.
Big Data LDN 2018: DEUTSCHE BANK: THE PATH TO AUTOMATION IN A HIGHLY REGULATE...Matt Stubbs
Date: 13th November 2018
Location: Data-Driven Ldn Theatre
Time: 12:30 - 13:00
Speaker: Paul Wilkinson, Naveen Gupta
Organisation: Cloudera
About: Investment banks are faced with some of the toughest regulatory requirements in the world. In a market where data is increasing and changing at extraordinary rates the journey with data governance never ends.
In this session, Deutsche Bank will share their journey with big data and explain some of the processes and techniques they have employed to prepare the bank for today’s challenges and tomorrow’s opportunities.
Brought to you by Naveen Gupta, VP Software Engineering, Deutsche Bank and Paul Wilkinson, Principal Solutions Architect, Cloudera.
Big Data LDN 2018: FROM PROLIFERATION TO PRODUCTIVITY: MACHINE LEARNING DATA ...Matt Stubbs
Date: 14th November 2018
Location: Self-Service Analytics Theatre
Time: 13:50 - 14:20
Speaker: Stephanie McReynolds
Organisation: Alation
About: Raw data is proliferating at an enormous rate. But so are our derived data assets - hundreds of dashboards, thousands of reports, millions of transformed data sets. Self-service analytics have ensured that this noise is making it increasingly hard to understand and trust data for decision-making. This trust gap is holding your organisation back from business outcomes.
European analytics leaders have found a way to close the gap between data and decision-making. From MunichRe to Pfizer and Daimler, analytics teams are adopting data catalogues for thousands of self-service analytics users.
Join us in this session to hear how data catalogues that activate data by incorporating machine learning can:
• Increase analyst productivity 20-40%
• Boost the understanding of the nuances of data and
• Establish trust in data-driven decisions with agile stewardship
Big Data LDN 2018: DATA APIS DON’T DISCRIMINATEMatt Stubbs
Date: 13th November 2018
Location: Self-Service Analytics Theatre
Time: 15:50 - 16:20
Speaker: Nishanth Kadiyala
Organisation: Progress
About: The exploding API economy, combined with an advanced analytics market projected to reach $30 billion by 2019, is forcing IT to expose more and more data through APIs. Business analysts, data engineers, and data scientists are still not happy because their needs never really made it into the existing API strategies. This is because most APIs are designed for application integration, but not for the data workers who are looking for APIs that facilitate direct data access to run complex analytics. Data APIs are specifically designed to provide that frictionless data access experience to support analytics across standard interoperable interfaces such as OData (REST) or ODBC/JDBC (SQL). Consider expanding your API strategy to service the developers with open analytics in this $30 billion market.
Enhanced Enterprise Intelligence with your personal AI Data Copilot.pdfGetInData
Recently we have observed the rise of open-source Large Language Models (LLMs) that are community-driven or developed by the AI market leaders, such as Meta (Llama3), Databricks (DBRX) and Snowflake (Arctic). On the other hand, there is a growth in interest in specialized, carefully fine-tuned yet relatively small models that can efficiently assist programmers in day-to-day tasks. Finally, Retrieval-Augmented Generation (RAG) architectures have gained a lot of traction as the preferred approach for LLMs context and prompt augmentation for building conversational SQL data copilots, code copilots and chatbots.
In this presentation, we will show how we built upon these three concepts a robust Data Copilot that can help to democratize access to company data assets and boost performance of everyone working with data platforms.
Why do we need yet another (open-source ) Copilot?
How can we build one?
Architecture and evaluation
Levelwise PageRank with Loop-Based Dead End Handling Strategy : SHORT REPORT ...Subhajit Sahu
Abstract — Levelwise PageRank is an alternative method of PageRank computation which decomposes the input graph into a directed acyclic block-graph of strongly connected components, and processes them in topological order, one level at a time. This enables calculation for ranks in a distributed fashion without per-iteration communication, unlike the standard method where all vertices are processed in each iteration. It however comes with a precondition of the absence of dead ends in the input graph. Here, the native non-distributed performance of Levelwise PageRank was compared against Monolithic PageRank on a CPU as well as a GPU. To ensure a fair comparison, Monolithic PageRank was also performed on a graph where vertices were split by components. Results indicate that Levelwise PageRank is about as fast as Monolithic PageRank on the CPU, but quite a bit slower on the GPU. Slowdown on the GPU is likely caused by a large submission of small workloads, and expected to be non-issue when the computation is performed on massive graphs.
Techniques to optimize the pagerank algorithm usually fall in two categories. One is to try reducing the work per iteration, and the other is to try reducing the number of iterations. These goals are often at odds with one another. Skipping computation on vertices which have already converged has the potential to save iteration time. Skipping in-identical vertices, with the same in-links, helps reduce duplicate computations and thus could help reduce iteration time. Road networks often have chains which can be short-circuited before pagerank computation to improve performance. Final ranks of chain nodes can be easily calculated. This could reduce both the iteration time, and the number of iterations. If a graph has no dangling nodes, pagerank of each strongly connected component can be computed in topological order. This could help reduce the iteration time, no. of iterations, and also enable multi-iteration concurrency in pagerank computation. The combination of all of the above methods is the STICD algorithm. [sticd] For dynamic graphs, unchanged components whose ranks are unaffected can be skipped altogether.
Adjusting primitives for graph : SHORT REPORT / NOTESSubhajit Sahu
Graph algorithms, like PageRank Compressed Sparse Row (CSR) is an adjacency-list based graph representation that is
Multiply with different modes (map)
1. Performance of sequential execution based vs OpenMP based vector multiply.
2. Comparing various launch configs for CUDA based vector multiply.
Sum with different storage types (reduce)
1. Performance of vector element sum using float vs bfloat16 as the storage type.
Sum with different modes (reduce)
1. Performance of sequential execution based vs OpenMP based vector element sum.
2. Performance of memcpy vs in-place based CUDA based vector element sum.
3. Comparing various launch configs for CUDA based vector element sum (memcpy).
4. Comparing various launch configs for CUDA based vector element sum (in-place).
Sum with in-place strategies of CUDA mode (reduce)
1. Comparing various launch configs for CUDA based vector element sum (in-place).
The Building Blocks of QuestDB, a Time Series Databasejavier ramirez
Talk Delivered at Valencia Codes Meetup 2024-06.
Traditionally, databases have treated timestamps just as another data type. However, when performing real-time analytics, timestamps should be first class citizens and we need rich time semantics to get the most out of our data. We also need to deal with ever growing datasets while keeping performant, which is as fun as it sounds.
It is no wonder time-series databases are now more popular than ever before. Join me in this session to learn about the internal architecture and building blocks of QuestDB, an open source time-series database designed for speed. We will also review a history of some of the changes we have gone over the past two years to deal with late and unordered data, non-blocking writes, read-replicas, or faster batch ingestion.
The Building Blocks of QuestDB, a Time Series Database
Big Data LDN 2017: Deep Learning, DeepAd Car Recognition Project
1. Deep Learning:
DeepAd Car Recognition Project
Neil Stobart
Global System Engineering Director
Cloudian
1
2. Machine Learning – Training AI
AI has the power to push ideas and actions beyond human biological limits,
providing us with capabilities to solve problems that were previously too
strenuous or tedious for humans.
ML is a large component of AI that gives
"computers the ability to learn without being explicitly programmed,“
AI pioneer Arthur Samuel, IBM, 1959.
The foundation of AI and ML are data management systems that organize
vast amounts of training data, the essential ingredient for all machine learning
and intelligence.
3. The “DeepAd project” – Making Big Data Smart Data
The “DeepAd project” delivered a digital billboard dynamic content system in Tokyo. DeepAd used
Artificial Intelligence combined with the Internet of Things and Big Data to detect and identify cars—with
94 percent accuracy—on one of Japan’s busy expressways. The system then selected and displayed
content based on the types of cars.
Project Goals
Create an out of home (OOH) advertising platform to target audiences with specific content just for them
Similar to highly targeted ads we experience while surfing the web via a billboard platform
Benefits advertisers with more effective content delivery
Connecting with Consumers through Effective Content Targeting
Improve the effectiveness of billboard outreach.
Big roadside signs and other OOH media do have an impact on behaviour, according to a 2013 Arbitron study
75% of adults notice OOH ads in a month
Over 25% of viewers visit a store, business, or restaurant immediately after seeing an OOH advertisement.
4. Camera installed under a billboard to live-stream
vehicles on Tokyo Metropolitan Expressway
6. Targeted Audiences
The system was trained to recognize specific car models, which
were classified in three different categories for the purpose of
delivering targeted messages:
1. Luxury cars—including all models of Mercedes, BMW, Audi,
and Lexus
2. Family cars—Toyota Prius*, Aqua*, and Vitz,* plus Honda Fit*
3. Project member cars—a 2001 Honda Odyssey*, a 2010
Subaru Outback* BR9, and a 2001 Toyota Bb*.
Control - These cars, driven by members of the DeepAd project passed
by the camera several times
No car detected
Current weather data
Luxury car
Golfing resort
Family car drivers
Local amusement park
Project members
Unique graphic image
7. Training Material
5,000 images of each targeted car type to train the
algorithm on the automobiles the system was to
target.
Nothing special about the images; they were
acquired publicly from automobile manufacturers’
and dealerships’ websites.
The algorithm looked at key features of each car,
such as fender radii (corner angle), headlights, and
other characteristics, to determine the automobile
maker, model, and year.
8. Smart Data Example
Car make, model, year,
and its view angle is
recognized and classified
by S3 key
9. Smarter Storage
One of the reasons why this technology is possible is
through the use of metadata. Typically, big data is just
stored passively for future analysis.
Because this data is unorganized and untagged, it
requires a good amount of effort in order to discover
and pull out specific information.
Object storage allows metadata tags attached to data objects.
As data runs through real-time classification and auto-
recognition/discrimination, metadata tags are attached on the fly.
As a result, we use this ‘deep learning’ to turn big data into smart
data.
11. Sensor
(Video Camera)
Automobiles,
robots,
manufacturing
machines,
monitors …
DL execution
environment
Neural
Model
Sensor Data
(photographed
images)
Real time processing
Recog-
nition Decision Control
CloudIoT / On-site
DL Learning
Environment
Training data
(big volume of
sample images)
(Video Data)
Control Signal
/ Feedback
Neural Model Generation
Edge
Sample Data
DevelopmentExecution / Operation
Sensor Data
Collaboration w/ DL Solution Companies
Deep Learning + HyperStore
✓ HyperStore enables meta data management, which is a good fit as storage for a DL
development environment where labeling of big volume of training data is required.
12. AI-based automated traffic census
AI and IoT can drive a shift of traffic survey from manual sampling to 24-hour, high-speed
measurement and aggregation
Deep learning-based traffic volume survey
• Automated counting, and drastic reduction of
measurement cost
• 24-hour measurement of traffic volume in details, e.g.,
congestion at multiple locations
• Instantaneous data aggregation and display
13. Measurement of buses and passengers at Shinjuku
Bus Terminal
• Automated AI-based measurement,
instead of data collection from bus
companies
• Conventional approach: Difficult to fully
collect data from 180 or more bus
companies
• Conventional approach: One month or
more to collect data which contains
significant errors and is based on
different measurement standards
• Target: AI-based data aggregation within
the next day and 365-day full automation
• Released on the 1st anniversary of
opening of Shinjuku Bus Terminal
14. What else? Security Cameras..
Road Monitoring
❖ traffic analysis
➢ vehicle counts
➢ pedestrian counts
❖ event detection
➢ falling objects
➢ flooding
Human Monitoring
❖ human behavior analysis
➢ suspicious
■ looking around
■ leaving objects
➢ help needed
■ disabled
■ elder
Why DL so interesting?
Provides very high accuracy in generic
object recognition with ease
NOTE: Good at discriminative object recognition like face
recognition and number plate recognition under strict proved
environments
OUR FOCUS PARTNER
15. Cloudian: AI Data Management
The AI Gold Rush = Massive Data Needs
Industrial IoT
Outdoor advertising
Traffic management
Server failure prediction
Electricity Management
Semiconductor defect detection
Machine parts routing
Movie pixilation
16. HyperStore Use Cases
Storage as a Service
QoS
Billing Multi Tenancy
Clients
S3 Storage
Service
APPLICATIONS BACKUP SERVERS
Backup and Archive
S3
INPUTS PROCESSING PACKAGING STORAGE DISTRIBUTIO
N
Media and Entertainment
Result of
Analysis
Hadoop Map Reduce
Apache Spark on Cloudian platform
Cloudian Platform
Cloudian HyperStore
Analytics
Social Media
Device Tracking
& Logs
Bioinformatics
AI, ML, Analytics