This document discusses using User-Managed Access (UMA) to protect personal data in an Internet of Things (IoT) network for a patient-centric use case. UMA allows an individual to control access to their personal data stored across different devices and systems. The document outlines how UMA could work in an example scenario where a patient's heart rate data is collected from a smart stethoscope and stored in an electronic health record. UMA provides a centralized authorization system to govern access to the patient's data based on their policies. This approach improves patient experience and empowerment over their personal health information.
Originally Published on Sep 17, 2014
What is a data refinery?
A data refinery is a facility for transforming raw data into relevant and actionable information. Data refinement services take the uncertainty out of the data foundation for analysis and operations. Refined data is timely, clean and well understood.
Who needs a data refinery?
Data access and refinement services provided by a data refinery on the cloud can make useful data readily available to everyone who needs it.
- Business people
- Application developers
- IT practitioners and data stewards
To learn more about data refinement, please visit:
http://www-01.ibm.com/software/data/refinery/
The document discusses how combining data across organizations within a sector can create "Sector Advantage". It proposes that a neutral third party like Digital Catapult could host combined data through its Data Catalyser initiative, allowing smaller companies to explore the data and extract value, while maintaining security and competitive advantages for data providers. The Data Catalyser uses technical and legal frameworks to allow innovative data exploration across organizations in a controlled way to enable new opportunities for sector advantage.
Data Refinery Is Fuelling Next Generation Big Data AnalyticsJean-Michel Franco
Latest Big Data technologies are allowing businesses to process ever growing volumes of data. But it is data quality, not quantity that's key to maximizing insight with Big Data analytics.
Is it possible to orchestrate and apply governance to all of that data and deliver it in a way that it can easily be consumed by the end user?
Learn how to take the uncertainty out of the data foundation for analytics by turning raw data into relevant and actionable information.
Meander Medical Center sought to improve patient care through a digital portal and mobile apps. They partnered with IBM Premier Business Partner Funatic to design a solution using IBM Digital Experience software. This included a patient portal, intranet, and mobile apps allowing patients to access records, appointments, and communicate with doctors. The growing user base and new capabilities are helping Meander enhance care coordination and expand their services digitally.
STAT Data Solutions provides an IoT data acquisition platform called IoTconnect that allows companies to collect data from devices and sensors in real-time to improve efficiencies and performance. IoTconnect supports over 30 protocols and can universally access any data source to perform analytics and integrate with backend systems. It helps companies build on their existing technology assets and data to derive business value from IoT. IoTconnect provides a versatile and fail-safe platform for automated data collection from a wide range of systems and devices.
This document discusses using User-Managed Access (UMA) to protect personal data in an Internet of Things (IoT) network for a patient-centric use case. UMA allows an individual to control access to their personal data stored across different devices and systems. The document outlines how UMA could work in an example scenario where a patient's heart rate data is collected from a smart stethoscope and stored in an electronic health record. UMA provides a centralized authorization system to govern access to the patient's data based on their policies. This approach improves patient experience and empowerment over their personal health information.
Originally Published on Sep 17, 2014
What is a data refinery?
A data refinery is a facility for transforming raw data into relevant and actionable information. Data refinement services take the uncertainty out of the data foundation for analysis and operations. Refined data is timely, clean and well understood.
Who needs a data refinery?
Data access and refinement services provided by a data refinery on the cloud can make useful data readily available to everyone who needs it.
- Business people
- Application developers
- IT practitioners and data stewards
To learn more about data refinement, please visit:
http://www-01.ibm.com/software/data/refinery/
The document discusses how combining data across organizations within a sector can create "Sector Advantage". It proposes that a neutral third party like Digital Catapult could host combined data through its Data Catalyser initiative, allowing smaller companies to explore the data and extract value, while maintaining security and competitive advantages for data providers. The Data Catalyser uses technical and legal frameworks to allow innovative data exploration across organizations in a controlled way to enable new opportunities for sector advantage.
Data Refinery Is Fuelling Next Generation Big Data AnalyticsJean-Michel Franco
Latest Big Data technologies are allowing businesses to process ever growing volumes of data. But it is data quality, not quantity that's key to maximizing insight with Big Data analytics.
Is it possible to orchestrate and apply governance to all of that data and deliver it in a way that it can easily be consumed by the end user?
Learn how to take the uncertainty out of the data foundation for analytics by turning raw data into relevant and actionable information.
Meander Medical Center sought to improve patient care through a digital portal and mobile apps. They partnered with IBM Premier Business Partner Funatic to design a solution using IBM Digital Experience software. This included a patient portal, intranet, and mobile apps allowing patients to access records, appointments, and communicate with doctors. The growing user base and new capabilities are helping Meander enhance care coordination and expand their services digitally.
STAT Data Solutions provides an IoT data acquisition platform called IoTconnect that allows companies to collect data from devices and sensors in real-time to improve efficiencies and performance. IoTconnect supports over 30 protocols and can universally access any data source to perform analytics and integrate with backend systems. It helps companies build on their existing technology assets and data to derive business value from IoT. IoTconnect provides a versatile and fail-safe platform for automated data collection from a wide range of systems and devices.
Personium - Open Source PDS envisioning the Web of MyData暁生 下野
How can we citizens maximize the benefits of the new right to data portability, which is now rapidly being recognized globally?
Personal Data Store is a technology that will receive all “My Data” from hundreds of services. It aggregates and integrates them, and at times discloses a portion of them to others under user’s control for creating new values.
This talk will introduce an open-source Personal Data Store (PDS) server “Personium”, providing details on its technical implementation, the underpinning business models, and the actual implemented and future use cases.
Self-learning machines can analyze big data from business & from digitized real world (IoT) rapidly and objectively in a trusted way using Blockchain than human beings (AI) to create additional value (Big Data).
Self-learning machines can analyze big data from business & from digitized real world (IoT) rapidly and objectively in a trusted way using Blockchain than human beings (AI) to create additional value (Big Data).
Defining a Digitalization Reference Architecture for the Pharma IndustryCapgemini
The rapid pace of technological advancement and an increasingly volatile business environment causes pharma industry players to face three main challenges. The first
challenge is to develop the critical ability to create and maintain a common view on the big picture of the organization’s architecture and environment. Linked to this is the challenge of making it decisible, which actions must be taken on strategical, business and technological level. The last challenge consists in identifying/considering the best
architectural approach and solution on a fine-grained level. This may include answer on question such as “Which approach we choose to implement a read replica in the cloud
(transactional replication using publish-subscribe between relational database systems, stream into a Redis Cache, …)?”
We develop a digitalization RA for pharma industry and incorporate three main areas, Internet of Things (IoT), Cognitive Computing (CC), and Augmented Reality (AR).
Digitalization is horizontally divisible into four sequentially arranged domains that can be understood as the stages of digitalization: Tag, Sense and Wire (1), Ingest (2), Analyze and Prepare (3), and Utilize (4). Technology components may be used across different industries. Applications developed and running in different industries may both interact or be integrated with each other and exchange data through these
industry-specific applications (cool chain monitoring while transportation, healthcare research data for public, origination & distribution information for authorities). For this reason, our frame possesses a third dimension: connected industries (in pharma these include transportation & logistics and the public sector). This approach generates a big picture that transparently illustrates the pharma RA for digitalization including its stages, architecture layers and connections between building blocks across organizational boundaries and different industries.
Author of paper and presentation: Eldar Sultanow, Carsten Brockmann, Levent Sözer, Capgemini Germany
The document discusses Data Analytics as a Service (DAaaS), which is an extensible analytical platform provided using cloud-based delivery models. DAaaS allows users to efficiently process and analyze large amounts of heterogeneous data. It provides analytical insights through configurable Analytical Apps that orchestrate data analysis workflows using analytical algorithms and machine learning. The platform must address challenges such as supporting real-time and non-real-time processing, hybrid cloud models, and the needs of data scientists.
Why Can't All Of Our Data Silos Just Get Along?Michelle Bruno
The technology landscape in the event industry is fragmented. There are lots of platforms that do lots of different things. The problem is that many don't share data and functionality, i.e. they don't work well together. Integration would be immensely helpful to change the situation, but there is still pushback and a layer of complexity that many event professionals aren't equipped to address. Read about what's going on and what needs to happen to change the situation.
The advantages in using data science as a business driver is tremendous.
If the data collected in a business is correctly applied it changes a business from a reactive and optimized entity to a proactive and optimized company.
Increasing profit margins and sustainability as well as giving a business the edge over it's competitors.
More and more leading companies are using custom analytics now to achieve an edge.
Gdpr ccpa steps to near as close to compliancy as possible with low risk of f...Steven Meister
The document outlines a 5 step plan to become compliant with GDPR and CCPA data protection laws:
1. Complete a Data Protection Impact Assessment to discover all personal data across systems.
2. Develop a remediation plan to encrypt personal data in key applications and files.
3. Begin remediation and testing by connecting encryption APIs to applications.
4. Ensure new personal data added is encrypted.
5. Prepare modified applications for production use after verifying no issues.
The goal is to protect personal data while maintaining business operations.
Patents are a good information resource for obtaining IoT (Internet of Things) technology development status. IOT big data analytics is becoming important to process unimaginably large amounts of information and data that are obtained by the sensor embedded interconnected IoT devices. The typical IoT big data analytics is Hadoop, an open-source software framework that supports data-intensive distributed applications, and the running of applications on large clusters of commodity hardware. Hadoop, that is based on the architectural framework MapReduce, collects both structured data and unstructured data, processes the collected data set in a distributed network cluster in parallel, and extracts valuable information from the processed data set within a short time. Followings illustrate some examples of patents that provide current status of the IoT big data analytics technology development.
Big data refers to large and diverse data sets that are analyzed to reveal insights. This document discusses how big data is important for the mobile market. It provides examples of how mobile data is gathered from loyalty programs, social media activity, emails, and sensors in phones. Big data helps mobile applications meet customer expectations, analyze user experience, access real-time data, increase revenue, and allow for personalization. Industries that commonly use big data include retail, healthcare, and banking. The growing mobile market relies on big data to develop engaging apps tailored to user needs.
Big data refers to large and diverse data sets that are analyzed to reveal insights. This document discusses how big data is important for the mobile market. It is gathered from various sources like loyalty programs, social media activity, emails, and sensors in phones. Big data helps mobile applications meet customer expectations, analyze user experience, access real-time data, improve revenue, and personalize services. Industries like retail, healthcare, and banking widely use big data. It will transform future mobile apps by helping businesses understand user needs better.
Big data refers to large and diverse data sets that are analyzed to reveal insights. This document discusses how big data is important for the mobile market. It is gathered from various sources like loyalty programs, social media activity, emails, and sensors in phones. Big data helps mobile applications meet customer expectations, analyze user experience, access real-time data, improve revenue, and personalize services. Industries like retail, healthcare, and banking widely use big data. It will transform future mobile apps by helping businesses understand user needs better.
This document discusses emerging mobile data platforms that integrate different data sets to provide insights into audience behaviors. It describes several platforms, including Lotame and AudienceArchitect, that combine first, second, and third party data to build robust audience profiles and enable targeted mobile marketing campaigns. As mobile data consumption increases, these platforms are gaining traction by helping marketers engage audiences across devices through personalized ads and experiences.
Two weeks ago we released an infographic showing the life of a normal person as we imagine it in 2025. The kinds of automated services illustrated in that graphic that will appear over the next decade are all enabled by the same piece of technology: the API. What APIs, or Application Programming Interfaces, do is provide programmers with a simple way of connecting their programming into the data and services of an existing program. Any websites that embed Google or Facebook logins are accessing an API provided by Google or Facebook to authenticate users' identities. Similarly, as personal data becomes stored in the cloud, companies with permission to access such data are increasingly plugged in to their consumers' changing status and needs.
Beyond just explaining how APIs are driving technological advancements across every industry, our latest Digital Lab Thought Piece provides useful tips for any business on navigating a world based on APIs - whether that means activating the APIs that are already out there, or building one of your own to create new revenue stream or make your services indispensable.
GDPR BigDataRevealed Readiness Requirements and EvaluationSteven Meister
This GDPR methodology can evaluate your GDPR readiness. For those feeling GDPR ready, you may uncover complex issues often neglected. For those that have waited, you can gain knowledge providing for a more successful GDPR outcome.
https://youtu.be/uE4Q7u0LatU https://youtu.be/R37S9mIiVAk https://youtu.be/AQf3if7DnuM
This document discusses the future of big data, including predictions such as machine learning becoming prominent and data scientists being in high demand. It outlines trends like the growth of open source technologies, in-memory computing, machine learning, predictive analytics, intelligent applications, integrating big data with security and the internet of things. Challenges mentioned include dealing with large amounts of data from IoT and high salaries for data professionals.
This document discusses the implementation of a Bring Your Own Device (BYOD) policy and program. It begins by explaining how the proliferation of mobile devices in the workplace has led to the rise of BYOD. It notes that most employees are already using their own devices for work purposes. The rest of the document outlines "The Ten Commandments of BYOD" which provide guidance on how to create a secure and productive mobile environment that supports BYOD while protecting corporate data. The ten commandments cover topics like creating a BYOD policy, identifying existing devices, simplifying enrollment, configuring devices remotely, giving users self-service options, and protecting personal information.
The document discusses the rise of Bring Your Own Device (BYOD) programs in workplaces and provides 10 commandments for effectively managing a BYOD program. It recommends that organizations first create a mobile device policy that considers what devices and apps will be allowed and how corporate and personal data will be separated and secured. It also stresses the importance of allowing simple, self-service enrollment and configuration of devices to reduce burden on IT staff and encourage user adoption. Continuous monitoring of devices is advised to ensure compliance with security policies and allow for automated responses to issues.
Open Source Contributions to Postgres: The Basics POSETTE 2024ElizabethGarrettChri
Postgres is the most advanced open-source database in the world and it's supported by a community, not a single company. So how does this work? How does code actually get into Postgres? I recently had a patch submitted and committed and I want to share what I learned in that process. I’ll give you an overview of Postgres versions and how the underlying project codebase functions. I’ll also show you the process for submitting a patch and getting that tested and committed.
Personium - Open Source PDS envisioning the Web of MyData暁生 下野
How can we citizens maximize the benefits of the new right to data portability, which is now rapidly being recognized globally?
Personal Data Store is a technology that will receive all “My Data” from hundreds of services. It aggregates and integrates them, and at times discloses a portion of them to others under user’s control for creating new values.
This talk will introduce an open-source Personal Data Store (PDS) server “Personium”, providing details on its technical implementation, the underpinning business models, and the actual implemented and future use cases.
Self-learning machines can analyze big data from business & from digitized real world (IoT) rapidly and objectively in a trusted way using Blockchain than human beings (AI) to create additional value (Big Data).
Self-learning machines can analyze big data from business & from digitized real world (IoT) rapidly and objectively in a trusted way using Blockchain than human beings (AI) to create additional value (Big Data).
Defining a Digitalization Reference Architecture for the Pharma IndustryCapgemini
The rapid pace of technological advancement and an increasingly volatile business environment causes pharma industry players to face three main challenges. The first
challenge is to develop the critical ability to create and maintain a common view on the big picture of the organization’s architecture and environment. Linked to this is the challenge of making it decisible, which actions must be taken on strategical, business and technological level. The last challenge consists in identifying/considering the best
architectural approach and solution on a fine-grained level. This may include answer on question such as “Which approach we choose to implement a read replica in the cloud
(transactional replication using publish-subscribe between relational database systems, stream into a Redis Cache, …)?”
We develop a digitalization RA for pharma industry and incorporate three main areas, Internet of Things (IoT), Cognitive Computing (CC), and Augmented Reality (AR).
Digitalization is horizontally divisible into four sequentially arranged domains that can be understood as the stages of digitalization: Tag, Sense and Wire (1), Ingest (2), Analyze and Prepare (3), and Utilize (4). Technology components may be used across different industries. Applications developed and running in different industries may both interact or be integrated with each other and exchange data through these
industry-specific applications (cool chain monitoring while transportation, healthcare research data for public, origination & distribution information for authorities). For this reason, our frame possesses a third dimension: connected industries (in pharma these include transportation & logistics and the public sector). This approach generates a big picture that transparently illustrates the pharma RA for digitalization including its stages, architecture layers and connections between building blocks across organizational boundaries and different industries.
Author of paper and presentation: Eldar Sultanow, Carsten Brockmann, Levent Sözer, Capgemini Germany
The document discusses Data Analytics as a Service (DAaaS), which is an extensible analytical platform provided using cloud-based delivery models. DAaaS allows users to efficiently process and analyze large amounts of heterogeneous data. It provides analytical insights through configurable Analytical Apps that orchestrate data analysis workflows using analytical algorithms and machine learning. The platform must address challenges such as supporting real-time and non-real-time processing, hybrid cloud models, and the needs of data scientists.
Why Can't All Of Our Data Silos Just Get Along?Michelle Bruno
The technology landscape in the event industry is fragmented. There are lots of platforms that do lots of different things. The problem is that many don't share data and functionality, i.e. they don't work well together. Integration would be immensely helpful to change the situation, but there is still pushback and a layer of complexity that many event professionals aren't equipped to address. Read about what's going on and what needs to happen to change the situation.
The advantages in using data science as a business driver is tremendous.
If the data collected in a business is correctly applied it changes a business from a reactive and optimized entity to a proactive and optimized company.
Increasing profit margins and sustainability as well as giving a business the edge over it's competitors.
More and more leading companies are using custom analytics now to achieve an edge.
Gdpr ccpa steps to near as close to compliancy as possible with low risk of f...Steven Meister
The document outlines a 5 step plan to become compliant with GDPR and CCPA data protection laws:
1. Complete a Data Protection Impact Assessment to discover all personal data across systems.
2. Develop a remediation plan to encrypt personal data in key applications and files.
3. Begin remediation and testing by connecting encryption APIs to applications.
4. Ensure new personal data added is encrypted.
5. Prepare modified applications for production use after verifying no issues.
The goal is to protect personal data while maintaining business operations.
Patents are a good information resource for obtaining IoT (Internet of Things) technology development status. IOT big data analytics is becoming important to process unimaginably large amounts of information and data that are obtained by the sensor embedded interconnected IoT devices. The typical IoT big data analytics is Hadoop, an open-source software framework that supports data-intensive distributed applications, and the running of applications on large clusters of commodity hardware. Hadoop, that is based on the architectural framework MapReduce, collects both structured data and unstructured data, processes the collected data set in a distributed network cluster in parallel, and extracts valuable information from the processed data set within a short time. Followings illustrate some examples of patents that provide current status of the IoT big data analytics technology development.
Big data refers to large and diverse data sets that are analyzed to reveal insights. This document discusses how big data is important for the mobile market. It provides examples of how mobile data is gathered from loyalty programs, social media activity, emails, and sensors in phones. Big data helps mobile applications meet customer expectations, analyze user experience, access real-time data, increase revenue, and allow for personalization. Industries that commonly use big data include retail, healthcare, and banking. The growing mobile market relies on big data to develop engaging apps tailored to user needs.
Big data refers to large and diverse data sets that are analyzed to reveal insights. This document discusses how big data is important for the mobile market. It is gathered from various sources like loyalty programs, social media activity, emails, and sensors in phones. Big data helps mobile applications meet customer expectations, analyze user experience, access real-time data, improve revenue, and personalize services. Industries like retail, healthcare, and banking widely use big data. It will transform future mobile apps by helping businesses understand user needs better.
Big data refers to large and diverse data sets that are analyzed to reveal insights. This document discusses how big data is important for the mobile market. It is gathered from various sources like loyalty programs, social media activity, emails, and sensors in phones. Big data helps mobile applications meet customer expectations, analyze user experience, access real-time data, improve revenue, and personalize services. Industries like retail, healthcare, and banking widely use big data. It will transform future mobile apps by helping businesses understand user needs better.
This document discusses emerging mobile data platforms that integrate different data sets to provide insights into audience behaviors. It describes several platforms, including Lotame and AudienceArchitect, that combine first, second, and third party data to build robust audience profiles and enable targeted mobile marketing campaigns. As mobile data consumption increases, these platforms are gaining traction by helping marketers engage audiences across devices through personalized ads and experiences.
Two weeks ago we released an infographic showing the life of a normal person as we imagine it in 2025. The kinds of automated services illustrated in that graphic that will appear over the next decade are all enabled by the same piece of technology: the API. What APIs, or Application Programming Interfaces, do is provide programmers with a simple way of connecting their programming into the data and services of an existing program. Any websites that embed Google or Facebook logins are accessing an API provided by Google or Facebook to authenticate users' identities. Similarly, as personal data becomes stored in the cloud, companies with permission to access such data are increasingly plugged in to their consumers' changing status and needs.
Beyond just explaining how APIs are driving technological advancements across every industry, our latest Digital Lab Thought Piece provides useful tips for any business on navigating a world based on APIs - whether that means activating the APIs that are already out there, or building one of your own to create new revenue stream or make your services indispensable.
GDPR BigDataRevealed Readiness Requirements and EvaluationSteven Meister
This GDPR methodology can evaluate your GDPR readiness. For those feeling GDPR ready, you may uncover complex issues often neglected. For those that have waited, you can gain knowledge providing for a more successful GDPR outcome.
https://youtu.be/uE4Q7u0LatU https://youtu.be/R37S9mIiVAk https://youtu.be/AQf3if7DnuM
This document discusses the future of big data, including predictions such as machine learning becoming prominent and data scientists being in high demand. It outlines trends like the growth of open source technologies, in-memory computing, machine learning, predictive analytics, intelligent applications, integrating big data with security and the internet of things. Challenges mentioned include dealing with large amounts of data from IoT and high salaries for data professionals.
This document discusses the implementation of a Bring Your Own Device (BYOD) policy and program. It begins by explaining how the proliferation of mobile devices in the workplace has led to the rise of BYOD. It notes that most employees are already using their own devices for work purposes. The rest of the document outlines "The Ten Commandments of BYOD" which provide guidance on how to create a secure and productive mobile environment that supports BYOD while protecting corporate data. The ten commandments cover topics like creating a BYOD policy, identifying existing devices, simplifying enrollment, configuring devices remotely, giving users self-service options, and protecting personal information.
The document discusses the rise of Bring Your Own Device (BYOD) programs in workplaces and provides 10 commandments for effectively managing a BYOD program. It recommends that organizations first create a mobile device policy that considers what devices and apps will be allowed and how corporate and personal data will be separated and secured. It also stresses the importance of allowing simple, self-service enrollment and configuration of devices to reduce burden on IT staff and encourage user adoption. Continuous monitoring of devices is advised to ensure compliance with security policies and allow for automated responses to issues.
Similar to Introduction to the HAT Microserver (20)
Open Source Contributions to Postgres: The Basics POSETTE 2024ElizabethGarrettChri
Postgres is the most advanced open-source database in the world and it's supported by a community, not a single company. So how does this work? How does code actually get into Postgres? I recently had a patch submitted and committed and I want to share what I learned in that process. I’ll give you an overview of Postgres versions and how the underlying project codebase functions. I’ll also show you the process for submitting a patch and getting that tested and committed.
"Financial Odyssey: Navigating Past Performance Through Diverse Analytical Lens"sameer shah
Embark on a captivating financial journey with 'Financial Odyssey,' our hackathon project. Delve deep into the past performance of two companies as we employ an array of financial statement analysis techniques. From ratio analysis to trend analysis, uncover insights crucial for informed decision-making in the dynamic world of finance."
Predictably Improve Your B2B Tech Company's Performance by Leveraging DataKiwi Creative
Harness the power of AI-backed reports, benchmarking and data analysis to predict trends and detect anomalies in your marketing efforts.
Peter Caputa, CEO at Databox, reveals how you can discover the strategies and tools to increase your growth rate (and margins!).
From metrics to track to data habits to pick up, enhance your reporting for powerful insights to improve your B2B tech company's marketing.
- - -
This is the webinar recording from the June 2024 HubSpot User Group (HUG) for B2B Technology USA.
Watch the video recording at https://youtu.be/5vjwGfPN9lw
Sign up for future HUG events at https://events.hubspot.com/b2b-technology-usa/
End-to-end pipeline agility - Berlin Buzzwords 2024Lars Albertsson
We describe how we achieve high change agility in data engineering by eliminating the fear of breaking downstream data pipelines through end-to-end pipeline testing, and by using schema metaprogramming to safely eliminate boilerplate involved in changes that affect whole pipelines.
A quick poll on agility in changing pipelines from end to end indicated a huge span in capabilities. For the question "How long time does it take for all downstream pipelines to be adapted to an upstream change," the median response was 6 months, but some respondents could do it in less than a day. When quantitative data engineering differences between the best and worst are measured, the span is often 100x-1000x, sometimes even more.
A long time ago, we suffered at Spotify from fear of changing pipelines due to not knowing what the impact might be downstream. We made plans for a technical solution to test pipelines end-to-end to mitigate that fear, but the effort failed for cultural reasons. We eventually solved this challenge, but in a different context. In this presentation we will describe how we test full pipelines effectively by manipulating workflow orchestration, which enables us to make changes in pipelines without fear of breaking downstream.
Making schema changes that affect many jobs also involves a lot of toil and boilerplate. Using schema-on-read mitigates some of it, but has drawbacks since it makes it more difficult to detect errors early. We will describe how we have rejected this tradeoff by applying schema metaprogramming, eliminating boilerplate but keeping the protection of static typing, thereby further improving agility to quickly modify data pipelines without fear.
Build applications with generative AI on Google CloudMárton Kodok
We will explore Vertex AI - Model Garden powered experiences, we are going to learn more about the integration of these generative AI APIs. We are going to see in action what the Gemini family of generative models are for developers to build and deploy AI-driven applications. Vertex AI includes a suite of foundation models, these are referred to as the PaLM and Gemini family of generative ai models, and they come in different versions. We are going to cover how to use via API to: - execute prompts in text and chat - cover multimodal use cases with image prompts. - finetune and distill to improve knowledge domains - run function calls with foundation models to optimize them for specific tasks. At the end of the session, developers will understand how to innovate with generative AI and develop apps using the generative ai industry trends.
Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You...Aggregage
This webinar will explore cutting-edge, less familiar but powerful experimentation methodologies which address well-known limitations of standard A/B Testing. Designed for data and product leaders, this session aims to inspire the embrace of innovative approaches and provide insights into the frontiers of experimentation!
Analysis insight about a Flyball dog competition team's performanceroli9797
Insight of my analysis about a Flyball dog competition team's last year performance. Find more: https://github.com/rolandnagy-ds/flyball_race_analysis/tree/main
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...sameer shah
"Join us for STATATHON, a dynamic 2-day event dedicated to exploring statistical knowledge and its real-world applications. From theory to practice, participants engage in intensive learning sessions, workshops, and challenges, fostering a deeper understanding of statistical methodologies and their significance in various fields."
Codeless Generative AI Pipelines
(GenAI with Milvus)
https://ml.dssconf.pl/user.html#!/lecture/DSSML24-041a/rate
Discover the potential of real-time streaming in the context of GenAI as we delve into the intricacies of Apache NiFi and its capabilities. Learn how this tool can significantly simplify the data engineering workflow for GenAI applications, allowing you to focus on the creative aspects rather than the technical complexities. I will guide you through practical examples and use cases, showing the impact of automation on prompt building. From data ingestion to transformation and delivery, witness how Apache NiFi streamlines the entire pipeline, ensuring a smooth and hassle-free experience.
Timothy Spann
https://www.youtube.com/@FLaNK-Stack
https://medium.com/@tspann
https://www.datainmotion.dev/
milvus, unstructured data, vector database, zilliz, cloud, vectors, python, deep learning, generative ai, genai, nifi, kafka, flink, streaming, iot, edge
2. The HAT
microserver is an
individual person’s
own database
wrapped with
microservices
Your Data
3. These microservices
are little pieces of
computer code that the
person controls. The
code give permissions
and instructions for
giving data and
accepting new data
Your Data
GET DATA
GIVE DATA
WRITE DATA
People who have a
HAT are HAT
owners. They own
the rights to the data
in the HAT
microserver
database and control
the microservices
4. A HAT owner can execute the
code with a touch of a button
on the HAT app, a special type
of app that allows the HAT
owner to operate the
microservices in the HAT
through “APIs”
Your HAT App Your Data
WRITE DATA
GIVE DATA
GET DATA
5. HAT owners can bring the data
they have on the Internet into
their HATs with Data Plugs. Plugs
are third party services that help
HAT owners grab a copy of their
data from other companies and
push it into their HATs. Data plugs
do not ever see or hold HAT data.
Your HAT App Your Data
DATA PLUGS
G
ET
DATA
Plugs
Social,
Calendar,
Payments,
Photos, Health
data
Data comes into HATs
unencumbered, as subject
access request. The data in
HAT is legally no longer
source (e.g. Facebook) data,
but HAT data.
6. Your HAT App
Third-party
App
Third-party
App
Your Data
Since it is the individual that
control their HAT, it means the
HAT microserver can act like a
universal user account, or a
private data account for apps, but
with the individual in control. Data
is given from the HAT through a
data debit.
GIVE DATA
Apps
Data Debits
7. Third-party
App
Third-party
App
Third-party
App
Previously, individuals interact with apps directly
and give their data away each time. Although
they get a good service in return, they do not get
to keep their data and so they cannot re-use or
re-share with other apps.
Instead, the apps acquire the data from others and
share amongst themselves, with their consent
The acquiring model
Every app acquiring
personal data
8. With a HAT micro server, HAT
owners get to keep the data
and can re-use and re-share
the data with other apps
Third-party
App
Third-party
App
Third-party
App
Since the HAT is API-based, corporations can build in the
ability for the individual to operate the microservices in the
HAT within their own app to inquire data, with permission, as
and when needed, without having to store the data
The inquiring model
Every app inquiring
personal data
9. Businesses like this because they can ask the individual, their
own customers, for more data instead of asking third parties. All
they have to do is give their customers a HAT
Third-party
App
Third-party
App
Third-party
App
On the HAT, the individual is the data controller and the data processor. There is no third party
involved in the relationship between the app and the user. The HAT is an infrastructure for
apps and websites, to have a direct relationship with their users
10. The Smart HAT Engine
can take on new AI
tools like algorithms,
analytics and bots into
the HAT. These tools
have been pre-trained,
and do not take any
data out of the HAT.
WRITE DATA
Algorithms / Analytic Tools
Tools
The HAT
Microserver is
also equipped
with a Smart HAT
Engine
11. These tools, once installed into the Smart HAT engine,
analyses HAT data and create insights and new data that
only goes back into the HAT.
Tools make the HAT smarter
Your Data
12. The HAT owner can share the new data with
other apps the normal way like all other
HAT data (through a data debit)
Your Data
Third-party
App
Third-party
App
Third-party
App
GIVE DATA
WRITE DATA
Data Debits
13. Your Data
The HAT owner can also view the new data generated by SHE in their
HAT app to give them more insights into their digital life, health and
activities
WRITE DATA
14. HATs are disruptive because companies (and everyone, really) need machine learning, predictive
analytics, and personalisation (i.e. AI) if they want to compete.
Third-party
App
Third-party
App
Third-party
App
Ai Corporations
The traditional AI model takes app data some place else. It is analysed and the intelligence it creates stay with
the company that analysed it. E.g. Alexa or Watson or Google. So if anyone else wants that intelligence, they
would have to go and ask these companies or create it themselves.
Traditional Model
15. With HATs, instead of getting AI from large tech companies, they can get it from their own
customers, if they give them a HAT.
Third-party
App
Third-party
App
Third-party
App
Tools
Data Scientists that install their tools into HATs get rewarded with
royalties when an app uses the data or insight generated
Apps
16. Algorithms / Analytics
Third-party
App
Third-party
App
GET DATA
GIVE DATA
WRITE DATA
Tools
Plugs
Apps
Over time, more plugs
and apps give more data
that will fuel more tools
that would will create
more data that will fuel
more apps!
17. Algorithms / Analytics
Third-party
App
Third-party
App
GET DATA
GIVE DATA
WRITE DATA
Tools
Plugs
Apps
The HAT becomes more intelligent, the HAT
data become more precious, and the individual
become more valuable!
A WIN-WIN SOLUTION FOR ALL - THAT’S HOW WE
CHANGE THE INTERNET
Apps from
organisations
get better too,
without being
creepy
Our best data
scientists get rewards!
Check out our
video!
https://youtu.be/aZBLTxS-RQc
18. If you are an individual: try the HAT app and see the TOOLS, APPS and PLUGS
and the Feed of your Digital Life
https://hatdex.org/hatstore
19. If you are a website or
application : you can
give your customers a
HAT and ask for more
data to personalise
your website with a
single sign on, and
without the risk of
holding personal
data. Find out more at
https://HATDeX.org
20. HATDeX is the commercial organisation that built
services around the open sourced HAT so that
organisations can issue HATs to their customers and
the data that individuals own can be moved around
and ‘spent’ on personalisation, recommendations
and insights.
HAT Microservers commercial deployment is on
HATDeX platform infrastructure
21. The HAT Community
Foundation governs the
whole ecosystem - the
certification of tools,
apps and plugs, legal
terms and conditions
and regulating good
behaviour on the
technology platform
The HAT Accelerator catalyses new apps,
tools and plugs through entrepreneurship
and partnership programmes
HATLAB is the innovation space and the centre for
research, education and policy for HATs and the
personal data economy
HATDeX
platform
infrastructure
https://hat-lab.org
https://hataccelerator.org
https://HATDeX.org
https://hatcommunity.org
22. 2013 Q2
HAT
£1.1M
• University of Warwick
• University of Cambridge
• University of Surrey
• University of Edinburgh
• University of W. England
• University of Nottingham
2014 Q3
HARRIET
£486k
• Warwick
HAT Living Labs
£1.2M
• Warwick
• Surrey
• Cambridge
• UWE
2016 Q2
COMEHERE
£420k
• Surrey
2017 Q3
ACCEPT
£1.08M
• Kent
• Surrey
• Warwick
2017 Q4
DROPS
£1.25M
• Warwick
• Surrey
• UWE
2018 Q2
FOOD
£1.1M
• Surrey
ASNNet+
£1.08M
• Surrey
DEAS
£1.2M
• Surrey
2018 Q3
HATDeX pre-seed
£300k
HATDeX community
£120k
HATDeX
seed
£150k
HAT Accelerator
Formalised
PriVelt
£1.2M
• Kent
HAT Community
Foundation &
HATDeX formed
£20k (founders)
HATLAB
formalised
The HAT was first created from an RCUK grant in 2013 with Professor Irene Ng as the lead. Since then, more than £10m grant
funding have been awarded to continue the research, as well as private investments into HATDeX for the commercial operation