2024 State of Marketing Report – by HubspotMarius Sescu
https://www.hubspot.com/state-of-marketing
· Scaling relationships and proving ROI
· Social media is the place for search, sales, and service
· Authentic influencer partnerships fuel brand growth
· The strongest connections happen via call, click, chat, and camera.
· Time saved with AI leads to more creative work
· Seeking: A single source of truth
· TLDR; Get on social, try AI, and align your systems.
· More human marketing, powered by robots
ChatGPT is a revolutionary addition to the world since its introduction in 2022. A big shift in the sector of information gathering and processing happened because of this chatbot. What is the story of ChatGPT? How is the bot responding to prompts and generating contents? Swipe through these slides prepared by Expeed Software, a web development company regarding the development and technical intricacies of ChatGPT!
Product Design Trends in 2024 | Teenage EngineeringsPixeldarts
The realm of product design is a constantly changing environment where technology and style intersect. Every year introduces fresh challenges and exciting trends that mold the future of this captivating art form. In this piece, we delve into the significant trends set to influence the look and functionality of product design in the year 2024.
How Race, Age and Gender Shape Attitudes Towards Mental HealthThinkNow
Mental health has been in the news quite a bit lately. Dozens of U.S. states are currently suing Meta for contributing to the youth mental health crisis by inserting addictive features into their products, while the U.S. Surgeon General is touring the nation to bring awareness to the growing epidemic of loneliness and isolation. The country has endured periods of low national morale, such as in the 1970s when high inflation and the energy crisis worsened public sentiment following the Vietnam War. The current mood, however, feels different. Gallup recently reported that national mental health is at an all-time low, with few bright spots to lift spirits.
To better understand how Americans are feeling and their attitudes towards mental health in general, ThinkNow conducted a nationally representative quantitative survey of 1,500 respondents and found some interesting differences among ethnic, age and gender groups.
Technology
For example, 52% agree that technology and social media have a negative impact on mental health, but when broken out by race, 61% of Whites felt technology had a negative effect, and only 48% of Hispanics thought it did.
While technology has helped us keep in touch with friends and family in faraway places, it appears to have degraded our ability to connect in person. Staying connected online is a double-edged sword since the same news feed that brings us pictures of the grandkids and fluffy kittens also feeds us news about the wars in Israel and Ukraine, the dysfunction in Washington, the latest mass shooting and the climate crisis.
Hispanics may have a built-in defense against the isolation technology breeds, owing to their large, multigenerational households, strong social support systems, and tendency to use social media to stay connected with relatives abroad.
Age and Gender
When asked how individuals rate their mental health, men rate it higher than women by 11 percentage points, and Baby Boomers rank it highest at 83%, saying it’s good or excellent vs. 57% of Gen Z saying the same.
Gen Z spends the most amount of time on social media, so the notion that social media negatively affects mental health appears to be correlated. Unfortunately, Gen Z is also the generation that’s least comfortable discussing mental health concerns with healthcare professionals. Only 40% of them state they’re comfortable discussing their issues with a professional compared to 60% of Millennials and 65% of Boomers.
Race Affects Attitudes
As seen in previous research conducted by ThinkNow, Asian Americans lag other groups when it comes to awareness of mental health issues. Twenty-four percent of Asian Americans believe that having a mental health issue is a sign of weakness compared to the 16% average for all groups. Asians are also considerably less likely to be aware of mental health services in their communities (42% vs. 55%) and most likely to seek out information on social media (51% vs. 35%).
AI Trends in Creative Operations 2024 by Artwork Flow.pdfmarketingartwork
This article is all about what AI trends will emerge in the field of creative operations in 2024. All the marketers and brand builders should be aware of these trends for their further use and save themselves some time!
A report by thenetworkone and Kurio.
The contributing experts and agencies are (in an alphabetical order): Sylwia Rytel, Social Media Supervisor, 180heartbeats + JUNG v MATT (PL), Sharlene Jenner, Vice President - Director of Engagement Strategy, Abelson Taylor (USA), Alex Casanovas, Digital Director, Atrevia (ES), Dora Beilin, Senior Social Strategist, Barrett Hoffher (USA), Min Seo, Campaign Director, Brand New Agency (KR), Deshé M. Gully, Associate Strategist, Day One Agency (USA), Francesca Trevisan, Strategist, Different (IT), Trevor Crossman, CX and Digital Transformation Director; Olivia Hussey, Strategic Planner; Simi Srinarula, Social Media Manager, The Hallway (AUS), James Hebbert, Managing Director, Hylink (CN / UK), Mundy Álvarez, Planning Director; Pedro Rojas, Social Media Manager; Pancho González, CCO, Inbrax (CH), Oana Oprea, Head of Digital Planning, Jam Session Agency (RO), Amy Bottrill, Social Account Director, Launch (UK), Gaby Arriaga, Founder, Leonardo1452 (MX), Shantesh S Row, Creative Director, Liwa (UAE), Rajesh Mehta, Chief Strategy Officer; Dhruv Gaur, Digital Planning Lead; Leonie Mergulhao, Account Supervisor - Social Media & PR, Medulla (IN), Aurelija Plioplytė, Head of Digital & Social, Not Perfect (LI), Daiana Khaidargaliyeva, Account Manager, Osaka Labs (UK / USA), Stefanie Söhnchen, Vice President Digital, PIABO Communications (DE), Elisabeth Winiartati, Managing Consultant, Head of Global Integrated Communications; Lydia Aprina, Account Manager, Integrated Marketing and Communications; Nita Prabowo, Account Manager, Integrated Marketing and Communications; Okhi, Web Developer, PNTR Group (ID), Kei Obusan, Insights Director; Daffi Ranandi, Insights Manager, Radarr (SG), Gautam Reghunath, Co-founder & CEO, Talented (IN), Donagh Humphreys, Head of Social and Digital Innovation, THINKHOUSE (IRE), Sarah Yim, Strategy Director, Zulu Alpha Kilo (CA).
Trends In Paid Search: Navigating The Digital Landscape In 2024Search Engine Journal
The search marketing landscape is evolving rapidly with new technologies, and professionals, like you, rely on innovative paid search strategies to meet changing demands.
It’s important that you’re ready to implement new strategies in 2024.
Check this out and learn the top trends in paid search advertising that are expected to gain traction, so you can drive higher ROI more efficiently in 2024.
You’ll learn:
- The latest trends in AI and automation, and what this means for an evolving paid search ecosystem.
- New developments in privacy and data regulation.
- Emerging ad formats that are expected to make an impact next year.
Watch Sreekant Lanka from iQuanti and Irina Klein from OneMain Financial as they dive into the future of paid search and explore the trends, strategies, and technologies that will shape the search marketing landscape.
If you’re looking to assess your paid search strategy and design an industry-aligned plan for 2024, then this webinar is for you.
5 Public speaking tips from TED - Visualized summarySpeakerHub
From their humble beginnings in 1984, TED has grown into the world’s most powerful amplifier for speakers and thought-leaders to share their ideas. They have over 2,400 filmed talks (not including the 30,000+ TEDx videos) freely available online, and have hosted over 17,500 events around the world.
With over one billion views in a year, it’s no wonder that so many speakers are looking to TED for ideas on how to share their message more effectively.
The article “5 Public-Speaking Tips TED Gives Its Speakers”, by Carmine Gallo for Forbes, gives speakers five practical ways to connect with their audience, and effectively share their ideas on stage.
Whether you are gearing up to get on a TED stage yourself, or just want to master the skills that so many of their speakers possess, these tips and quotes from Chris Anderson, the TED Talks Curator, will encourage you to make the most impactful impression on your audience.
See the full article and more summaries like this on SpeakerHub here: https://speakerhub.com/blog/5-presentation-tips-ted-gives-its-speakers
See the original article on Forbes here:
http://www.forbes.com/forbes/welcome/?toURL=http://www.forbes.com/sites/carminegallo/2016/05/06/5-public-speaking-tips-ted-gives-its-speakers/&refURL=&referrer=#5c07a8221d9b
ChatGPT and the Future of Work - Clark Boyd Clark Boyd
Everyone is in agreement that ChatGPT (and other generative AI tools) will shape the future of work. Yet there is little consensus on exactly how, when, and to what extent this technology will change our world.
Businesses that extract maximum value from ChatGPT will use it as a collaborative tool for everything from brainstorming to technical maintenance.
For individuals, now is the time to pinpoint the skills the future professional will need to thrive in the AI age.
Check out this presentation to understand what ChatGPT is, how it will shape the future of work, and how you can prepare to take advantage.
A brief introduction to DataScience with explaining of the concepts, algorithms, machine learning, supervised and unsupervised learning, clustering, statistics, data preprocessing, real-world applications etc.
It's part of a Data Science Corner Campaign where I will be discussing the fundamentals of DataScience, AIML, Statistics etc.
Time Management & Productivity - Best PracticesVit Horky
Here's my presentation on by proven best practices how to manage your work time effectively and how to improve your productivity. It includes practical tips and how to use tools such as Slack, Google Apps, Hubspot, Google Calendar, Gmail and others.
The six step guide to practical project managementMindGenius
The six step guide to practical project management
If you think managing projects is too difficult, think again.
We’ve stripped back project management processes to the
basics – to make it quicker and easier, without sacrificing
the vital ingredients for success.
“If you’re looking for some real-world guidance, then The Six Step Guide to Practical Project Management will help.”
Dr Andrew Makar, Tactical Project Management
2024 State of Marketing Report – by HubspotMarius Sescu
https://www.hubspot.com/state-of-marketing
· Scaling relationships and proving ROI
· Social media is the place for search, sales, and service
· Authentic influencer partnerships fuel brand growth
· The strongest connections happen via call, click, chat, and camera.
· Time saved with AI leads to more creative work
· Seeking: A single source of truth
· TLDR; Get on social, try AI, and align your systems.
· More human marketing, powered by robots
ChatGPT is a revolutionary addition to the world since its introduction in 2022. A big shift in the sector of information gathering and processing happened because of this chatbot. What is the story of ChatGPT? How is the bot responding to prompts and generating contents? Swipe through these slides prepared by Expeed Software, a web development company regarding the development and technical intricacies of ChatGPT!
Product Design Trends in 2024 | Teenage EngineeringsPixeldarts
The realm of product design is a constantly changing environment where technology and style intersect. Every year introduces fresh challenges and exciting trends that mold the future of this captivating art form. In this piece, we delve into the significant trends set to influence the look and functionality of product design in the year 2024.
How Race, Age and Gender Shape Attitudes Towards Mental HealthThinkNow
Mental health has been in the news quite a bit lately. Dozens of U.S. states are currently suing Meta for contributing to the youth mental health crisis by inserting addictive features into their products, while the U.S. Surgeon General is touring the nation to bring awareness to the growing epidemic of loneliness and isolation. The country has endured periods of low national morale, such as in the 1970s when high inflation and the energy crisis worsened public sentiment following the Vietnam War. The current mood, however, feels different. Gallup recently reported that national mental health is at an all-time low, with few bright spots to lift spirits.
To better understand how Americans are feeling and their attitudes towards mental health in general, ThinkNow conducted a nationally representative quantitative survey of 1,500 respondents and found some interesting differences among ethnic, age and gender groups.
Technology
For example, 52% agree that technology and social media have a negative impact on mental health, but when broken out by race, 61% of Whites felt technology had a negative effect, and only 48% of Hispanics thought it did.
While technology has helped us keep in touch with friends and family in faraway places, it appears to have degraded our ability to connect in person. Staying connected online is a double-edged sword since the same news feed that brings us pictures of the grandkids and fluffy kittens also feeds us news about the wars in Israel and Ukraine, the dysfunction in Washington, the latest mass shooting and the climate crisis.
Hispanics may have a built-in defense against the isolation technology breeds, owing to their large, multigenerational households, strong social support systems, and tendency to use social media to stay connected with relatives abroad.
Age and Gender
When asked how individuals rate their mental health, men rate it higher than women by 11 percentage points, and Baby Boomers rank it highest at 83%, saying it’s good or excellent vs. 57% of Gen Z saying the same.
Gen Z spends the most amount of time on social media, so the notion that social media negatively affects mental health appears to be correlated. Unfortunately, Gen Z is also the generation that’s least comfortable discussing mental health concerns with healthcare professionals. Only 40% of them state they’re comfortable discussing their issues with a professional compared to 60% of Millennials and 65% of Boomers.
Race Affects Attitudes
As seen in previous research conducted by ThinkNow, Asian Americans lag other groups when it comes to awareness of mental health issues. Twenty-four percent of Asian Americans believe that having a mental health issue is a sign of weakness compared to the 16% average for all groups. Asians are also considerably less likely to be aware of mental health services in their communities (42% vs. 55%) and most likely to seek out information on social media (51% vs. 35%).
AI Trends in Creative Operations 2024 by Artwork Flow.pdfmarketingartwork
This article is all about what AI trends will emerge in the field of creative operations in 2024. All the marketers and brand builders should be aware of these trends for their further use and save themselves some time!
A report by thenetworkone and Kurio.
The contributing experts and agencies are (in an alphabetical order): Sylwia Rytel, Social Media Supervisor, 180heartbeats + JUNG v MATT (PL), Sharlene Jenner, Vice President - Director of Engagement Strategy, Abelson Taylor (USA), Alex Casanovas, Digital Director, Atrevia (ES), Dora Beilin, Senior Social Strategist, Barrett Hoffher (USA), Min Seo, Campaign Director, Brand New Agency (KR), Deshé M. Gully, Associate Strategist, Day One Agency (USA), Francesca Trevisan, Strategist, Different (IT), Trevor Crossman, CX and Digital Transformation Director; Olivia Hussey, Strategic Planner; Simi Srinarula, Social Media Manager, The Hallway (AUS), James Hebbert, Managing Director, Hylink (CN / UK), Mundy Álvarez, Planning Director; Pedro Rojas, Social Media Manager; Pancho González, CCO, Inbrax (CH), Oana Oprea, Head of Digital Planning, Jam Session Agency (RO), Amy Bottrill, Social Account Director, Launch (UK), Gaby Arriaga, Founder, Leonardo1452 (MX), Shantesh S Row, Creative Director, Liwa (UAE), Rajesh Mehta, Chief Strategy Officer; Dhruv Gaur, Digital Planning Lead; Leonie Mergulhao, Account Supervisor - Social Media & PR, Medulla (IN), Aurelija Plioplytė, Head of Digital & Social, Not Perfect (LI), Daiana Khaidargaliyeva, Account Manager, Osaka Labs (UK / USA), Stefanie Söhnchen, Vice President Digital, PIABO Communications (DE), Elisabeth Winiartati, Managing Consultant, Head of Global Integrated Communications; Lydia Aprina, Account Manager, Integrated Marketing and Communications; Nita Prabowo, Account Manager, Integrated Marketing and Communications; Okhi, Web Developer, PNTR Group (ID), Kei Obusan, Insights Director; Daffi Ranandi, Insights Manager, Radarr (SG), Gautam Reghunath, Co-founder & CEO, Talented (IN), Donagh Humphreys, Head of Social and Digital Innovation, THINKHOUSE (IRE), Sarah Yim, Strategy Director, Zulu Alpha Kilo (CA).
Trends In Paid Search: Navigating The Digital Landscape In 2024Search Engine Journal
The search marketing landscape is evolving rapidly with new technologies, and professionals, like you, rely on innovative paid search strategies to meet changing demands.
It’s important that you’re ready to implement new strategies in 2024.
Check this out and learn the top trends in paid search advertising that are expected to gain traction, so you can drive higher ROI more efficiently in 2024.
You’ll learn:
- The latest trends in AI and automation, and what this means for an evolving paid search ecosystem.
- New developments in privacy and data regulation.
- Emerging ad formats that are expected to make an impact next year.
Watch Sreekant Lanka from iQuanti and Irina Klein from OneMain Financial as they dive into the future of paid search and explore the trends, strategies, and technologies that will shape the search marketing landscape.
If you’re looking to assess your paid search strategy and design an industry-aligned plan for 2024, then this webinar is for you.
5 Public speaking tips from TED - Visualized summarySpeakerHub
From their humble beginnings in 1984, TED has grown into the world’s most powerful amplifier for speakers and thought-leaders to share their ideas. They have over 2,400 filmed talks (not including the 30,000+ TEDx videos) freely available online, and have hosted over 17,500 events around the world.
With over one billion views in a year, it’s no wonder that so many speakers are looking to TED for ideas on how to share their message more effectively.
The article “5 Public-Speaking Tips TED Gives Its Speakers”, by Carmine Gallo for Forbes, gives speakers five practical ways to connect with their audience, and effectively share their ideas on stage.
Whether you are gearing up to get on a TED stage yourself, or just want to master the skills that so many of their speakers possess, these tips and quotes from Chris Anderson, the TED Talks Curator, will encourage you to make the most impactful impression on your audience.
See the full article and more summaries like this on SpeakerHub here: https://speakerhub.com/blog/5-presentation-tips-ted-gives-its-speakers
See the original article on Forbes here:
http://www.forbes.com/forbes/welcome/?toURL=http://www.forbes.com/sites/carminegallo/2016/05/06/5-public-speaking-tips-ted-gives-its-speakers/&refURL=&referrer=#5c07a8221d9b
ChatGPT and the Future of Work - Clark Boyd Clark Boyd
Everyone is in agreement that ChatGPT (and other generative AI tools) will shape the future of work. Yet there is little consensus on exactly how, when, and to what extent this technology will change our world.
Businesses that extract maximum value from ChatGPT will use it as a collaborative tool for everything from brainstorming to technical maintenance.
For individuals, now is the time to pinpoint the skills the future professional will need to thrive in the AI age.
Check out this presentation to understand what ChatGPT is, how it will shape the future of work, and how you can prepare to take advantage.
A brief introduction to DataScience with explaining of the concepts, algorithms, machine learning, supervised and unsupervised learning, clustering, statistics, data preprocessing, real-world applications etc.
It's part of a Data Science Corner Campaign where I will be discussing the fundamentals of DataScience, AIML, Statistics etc.
Time Management & Productivity - Best PracticesVit Horky
Here's my presentation on by proven best practices how to manage your work time effectively and how to improve your productivity. It includes practical tips and how to use tools such as Slack, Google Apps, Hubspot, Google Calendar, Gmail and others.
The six step guide to practical project managementMindGenius
The six step guide to practical project management
If you think managing projects is too difficult, think again.
We’ve stripped back project management processes to the
basics – to make it quicker and easier, without sacrificing
the vital ingredients for success.
“If you’re looking for some real-world guidance, then The Six Step Guide to Practical Project Management will help.”
Dr Andrew Makar, Tactical Project Management
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Tutorial on Bootstrap resampling with independent data
1. empirical bayesian@ieee.org
Bootstrap resampling with independent data
Some theory and applications
Jan Galkowski
21st November 2012
(last rev November 21, 2012)
J. T. Galkowski Bootstrapping Independent Data 1 / 44
2. empirical bayesian@ieee.org
Why do we care about Uncertainty Qualification (UQ)?
Inherent variability of mechanisms, measurements
Statistics are themselves random variables, e.g., sample mean, sample standard
deviation, sample median
For simple distributions, knowing the first few moments suffices to characterize
them
For complicated or mixture distributions . . . ?
We often pretend distributions of measurements are stationary. Are they?
Diagnostics! What to do if aren’t?
Will have more to say about latter in next lecture on bootstrapping with
dependent data
J. T. Galkowski Bootstrapping Independent Data 2 / 44
4. empirical bayesian@ieee.org
Sketch of Why the Bootstrap Works
400
300 Growth in Number of Possible Bootstrap Resamplings with Sample Size
log10(N)
200
100
0
0 50 100 150 200
size of sample, N
J. T. Galkowski Bootstrapping Independent Data 4 / 44
5. empirical bayesian@ieee.org
Test Data Model, Easy Case
0.12
0.10
0.08
0.06
0.04
0.02
0.00 Synthetic Gaussians for Easy Test Case
10 20 30 40 50
x
J. T. Galkowski Bootstrapping Independent Data 5 / 44
6. empirical bayesian@ieee.org
Boxplot of well-separated Gaussians
30
25 Simple case of two well separated Gaussians
sample value
20
15
10
Left Right
Contributing Distribution
J. T. Galkowski Bootstrapping Independent Data 6 / 44
7. empirical bayesian@ieee.org
Histogram of same well-separated Gaussians
0.25 Left Sample of Two Well−separated Gaussians Right Sample of Two Well−separated Gaussians
0.25
0.20
0.20
0.15
0.15
0.10
0.10
0.05
0.05
0.00
0.00
10 15 20 25 30 35 40 10 15 20 25 30 35 40
Sample of Left Well−Separated Gaussian Sample of Right Gaussian
J. T. Galkowski Bootstrapping Independent Data 7 / 44
8. empirical bayesian@ieee.org
Standard t-test on well-separated Gaussians
values values
Group n average s.d.
1: left Gaussians 24 19.540998 4.867407
2: right Gaussians 35 28.471698 2.820691
3: POOLED 59 24.838871 3.7822811
¯ ¯
(Y2 − Y1 ) − 0.0 = 8.930700 ¯ ¯
SE(Y2 − Y1 ) = 1.002398
8.930700−0.0
two-sided t-statistic = 1.002398
= 8.909336
P = 0.0000 t-statistic = 8.90934 = t57 (0.0000)
1
Adjusted for d.o.f.
J. T. Galkowski Bootstrapping Independent Data 8 / 44
9. empirical bayesian@ieee.org
Bootstrap characterization of well-separated Gaussians
Two well−separated Gaussians, differences
after 10,000 bootstrap replications
0.5
z t test
z t test − se t test
z t test + se t test
z bootstrap
z bootstrap − se bootstrap
z bootstrap + se bootstrap
0.4
0.3
0.2
0.1
0.0
6 8 10 12
difference between Right Gaussians and Left Gaussians
J. T. Galkowski Bootstrapping Independent Data 9 / 44
10. empirical bayesian@ieee.org
Bootstrap test of well-separated Gaussians
values values
Group n average s.d.
1: resampled Left 24 19.540998 4.867407
2: resampled Right 35 28.471698 2.820691
3: POOLED 59 24.838871 3.7822812
¯ ¯
Recall original (Y2 − Y1 ) − 0.0 = 8.930700
baseline two-sided t statistic P = 0.0000 t-statistic = 8.90934 = t57 (0.0000)
bootstrap resampled Left two-sided t statistic P = 0.0001 0 of 10000 exceed baseline t
bootstrap resampled Right t statistic P = 0.0002 1 of 10000 exceed baseline t
2
Adjusted for d.o.f.
J. T. Galkowski Bootstrapping Independent Data 10 / 44
11. empirical bayesian@ieee.org
Test Data Model, Hard Case
0.06
0.05
0.04
0.03
0.02
0.01
0.00 Synthetic Gaussians for Hard Test Case
10 20 30 40 50
x
J. T. Galkowski Bootstrapping Independent Data 11 / 44
12. empirical bayesian@ieee.org
Boxplot of well-mixed Gaussians
40
35
30 Hard case of two well mixed Gaussians
sample value
25
20
15
10
Left Right
Contributing Distribution
J. T. Galkowski Bootstrapping Independent Data 12 / 44
13. empirical bayesian@ieee.org
Histogram of same well-mixed Gaussians
0.15
0.10
0.05
0.00 Two well−mixed Gaussians
10 15 20 25 30 35 40
Pooled Sample of Right and Left Mixtures of Gaussians
J. T. Galkowski Bootstrapping Independent Data 13 / 44
14. empirical bayesian@ieee.org
Standard t-test on mixture distribution of Gaussians
values values
Group n average s.d.
1: left Gaussians 24 22.635617 6.534154
2: right Gaussians 35 27.558842 6.140611
3: POOLED 59 25.556174 6.3023673
¯ ¯
(Y2 − Y1 ) − 0.0 = 4.923225 ¯ ¯
SE(Y2 − Y1 ) = 1.670283
4.923225−0.0
two-sided t-statistic = 1.670283
= 2.947539
P = 0.0023 t-statistic = 2.94754 = t57 (0.0023)
3
Adjusted for d.o.f.
J. T. Galkowski Bootstrapping Independent Data 14 / 44
15. empirical bayesian@ieee.org
Resampling from each population separately
Bootstrap Resampling Pseudo−Populations from Bootstrap Resampling Pseudo−Populations from
Left Case of Two well−mixed Gaussians Right Case of Two well−mixed Gaussians
0.4
0.4
0.3
0.3
0.2
0.2
0.1
0.1
0.0
0.0
15 20 25 30 35 40 15 20 25 30 35 40
Resampling of Left Well−Mixed Gaussians Resampling of Right Well−Mixed Gaussians
(20,000 resamplings) (20,000 resamplings)
J. T. Galkowski Bootstrapping Independent Data 15 / 44
16. empirical bayesian@ieee.org
What a pooled bootstrap resampling population looks like
Bootstrap Resampling Pseudo−Populations from
Pooled Case of Two well−mixed Gaussians
0.25
0.20
0.15
0.10
0.05
0.00
15 20 25 30 35 40
Pooled Resampling of Well−Mixed Gaussians
J. T. Galkowski Bootstrapping Independent Data 16 / 44
17. empirical bayesian@ieee.org
Bootstrap test of well-mixed Gaussians
values values
Group n average s.d.
1: resampled Left 24 22.635617 6.534154
2: resampled Right 35 27.558842 6.140611
3: POOLED 59 25.556174 6.3023674
¯ ¯
Recall original (Y2 − Y1 ) − 0.0 = 4.923225
baseline two-sided t statistic P = 0.0023 t-statistic = 2.94754 = t57 (0.0023)
bootstrap resampled Left two-sided t statistic P = 0.1328 2656 of 20000 exceed baseline t
bootstrap resampled Right t statistic P = 0.1331 2662 of 20000 exceed baseline t
4
Adjusted for d.o.f.
J. T. Galkowski Bootstrapping Independent Data 17 / 44
18. empirical bayesian@ieee.org
Resampled t-values
Bootstrap resampled t−values and Baseline from Bootstrap resampled t−values and Baseline from
Left Sample of Well−mixed Gaussians Right Sample of Well−mixed Gaussians
0.4
baseline t−value = 2.9475
0.4
baseline t−value = 2.9475
0.3
0.3
0.2
0.2
0.1
0.1
0.0
0.0
0 1 2 3 4 5 6 0 1 2 3 4 5
t t
J. T. Galkowski Bootstrapping Independent Data 18 / 44
19. empirical bayesian@ieee.org
A little Bootstrap theory: Glivenko-Cantelli
Empirical distribution function: i.i.d. X1 , X2 , . . . , Xn ∼ F then
n
def 1
Fn = ∑ IX ≤x (x )
k
n k =1
Glivenko-Cantelli Theorem: As n → ∞,
∑ |Fn − F | → 0
x
almost surely (abbreviated “a.s.”).
Given
def
Ω0 = {ω : lim Xn exists} = {ω : (lim sup Xn ) − (lim inf Xn ) = 0}
n→∞ n→∞ n→∞
if P (Ω0 ) = 1, we say that Xn converges almost surely5 .
5
See Durrett, 4th edition, 2010, pp 15-16.
J. T. Galkowski Bootstrapping Independent Data 19 / 44
20. empirical bayesian@ieee.org
Proof of Glivenko-Cantelli Theorem
After R. Durrett, 2010, 4th edition, Probability: Theory and Examples, 76-77
1 Fix x. Let Zn = IXn ≤x (x ).
2 Since Zn i.i.d. with E [Zn ] = P (Xn < x ) = F (x − ) = limy →x F (y ), strong law ⇒ Fn (x − ) = n ∑n =1 Zk → F (x − ) a.s.
1
k
3 i
For 1 ≤ i ≤ k − 1, let xi ,k = inf{y : F (y ) ≥ k }.
4 Pointwise convergence of Fn (x ), Fn (x − ) ⇒ can pick Wk so
1
n ≥ Wk ⇒ |Fn (xi ,k ) − F (xi ,k )| <
k
1
n ≥ Wk ⇒ |Fn (xi− ) − F (xi− )| <
,k ,k k
for 1 ≤ i ≤ k − 1.
5 Let x0,k = −∞ and xk ,k = ∞. Then last inequalities valid for i = 0 or i = k .
6 Given u = xi −1,k or u = xi ,k , 1 ≤ i ≤ k − 1, n ≥ Wk , monotonicity of Fn , monotonicity of F , F (xi− ) − F (xi −1,k ) ≤ k ,
,k
1
1 2 2
Fn (u ) ≤ Fn (xi− ) ≤ F (xi− ) +
,k ,k ≤ F (xi −1,k ) + ≤ F (u ) +
k k k
1 2 2
Fn (u ) ≥ Fn (xi −1,k ) ≥ F (xi −1,k ) − ≥ F (xi− ) −
,k ≥ F (u ) −
k k k
7 2
Consequently, supu |Fn (u ) − F (u )| ≤ k . Result proved.
J. T. Galkowski Bootstrapping Independent Data 20 / 44
21. empirical bayesian@ieee.org
A little Bootstrap theory: Edgeworth expansions
Edgeworth expansion:
1 ˆ
θ is constructed from sample of size n
2 θ0 is a “parameter”
√
3 n(θ − θ0 ) ∼ N (0, σ 2 )
ˆ
√ θ −θ0
ˆ √ √
4 P {( n σ ≤ x } = Φ(x ) + ( n)1 p1 (x )φ (x ) + · · · + ( n)j pj φ (x ) + . . .
5 φ (x ) = N (0, 1) and Φ(x ) = −∞ φ (u ) du
x
g (X∗ −g (X)
¯ ¯
Then letting A(X∗ ) =
ˆ ¯ ¯
h(X)
, where s∗ denotes the corresponding bootstrap
estimate of the sample statistic s,
1 Characteristic function χ of X satisfies lim sup|χ(t)| < 1
t →0
g (X∗ −g (X)
¯ ¯
2 Asymptotic variance of ¯
h(X)
=1
√ g (X∗ −g (X)
¯ ¯ √
3 Then P { n ¯
h(X)
≤ x } = Φ(x ) + ∑ν=1 ( n)j πj (x )φ (x ) + o(n−ν/2 )
j
4 πj is a polynomial of degree 3j − 1, odd for even j and even for odd j with
coefficients a function of moments of X up to order j + 2.
J. T. Galkowski Bootstrapping Independent Data 21 / 44
22. empirical bayesian@ieee.org
Bumpus 1898 data
0.78
0.76
0.74 Sparrow Survival after Severe Snowstorm (Bumpus, 1898, Woods Hole, MA)
Humerus length in inches
0.72
0.70
0.68
0.66
PERISHED SURVIVED
status of sparrow
J. T. Galkowski Bootstrapping Independent Data 22 / 44
23. empirical bayesian@ieee.org
t-test results from Bumpus 1898 data
Humerus Humerus
average s.d.
Group n (inches) (inches)
1: perished 24 0.727917 0.023543
2: survived 35 0.738000 0.019839
3: POOLED 59 0.733898 0.0214116
¯ ¯
(Y2 − Y1 ) − 0.0 = 0.010083 ¯ ¯
SE(Y2 − Y1 ) = 0.005674
0.010083−0.0
two-sided t-statistic = 0.005674
= 1.776998
P = 0.0405 t-statistic = 1.77700 = t57 (0.0405)
6
Adjusted for d.o.f.
J. T. Galkowski Bootstrapping Independent Data 23 / 44
24. empirical bayesian@ieee.org
Bootstrap results from Bumpus 1898 data
values values
Group n average s.d.
1: perished 24 0.727917 0.023543
2: survived 35 0.738000 0.019839
3: POOLED 59 0.733898 0.0214117
¯ ¯
Recall original (Y2 − Y1 ) − 0.0 = 0.010083
baseline two-sided t statistic P = 0.0405 t-statistic = 1.77700 = t57 (0.0405)
bootstrap perished two-sided t statistic P = 0.2711 2710 of 10000 exceed baseline t
bootstrap survived t statistic P = 0.2564 2563 of 10000 exceed baseline t
7
Adjusted for d.o.f.
J. T. Galkowski Bootstrapping Independent Data 24 / 44
25. empirical bayesian@ieee.org
Can go elsewhere: Stroke risk and proportions
strokes8 subjects
aspirin group 119 11,037
placebo group 98 11,034
How to bootstrap this problem . . .
1 Create an “aspirin sample” of length 11,037 having 119 ones and 11,037 - 119 zeros.
2 Create a “placebo sample” of length 11,034 having 98 ones and 11,037 - 98 zeros.
3 Draw with replacement of length 11,037 from “aspirin sample”.
4 Draw with replacement of length 11,034 from “placebo sample”.
5 Calculate proportion ones in draw from “aspirin sample” to number of ones in draw from
“placebo sample”.
6 Repeat drawing and calculation a large number of times, say, 1000.
7 Calculate the sample statistics you like from the resulting population.
8
From New York Times of 27th January 1987, as quoted by Efron and Tibshirani in An Introduction to the Bootstrap, 1998.
J. T. Galkowski Bootstrapping Independent Data 25 / 44
26. empirical bayesian@ieee.org
Bootstrap display of proportions in stroke risk
Resampled proportions for stroke risk
using 10000 bootstrap replicas
300
median risk ratio = 1.2140, p = 0.548
250
200
150
100
50
0
1.0 1.5 2.0
aspirin placebo
J. T. Galkowski Bootstrapping Independent Data 26 / 44
27. empirical bayesian@ieee.org
Can go elsewhere: In lieu of cross-validation
Cross-validation is a bag of techniques for trying to assess prediction error.
There are other “bags of techniques” as well, one will be mentioned later, but a
review needs await a lecture on predictive inference.
y + δ y = [f + δ f ](x + δ x ) −→ δ y = f(δ x ) + δ f (x + δ x )
This is gotten wrong a lot. Important to separate variation in data from variation
in model.
Will present the standard set of methods, and then offer one Bootstrap-based
alternative.
J. T. Galkowski Bootstrapping Independent Data 27 / 44
28. empirical bayesian@ieee.org
Cross-validation: A review
Classical methods
Split-half:
1 Randomly pick half data subset from sample without replacement.
2 Fit to that.
3 Test fit on half not selected.
4 Report measure like MSE for test.
K -fold: Let F = {k : k = 1, . . . , K }.
1 Divide sample into K randomly picked disjoint subsets.
2 For each j , j ∈ F , fit to all i = j , i ∈ F .
3 Then test the fit obtained on subset j.
4 Report measure like MSE by averaging over all K of the “folds”.
Leave one out cross-validation: K -fold with K = 1. (Not covered here.)
J. T. Galkowski Bootstrapping Independent Data 28 / 44
29. empirical bayesian@ieee.org
Cross-validation: A review
Bootstrap-related methods
Bootstrap validation:
1 Take B Bootstrap resamplings from the sample.
2 For each resampling, fit the model to the resampling, then predict the model on the
resampling and on the entire original sample.
3 Note the difference between performance on the entire original sample and the performance
on the resampling in each case. Each of these values are called an estimate of optimism.
4 Average the optimism values, and then add that to the average residual squared error of the
original fit to get an estimate of prediction error.
Two versions of Bootstrap-based cross validation illustrated here. The first is the “ordinary
Bootstrap” realization of validation9 , and the “0.632+ bootstrap”, a refined version
designed to replace classical cross-validation.
9
B. Efron, G. Gong, “A leisurely look at the Bootstrap, the Jackknife, and cross-validation”, The American Statistician, 37(1), 1983, 36-48,
as realized by F. E. Harrell, Jr, in the R package rms.
J. T. Galkowski Bootstrapping Independent Data 29 / 44
30. empirical bayesian@ieee.org
Data to model: Salinity as predicted by temperature, pressure, latitude, month
ARGO float, platform 1900764 (WHOI, IFREMER)
ARGO Float, Platform 1900764 (WHOI, IFREMER)
25
20
Adjusted Temperature (Celsius)
15
10
5
35.0 35.5 36.0 36.5 37.0
Salinity Adjusted (psu)
J. T. Galkowski Bootstrapping Independent Data 30 / 44
32. empirical bayesian@ieee.org
Location of collection: Equatorial Atlantic, near Africa
Canary Current dominates at surface, flows southwest
J. T. Galkowski Bootstrapping Independent Data 32 / 44
33. empirical bayesian@ieee.org
Location of collection: Near Cape Verde Islands
J. T. Galkowski Bootstrapping Independent Data 33 / 44
34. empirical bayesian@ieee.org
Models to be compared: Linear regression to predict Salinity
Two competitive models
Table: salinity = f (temperature, pressure, month, latitude, intercept)
Estimate Std. Error t value Pr(>|t|)
(Intercept) 33.58993 0.04224 795.28623 0.00000
temperature 0.11911 0.00082 144.46335 0.00000
pressure 0.00014 0.00001 11.87072 0.00000
month -0.00069 0.00061 -1.14065 0.25404
latitude 0.02783 0.00194 14.36736 0.00000
Table: salinity = f (temperature, pressure, month, latitude)
Estimate Std. Error t value Pr(>|t|)
temperature 0.35984 0.00595 60.43834 0.00000
pressure 0.00347 0.00008 40.91654 0.00000
month -0.06683 0.00466 -14.33299 0.00000
latitude 1.43767 0.00606 237.19820 0.00000
J. T. Galkowski Bootstrapping Independent Data 34 / 44
35. empirical bayesian@ieee.org
Models to be compared: Linear regression to predict Salinity
Two more competitive models
Table: salinity = f (temperature, pressure, month, intercept)
Estimate Std. Error t value Pr(>|t|)
(Intercept) 34.14529 0.01719 1986.78239 0.00000
temperature 0.11940 0.00083 143.49286 0.00000
pressure 0.00014 0.00001 12.02606 0.00000
month 0.00106 0.00060 1.76639 0.07736
Table: salinity = f (temperature, pressure, month)
Estimate Std. Error t value Pr(>|t|)
temperature 1.71963 0.00403 427.12858 0.00000
pressure 0.02196 0.00008 264.55418 0.00000
month 0.14220 0.01147 12.40126 0.00000
J. T. Galkowski Bootstrapping Independent Data 35 / 44
36. empirical bayesian@ieee.org
Models to be compared: Linear regression to predict Salinity
Still two more competitive models
Table: salinity = f (temperature, pressure, intercept)
Estimate Std. Error t value Pr(>|t|)
(Intercept) 34.14888 0.01707 2000.88558 0.00000
temperature 0.11951 0.00083 144.02016 0.00000
pressure 0.00014 0.00001 12.17462 0.00000
Table: salinity = f (temperature, pressure)
Estimate Std. Error t value Pr(>|t|)
temperature 1.75765 0.00263 668.68046 0.00000
pressure 0.02246 0.00007 308.89198 0.00000
J. T. Galkowski Bootstrapping Independent Data 36 / 44
37. empirical bayesian@ieee.org
Variation by season: Salinity as predicted by temperature, pressure, latitude, month
ARGO float, platform 1900764 (WHOI, IFREMER)
ARGO Float, Platform 1900764 (WHOI, IFREMER)
(showing dependency by month)
25
20
Adjusted Temperature (Celsius)
15
10
5
35.0 35.5 36.0 36.5 37.0
Salinity Adjusted (psu)
J. T. Galkowski Bootstrapping Independent Data 37 / 44
38. empirical bayesian@ieee.org
Split-Half cross-validation example
Also called “Hold Out cross-validation” or “Split-Sample cross-validation”
Table: Split-half cross-validation summary
MSE
temperature, pressure, month, latitude; with intercept 0.219357
temperature, pressure, month; with intercept 0.180320
temperature, pressure; with intercept 0.133647
temperature, pressure, month, latitude; without intercept 10.743651
temperature, pressure, month; without intercept 48.300449
temperature, pressure; without intercept 33.191302
J. T. Galkowski Bootstrapping Independent Data 38 / 44
39. empirical bayesian@ieee.org
K-fold Cross-validation example
Table: 10-fold cross-validation summary
MSE
temperature, pressure, month, latitude; with intercept 0.209468
temperature, pressure, month; with intercept 0.211403
temperature, pressure; with intercept 0.211403
temperature, pressure, month, latitude; without intercept 1.625414
temperature, pressure, month; without intercept 4.071300
temperature, pressure; without intercept 4.099267
J. T. Galkowski Bootstrapping Independent Data 39 / 44
40. empirical bayesian@ieee.org
Bootstrap validation instead
Table: Bootstrap Validation summary (100 resamplings, Efron & Gong Boostrap)
MSE
temperature, pressure, month, latitude; with intercept 0.043891
temperature, pressure, month; with intercept 0.044472
temperature, pressure; with intercept 0.044584
temperature, pressure, month, latitude; without intercept 0.043893
temperature, pressure, month; without intercept 0.044507
temperature, pressure; without intercept 0.044723
Table: Bootstrap Validation summary (100 resamplings, 0.632+ Boostrap)
MSE
temperature, pressure, month, latitude; with intercept 0.043733
temperature, pressure, month; with intercept 0.044760
temperature, pressure; with intercept 0.044618
temperature, pressure, month, latitude; without intercept 0.043833
temperature, pressure, month; without intercept 0.044626
temperature, pressure; without intercept 0.044672
J. T. Galkowski Bootstrapping Independent Data 40 / 44
41. empirical bayesian@ieee.org
Model Selection and Comparison: Information Criteria
Something to be taught another day ... lecture on predictive inference
Table: Information Criteria Judging Quality of Several Models
AIC AICc BIC R2
temperature, pressure, month, latitude; with intercept -3086.41 -3086.40 -3042.75 0.91
temperature, pressure, month; with intercept -2883.86 -2883.86 -2847.48 0.91
temperature, pressure; with intercept -2882.74 -2882.74 -2853.64 0.91
temperature, pressure, month, latitude; without intercept 0.00 0.00 0.00 1.00
temperature, pressure, month; without intercept 0.00 0.00 0.00 0.99
temperature, pressure; without intercept 0.00 0.00 0.00 0.99
Table: Relative Importance of Various Attributes in Several Models
temperature pressure month latitude
temperature+pressure+month+latitude+intercept model 0.5916 0.4048 0.0006 0.0030
temperature+pressure+month+intercept model 0.5935 0.4057 0.0008 0.0000
temperature+pressure+intercept model 0.5942 0.4058 0.0000 0.0000
J. T. Galkowski Bootstrapping Independent Data 41 / 44
42. empirical bayesian@ieee.org
Computation
R
SEQBoot special purpose and free
Nothing in GNU Scientific Library, unfortunately.
For non-delicate inferences, can simply write the sampling yourself.
Kleiner, Talwalkar, Sarkar, Jordan “Big data bootstrap”, 2012
J. T. Galkowski Bootstrapping Independent Data 42 / 44
43. empirical bayesian@ieee.org
When do these fail?
Fail principally when data are co-correlated or dependent
Special techniques for dependent data: Next lecture!
Fail when trying to characterize extremes of distributions like minimum,
maximum: Insufficient number of samples
Deteriorates when sampling distribution of population is not smooth, in a formal
sense
J. T. Galkowski Bootstrapping Independent Data 43 / 44
44. empirical bayesian@ieee.org
Bibliography
M. R. Chernick, Bootstrap Methods: A Guide for Practitioners and Researches, 2nd edition, 2008, John Wiley & Sons.
A. C. Davison, D. V. Hinkley, Bootstrap Methods and their Application, first published 1997, 11th printing, 2009, Cambridge University
Press.
B. Efron, R. J. Tibshirani, An Introduction to the Bootstrap, 1993, Chapman & Hall/CRC.
P. I. Good, Resampling Methods – A Practical Guide to Data Analysis, 2006.
B. Efron, The Jackknife, the Bootstrap and Other Resampling Plans, CBMS-NSF Regional Conference Series in Applied Mathematics,
1982.
P. Hall, The Bootstrap and Edgeworth Expansion, Springer-Verlag, 1992.
K. Singh, G. J. Babu, “On the asymptotic optimality of the bootstrap”, Scandanavian Journal of Statistics, 17, 1990, 1-9.
A. Kleiner, A. Talwalkar, P. Sarkar, M. I. Jordan, “The Big Data Bootstrap”, in J. Langford and J. Pineau (eds.), Proceedings of the 29th
International Conference on Machine Learning (ICML), Edinburgh, UK, 2012.
A. Kleiner, A. Talwalkar, P. Sarkar, M. I. Jordan, “A scalable bootstrap for massive data”, http://arxiv.org/abs/1112.5016, last
accessed 16th November 2012.
B. Efron, R. Tibshirani, “Improvements on cross-validation: The 0.632+ Bootstrap method”, JASA, 92(438), June 1997, 548-560.
J. T. Galkowski Bootstrapping Independent Data 44 / 44