2024 State of Marketing Report – by HubspotMarius Sescu
https://www.hubspot.com/state-of-marketing
· Scaling relationships and proving ROI
· Social media is the place for search, sales, and service
· Authentic influencer partnerships fuel brand growth
· The strongest connections happen via call, click, chat, and camera.
· Time saved with AI leads to more creative work
· Seeking: A single source of truth
· TLDR; Get on social, try AI, and align your systems.
· More human marketing, powered by robots
ChatGPT is a revolutionary addition to the world since its introduction in 2022. A big shift in the sector of information gathering and processing happened because of this chatbot. What is the story of ChatGPT? How is the bot responding to prompts and generating contents? Swipe through these slides prepared by Expeed Software, a web development company regarding the development and technical intricacies of ChatGPT!
Product Design Trends in 2024 | Teenage EngineeringsPixeldarts
The realm of product design is a constantly changing environment where technology and style intersect. Every year introduces fresh challenges and exciting trends that mold the future of this captivating art form. In this piece, we delve into the significant trends set to influence the look and functionality of product design in the year 2024.
How Race, Age and Gender Shape Attitudes Towards Mental HealthThinkNow
Mental health has been in the news quite a bit lately. Dozens of U.S. states are currently suing Meta for contributing to the youth mental health crisis by inserting addictive features into their products, while the U.S. Surgeon General is touring the nation to bring awareness to the growing epidemic of loneliness and isolation. The country has endured periods of low national morale, such as in the 1970s when high inflation and the energy crisis worsened public sentiment following the Vietnam War. The current mood, however, feels different. Gallup recently reported that national mental health is at an all-time low, with few bright spots to lift spirits.
To better understand how Americans are feeling and their attitudes towards mental health in general, ThinkNow conducted a nationally representative quantitative survey of 1,500 respondents and found some interesting differences among ethnic, age and gender groups.
Technology
For example, 52% agree that technology and social media have a negative impact on mental health, but when broken out by race, 61% of Whites felt technology had a negative effect, and only 48% of Hispanics thought it did.
While technology has helped us keep in touch with friends and family in faraway places, it appears to have degraded our ability to connect in person. Staying connected online is a double-edged sword since the same news feed that brings us pictures of the grandkids and fluffy kittens also feeds us news about the wars in Israel and Ukraine, the dysfunction in Washington, the latest mass shooting and the climate crisis.
Hispanics may have a built-in defense against the isolation technology breeds, owing to their large, multigenerational households, strong social support systems, and tendency to use social media to stay connected with relatives abroad.
Age and Gender
When asked how individuals rate their mental health, men rate it higher than women by 11 percentage points, and Baby Boomers rank it highest at 83%, saying it’s good or excellent vs. 57% of Gen Z saying the same.
Gen Z spends the most amount of time on social media, so the notion that social media negatively affects mental health appears to be correlated. Unfortunately, Gen Z is also the generation that’s least comfortable discussing mental health concerns with healthcare professionals. Only 40% of them state they’re comfortable discussing their issues with a professional compared to 60% of Millennials and 65% of Boomers.
Race Affects Attitudes
As seen in previous research conducted by ThinkNow, Asian Americans lag other groups when it comes to awareness of mental health issues. Twenty-four percent of Asian Americans believe that having a mental health issue is a sign of weakness compared to the 16% average for all groups. Asians are also considerably less likely to be aware of mental health services in their communities (42% vs. 55%) and most likely to seek out information on social media (51% vs. 35%).
AI Trends in Creative Operations 2024 by Artwork Flow.pdfmarketingartwork
This article is all about what AI trends will emerge in the field of creative operations in 2024. All the marketers and brand builders should be aware of these trends for their further use and save themselves some time!
A report by thenetworkone and Kurio.
The contributing experts and agencies are (in an alphabetical order): Sylwia Rytel, Social Media Supervisor, 180heartbeats + JUNG v MATT (PL), Sharlene Jenner, Vice President - Director of Engagement Strategy, Abelson Taylor (USA), Alex Casanovas, Digital Director, Atrevia (ES), Dora Beilin, Senior Social Strategist, Barrett Hoffher (USA), Min Seo, Campaign Director, Brand New Agency (KR), Deshé M. Gully, Associate Strategist, Day One Agency (USA), Francesca Trevisan, Strategist, Different (IT), Trevor Crossman, CX and Digital Transformation Director; Olivia Hussey, Strategic Planner; Simi Srinarula, Social Media Manager, The Hallway (AUS), James Hebbert, Managing Director, Hylink (CN / UK), Mundy Álvarez, Planning Director; Pedro Rojas, Social Media Manager; Pancho González, CCO, Inbrax (CH), Oana Oprea, Head of Digital Planning, Jam Session Agency (RO), Amy Bottrill, Social Account Director, Launch (UK), Gaby Arriaga, Founder, Leonardo1452 (MX), Shantesh S Row, Creative Director, Liwa (UAE), Rajesh Mehta, Chief Strategy Officer; Dhruv Gaur, Digital Planning Lead; Leonie Mergulhao, Account Supervisor - Social Media & PR, Medulla (IN), Aurelija Plioplytė, Head of Digital & Social, Not Perfect (LI), Daiana Khaidargaliyeva, Account Manager, Osaka Labs (UK / USA), Stefanie Söhnchen, Vice President Digital, PIABO Communications (DE), Elisabeth Winiartati, Managing Consultant, Head of Global Integrated Communications; Lydia Aprina, Account Manager, Integrated Marketing and Communications; Nita Prabowo, Account Manager, Integrated Marketing and Communications; Okhi, Web Developer, PNTR Group (ID), Kei Obusan, Insights Director; Daffi Ranandi, Insights Manager, Radarr (SG), Gautam Reghunath, Co-founder & CEO, Talented (IN), Donagh Humphreys, Head of Social and Digital Innovation, THINKHOUSE (IRE), Sarah Yim, Strategy Director, Zulu Alpha Kilo (CA).
Trends In Paid Search: Navigating The Digital Landscape In 2024Search Engine Journal
The search marketing landscape is evolving rapidly with new technologies, and professionals, like you, rely on innovative paid search strategies to meet changing demands.
It’s important that you’re ready to implement new strategies in 2024.
Check this out and learn the top trends in paid search advertising that are expected to gain traction, so you can drive higher ROI more efficiently in 2024.
You’ll learn:
- The latest trends in AI and automation, and what this means for an evolving paid search ecosystem.
- New developments in privacy and data regulation.
- Emerging ad formats that are expected to make an impact next year.
Watch Sreekant Lanka from iQuanti and Irina Klein from OneMain Financial as they dive into the future of paid search and explore the trends, strategies, and technologies that will shape the search marketing landscape.
If you’re looking to assess your paid search strategy and design an industry-aligned plan for 2024, then this webinar is for you.
5 Public speaking tips from TED - Visualized summarySpeakerHub
From their humble beginnings in 1984, TED has grown into the world’s most powerful amplifier for speakers and thought-leaders to share their ideas. They have over 2,400 filmed talks (not including the 30,000+ TEDx videos) freely available online, and have hosted over 17,500 events around the world.
With over one billion views in a year, it’s no wonder that so many speakers are looking to TED for ideas on how to share their message more effectively.
The article “5 Public-Speaking Tips TED Gives Its Speakers”, by Carmine Gallo for Forbes, gives speakers five practical ways to connect with their audience, and effectively share their ideas on stage.
Whether you are gearing up to get on a TED stage yourself, or just want to master the skills that so many of their speakers possess, these tips and quotes from Chris Anderson, the TED Talks Curator, will encourage you to make the most impactful impression on your audience.
See the full article and more summaries like this on SpeakerHub here: https://speakerhub.com/blog/5-presentation-tips-ted-gives-its-speakers
See the original article on Forbes here:
http://www.forbes.com/forbes/welcome/?toURL=http://www.forbes.com/sites/carminegallo/2016/05/06/5-public-speaking-tips-ted-gives-its-speakers/&refURL=&referrer=#5c07a8221d9b
ChatGPT and the Future of Work - Clark Boyd Clark Boyd
Everyone is in agreement that ChatGPT (and other generative AI tools) will shape the future of work. Yet there is little consensus on exactly how, when, and to what extent this technology will change our world.
Businesses that extract maximum value from ChatGPT will use it as a collaborative tool for everything from brainstorming to technical maintenance.
For individuals, now is the time to pinpoint the skills the future professional will need to thrive in the AI age.
Check out this presentation to understand what ChatGPT is, how it will shape the future of work, and how you can prepare to take advantage.
A brief introduction to DataScience with explaining of the concepts, algorithms, machine learning, supervised and unsupervised learning, clustering, statistics, data preprocessing, real-world applications etc.
It's part of a Data Science Corner Campaign where I will be discussing the fundamentals of DataScience, AIML, Statistics etc.
Time Management & Productivity - Best PracticesVit Horky
Here's my presentation on by proven best practices how to manage your work time effectively and how to improve your productivity. It includes practical tips and how to use tools such as Slack, Google Apps, Hubspot, Google Calendar, Gmail and others.
The six step guide to practical project managementMindGenius
The six step guide to practical project management
If you think managing projects is too difficult, think again.
We’ve stripped back project management processes to the
basics – to make it quicker and easier, without sacrificing
the vital ingredients for success.
“If you’re looking for some real-world guidance, then The Six Step Guide to Practical Project Management will help.”
Dr Andrew Makar, Tactical Project Management
2024 State of Marketing Report – by HubspotMarius Sescu
https://www.hubspot.com/state-of-marketing
· Scaling relationships and proving ROI
· Social media is the place for search, sales, and service
· Authentic influencer partnerships fuel brand growth
· The strongest connections happen via call, click, chat, and camera.
· Time saved with AI leads to more creative work
· Seeking: A single source of truth
· TLDR; Get on social, try AI, and align your systems.
· More human marketing, powered by robots
ChatGPT is a revolutionary addition to the world since its introduction in 2022. A big shift in the sector of information gathering and processing happened because of this chatbot. What is the story of ChatGPT? How is the bot responding to prompts and generating contents? Swipe through these slides prepared by Expeed Software, a web development company regarding the development and technical intricacies of ChatGPT!
Product Design Trends in 2024 | Teenage EngineeringsPixeldarts
The realm of product design is a constantly changing environment where technology and style intersect. Every year introduces fresh challenges and exciting trends that mold the future of this captivating art form. In this piece, we delve into the significant trends set to influence the look and functionality of product design in the year 2024.
How Race, Age and Gender Shape Attitudes Towards Mental HealthThinkNow
Mental health has been in the news quite a bit lately. Dozens of U.S. states are currently suing Meta for contributing to the youth mental health crisis by inserting addictive features into their products, while the U.S. Surgeon General is touring the nation to bring awareness to the growing epidemic of loneliness and isolation. The country has endured periods of low national morale, such as in the 1970s when high inflation and the energy crisis worsened public sentiment following the Vietnam War. The current mood, however, feels different. Gallup recently reported that national mental health is at an all-time low, with few bright spots to lift spirits.
To better understand how Americans are feeling and their attitudes towards mental health in general, ThinkNow conducted a nationally representative quantitative survey of 1,500 respondents and found some interesting differences among ethnic, age and gender groups.
Technology
For example, 52% agree that technology and social media have a negative impact on mental health, but when broken out by race, 61% of Whites felt technology had a negative effect, and only 48% of Hispanics thought it did.
While technology has helped us keep in touch with friends and family in faraway places, it appears to have degraded our ability to connect in person. Staying connected online is a double-edged sword since the same news feed that brings us pictures of the grandkids and fluffy kittens also feeds us news about the wars in Israel and Ukraine, the dysfunction in Washington, the latest mass shooting and the climate crisis.
Hispanics may have a built-in defense against the isolation technology breeds, owing to their large, multigenerational households, strong social support systems, and tendency to use social media to stay connected with relatives abroad.
Age and Gender
When asked how individuals rate their mental health, men rate it higher than women by 11 percentage points, and Baby Boomers rank it highest at 83%, saying it’s good or excellent vs. 57% of Gen Z saying the same.
Gen Z spends the most amount of time on social media, so the notion that social media negatively affects mental health appears to be correlated. Unfortunately, Gen Z is also the generation that’s least comfortable discussing mental health concerns with healthcare professionals. Only 40% of them state they’re comfortable discussing their issues with a professional compared to 60% of Millennials and 65% of Boomers.
Race Affects Attitudes
As seen in previous research conducted by ThinkNow, Asian Americans lag other groups when it comes to awareness of mental health issues. Twenty-four percent of Asian Americans believe that having a mental health issue is a sign of weakness compared to the 16% average for all groups. Asians are also considerably less likely to be aware of mental health services in their communities (42% vs. 55%) and most likely to seek out information on social media (51% vs. 35%).
AI Trends in Creative Operations 2024 by Artwork Flow.pdfmarketingartwork
This article is all about what AI trends will emerge in the field of creative operations in 2024. All the marketers and brand builders should be aware of these trends for their further use and save themselves some time!
A report by thenetworkone and Kurio.
The contributing experts and agencies are (in an alphabetical order): Sylwia Rytel, Social Media Supervisor, 180heartbeats + JUNG v MATT (PL), Sharlene Jenner, Vice President - Director of Engagement Strategy, Abelson Taylor (USA), Alex Casanovas, Digital Director, Atrevia (ES), Dora Beilin, Senior Social Strategist, Barrett Hoffher (USA), Min Seo, Campaign Director, Brand New Agency (KR), Deshé M. Gully, Associate Strategist, Day One Agency (USA), Francesca Trevisan, Strategist, Different (IT), Trevor Crossman, CX and Digital Transformation Director; Olivia Hussey, Strategic Planner; Simi Srinarula, Social Media Manager, The Hallway (AUS), James Hebbert, Managing Director, Hylink (CN / UK), Mundy Álvarez, Planning Director; Pedro Rojas, Social Media Manager; Pancho González, CCO, Inbrax (CH), Oana Oprea, Head of Digital Planning, Jam Session Agency (RO), Amy Bottrill, Social Account Director, Launch (UK), Gaby Arriaga, Founder, Leonardo1452 (MX), Shantesh S Row, Creative Director, Liwa (UAE), Rajesh Mehta, Chief Strategy Officer; Dhruv Gaur, Digital Planning Lead; Leonie Mergulhao, Account Supervisor - Social Media & PR, Medulla (IN), Aurelija Plioplytė, Head of Digital & Social, Not Perfect (LI), Daiana Khaidargaliyeva, Account Manager, Osaka Labs (UK / USA), Stefanie Söhnchen, Vice President Digital, PIABO Communications (DE), Elisabeth Winiartati, Managing Consultant, Head of Global Integrated Communications; Lydia Aprina, Account Manager, Integrated Marketing and Communications; Nita Prabowo, Account Manager, Integrated Marketing and Communications; Okhi, Web Developer, PNTR Group (ID), Kei Obusan, Insights Director; Daffi Ranandi, Insights Manager, Radarr (SG), Gautam Reghunath, Co-founder & CEO, Talented (IN), Donagh Humphreys, Head of Social and Digital Innovation, THINKHOUSE (IRE), Sarah Yim, Strategy Director, Zulu Alpha Kilo (CA).
Trends In Paid Search: Navigating The Digital Landscape In 2024Search Engine Journal
The search marketing landscape is evolving rapidly with new technologies, and professionals, like you, rely on innovative paid search strategies to meet changing demands.
It’s important that you’re ready to implement new strategies in 2024.
Check this out and learn the top trends in paid search advertising that are expected to gain traction, so you can drive higher ROI more efficiently in 2024.
You’ll learn:
- The latest trends in AI and automation, and what this means for an evolving paid search ecosystem.
- New developments in privacy and data regulation.
- Emerging ad formats that are expected to make an impact next year.
Watch Sreekant Lanka from iQuanti and Irina Klein from OneMain Financial as they dive into the future of paid search and explore the trends, strategies, and technologies that will shape the search marketing landscape.
If you’re looking to assess your paid search strategy and design an industry-aligned plan for 2024, then this webinar is for you.
5 Public speaking tips from TED - Visualized summarySpeakerHub
From their humble beginnings in 1984, TED has grown into the world’s most powerful amplifier for speakers and thought-leaders to share their ideas. They have over 2,400 filmed talks (not including the 30,000+ TEDx videos) freely available online, and have hosted over 17,500 events around the world.
With over one billion views in a year, it’s no wonder that so many speakers are looking to TED for ideas on how to share their message more effectively.
The article “5 Public-Speaking Tips TED Gives Its Speakers”, by Carmine Gallo for Forbes, gives speakers five practical ways to connect with their audience, and effectively share their ideas on stage.
Whether you are gearing up to get on a TED stage yourself, or just want to master the skills that so many of their speakers possess, these tips and quotes from Chris Anderson, the TED Talks Curator, will encourage you to make the most impactful impression on your audience.
See the full article and more summaries like this on SpeakerHub here: https://speakerhub.com/blog/5-presentation-tips-ted-gives-its-speakers
See the original article on Forbes here:
http://www.forbes.com/forbes/welcome/?toURL=http://www.forbes.com/sites/carminegallo/2016/05/06/5-public-speaking-tips-ted-gives-its-speakers/&refURL=&referrer=#5c07a8221d9b
ChatGPT and the Future of Work - Clark Boyd Clark Boyd
Everyone is in agreement that ChatGPT (and other generative AI tools) will shape the future of work. Yet there is little consensus on exactly how, when, and to what extent this technology will change our world.
Businesses that extract maximum value from ChatGPT will use it as a collaborative tool for everything from brainstorming to technical maintenance.
For individuals, now is the time to pinpoint the skills the future professional will need to thrive in the AI age.
Check out this presentation to understand what ChatGPT is, how it will shape the future of work, and how you can prepare to take advantage.
A brief introduction to DataScience with explaining of the concepts, algorithms, machine learning, supervised and unsupervised learning, clustering, statistics, data preprocessing, real-world applications etc.
It's part of a Data Science Corner Campaign where I will be discussing the fundamentals of DataScience, AIML, Statistics etc.
Time Management & Productivity - Best PracticesVit Horky
Here's my presentation on by proven best practices how to manage your work time effectively and how to improve your productivity. It includes practical tips and how to use tools such as Slack, Google Apps, Hubspot, Google Calendar, Gmail and others.
The six step guide to practical project managementMindGenius
The six step guide to practical project management
If you think managing projects is too difficult, think again.
We’ve stripped back project management processes to the
basics – to make it quicker and easier, without sacrificing
the vital ingredients for success.
“If you’re looking for some real-world guidance, then The Six Step Guide to Practical Project Management will help.”
Dr Andrew Makar, Tactical Project Management
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
[kakaobrain]beep
1. BEEP! Korean Corpus of Online News
Comments for Toxic Speech Detection
Jihyung Moon*, Won Ik Cho*, Junbum Lee
SocialNLP@ACL2020
KOCO
2. Introduction
• The topic to be covered:
Korean Corpus of Online News Comments for Toxic Speech Detection
• Main Contribution
• Release the first Korean corpus manually annotated on toxic
attributes
• Provide benchmarks for both bias classification and hate
speech detection
• Boost researches on this topic
2
3. Introduction
• The topic to be covered:
Korean Corpus of Online News Comments for Toxic Speech Detection
• Main Contribution
• Release the first Korean corpus manually annotated on toxic
attributes
• Set up the annotation guideline
• Provide benchmarks for both bias classification and hate
speech detection
• Boost researches on this topic
3
4. Motivation
• Recently, Korea had suffered a series of tragic incidents of
two young celebrities that are presumed to be caused by
toxic comments in Entertainment news platform.
• Toxic comments on online platforms have emerged as a
critical social issue.
• Causes mental harm to the target (usually explicit)
• Offends many comment viewers
4
5. Motivation
• Current Approaches
• Close the comment system in Entertainment news
• (pros) Demolishes all the issues
• (cons) Not the fundamental solution
• Term-based toxic speech detection
• (pros) Easy to apply
• (cons) False predictions / Not all hate speech is detectable using
terms.
• Learning algorithm-based toxic speech detection
• (pros) Better detection ability
• (cons) Require labeled data
5
However, there has been
no Korean corpus yet.
6. Motivation
• We need to build corpus!
• that can reduce toxic speech in entertainment news comments
• and that are manually labeled
• based on a guideline
• reflecting not only toxic speech features
• but also socio-linguistic features of Korean online texts
6
8. Data Collection
• Comments from the most popular Korean entertainment
news platform
• Most-viewed articles in Jan. 2018 ~ Feb. 2020
• 10,403,368 comments from 23,700 articles
• Sampling and Filtering
• Stratified sample articles and get 1,580 articles
• Top 20 comments in the order of Wilson score on the
downvote for each 1,580 articles
• Filter the duplicates and leave comments having more than
single token and less than 100 characters
à Random sampled 10K comments
10
comment example
9. Data Annotation
• Overall Process
• Set-up a guideline
• We had looked into 1k sample sentences and made a guideline
• Work with crowd-workers (at least 3 annotators on 1 sentence)
• Pilot study with crowd-workers
• To train and find crowd-workers whose annotation sense is similar
to ours.
• Batch annotation from the selected crowd-workers
• Filter out cases where no majority result exists
11
10. • 우리가 악성 댓글이라고 생각하는 댓글은 어떤 특징이 있을까?
• 편견
• 혐오
• 모욕
12
Data Annotation
Guideline
11. • 우리가 악성 댓글이라고 생각하는 댓글은 어떤 특징이 있을까?
• 편견: “이러이러한 사람”은 “이러이러할 것이다 / 이러이러해야 한다.”
• 개인에 대한 판단이 아님
• 개인의 (눈에 보이는) 일부 특성을 그 사람이 속한 집단의 특성과
동일시하여 집단에 대한 생각을 투영시키는 것
• 예) (여자 연예인에게) 가정을 지키고 싶으면 요리배우고 살빼야할듯.
• 이 댓글 작성자의 생각의 이면에는 1) 여성의 가정 속 역할은 상대에게
맛있는 식사를 제공하는 것과 2) 아름다워야 한다는 것이 있음
• 혐오
• 모욕
13
Data Annotation
Guideline
12. • 우리가 악성 댓글이라고 생각하는 댓글은 어떤 특징이 있을까?
• 편견
• 혐오 ~= 적대감
• 편견과 동반된 경우, 개인이 속한 집단에 대한 혐오가 투영됨
• 예) 한남은 믿거해야쥬
• 예) 동성애가 무슨 자랑도 아니고 역겹네 XXXX들
• 합리적인 비판과 거리가 있음.
• 예) OOO은 빼라..목소리 자체가 짜증이다
• Profanity terms 로도 표출될 수 있으나 단순히 이 용어가 등장한다고
해서 혐오가 느껴지는 것은 아님
• 예) ㅅㅂㅋㅋㅋㅋㅋㅋㅋㅋㅋㅋㅋㅋㅋㅋㅋㅋㅋㅋ
• 모욕
14
Data Annotation
Guideline
13. • 우리가 악성 댓글이라고 생각하는 댓글은 어떤 특징이 있을까?
• 편견
• 혐오
• 모욕: 대상이 모욕감 혹은 수치감을 느낄 수 있는 발언
• 대상의 사회적 평가를 저하시킬 만한 추상적 판단이나 경멸적 감정을
표현하는 것 1
• 예) 저뮤탈리스크같이 생긴애는 왜자꾸나옴?ㅎ
• 혐오가 동반되지 않을 수 있음
• 예) 스타킹 벗겨서 발가락빨구시퍼용.
• 사실 적시를 통한 모욕도 포함
• 예) ㅋ 접대와 조작의 아이콘 아이즈원 엑스원
151) http://www.law.go.kr/%ED%8C%90%EB%A1%80/(2015%EB%8F%842229)
Data Annotation
Guideline
14. • 편견
• 분류의 문제에 가까움
• 성에 대한 편견인가?
• 지역에 대한 편견인가?
• 인종에 대한 편견인가?
• …
• 혐오 및 모욕
• 정도의 문제에 가까움
• 성희롱 혹은 성적 대상화 외의 모욕적 표현은 혐오와 깊이 연관
16
Data Annotation
Guideline
15. • 편견
• Gender-related bias
• Other biases
• None
• 혐오 및 모욕
• Severe hate or insulting
• Not hateful but offensive or sarcastic
• None
17
Data Annotation
Guideline
16. • 가이드라인 세부 내용
• 예외 케이스
• 댓글의 내용이 미완성이거나 불분명하여 이해되지 않는 경우
• 예) 이거 보는 주 시청자가
• 예) 귄있는 스타일
• 댓글의 내용만으로는 판단되지 않는 경우
• 예) 역시.. 왠지 그럴 것 같았음
• 위 댓글이 열애 기사에서 등장한 경우와 미투 기사에서 등장한 경우
판단 결과가 다름
• 한국어 외의 언어로 작성된 경우
18
Data Annotation
Guideline
17. • 가이드라인 세부 내용
• 편견: Gender-related bias
19
Data Annotation
Guideline
18. • 가이드라인 세부 내용
• 편견: Other biases
20
Data Annotation
Guideline
19. • 가이드라인 세부 내용
• 편견: Other biases
21
Data Annotation
Guideline
20. • 가이드라인 세부 내용
• 혐오 및 모욕: Severe hate or insulting
22
Data Annotation
Guideline
21. • 가이드라인 세부 내용
• 혐오 및 모욕: Severe hate or insulting
23
Data Annotation
Guideline
22. • 가이드라인 세부 내용
• 혐오 및 모욕: Not hateful but offensive or sarcastic
24
Data Annotation
Guideline
23. • 가이드라인 세부 내용
• 혐오 및 모욕: None
25
Data Annotation
Guideline
24. • 가이드라인 세부 내용
• 예시 문항
26
Data Annotation
Guideline
25. • 가이드라인 세부 내용
• 예시 문항
27
Data Annotation
Guideline
26. • 가이드라인 세부 내용
• 예시 문항
28
Data Annotation
Guideline
27. Data Annotation
Crowdsource
• Pilot study
• 목표: annotation sense 가 비슷한 작업자 선정 및 작업자 훈련
• 어떤 가이드라인도 완벽하게 모든 케이스를 커버할 수 없음
• 가이드라인에 기술되지 않은 케이스에 대해서 우리가 한 것과 유사한
라벨을 달 수 있는 작업자가 필요
• 플랫폼 상에서 피드백을 주면서 반려할 수 있었음
• 어려운 문항에 대해 좋은 라벨을 달아준 작업자에게 가중치 부여
• 피드백 반영 속도가 빠른 작업자에게 가중치 부여
• 이 과정에서 성 감수성, 혐오 감수성이 다른 작업자들 탈락
29
28. • Batch annotation
• Pilot study 에서 선정된 작업자들에게만 할당
• 8,000개의 문장을 2명이서 전수 리뷰할 수 없으므로 작업자들의
결과물을 믿고 accept
• Filtering
• 3명의 검수자 의견이 모두 다른 경우
• 검수자 의견이 크게 다른 경우 (혐오 존재 vs. 혐오 존재 x)
• 659 문장이 제거됨
30
Data Annotation
Crowdsource & Filtering
29. Data Statistics
• Data composition
• Train: 7,896
• Batch annotation 결과
• Validation: 471
• Pilot study 에서 직접 태깅 결과를 보고 반려/승인 절차를 거친 결과
• Test: 974
• 가이드라인 작성자들의 의도가 가장 잘 반영된 데이터
32
31. Benchmark Results
• CharCNN vs. BiLSTM vs. BERT
• Models trained on our corpus > Term matching method
• BERT shows the bect performance for all tasks
• Pretrained model’s linguistic knowledge and semantic
information help.
• Bias is useful information to catch the hate speech.
34
33. Release and Competitions
• Github repository (Dataset and Guideline)
• https://github.com/kocohub/korean-hate-speech
• Kaggle competition
• https://www.kaggle.com/c/korean-gender-bias-detection
• https://www.kaggle.com/c/korean-bias-detection/
• https://www.kaggle.com/c/korean-hate-speech-detection/
• Used for an official government AI hackathon (6/17~6/30)
• http://www.aichallenge.or.kr/task/detail.do?taskId=T000018
36
34. Data Import Package
• For more convenient data importing, we released koco
• https://github.com/inmoonlight/koco
37
35. Takeaways
• 아직도 “혐오란 무엇인가?”에 대해 똑부러진 답을 하긴 어렵다.
• 글을 쓰는 사람의 (합리적 비판이라고 주장하는) 표현의 자유와
글을 보는 사람의 불편한 감정의 경계 어디쯤에 있는 것 같다.
• 혐오의 기준은 데이터의 활용 목적에 따라 달라져야 할 것이다.
• 우리는 작성자의 표현의 자유를 고려하기보다, 작성된 글을 읽을 사람과 대상의
입장에서 기준을 세웠다.
• 하지만 회사라면 사용자의 표현의 자유를 많이 고려해야 할 것이다.
• 댓글은 너무나 창의적이다.
• 기사의 내용만이 컨텍스트가 아니다. 댓글 작성 시기의 사회적 맥락과
연예인에 대한 메타 정보 또한 중요하다.
• 예) 살빠진 승리같애
38