2024 State of Marketing Report – by HubspotMarius Sescu
https://www.hubspot.com/state-of-marketing
· Scaling relationships and proving ROI
· Social media is the place for search, sales, and service
· Authentic influencer partnerships fuel brand growth
· The strongest connections happen via call, click, chat, and camera.
· Time saved with AI leads to more creative work
· Seeking: A single source of truth
· TLDR; Get on social, try AI, and align your systems.
· More human marketing, powered by robots
ChatGPT is a revolutionary addition to the world since its introduction in 2022. A big shift in the sector of information gathering and processing happened because of this chatbot. What is the story of ChatGPT? How is the bot responding to prompts and generating contents? Swipe through these slides prepared by Expeed Software, a web development company regarding the development and technical intricacies of ChatGPT!
Product Design Trends in 2024 | Teenage EngineeringsPixeldarts
The realm of product design is a constantly changing environment where technology and style intersect. Every year introduces fresh challenges and exciting trends that mold the future of this captivating art form. In this piece, we delve into the significant trends set to influence the look and functionality of product design in the year 2024.
How Race, Age and Gender Shape Attitudes Towards Mental HealthThinkNow
Mental health has been in the news quite a bit lately. Dozens of U.S. states are currently suing Meta for contributing to the youth mental health crisis by inserting addictive features into their products, while the U.S. Surgeon General is touring the nation to bring awareness to the growing epidemic of loneliness and isolation. The country has endured periods of low national morale, such as in the 1970s when high inflation and the energy crisis worsened public sentiment following the Vietnam War. The current mood, however, feels different. Gallup recently reported that national mental health is at an all-time low, with few bright spots to lift spirits.
To better understand how Americans are feeling and their attitudes towards mental health in general, ThinkNow conducted a nationally representative quantitative survey of 1,500 respondents and found some interesting differences among ethnic, age and gender groups.
Technology
For example, 52% agree that technology and social media have a negative impact on mental health, but when broken out by race, 61% of Whites felt technology had a negative effect, and only 48% of Hispanics thought it did.
While technology has helped us keep in touch with friends and family in faraway places, it appears to have degraded our ability to connect in person. Staying connected online is a double-edged sword since the same news feed that brings us pictures of the grandkids and fluffy kittens also feeds us news about the wars in Israel and Ukraine, the dysfunction in Washington, the latest mass shooting and the climate crisis.
Hispanics may have a built-in defense against the isolation technology breeds, owing to their large, multigenerational households, strong social support systems, and tendency to use social media to stay connected with relatives abroad.
Age and Gender
When asked how individuals rate their mental health, men rate it higher than women by 11 percentage points, and Baby Boomers rank it highest at 83%, saying it’s good or excellent vs. 57% of Gen Z saying the same.
Gen Z spends the most amount of time on social media, so the notion that social media negatively affects mental health appears to be correlated. Unfortunately, Gen Z is also the generation that’s least comfortable discussing mental health concerns with healthcare professionals. Only 40% of them state they’re comfortable discussing their issues with a professional compared to 60% of Millennials and 65% of Boomers.
Race Affects Attitudes
As seen in previous research conducted by ThinkNow, Asian Americans lag other groups when it comes to awareness of mental health issues. Twenty-four percent of Asian Americans believe that having a mental health issue is a sign of weakness compared to the 16% average for all groups. Asians are also considerably less likely to be aware of mental health services in their communities (42% vs. 55%) and most likely to seek out information on social media (51% vs. 35%).
AI Trends in Creative Operations 2024 by Artwork Flow.pdfmarketingartwork
Creative operations teams expect increased AI use in 2024. Currently, over half of tasks are not AI-enabled, but this is expected to decrease in the coming year. ChatGPT is the most popular AI tool currently. Business leaders are more actively exploring AI benefits than individual contributors. Most respondents do not believe AI will impact workforce size in 2024. However, some inhibitions still exist around AI accuracy and lack of understanding. Creatives primarily want to use AI to save time on mundane tasks and boost productivity.
Organizational culture includes values, norms, systems, symbols, language, assumptions, beliefs, and habits that influence employee behaviors and how people interpret those behaviors. It is important because culture can help or hinder a company's success. Some key aspects of Netflix's culture that help it achieve results include hiring smartly so every position has stars, focusing on attitude over just aptitude, and having a strict policy against peacocks, whiners, and jerks.
PEPSICO Presentation to CAGNY Conference Feb 2024Neil Kimberley
PepsiCo provided a safe harbor statement noting that any forward-looking statements are based on currently available information and are subject to risks and uncertainties. It also provided information on non-GAAP measures and directing readers to its website for disclosure and reconciliation. The document then discussed PepsiCo's business overview, including that it is a global beverage and convenient food company with iconic brands, $91 billion in net revenue in 2023, and nearly $14 billion in core operating profit. It operates through a divisional structure with a focus on local consumers.
Content Methodology: A Best Practices Report (Webinar)contently
This document provides an overview of content methodology best practices. It defines content methodology as establishing objectives, KPIs, and a culture of continuous learning and iteration. An effective methodology focuses on connecting with audiences, creating optimal content, and optimizing processes. It also discusses why a methodology is needed due to the competitive landscape, proliferation of channels, and opportunities for improvement. Components of an effective methodology include defining objectives and KPIs, audience analysis, identifying opportunities, and evaluating resources. The document concludes with recommendations around creating a content plan, testing and optimizing content over 90 days.
How to Prepare For a Successful Job Search for 2024Albert Qian
The document provides guidance on preparing a job search for 2024. It discusses the state of the job market, focusing on growth in AI and healthcare but also continued layoffs. It recommends figuring out what you want to do by researching interests and skills, then conducting informational interviews. The job search should involve building a personal brand on LinkedIn, actively applying to jobs, tailoring resumes and interviews, maintaining job hunting as a habit, and continuing self-improvement. Once hired, the document advises setting new goals and keeping skills and networking active in case of future opportunities.
A report by thenetworkone and Kurio.
The contributing experts and agencies are (in an alphabetical order): Sylwia Rytel, Social Media Supervisor, 180heartbeats + JUNG v MATT (PL), Sharlene Jenner, Vice President - Director of Engagement Strategy, Abelson Taylor (USA), Alex Casanovas, Digital Director, Atrevia (ES), Dora Beilin, Senior Social Strategist, Barrett Hoffher (USA), Min Seo, Campaign Director, Brand New Agency (KR), Deshé M. Gully, Associate Strategist, Day One Agency (USA), Francesca Trevisan, Strategist, Different (IT), Trevor Crossman, CX and Digital Transformation Director; Olivia Hussey, Strategic Planner; Simi Srinarula, Social Media Manager, The Hallway (AUS), James Hebbert, Managing Director, Hylink (CN / UK), Mundy Álvarez, Planning Director; Pedro Rojas, Social Media Manager; Pancho González, CCO, Inbrax (CH), Oana Oprea, Head of Digital Planning, Jam Session Agency (RO), Amy Bottrill, Social Account Director, Launch (UK), Gaby Arriaga, Founder, Leonardo1452 (MX), Shantesh S Row, Creative Director, Liwa (UAE), Rajesh Mehta, Chief Strategy Officer; Dhruv Gaur, Digital Planning Lead; Leonie Mergulhao, Account Supervisor - Social Media & PR, Medulla (IN), Aurelija Plioplytė, Head of Digital & Social, Not Perfect (LI), Daiana Khaidargaliyeva, Account Manager, Osaka Labs (UK / USA), Stefanie Söhnchen, Vice President Digital, PIABO Communications (DE), Elisabeth Winiartati, Managing Consultant, Head of Global Integrated Communications; Lydia Aprina, Account Manager, Integrated Marketing and Communications; Nita Prabowo, Account Manager, Integrated Marketing and Communications; Okhi, Web Developer, PNTR Group (ID), Kei Obusan, Insights Director; Daffi Ranandi, Insights Manager, Radarr (SG), Gautam Reghunath, Co-founder & CEO, Talented (IN), Donagh Humphreys, Head of Social and Digital Innovation, THINKHOUSE (IRE), Sarah Yim, Strategy Director, Zulu Alpha Kilo (CA).
Trends In Paid Search: Navigating The Digital Landscape In 2024Search Engine Journal
The search marketing landscape is evolving rapidly with new technologies, and professionals, like you, rely on innovative paid search strategies to meet changing demands.
It’s important that you’re ready to implement new strategies in 2024.
Check this out and learn the top trends in paid search advertising that are expected to gain traction, so you can drive higher ROI more efficiently in 2024.
You’ll learn:
- The latest trends in AI and automation, and what this means for an evolving paid search ecosystem.
- New developments in privacy and data regulation.
- Emerging ad formats that are expected to make an impact next year.
Watch Sreekant Lanka from iQuanti and Irina Klein from OneMain Financial as they dive into the future of paid search and explore the trends, strategies, and technologies that will shape the search marketing landscape.
If you’re looking to assess your paid search strategy and design an industry-aligned plan for 2024, then this webinar is for you.
5 Public speaking tips from TED - Visualized summarySpeakerHub
From their humble beginnings in 1984, TED has grown into the world’s most powerful amplifier for speakers and thought-leaders to share their ideas. They have over 2,400 filmed talks (not including the 30,000+ TEDx videos) freely available online, and have hosted over 17,500 events around the world.
With over one billion views in a year, it’s no wonder that so many speakers are looking to TED for ideas on how to share their message more effectively.
The article “5 Public-Speaking Tips TED Gives Its Speakers”, by Carmine Gallo for Forbes, gives speakers five practical ways to connect with their audience, and effectively share their ideas on stage.
Whether you are gearing up to get on a TED stage yourself, or just want to master the skills that so many of their speakers possess, these tips and quotes from Chris Anderson, the TED Talks Curator, will encourage you to make the most impactful impression on your audience.
See the full article and more summaries like this on SpeakerHub here: https://speakerhub.com/blog/5-presentation-tips-ted-gives-its-speakers
See the original article on Forbes here:
http://www.forbes.com/forbes/welcome/?toURL=http://www.forbes.com/sites/carminegallo/2016/05/06/5-public-speaking-tips-ted-gives-its-speakers/&refURL=&referrer=#5c07a8221d9b
ChatGPT and the Future of Work - Clark Boyd Clark Boyd
Everyone is in agreement that ChatGPT (and other generative AI tools) will shape the future of work. Yet there is little consensus on exactly how, when, and to what extent this technology will change our world.
Businesses that extract maximum value from ChatGPT will use it as a collaborative tool for everything from brainstorming to technical maintenance.
For individuals, now is the time to pinpoint the skills the future professional will need to thrive in the AI age.
Check out this presentation to understand what ChatGPT is, how it will shape the future of work, and how you can prepare to take advantage.
The document provides career advice for getting into the tech field, including:
- Doing projects and internships in college to build a portfolio.
- Learning about different roles and technologies through industry research.
- Contributing to open source projects to build experience and network.
- Developing a personal brand through a website and social media presence.
- Networking through events, communities, and finding a mentor.
- Practicing interviews through mock interviews and whiteboarding coding questions.
Google's Just Not That Into You: Understanding Core Updates & Search IntentLily Ray
1. Core updates from Google periodically change how its algorithms assess and rank websites and pages. This can impact rankings through shifts in user intent, site quality issues being caught up to, world events influencing queries, and overhauls to search like the E-A-T framework.
2. There are many possible user intents beyond just transactional, navigational and informational. Identifying intent shifts is important during core updates. Sites may need to optimize for new intents through different content types and sections.
3. Responding effectively to core updates requires analyzing "before and after" data to understand changes, identifying new intents or page types, and ensuring content matches appropriate intents across video, images, knowledge graphs and more.
A brief introduction to DataScience with explaining of the concepts, algorithms, machine learning, supervised and unsupervised learning, clustering, statistics, data preprocessing, real-world applications etc.
It's part of a Data Science Corner Campaign where I will be discussing the fundamentals of DataScience, AIML, Statistics etc.
Time Management & Productivity - Best PracticesVit Horky
Here's my presentation on by proven best practices how to manage your work time effectively and how to improve your productivity. It includes practical tips and how to use tools such as Slack, Google Apps, Hubspot, Google Calendar, Gmail and others.
The six step guide to practical project managementMindGenius
The six step guide to practical project management
If you think managing projects is too difficult, think again.
We’ve stripped back project management processes to the
basics – to make it quicker and easier, without sacrificing
the vital ingredients for success.
“If you’re looking for some real-world guidance, then The Six Step Guide to Practical Project Management will help.”
Dr Andrew Makar, Tactical Project Management
2024 State of Marketing Report – by HubspotMarius Sescu
https://www.hubspot.com/state-of-marketing
· Scaling relationships and proving ROI
· Social media is the place for search, sales, and service
· Authentic influencer partnerships fuel brand growth
· The strongest connections happen via call, click, chat, and camera.
· Time saved with AI leads to more creative work
· Seeking: A single source of truth
· TLDR; Get on social, try AI, and align your systems.
· More human marketing, powered by robots
ChatGPT is a revolutionary addition to the world since its introduction in 2022. A big shift in the sector of information gathering and processing happened because of this chatbot. What is the story of ChatGPT? How is the bot responding to prompts and generating contents? Swipe through these slides prepared by Expeed Software, a web development company regarding the development and technical intricacies of ChatGPT!
Product Design Trends in 2024 | Teenage EngineeringsPixeldarts
The realm of product design is a constantly changing environment where technology and style intersect. Every year introduces fresh challenges and exciting trends that mold the future of this captivating art form. In this piece, we delve into the significant trends set to influence the look and functionality of product design in the year 2024.
How Race, Age and Gender Shape Attitudes Towards Mental HealthThinkNow
Mental health has been in the news quite a bit lately. Dozens of U.S. states are currently suing Meta for contributing to the youth mental health crisis by inserting addictive features into their products, while the U.S. Surgeon General is touring the nation to bring awareness to the growing epidemic of loneliness and isolation. The country has endured periods of low national morale, such as in the 1970s when high inflation and the energy crisis worsened public sentiment following the Vietnam War. The current mood, however, feels different. Gallup recently reported that national mental health is at an all-time low, with few bright spots to lift spirits.
To better understand how Americans are feeling and their attitudes towards mental health in general, ThinkNow conducted a nationally representative quantitative survey of 1,500 respondents and found some interesting differences among ethnic, age and gender groups.
Technology
For example, 52% agree that technology and social media have a negative impact on mental health, but when broken out by race, 61% of Whites felt technology had a negative effect, and only 48% of Hispanics thought it did.
While technology has helped us keep in touch with friends and family in faraway places, it appears to have degraded our ability to connect in person. Staying connected online is a double-edged sword since the same news feed that brings us pictures of the grandkids and fluffy kittens also feeds us news about the wars in Israel and Ukraine, the dysfunction in Washington, the latest mass shooting and the climate crisis.
Hispanics may have a built-in defense against the isolation technology breeds, owing to their large, multigenerational households, strong social support systems, and tendency to use social media to stay connected with relatives abroad.
Age and Gender
When asked how individuals rate their mental health, men rate it higher than women by 11 percentage points, and Baby Boomers rank it highest at 83%, saying it’s good or excellent vs. 57% of Gen Z saying the same.
Gen Z spends the most amount of time on social media, so the notion that social media negatively affects mental health appears to be correlated. Unfortunately, Gen Z is also the generation that’s least comfortable discussing mental health concerns with healthcare professionals. Only 40% of them state they’re comfortable discussing their issues with a professional compared to 60% of Millennials and 65% of Boomers.
Race Affects Attitudes
As seen in previous research conducted by ThinkNow, Asian Americans lag other groups when it comes to awareness of mental health issues. Twenty-four percent of Asian Americans believe that having a mental health issue is a sign of weakness compared to the 16% average for all groups. Asians are also considerably less likely to be aware of mental health services in their communities (42% vs. 55%) and most likely to seek out information on social media (51% vs. 35%).
AI Trends in Creative Operations 2024 by Artwork Flow.pdfmarketingartwork
Creative operations teams expect increased AI use in 2024. Currently, over half of tasks are not AI-enabled, but this is expected to decrease in the coming year. ChatGPT is the most popular AI tool currently. Business leaders are more actively exploring AI benefits than individual contributors. Most respondents do not believe AI will impact workforce size in 2024. However, some inhibitions still exist around AI accuracy and lack of understanding. Creatives primarily want to use AI to save time on mundane tasks and boost productivity.
Organizational culture includes values, norms, systems, symbols, language, assumptions, beliefs, and habits that influence employee behaviors and how people interpret those behaviors. It is important because culture can help or hinder a company's success. Some key aspects of Netflix's culture that help it achieve results include hiring smartly so every position has stars, focusing on attitude over just aptitude, and having a strict policy against peacocks, whiners, and jerks.
PEPSICO Presentation to CAGNY Conference Feb 2024Neil Kimberley
PepsiCo provided a safe harbor statement noting that any forward-looking statements are based on currently available information and are subject to risks and uncertainties. It also provided information on non-GAAP measures and directing readers to its website for disclosure and reconciliation. The document then discussed PepsiCo's business overview, including that it is a global beverage and convenient food company with iconic brands, $91 billion in net revenue in 2023, and nearly $14 billion in core operating profit. It operates through a divisional structure with a focus on local consumers.
Content Methodology: A Best Practices Report (Webinar)contently
This document provides an overview of content methodology best practices. It defines content methodology as establishing objectives, KPIs, and a culture of continuous learning and iteration. An effective methodology focuses on connecting with audiences, creating optimal content, and optimizing processes. It also discusses why a methodology is needed due to the competitive landscape, proliferation of channels, and opportunities for improvement. Components of an effective methodology include defining objectives and KPIs, audience analysis, identifying opportunities, and evaluating resources. The document concludes with recommendations around creating a content plan, testing and optimizing content over 90 days.
How to Prepare For a Successful Job Search for 2024Albert Qian
The document provides guidance on preparing a job search for 2024. It discusses the state of the job market, focusing on growth in AI and healthcare but also continued layoffs. It recommends figuring out what you want to do by researching interests and skills, then conducting informational interviews. The job search should involve building a personal brand on LinkedIn, actively applying to jobs, tailoring resumes and interviews, maintaining job hunting as a habit, and continuing self-improvement. Once hired, the document advises setting new goals and keeping skills and networking active in case of future opportunities.
A report by thenetworkone and Kurio.
The contributing experts and agencies are (in an alphabetical order): Sylwia Rytel, Social Media Supervisor, 180heartbeats + JUNG v MATT (PL), Sharlene Jenner, Vice President - Director of Engagement Strategy, Abelson Taylor (USA), Alex Casanovas, Digital Director, Atrevia (ES), Dora Beilin, Senior Social Strategist, Barrett Hoffher (USA), Min Seo, Campaign Director, Brand New Agency (KR), Deshé M. Gully, Associate Strategist, Day One Agency (USA), Francesca Trevisan, Strategist, Different (IT), Trevor Crossman, CX and Digital Transformation Director; Olivia Hussey, Strategic Planner; Simi Srinarula, Social Media Manager, The Hallway (AUS), James Hebbert, Managing Director, Hylink (CN / UK), Mundy Álvarez, Planning Director; Pedro Rojas, Social Media Manager; Pancho González, CCO, Inbrax (CH), Oana Oprea, Head of Digital Planning, Jam Session Agency (RO), Amy Bottrill, Social Account Director, Launch (UK), Gaby Arriaga, Founder, Leonardo1452 (MX), Shantesh S Row, Creative Director, Liwa (UAE), Rajesh Mehta, Chief Strategy Officer; Dhruv Gaur, Digital Planning Lead; Leonie Mergulhao, Account Supervisor - Social Media & PR, Medulla (IN), Aurelija Plioplytė, Head of Digital & Social, Not Perfect (LI), Daiana Khaidargaliyeva, Account Manager, Osaka Labs (UK / USA), Stefanie Söhnchen, Vice President Digital, PIABO Communications (DE), Elisabeth Winiartati, Managing Consultant, Head of Global Integrated Communications; Lydia Aprina, Account Manager, Integrated Marketing and Communications; Nita Prabowo, Account Manager, Integrated Marketing and Communications; Okhi, Web Developer, PNTR Group (ID), Kei Obusan, Insights Director; Daffi Ranandi, Insights Manager, Radarr (SG), Gautam Reghunath, Co-founder & CEO, Talented (IN), Donagh Humphreys, Head of Social and Digital Innovation, THINKHOUSE (IRE), Sarah Yim, Strategy Director, Zulu Alpha Kilo (CA).
Trends In Paid Search: Navigating The Digital Landscape In 2024Search Engine Journal
The search marketing landscape is evolving rapidly with new technologies, and professionals, like you, rely on innovative paid search strategies to meet changing demands.
It’s important that you’re ready to implement new strategies in 2024.
Check this out and learn the top trends in paid search advertising that are expected to gain traction, so you can drive higher ROI more efficiently in 2024.
You’ll learn:
- The latest trends in AI and automation, and what this means for an evolving paid search ecosystem.
- New developments in privacy and data regulation.
- Emerging ad formats that are expected to make an impact next year.
Watch Sreekant Lanka from iQuanti and Irina Klein from OneMain Financial as they dive into the future of paid search and explore the trends, strategies, and technologies that will shape the search marketing landscape.
If you’re looking to assess your paid search strategy and design an industry-aligned plan for 2024, then this webinar is for you.
5 Public speaking tips from TED - Visualized summarySpeakerHub
From their humble beginnings in 1984, TED has grown into the world’s most powerful amplifier for speakers and thought-leaders to share their ideas. They have over 2,400 filmed talks (not including the 30,000+ TEDx videos) freely available online, and have hosted over 17,500 events around the world.
With over one billion views in a year, it’s no wonder that so many speakers are looking to TED for ideas on how to share their message more effectively.
The article “5 Public-Speaking Tips TED Gives Its Speakers”, by Carmine Gallo for Forbes, gives speakers five practical ways to connect with their audience, and effectively share their ideas on stage.
Whether you are gearing up to get on a TED stage yourself, or just want to master the skills that so many of their speakers possess, these tips and quotes from Chris Anderson, the TED Talks Curator, will encourage you to make the most impactful impression on your audience.
See the full article and more summaries like this on SpeakerHub here: https://speakerhub.com/blog/5-presentation-tips-ted-gives-its-speakers
See the original article on Forbes here:
http://www.forbes.com/forbes/welcome/?toURL=http://www.forbes.com/sites/carminegallo/2016/05/06/5-public-speaking-tips-ted-gives-its-speakers/&refURL=&referrer=#5c07a8221d9b
ChatGPT and the Future of Work - Clark Boyd Clark Boyd
Everyone is in agreement that ChatGPT (and other generative AI tools) will shape the future of work. Yet there is little consensus on exactly how, when, and to what extent this technology will change our world.
Businesses that extract maximum value from ChatGPT will use it as a collaborative tool for everything from brainstorming to technical maintenance.
For individuals, now is the time to pinpoint the skills the future professional will need to thrive in the AI age.
Check out this presentation to understand what ChatGPT is, how it will shape the future of work, and how you can prepare to take advantage.
The document provides career advice for getting into the tech field, including:
- Doing projects and internships in college to build a portfolio.
- Learning about different roles and technologies through industry research.
- Contributing to open source projects to build experience and network.
- Developing a personal brand through a website and social media presence.
- Networking through events, communities, and finding a mentor.
- Practicing interviews through mock interviews and whiteboarding coding questions.
Google's Just Not That Into You: Understanding Core Updates & Search IntentLily Ray
1. Core updates from Google periodically change how its algorithms assess and rank websites and pages. This can impact rankings through shifts in user intent, site quality issues being caught up to, world events influencing queries, and overhauls to search like the E-A-T framework.
2. There are many possible user intents beyond just transactional, navigational and informational. Identifying intent shifts is important during core updates. Sites may need to optimize for new intents through different content types and sections.
3. Responding effectively to core updates requires analyzing "before and after" data to understand changes, identifying new intents or page types, and ensuring content matches appropriate intents across video, images, knowledge graphs and more.
A brief introduction to DataScience with explaining of the concepts, algorithms, machine learning, supervised and unsupervised learning, clustering, statistics, data preprocessing, real-world applications etc.
It's part of a Data Science Corner Campaign where I will be discussing the fundamentals of DataScience, AIML, Statistics etc.
Time Management & Productivity - Best PracticesVit Horky
Here's my presentation on by proven best practices how to manage your work time effectively and how to improve your productivity. It includes practical tips and how to use tools such as Slack, Google Apps, Hubspot, Google Calendar, Gmail and others.
The six step guide to practical project managementMindGenius
The six step guide to practical project management
If you think managing projects is too difficult, think again.
We’ve stripped back project management processes to the
basics – to make it quicker and easier, without sacrificing
the vital ingredients for success.
“If you’re looking for some real-world guidance, then The Six Step Guide to Practical Project Management will help.”
Dr Andrew Makar, Tactical Project Management
5. 행동심리학에서 강화라는 개념은 상당히 보편적으로
알려진 개념이다.
강화는 동물이 시행착오를 통해 학습하는 방법 중 하나이다.
강화라는 것은 동물이 이전에 배우지 않았지만 직접 시도하면서
행동과 그 결과로 나타나는 좋은 보상 사이의 상관관계를
학습하는 것이다.
그러면서 동물이 좋은 보상을 얻게 해주는 행동을
점점 더 많이 하는 것을 말한다.
7. 에이전트는 환경에 대해 사전지식이 없는 상태에서 학습을 한다.
에이전트는 자신이 놓인 환경에서 자신의 상태를 인식한 후 행동한다.
그러면 환경은 에이전트에게 보상을 주고 다음 상태를 알려준다.
이 보상을 통해 에이전트는 어떤 행동이 좋은 행동인지 간접적으로 알게 된다.
강화학습의 목적은 에이전트가 환경을 탐색하면서 얻는 보상들의 합을
최대화하는 "최적의 행동양식, 또는 정책"을 학습하는 것이다.
12. 강화학습에서 가장 간단한 형태의 문제는 n개의 손잡이가 달린 밴딧,
즉 멀티암드 밴딧이다.
밴딧은 손잡이가 n개 달린 슬롯머신이다. 각각의 손잡이는 다른 확률
로 보상을 제공한다.
에이전트의 목표는 시간의 흐름에 따라 가장 높은 보상을 제공하는
손잡이를 찾아내고 항상 이 손잡이를 선택함으로써
돌아오는 보상을 최대화하는 것이다.
이 문제에서는 액션 의존성만을 고려하면 된다. 어떤 보상이 어떤 액
션과 연관되어 있는지, 그리고 우리가 최적의 액션을 선택하도록
보장하는 것이 전부이다.
멀티암드 밴딧의 경우 환경은 상태를 가지지 않으며 에이전트는 단순
히 최선의 액션을 선택하도록 학습하면 된다.
상태가 주어지지 않기 때문에 어떤 순간에서의 최선의 액션은 곧 모
든 순간에서의 최선의 액션이 된다.
14. 콘텍스트 밴딧에서는 상태(state)라는 개념을 도입한다.
상태는 환경에 대한 기술(description)로 이루어지며,
에이전트는 이를 이용해 좀 더 정보에 기반을 두고
액션을 취할 수 있다.
각각의 밴딧은 각각의 손잡이에 대해 잠재적 보상이 다르기 때문에
에이전트는 환경의 상태에 기반을 두고
취할 액션의 조건을 학습해야 할 필요가 있다.
18. 상태 :
S는 에이전트가 관찰 가능한 상태의 집합이다.
자신의 상황에 대한 관찰이 상태에 대한 가장 정확한 표현이다.
행동 :
에이전트가 상태 에서 할 수 있는 가능한 행동의 집합은 A이다.
는 어떤 t라는 시간에 집합 A에서 선택한 행동이다
보상함수 :
보상은 에이전트가 학습할 수 있는 유일한 정보로서
환경이 에이전트에게 주는 정보이다.
19. 상태 변환 확률 :
앞으로 나아가는 행동을 했을때 바람이 불거나 미끄러지거나 해서
앞에 있는 상태로 도달하지 못할 수도 있다.
이처럼 상태의 변화에는 확률적인 요인이 들어간다.
이를 수치적으로 표현한 것이 상태 변환 확률이다.
감가율 :
현재에 가까운 보상일수록 더 큰 가치를 가진다.
에이전트는 그 보상이 얼마나 시간이 지나서
받는지를 고려해서 감가시켜 현재의 가치로 따진다.
0과 1사이의 감가율로 미래 보상을 감가시킨다.
20. 또 하나의 중요한 개념.. 정책
정책은 모든 상태에서 에이전트가 할 행동이다.
상태가 입력으로 들어오면 행동을 출력으로 내보내는 일종의 함수다.
에이전트가 강화학습을 통해 학습해야 할 것은
수많은 정책 중에서 최적 정책이다.
시간 t에 상태 s에 에이전트가 있을 때
가능한 행동 중에서 a를 할 확률을 나타낸다
23. 현재 시간 t로부터 얻은 보상들을 단순히 합하면 다음과 같다
하지만 보상들을 감가하지 않고 단순히 더하면
세가지 문제가 생긴다.
24. 1. 에이전트 입장에서는 지금 받은 보상이나
미래에 받는 보상이나 똑같이 취급한다.
(보상을 빨리 최대화 하는 방향으로 에이전트가
학습이 안될 수 있다.)
2. 100이라는 보상을 1번 받는것과
20이라는 보상을 5번 받는 것을 구분할 방법이 없다.
(100이라는 보상을 빨리 1번 받고 끝내는게 좋다)
3. 시간이 무한대라고 하면 보상을 시간마다 0.1씩 받아도
합이 무한대이고 1씩 받아도 합이 무한대이다.
수치적으로 두 경우를 구분할 수 없다.
25. 앞에서 본 3가지 이유 때문에
에이전트는 단순한 보상의 합으로는 판단을 내리기 어렵다.
따라서 좀 더 정확한 판단을 위해 감가율을 적용한다.
감가율을 적용한 보상들의 합을 반환값이라고 한다.
반환값이라는 것은 에이전트가 실제로
환경을 탐험하며 받은 보상의 합이다.
에이전트는 직접 다 경험을 해서 보상을 받아보지 않더라도
얼마의 보상을 받을 것이라고 예측할 수 있다.
어떠한 상태에 있으면 앞으로 얼마의 보상을
받을 것인지에 대한 기댓값을 고려해볼 수 있다.
그것이 바로 가치함수이다.
26. 가치함수는 반환값의 기댓값이다.
상태의 가치를 고려하는 이유는
만약 현재 에이전트가 갈 수 있는
상태들의 가치를 안다면 그중에서
가장 가치가 높은 상태를 선택할 수 있기 때문이다.
에이전트는 이러한 개념인 가치함수를 통해
어느 상태가 좋을지 판단한다.
27.
28. 에이전트가 앞으로 받을 보상에 대해 생각할 때
정책을 고려하지 않으면 안된다.
각 상태에서 행동을 하는 것이 에이전트의 정책이기 때문이다.
정책에 따라서 계산하는 가치함수는 당연히 달라질 수 밖에 없다.
MDP에서의 가치함수는 항상 정책을 고려해야 한다.
바로 위 수식이 벨만 기대 방정식이다. (Bellman Expectation Equation)
강화학습은 벨만 방정식을 풀어나가는게 핵심이다.
29. 에이전트는 가치함수를 통해
다음에 어떤 상태로 가야 할지 판단할 수 있다.
어떤 상태로 가면 좋을지 판단한 후에
그 상태로 가기 위한 행동을 따져볼 것이다.
하지만 어떤 상태에서 각 행동에 대해
따로 가치함수를 만들어서
그 정보를 얻어올 수 있다면,
에이전트는 굳이 다음 상태의
가치함수를 따져보지 않아도
어떤 행동을 해야 할지 선택할 수 있다.
이처럼 어떤 상태에서 어떤 행동이
얼마나 좋은지 알려주는 함수를 행동 가치함수라고 하며
큐함수라고도 부른다.
큐함수
30. 특정 상태 s에서 취할 수 있는 행동이 행동1과 행동2로
두가지라면 하나의 상태 s에서 2개의 행동 상태를 가지는 것이다.
2개의 행동 상태에서 따로 가치함수를 계산할 수 있는데,
그것이 바로 큐함수이다.
따라서 큐함수는 상태, 행동이라는 두 가지 변수를 가진다.
32. 1. 각 행동을 했을 때 앞으로 받을 보상인
큐함수 를 정책 π(a | s)에 곱한다.
2. 모든 행동에 대해 큐함수와 정책을 곱한 값을 더하면 가치함수가 된다.
큐함수 또한 벨만 기대 방정식의 형태로 나타낼 수 있다.
가치함수의 벨만 기대 방정식과 다른 점은
조건문에 행동이 추가된다는 점이다.
33.
34. 가치함수는 어떤 상태의 가치에 대한 기대를 나타낸다.
어떤 상태의 가치함수는 에이전트가 그 상태로
갈 경우에 앞으로 받을 보상의 합에 대한 기댓값이다.
벨만 기대 방정식은 현재 상태의 가치함수와
다음 상태의 가치함수 사이의 관계를 식으로 나타낸 것이다.
이 벨만 기대 방정식을 풀어나감으로써
현재 가치함수 값을 업데이트 해나가야 한다.
업데이트 하려면 방정식에서의 기댓값을 계산해야 하는데
기댓값에는 어떠한 행동을 할 확률인 정책 과
그 행동을 했을 때 어떤 상태로 가게 되는 확률인
상태 변환 확률 이 포함돼 있다.
35. 기댓값이 계산 가능한 형태의 벨만 기대 방정식
deterministic한 문제를 생각해보자.
즉 왼쪽으로 행동을 취하면 왼쪽에 있는 상태로
무조건 가게 되는 환경을 고려해보자
이 경우 상태 변환 확률은 모든 s와 a에 대해 1이다.
위의 수식을 상태 변환 확률이 모두 1인 경우로 바꾸면 다음과 같다.
36. 벨만 기대 방정식은 다음과 같이 설명할 수 있다.
1. 각 행동에 대해 그 행동을 할 확률을 고려하고(정책)π(a | s)
2. 각 행동을 했을 때 받을 보상과
3. 다음 상태의 가치함수를 고려한다.
벨만 기대 방정식을 이용해 현재의 가치함수를
계속 업데이트하다 보면 참값을 구할 수 있다.
참값이라는 것은 최대로 받을 보상을 이야기 하는 것이 아니다.
현재의 정책을 따라갔을 경우에 에이전트가 얻을
실제 보상의 값에 대한 참 기댓값이다.
37. 처음에 가치함수의 값들은 의미 없는 값으로 초기화된다.
초깃값으로부터 시작해서 벨만 기대 방정식으로 반복적으로 계산을 해나가면
의 값이 수렴하게 된다.
즉 현재 정책 π에 대한 참 가치함수를 구한 것이다.
참 가치함수와 최적 가치함수(Optimal Value Function)는 다르다.
참 가치함수는 '어떤 정책'을 따라서 움직였을 경우에
받게 되는 보상에 대한 참값이다.
하지만 최적 가치함수는 수많은 정책 중에서
가장 높은 보상을 주는 가치함수이다.
38. 벨만 기대 방정식을 다시 살펴보자
는 현재 정책에 따라 k+1번째 계산한 가치함수를 뜻하고
그중에서 상태 s의 가치함수를 의미한다.
k+1번째의 가치함수는 k번째 가치함수 중에서
주변 상태들 s ́ 을 이용해 구한다.
이 계산은 모든 상태에 대해 동시에 진행한다.
39. 이 수식은 상태집합에 속한 모든 상태에 대해
가능한 행동들을 고려한다.
주변 상태에 저장돼 있는 가치함수를 통해
현재의 가치함수를 업데이트한다.
이 수식을 통해 현재 정책에 대한 참 가치함수를 구할 수 있다.
하지만 단순히 현 정책에 대한 가치함수를 찾는 것이 아니라
더 좋은 정책으로 현재의 정책을 업데이트해나가야 한다.
가치함수가 결국 정책이 얼마나 좋은지를 말해주게 되는데
모든 정책에 대해 가장 큰 가치함수를 주는 정책이 최적 정책이다.
최적 정책을 따라갔을 때 받을 보상의 합이 최적 가치함수이다.
41. 가장 높은 큐함수를 에이전트가 찾았다고 가정해 보자.
이때 최적 정책은 각 상태 s에서의 최적의 큐함수 중에서
가장 큰 큐함수를 가진 행동을 하는 것이다.
즉, 선택 상황에서 판단 기준은 큐함수이며,
최적 정책은 언제나 이 큐함수 중에서 가장 높은 행동 하나를 하는 것이다.
에이전트는 큐함수를 기준으로 어떤 행동이 가장 좋은지를 결정한다.
따라서 최적의 큐함수 중에서 max(최댓값)를 취하는 것이
최적의 가치함수이다.
42. 앞 수식에서 큐함수를 가치함수로 고쳐서 표현을 해보자
이를 벨만 최적 방정식이라고 부른다.(Bellman Optimality Equation)
큐함수에 대해서도 벨만 최적 방정식을 표현할 수 있다.
최적 정책을 따라갈 때 현재 상태의 큐함수는 다음 상태에 선택 가능한
행동 중에서 가장 높은 값의 큐함수를 1번 감가하고 보상을 더한 것과 같다.
43. 벨만 기대 방정식과 벨만 최적 방정식을 이용해
MDP로 정의된 문제를 계산으로 푸는 방법인
다이내믹 프로그래밍