Digital Wellbeing Technology through a Social Semiotic Multimodal Lens: A Case Study

1. Omar Sosa-Tzec San Francisco State University,School of Design DigitalWellbeingTechnologythrougha SocialSemioticMultimodalLens A Case Study The 46th Annual Conference of the Semiotic Society of America October 15,2022–Virtual

2. Photo Credit: Nick Beardslee PersonalIntroduction • Designer • Background in Human-Computer Interaction Design,Information Design, and Computer Science • Assistant Professor of Visual Communication Design

3. Background

4. Photo Credit: Keira Burton SmartphoneasaCentralArtifactinModernSociety • Global communication and connectivity • Flexible work practices • Mobile entertainment • Ubiquitous access to services and products Cecchinato et al.(2019) Vanden Abeele and Nguyen (2022) Büchi (2021)

5. Photo Credit: CottonBro SmartphoneOveruseandWellbeing • Loneliness • Anxiety • Depression • Concentration issues • Sleep deprivation • Physical pain • Lack of boundaries between personal and professional life Bernroider, Krumay,and Marigold (2014)

6. Photo Credit: Monstera DigitalWellbeing • Emerging concept in humanities and social sciences that has now become a research domain addressed by scholars of multiple disciplines • Happiness (hedonia and eudaimonia) in social environments where digital media is omnipresent • A balanced relationship with technology: • Controlled pleasure • Maximizing the support to achieve goals • Minimizing loss of control and functional impairment Vanden Abeele (2021) Vanden Abeele and Nguyen (2022) Büchi (2021)

7. Photo Credit: Monstera DesignforDigitalWellbeing • “Interest in understanding how to support more meaningful interactions with technology that allow users to be in control and better self-manage their use to limit any potential negative e f fects and achieve their goals” • Designers in academia and industry create and evaluate digital tools or products that promote digital wellbeing • Design challenges introduced by the variety of occupations,personal preferences,and social contexts of use Cecchinato et al.(2019)

8. Howtodesigne f fectiveanddelightful digitalwellbeingtoolsandproducts? central question for digital wellbeing design practice

9. WhytoExploretheApplicationof SocialSemioticsandMultimodality?

10. Photo Credit: Anete Lusina Design,Semiotics,Rhetoric,andMultimodality • Design as a practice focused on communication connects directly with semiotics • Designers expect that signs will a f fect behavior,belief,or attitude in the users (viewers) • Design products are rei fi ed arguments by which designers attempt to prescribe how users should live the everyday life • Argument by experience: multimodal argumentation • A bridge between design and social semiotics and multimodality appears (for designers) from identifying design products in context as a source of meaning and experience Ehses (2008) Buchanan (2001) Groarke (2015)

11. Photo Credit: Anete Lusina Design,Semiotics,Rhetoric,andMultimodality •Design as a practice focused on communication connects directly with semiotics •Designers expect that signs will a f fect behavior,belief,or attitude in the users (viewers) •Design products are rei fi ed arguments by which designers attempt to prescribe how users should live the everyday life •Argument by experience: multimodal argumentation •A bridge between design and social semiotics and multimodality appears (for designers) from identifying design products in context as a source of design and experience Ehses (2008) Buchanan (2001) Groarke (2015) Nelson & Stolterman (2012) Kress (2010) Norman (2013) De Souza (2005) Fogg (2003) central question in design practice Whatarethedecisionsthatwillproducean e f fectiveanddelightfuldesign? Choice Resource Semiosis A f fect Persuasion Intent A f fordance Interfacesignembodiment Multimodality Interactionand UX

12. Photo Credit: Anete Lusina Design,Semiotics,Rhetoric,andMultimodality •Design as a practice focused on communication connects directly with semiotics •Designers expect that signs will a f fect behavior,belief,or attitude in the users (viewers) •Design products are rei fi ed arguments by which designers attempt to prescribe how users should live the everyday life •Argument by experience: multimodal argumentation •A bridge between design and social semiotics and multimodality appears (for designers) from identifying design products in context as a source of design and experience Ehses (2008) Buchanan (2001) Groarke (2015) embodied/embedded knowledge that can help promote best practices central question in design practice Whatarethedecisionsthatwillproducean e f fectiveanddelightfuldesign? Choice Resource Semiosis A f fect Persuasion Intent A f fordance Interfacesignembodiment Multimodality Interactionand UX Nelson & Stolterman (2012) Kress (2010) Norman (2013) De Souza (2005) Fogg (2003)

13. Doesasocialsemioticmultimodalapproachhelpcaptureand articulatedesignknowledgetopromotebestpracticesfor digitalwellbeingdesign? methodological question

14. CaseStudy

15. Photo Credit: little signals.withgoogle.com GoogleLittleSignals • Part of the Digital Wellbeing Experiments showcased on Experiments with Google website • Little Signals is a 6-object collection that explores“how to stay up-to-date with digital information while maintaining moments of calm” • They represent an application of the Calm Technology framework to digital wellbeing design Little Signals (2022) Weiser and Brown (1997)

16. Approach social semiotics multimodality •Inspection of semiotic resources: a f fordances and meaning potentials •Analysis of design: organization and dominance of semiotic resources •Analysis of intent: designer ’ s choice and context of use Semiotic Resources: material resources and immaterial conceptual resources Mode: socially organized set of semiotic resources for making meaning Jewitt,Bezemer,and O ’ Halloran (2016) Kress (2010) Kress and Van Leeuwen (2020) data •Separate Website: emphasis on introductory video and object gallery •Webpage on Digital Wellbeing Experiments (Experiments with Google) The e f fective communication of this project is possible due to all the content and its integration—text,image, sound,and interaction—hence it became necessary to consider all the content for analysis not just the artifacts—especially,given the lack of fi rst-hand user experience with the artifacts observations

17. Observations

18. StatedResourcesAppliedtoLittleSignals’Design Photo Credit: little signals.withgoogle.com Google Little Signals Blow Wiggle Spin Grow Chime Bounce Push Twist Move Graph Bounce Flip Breath Shake Stretch Rotate Knock Point Wave

19. IntroductoryVideo ’ sChoicesandMeaningPotentials storytelling choices •Depth •Proximity •Focus •Analogy meaning potentials •Unobtrusiveness •Invisibility •Ephemerality •Intimacy •Control •Familiarity •Signi fi cance Source: little signals.withgoogle.com

20. Identi fi edResourcesforDigitalWellbeingDesign Detachable Indivisible Modular Unibody Exact Prism Approximate Prism Geometric Organic Solid Hollow Uniform Long Compact Slim Form Body Shape Proportion Color Hue Saturation Value Material Tactility Opaqueness Rough Smooth Opaque Translucent Occurrence Direction Duration Speed Motion Periodic Episodic Unidirectional Multidirectional Linear Rotatory Brief Long Quick Slow Calm Technology Object for Digital Wellbeing Interactivity Action Engagement Sound Gentle Absent Tap Push Flip Observe Attend Ignore

21. Discussion

22. Photo Credit: little signals.withgoogle.com OnSocialSemioticsandMultimodalityasaCriticalFrameworkforDesigners • Social semiotics and multimodality provide (interface/experience/HCI) designers with a framework to think of digital wellbeing technology as multimodal artifact • The focus becomes the chosen resources,their integration,the meaning potentials that they could unfold during the user experience,and the signi fi cance of such meanings • This framework highlights the relation between creation, contextualized meaning,user experience,and agency from both designer and user • Designers would fi nd transductionas a key concept to understand the evolution and interrelation between design of di f ferent kinds

23. Photo Credit: little signals.withgoogle.com DrawbacksConcerningtheApplicationofSSM • A major drawback in the application of this framework is the circularity or unclear boundaries concerning the notions of resource and modes • It seems seems necessary to set the delimit of the notions and theory beforehand • Experiential knowledge derived from using the design being inspected becomes a crucial factor to respect the contextual dimension of meaning-making—in this case,the meaning produced during a real user experience

24. Photo Credit: little signals.withgoogle.com OnTechnologytoFixTechnologyandAttainingDigitalWellbeing • The project ’ s information online—especially the introductory video— works as a multimodal argument that presents a vision where (more) technology is the solution for the uncontrolled use of technology • Technology that still seeks to delight—just like any other kind of digital design—in order to work as a product • Technology that portrays technologizing home as a benign and desirable enterprise • Technology that suggests some technological,economic,and social requirements to live a happy life

25. Photo Credit: little signals.withgoogle.com NotableAspectsofGoogleLittleSignals • The project explores changes from the center and periphery as one way to engage with digital services and products o f f the screen • It is an e f fective demonstration of how the physicalization of data and material explorations can be applied in design for the everyday life • It promotes the idea of home as the space to nurture oneself and disconnect, • It also emphasizes being o f f the screen as a fundamental factor

26. Thankyou! tzec.com https://design.sfsu.edu/

Digital Wellbeing Technology through a Social Semiotic Multimodal Lens: A Case Study

Digital Wellbeing Technology through a Social Semiotic Multimodal Lens: A Case Study

Recommended

More Related Content

Similar to Digital Wellbeing Technology through a Social Semiotic Multimodal Lens: A Case Study

Similar to Digital Wellbeing Technology through a Social Semiotic Multimodal Lens: A Case Study(20)

More from Omar Sosa-Tzec

More from Omar Sosa-Tzec(20)

Recently uploaded

Recently uploaded(20)

Digital Wellbeing Technology through a Social Semiotic Multimodal Lens: A Case Study