SlideShare a Scribd company logo
To what extent will developments of
feedback in touchscreen devices impact its
future functionalities?
Extended Project Qualification (AQA Level III)
Harry Vigolo
10/19/2015
1
Table of contents
Table of contents………………………………………………………………………...…1
Abstract……………………………………………………………………………………..2
Introduction……………………………………………………………………………...…3
Microsoft and the Journey of Key-Click Feedback
- Introduction……………………………………………………………………..…...4
- Method………………………………………………………………………………6
- Results.........................................................................................................................7
Microsoft and Haptic Feedback Part II: SlickFeel
- Introduction………………………………………………………………………….8
- Method………………………………………………………………………………9
- Results……………………………………………………………………………….9
Final Conclusions…………………………………………………………………………..10
2
Abstract
Touchscreens have become a staple part of the way we work and interact with others. Through
touchscreen,we can have a larger screen space that can be used for both typing, and various other
uses, but so far,it has been a mainly two-dimensional experience where the actions on screen have not
matched the precision of manual inputs. Most of us continue to use computer keyboards over the ones
on our tablets because they provide more reliable and consistent typing, mainly due to the presence of
physical buttons which allow us to be precise yet swift in actions. This study examines the increasing
the level of haptic feedback,which is the sensory feedback received by our fingers as we type, using
small, inconspicuous actuators to deliver a much more enhanced responsiveness from the screens we
use, and has found that existing technology does indeed allow for much more than we currently have
whilst using our devices.
3
To what extent will developments of feedback in touchscreen devices impact its future
functionalities?
In every generation there exists a diverse amount of celebrated and acclaimed technological
innovations that have defined those eras,be it the invention of the lightbulb by Edison in the late
nineteenth century, or the discovery of penicillin by Alexander Fleming and sustained man made
flight from the Wright brothers in the early twentieth centuries, these revelations went on to define
their respective ages and create a lasting impression on ages to come. They also share similarities. For
example, most of the aforementioned innovations can be attributed to very few individuals, yet have
undergone countless modifications, to refine and perfect their technologies, bringing new uses and
comfort. Simple, yet highly important advancements such as the introduction of the inert gas to the
lightbulb bring us longer lasting light, the application of penicillin to discover new antibiotic remedies
have saved countless, and the jet engine has allowed quick and convenient trips. Although the starting
model remains essential, all the countless evolutions of such technology have impacted the world just
as much as the original. For reference,electricity is STILL produced, by and large, through the
Victorian age method of turning a big fan with steam to create an alternating current, yet we have
developed far more methods of creating steam other than burning coal.
From my point of view as an enthusiast in portable devices, and as a keen follower of recent
occurrences in the current technology world, I believe that the common touchscreen will be one of
those generation defining advancements,seeing new uses and adaptations in the coming decades.
Already, the onset of the touchscreen device has found incredible mainstream popularity, the
smartphone industry alone becoming worth $270 billion in the short space in between 2000 to 20151
,
with Apple Inc. making sales of nearly 170 million iPhones in 20142
.
The technology in question has just achieved its golden age, and today, it seems that many firms that
are already involved in the manufacture of touchscreen devices are keen to find the next big thing.
This is a time where the average smartphone is seen everywhere,even in the developing world due to
the appearance of low-cost smartphones and tablets, therefore it is not impossible to imagine a new
change that will revolutionise the way some use touchscreen. This study will focus on the research
behind the latest ideas in making users feel immersed in the experience that is touchscreen, may they
be from researchers,or the market leaders in tech, new applications and modifications to existing
touchscreen that will improve our experiences are being worked on as this is read,and perhaps in the
coming years it will manifest itself in our daily lives.
3
1
Statista.com, (2015). Global revenue from smartphones from 2008 to 2015 (in billion U.S. dollars). Retrieved June 23,
2015, from: http://www.statista.com/statistics/237505/global-revenue-from-smartphones-since-2008/
2 Statista.com, (2015). Global iPhone sales in the fiscal years 2007 to 2014 (in million units). Retrieved June 23, 2015, from:
http://www.statista.com/statistics/276306/global-apple-iphone-sales-since-fiscal-year-2007/
3
Google.com, (2015). Search: “Origin of word haptic.” Retrieved September 29, 2015, from:
https://www.google.co.uk/search?q=origin+of+word+haptic&oq=origin+of+word+haptic&aqs=chrome..69i57j0.9888j0j7&s
ourceid=chrome&es_sm=93&ie=UTF-8
4
Microsoft and the Journey of Key-Click Feedback
The word haptic originates from the classic Greek haptikos,meaning able to grasp or to touch and
when one thinks of touchscreen,their immediate reaction is to imagine the screen,and of course,its
ability to be operated using their sense of touch.
However the touchscreen is not particularly touchy at all. At least that’s what the ladies and
gentlemen at Microsoft’s research centres would suggest.
Many touchscreen smartphones have a system of what is called
haptic feedback in place. These tend to be crude implementations
of vibrations in the device to inform the user of a certain activated
function. A simple use of this is present on my old HTC Wildfire:
whenever any of the four ‘keys’ on the bottom of the device were
touched, the phone vibrated, alerting me that I had just pressed
something. But that was it. The vibration gave me no indication of
whether I had just touched the home button, or the menu button,
and the separate keys did not feeldifferent at all, as the glass was not
raised. Therefore by the second week of using the phone, I had already
deactivated not only these vibrations, but also the vibrations for the
keyboard. In my opinion, the functionality of such a feature was palpable at best, as the entire
smartphone was vibrating, limiting the use of having haptic feedback as a way to give the user more
information.
If separate,individual objects displayed on the screen could be identified through the sense of touch,
not only would it give greater user experience and comfort, but also open up a new way that we as
media consumers and workers use something such as smartphones and tablets. The haptic research
team at Microsoft have spent years developing their own breeds of haptic feedback, both of which
seek to enhance the interactions between user and device. In doing so, Microsoft hope to be able to
bring us into the world of faux 3D screens, to give us the ability to experience our sense of touch on a
touchscreen the same way we would do from pressing a button on a camera,or feeling the buttons on
a keyboard sink as we type.
This has become the main aim of Hong Z. Tan4
, senior researcher at Microsoft Research5
,and the
recent study undertaken on ‘Key-click Feedback Signals on a Virtual Keyboard.’6
In this study, the developers used existing and well received experimental results from a range of
earlier sources7 8 9 10
, which had previously attempted to create an immersive tactile feedback,
4
https://engineering.purdue.edu/~hongtan/
5 http://research.microsoft.com/en-us/news/features/haptics-080514.aspx
6 Jin R. K., Xiaowei D., Xiang C., Carl P., Desney T. and Hong. Z. T. (2012). A Masking Study of Key-Click
Feedback Signalson a Virtual Keyboard.Retrieved July 4, 2015, from:
http://research.microsoft.com/pubs/184101/C58_JRKim_etal_EH2012.pdf
ALL ARTICLES USED BY THE MICROSOFT RESEARCH TEAM:
7
Winfield, L., Glassmire, J., Colgate, J.E., Peshkin, M.:T-PaD:Tactile pattern display through variable friction reduction.
In: Proceedings of World Haptics Conference 2007,
pp. 421–426 (2007)
8
Bau, O., Poupyrev, I., Israr, A., Harrison, C.: TeslaTouch: Electrovibration for touch surfaces. In: Proceedings of UIST
2010, pp. 283–292 (2010)
9
TouchSense 1000 HapticSystem (January 30, 2012), http://www.immersion.com/products/touchsense-tactilefeedback/
Figure 1.
A HTC Wildfire, with its capacitive
touch buttons visible. 1
5
without vibrations permeating throughout the entire device, known as global feedback,or to
implement various levels of vibrations to simulate the difference in friction of some surfaces.
They state that there are many options available to achieve this, such as attaching actuators on the
surface of glass, instead of inside the circuitry to provide direct interactions to the touching finger, or
employing ultrasound/electrovibration to rapidly vibrate glass and therefore allow a greater level of
feedback. The limitations, and one of the major points that the team wished to improve upon was the
use of this technology with multiple fingers touching the device.
This is obviously influential in making the recent technology viable to implement on current devices,
as the vast majority of users type quickly on their keyboards, so any inaccuracies caused by the
limitations of hardware would be commonly seen and quite literally felt frequently. It had been seen
that these methods were unable to offer feedback to all fingers on the screen and for this reason
Microsoft made fixing this one of their priorities. This had been tried before by applying various
actuators to the screen at each key of the keyboard11
, or by keeping the area of tactic feedback
restricted to a small area12
. The drawback for the latter would be of decreased user experience,as a
number of the keys emitting feedback is lost, which was viewed as too great a loss.
Instead, the team focused on improving and manipulating the former idea of applying multiple
actuators below the surface,using a larger touchscreen to make up for the fact that each vibrating part
would be closer together. This is the most obvious limitation of such technique. With current
technology, parts that would vibrate and cause these key-click simulations are far too large to
implement numerously and effectively. There is also the large cost and efficiency of placing so many
small devices in a gadget, so the product may end up costing much more than a regular touchscreen,
affecting its total sales and the overall success of a release. However,there have been previous
innovations that have garnered much attention, such as the curved screens of some LG smartphones
and the Samsung Galaxy S6 Edge’s bent corners. Furthermore, Apple Inc. recently released their
Apple watch previously this year,which was extremely popular and used a similar set of vibrating
parts to create a tap-like sensation on one’s wrist whenever a notification was available. Although the
smartwatch was sold at the high retail price of US$350 in the US, there was significant demand for it,
swaying some doubts about whether a manufacturer would be able to afford such a product.
In the experiment, two identical simulators were constructed to mimic the keys of a zero-travel
keyboard which are commonly seen on smartphones; tablets etc. that emulated the tactile (felt)
feedback from a keyboard on something such as a laptop or PC. This was achieved by implementing
two-layered piezoelectric actuators into the surface. Piezoelectricity refers to the accumulated electric
charge ‘in certain solid materials in response to applied mechanical stresses.13
Once again,the word is
derived from Greek, piezo or piezein, meaning to squeeze or to press.
1000-series/index.html
CONT.
9
Poupyrev, I., Maruyama, S.: Tactile interfaces for small touch screens. In: Proceedings of UIST 2003, vol. 5, pp. 217–220
(2003)
11
Jansen, Y., Karrer, T., Borchers, J.: MudPad:localized tactile feedback on touch surfaces.
In: Adjunct Proceedings of UIST 2010, pp. 385–386 (2010)10
12
Luk, J., Pasquero, J., Little, S., MacLean, K., Levesque, V., Hayward, V.: A role for haptics
in mobile interaction: Initial design using a handheld tactile display prototype. In:Proceedings
of CHI 2006, pp. 171–180 (2006)
13 Piezoelectricity,From Wikipedia, the free encyclopaedia. Retrieved July 8, 2015, from:
https://en.wikipedia.org/wiki/Piezoelectricity
6
Figure 2; (b)
The two FSRs below each key,
measuring a 4mm diameter active
area taped on with clear tape.
Figure 2; (c)
The pink dot on each key.
Figure 2; (d)
A participant during testing: the plastic
container with opaque top and the foam
pad clearly visible.
This is incredibly important for the function of emulating the press of a
keyboard, whilst remaining efficient in its method of producing the
sensation of touch. As one could assume, using piezoelectric actuators
removes the need for a lot of electricity to be supplied, which could
conserve battery life as the charge is produced on demand from the simple
action of pressing the screen. As soon as the finger presses the key, the
force sensing resistors (FSRs) are triggered and a signal is delivered to
an appropriate actuator, allowing it to initiate its vibration, giving the
key-click feedback.
For each key, a two layered piezoelectric actuator was placed between
two clear plastic layers with two individual FSRs on one side. Figure 2;
(a) displays the piezoelectric actuator whilst Figure 2; (b) gives a view of
the two FSRs attached to the bottom of the key, two being to minimise
the chance of a press not being identified. A red dot was also placed on
top of each actuator/key which acted to help participants remain each
finger on the key (Figure 2; (c)).
Each key was then placed on top of a thick foam pad to isolate the active
vibrations of each actuator from the hard surface of the table and the two
keys were placed in a clear box with an opaque top and open front.
Allowing the user to place their wrist and hand on the table, yet not be
able to see the keys. This acted to prevent foreign interferences,whilst
each participant wore noise-reduction headphones, listening to pink noise
to avoid auditory cues from the moving parts.
The experiment carried out in the study covered a range of test subjects of
equal number male and female and an age of average 27 years. Of the 12
participants, 4 identified as left handed, whilst the other 8 were right
handed.
After each FSR was activated, the waveforms sent were independently
controlled by two output channels from a single soundcard (a device
which can be slotted into a computer to allow the use of audio components
for multimedia applications.14
) that went through a voltage amplifier with a
gain of 100, before reaching the piezoelectric actuators. When the
piezoelectric actuators received the sinusoidal pulse, which is a wave
frequency that oscillates like a sine/cosine curve, it emits a response that
feels like a ‘”crisp” key click.’ To measure the vibration speed and
frequency, the acceleration of the red dot per voltage (ms^-2V^-2) was
noted. Maximum acceleration was found to be 6.83 ms^-2, where each
voltage gain corresponded to +0.0683 ms^-2. Therefore the voltage of
100 is suitable to reach the desired acceleration.
14 As defined by Google.com;
https://www.google.co.uk/search?num=100&safe=off&espv=2&q=define+sound+card&spell=1&sa=X&ei=4U -
iVd-mKoLWU7fngfAH&ved=0CBsQBSgA&biw=1920&bih=955
Figure 2; (a)
The piezoelectric actuator, consisting of
a 22mm ceramic disk mounted on a
35mm metal disk.
7
For each participant, out of three cues,one included the
haptic feedback emanating from the makeshift key. If a
participant could not distinguish the key-click sensation
from the others, the next round included a stronger haptic
feedback. Likewise if it was identified, the participant
would receive a slightly lower strength feedback.
This was repeated until a precise threshold for
identification from each candidate was identified. For
example, on top, Figure 3 contains the results of trials on
one test subject. It is clearly shown that the level of
vibration required changing severaltimes until the average,
most accurate minimum threshold was identified, to identify the key-click on the finger and nowhere
else. This was carried out to ensure that the technology would be able to function not only with a wide
range of users, but also able to personalise itself to any person, and change the strength depending on
each user’s needs.
For this participant, the minimum required level of vibration was at 36.3 dB, but this varied amongst
the persons involved in the experiment. Results also differed, as each candidate used different fingers
from both hands, and eventually it was found that the sensitivity of the middle finger was higher than
the index when both fingers were being used during testing, and that it was harder to identify the
haptic feedback when only one hand with both fingers was being tested.
In light of the results from testing, it is shown that when using a device with both hands, a lower level
of vibration will be needed than with only one hand. This is because participants noted the vibration
as being easier to detect with inactive fingers that are not typing when using a single hand. This
dilemma is one of the main concerns surrounding the use of vibrating tactile feedback,and can be
explained due to an illusion, where different stimuli on two locations on the skin create the illusion of
there being a single stimulus in between both points of skin, known as the topographic location. This
limitation may have made it harder for participants to identify haptic feedback when they were using
two adjacent fingers from the same hand and therefore understandably manipulate the effectiveness of
implementation. However, I believe that if used correctly, it can be compensated for by simply
calculating the midpoint between two fingers, and instead creating feedback that would have a
topographic location on the finger that has actually pressed a button.
Yet another limitation involved is mentioned as being the way in which, only two fingers, one
receiving actual feedback,and the other a passive signal that emanates from the mechanism, were
used. In actual circumstances,one would be utilising a variety of fingers from both hands on a
keyboard, and therefore the results for only two fingers may predict, but not accurately measure the
overall impact of residual vibrations on surrounding fingers. In such occasions where double handed
typing is being used, then the keyboard may use a lower vibration, as shown in the conclusions, when
using both hands, the feedback is more discernible. The result of this would be that users would still
feel the appropriate amount of feedback, but not experience less “leaking” sensations from other
fingers.
The cost of implementing piezoelectric actuators may also be quite small, or large depending on the
model that is to be used. I have found that the larger and more specified the actuator is, the more
expensive they become, although it is according to my research that those of standard quality are
Figure 3
Test results for one participant, using
both index and middle finger on the
right hand (Middle finger receives
feedback.)
8
Figure 4.
The SlickFeel setup, clearly showing the
piezoelectric actuators surrounding the
back of the touchscreen, the larger ones
for click feedback and the smaller for
manipulating vibrations.
priced at $0.5-$5.0 each15
. I believe that depending on the amount of actuators used, it would add a
hefty sum to overall cost of production, as including disks to a tablet-sized screen would be a total of
up to 7x4x5=$140 per product. Obviously this will add far too much to the finished cost of each
device, but depending on the actuator used, the total cost of implementation could be as low as
7x4x0.5=$14. It is for this reason that I also question the affordability of this method of haptic
feedback,especially if it is to be used in smaller screens,where more,smaller piezoelectric actuators
must be utilised.
So far, I have explored simple uses of notifications, the feedback remains what it was before, only
feedback. It has been a way to give users more information regarding the actions of the device. While
it may improve efficiency of typing by simulating the real world sensation, and add to the immersive
experience that has become the aim of modern technology, it still fails to be elegant enough. A user is
still unable to type as well as they would when using a regular keyboard or playing a game that
involves a button. To emphasise, as I am rather unskilled at using a keyboard, typing still requires my
sense of touch. I rely on my fingers to be able to feel around the keys, using all sensory information
available to me and not just the click that occurs when I type. Being able to actually distinguish from
the sense of touch, all the individual keys whilst typing, is a great benefit that cannot be overstated to
the average user,and combined with the key-click feedback that Microsoft have developed could
possibly revolutionise the efficacy that feedback presents. Imagine a workplace where large tablets
are main-purpose computers, as there is no obvious drawback to using it. The screen is large enough
to support a full-range keyboard that types as quickly and effectively as a traditional one, and can
possibly be combined with a USB mouse, making the experience as seamless as possible. This is
beneficial as work could be done on the move, particularly in the U.S. where New York is due to
receive wide-ranging free,high speed Wi-Fi through the project LinkNYC.16 17
. Of course, replacing
full sized computers with touchscreen devices may take a while, however the same team at Microsoft
have been developing the complementary project to this technology.
The previous study had mentioned in its introduction various anterior publications of past uses of
using a variety of pre-calibrated vibrations. Microsoft decided to do their own experiments, hoping to
find a way to build on previous successes18
.
Microsoft and Haptic Feedback Part II: SlickFeel
Attempting to recreate the sensations of click using
vibrations had been tried before, in reference to past
papers looked at by Microsoft, but in this situation
actuators were applied to increasing or decreasing the
amount of friction felt by a user whilst sliding one’s
15 http://www.alibaba.com/product-detail/piezo-electric-ceramics-Piezo-multilayer-
actuator_2001485334.html?spm=a2700.7724857.35.1.bmYaHq
16Nate B.: New Yorkers to get free Wi-Fi via old phone booths in Google-funded project, 2015 Guardian News
and Media Limited. Retrieved 12 August,2015, from: http://www.theguardian.com/cities/2015/jun/29/new-
york-free-wi-fi-phone-booths-google-sidewalk-labs
17Link NYC Official Web Page: http://www.link.nyc/
18
Xiaowei D., Jiawei G., Xiang C., J. Ed C., and Hong Z. T. (2012), SlickFeel:Sliding and Clicking Haptic
Feedback on a Touchscreen,Retrieved September 10, 2015, from;
http://research.microsoft.com/apps/pubs/default.aspx?id=184102
9
finger over a screen. Just as a keyboard has gaps in between keys with no friction, the research project
centred on the ability to identify different areas of the screen using only the level of friction. This can
be deemed as the next step in making touchscreens feelmore lifelike and genuine whilst typing but at
the same time remain with a completely flat screen.
A Kindle Fire tablet was used during testing, suggesting that the vibrations can be easily applied to
other devices with large touchscreens. As a large basis of the testing was to try and reduce the time
spent by users looking at the screen instead of being able to do what many more skilled at typing do
naturally, which consists of using touch senses instead of vision.
Once again, piezoelectric actuators acted to produce the feedback, placing 4 on all vertices of the
tablet, known as “haptic units”. Each were made of 1.7mm glass and two sets of piezos glued to each
other, activated at around 30 kilohertz (kHz), just enough to vibrate the glass and influence the
amount of friction felt by the user from certain parts of the screen.
This is done by mapping where the finger is on the screen of the Kindle, which then sends information
over Wi-Fi to a local PC. The PC interprets the information and relays input signals via USB to a
driver board, which acts to amplify and control the signal sent to each actuator to produce the exact
amount of friction. Depending on the frequency of the vibrations created,a different level of drag
across the screen is formed. This is due to a very small amount of air trapped between the screen and
one’s finger creating the illusion of the screen feeling smoother or stickier depending on the
frequency. (No data was provided in the study regarding the frequencies used, only that by
manipulating the way the glass above the screen vibrates,it createsmore or less drag.)
The mechanism was then used to adapt rendered buttons and
other features on screen to contain different levels of
friction, which were then used to carry out tests on its
efficacy in improving user experiences. Two different
activities were prepared; one to distinguish between four
different types of virtual buttons and the other a simulated
typing exercise using the key-click haptic feedback
previously investigated.
The first test consisted of familiarising the users with
sensing the friction change throughout the screen itself and
contained four buttons, two ‘plastic’ and two ‘metal’ which
varied in style, one of each pair appearing flat and the other
of teach pair perceived to be raised. The raised buttons were
given a higher friction whilst the others were given less and
the metal buttons also appeared to be less sticky than their
plastic counterparts. The participants were asked to run their
finger over the different buttons until they became
accustomed to differentiating each individual button by
touch, and eventually each of the persons involved were
given the extra challenge of typing.
The simulated typing exercise was specially created for the
testing of how well friction worked on identifying the
separate keys on a screen,with one thumb on the left used to
cycle between letters of the alphabet in sets of three
consecutive letters by scrolling through the “ribbon.” This
ribbon had a higher level of friction compared to the rest of
the screen that allowed it to be identified without removing
Figure 5.
The buttons rendered on the screen of
the Kindle Fire Tablet. Each buttons has
a distinct feel and consist of 2 “metal”
and 2 “plastic” buttons.
Figure 6.
The simulated typing exercise, with the
letter scroll and buttons visible.
10
eyesight from the line that was being typed on. On the other side of the screen, the right thumb had
three “buttons” which acted to choose one of the three letters: top, middle, bottom, which also had a
higher friction so that they could be used without sight. To enforce this rule of no eye contact with the
input mechanism, the light grey outlines of the buttons and letter ribbon faded after 3 seconds of
continuous input.
These experiments also had no data to refer to. Unlike the Key-click feedback study, there exists no
information on the range of tested frequencies or concentration of actuators that were under the
screen,therefore making it harder to tell if this was difficult or costly to implement. We are only told
that the system was hooked up, by Wi-Fi to a PC, which sends data back via USB. This process would
drain the battery life of a device and become cumbersome if the USB cable were to be dangling from
the touchscreen to the PC. There was no typing error tally to test efficiency, nor a time trial to see how
utilising varying levels of friction on input could change the speed of typing on a normal keyboard.
Conclusions
Although the research paper concludes that “SlickFeel provides both sliding and clicking feedback on
a touchscreen through a single hardware setup.19
” This setup does not seem to be as refined and as the
previous study regarding emulating clicks on a keyboard, as the hardware is rather excessive and the
permanent need for Wi-Fi adds to the problem by requiring a constant connection. Although I feel that
this is a prominent issue with the technology, I have no doubt that continued efforts would allow for a
more compact and effective method of friction integration. After all, integration with a Kindle Fire
proves that it already functions well with existing technology, simply requiring few adaptations to
improve leaps and bounds. Both techniques of Key-Click feedback and SlickFeel have already been
demonstrated in a video published on the Microsoft Research website20
,showing a demo of all their
haptic projects over the years,one of which named “Electrostatic Haptics” which acts to use opposite
charges between the finger and the screen to attract them and create friction. Recently, different
developers such as Fujitsu21
have begun the process of creating their own haptic tablets that
implement different levels of friction, making the introduction of such technology all the more
possible.
Furthermore, in a boon to the possible implementation of SlickFeel and Key-Click feedback,the
release of the iPhone 6s has led to the introduction of 3D Touch, previously known as Force Touch
(As Huaweibeat Apple to the trademark) that existed on the Apple Watch. Force Touch was a method
by which pressing the screen harder would introduce new options, such as pressing with greater force
on the clock face to customise it to your liking, and was implemented into the iPhone 6s. On the
iPhone it is mainly used to bring extra options, as was done with the Apple Watch and another few
gimmicks, introducing a new haptics engine. As this technology is recent and patented, it is
impossible to find all information on how the technology works, but the patent includes much detail
on how it functions and is believed to have been in development since 200622
. For reference,I have
19
Xiaowei D., Jiawei G., Xiang C., J. Ed C., and Hong Z. T. (2012), SlickFeel:Sliding and Clicking Haptic
Feedback on a Touchscreen,Retrieved September 10, 2015, from;
http://research.microsoft.com/apps/pubs/default.aspx?id=184102
20Microsoft Research, (2014), Haptic Feedback at the Fingertips,Retrieved 13 September, 2015, from;
http://research.microsoft.com/apps/video/default.aspx?id=226096
21 Fujitsu Develops Prototype Haptic Sensory Tablet;
http://www.fujitsu.com/global/about/resources/news/press-releases/2014/0224-01.html
22 Patentlyapple.com, (2015), Apple's Force Touch for iPhone Invention was Published Today, Retrieved
September 29, 2015, from;
11
included the link to the official YouTube video.23
My interest is not to see how photos are enlarged by
pressing with force on the screen,but how Apple’s new technology can be combined with these
studies of haptics carried out by Microsoft Research.
Picture a touchscreen tablet, one which you could feel around the keyboard with, identifying all keys
by touch and dismissing the need to look. Next, as one begins to type, he feels the button click, and
click only once you have pressed the key with enough force to activate it. This is a perfect outcome.
With SlickFeel, the tablet will be able to reproduce the friction felt on the keys of a keyboard, and
how it disappears for a split second as you move to another. Although you may be in contact with the
screen, the keys are not activated and text is not typed until you have pressed with the force required,
which the 3D touch from the iPhone 6s has demonstrated works perfectly with no exemption. Finally,
as you click the key, the sensation of the key dropping then popping back up hits your finger to allow
you, the user, to continue typing when the letter has been confirmed as pressed. Overall, it creates a
lifelike simulation of a real keyboard, but without the lump of plastic mechanisms found on a physical
one. The buttons are easily identifiable and require force to be pressed, and finally the user themselves
receives confirmation, allowing them to know if a mistake has been made.
The result will be a function of the device that has no real operational flaws, although it may add a
few extra grams to the final product, due to SlickFeels rather rugged setup, the electrostatic haptics
introduced by Microsoft remove the need for external processing of data, while maintaining the
functionalities seen previously.
In the future, I sincerely hope to see a product retail which offers such benefits to the user. Most
consumers prefer original computers to large tablets as so far,they have been proven to be inefficient
and much more difficult to perform typing effectively. At the present time a set of virtual keys cannot
surpass physical keyboards, with a limited amount of sensory feedback, it is for this reason why at
tech conferences,instead of rows of journalists with tablets, portable computers are used instead. The
same is true for workplaces, whilst devices such as the Microsoft Surface that use a large touchscreen
are already on the market, few take advantage of the portability and power that comes with a full
Windows experience on what is effectively a tablet. They are built for on the go work and in tight
spaces such as public transport or conferences,where space is tight but are shunned for more
demanding, power hungry laptops.
The complete package may require a few more years of development, however at the rate the
technology industry has been releasing new ideas. The touchscreen is in its golden age of evolution
and it is possible that in a few years a larger sized species of tablet takes the world by storm with its
convincing haptic feedback, changing the places and methods we use touchscreen in the workplace,
homes and in schools as fully sized computers become a simple place for storage and high demand
processing power.
http://www.patentlyapple.com/patently-apple/2015/07/apples-force-touch-for-iphone-invention-was-published-
today.html
23 Apple, Introducing iPhone 6sand iPhone 6s Plus with 3D touch, (2015), Retrieved 13 September 2015, from:
https://www.youtube.com/watch?v=cSTEB8cdQwo
12
Bibliography
Jin R. K., Xiaowei D., Xiang C., Carl P., Desney T. and Hong. Z. T. (2012). A Masking Study of Key-Click
Feedback Signalson a Virtual Keyboard.Retrieved July 4, 2015, from:
http://research.microsoft.com/pubs/184101/C58_JRKim_etal_EH2012.pdf
Xiaowei D., Jiawei G., Xiang C., J. Ed C., and Hong Z. T. (2012), SlickFeel:Sliding and Clicking Haptic
Feedback on a Touchscreen,Retrieved September 10, 2015, from;
http://research.microsoft.com/apps/pubs/default.aspx?id=184102
Winfield, L., Glassmire, J., Colgate, J.E., Peshkin, M.: T-PaD: Tactile pattern display through variable friction
reduction. In: Proceedings of World Haptics Conference 2007,
pp. 421–426 (2007)
Bau, O., Poupyrev, I., Israr, A., Harrison, C.: TeslaTouch: Electrovibration for touch surfaces. In: Proceedings
of UIST 2010, pp. 283–292 (2010)
TouchSense 1000 Haptic System (January 30, 2012),
http://www.immersion.com/products/touchsensetactilefeedback/1000-series/index.html
Poupyrev, I., Maruyama, S.: Tactile interfaces for small touch screens.In: Proceedings of UIST 2003, vol. 5, pp.
217–220 (2003)
Jansen, Y., Karrer, T., Borchers, J.: MudPad: localized tactile feedback on touch surfaces.
In: Adjunct Proceedings of UIST 2010, pp. 385–386 (2010)10
Luk, J., Pasquero, J., Little, S., MacLean, K., Levesque, V., Hayward, V.: A role for haptics
in mobile interaction: Initial design using a handheld tactile display prototype.In: Proceedings
of CHI 2006, pp. 171–180 (2006)
Piezoelectricity,From Wikipedia, the free encyclopaedia. Retrieved July 8, 2015, from:
https://en.wikipedia.org/wiki/Piezoelectricity
Nate B.: New Yorkers to get free Wi-Fi via old phone booths in Google-funded project, 2015 Guardian News
and Media Limited. Retrieved 12 August,2015, from: http://www.theguardian.com/cities/2015/jun/29/new-
york-free-wi-fi-phone-booths-google-sidewalk-labs

More Related Content

What's hot

UX Flows - Skillshare
UX Flows - SkillshareUX Flows - Skillshare
UX Flows - Skillshare
Carl Nelson
 
Future of interface design 2010
Future of interface design 2010Future of interface design 2010
Future of interface design 2010
Pavel Růžička
 
Microsoft's HoloLens
Microsoft's HoloLensMicrosoft's HoloLens
Microsoft's HoloLens
Shoaib Belim
 
Virtual Reality: Creating Accessible Virtual Worlds
Virtual Reality: Creating Accessible Virtual WorldsVirtual Reality: Creating Accessible Virtual Worlds
Virtual Reality: Creating Accessible Virtual Worlds
Colson Rice
 
Hololens
HololensHololens
Hololens
aminpathan11
 
WassUp Recap - June 2012 - part 2
WassUp Recap -  June 2012 - part 2WassUp Recap -  June 2012 - part 2
WassUp Recap - June 2012 - part 2
LUON
 
LUON WassUp Recap March 2013 - 3. the mobile landscape
LUON WassUp Recap March 2013 - 3. the mobile landscapeLUON WassUp Recap March 2013 - 3. the mobile landscape
LUON WassUp Recap March 2013 - 3. the mobile landscape
LUON
 
Microsoft Hololens
Microsoft HololensMicrosoft Hololens
Microsoft Hololens
Sri Teja
 
Microinteractions
MicrointeractionsMicrointeractions
Microinteractions
Dan Saffer
 
HoloLens
HoloLensHoloLens
HoloLens
Rudra Bhatt
 
Microsoft HoloLens
Microsoft HoloLensMicrosoft HoloLens
Microsoft HoloLens
sahibe alam
 
Smartouch
SmartouchSmartouch
Smartouch
aido Cho
 
Presentation on HOLOLENS
Presentation on HOLOLENSPresentation on HOLOLENS
Presentation on HOLOLENS
Vishal Sharma
 
LinxFIT 7 GDK UX Tips
LinxFIT 7 GDK UX TipsLinxFIT 7 GDK UX Tips
LinxFIT 7 GDK UX Tips
Mauro Canziani
 
Microsoft holo lens
Microsoft holo lensMicrosoft holo lens
LUON WassUp recap - mar 2014 - 2. the mobile landscape
LUON WassUp recap  - mar 2014 - 2. the mobile landscapeLUON WassUp recap  - mar 2014 - 2. the mobile landscape
LUON WassUp recap - mar 2014 - 2. the mobile landscape
LUON
 
Hyper personalized spaces with @Octoblu
Hyper personalized spaces with @Octoblu Hyper personalized spaces with @Octoblu
Hyper personalized spaces with @Octoblu
Zachary Weiner
 
Designing Microinteractions for Better UX
Designing Microinteractions for Better UXDesigning Microinteractions for Better UX
Designing Microinteractions for Better UX
Justin Stollsteimer
 
Performics at CES: Day 2
Performics at CES: Day 2 Performics at CES: Day 2
Performics at CES: Day 2
Performics
 
Wearable Technology Part 1 Blue Paper
Wearable Technology Part 1 Blue Paper Wearable Technology Part 1 Blue Paper
Wearable Technology Part 1 Blue Paper
4imprint
 

What's hot (20)

UX Flows - Skillshare
UX Flows - SkillshareUX Flows - Skillshare
UX Flows - Skillshare
 
Future of interface design 2010
Future of interface design 2010Future of interface design 2010
Future of interface design 2010
 
Microsoft's HoloLens
Microsoft's HoloLensMicrosoft's HoloLens
Microsoft's HoloLens
 
Virtual Reality: Creating Accessible Virtual Worlds
Virtual Reality: Creating Accessible Virtual WorldsVirtual Reality: Creating Accessible Virtual Worlds
Virtual Reality: Creating Accessible Virtual Worlds
 
Hololens
HololensHololens
Hololens
 
WassUp Recap - June 2012 - part 2
WassUp Recap -  June 2012 - part 2WassUp Recap -  June 2012 - part 2
WassUp Recap - June 2012 - part 2
 
LUON WassUp Recap March 2013 - 3. the mobile landscape
LUON WassUp Recap March 2013 - 3. the mobile landscapeLUON WassUp Recap March 2013 - 3. the mobile landscape
LUON WassUp Recap March 2013 - 3. the mobile landscape
 
Microsoft Hololens
Microsoft HololensMicrosoft Hololens
Microsoft Hololens
 
Microinteractions
MicrointeractionsMicrointeractions
Microinteractions
 
HoloLens
HoloLensHoloLens
HoloLens
 
Microsoft HoloLens
Microsoft HoloLensMicrosoft HoloLens
Microsoft HoloLens
 
Smartouch
SmartouchSmartouch
Smartouch
 
Presentation on HOLOLENS
Presentation on HOLOLENSPresentation on HOLOLENS
Presentation on HOLOLENS
 
LinxFIT 7 GDK UX Tips
LinxFIT 7 GDK UX TipsLinxFIT 7 GDK UX Tips
LinxFIT 7 GDK UX Tips
 
Microsoft holo lens
Microsoft holo lensMicrosoft holo lens
Microsoft holo lens
 
LUON WassUp recap - mar 2014 - 2. the mobile landscape
LUON WassUp recap  - mar 2014 - 2. the mobile landscapeLUON WassUp recap  - mar 2014 - 2. the mobile landscape
LUON WassUp recap - mar 2014 - 2. the mobile landscape
 
Hyper personalized spaces with @Octoblu
Hyper personalized spaces with @Octoblu Hyper personalized spaces with @Octoblu
Hyper personalized spaces with @Octoblu
 
Designing Microinteractions for Better UX
Designing Microinteractions for Better UXDesigning Microinteractions for Better UX
Designing Microinteractions for Better UX
 
Performics at CES: Day 2
Performics at CES: Day 2 Performics at CES: Day 2
Performics at CES: Day 2
 
Wearable Technology Part 1 Blue Paper
Wearable Technology Part 1 Blue Paper Wearable Technology Part 1 Blue Paper
Wearable Technology Part 1 Blue Paper
 

Similar to EPQ Study(1)

IRJET - Touchless Technology
IRJET - Touchless TechnologyIRJET - Touchless Technology
IRJET - Touchless Technology
IRJET Journal
 
Mobile Phone Design - Lifecycle
Mobile Phone Design - LifecycleMobile Phone Design - Lifecycle
Mobile Phone Design - Lifecycle
Jaime Brown
 
Future of UX
Future of UXFuture of UX
Future of UX
Christopher Azar
 
Investigating_the_effectiveness_of_tactile_feedbac.pdf
Investigating_the_effectiveness_of_tactile_feedbac.pdfInvestigating_the_effectiveness_of_tactile_feedbac.pdf
Investigating_the_effectiveness_of_tactile_feedbac.pdf
EmiliaFelicia1
 
CES 2015: An Overview
CES 2015: An OverviewCES 2015: An Overview
CES 2015: An Overview
What's Next
 
7 top technology trends and what they mean for brand experience
7 top technology trends and what they mean for brand experience7 top technology trends and what they mean for brand experience
7 top technology trends and what they mean for brand experience
Jack Morton Worldwide
 
Touchscreen technology report
Touchscreen technology reportTouchscreen technology report
Touchscreen technology report
Vivektech
 
Microsoft Surface Report
Microsoft Surface ReportMicrosoft Surface Report
Microsoft Surface Report
piyush khadse
 
A critical research study on the impact of virtual reality.docx
A critical research study on the impact of virtual reality.docxA critical research study on the impact of virtual reality.docx
A critical research study on the impact of virtual reality.docx
patrickwaweru11
 
New Interactions
New InteractionsNew Interactions
New Interactions
nowtom
 
Sixth sense technology
Sixth sense technologySixth sense technology
Sixth sense technology
Niraj Bharambe
 
Being Human
Being HumanBeing Human
Being Human
Rahul Singh
 
An Overview of Interactive Surfaces: Applications, Sensors, and Dimensions
An Overview of Interactive Surfaces: Applications, Sensors, and DimensionsAn Overview of Interactive Surfaces: Applications, Sensors, and Dimensions
An Overview of Interactive Surfaces: Applications, Sensors, and Dimensions
gaup_geo
 
The Future of UX: What designers need to know to stay ahead
The Future of UX: What designers need to know to stay aheadThe Future of UX: What designers need to know to stay ahead
The Future of UX: What designers need to know to stay ahead
Doralin Kelly
 
New to the touch
New to the touchNew to the touch
New to the touch
R K Tiwari Sagar
 
2015 05 NO INTERFACE
2015 05 NO INTERFACE2015 05 NO INTERFACE
2015 05 NO INTERFACE
TrendWatching
 
Tecnology vocabulary
Tecnology vocabularyTecnology vocabulary
Tecnology vocabulary
jesusmlaz
 
HCI First Lecture.pptx
HCI First Lecture.pptxHCI First Lecture.pptx
HCI First Lecture.pptx
MuhammadFarhan947035
 
Windows Phone UX - a 101 @ BizSpark
Windows Phone UX - a 101 @ BizSparkWindows Phone UX - a 101 @ BizSpark
Windows Phone UX - a 101 @ BizSpark
Antony Ribot
 
HCI in Smart glasses
HCI in Smart glassesHCI in Smart glasses
HCI in Smart glasses
Pasquale Matrone
 

Similar to EPQ Study(1) (20)

IRJET - Touchless Technology
IRJET - Touchless TechnologyIRJET - Touchless Technology
IRJET - Touchless Technology
 
Mobile Phone Design - Lifecycle
Mobile Phone Design - LifecycleMobile Phone Design - Lifecycle
Mobile Phone Design - Lifecycle
 
Future of UX
Future of UXFuture of UX
Future of UX
 
Investigating_the_effectiveness_of_tactile_feedbac.pdf
Investigating_the_effectiveness_of_tactile_feedbac.pdfInvestigating_the_effectiveness_of_tactile_feedbac.pdf
Investigating_the_effectiveness_of_tactile_feedbac.pdf
 
CES 2015: An Overview
CES 2015: An OverviewCES 2015: An Overview
CES 2015: An Overview
 
7 top technology trends and what they mean for brand experience
7 top technology trends and what they mean for brand experience7 top technology trends and what they mean for brand experience
7 top technology trends and what they mean for brand experience
 
Touchscreen technology report
Touchscreen technology reportTouchscreen technology report
Touchscreen technology report
 
Microsoft Surface Report
Microsoft Surface ReportMicrosoft Surface Report
Microsoft Surface Report
 
A critical research study on the impact of virtual reality.docx
A critical research study on the impact of virtual reality.docxA critical research study on the impact of virtual reality.docx
A critical research study on the impact of virtual reality.docx
 
New Interactions
New InteractionsNew Interactions
New Interactions
 
Sixth sense technology
Sixth sense technologySixth sense technology
Sixth sense technology
 
Being Human
Being HumanBeing Human
Being Human
 
An Overview of Interactive Surfaces: Applications, Sensors, and Dimensions
An Overview of Interactive Surfaces: Applications, Sensors, and DimensionsAn Overview of Interactive Surfaces: Applications, Sensors, and Dimensions
An Overview of Interactive Surfaces: Applications, Sensors, and Dimensions
 
The Future of UX: What designers need to know to stay ahead
The Future of UX: What designers need to know to stay aheadThe Future of UX: What designers need to know to stay ahead
The Future of UX: What designers need to know to stay ahead
 
New to the touch
New to the touchNew to the touch
New to the touch
 
2015 05 NO INTERFACE
2015 05 NO INTERFACE2015 05 NO INTERFACE
2015 05 NO INTERFACE
 
Tecnology vocabulary
Tecnology vocabularyTecnology vocabulary
Tecnology vocabulary
 
HCI First Lecture.pptx
HCI First Lecture.pptxHCI First Lecture.pptx
HCI First Lecture.pptx
 
Windows Phone UX - a 101 @ BizSpark
Windows Phone UX - a 101 @ BizSparkWindows Phone UX - a 101 @ BizSpark
Windows Phone UX - a 101 @ BizSpark
 
HCI in Smart glasses
HCI in Smart glassesHCI in Smart glasses
HCI in Smart glasses
 

EPQ Study(1)

  • 1. To what extent will developments of feedback in touchscreen devices impact its future functionalities? Extended Project Qualification (AQA Level III) Harry Vigolo 10/19/2015
  • 2. 1 Table of contents Table of contents………………………………………………………………………...…1 Abstract……………………………………………………………………………………..2 Introduction……………………………………………………………………………...…3 Microsoft and the Journey of Key-Click Feedback - Introduction……………………………………………………………………..…...4 - Method………………………………………………………………………………6 - Results.........................................................................................................................7 Microsoft and Haptic Feedback Part II: SlickFeel - Introduction………………………………………………………………………….8 - Method………………………………………………………………………………9 - Results……………………………………………………………………………….9 Final Conclusions…………………………………………………………………………..10
  • 3. 2 Abstract Touchscreens have become a staple part of the way we work and interact with others. Through touchscreen,we can have a larger screen space that can be used for both typing, and various other uses, but so far,it has been a mainly two-dimensional experience where the actions on screen have not matched the precision of manual inputs. Most of us continue to use computer keyboards over the ones on our tablets because they provide more reliable and consistent typing, mainly due to the presence of physical buttons which allow us to be precise yet swift in actions. This study examines the increasing the level of haptic feedback,which is the sensory feedback received by our fingers as we type, using small, inconspicuous actuators to deliver a much more enhanced responsiveness from the screens we use, and has found that existing technology does indeed allow for much more than we currently have whilst using our devices.
  • 4. 3 To what extent will developments of feedback in touchscreen devices impact its future functionalities? In every generation there exists a diverse amount of celebrated and acclaimed technological innovations that have defined those eras,be it the invention of the lightbulb by Edison in the late nineteenth century, or the discovery of penicillin by Alexander Fleming and sustained man made flight from the Wright brothers in the early twentieth centuries, these revelations went on to define their respective ages and create a lasting impression on ages to come. They also share similarities. For example, most of the aforementioned innovations can be attributed to very few individuals, yet have undergone countless modifications, to refine and perfect their technologies, bringing new uses and comfort. Simple, yet highly important advancements such as the introduction of the inert gas to the lightbulb bring us longer lasting light, the application of penicillin to discover new antibiotic remedies have saved countless, and the jet engine has allowed quick and convenient trips. Although the starting model remains essential, all the countless evolutions of such technology have impacted the world just as much as the original. For reference,electricity is STILL produced, by and large, through the Victorian age method of turning a big fan with steam to create an alternating current, yet we have developed far more methods of creating steam other than burning coal. From my point of view as an enthusiast in portable devices, and as a keen follower of recent occurrences in the current technology world, I believe that the common touchscreen will be one of those generation defining advancements,seeing new uses and adaptations in the coming decades. Already, the onset of the touchscreen device has found incredible mainstream popularity, the smartphone industry alone becoming worth $270 billion in the short space in between 2000 to 20151 , with Apple Inc. making sales of nearly 170 million iPhones in 20142 . The technology in question has just achieved its golden age, and today, it seems that many firms that are already involved in the manufacture of touchscreen devices are keen to find the next big thing. This is a time where the average smartphone is seen everywhere,even in the developing world due to the appearance of low-cost smartphones and tablets, therefore it is not impossible to imagine a new change that will revolutionise the way some use touchscreen. This study will focus on the research behind the latest ideas in making users feel immersed in the experience that is touchscreen, may they be from researchers,or the market leaders in tech, new applications and modifications to existing touchscreen that will improve our experiences are being worked on as this is read,and perhaps in the coming years it will manifest itself in our daily lives. 3 1 Statista.com, (2015). Global revenue from smartphones from 2008 to 2015 (in billion U.S. dollars). Retrieved June 23, 2015, from: http://www.statista.com/statistics/237505/global-revenue-from-smartphones-since-2008/ 2 Statista.com, (2015). Global iPhone sales in the fiscal years 2007 to 2014 (in million units). Retrieved June 23, 2015, from: http://www.statista.com/statistics/276306/global-apple-iphone-sales-since-fiscal-year-2007/ 3 Google.com, (2015). Search: “Origin of word haptic.” Retrieved September 29, 2015, from: https://www.google.co.uk/search?q=origin+of+word+haptic&oq=origin+of+word+haptic&aqs=chrome..69i57j0.9888j0j7&s ourceid=chrome&es_sm=93&ie=UTF-8
  • 5. 4 Microsoft and the Journey of Key-Click Feedback The word haptic originates from the classic Greek haptikos,meaning able to grasp or to touch and when one thinks of touchscreen,their immediate reaction is to imagine the screen,and of course,its ability to be operated using their sense of touch. However the touchscreen is not particularly touchy at all. At least that’s what the ladies and gentlemen at Microsoft’s research centres would suggest. Many touchscreen smartphones have a system of what is called haptic feedback in place. These tend to be crude implementations of vibrations in the device to inform the user of a certain activated function. A simple use of this is present on my old HTC Wildfire: whenever any of the four ‘keys’ on the bottom of the device were touched, the phone vibrated, alerting me that I had just pressed something. But that was it. The vibration gave me no indication of whether I had just touched the home button, or the menu button, and the separate keys did not feeldifferent at all, as the glass was not raised. Therefore by the second week of using the phone, I had already deactivated not only these vibrations, but also the vibrations for the keyboard. In my opinion, the functionality of such a feature was palpable at best, as the entire smartphone was vibrating, limiting the use of having haptic feedback as a way to give the user more information. If separate,individual objects displayed on the screen could be identified through the sense of touch, not only would it give greater user experience and comfort, but also open up a new way that we as media consumers and workers use something such as smartphones and tablets. The haptic research team at Microsoft have spent years developing their own breeds of haptic feedback, both of which seek to enhance the interactions between user and device. In doing so, Microsoft hope to be able to bring us into the world of faux 3D screens, to give us the ability to experience our sense of touch on a touchscreen the same way we would do from pressing a button on a camera,or feeling the buttons on a keyboard sink as we type. This has become the main aim of Hong Z. Tan4 , senior researcher at Microsoft Research5 ,and the recent study undertaken on ‘Key-click Feedback Signals on a Virtual Keyboard.’6 In this study, the developers used existing and well received experimental results from a range of earlier sources7 8 9 10 , which had previously attempted to create an immersive tactile feedback, 4 https://engineering.purdue.edu/~hongtan/ 5 http://research.microsoft.com/en-us/news/features/haptics-080514.aspx 6 Jin R. K., Xiaowei D., Xiang C., Carl P., Desney T. and Hong. Z. T. (2012). A Masking Study of Key-Click Feedback Signalson a Virtual Keyboard.Retrieved July 4, 2015, from: http://research.microsoft.com/pubs/184101/C58_JRKim_etal_EH2012.pdf ALL ARTICLES USED BY THE MICROSOFT RESEARCH TEAM: 7 Winfield, L., Glassmire, J., Colgate, J.E., Peshkin, M.:T-PaD:Tactile pattern display through variable friction reduction. In: Proceedings of World Haptics Conference 2007, pp. 421–426 (2007) 8 Bau, O., Poupyrev, I., Israr, A., Harrison, C.: TeslaTouch: Electrovibration for touch surfaces. In: Proceedings of UIST 2010, pp. 283–292 (2010) 9 TouchSense 1000 HapticSystem (January 30, 2012), http://www.immersion.com/products/touchsense-tactilefeedback/ Figure 1. A HTC Wildfire, with its capacitive touch buttons visible. 1
  • 6. 5 without vibrations permeating throughout the entire device, known as global feedback,or to implement various levels of vibrations to simulate the difference in friction of some surfaces. They state that there are many options available to achieve this, such as attaching actuators on the surface of glass, instead of inside the circuitry to provide direct interactions to the touching finger, or employing ultrasound/electrovibration to rapidly vibrate glass and therefore allow a greater level of feedback. The limitations, and one of the major points that the team wished to improve upon was the use of this technology with multiple fingers touching the device. This is obviously influential in making the recent technology viable to implement on current devices, as the vast majority of users type quickly on their keyboards, so any inaccuracies caused by the limitations of hardware would be commonly seen and quite literally felt frequently. It had been seen that these methods were unable to offer feedback to all fingers on the screen and for this reason Microsoft made fixing this one of their priorities. This had been tried before by applying various actuators to the screen at each key of the keyboard11 , or by keeping the area of tactic feedback restricted to a small area12 . The drawback for the latter would be of decreased user experience,as a number of the keys emitting feedback is lost, which was viewed as too great a loss. Instead, the team focused on improving and manipulating the former idea of applying multiple actuators below the surface,using a larger touchscreen to make up for the fact that each vibrating part would be closer together. This is the most obvious limitation of such technique. With current technology, parts that would vibrate and cause these key-click simulations are far too large to implement numerously and effectively. There is also the large cost and efficiency of placing so many small devices in a gadget, so the product may end up costing much more than a regular touchscreen, affecting its total sales and the overall success of a release. However,there have been previous innovations that have garnered much attention, such as the curved screens of some LG smartphones and the Samsung Galaxy S6 Edge’s bent corners. Furthermore, Apple Inc. recently released their Apple watch previously this year,which was extremely popular and used a similar set of vibrating parts to create a tap-like sensation on one’s wrist whenever a notification was available. Although the smartwatch was sold at the high retail price of US$350 in the US, there was significant demand for it, swaying some doubts about whether a manufacturer would be able to afford such a product. In the experiment, two identical simulators were constructed to mimic the keys of a zero-travel keyboard which are commonly seen on smartphones; tablets etc. that emulated the tactile (felt) feedback from a keyboard on something such as a laptop or PC. This was achieved by implementing two-layered piezoelectric actuators into the surface. Piezoelectricity refers to the accumulated electric charge ‘in certain solid materials in response to applied mechanical stresses.13 Once again,the word is derived from Greek, piezo or piezein, meaning to squeeze or to press. 1000-series/index.html CONT. 9 Poupyrev, I., Maruyama, S.: Tactile interfaces for small touch screens. In: Proceedings of UIST 2003, vol. 5, pp. 217–220 (2003) 11 Jansen, Y., Karrer, T., Borchers, J.: MudPad:localized tactile feedback on touch surfaces. In: Adjunct Proceedings of UIST 2010, pp. 385–386 (2010)10 12 Luk, J., Pasquero, J., Little, S., MacLean, K., Levesque, V., Hayward, V.: A role for haptics in mobile interaction: Initial design using a handheld tactile display prototype. In:Proceedings of CHI 2006, pp. 171–180 (2006) 13 Piezoelectricity,From Wikipedia, the free encyclopaedia. Retrieved July 8, 2015, from: https://en.wikipedia.org/wiki/Piezoelectricity
  • 7. 6 Figure 2; (b) The two FSRs below each key, measuring a 4mm diameter active area taped on with clear tape. Figure 2; (c) The pink dot on each key. Figure 2; (d) A participant during testing: the plastic container with opaque top and the foam pad clearly visible. This is incredibly important for the function of emulating the press of a keyboard, whilst remaining efficient in its method of producing the sensation of touch. As one could assume, using piezoelectric actuators removes the need for a lot of electricity to be supplied, which could conserve battery life as the charge is produced on demand from the simple action of pressing the screen. As soon as the finger presses the key, the force sensing resistors (FSRs) are triggered and a signal is delivered to an appropriate actuator, allowing it to initiate its vibration, giving the key-click feedback. For each key, a two layered piezoelectric actuator was placed between two clear plastic layers with two individual FSRs on one side. Figure 2; (a) displays the piezoelectric actuator whilst Figure 2; (b) gives a view of the two FSRs attached to the bottom of the key, two being to minimise the chance of a press not being identified. A red dot was also placed on top of each actuator/key which acted to help participants remain each finger on the key (Figure 2; (c)). Each key was then placed on top of a thick foam pad to isolate the active vibrations of each actuator from the hard surface of the table and the two keys were placed in a clear box with an opaque top and open front. Allowing the user to place their wrist and hand on the table, yet not be able to see the keys. This acted to prevent foreign interferences,whilst each participant wore noise-reduction headphones, listening to pink noise to avoid auditory cues from the moving parts. The experiment carried out in the study covered a range of test subjects of equal number male and female and an age of average 27 years. Of the 12 participants, 4 identified as left handed, whilst the other 8 were right handed. After each FSR was activated, the waveforms sent were independently controlled by two output channels from a single soundcard (a device which can be slotted into a computer to allow the use of audio components for multimedia applications.14 ) that went through a voltage amplifier with a gain of 100, before reaching the piezoelectric actuators. When the piezoelectric actuators received the sinusoidal pulse, which is a wave frequency that oscillates like a sine/cosine curve, it emits a response that feels like a ‘”crisp” key click.’ To measure the vibration speed and frequency, the acceleration of the red dot per voltage (ms^-2V^-2) was noted. Maximum acceleration was found to be 6.83 ms^-2, where each voltage gain corresponded to +0.0683 ms^-2. Therefore the voltage of 100 is suitable to reach the desired acceleration. 14 As defined by Google.com; https://www.google.co.uk/search?num=100&safe=off&espv=2&q=define+sound+card&spell=1&sa=X&ei=4U - iVd-mKoLWU7fngfAH&ved=0CBsQBSgA&biw=1920&bih=955 Figure 2; (a) The piezoelectric actuator, consisting of a 22mm ceramic disk mounted on a 35mm metal disk.
  • 8. 7 For each participant, out of three cues,one included the haptic feedback emanating from the makeshift key. If a participant could not distinguish the key-click sensation from the others, the next round included a stronger haptic feedback. Likewise if it was identified, the participant would receive a slightly lower strength feedback. This was repeated until a precise threshold for identification from each candidate was identified. For example, on top, Figure 3 contains the results of trials on one test subject. It is clearly shown that the level of vibration required changing severaltimes until the average, most accurate minimum threshold was identified, to identify the key-click on the finger and nowhere else. This was carried out to ensure that the technology would be able to function not only with a wide range of users, but also able to personalise itself to any person, and change the strength depending on each user’s needs. For this participant, the minimum required level of vibration was at 36.3 dB, but this varied amongst the persons involved in the experiment. Results also differed, as each candidate used different fingers from both hands, and eventually it was found that the sensitivity of the middle finger was higher than the index when both fingers were being used during testing, and that it was harder to identify the haptic feedback when only one hand with both fingers was being tested. In light of the results from testing, it is shown that when using a device with both hands, a lower level of vibration will be needed than with only one hand. This is because participants noted the vibration as being easier to detect with inactive fingers that are not typing when using a single hand. This dilemma is one of the main concerns surrounding the use of vibrating tactile feedback,and can be explained due to an illusion, where different stimuli on two locations on the skin create the illusion of there being a single stimulus in between both points of skin, known as the topographic location. This limitation may have made it harder for participants to identify haptic feedback when they were using two adjacent fingers from the same hand and therefore understandably manipulate the effectiveness of implementation. However, I believe that if used correctly, it can be compensated for by simply calculating the midpoint between two fingers, and instead creating feedback that would have a topographic location on the finger that has actually pressed a button. Yet another limitation involved is mentioned as being the way in which, only two fingers, one receiving actual feedback,and the other a passive signal that emanates from the mechanism, were used. In actual circumstances,one would be utilising a variety of fingers from both hands on a keyboard, and therefore the results for only two fingers may predict, but not accurately measure the overall impact of residual vibrations on surrounding fingers. In such occasions where double handed typing is being used, then the keyboard may use a lower vibration, as shown in the conclusions, when using both hands, the feedback is more discernible. The result of this would be that users would still feel the appropriate amount of feedback, but not experience less “leaking” sensations from other fingers. The cost of implementing piezoelectric actuators may also be quite small, or large depending on the model that is to be used. I have found that the larger and more specified the actuator is, the more expensive they become, although it is according to my research that those of standard quality are Figure 3 Test results for one participant, using both index and middle finger on the right hand (Middle finger receives feedback.)
  • 9. 8 Figure 4. The SlickFeel setup, clearly showing the piezoelectric actuators surrounding the back of the touchscreen, the larger ones for click feedback and the smaller for manipulating vibrations. priced at $0.5-$5.0 each15 . I believe that depending on the amount of actuators used, it would add a hefty sum to overall cost of production, as including disks to a tablet-sized screen would be a total of up to 7x4x5=$140 per product. Obviously this will add far too much to the finished cost of each device, but depending on the actuator used, the total cost of implementation could be as low as 7x4x0.5=$14. It is for this reason that I also question the affordability of this method of haptic feedback,especially if it is to be used in smaller screens,where more,smaller piezoelectric actuators must be utilised. So far, I have explored simple uses of notifications, the feedback remains what it was before, only feedback. It has been a way to give users more information regarding the actions of the device. While it may improve efficiency of typing by simulating the real world sensation, and add to the immersive experience that has become the aim of modern technology, it still fails to be elegant enough. A user is still unable to type as well as they would when using a regular keyboard or playing a game that involves a button. To emphasise, as I am rather unskilled at using a keyboard, typing still requires my sense of touch. I rely on my fingers to be able to feel around the keys, using all sensory information available to me and not just the click that occurs when I type. Being able to actually distinguish from the sense of touch, all the individual keys whilst typing, is a great benefit that cannot be overstated to the average user,and combined with the key-click feedback that Microsoft have developed could possibly revolutionise the efficacy that feedback presents. Imagine a workplace where large tablets are main-purpose computers, as there is no obvious drawback to using it. The screen is large enough to support a full-range keyboard that types as quickly and effectively as a traditional one, and can possibly be combined with a USB mouse, making the experience as seamless as possible. This is beneficial as work could be done on the move, particularly in the U.S. where New York is due to receive wide-ranging free,high speed Wi-Fi through the project LinkNYC.16 17 . Of course, replacing full sized computers with touchscreen devices may take a while, however the same team at Microsoft have been developing the complementary project to this technology. The previous study had mentioned in its introduction various anterior publications of past uses of using a variety of pre-calibrated vibrations. Microsoft decided to do their own experiments, hoping to find a way to build on previous successes18 . Microsoft and Haptic Feedback Part II: SlickFeel Attempting to recreate the sensations of click using vibrations had been tried before, in reference to past papers looked at by Microsoft, but in this situation actuators were applied to increasing or decreasing the amount of friction felt by a user whilst sliding one’s 15 http://www.alibaba.com/product-detail/piezo-electric-ceramics-Piezo-multilayer- actuator_2001485334.html?spm=a2700.7724857.35.1.bmYaHq 16Nate B.: New Yorkers to get free Wi-Fi via old phone booths in Google-funded project, 2015 Guardian News and Media Limited. Retrieved 12 August,2015, from: http://www.theguardian.com/cities/2015/jun/29/new- york-free-wi-fi-phone-booths-google-sidewalk-labs 17Link NYC Official Web Page: http://www.link.nyc/ 18 Xiaowei D., Jiawei G., Xiang C., J. Ed C., and Hong Z. T. (2012), SlickFeel:Sliding and Clicking Haptic Feedback on a Touchscreen,Retrieved September 10, 2015, from; http://research.microsoft.com/apps/pubs/default.aspx?id=184102
  • 10. 9 finger over a screen. Just as a keyboard has gaps in between keys with no friction, the research project centred on the ability to identify different areas of the screen using only the level of friction. This can be deemed as the next step in making touchscreens feelmore lifelike and genuine whilst typing but at the same time remain with a completely flat screen. A Kindle Fire tablet was used during testing, suggesting that the vibrations can be easily applied to other devices with large touchscreens. As a large basis of the testing was to try and reduce the time spent by users looking at the screen instead of being able to do what many more skilled at typing do naturally, which consists of using touch senses instead of vision. Once again, piezoelectric actuators acted to produce the feedback, placing 4 on all vertices of the tablet, known as “haptic units”. Each were made of 1.7mm glass and two sets of piezos glued to each other, activated at around 30 kilohertz (kHz), just enough to vibrate the glass and influence the amount of friction felt by the user from certain parts of the screen. This is done by mapping where the finger is on the screen of the Kindle, which then sends information over Wi-Fi to a local PC. The PC interprets the information and relays input signals via USB to a driver board, which acts to amplify and control the signal sent to each actuator to produce the exact amount of friction. Depending on the frequency of the vibrations created,a different level of drag across the screen is formed. This is due to a very small amount of air trapped between the screen and one’s finger creating the illusion of the screen feeling smoother or stickier depending on the frequency. (No data was provided in the study regarding the frequencies used, only that by manipulating the way the glass above the screen vibrates,it createsmore or less drag.) The mechanism was then used to adapt rendered buttons and other features on screen to contain different levels of friction, which were then used to carry out tests on its efficacy in improving user experiences. Two different activities were prepared; one to distinguish between four different types of virtual buttons and the other a simulated typing exercise using the key-click haptic feedback previously investigated. The first test consisted of familiarising the users with sensing the friction change throughout the screen itself and contained four buttons, two ‘plastic’ and two ‘metal’ which varied in style, one of each pair appearing flat and the other of teach pair perceived to be raised. The raised buttons were given a higher friction whilst the others were given less and the metal buttons also appeared to be less sticky than their plastic counterparts. The participants were asked to run their finger over the different buttons until they became accustomed to differentiating each individual button by touch, and eventually each of the persons involved were given the extra challenge of typing. The simulated typing exercise was specially created for the testing of how well friction worked on identifying the separate keys on a screen,with one thumb on the left used to cycle between letters of the alphabet in sets of three consecutive letters by scrolling through the “ribbon.” This ribbon had a higher level of friction compared to the rest of the screen that allowed it to be identified without removing Figure 5. The buttons rendered on the screen of the Kindle Fire Tablet. Each buttons has a distinct feel and consist of 2 “metal” and 2 “plastic” buttons. Figure 6. The simulated typing exercise, with the letter scroll and buttons visible.
  • 11. 10 eyesight from the line that was being typed on. On the other side of the screen, the right thumb had three “buttons” which acted to choose one of the three letters: top, middle, bottom, which also had a higher friction so that they could be used without sight. To enforce this rule of no eye contact with the input mechanism, the light grey outlines of the buttons and letter ribbon faded after 3 seconds of continuous input. These experiments also had no data to refer to. Unlike the Key-click feedback study, there exists no information on the range of tested frequencies or concentration of actuators that were under the screen,therefore making it harder to tell if this was difficult or costly to implement. We are only told that the system was hooked up, by Wi-Fi to a PC, which sends data back via USB. This process would drain the battery life of a device and become cumbersome if the USB cable were to be dangling from the touchscreen to the PC. There was no typing error tally to test efficiency, nor a time trial to see how utilising varying levels of friction on input could change the speed of typing on a normal keyboard. Conclusions Although the research paper concludes that “SlickFeel provides both sliding and clicking feedback on a touchscreen through a single hardware setup.19 ” This setup does not seem to be as refined and as the previous study regarding emulating clicks on a keyboard, as the hardware is rather excessive and the permanent need for Wi-Fi adds to the problem by requiring a constant connection. Although I feel that this is a prominent issue with the technology, I have no doubt that continued efforts would allow for a more compact and effective method of friction integration. After all, integration with a Kindle Fire proves that it already functions well with existing technology, simply requiring few adaptations to improve leaps and bounds. Both techniques of Key-Click feedback and SlickFeel have already been demonstrated in a video published on the Microsoft Research website20 ,showing a demo of all their haptic projects over the years,one of which named “Electrostatic Haptics” which acts to use opposite charges between the finger and the screen to attract them and create friction. Recently, different developers such as Fujitsu21 have begun the process of creating their own haptic tablets that implement different levels of friction, making the introduction of such technology all the more possible. Furthermore, in a boon to the possible implementation of SlickFeel and Key-Click feedback,the release of the iPhone 6s has led to the introduction of 3D Touch, previously known as Force Touch (As Huaweibeat Apple to the trademark) that existed on the Apple Watch. Force Touch was a method by which pressing the screen harder would introduce new options, such as pressing with greater force on the clock face to customise it to your liking, and was implemented into the iPhone 6s. On the iPhone it is mainly used to bring extra options, as was done with the Apple Watch and another few gimmicks, introducing a new haptics engine. As this technology is recent and patented, it is impossible to find all information on how the technology works, but the patent includes much detail on how it functions and is believed to have been in development since 200622 . For reference,I have 19 Xiaowei D., Jiawei G., Xiang C., J. Ed C., and Hong Z. T. (2012), SlickFeel:Sliding and Clicking Haptic Feedback on a Touchscreen,Retrieved September 10, 2015, from; http://research.microsoft.com/apps/pubs/default.aspx?id=184102 20Microsoft Research, (2014), Haptic Feedback at the Fingertips,Retrieved 13 September, 2015, from; http://research.microsoft.com/apps/video/default.aspx?id=226096 21 Fujitsu Develops Prototype Haptic Sensory Tablet; http://www.fujitsu.com/global/about/resources/news/press-releases/2014/0224-01.html 22 Patentlyapple.com, (2015), Apple's Force Touch for iPhone Invention was Published Today, Retrieved September 29, 2015, from;
  • 12. 11 included the link to the official YouTube video.23 My interest is not to see how photos are enlarged by pressing with force on the screen,but how Apple’s new technology can be combined with these studies of haptics carried out by Microsoft Research. Picture a touchscreen tablet, one which you could feel around the keyboard with, identifying all keys by touch and dismissing the need to look. Next, as one begins to type, he feels the button click, and click only once you have pressed the key with enough force to activate it. This is a perfect outcome. With SlickFeel, the tablet will be able to reproduce the friction felt on the keys of a keyboard, and how it disappears for a split second as you move to another. Although you may be in contact with the screen, the keys are not activated and text is not typed until you have pressed with the force required, which the 3D touch from the iPhone 6s has demonstrated works perfectly with no exemption. Finally, as you click the key, the sensation of the key dropping then popping back up hits your finger to allow you, the user, to continue typing when the letter has been confirmed as pressed. Overall, it creates a lifelike simulation of a real keyboard, but without the lump of plastic mechanisms found on a physical one. The buttons are easily identifiable and require force to be pressed, and finally the user themselves receives confirmation, allowing them to know if a mistake has been made. The result will be a function of the device that has no real operational flaws, although it may add a few extra grams to the final product, due to SlickFeels rather rugged setup, the electrostatic haptics introduced by Microsoft remove the need for external processing of data, while maintaining the functionalities seen previously. In the future, I sincerely hope to see a product retail which offers such benefits to the user. Most consumers prefer original computers to large tablets as so far,they have been proven to be inefficient and much more difficult to perform typing effectively. At the present time a set of virtual keys cannot surpass physical keyboards, with a limited amount of sensory feedback, it is for this reason why at tech conferences,instead of rows of journalists with tablets, portable computers are used instead. The same is true for workplaces, whilst devices such as the Microsoft Surface that use a large touchscreen are already on the market, few take advantage of the portability and power that comes with a full Windows experience on what is effectively a tablet. They are built for on the go work and in tight spaces such as public transport or conferences,where space is tight but are shunned for more demanding, power hungry laptops. The complete package may require a few more years of development, however at the rate the technology industry has been releasing new ideas. The touchscreen is in its golden age of evolution and it is possible that in a few years a larger sized species of tablet takes the world by storm with its convincing haptic feedback, changing the places and methods we use touchscreen in the workplace, homes and in schools as fully sized computers become a simple place for storage and high demand processing power. http://www.patentlyapple.com/patently-apple/2015/07/apples-force-touch-for-iphone-invention-was-published- today.html 23 Apple, Introducing iPhone 6sand iPhone 6s Plus with 3D touch, (2015), Retrieved 13 September 2015, from: https://www.youtube.com/watch?v=cSTEB8cdQwo
  • 13. 12 Bibliography Jin R. K., Xiaowei D., Xiang C., Carl P., Desney T. and Hong. Z. T. (2012). A Masking Study of Key-Click Feedback Signalson a Virtual Keyboard.Retrieved July 4, 2015, from: http://research.microsoft.com/pubs/184101/C58_JRKim_etal_EH2012.pdf Xiaowei D., Jiawei G., Xiang C., J. Ed C., and Hong Z. T. (2012), SlickFeel:Sliding and Clicking Haptic Feedback on a Touchscreen,Retrieved September 10, 2015, from; http://research.microsoft.com/apps/pubs/default.aspx?id=184102 Winfield, L., Glassmire, J., Colgate, J.E., Peshkin, M.: T-PaD: Tactile pattern display through variable friction reduction. In: Proceedings of World Haptics Conference 2007, pp. 421–426 (2007) Bau, O., Poupyrev, I., Israr, A., Harrison, C.: TeslaTouch: Electrovibration for touch surfaces. In: Proceedings of UIST 2010, pp. 283–292 (2010) TouchSense 1000 Haptic System (January 30, 2012), http://www.immersion.com/products/touchsensetactilefeedback/1000-series/index.html Poupyrev, I., Maruyama, S.: Tactile interfaces for small touch screens.In: Proceedings of UIST 2003, vol. 5, pp. 217–220 (2003) Jansen, Y., Karrer, T., Borchers, J.: MudPad: localized tactile feedback on touch surfaces. In: Adjunct Proceedings of UIST 2010, pp. 385–386 (2010)10 Luk, J., Pasquero, J., Little, S., MacLean, K., Levesque, V., Hayward, V.: A role for haptics in mobile interaction: Initial design using a handheld tactile display prototype.In: Proceedings of CHI 2006, pp. 171–180 (2006) Piezoelectricity,From Wikipedia, the free encyclopaedia. Retrieved July 8, 2015, from: https://en.wikipedia.org/wiki/Piezoelectricity Nate B.: New Yorkers to get free Wi-Fi via old phone booths in Google-funded project, 2015 Guardian News and Media Limited. Retrieved 12 August,2015, from: http://www.theguardian.com/cities/2015/jun/29/new- york-free-wi-fi-phone-booths-google-sidewalk-labs