Neuroscience Meets Cryptography:        Designing Crypto Primitives Secure Against Rubber Hose Attacks       Hristo Bojino...
ments compared to untrained, the participant validates              lower bound the entropy of the learned secrets, is sim...
pend on this type of implicit, non-conscious learning sys-        intercept every object as it nears the sink. Interceptio...
as a secret key is increased exponentially from only 256to nearly 248 billion, as explained in the next section.Second, we...
SISL Authentication. To authenticate at a later time, a               • Usability: can a trained user complete the authent...
sequence as familiar at an average of 6.5 (SE 0.4) on the                                                                 ...
Figure 4: Across training participants gradually beginto express knowledge of the repeating sequence by ex-hibiting a perf...
us was to price the initial (much lengthier) part at $2.00                                              Submissionsand the...
of participants in a cryptographic protocol are as enti-           phases, alternating between extraction and testing, lat...
it can be difficult for a human attacker to memorize an             be detained for questioning and the legitimate user ise...
Participants were again recruited through Mechanical             We leave it for future work to design a coersion-resistan...
Deniable encryption. In the context of encryption, de-            guishable to an attacker versus a legitimate authenticat...
[6] Julien Doyon, Pierre Bellec, Rhonda Amsel,                  [17] D. Sanchez, E. Gobel, and P. Reber. Performing the   ...
Upcoming SlideShare
Loading in …5

Neuroscience Meets Cryptography: Designing Crypto Primitives Secure Against Rubber Hose Attacks


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Neuroscience Meets Cryptography: Designing Crypto Primitives Secure Against Rubber Hose Attacks

  1. 1. Neuroscience Meets Cryptography: Designing Crypto Primitives Secure Against Rubber Hose Attacks Hristo Bojinov Daniel Sanchez, Paul Reber Dan Boneh Patrick Lincoln Stanford University Northwestern University Stanford University SRI Abstract feat cryptography [22]. The problem is that an authen- ticated user must possess authentication credentials andCryptographic systems often rely on the secrecy of cryp- these credentials can be extracted by force [19] or bytographic keys given to users. Many schemes, however, other means.cannot resist coercion attacks where the user is forciblyasked by an attacker to reveal the key. These attacks, In this work we present a new approach to preventingknown as rubber hose cryptanalysis, are often the easiest rubber hose attacks using the concept of implicit learn-way to defeat cryptography. We present a defense against ing [5, 17] from cognitive psychology. Implicit learn-coercion attacks using the concept of implicit learning ing is believed to involve the part of the brain called thefrom cognitive psychology. Implicit learning refers to basal ganglia that learns tasks such as riding a bicycle orlearning of patterns without any conscious knowledge of playing golf by repeatedly performing those tasks. Ex-the learned pattern. We use a carefully crafted computer periments designed to trigger implicit learning show thatgame to plant a secret password in the participant’s brain knowledge learned this way is not consciously accessiblewithout the participant having any conscious knowledge to the person being trained [17]. An everyday exampleof the trained password. While the planted secret can of this phenomenon is riding a bicycle: we know how tobe used for authentication, the participant cannot be co- ride a bicycle, but cannot explain how we do it. Section 2erced into revealing it since he or she has no conscious gives more background of the relevant neuroscience.knowledge of it. We performed a number of user studies Implicit learning presents a fascinating tool for design-using Amazon’s Mechanical Turk to verify that partici- ing coercion-resistant security systems. In this paper wepants can successfully re-authenticate over time and that focus on user authentication where implicit learning isthey are unable to reconstruct or even recognize short used to plant a password in the human brain that can befragments of the planted secret. detected during authentication, but cannot be explicitly described by the user. Such a system avoids the problem that people can be persuaded to reveal their password.1 Introduction To use this system, participants would be initially trained to do a specific task called Serial Interception SequenceConsider the following scenario: a high security facility Learning (SISL), described in the next section. Trainingemploys a sophisticated authentication system to check is done using a computer game that results in implicitthat only persons who know a secret key, possess a hard- learning of a specific sequence of key strokes that func-ware token, and have an authorized biometric can enter. tions as an authentication password. In our experiments,Guards ensure that only people who successfully authen- training sessions last approximately 30 to 45 minutes andticate can enter the facility. Now, suppose a clever at- participants learn a random password that has about 38tacker captures an authenticated user. The attacker can bits of entropy. We conducted experiments to show thatsteal the user’s hardware token, fake the user’s biomet- after training, participants cannot reconstruct the trainedrics, and coerce the victim into revealing his or her secret sequence and cannot even recognize short fragments ofkey. At this point the attacker can impersonate the victim it.and defeat the expensive authentication system deployed To be authenticated at a later time, a participant is pre-at the facility. sented with multiple SISL tasks where one of the tasks So-called rubber hose attacks have long been the bane contains elements from the trained sequence. By ex-of security systems and are often the easiest way to de- hibiting reliably better performance on the trained ele- 1
  2. 2. ments compared to untrained, the participant validates lower bound the entropy of the learned secrets, is simplehis or her identity within 5 to 6 minutes. An attacker to set up, and is designed to leave no conscious trace ofwho does not know the trained sequence cannot exhibit the trained sequences.the user’s performance characteristics measured at the User studies. To validate our proposal we performedend of training. Note that the authentication procedure a number of user studies using Amazon’s Mechanicalis an interactive game in which the server knows the Turk. We asked the following core questions that exploreparticipant’s secret training sequence and uses it to au- the feasibility of authentication via implicit learning:thenticate the participant. Readers who want to playwith the system can check out the training game at • Is individual identification reliable? That is, trained users re-authenticate and can they do it over While in this paper we focus on coercion-resistant time?user authentication systems, authentication is just the tip • Can an attacker reverse engineer the sequence fromof the iceberg. We expect that many other coercion- easily obtained performance data from a trained par-resistant security primitives can be designed using im- ticipant?plicit learning. Across three experiments, we present promising initialThreat model. The proposed system is designed to be results supporting the practical implementation of ourused as a local password mechanism requiring physical design. First, we show that identification is possiblepresence. That is, we consider authentication at the en- with relatively short training and a simple test. Second,trance to a secure location where a guard can ensure that the information learned by the user persists over delaysa real person is taking the test without the aid of any elec- of one and two weeks: while there is some forgettingtronics. over a week, there is little additional forgetting at two To fool the authentication test the adversary is allowed weeks suggesting a long (exponentially shaped) forget-to intercept one or more trained users and get them to re- ting curve. Finally, in a third experiment we examined anveal as much as they can, possibly using coercion. Then attack based on having participants complete sequencesthe adversary, on his own, engages in the live authentica- containing all minimal-length fragments needed to try totion test and his goal is to pass the test. reconstruct the identification sequence: our results show We stress that as with standard password authentica- that participants do not express reliable sequence knowl-tion, the system is not designed to resist eavesdropping edge under this condition, indicating that the underlyingattacks such as shoulder surfing during the authentica- sequence information is resistant to attack until longertion process. While challenge-response protocols are a subsequences are guessed correctly by the attacker.standard defense against eavesdropping, it is currentlyan open problem to design a challenge-response proto-col based on implicit learning. We come back to this 2 An Overview of the Human Memoryquestion at the end of the paper. SystemBenefits over biometric authentication. The trained The difference between knowing how to perform a well-secret sequence can be thought of as a biometric key learned skill and being able to explain that performanceauthenticating the trained participant. However, unlike is familiar to anyone who has acquired skilled expertise.biometric keys the authenticating information cannot be This dissociation reflects the multiple memory systemssurreptitiously duplicated and participants cannot reveal in the human brain [14]. Memory for verbally reportablethe trained secret even if they want to. In addition, if facts, events and episodes depends on the medial tem-the trained sequence is compromised, a new identifying poral lobe memory system (including the hippocampus).sequence can be trained as a replacement, resulting in a Damage to this system due to stroke, Alzheimer’s dis-change of password. ease neuropathology, or aging leads to impairments in We discuss other related work in Section 6, but briefly conscious, explicit memory. However, patients with im-mention here a related result of Denning et al. [4] that pairments to explicit memory often show an intact abil-uses images to train users to implicitly memorize pass- ity to acquire new information implicitly, including ex-words. This approach is not as resistant to rubber hose hibiting normal learning of several kinds of skills. Theattacks since users will remember images they have seen types of learning preserved in memory-disordered pa-versus ones they have not, giving an attacker informa- tients are those learned incidentally through practice:tion that can be used for authentication. Additionally, even in healthy participants the information thus ac-image-based methods require large sets of images to be quired cannot be easily verbally described.prepared and used only once per user making the system Several decades of experimental cognitive psychologydifficult to deploy. Our combinatorial approach lets us have led to the development of tasks that selectively de- 2
  3. 3. pend on this type of implicit, non-conscious learning sys- intercept every object as it nears the sink. Interceptiontem. These tasks typically present information covertly is performed by pressing the key that corresponds to thewith embedded structure in a set of experimental stim- object’s column when the object is in the correct verti-uli. Although participants are not attempting to learn this cal position. Pressing the wrong key or not pressing anystructure, evidence for learning can be observed in their key results in an incorrect outcome for that object. Inperformance. a typical training session of 30-60 minutes, participants The covertly embedded information often takes the complete several thousand trials and the order of the cuesform of a statistical structure to a sequence of responses. follows a covertly embedded repeating sequence on 80%Participants exhibit improved performance when the re- of trials. The game is designed to keep each user at (butsponses follow this sequence and performance declines not beyond) the limit of his or her abilities by graduallyif the structure is changed [12]. The improvement in varying the speed of the falling circles to achieve a hitperformance can occur completely outside of awareness, rate of about 70%. Knowledge of the embedded repeat-that is, participants do not realize there is any structure ing sequence is assessed by comparing the performancenor can they recognize the structure when shown [17]. rate (percent correct) during times when the cues followThe lack of awareness of learning indicates the mem- the trained sequence to that during periods when the cuesory system supporting learning is not part of the explicit, follow an untrained sequence.declarative memory system and instead is hypothesized All of the sequences presented to the user are de-to depend on the basal ganglia and connections to motor signed to prevent conspicuous, easy to remember pat-cortical areas [6]. terns from emerging. Specifically, training as well as Less is known about the information processing char- random sequences are designed to contain every orderedacteristics of the cortico-striatal memory system oper- pair of characters exactly once with no character appear-ating in the connections between the basal ganglia and ing twice in a row, and thus the sequence length mustmotor cortical areas. Most prior research has examined be 4 × 3 = 12 when four columns (characters) are used.learning of simple structures with small amounts of in- The result is that while the trained sequence is performedformation, typically repeating sequences of actions 10- better than an untrained sequence, the participant usually12 items in length. However, more recent studies have does not consciously recognize the trained sequence. Infound that long, complex sequences can be learned fairly order to confirm this in experimental work, after SISLrapidly by this memory system and that learning is rela- participants are typically asked to complete tests of ex-tively unaffected by noise [18]. The ability to learn re- plicit recognition in which they specify how familiar var-peating sequences that are at least 80 items long rela- ious sequences look to them.tively rapidly and the fact that this training can be hid-den within irrelevant responses (noise) during trainingsuggests an intriguing possibility for covertly embeddingnon-reportable cryptographic data within the cortico-striatal memory system in the human brain.2.1 The SISL Task and AppletThe execution of the Serial Interception Sequence Learn-ing (SISL) task is central to the authentication systemthat we have developed. Here we introduce the SISL taskin the context of the human memory system in order toprovide background for describing our design and prac-tical experiments. Figure 1: Screenshot of the SISL task in progress. Originally introduced in [17], SISL is a task inwhich human participants develop sensitivity to struc-tured information without being aware of what they have For the current application, we extended the traditionallearned. The task requires participants to intercept mov- definition of the SISL task in order to accommodate itsing objects (circles) delivered in a pre-determined se- use as an authentication mechanism. First, we increasedquence, much like this is done in the popular game “Gui- the number of columns to six, which increases the poten-tar Hero”. Initially each object appears at the top of one tial complexity of the trained sequence. Using the sameof four different columns, and falls vertically at a con- constraints on sequence order as the 4-column version ofstant speed until it reaches the “sink” at the bottom, at the task, the training sequences are 30 items long. As awhich point it disappears. The goal for the player is to result, the number of possible sequences that can be used 3
  4. 4. as a secret key is increased exponentially from only 256to nearly 248 billion, as explained in the next section.Second, we added an empty column in the middle of thelayout (Figure 1). In early experimental testing we foundout that the empty column facilitates the visual percep-tion of the falling objects and helps the user to “map”them to the correct hand, especially for objects in themiddle columns which are otherwise easily confused athigh speed. The SISL task is delivered to users as a Flash appli-cation via a web browser. Participants navigate to ourweb site,, and are presented with aconsent form. Once they agree to participate, the ap-plet downloads a random training sequence and starts thegame. Upon completion of the training and test trials, theexplicit recognition test is administered, and results are Figure 2: The secret key we generate is a random 30-uploaded to the server. Once we describe our authenti- character sequence from the set of Euler cycles in thiscation system, we will return to describe how the SISL directed graph. The resulting sequence contains all bi-applet functions in the bigger scheme of our experiments grams exactly once, excluding repeating characters.with multiple users. which gives3 The Basic Authentication System Using #keys = 64 · 246 ≈ 237.8 . Implicit Learning Hence the learned random secret has about 38 bits ofThe SISL task provides a method for storing a secret key entropy which is far more than the entropy of standardwithin the human brain that can be detected during au- memorized passwords.thentication, but cannot be explicitly described by theuser. Such a system avoids the problem that people can Training. Users learn a random 30-item secret key k ∈ Σbe persuaded to reveal their password and can form the by playing the SISL game in a trusted environment. Tobasis of a coercion-resistant authentication protocol. If train users we experimented with the following proce-the information is compromised, a new identifying se- dure:quence can be trained as a replacement—resulting in a • While performing the SISL task the trainee is pre-change of password. sented with the 30-item secret key sequence re- The identification system operates in two steps: train- peated three times followed by 18 items selecteding followed by authentication. In the training phase, the from a random other sequence (subject to the con-secret key learned by the user is as in the expanded SISL straint that there will be no back-to-back repetitionstask, namely a sequence of 30 characters over the set of the same cue), for a total of 108 items.S = {s, d, f , j, k, l}. We only use 30-character sequencesthat correspond to an Euler cycle in the graph shown in • This sequence is repeated five times, so that theFigure 2 (i.e. a cycle where every edge appears exactly trainee is presented with a total of 540 items.once). These sequences have the property that every non- • At the end of this sequence there is a short pause inrepeating bigram over S (such as ‘sd’, ‘dj’, ’fk’) appears the SISL game and then the entire sequence of 540exactly once. In order to anticipate the next item (e.g., to items (including the pause at the end) is repeated sixshow a performance advantage), it is necessary to learn more times.associations among groups of three or more items. Thiseliminates learning of letter frequencies or common pairs During the entire training session the trainee is presentedof letters, which reduces conscious recognition of the with 7 × 540 = 3780 items which takes approximatelyembedded repeating sequence [5]. 30-45 minutes to complete. After the training phase Let Σ denote the set of all possible secret keys, namely completes, the trainee runs through the authenticationthe set of 30-character sequences corresponding to Eu- test described next to ensure that training succeeded.ler cycles in Figure 2. The number of Euler cycles in The system records the final playing speed that the userthis graph can be computed using the BEST theorem [20] achieved. 4
  5. 5. SISL Authentication. To authenticate at a later time, a • Usability: can a trained user complete the authenti-trained user is presented with the SISL game where the cation task reliably over time?structure of the cues contains elements from the trainedauthentication sequence and untrained elements for com- • Security: can an attacker who intercepts a trainedparison. By exhibiting reliably better performance on user coerce enough information out of the user tothe trained elements compared to untrained, the partic- properly authenticate?ipant validates his or her identity. Specifically we exper-imented with the following authentication procedure: 4 Usability Experiments • Let k0 be the trained 30-item sequence and let k1 , k2 be two additional 30-item sequences chosen at ran- We report on preliminary experiments that demonstrate dom from Σ. The same sequences (k0 , k1 , k2 ) are feasibility and promise of the SISL authentication sys- used for all authentication sessions. tem. We carried out the experiments in three stages. First, we established that reliable learning was observed • The system chooses a random permutation π with the new expanded version of the SISL task using of (0, 1, 2, 0, 1, 2) (e.g., π = (2, 1, 0, 0, 2, 1) ) and Mechanical Turk. Second, we verified that users retain presents the user with a SISL game with the fol- the knowledge of the trained sequence after delays of one lowing sequence of 540 = 18 × 30 items: and two weeks. Finally, we investigated the effectiveness of an attack on participants’ sequence knowledge based kπ1 , kπ1 , kπ1 , . . . , kπ6 , kπ6 , kπ6 . on sampling the smallest fragments from which the orig- That is, each of k0 , k1 , k2 is shown to the user ex- inal sequence could potentially be reconstructed. actly six times (two groups of three repetitions), but The experiments were carried out online within Ama- ordering is random. The game begins at the speed zon’s Mechanical Turk platform. The advantages of Me- at which the training for that user ended. chanical Turk involve a practically unlimited base of par- ticipants, and a relatively low cost. One drawback of run- • For i = 0, 1, 2 let pi be the fraction of correct keys ning the experiments online is the relative lack of control the user entered during all plays of the sequence ki . we had over users coming back at a later time for repeat The system declares that authentication succeeded evaluations. We discuss all of these considerations to- if wards the end of the section. p0 > average(p1 , p2 ) + σ (3.1) Where σ > 0 is sufficiently large to minimize the 4.1 Experiment 1: Implicit and Explicit possibility that this gap occurred by chance, but without causing authentication failures. Learning Our first experiment confirmed that implicit learning can In the above, preliminary formulation, the authenti- be clearly detected while explicit conscious sequencecation process is potentially vulnerable to an attack by knowledge was minimal. Experimental data from 35 par-which an untrained user degrades his performance across ticipants were included in the analysis.two blocks hoping to exhibit an artificial performance The experiment used the training procedure describeddifference in favor of the trained sequence (and obtain- in the previous section where the training phase con-ing a 1/3 chance of passing authentication). We discuss tained 3780 total trials and took approximately 30-45a robust defense against this in Section 5, but for now minutes to complete. Recall that training consists ofwe mention that two simple precautions offer some pro- seven 540-trial training blocks. After the training ses-tection, even for this simple assessment procedure. First, sion, participants completed a SISL authentication testverifying that the authenticator is a live human makes it that compares performance on the trained sequence todifficult to consistently change performance across the performance on two random test sequences.foil blocks k1 , k2 . Second, the final training speed ob- Learning of the trained sequence is shown in Figure 3tained during acquisition of the sequence is known to as a function of the performance advantage (increase inthe authentication server and the attacker is unlikely to percent correct responses) for the trained sequence com-match that performance difference between the trained pared with the randomly occurring noise segments. Onand foil blocks. A performance gap that is substantially the test block following training, participants performeddifferent from the one obtained after training indicates an the SISL task at an average rate of 79.2% correct forattack. the trained sequence and 70.6% correct for the untrained sequence. The difference of 8.6% correct (SE 2.4%)1Analysis. The next two sections discuss two critical as-pects of this system: 1 SE is short hand for Standard Error. 5
  6. 6. sequence as familiar at an average of 6.5 (SE 0.4) on the 0-10 scale and rated novel untrained sequences at 5.15 (SE 0.3). The modestly higher recognition of the trained sequence was reliable across the group, t(34) = 3.69, p < .01, but did not correlate with SISL performance (r = 0.13) indicating that it did not contribute to the im- plicit test. Slightly higher recognition of the trained se- quence is often seen in implicit learning experiments as healthy participants find some parts of the training se- quence familiar after practice. It is worth noting thatFigure 3: Across training participants gradually begin implicit memory does not transform into explicit knowl-to express knowledge of the repeating sequence by ex- edge, even with repeated use, and the structure and lengthhibiting a performance advantage for the trained se- of the training and test sequences specifically aim to re-quence compared to randomly interspersed noise seg- duce the possibility that explicit knowledge is accumu-ments. Note that overall performance on the task stays lated over around 70% throughout due to the adaptive nature of The general small difference in recognition ratingsthe task by which the speed is increased as participants (5.15 vs. 6.5) indicates that participants would not bebecome better at general SISL performance. able to recall the 30-item sequence meaning that they could not consciously produce the training information (e.g. to compromise the security of the authentication method). One participant remarked in a follow-up emailindicated reliably better performance for the trained se- message:quence. By one-sample t-test versus zero, the expecteddifference between trained and untrained if there was no “... To be honest I was not that sure of the quizzeslearning2 would be t(34) = 3.55, p < .01. at the end. When I played the tempo was so high it Group-level differences in performance are commonly was incredibly difficult to keep a track of the circles.seen on tests of implicit learning, but being able to reli- Most of the time my fingers moved by themselves, atably assess individual learning is necessary for an au- least it felt that way. I noticed two repeating pat-thentication method. On an individual participant ba- terns over all the levels. (I’m not totally sure whatsis, performance on the trained sequence could be dis- the buttons were, was it DFG JKL?) One was D-criminated from the untrained sequence on the 540 test F-G-F-D I think and the other I’m not quite suretrials (by chi-squared analysis at p < .05) in 25 of 35 the sequence but it was a four or five button seriescases. For authentication purposes, the individual relia- which went from the left to the right and back to thebility of the assessment will need to be further improved left...”.by longer training to establish the implicitly learned se-quence. However, the ability to identify learning in a We discuss the reconstruction question further in ourlarge fraction of individuals with relatively short train- third is a feature of the SISL task not seen in most tests ofimplicit learning. 4.2 Experiment 2: Recall Over TimeExplicit recognition test. After the training and test An authentication mechanism is only useful if authen-blocks, participants were presented with five different an- tication can still be accurately performed at some timeimated sequences and asked how familiar each looked on after the password is memorized. In Experiment 2,a scale of 0 to 10). Of the five sequences, one was the we confirmed that sequence-specific knowledge acquiredtrained sequence and the other four were randomly se- by users was retained over prolonged periods of time.lected foils. This test assessed explicit recognition mem- Although skill learning generally persists over time, aory for the trained sequence. SISL-based test had never been conducted with a sub- On the recognition test, participants rated the trained stantial delay and a sufficient number of participants. 2 In other words, if the percent correct measurements for trained In Experiment 2, participants agreed to perform theand untrained sequences followed the same normal distribution, the t- SISL task over two sessions. In the first session, par-value calculated with N = 35 samples (and thus N − 1 = 34 degrees ticipants completed a training sequence which the sameof freedom), should be near zero—less than 3.55 with 99% probability structure as the one in Experiment 1. The training was(p = 0.01); in contrast, the value we obtained was 8.6. The t-test is astandard statistical method used to confirm that the manipulated vari- immediately followed by the same SISL test to assessable (here, sequence type) affects the measured variable (performance sequence knowledge before the delay. A group of 32correct). participants returned to the online applet after 1 week to 6
  7. 7. Figure 4: Across training participants gradually beginto express knowledge of the repeating sequence by ex-hibiting a performance advantage for the trained se- Figure 5: Participants exhibited reliable sequence knowl-quence compared to randomly interspersed noise seg- edge on both immediate assessments (shown for Exper-ments. Learning performance was similar across both iment 1 and both conditions of Experiment 2) shown bygroups and similar to Experiment 1, as expected. a performance advantage for the trained sequence com- pared with untrained, novel sequences at test. Sequence knowledge is retained at both the 1 and 2 week delay test sessions. While there is some reduction in expressedperform a retention test and recognition assessment for knowledge after either delay, the lack of significant ad-the trained sequence. A separate group of 80 participants ditional decay from 1 to 2 weeks suggests that informa-returned after a 2 week delay for the retention and recog- tion is likely to persist for significant periods following 2nition tests. For the 1-week group, the test session con- weeks (exponential or power-law decay curves are com-sisted of a 540-trial implicit sequence learning assess- monly observed for many types of memory).ment. For the 2-week group, the test session was doubledin length to additionally evaluate whether a longer testprovided better sensitivity to individual sequence knowl-edge. For both groups, the initial speed of the test on the ability to recall the entire 30-item trained sequences.delay session was set to match the speed with which theparticipants had been performing the task at the end ofthe training session. A short warm-up block of 180 trials 4.3 Mechanical Turkwas used to adjust this initial speed so that participants Running our experiments over Mechanical Turk requiredwere performing at around the target 70% correct at the considerable thought and effort to ensure that the experi-beginning of the retention test. ments do not suffer from selection bias and are conducted Figure 4 shows gradual learning of the trained se- fairly for both participants and researchers.quence during the first session for both groups as in Ex- One of the early initial questions was that of setting theperiment 1. Implicit sequence knowledge at both im- price for user participation. The training block, whichmediate and delayed tests is shown in Figure 5. On comprises the bulk of the initial session, takes approxi-all five assessments, participants exhibited reliable se- mately 30-40 minutes to complete depending on playerquence learning as a group, ts > 4.3, ps < .01. On the skill. We wanted to motivate our participants to per-one-week delay test, 15 of 32 participants exhibited in- form to the best of their abilities, and thus set a pricedividually reliable sequence knowledge. However, for of $5.00 for standalone sessions, assuming a total ofthe two-week delay group, 49 of 80 participants exhib- approximately one hour of work involved. Apart fromited reliable sequence knowledge reflecting the increased isolated complaints from users who thought the gamesensitivity in the longer assessment test used. Future moved too slowly (likely due to them not pressing keys,research will examine both increased training time and or playing incorrectly), most users were happy to partic-assessment tests with increased sensitivity to individual ipate and even solicited additional work. We defined ourknowledge to provide a reliable and accurate identifica- HIT (Human Intelligence Task) such that each workertion method by SISL performance. could participate only once in it and we believe that there Even at one and two weeks delay, participants exhib- were few—if any—cases where the same user submittedited the same modest tendency for better recognition of multiple responses.the trained sequence, ts > 2.8, ps < .05. Again, recog- We had to design special incentives for participants tonition performance did not correlate with expression of return and complete the second part in the case of two-sequence knowledge, rs < .16 and did not suggest any session experiments. The approach that worked well for 7
  8. 8. us was to price the initial (much lengthier) part at $2.00 Submissionsand the follow-up 15-minute session at $6.00. We also Experiment Part All Paid Usedexplained clearly that this is a two-HIT sequence, and baseline 46 39 34that payment for both parts will only be processed once 1 week delay initial 35 32 32the second part is done. No-shows at the second ses- 1 week delay follow-up 45 32 32sion would get no payment at all. Additionally we used 2 week delay initial 100 95 (a) 82Amazon’s command line tools to automatically send re- 2 week delay follow-up 111 84 (b) 82minders to participants when the second session was trigrams 37 34 32available and due. As a result, we saw over 90% of thepeople who completed the first session return and finishthe second part. Table 1: Total number of participants in each experi- Due to the special requirements of the SISL applica- ment. The higher number of submissions on follow-uption we had to create what is considered to be an “ex- session are due to more failed opportunistic attempts byternal HIT”, exposing the task as a public website. In users to get paid $6.00 for no work because HIT assign-order to make sure that results submitted in Amazon cor- ments were remaining available longer, waiting for eli-respond to valid submissions in our system, we designed gible users to show up. Notes: (a) we paid more peoplea system that involves a receipt code for every success- than necessary due to the 16-day auto-approval config-fully completed session. The code is a 6-digit number uration of the HIT; (b) we paid, but did not evaluate abetween 100000 and 999999—we chose this size to pre- submission which came in after the cut-off time; (c) thevent people from easily guessing the code, but not make variation in number of participants across experimentsit difficult for them to write it down (especially useful was due to varying response and acceptance rates—ourin two-session experiments, where we also have to fetch primary goal was to collect enough data to be able tothe correct follow-up sequence that matches the user’s make statistical inferences, and we deliberately collectedfirst visit). After follow-up sessions we provided the user more data for the most difficult experiment (the 2-weekwith a second code that needed to be submitted to the delay).separate second HIT in order to receive payment. Naturally we were concerned about the security of our organization of the Mechanical Turk system is at leastsystem, so we took measures to only accept limited types partially to thank: workers need to register, and provideof input as parameters, leaving the website open mostly some sort of payment account which makes their identityto denial of service attacks which we had no reason to relatively easy to track; moreover, rejected work nega-expect. In comparison, our fear of legitimate users trying tively affects a worker’s score and as a result most usersto cheat the system and getting paid without completing genuinely try to do the best they can, get entertainedquality work was somewhat more justified. We saw some if possible, and earn some extra money in the instances of behavior in this category: Overall, we consider our use of Mechanical Turk to have • There were users who, against the instructions, sub- been a big success: it allowed us to conduct each exper- mitted an invalid receipt code. We immediately re- iment practically overnight, drawing on the huge avail- jected any such submissions. able pool of participants. • Some users submitted sequences that were so long that they did not fit in our generous allowance on the 5 Security Analysis server. Upon examination we found out that these were due primarily to excessive wrong key presses In this section we analyze the security of the basic au- (sometimes 5 or more key presses for the same ob- thentication protocol from Section 3 and propose a num- ject, which suggests that possibly an automated tool ber of extensions that improve security. We also experi- was used to complete the task). ment with a particular attack that attempts to extract the secret sequence from the user one fragment at a time. • In relatively few situations we noticed users who Our Mechanical Turk experiment shows that this attack had unusually long intervals of inactivity. We ex- works poorly on humans. cluded the most outrageous submissions but leaned towards including the rest in the results of the study in order to avoid biasing our data towards people 5.1 Implicit Learning as a Cryptographic who did well. Primitive The scope of these abuses never amounted to more We begin with an abstract model of the new function-than 5% of the submissions, and we believe that the ality enabled by implicit learning. Traditional modeling 8
  9. 9. of participants in a cryptographic protocol are as enti- phases, alternating between extraction and testing, laterties who hold secrets unknown to the adversary. These on in this section.assumptions fall apart in the face of coercion since all We also note that the basic threat model assumes thatsecrets can be extracted from the participant. during the training phase, when users are taught the cre- Implicit learning provides the following new abstract dential, users are following the instructions and are notfunctionality: the training phase embeds a predicate deliberately trying to mislead the training process. In ef- fect, the adversary is only allowed to coerce a user after p : Σ → {0, 1} the training process the user’s brain for some large set Σ. Anyone can ask It is straight-forward to show that the system of Sec-the user to evaluate his or her predicate p at a point k ∈ tion 3 is secure under this basic threat model, assum-Σ. The predicate evaluates to 1 when k has been learned ing the training procedure embeds an implicitly learnedby the user and evaluates to 0 otherwise. The number predicate p in the user’s brain. Indeed, if the attackerof inputs at which p evaluates to 1 is relatively small. intercepts u trained users and subjects each one to qMost often p will only evaluate to 1 at a single point queries, his chances of finding a valid sequence is atmeaning that the user has been trained on only one secret most qu/|Σ|. Since each test takes about five minutes,sequence. we can assume an upper bound of q = 105 trials per The key feature of implicit learning is that even under captured user (this amounts to about one year of non-duress it is impossible to extract a point k ∈ Σ from the stop testing per user which will either interfere with theuser for which p(k) = 1. This abstract property captures user’s learned password rendering the user useless tothe fact that the secret sequence k is implicitly learned by the attacker, or alert security administrators due to thethe user and not consciously accessible. In this paper, we user’s absence prompting a revocation of the creden-use the implicit learning primitive to construct an authen- tials). Hence, even after capturing u = 100 users, thetication system, but one can imagine it being used more attacker’s success probability is onlybroadly in security systems. The authentication procedure described in Section 3 100 × 105 /|Σ| ≈ 2−16 .provides an implementation of the predicate p(·) forsome sequence k0 in Σ. If the procedure declares suc- Further complicating the attacker’s life is the fact thatcess we say that p(k0 ) = 1 and otherwise p(k0 ) = 0. The subjecting a person to many random SISL games maypredicate p is embedded in the user’s brain during the obliterate the learned sequence or cause the person totraining session. learn an incorrect sequence thereby making extraction impossible.The basic coercion threat model. The SISL authenti- We note that physical presence is necessary in authen-cation system from Section 3 is designed to resist an ad- tication systems designed to resist coercion attacks. Ifversary who tries to fool the authentication test. We as- the system supported remote authentication then an at-sume the test requires physical presence and begins with tacker could coerce a trained user to authenticate to a re-a liveness check to ensure that a real person is taking the mote server and then hijack the session.test without the aid of any instruments. To fool the au-thentication test the adversary is allowed the following Security enhancements. The security model abovesequence of steps: gives the attacker one chance to authenticate and the at- tacker must succeed with non-negligible probability. If • Extraction phase: intercept one or more trained the attacker is allowed multiple authentication attempts users and get them to reveal as much as they can, — iterating the extraction and test phases, alternating be- possibly using coercion. tween the two — then the protocol may become insecure. The reason is that during an authentication attempt the at- • Test phase: the adversary, on his own, submits to tacker sees the three sequences k0 , k1 , k2 and could mem- the authentication test and his or her goal is to pass orize one of them (30 symbols). He would then train the test. In real life this could mean that the adver- offline on that sequence so that at the next authentica- sary shows up at the entrance to a secure facility and tion attempt he would have a 1/3 chance in succeeding. attempts to pass the authentication test there. If he If the attacker could memorize all three sequences (90 fails he could be detained for questioning. symbols), he could offline subject a trained user to all three sequences and reliably determine which is the cor- This basic threat model gives the attacker a single rect one and then train himself on that sequence. He ischance at the authentication test. We consider a model then guaranteed success at the next authentication trial.where the attacker may iterate the extraction and test We note that this attack is non-trivial to pull off since 9
  10. 10. it can be difficult for a human attacker to memorize an be detained for questioning and the legitimate user isentire sequence at the speed the game is played. blocked from authenticating with the system until he or Another potential attack, already discussed in Sec- she is retrained on a new set of sequences.tion 3, is an attacker who happens to be an expert player,but deliberately degrades his performance on two of the Eavesdropping security. Traditional password authen-sequences presented. With probability 1/3 he will show tication is vulnerable to eavesdropping (either via client-a performance gap on the correct sequence and pass the side malware or shoulder surfing) and so is the authenti-authentication test. We described a number of defenses cation system presented here. An eavesdropper who ob-in Section 3. Here we describe a more robust defense. tains a number of valid authentication transcripts with a trained user will be able to reconstruct the learned se- Both attacks above can be defeated with combina- quence(s). It is a fascinating direction for future researchtorics. Instead of training the user on a single sequence, to devise a coercion-resistant system where an implicitlywe train the user on a small number of sequences, say learned secret is used in a challenge-response protocolfour. Experiments [18] suggest that the human brain can with the server. We come back to this question at the endlearn multiple sequences and these learned sequences do of the paper.not interfere with one another. Equivalently we couldtrain the user on a longer sequence and use its fragmentsduring authentication. While this will increase training 5.2 An Experiment: Extracting Sequencetime, we show that it can enhance security. Fragments During authentication, instead of using one correct se-quence and two foils, we use the four correct sequences One of the potential attacks on our system involves arandomly interspersed within 8 foils. Authentication malicious party profiling the legitimate user’s knowledgesucceeds if the attacker shows a measurable performance and using that information to reverse engineer the trainedgap on the correct 4 out of 12 presented sequences. An sequence to be able to pass the authentication test. Al-attacker who slows down on random sequences will now though the number of possible trained sequences is toohave at most a 1/ 12 ≈ 1/500 chance in passing the test. 4 large to exhaustively test on any single individual eachThe number of trained sequences (4) and the number of sequence is constructed according to known constraintsfoils (8) can be adjusted to achieve an acceptable tradeoff and knowledge of subsequence fragments might enablebetween security and usability. the attacker to either reconstruct the original sequence or Similarly, a small number of authentication attempts enough of it to pass an authentication test.will not help a direct attacker pass the test. However, The training sequences are constrained to use all 6 re-memorizing the authentication test (360 symbols) and sponse keys equally often, so analysis of individual re-later presenting it to a coerced user could give the adver- sponse probabilities cannot provide information aboutsary an advantage. To further defend against this memo- the trained sequence. Likewise all 30 possible responserization attack we add one more step to the authentication key pairs (6 ∗ 5 = 30, since keys are not repeated) occurprocedure: once the authentication server observes that equally often during training meaning that bigram fre-the user failed to demonstrate a measurable gap on some quency also provides no information about the trained se-of the trained sequences, all remaining trained sequences quence. However, each 30-item sequence has 30 uniqueare replaced with random foils. This ensures that an trigrams (of 150 possible). If the specific training trigramattacker who tries to authenticate with no prior knowl- fragments could be identified, the underlying training se-edge will not see all the trained sequences and therefore quence could be reconstructed.cannot extract all trained sequences from a coerced user. An attack based on this information would be to haveConsequently, a one-shot attack on a coerced user is not a trained user perform a SISL test that contains all 150possible. Nevertheless, by iterating this process — tak- trigrams equally often. If the user exhibited better perfor-ing the authentication test, memorizing the observed se- mance on the 30 trained trigrams than the 120 untrained,quences, and then testing them out on a coerced trained the sequence could be reconstructed. This attack woulduser — the attacker may eventually learn all trained se- weaken the method’s relative resistance to external pres-quences and succeed in fooling the authentication test. sure to reveal the authentication information.During this process, however, the attacker must engage However, while the sequence information can be de-in the authentication test where he demonstrates knowl- termined at the trigram level it is not known if partici-edge of a strict subset of the trained sequences, but can- pants reliably exhibit sequence knowledge in such shortnot demonstrate knowledge of all sequences. This is a fragments. In Experiment 3, we evaluated performanceclear signal to the system that it is under attack at which on this type of trigram test to assess whether the sequencepoint the person engaging in the authentication test could information could be reconstructed. 10
  11. 11. Participants were again recruited through Mechanical We leave it for future work to design a coersion-resistantTurk and completed the same training sessions used in authentication protocol that remains secure when usersExperiments 1 and 2. At test, participants performed a can be coerced during the training phase.sequence constructed to provide each of the 150 trigramsexactly 10 times by constructing ten different 150-trialunits that each contain all possible trigrams in varying 6 Related Workorder. Performance on each trigram was measured bypercent correct as a function of the current response and There is a large body of related work in user authenti-two responses prior. cation and biometrics for user access control. The work To evaluate whether these data could be used to recon- can been broken down into biometrics (“who you are”),struct the sequence, the percent correct on each trigram tokens (“what you have”), and passwords (“what youwas individually calculated and a rank order of all tri- know”). There is significant past work in each of thegrams was created for each individual. If performance three main areas. Our work may fall into a new cate-on the trained trigrams was superior to others, the trained gory of implicit learning (“what you know you know buttrigram ranks should tend to be lower (e.g., performance do not know”), or could be categorized as a subclass ofexpression would lead the sequence trigrams to be the 30 behavioral biometric performed responses). However, average rank and Classic biometrics identifying a user based on whoaverage percent correct on the trained trigrams was in- they physically are can be grouped into physiologi-distinguishable from untrained trigrams. Participants did cal and behavior categories. Physiological characteris-not exhibit their trained sequence knowledge on this type tics include fingerprint, face recognition, DNA prints,of test, indicating that their sequence knowledge cannot palm print, hand geometry, iris recognition, and retinalbe attacked with a trigram-based method. More specifi- scans. Behavioral characteristics include measurementscally, for each user we compared the average percent cor- of typing rhythm and other dynamics, dynamic signa-rect measurements for the 30 trained-sequence trigrams ture, walking gait, voiceprints, and eye movement pat-to those for the 120 remaining trigrams. The 34 par- terns [11, 10, 2, 15]. Our work differs from these in en-ticipants averaged 73.9% correct (SE 1.2%) for trigrams abling quick training in new randomly seeded patterns.from the trained sequence and 73.2% correct (SE 1.1%) It might be very difficult to learn to walk a new way,for the rest. The difference was not reliable. and nearly impossible to change one’s iris pattern, but it While the trigram test did not lead to expression of se- should be easy to learn a new cortical crypto sequencequence knowledge, it is likely that participants’ sequence with a modest training regime. Further, if one relies onknowledge could be assessed for some longer fragments. retinal patterns for identification, each system could cap-However, the number of fragments to assess grows ex- ture all the information content of the retina, and thus aponentially with the length to be assessed and the abil- single compromised retina reader could reveal to an ad-ity to test all fragments is limited by the need to rely versary the entire set of information. Our approach en-on human performance to do the assessment. For exam- ables key revocation and multiple keys per user for dif-ple, for length 4 fragments (quad-grams), there are 750 ferent systems where there need not be any informationfragments to assess multiple times each to try to identify leakage from one system to the next.which ones had been trained. Denning et al. [4] propose an authentication model based on implicit learning of sets of images. An ear-Future work. In future work we will assess sequence lier study [21] compared the learning of images, artifi-expression at various lengths to find the minimal length cial words, and outputs from finite-state automata. Bothat which sequence knowledge can be expressed. This of these works develop authentication systems that al-minimal length likely reflects a basic operating charac- low users to easily memorize strong passwords, how-teristic of the brain regions that support implicit sequen- ever the resulting systems are not as resistant to rubbertial skill learning. If this length suggests the possibility hose attacks because they depend on the user consciouslyof attack, the sequence can be increased in complexity by studying sets of images or strings and as a result theincreasing the number of characters, using inter-response user retains some conscious knowledge of the creden-timing (known to be important to learning [7]) or more tial. When using the SISL task we were able to verifycomplex sequence structures than simple repeating se- that little conscious knowledge of the trained secret isquences. retained. Image-based authentication mechanisms also require curated image sets in order to reduce errors in the Recall that in our experiments we assumed that users authentication process; in contrast SISL-based authenti-are honest during the training phase and the adversary cation uses automatically generated sequences sampledonly gets to coerce users after they have been trained. from a well-defined high entropy combinatorial space. 11
  12. 12. Deniable encryption. In the context of encryption, de- guishable to an attacker versus a legitimate authenticator,niable encryption [3, 13] enables a user who encrypts a as well as by strengthening the test procedures and analy-message to open the ciphertext in multiple ways to pro- sis to increase reliability across a larger fraction of users,duce different cleartexts from the same ciphertext. Such or reduce the required testing time, false positives, andsystems enable a user to reveal an encryption key, which false negatives. Using variable timing between cues andproduces a document that contains plausible cleartext, measuring user performance as a function of game speedbut which is different from the actual document the user can further help in making the test protocol more reli-wishes to protect. This technique protects encrypted doc- able. Implicit learning of multiple credentials is yet an-uments, but does not apply to authentication credentials. other area that can benefit from additional experiments,Further, a properly motivated user of deniable encryp- building upon prior work that has so far found no evi-tion could choose to reveal the correct decryption key, dence of interference when users learn distinct 12-itemenabling the coercive adversary offline access to all ver- sequences, while also being capable of learning implic-sions of the document. Our approach develops a sys- itly sequences as long as 80 items.tem where the user cannot, even if strongly motivated, Another future direction for this work is in testingreveal to another any information useful for an adver- whether more complex structures—for example Markovsary to replicate the user’s access without the user being models—can be learned implicitly. We would like to usepresent. Deniability has also been studied in the context such learning to build challenge-response authenticationof elections [9]. which is resistant to eavesdropping in addition to coer- cion. Finally, beyond authentication, we would like toCoercion detection. Since our aim is to prevent users investigate the construction of a variety of cryptographicfrom effectively transmitting the ability to authenticate primitives based on implicit others, there remains an attack where an adversarycoerces a user to authenticate while they are under ad-versary control. It is possible to reduce the effective- Acknowledgmentness of this technique if the system could detect if theuser is under duress. Some behaviors such as timed re- We would like to thank all the paid volunteers who havesponses to stimuli may detectably change when the user contributed to our user studies through their participa-is under duress. Alternately, we might imagine other tion. This work was funded by NSF and a MURI grant.modes of detection of duress, including video monitor-ing, voice stress detection, and skin conductance moni-toring [8, 16, 1]. The idea here would be to detect by Referencesout-of-band techniques the effects of coercion. Together [1] J. Benaloh and D. Tuinstra. Uncoercible communi-with in-band detection of altered performance, we may cation. Technical Report TR-MCS-94-1, Clarksonbe able to reliably detect coerced users. University, 1994.7 Conclusions and Future Work [2] Christoph Bregler. Learning and recognizing hu- man dynamics in video sequences. In IEEE Conf.We have presented a new approach to protecting against on Computer Vision and Pattern Recognition, pagescoercion attacks using the concept of implicit learning 568–574, 1997.from cognitive psychology. We described a proof of [3] Ran Canetti, Cynthia Dwork, Moni Naor, andconcept protocol and preliminary experiments conducted Rafail Ostrovsky. Deniable encryption. Inthrough Mechanical Turk demonstrating a basis for con- CRYPTO, pages 90–104, 1997.fidence that it is possible to construct rubber hose resis-tant authentication. [4] Tamara Denning, Kevin D. Bowers, Marten van Much work remains. We hope to further analyze the Dijk, and Ari Juels. Exploring implicit memoryrate at which implicitly learned passwords are forgotten, for painless password recovery. In Desney S. Tan,and the required frequency of refresher sessions. In ad- Saleema Amershi, Bo Begole, Wendy A. Kellogg,dition we would like to find methods to detect or predict and Manas Tungare, editors, CHI, pages 2615–when individual users reliably learn (collecting more de- 2618. ACM, 2011.mographic data about our users might be a good first stepin this direction, along with multi-session long-term ex- [5] A. Destrebecqz and A. Cleeremans. Can sequenceperiments). We also hope to explore some of the limits of learning be implicit? new evidence with the pro-the approach, for example by finding out the minimum cess dissociation procedure. Psychonomic Bulletinlengths at which parts of learned sequences are distin- & Review, 8:343–350, 2001. 12
  13. 13. [6] Julien Doyon, Pierre Bellec, Rhonda Amsel, [17] D. Sanchez, E. Gobel, and P. Reber. Performing the Virginia Penhune, Oury Monchi, Julie Carrier, unexplainable: Implicit task performance reveals St´ phane Leh´ ricy, and Habib Benali. Contribu- e e individually reliable sequence learning without ex- tions of the basal ganglia and functionally related plicit knowledge. Psychonomic Bulletin & Review, brain structures to motor learning. Behavioural 17:790–796, 2010. Brain Research, 199(1):61–75, April 2009. [18] D.J. Sanchez and P.J. Reber. Operating character- [7] E. Gobel, D. Sanchez, and P. Reber. Integration istics of the implicit learning system during serial of temporal and ordinal information during serial interception sequence learning. Journal of Experi- interception sequence learning. Journal of Exper- mental Psychology: Human Perception and Perfor- imental Psychology: Learning, Memory & Cogni- mance, in press. tion, 37:994–1000, 2011. [19] Chris Soghoian. Turkish police may have [8] Payas Gupta and Debin Gao. Fighting coercion beaten encryption key out of TJ Maxx sus- attacks in key generation using skin conductance. pect, 2008. In USENIX Security Symposium, pages 469–484, 3-10069776-46.html. 2010. [20] T. van Aardenne-Ehrenfest and N. G. de Bruijn. Circuits and trees in oriented linear graphs. Simon [9] Ari Juels, Dario Catalano, and Markus Jakobsson. Stevin, 28:203–217, 1951. Coercion-resistant electronic elections. In Proceed- ings of the 2005 ACM workshop on Privacy in the [21] Daphna Weinshall and Scott Kirkpatrick. Pass- electronic society, WPES ’05, pages 61–70, New words you’ll never forget, but can’t recall. In CHI York, NY, USA, 2005. ACM. Extended Abstracts, pages 1399–1402, 2004.[10] A. Kale, A.N. Rajagopalan, N. Cuntoor, [22] Wikipedia. Rubber-hose cryptanalysis, 2011. V. Krueger, and R. Chellappa. Identification of humans using gait. IEEE Transactions on Image Processing, 13:1163–1173, 2002.[11] Fabian Monrose, Michael Reiter, and Susanne Wet- zel. Password hardening based on keystroke dy- namics. Int. J. of Inf. Sec., 1(2):69–83, 2002.[12] Mary J. Nissen and Peter Bullemer. Attentional requirements of learning: Evidence from perfor- mance measures. Cognitive Psychology, 19(1):1– 32, January 1987.[13] Adam O’Neill, Chris Peikert, and Brent Waters. Bi-deniable public-key encryption. In Proc. of Crypto’11, volume 6841 of LNCS, pages 525–542, 2011.[14] Paul Reber. Cognitive neuroscience of declarative and non-declarative memory. Parallels in Learning and Memory, Eds. M.Guadagnoli, M.S. deBelle, B. Etnyre, T. Polk, A. Benjamin, pages 113–123, 2008.[15] Douglas A. Reynolds, Thomas F. Quatieri, and Robert B. Dunn. Speaker verification using adapted gaussian mixture models. In Digital Signal Pro- cessing, 2000.[16] Robert Ruiz, Claude Legros, and Antonio Guell. Voice analysis to predict the psychological or phys- ical state of a speaker, 1990. 13