Solving the Challenge of New
Entropy Standards
Ray Potter
ICMC
November 20, 2014
Flow
• Quick recap of entropy and its purpose
• Standards review
• Our work
2
Entropy
• Average amount of information contained
in data stream
• A measure of uncertainty /
unpredictability
3
Practical Entropy
4
S a f L o g i c
R u l s
Entropy in the Real World
5
Entropy in Crypto
• Provide random bits
• Challenges in authentication protocols
• Seeds for algorithms
• Use to seed DRBG
• Value is unpredictable output
6
Issues
• Truly random data difficult / impossible
to generate on a computer
• How to measure it
7
Entropy Quantified
• log2 (max p(xi))
• −􏰂 P(X = x)log P(X = x)
8
NIST 800 Series
• SP 800-90B: requirements for entropy
source
• SP 800-90A: deterministic algorithms
• SP 800-90C: implement an RBG with -90A
and -90B components
9
Effect to FIPS 140
• Current Requirement:
“Compromising the security of the key
generation method (e.g., guessing the seed
value to initialize the deterministic RNG) shall
require as least as many operations as
determining the value of the generated key.”
10
Draft IG
• First socialized last year
• Entropy estimation mandatory for…
software modules which include entropy
gathering mechanisms that are within the
logical boundary of the module
11
Entropy Gathered within SW
Module Logical Boundary
• CMTL needs to submit entropy rationale
• If DRBG is reseeded frequently, the
vendor shall make a reasonable heuristic
claim of independence of the added
entropy values.
12
Entropy Gathered Outside the
SW Module Logical Boundary
• Entropy estimate should be in SP
1. Entropy originates from another
validated module
2. Entropy originates from the
operational environment
13
ChaosControl
• Cryptographically secure DRBG
• Available for mobile and desktop / server
environments
• Compliant to 800-90 and draft FIPS 140 IG
14
Logical View of Entropy Sources
for iOS Platform
15
Words from Whit
• The right way to use tests
in random number
generation is to look for
failure of the particular
mechanism.
• Test each source
independently (for stuck
faults and other things that
can be detected by
correlation) and shut down
if fewer than some
16
Checks Performed
• Entropy estimates for each source is
recorded with that source
• Exception / reinitialize if not enough
entropy
• CRNGT (CREGT?)
17
Initial Seeding
• Ensures sufficient entropy before allowing
clients to request random bytes
• Checks for suitable amount of entropy
before initialization
• Seed file is persisted to disk
18
More about Tests
• Heuristic
• log2 (max p(xi)) / min-entropy from 800-
90b
• Statistical Tests from 800-90b
• Full test suite documented by NIST SP800-
22rev1a
19
Results from 800-22rev1a
20
Statistical Analysis Results
21
Compression Bins Collision
Output Space Size = 256 Minimum
Possible Score: 0.000000 Maximum
Possible Score: 7.183666 Filename:
out.bin Test name:
compression Output Space Size
256 Numberof samples:
58321 Numberof events:
57321 Mean score:
7.139077 Adjusted mean score:
7.126542 Standard deviation:
1.818899 Entropy type: min-
entropy Entropy estimate:
4.936194 Entropy/outputdimension
estimate: 0.617024
Output Space Size =
256 Warning: Shannon entropy
estimate = 7.97 Filename:
out.bin Test name: bins Output
Space Size 256 Number of
samples: 58321 Numberof
events: 58321 Mean score:
0.006927 Adjusted mean score:
0.015471 Standard deviation:
0.000000 Entropy type: min-
entropy Entropy estimate:
6.014244 Entropy/outputdimens
ion estimate: 0.751781
Output Space Size =
256 MinimumPossible Score:
2.000000 MaximumPossible
Score: 20.726106 Filename:
out.bin Test name:
collision Output Space Size
256 Numberof samples:
58313 Numberof events:
2803 Mean score:
20.803782 Adjusted mean score:
20.493429 Standard deviation:
9.956489 Entropy type: min-
entropy Entropy estimate:
6.103266 Entropy/outputdimensi
on estimate: 0.762908
Hey Guess What…
• ChaosControl is included with
CryptoComply
• ChaosControl is (nearly) patented
• SafeLogic is offering it standalone at NO
COST license until the end of Q1CY2015
22
Let’s Connect
• @SafeLogic
• @SafeLogic_Ray
• www.SafeLogic.com

Solving 800-90 Entropy Requirements in Software

  • 1.
    Solving the Challengeof New Entropy Standards Ray Potter ICMC November 20, 2014
  • 2.
    Flow • Quick recapof entropy and its purpose • Standards review • Our work 2
  • 3.
    Entropy • Average amountof information contained in data stream • A measure of uncertainty / unpredictability 3
  • 4.
    Practical Entropy 4 S af L o g i c R u l s
  • 5.
    Entropy in theReal World 5
  • 6.
    Entropy in Crypto •Provide random bits • Challenges in authentication protocols • Seeds for algorithms • Use to seed DRBG • Value is unpredictable output 6
  • 7.
    Issues • Truly randomdata difficult / impossible to generate on a computer • How to measure it 7
  • 8.
    Entropy Quantified • log2(max p(xi)) • −􏰂 P(X = x)log P(X = x) 8
  • 9.
    NIST 800 Series •SP 800-90B: requirements for entropy source • SP 800-90A: deterministic algorithms • SP 800-90C: implement an RBG with -90A and -90B components 9
  • 10.
    Effect to FIPS140 • Current Requirement: “Compromising the security of the key generation method (e.g., guessing the seed value to initialize the deterministic RNG) shall require as least as many operations as determining the value of the generated key.” 10
  • 11.
    Draft IG • Firstsocialized last year • Entropy estimation mandatory for… software modules which include entropy gathering mechanisms that are within the logical boundary of the module 11
  • 12.
    Entropy Gathered withinSW Module Logical Boundary • CMTL needs to submit entropy rationale • If DRBG is reseeded frequently, the vendor shall make a reasonable heuristic claim of independence of the added entropy values. 12
  • 13.
    Entropy Gathered Outsidethe SW Module Logical Boundary • Entropy estimate should be in SP 1. Entropy originates from another validated module 2. Entropy originates from the operational environment 13
  • 14.
    ChaosControl • Cryptographically secureDRBG • Available for mobile and desktop / server environments • Compliant to 800-90 and draft FIPS 140 IG 14
  • 15.
    Logical View ofEntropy Sources for iOS Platform 15
  • 16.
    Words from Whit •The right way to use tests in random number generation is to look for failure of the particular mechanism. • Test each source independently (for stuck faults and other things that can be detected by correlation) and shut down if fewer than some 16
  • 17.
    Checks Performed • Entropyestimates for each source is recorded with that source • Exception / reinitialize if not enough entropy • CRNGT (CREGT?) 17
  • 18.
    Initial Seeding • Ensuressufficient entropy before allowing clients to request random bytes • Checks for suitable amount of entropy before initialization • Seed file is persisted to disk 18
  • 19.
    More about Tests •Heuristic • log2 (max p(xi)) / min-entropy from 800- 90b • Statistical Tests from 800-90b • Full test suite documented by NIST SP800- 22rev1a 19
  • 20.
  • 21.
    Statistical Analysis Results 21 CompressionBins Collision Output Space Size = 256 Minimum Possible Score: 0.000000 Maximum Possible Score: 7.183666 Filename: out.bin Test name: compression Output Space Size 256 Numberof samples: 58321 Numberof events: 57321 Mean score: 7.139077 Adjusted mean score: 7.126542 Standard deviation: 1.818899 Entropy type: min- entropy Entropy estimate: 4.936194 Entropy/outputdimension estimate: 0.617024 Output Space Size = 256 Warning: Shannon entropy estimate = 7.97 Filename: out.bin Test name: bins Output Space Size 256 Number of samples: 58321 Numberof events: 58321 Mean score: 0.006927 Adjusted mean score: 0.015471 Standard deviation: 0.000000 Entropy type: min- entropy Entropy estimate: 6.014244 Entropy/outputdimens ion estimate: 0.751781 Output Space Size = 256 MinimumPossible Score: 2.000000 MaximumPossible Score: 20.726106 Filename: out.bin Test name: collision Output Space Size 256 Numberof samples: 58313 Numberof events: 2803 Mean score: 20.803782 Adjusted mean score: 20.493429 Standard deviation: 9.956489 Entropy type: min- entropy Entropy estimate: 6.103266 Entropy/outputdimensi on estimate: 0.762908
  • 22.
    Hey Guess What… •ChaosControl is included with CryptoComply • ChaosControl is (nearly) patented • SafeLogic is offering it standalone at NO COST license until the end of Q1CY2015 22
  • 23.
    Let’s Connect • @SafeLogic •@SafeLogic_Ray • www.SafeLogic.com

Editor's Notes

  • #2 Who was here last year? Remember Entropy? Much like entropy itself, the details and usefulness of this talk will be completely unpredictable.
  • #3 Walk you through our journey. I’m not an expert in entropy. This is not technical.
  • #4 As entropy is larger for more random sources. We’re talking about random data. A coin flip is not much entropy. Let’s look at a simple use case
  • #5 26^14
  • #6 Explain roulette If wheel is not true, value of entropy decreases. Players will bet more often on black because it hits more. That’s also the reason for the two greens. Increase in entropy. But we’re not here to gamble… security professionals don’t like “risk”
  • #7 Seeds for algs like Diffie Hellman Stronger the entropy, the stronger the output from DRBG
  • #8 Hardware can use Geiger counters, ring oscillators, thermal noise, etc. Behavior of computers is deterministic. Measure: Mary Ann- “Know it when I see it”
  • #9 How do you quantify random / unpredictable output? With MATH First is min-entropy: a worst-case measure of the uncertainty Second is a simple calculation of entropy source 3/2 people have problems with fractions Stanford Professor Tsachy Weissman the compression ratio and the ratio of the log of the compression time,” normalized against an industry standard compressor used for the same data
  • #10 B: not specific instructions because every environment is different. Implementation is up to developer 90A: that take an entropy input and use it to produce pseudorandom values.
  • #11 Basically just say that the length of the seed is greater than or equal to the length of the key. or it’s outside the module. standards do not yet exist for the embodiment or construction of an entropy source or the mechanisms to gather entropy. No real documentation of the estimate of the entropy that the module receives or generates to seed RBG. No real mechanism for the testing laboratory verify the vendor claims
  • #12 illustrates all of the components, sources and mechanisms that constitute the NDRNG implemented within the module vendor provided heuristic analysis of an entropy source along with the justifications of the entropy claim based on this analysis Talking about this from a SOFTWARE perspective
  • #13 1. Detailed logical diagram: Include conditioning components, service calls 2. Output of statistical tests from 800-90b: QUESTION - CAVP has or will have a tool that must be run on 1 million samples of raw entropy 3. Heuristic analysis and justification Again, make sure it’s UNCERTAIN
  • #14 or within the Operating Environment outside the software modules logical boundary Inherit the applicable restrictions. No entropy estimation is necessary. A statement and rationale shall be made of the entropy source and the entropy estimate for each tested OE. No estimate needed if outside the physical boundary or third-party applications running on the same platform as intermediaries that pass the seed and the seed key (if applicable)
  • #15 We did what any good SV company would do… got busy solving the problem.
  • #16 Just one example. 32 pools of entropy, each with 256 bits. Total of 8192 bits of potential entropy
  • #17 Whit Diffie is on our advisory board. Sweetheart of a guy and sometimes very intimidating to talk to! Embraced the concept and provided input on design
  • #18 1. If that source fails(any error condition), then the entropy estimate is subtracted from the over all entropy of the system. 2. If the entropy of the system falls below a certain threshold, then the system fails and will need to be shutdown and re-initialized before any more random bytes can be extracted 3. For every source, the last hash of the injection is recorded. If that hash repeats itself over two iterations, then an error count is incremented. If that error count goes over a certain threshold, then the system will be required to be re-initialized
  • #19 The seed file is backed up (twice, in case the first backup fails), so on next startup it will utilize the current state of the system and does not need to re-establish entropy
  • #20 100000 bits / 10 tests for each entropy source after randomness is extracted and ensure that the output passes the appropriate proportion of tests for randomness
  • #22 1. Compression Test 2. Bins Test – performs a frequency test of each of the possible output states. 3. Collision Test – estimate entropy from collision rates within the data stream.
  • #23 You should know where your entropy comes from.