We used to believe that some software is secure. I'd like to decompose our faith in it by touching every prime factor of our daily computing environments. This implies the roller-coaster ride from user space to the farthest parts of the Universe by 0.1-days and oddities of software that we use each day. Our goal is to realize how many assumptions are hidden in saying that some piece of code is secure.

2. Trust boundaries...
Mateusz Kocielski
m.kocielski@logicaltrust.net
LogicalTrust
Confidence
Kraków, Poland, May 2015

3. $ whoami
pentester at LogicalTrust as $DAILYJOB
open source committer:
PHP - bug fixing
NetBSD - libsaslc(3) & random things...
security:
PHP - CVE-2010-1868, CVE-2010-1917, CVE-2010-4150,
CVE-2010-4156, CVE-2011-1938, ...
stunnel - CVE-2013-1762
OpenSSH - CVE-2011-0539
Apache - CVE-2014-0117, CVE-2014-0226
FreeBSD - CVE-2015-1414
...

4. Trust boundaries?
Our goal is to investigate how many assumptions are hidden in belief
that some piece of code is secure.
We want to harbour a seed of doubt in those, who are too confident
and encourage those, who cannot sleep at night.
I want to share with you some of my findings...

5. What is trust? - Wikitionary
trust (source: http://en.wiktionary.org/wiki/trust):
1. Confidence in or reliance on some person or quality.
4. That which is committed or entrusted; something received in
confidence; a charge.
5. That upon which confidence is reposed; ground of reliance; hope
7. The condition or obligation of one to whom anything is confided;
responsible charge or office.
...

6. What is trust?
Source: http://pl.wiktionary.org/wiki/Plik:Male˙House˙Sparrow˙%28Passer˙domesticus%29˙feeding˙from˙hand.jpg

7. In software we trust?
By saying ”I trust this software”, we’d like to think that:
it doesn’t hurt us lack of bugs (at least those connected
somehow with security).
it does what we think it does (and nothing else) lack of
backdoors etc.
We all have our own definition of trusted software, moreover this
definition varies on many factors like requirements etc. For this
presentation we can assume that everyone in this room are (at
least) a bit paranoid.

8. A short survey...
Can we trust following software:
Linux
Windows
MS Office
Apache
OpenBSD
OpenSSL
OpenSSH
working as ”cloud” (innovative clusters placed in cloud operating
with synergy crap) - GMail, Dropbox, Google Drive, ...

9. Keeping data in cloud

10. Keeping data in cloud
Source: http://download.fsfe.org/advocacy/stickers/thereisnocloud/thereisnocloud-v2-74x74.pdf, CC-By-Sa Markus
Meier

11. Simple observations
It’s easier to trust software:
small (e.g. bc vs. Excel)
open source
”given enough eyeballs, all bugs are shallow” - Linus’s Law
we can verify code ourself
who is paranoid enough not to trust binary sets of your favourite
distro? :)
Reproducible builds can solve that problem
with good reputation (e.g. OpenBSD vs. Windows)

12. ...but let’s talk about the facts
Here is the plan:
take a piece of software that we believe is secure
run it and expose it to the internet
verify what we meant by claiming it is secure?
think about the consequences
So which software to choose?...

13. Who’ll be our hero?!
...by acclamation we’re choosing:
openssh

14. Why OpenSSH?
small - version 6.8 has around 90K LOC
good design (priv. separation etc.)
well written (security in mind...)
”I am always looking for bugs in OpenSSH as it is written in clear
to read source code and has very strong security.” - Kingcope
(source: http://kingcope.wordpress.com/2013/09/13/
opensslopenssh-ecdsa-authentication-code-inconsistent-return-values-no-v
Everybody uses it:
zmap (source: https://zmap.io/paper.pdf): (...) port 22 hit rate:
0.57% (...)
generally recognized as a safe and robust software

15. OpenSSH on the screen
Source: http://nmap.org/movies/matrix/trinity-nmapscreen-hd-crop-1200x728.jpg
...so yesterday...

16. Trust boundaries
OpenSSH
libopenbsd-
compat
openssl
libc
libz
kernel CPU/hw physics ???

17. Simple observations pt. 2
layer n (lower) is broken → layer n + 1 (higher) is broken
even (relatively) simple software has complex foundations
Source: http://vignette1.wikia.nocookie.net/uncyclopedia/images/c/c1/CaptobviousChooseOption.gif/revision/latest?
cb=20070106161415

18. Trust boundaries
OpenSSH
libopenbsd-
compat
openssl
libc
libz
kernel CPU/hw physics ???

19. OpenSSH - potential stack overflow
key.c:
static int
cert_parse(Buffer *b, Key *key, const u_char *blob, u_int blen)
{
u_char *principals, *critical, *exts, *sig_key, *sig;
u_int signed_len, plen, clen, sklen, slen, kidlen, elen;
Buffer tmp;
char *principal;
int ret = -1;
int v00 = key->type == KEY_DSA_CERT_V00 ||
key->type == KEY_RSA_CERT_V00;
[...]
if ((key->cert->signature_key = key_from_blob(sig_key,
sklen)) == NULL) {
[...]
Key *
key_from_blob(const u_char *blob, u_int blen)
{
[...]
if (key_is_cert(key) && cert_parse(&b, key, blob, blen) == -1) {
error("key_from_blob: can’t parse cert data");
goto badkey;
}
[...]
Fixed a year or two ago, left here for historical reasons... key from blob can
be called remotely using pubkey authentication. If you’re interested then take
a look to previous versions of auth2-pubkey.c.

20. OpenSSH - potential stack overflow
(Un)fortunately certificate is handled by the Buffer structure, which
maximum length is bounded:
buffer.c:
#define BUFFER_MAX_CHUNK 0x100000
[...]
void *
buffer_append_space(Buffer *buffer, u_int len)
{
u_int newlen;
void *p;
if (len > BUFFER_MAX_CHUNK)
fatal("buffer_append_space: len %u not supported", len);
No cookies this time, but maybe somewhere in space there are systems (or
configurations) which are exploitable. (Bounded stack + something
important near to it).

21. OpenSSH - CVE-2011-0539
”OpenSSH does not properly initialise a nonce field with random data
when generating legacy certificates (”-t” command line option of
ssh-keygen). This can result in certain stack memory being used as
nonce, which can lead to the disclosure of potentially sensitive
information.” - source: http://secunia.com/advisories/43181
key.c patch:
/* -v01 certs put nonce first */
+ arc4random_buf(&nonce, sizeof(nonce));
if (!key_cert_is_legacy(k)) {
- arc4random_buf(&nonce, sizeof(nonce));
buffer_put_string(&k->cert->certblob, nonce, sizeof(nonce));
}

22. Trust boundaries
OpenSSH
libopenbsd-
compat
openssl
libc
libz
kernel CPU/hw physics ???

23. Libraries - libopenssh-compat
OpenBSD specific functions for !OpenBSD platforms
heavily relies on the OpenSSL (e.g. rng implementation)
let’s trust it! :)

24. Libraries - OpenSSL
¨uber complex library which implements various crypto stuff
around 450 K LOC of hard-core C
there are rumours, that some people learnt C by writing this
library....
among developers this library has rather a bad reputation
Why no SSL? -
https://www.varnish-cache.org/docs/trunk/phk/ssl.html
OpenSSL is written by monkeys -
http://www.peereboom.us/assl/assl/html/openssl.html
lots of projects rely on OpenSSL (700+ ports in the FreeBSD
ports tree)
we would rather be sorry if someone found a bug in this code...

25. Libraries - OpenSSL - top comments
grep -Ri xxx .:
crypto/asn1/a_strex.c: fld_len = 0; /* XXX: what should this be? */
apps/passwd.c: /* XXX: really we should know how to print a size_t, not cast it */
ssl/t1_enc.c: /* I should fix this up TLS TLS TLS TLS TLS XXXXXXXX */
ssl/d1_pkt.c: /* XXX: check what the second ’&& type’ is about */
include/openssl/pem.h: /* XXX(ben): don#t think this is used!... */
crypto/asn1/asn1_mac.h: /* BIG UGLY WARNING! This is so damn ugly I wanna puke. Unfortunately,
some macros that use ASN1_const_CTX still insist on writing in the input
stream. ARGH! ARGH! ARGH! Let’s get rid of this macro package.... */
crypto/objects/obj_dat.c: ad.obj=(ASN1_OBJECT *)a; /* XXX: ugly but harmless */
crypto/objects/obj_lib.c: return((ASN1_OBJECT *)o); /* XXX: ugh! Why? What kind of
duplication is this??? */
crypto/engine/eng_cryptodev.c: /* XXXX just disable all digests for now, because it sucks. */
More stuff @ https://twitter.com/OpenSSLFact

26. Libraries - LibreSSL
fork done by OpenBSD guys right after the famous
CVE-2014-0160
”libressl is a version of the tls/crypto stack forked from openssl in
2014, with goals of modernizing the codebase, improving security,
and applying best practice development processes.” -
http://www.libressl.org
The first 30 days, and where we go from here -
http://www.openbsd.org/papers/bsdcan14-libressl/
More than 30 Days Later -
http://www.openbsd.org/papers/eurobsdcon2014-libressl.html

27. Libraries - LibreSSL - favicon.ico
Source: http://www.libressl.org/favicon.ico

28. Libraries - LibreSSL - BN
Code is complex, so let’s pick some small sublibrary - BigNumber
implementation:
part of the lib which is used in many other parts of library
should be rather easy to implement (is it? :))

29. Libraries - LibreSSL - fun fact
LibreSSL uses (sometimes) three zero representations: 0, −0 and 0..0:
those ”zeros” are values returned by BN functions...
once 0 = 0..0 = −0, sometimes not...
...inconsistency can’t hurt us..., can it?

30. Libraries - LibreSSL - off-by-one #1
Let’s take BIGNUM −0 and apply it to the BN bn2hex
crypto/bn/bn print.c:
char *BN_bn2hex(const BIGNUM *a)
{
char *buf;
char *p;
buf=(char *)OPENSSL_malloc(a->top*BN_BYTES*2+2);
[...]
p=buf;
if (a->neg) *(p++)=’-’;
if (BN_is_zero(a)) *(p++)=’0’;
for (i=a->top-1; i >=0; i--)
[...]
*p=’0’;
How to get −0 in LibreSSL? It is a task for the listener

31. Libraries - LibreSSL - off-by-one #2
Let’s call BN rand(BN, 1, 1, 0)- bnrand crypto/bn/bn rand.c:
static int
bnrand(int pseudorand, BIGNUM *rnd, int bits, int top, int bottom)
{
unsigned char *buf = NULL;
[...]
bytes = (bits + 7) / 8;
bit = (bits - 1) % 8;
buf = OPENSSL_malloc(bytes);
if (top != -1) {
if (top) {
if (bit == 0) {
buf[0] = 1;
buf[1] |= 0x80;
[...]

32. Libraries - libz
zlib - easy, small library for data compression/decompression
inflate.c:
int ZEXPORT inflateInit_(strm, version, stream_size)
z_streamp strm;
const char *version;
int stream_size;
{
return inflateInit2_(strm, DEF_WBITS, version, stream_size);
}
[...]
int ZEXPORT inflateInit2_(strm, windowBits, version, stream_size)
z_streamp strm;
int windowBits;
const char *version;
int stream_size;
{
[...]
struct inflate_state FAR *state;
[...]
if (strm->zalloc == (alloc_func)0) {
[...]
}
state = (struct inflate_state FAR *)
ZALLOC(strm, 1, sizeof(struct inflate_state));
[...]
strm->state = (struct internal_state FAR *)state;
state->window = Z_NULL;
ret = inflateReset2(strm, windowBits);
[...]
}

33. Libraries - libz cont.
inflate.c:
int ZEXPORT inflateReset2(strm, windowBits)
z_streamp strm;
int windowBits;
{
int wrap;
struct inflate_state FAR *state;
if (strm == Z_NULL || strm->state == Z_NULL) return Z_STREAM_ERROR;
state = (struct inflate_state FAR *)strm->state;
if (windowBits < 0) {
wrap = 0;
windowBits = -windowBits;
} else {
wrap = (windowBits >> 4) + 1;
}
/* set number of window bits, free window if different */
if (windowBits && (windowBits < 8 || windowBits > 15))
return Z_STREAM_ERROR;
if (state->window != Z_NULL && state->wbits != (unsigned)windowBits) {
ZFREE(strm, state->window);
state->window = Z_NULL;
}
[...]
OpenSSH is not affected, but how about other popular software? YES IT IS!

34. Libraries - libc - dobule free - getaddrinfo IDN
$ traceroute $(printf "302a")
*** glibc detected *** traceroute: munmap_chunk(): invalid pointer: 0x00007fff1b43a547 ***
======= Backtrace: =========
/lib64/libc.so.6(cfree+0x166)[0x32244758c6]
/lib64/libc.so.6[0x32244bc116]
/lib64/libc.so.6(getaddrinfo+0x21a)[0x32244be94a]
traceroute[0x402926]
traceroute[0x4029f1]
traceroute[0x406281]
traceroute[0x403546]
/lib64/libc.so.6(__libc_start_main+0xf4)[0x322441d9f4]
traceroute[0x401619]
======= Memory map: ========
00400000-00409000 r-xp 00000000 68:06 7103807 /bin/traceroute
00608000-00609000 rw-p 00008000 68:06 7103807 /bin/traceroute
00609000-0060a000 rw-p 00609000 00:00 0
00808000-00809000 rw-p 00008000 68:06 7103807 /bin/traceroute
00ff7000-01018000 rw-p 00ff7000 00:00 0 [heap]
3224000000-322401c000 r-xp 00000000 68:06 7332914 /lib64/ld-2.5.so
http://www.openwall.com/lists/oss-security/2015/01/27/11

35. Trust boundaries
OpenSSH
libopenbsd-
compat
openssl
libc
libz
kernel CPU/hw physics ???

36. Kernel
Operating system kernel has usually millions code lines
Next to each other:
network protocols
filesystems
sound system
...programming ZOO

37. Kernel - FreeBSD - IGMP - CVE-2015-1414
netinet/igmp.c:
case IGMP_VERSION_3: {
struct igmpv3 *igmpv3;
uint16_t igmpv3len;
uint16_t srclen;
int nsrc;
[...]
igmpv3 = (struct igmpv3 *)igmp;
/* Validate length based on source count. */
nsrc = ntohs(igmpv3->igmp_numsrc);
srclen = sizeof(struct in_addr) * nsrc;
if (nsrc * sizeof(in_addr_t) > srclen) {
IGMPSTAT_INC(igps_rcv_tooshort);
return;
}
[...]
igmpv3len = iphlen + IGMP_V3_QUERY_MINLEN +
srclen;
if ((m->m_flags & M_EXT ||
m->m_len < igmpv3len) &&
(m = m_pullup(m, igmpv3len)) == NULL) {
IGMPSTAT_INC(igps_rcv_tooshort);
return;
}
igmpv3 = (struct igmpv3 *)(mtod(m, uint8_t *)
+ iphlen);
if (igmp_input_v3_query(ifp, ip, igmpv3) != 0) {
[...]

38. Kernel - NetBSD - libprop - #1
common/lib/libprop/*:
buf = malloc(pref->pref_len + 1, M_TEMP, M_WAITOK | M_CANFAIL);
if (buf == NULL)
return (ENOMEM);
error = copyin(pref->pref_plist, buf, pref->pref_len);
It’s not exploitable on x86/amd64, but may hurt sparc64.

39. Kernel - NetBSD - libprop - #2
common/lib/libprop/*:
#define _PROP_EOF(c) ((c) == ’0’)
#define _PROP_ISSPACE(c)
((c) == ’ ’ || (c) == ’t’ || (c) == ’n’ || (c) == ’r’ ||
_PROP_EOF(c))
struct _prop_object_internalize_context *
_prop_object_internalize_context_alloc(const char *xml)
{
[...]
/*
* Skip any whitespace and XML preamble stuff that we don’t
* know about / care about.
*/
for (;;) {
while (_PROP_ISSPACE(*xml))
xml++;

40. Trust boundaries
OpenSSH
libopenbsd-
compat
openssl
libc
libz
kernel CPU/hw physics ???

41. CPU/HW
Bugs in the CPUs
Kris Kaspersky - Remote Code Execution Through Intel CPU
Bugs - HITB 2009
Pentium F00F bug
Pentiun FDIV bug
Bugs in the processor’s microcode - XEON example:
(...) Erratum AAK167/BT248: ”If a logical processor has EPT
(Extended Page Tables) enabled, is using 32-bit PAE paging, and
accesses the virtual-APIC page then a complex sequence of
internal processor micro-architectural events may cause an
incorrect address translation or machine check on either logical
processor. (...) -
http://lists.debian.org/debian-user/2013/09/msg00126.html
Bugs in (random) devices (incl. its firmware)
have you ever wondered what’s inside the firmware?
Backdoors in devices
Intel, NSA & RDRAND ... :)

42. Trust boundaries
OpenSSH
libopenbsd-
compat
openssl
libc
libz
kernel CPU/hw physics ???

43. WARNING
Dragons live here!For the moment I’ll talk about things that:
I don’t understand
relate to philosophy rather than thug life...

44. Physics
Have you ever heard about bugs in physics?
What will happen if our openssh will run in space?
Have we tested it in such environment?
Are the physics’ laws consistent?
What if somebody can remotely change CPU/memory state?
Haha, I hope you didn’t believe that?
Can we defend somehow?
encoding correction (i.e. ECC)
mirroring hardware
OpenSSH will not suffer...But if we write a code to manage space
robot, then it’s not funny anymore...

45. Trust boundaries
OpenSSH
libopenbsd-
compat
openssl
libc
libz
kernel CPU/hw physics ???

46. ???
In God We Trust... Everything Else We Test...
...maybe some three letter agency can manipulate the physics’
laws... :)

47. Fallen actors
Protocols: ARP, IP, TCP, SSH, ...
Algorithms: DH, RSA, DSA, EC, ...

48. Fallen actors - protocols
There are bugs in protocols:
SSL - CRIME, BEAST...
TCP - SMURF, SYN
...
We can formally proof that there are no bugs in the protocol...
...but in order to do so we need a model which is usually
simplification of reality
”the absence of proof is not the proof of absence”

49. Forgotten elements - algorithms
We can use math tools to verify that RSA does the job
We can do it modulo some model
Do you know that RSA relies on Factorization /∈ P
Do you know that we don’t know if P = NP?
Do you know that three letters agencies spent lots on money on
breaking crypto?
Finally, do you know that basing on ZF or ZFC we can’t be sure
what we do? :)
Assuming that we verified our algorithm, there’s still long road:
we have to choose language (with formal semantics)
code our algorithm in chosen language
verify that we coded exactly what we meant (tests, formal proofs)
having verified code we should recall the forgotten actor...

50. Forgotten compiler
Attentive listener recalls that: ”OpenSSH is written in C, but
processor is fed with machine code...”
How do we know that we do not introduce any bugs in
compilation process?
Finding and Understanding Bugs in C Compilers
(http://www.cs.utah.edu/˜regehr/papers/pldi11-preprint.pdf):
”Compilers should be correct. To improve the quality of C compilers, we
created Csmith, a randomized test-case generation tool, and spent
three years using it to find compiler bugs. During this period we
reported more than 325 previously unknown bugs to compiler
developers. Every compiler we tested was found to crash and also to
silently generate wrong code when presented with valid input. (...)”

51. CompCert
CompCert - verified compiler from C90 subset to PPC, ARM, x86
Verified = result code is 100% consistent with C90 semantics
Useful when compiling critical code for embedded solutions
Its translation process is INSANELY complex:
Project’s homepage: http://compcert.inria.fr/

52. Conclusions
Trust in even the simplest piece of code implies belief in hidden
assumptions which we usually don’t even realize
We’re forced to believe that each ”layer” works well
For some reason three letters agencies are spending millions on
breaking things
Crypto is working, washing machines do the job
Some of the instances of OpenSSH which I run are reachable from
the Internet, and usually I sleep well...
Sometimes it’s good to turn on paranoia when thinking about our
computers
”Better paranoid than sorry” - comment from the OpenSSH
project
Read the code or not be surprised
We’ll never get rid of bugs in software :)

53. Credits
Large parts of this presentation were done in cooperation with
Marek Kroemeke and Filip Palian, THANKS!

54. Some reading material
http://c2.com/cgi/wiki?ProofsCantProveTheAbsenceOfBugs
http://cm.bell-labs.com/who/ken/trust.html
http://en.wikipedia.org/wiki/Argument˙from˙ignorance
http://reproducible.alioth.debian.org/presentations/
2014-02-01-FOSDEM14.pdf
https://wiki.freebsd.org/ReproducibleBuilds
https://wiki.debian.org/ReproducibleBuilds
https://fosdem.org/2015/schedule/event/stretching˙out˙for˙
trustworthy˙reproducible˙builds/attachments/paper/668/export/
events/attachments/stretching˙out˙for˙trustworthy˙reproducible˙
builds/paper/668/2015˙01˙31˙FOSDEM15.pdf
http://www.cl.cam.ac.uk/research/security/capsicum/

55. Time for questions (and maybe answers)
Q&A