This document summarizes a presentation on secure software development given by Rod Chapman. It discusses how safety-critical systems have historically used formal methods like correctness-by-construction (CbyC) to build reliable systems. However, secure systems operate in a malicious environment and must assume arbitrary attacks. While CbyC offers confidence by verifying properties, it is not a silver bullet and still requires solid security engineering. There are also concerns that efforts focus too much on legacy code instead of prevention and that security requirements are an infinite set that cannot be fully enumerated. The future may involve combining formal verification of critical components with other techniques for less critical parts and architecting to isolate systems of differing security needs.

1. High Assurance Software Symposium
Directions in Secure
Software Development
Rod Chapman

2. Contents
• Some observations
• Experiences with secure systems…
• Trends and Concerns
• A future?

3. Some observations
• The safety-critical industry has been building
remarkably reliable and high-quality systems for
years…
• Can we apply the same approach for secure
systems?
• At Altran Praxis, we first tested this hypothesis in
1999 with the MULTOS CA system.
• It seems the answer is “Yes…but with a few key
assumptions changed…”

4. Some observations
• In the last ten years or so, we have seen a huge rise
in interest in “security vulnerabilities” in
• operating systems,
• embedded systems,
• SCADA systems etc.
• What changed?

5. What changed?
• In “Software Security: Building Security In…”, Gary
McGraw cites three main areas of change…

6. What changed?
• Connectivity
• “Let’s network everything!”

7. What changed?
• Extensibility
• “Dynamically loadable everything!”

8. What changed?
• Growth in complexity
• More code implies more bugs

9. Safety and security…
the difference?
• Possibly the most notable difference is in the
environment itself.
• Safety-related systems – mostly operate in a benign
environment that we can control to some extent.
• Security-critical systems operate in a openly malicious
environment where we might have no control over inputs,
access, and so on
• ..and…attackers are arbitrarily intelligent, well-funded and
motivated.
• “Programming Satan’s Computer” [Needham and Anderson]

10. Experiences with Security
• In the worst-case, we must assume systems will be
attacked in a way that we haven’t even thought of (let
alone tested)…
• Therefore any sole claim of “we’ve tested it lots”, or
“we’ve tested it more…” offer limited confidence for
security properties
• So – why not use formal methods and sound static
verification – i.e. Correctness-by-Construction?
• Examples: MULTOS CA, Tokeneer, SPARKSkein, and other
SPARK users.

11. Experiences with Security
• At a technical level, strong static verification offers
confidence in a number of basic areas:
• Absence of “dumb bugs”…
• Verification of specific program properties of
interest.
• But...no silver bullet. There is still a need for solid
requirements engineering, security engineering,
architecture and so on.

12. Trends and Concerns
• Trend: massive resurgence in interest in static
analysis in general. (How many companies selling
“security vulnerability finder” tools???)
• Good! Raises awareness and average maturity.
• Concern: Almost all effort aimed at “low hanging
fruit” of retrospective analysis of legacy code-
bases. Gives the impression that this is the only
thing you can do…

13. Trends and Concerns
• Trend: lots of effort in “enumeration” of defects,
vulnerabilities, etc.
• Examples: SANS “Top 25” list, CVE, CWE, ISO OWGV
effort etc.
• Good! Provides a starting point..
• Concern: Encourages “box ticking” mentality.
• Concern: Application-specific security requirements
(i.e. the interesting stuff) is an infinite set. Trying to
enumerate it is unlikely to terminate!

14. A Future?
• We have shown that the CbyC/SPARK approach can
yield strong results, particularly for the highest-grade
secure systems.
• But…many systems incorporate:
• Legacy code, COTS, “SOUP”, many languages,
developed by many teams…
• The boundary between “safety” and “security” is
becoming blurred – many systems exhibit “both”…

15. A Future?
• Can we use architecture to isolate (sub-)systems of
differing security requirements?
• Can we combine verification evidence from multiple
sources to gain confidence where it’s most needed?
• Formal methods for the most critical components?
• Test, isolate, and contain for the less critical?
• What can we do to contractualize and enforce the
boundary between such components?

16. Altran Praxis Limited
22 St. Lawrence Street, Southgate,
Bath BA1 1AN
United Kingdom
Telephone +44 (0) 1225 466991
Facsimile +44 (0) 1225 469006
Website www.altran-praxis.com
Email rod.chapman@altran-praxis.com