The Use of Formal Methods on the iFACTS Air Traffic Control Project
Upcoming SlideShare
Loading in...5
×
 

The Use of Formal Methods on the iFACTS Air Traffic Control Project

on

  • 4,406 views

The next talk in our series from the recent Open-DO Conference is from Neil White,Principal Engineer with Altran Praxis. His talk provides an overview of the formal methods used on the iFACTS project. ...

The next talk in our series from the recent Open-DO Conference is from Neil White,Principal Engineer with Altran Praxis. His talk provides an overview of the formal methods used on the iFACTS project. iFACTS is delivering increased Air Traffic Control capability to the UK.

Statistics

Views

Total Views
4,406
Views on SlideShare
3,926
Embed Views
480

Actions

Likes
2
Downloads
90
Comments
0

4 Embeds 480

http://www.open-do.org 470
http://www.slideshare.net 5
http://arstechnica.com 3
http://webcache.googleusercontent.com 2

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Document reference: S.P9999.99.99, issue 1.0 Page
  • Document reference: S.P9999.99.99, issue 1.0 Page Good morning If you were expecting Rod Chapman, and are thinking “gosh Rod’s let himself go”, then the key news is that I’m not Rod. I’m Neil White. I work as a principal engineer for Altran Praxis. More significantly, I’m the engineering manager for several projects, including the large formal methods project I’m talking about today. I also run the software practice. For those of you who don’t yet know, Altran Praxis was recently formed by the merger of Praxis and SC2. Altran has been Praxis’ parent for over 10 years, and SC2 was a sister company based in the south of France. The two companies have been working closely for a while, and bring different technologies together but with a very similar ethos. It’s a great mix. Public marriage of existing private relationship. In today’s context, Praxis brings a long history of formal methods from the UK base, and a long history of Agile methods from the French base.
  • Document reference: S.P9999.99.99, issue 1.0 Page In 30 mins I can deal with one topic in depth, or go for a rapid canter across a wider spectrum. I’m going for the latter, not least to ward off the snooze effect of that food! I’m going to talk generally about a project called iFACTS to set the context, and then I’m going to talk about formal methods through the project lifecycle and give a personal opinion of the pro’s and con’s. I ought to fess up: I’m a long term formalist with a passion for mathematics and rigor, but I also like success, and delivery, and especially profit, so formal methods have to work for me in industry.
  • Document reference: S.P9999.99.99, issue 1.0 Page iFACTS is an ATC system being procured by NATS – the UK air traffic provider. NATS are world leaders in ATC innovation. Partly through necessity; UK ATC is very very busy because of the dense population and our geographic position below transatlantic air traffic. I can only scratch the surface of the project. There is loads more detail on the NATS website
  • Document reference: SPARK User Group, issue 1.1 Page UK airspace is divided into sectors.
  • Document reference: SPARK User Group, issue 1.1 Page Each sector has a team of 3 (currently) looking after the traffic in that sector. The planner accepts aircraft into the sector, and arranges for them to be handed off to the next sector. Arrangements include height, speed, heading, etc. He talks to other planning controllers. Ne deals with the boundary. The tactical is the controller in comms with the aircraft. She basically gets them from the in to the out of this sector. She deals with the interier. The assistant prints paper strips and generally helps out. Increased capacity comes through more sectors and thus more controllers. Except we have hit the limit. The hand-over burden now outweighs getting a new sector. We need tools to help.
  • Document reference: S.P9999.99.99, issue 1.0 Page iFACTS will allow greater capacity in the existing sectors through the provision of new tools.
  • Document reference: S.P9999.99.99, issue 1.0 Page We replace the paper flight strips with electronic ones. Not a great computer challenge. (Big usage challenge though.) An enabler. Enabler because once we have all the data that’s currently on paper into the system, we can do things with it. We create a trajectory through space and time for each flight. We add uncertainty as a cone along the trajectory. The closer to “now”, the more certain you can be. Some maneuvers increase or decrease uncertainty. We can then compare every trajectory with every other trajectory to identify possible conflicts up to 15 mins in advance. Currently controllers work with a much shorter look-ahead of only a few minutes. This also gives the controllers a “what if” capability so that we don’t maneuver aircraft into annoying places in the first place! So as you can see, we augmenting – not replacing – the current system. “ Biggest advance in ATC since the introduction of Radar”
  • Page This is an example of part of the HMI. Each symbol is a pair of aircraft. This is time to the closest approach between a pair of aircraft. This is the distance at closest approach. Note this says nothing about current gap. Symbols tell you the attitude of the approach. Colors tell you something about severity. White is a deviation – an aircraft not doing what it’s told. Document reference: SPARK User Group, issue 1.1
  • Document reference: S.P9999.99.99, issue 1.0 Page So lets’ start looking at formal methods.
  • Document reference: S.P9999.99.99, issue 1.0 Page The specification is large, and split into a couple of technologies. The dominant part is a formal Z specification. There is some inherited mathematics defining algorithms. We could re-write in Z, but it costs, and can only add defects. We don’t! It’s already unambiguous. We just tie functions to Z. The HMI specification is in state tables. A small amount – eg stating non-functional requirements on performance or resource usage is in English commentary.
  • Random bit of Z. Not expected to read this! English description and (more detailed) mathematical description. This is a schema. These are variables with types. This is a mathematical relationship between the variables. We can generate the document with and without the mathematics for distinct readerships. The English needs to work in both. The maths has more detail. Extends, doesn’t contradict, the English. 4250 pages. All customer reviewed. Everything flows from this: design, code, test, everything.
  • Document reference: S.P9999.99.99, issue 1.0 Page How do we get a body of Z engineers? Reading and writing are different skills. Teaching reading is easy, and we have a lot of data to support that. People are up to speed fast. We can teach almost anyone who si not scared of basic maths. Teaching writing is harder. We pre-select harder, and the learning curve is longer and steeper. Some people don’t make it. Not a surprise – not everyone can do anything. There are people who will never write good code, or write good tests.
  • Document reference: S.P9999.99.99, issue 1.0 Page Tools support is a key issues. We use Word. We don’t love it! However, when teaching people Z you really don’t want to simultaneously teach them other tools too. Fight selective battles! The template includes a Z font, an ability to kick off the FuZZ type checker, and the ability to launch a graphical analysis too that shows you the linkage and structure of the specification.
  • Document reference: S.P9999.99.99, issue 1.0 Page Word has made for an easier environment for new users. But it retains all the usual problems of large word documents. In particular, when developing a branch the merge can be tortuous. Binary word files means that you have little option but to use Word to do the merge. Going forward, a Z-aware merge tool for the underlying OO XML might be one option to help merge.
  • Document reference: S.P9999.99.99, issue 1.0 Page The HMI spec is a simple state machine Describe… We could clearly draw this, but we get more material on the page in tables. Leads to a clear mapping to code if we want. Under the look-and-feel, we tie operations into the Z. So a button press is the trigger for a Z operation. A text field is the output from a Z operation.
  • Document reference: S.P9999.99.99, issue 1.0 Page Beauty of this is the sheer simplicity.
  • Document reference: S.P9999.99.99, issue 1.0 Page
  • Document reference: S.P9999.99.99, issue 1.0 Page I don’t want to shock, but I’m actually going to pass up the opportunity to extol the virtues of SPARK. I think this audience is pretty SPARK-aware, but please grab me later if you want to talk about the language. In summary, it’s an annotated subset of Ada which is designed for people who want their programs to be safe, secure, or frankly just correct. We have 150KLSLOC, all of which has a proof of the absence of any possible Ada exception.
  • Document reference: S.P9999.99.99, issue 1.0 Page Again, we have trained a lot of people with a diverse background. All our SPARK coders read Z and do proof. Note that we are not doing a correctness proof. It’s not cost-effective for this project. The level of integrity doesn’t warrant the work. (Remember my comment on profit!)
  • Document reference: S.P9999.99.99, issue 1.0 Page In comparison to Z, the SPARK toolset is mature and excellent. And again, please see me for details or your AdaCore rep for a very reasonable quotation!
  • Document reference: S.P9999.99.99, issue 1.0 Page
  • Our testing is driven by the Z. We require specification and code coverage. We devise possible conditions by analysis of the mathematics. Partition analysis and equivalence classes. We write these in a Z-like notation.
  • Keeping test under control is – however – a big challenge. Just because you can devise a test case, doesn’t mean that you can afford to generate the test, or that it’s a good test. How many in this small example?
  • Document reference: S.P9999.99.99, issue 1.0 Page Far too many! We triage out the low-value test conditions. Drop duplicates or contradictions. Then by carefully crafting scripts we can knock off a large number of conditions in one go. This is an activity that takes skill and domain knowledge. If you take the easy option “we will test that” then out test program will grow to commercially un-viable proportions. Too long, and too costly. But of course, we do need to be sure of the verification – we need a safety argument at the end of the day! In summary: Maths tells you all possible tests, but so many that we could test for years. Trick is to pick high value tests
  • Document reference: S.P9999.99.99, issue 1.0 Page We use an independent implementation to help with the detailed trajectory algorithm testing. We use the “reference model” to generate the expected outcomes. Diverse implementation notation and programmer, so risk of common failure. (Outside of spec error.)
  • Document reference: S.P9999.99.99, issue 1.0 Page Interestingly, the reference model has proven very accurate. (Although not not eg fast enough for real use.) Need to be careful not to draw too many conclusions from a small study. Worthy of further evaluation. We could throw loads of tests at this automatically!
  • Document reference: S.P9999.99.99, issue 1.0 Page Conclusion Formal methods – being unambiguous – help us throughout the life cycle. We have a project of over 100 engineers, and have had precisely zero scaling problems in the technologies. In particular, the oft-cited problem reasons of “training” and “learning” doesn’t hold true. (Which of course argues that we can train any number of Ada programmers too.) The achilles heel – if there is one – is the tool support. We don’t have enough tool support, and enough integrated tooling. There are exceptions – like the Examiner – but for Z etc. we need more.
  • Document reference: S.P9999.99.99, issue 1.0 Page Document Control Altran Praxis Limited, 20 Manvers Street, Bath BA1 1PX. Copyright © Altran Praxis. All rights reserved. Changes history Issue 0.1 (date): Changes forecast

The Use of Formal Methods on the iFACTS Air Traffic Control Project The Use of Formal Methods on the iFACTS Air Traffic Control Project Presentation Transcript

  •  
  • Formal Methods in Air Traffic Control Neil White Copyright © Altran Praxis Open-DO
  • Agenda
    • A quick introduction
      • What is iFACTS?
    • Formal methods for Specification
      • Z, State machines.
    • Formal methods for Implementation
      • Implementation: SPARK.
    • Formal methods for Test
      • Verification: more Z, Mathematica.
    Copyright © Altran Praxis
  • Context
    • NATS, the UK’s leading air traffic services provider, has pioneered research and development of advanced air traffic control tools for several years from its simulator and research centre.  The iFACTS project will deliver a subset of these tools onto the system at the company’s main en-route Control Centre at Swanwick. 
    • Further information is available at: www.nats.co.uk/article/218/62/nats_pioneers_biggest_atc_advance_since_radar.html
    Copyright © Altran Praxis
  • UK Air Traffic Control Copyright © Altran Praxis limited 2010
          • The Notes of this slide and the previous slide give some guidance on style and usage.
    Copyright © Altran Praxis limited 2010
  • Why iFACTS?
    • iFACTS – Interim Future Area Control Tools Support – will further improve safety and provide Controllers with a set of advanced support tools, which will  enable them to increase the amount of traffic they can comfortably handle.  In trials, the system has delivered significant capacity increases.
    Copyright © Altran Praxis
  • What is iFACTS?
    • iFACTS provides tools to support the controllers
      • Electronic flight strips replace the paper flight strips.
      • Trajectory tools - including prediction, deviation alerts, and conflict detection – are added.
    • iFACTS is not an Air Traffic control system
      • Integrated with, but sits alongside, the existing system.
    Copyright © Altran Praxis
  • Medium Term Conflict Detection: Separation Monitor Copyright © Altran Praxis limited 2010
  • Agenda
    • A quick introduction
      • What is iFACTS?
    • Formal methods for Specification
      • Z, State machines.
    • Formal methods for Implementation
      • Implementation: SPARK.
    • Formal methods for Test
      • Verification: more Z, Mathematica.
    Copyright © Altran Praxis
  • The complete iFACTS specification
    • The functional specification
      • Z
    • The algorithm specification
      • Maths
    • The HMI specification
      • State tables
    • The rest of the specification!
      • English
    Copyright © Altran Praxis
  • The Z specification
  • Z training
    • Z reader training
      • 3 day course; fluency then comes after 1 week on the job.
      • We have trained 75 people to read Z.
      • Engineers, domain experts, ATCOs.
    • Z writer training
      • 3 day course, fluency then comes after 3 months on the job.
      • We have trained 11 people to write Z.
      • All engineers.
    Copyright © Altran Praxis
  • Z tools
    • Z written in Microsoft Word
      • To get acceptance, you need to work with what people know.
      • Supported by Word Add-ins.
        • A Z character set.
        • A simple interface to the fuzz type checker.
        • A graphical representation tool.
    Copyright © Altran Praxis
  • Z tools
    • Advantages
      • Easy to develop commentary and Z together.
      • Hyper linking of fuzz errors back to source.
      • Cross-referencing of Z names in final document.
    • Disadvantages
      • All the problems of large word documents.
      • Tools can be slow on 1000 page documents.
      • Merging branches is painful.
    • The Future
      • Open Office XML?
    Copyright © Altran Praxis
  • The state machine specification
    • Button 1 Checkbox 1
    • State 1 State 2 N/A
    • State 2 State 1 State 3
    • State 3 State 1 State 2
    • Transition Actions
    • State 1 -> State 2 : De-select Checkbox 1
    Copyright © Altran Praxis
  • State machine training & tools
    • Training
      • So trivial that we don’t train!
      • People “just get it”.
    • Tools
      • Err …. None.
    Copyright © Altran Praxis
  • Agenda
    • A quick introduction
      • What is iFACTS?
    • Formal methods for Specification
      • Z, State machines.
    • Formal methods for Implementation
      • Implementation: SPARK.
    • Formal methods for Test
      • Verification: more Z, Mathematica.
    Copyright © Altran Praxis
  • The SPARK Implementation
    • SPARK Ada
      • An annotated subset of Ada.
    • 150 KSLOC (Logical)
    • RTE Proof
      • Formal partial correctness proof against specification not considered cost-effective.
    Copyright © Altran Praxis
  • Code
  • SPARK Training
    • 57 people trained in SPARK
      • Mostly contractors and clients.
      • Diverse programming background.
      • All SPARK coders are also Z readers.
    • Effective as SPARK coders immediately
    • Picking up RTE proof takes longer.
      • About 2 months.
    • How long to pick up formal correctness proofs?
      • No data, but I suspect longer again.
    Copyright © Altran Praxis
  • SPARK Tools
    • The SPARK toolset
      • Examiner.
      • Proof Simplifier.
      • Proof Checker.
    • See me later!
    Copyright © Altran Praxis
  • Agenda
    • A quick introduction
      • What is iFACTS?
    • Formal methods for Specification
      • Z, State machines.
    • Formal methods for Implementation
      • Implementation: SPARK.
    • Formal methods for Test
      • Verification: more Z, Mathematica.
    Copyright © Altran Praxis
  • Test Design
  • The Challenge of Test Design How many potential tests for this fragment?
  • The Challenge of Test Design
    • If you just turn the handle there are 1134 conditions to test.
    • But if you work at it hard enough you can cover the required subset in just 6 test scripts.
    • Formal methods are not a substitute for initiative.
    Copyright © Altran Praxis
  • Test reference models
    • Algorithms are specified in pure mathematics.
      • Working out the expected answer for test cases is very difficult and error prone.
    • We generate test cases as usual.
    • We create a test reference implementation in Mathematica.
    • We do back-to-back testing of iFACTS against the reference.
      • Diverse tools and implementers reduce the possibility of a common failure.
    Copyright © Altran Praxis
  • Mathematica tools & training
    • Small team – only 5 trained.
    • Reference model has similar defect density to SPARK implementation.
    • Limited conclusions to draw from such a small activity.
    Copyright © Altran Praxis
  • Conclusions
    • Formal methods are applicable to all phases of the lifecycle.
    • Training engineers is not a barrier
      • It’s a one-off cost
      • Our data shows that training is easy and cheap.
    • Tool support is vital
      • The Achilles heel of formal methods
        • Except the SPARK Examiner!
    Copyright © Altran Praxis
    • Altran Praxis Limited
    • 20 Manvers Street
    • Bath BA1 1PX
    • United Kingdom
    • Telephone: +44 (0) 1225 466991
    • Facsimile: +44 (0) 1225 469006
    • Website: www.altran-praxis.com
    • Email: neil.white@altran-praxis.com
    Copyright © Altran Praxis