This document discusses the differences between tests and specifications, and how to write formal specifications for systems using TLA+. Tests check implementation details, while specifications check high-level design. TLA+ is introduced as a formal specification language that can be used to define abstract models of systems and check for complex bugs. Writing specifications in TLA+ helps validate system designs before implementation. Examples are provided showing how processes, variables, and properties can be specified in TLA+ to model behaviors like multi-device app installation.

1. Tests are not specs
How to write actual specifications with TLA+
Hillel Wayne
@hillelogram
hillel@esparklearning.com

2. Rules
• Interrupt me if I’m speaking too fast
• You don’t need to understand the code samples
• Ask questions if you have them
• I may punt them until later

3. Espark Learning
Or core, same thing

6. MDM
School

7. MDM
School

8. Business logic 

9. TDD
• Write a failing test
• Fix the failing test
• Refactor

11. Describe EmailSender
context “when called”
it “should send an email”
it “should not send two emails”

12. Describe EmailSender
context “when called”
context “and the email API is lagging out”
it “should send an email”
it “should not send two emails”

13. Describe EmailSender
context “when called”
context “and the email API is lagging out”
context “and there are three separate email services”
context “and our AP database has a partition”
it “should send an email”
it “should not send two emails”

14. TESTS ARE NOT SPECS

16. Programming Languages
• I/O
• Implementations
• Concrete
• Fast
• Designed to run
Specification Languages
• No I/O
• Definitions
• Abstract
• “Comprehensive”
• Designed to check

17. Maximum of Set
def maximum(numbers)
max = numbers.first
numbers.each do |n|
if n > max
max = n
end
max
end
Maximum(numbers) ==
CHOOSE max in numbers:
A n in numbers:
max >= n

19. Alice BobMoney

20. Variables alice_account = 4, bob_account = 0;
Process transfer in {1}
Variables to_transfer in 1..5
Begin
Withdraw:
alice_account := alice_account – to_transfer;
Deposit:
bob_account := bob_account + to_transfer;
End process;
NoOverDrafts == alice_account >= 0

23. 1 Withdraw Deposit
2 Withdraw Deposit
3 Withdraw Deposit
4 Withdraw Deposit
5 Withdraw Broken

24. Variables alice_account = 4, bob_account = 0;
Process transfer in {1}
Variables to_transfer in 1..5
Begin
Start:
if to_transfer <= alice_account then
Withdraw:
alice_account := alice_account – to_transfer;
Deposit:
bob_account := bob_account + to_transfer;
end if;
End process;

25. Variables alice_account = 4, bob_account = 0;
Process transfer in {1, 2}
Variables to_transfer in 1..5
Begin
Start:
if to_transfer <= alice_account then
Withdraw:
alice_account := alice_account – to_transfer;
Deposit:
bob_account := bob_account + to_transfer;
end if;
End process;

27. Check
Withdraw
Check
Withdraw
Deposit
Withdraw
WithdrawCheck

28. Variables alice_account = 4, bob_account = 0;
Process transfer in {1}
Variables to_transfer in 1..5
Begin
Start:
if to_transfer <= alice_account then
Withdraw:
alice_account := alice_account – to_transfer;
Deposit:
bob_account := bob_account + to_transfer;
end if;
End process;
<>[](alice_account + bob_account = 4)

30. Check Withdraw Crash

31. Use case
• For performance reasons, we should install the app on more than one
device per API call.
• For stability reasons, we could never show the MDM it was incorrect
about something. So if we tell it to install an app it thinks it installed,
the system wigs out.

32. EXTENDS Integers, TLC, Sequences
CONSTANTS Devices
(* --algorithm BatchInstall
variables
AppScope in [Devices -> {0, 1}];
Installs in [Devices -> BOOLEAN];
batch_pool = {};
lock = FALSE;
define
PoolNotEmpty == batch_pool # {}
end define
procedure ChangeAppScope()
variables cache;
begin
GetLock:
await ~lock;
lock := TRUE;
Cache:
cache := batch_pool;
Filter:
cache := cache intersect {d in Devices:
AppScope[d] = 0};
Add:
AppScope := [d in Devices |->
IF d in cache THEN AppScope[d] + 1
ELSE AppScope[d]
];
Clean:
batch_pool := batch_pool cache;
lock := FALSE;
return;
end procedure
fair process SyncDevice in Devices
begin
Sync:
if Installs[self] then
batch_pool := batch_pool union {self};
end if;
if PoolNotEmpty then
either
call ChangeAppScope();
or
skip;
end either;
end if;
end process;
fair process TimeLoop = 0
begin
Start:
while TRUE do
await PoolNotEmpty;
call ChangeAppScope();
end while;
end process
end algorithm;

33. Filter:
cache := cache intersect {d in Devices:
AppScope[d] = 0};
Clean:
batch_pool := batch_pool cache;

34. GetLock:
await ~lock;
lock := TRUE;

36. Slow

37. Unfriendly
• F[a] vs F(a) vs F a
• [A -> B] vs [A |-> B]
• = vs == vs := vs /= vs #
• Whitespace is sometimes significant
• …And More!

40. SPECS ARE NOT TESTS

41. Conclusions
• Tests check implementation of code
• Specs check design of system
• Formal specification helps find complicated bugs
• TLA+ will save your butt

42. Resources
• http://lamport.azurewebsites.net/tla/tla.html
• http://lamport.azurewebsites.net/tla/book.html
• https://learntla.com (me!)

44. Questions?
Hillel Wayne
h@learntla.com
Twitter: @hillelogram
Site: www.learntla.com