Successfully reported this slideshow.
Upcoming SlideShare
×

# L1 Sudoku

3,615 views

Published on

Sudoku solving with CVXOPT - London Python Dojo

• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

### L1 Sudoku

1. 1. Sudoku solving with CVXOPT Sudoku solving with CVXOPT Ben Moran http://transnite.wordpress.com 09 December 2009
2. 2. Sudoku solving with CVXOPT Background 1 Background 2 Sudoku as matrix and vectors 3 Sparsity and solving 4 Conclusion
3. 3. Sudoku solving with CVXOPT Background
4. 4. Sudoku solving with CVXOPT Background Based on. . . a paper by Babu, Pelckmans, Stoica: Linear Systems, Sparse Solutions, and Sudoku 1 Solve Sudoku by turning it into a linear programming problem, inspired by new signal processing paradigm: compressed sensing 2 I thought it was interesting and reimplemented it in Python with CVXOPT 1 http://www.it.uu.se/katalog/praba420/Sudoku.pdf 2 http://nuit-blanche.blogspot.com/
5. 5. Sudoku solving with CVXOPT Background Why? Ben Laurie - Sudoku is a denial of service attack on human intellect It's a solved problem!
6. 6. Sudoku solving with CVXOPT Background Existing solutions Knuth's Dancing Links (DLX) 3 Norvig's constraint propagation (also Python) 4 Sinkhorn method 5 And many others 3 http://en.wikipedia.org/wiki/Dancing_Links 4 http://norvig.com/sudoku.html 5 http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?isnumber=4802290arnumber=4804
7. 7. Sudoku solving with CVXOPT Background But. . . This is still an interesting way into the brand new eld of compressed sensing And what have Sudoku got to do with. . . signal processing sensor networks single-pixel cameras satellite imaging? Our solution will be interesting, as the solver won't know the rules of Sudoku at all!
8. 8. Sudoku solving with CVXOPT Sudoku as matrix and vectors What are linear systems? Matrix - Vector multiplication 1 3 2 11 = 4 −1 3 5 Ax =b In Python: import numpy numpy . m a t r i x ( ' 1 3 ; 4 −1 ' ) ∗ numpy . m a t r i x ( ' 2 ; 3 ' ) matrix ( [ [ 1 1 ] , [ 5]])
9. 9. Sudoku solving with CVXOPT Sudoku as matrix and vectors Solving a linear system Given 1 3 11 A = and b = , what is x? 4 −1 5 In Python: import numpy numpy . l i n a l g . s o l v e ( numpy . m a t r i x ( ' 1 3 ; 4 −1 ' ) , numpy . m a t r i x ( ' 1 1 ; 5 ' ) ) matrix ( [ [ 2 . ] , [ 3.]])
10. 10. Sudoku solving with CVXOPT Sudoku as matrix and vectors Sudoku board as an indicator vector We can turn a 9x9 Sudoku board into a single vector with 9x9x9 = 729 elements Each 9 entries corresponds to one cell, most are zero Put a 1 in the rst place for 1, in the second place for 2. . . p = Problem ( 1 2 . . ,N=2) numpy . m a t r i x ( p . t o _ i n d i c a t o r _ v e c t o r ( ) , ' i ' ) matrix ( [ [ 1 ] , [ 0 ], [ 0 ], [ 1 ], [ 0 ], [ 0 ], [ 0 ], [ 0 ]])
11. 11. Sudoku solving with CVXOPT Sudoku as matrix and vectors Sudoku rules as a matrix system Now we can set up a special matrix to enforce the rules of Sudoku It will have one row for each constraint, and 9x9x9 = 729 columns When you multiply this with the indicator vector, you get all 1's if the board is valid Here are some of the rules for 2x2 Sudoku: p = Problem ( 1 2 . . ,N=2) numpy . m a t r i x ( p . m a t r i x ( ) ) matrix ( [ [ 1 , 1 , 0 , 0 , 0 , 0, 0, 0.] , [ 0, 0, 1, 1, 0, 0, 0, 0.] , [ 0, 0, 0, 0, 1, 1, 0, 0.] , [ 0, 0, 0, 0, 0, 0, 1, 1.] , [ 1, 0, 1, 0, 0, 0, 0, 0.] , ... [ 0, 0, 0, 1, 0, 0, 0, 0.]])
12. 12. Sudoku solving with CVXOPT Sudoku as matrix and vectors Rules for 2x2 Sudoku 1 2 _ _
13. 13. Sudoku solving with CVXOPT Sudoku as matrix and vectors _ _ _ 1 5 _ _ 7 _    1 _ 6 _ _ _ 8 2 _  3 _ _ 8 6 _ _ 4 _     9 _ _ 4 _ _ 5 6 7     _ _ 4 7 _ 8 3 _ _     7 3 2 _ _ 6 _ _ 4     _ 4 _ _ 8 1 _ _ 9        _ 1 7 _ _ _ 2 _ 8  _ 5 _ _ 3 7 _ _ _ Figure: Full size 9x9 board
14. 14. Sudoku solving with CVXOPT Sudoku as matrix and vectors Figure: Full size 9x9 rules matrix
15. 15. Sudoku solving with CVXOPT Sparsity and solving What do we gain by writing Sudoku like that? We can check solutions with a matrix-vector multiply Turn the proposed solution into an indicator vector Multiply it with the the rules matrix If the solution is valid, the answer will be all ones Solve Ax = [1 1 1 1 1] to nd the answer? . . . No - there is an innite number of answers, most won't give valid boards Need something else to nd our true correct answer
16. 16. Sudoku solving with CVXOPT Sparsity and solving Sparsity The true solution if it exists will be the sparsest (Sparse vector == mostly zeros) 0 3      1   1  0 0.1         Sparse  0  vs non-sparse  4      0 −2         1 1         0 .5 That still doesn't help. . . no ecient method to solve that directly
17. 17. Sudoku solving with CVXOPT Sparsity and solving Dierent kinds of distance: L-2 norm, Pythagoras You measure the length of a vector with a norm This is an L-p norm (for a 3d vector): (|x1 |p + |x2 |p + |x3 |p )1/p If you plug in p=2, you get normal Euclidean distance (Pythagoras' theorem): (|x1 |2 + |x2 |2 + |x3 |2 ) We can choose the solution with the smallest L2-norm. . .
18. 18. Sudoku solving with CVXOPT Sparsity and solving Dierent kinds of distance: L-2 norm, Pythagoras But we don't get a valid answer # Least squares s o l u t i o n M = numpy . m a t r i x ( s a m p l e ( ) . m a t r i x ( ) ) o n e s =numpy . o n e s ( (M. s h a p e [ 0 ] , 1 ) ) v = numpy . l i n a l g . p i n v (M) ∗ o n e s (M∗ v ) [ : 4 ] matrix ( [ [ 1 . ] , [ 1.] , [ 1.] , [ 1.]]) v [ : 4 ] matrix ([[ −0.0735114 ] , [ 0.24178992] , [ 0.00310795] , [ 0.28139045]])
19. 19. Sudoku solving with CVXOPT Sparsity and solving Dierent kinds of distance: L-0 norm, sparsity Plug in p=0 and you get L0, sparsity (number of non-zeros): |x1 |0 + |x2 |0 + |x3 |0 The right solution will have the lowest L0 norm But we've no way of nding it, besides solving the Sudoku!
20. 20. Sudoku solving with CVXOPT Sparsity and solving Dierent kinds of distance: L-1 norm With p=1 you get the absolute sum, taxicab distance |x1 |1 + |x2 |1 + |x3 |1
21. 21. Sudoku solving with CVXOPT Sparsity and solving Why is L-1 important? There are ecient methods of solving it (linear programming) And for many matrices, the minimum L1 norm solution turns out to also minimize L0 (Smallest L1 then nds the sparsest solution) This insight is central to the compressed sensing revolution
22. 22. Sudoku solving with CVXOPT Sparsity and solving Python - CVXOPT Now we can install python-cvxopt 6 and solve the problem! CVXOPT is a GPL library for optimization linear programming Other kinds of convex optimization We can use it to nd the solution x of Ax=b, with the smallest L1-norm The result comes out as a binary vector of 0s and 1s (without being constrained to do so!) 6 http://abel.ee.ucla.edu/cvxopt/
23. 23. Sudoku solving with CVXOPT Sparsity and solving Python - CVXOPT After all that work to set up the question, the answer is one line! h e l p c v x o p t . s o l v e r s . l p Help on f u n c t i o n l p i n module c v x o p t . c o n e p r o g : l p ( c , G , h , A=None , b=None , s o l v e r=None , . . . ) S o l v e s a p a i r o f p r i m a l and d u a l LPs minimize c ' ∗x s u b j e c t t o G∗ x + s = h A∗ x = b s = 0 ... # S o l v e w i t h CVXOPT c v x o p t . s o l v e r s . l p ( c0 , G , hh )
24. 24. Sudoku solving with CVXOPT Conclusion Caveats This won't solve every Sudoku - it tends to solve the easier ones It fails when the L1 minimum doesn't nd the L0 minimum There are other methods, from the compressed sensing literature, like iterative reweighting, that solve more For Sudoku, which puzzles are hard why is still an open problem at the frontiers of research
25. 25. Sudoku solving with CVXOPT Conclusion Compressed sensing Very many important, real world signals are sparse: images, sounds, gene expressions Instead of capturing the raw signal and compressing. . . . . . compressed sensing means you can capture them pre-compressed, with fewer measurements (below the Nyquist rate: 44.1kHz CDs to play back 22kHz frequencies) Less hardware needed will mean better images or cheaper kit Single-pixel camera 7 7 http://dsp.rice.edu/cscamera
26. 26. Sudoku solving with CVXOPT Conclusion Code, links and slides Code is at http://github.com/benmoran/L1-Sudoku/ Sporadic blog: http://transnite.wordpress.com Find CVXOPT at http://abel.ee.ucla.edu/cvxopt/examples/index.html And the nice CVXMOD wrapper at http://cvxmod.net