#| Copyright © 2021 by Andrew Walter and Pete Manolios CS 4820 Fall 2021 Homework 6 Due: 11/4 (Midnight) For this assignment, work in groups of 2-3. Send Pete and Drew exactly one solution per team and make sure to follow the submission instructions on the course Web page. In particular, make sure that the subject of your email submission is "CS 4820 HWK 6". The group members are: ... |# #| In this homework, you will learn how to use Z3, a modern SMT (satisfiability modulo theories) solver from inside of ACL2s. Andrew has developed API bindings that provide a (property ...)-like interface to Z3. You'll develop a simple Sudoku solver using the Z3 bindings. You will also explore different ways of encoding problems and how that affects performance. SMT solvers combine SAT solvers with solvers for additional theories (for example, the theory of uninterpreted functions, or real arithmetic with addition and multiplication). In this way, SMT solvers can check the satisfiability of expressions that contain variables and functions from several different theories at once. Consider the following example: Let x and y be two strings. Is the following satisfiable? (^ (< (length x) 3) (< (length y) 3) (> (length (string-append x y)) 6)) It is not - the length of (string-append x y) is at most (length x) + (length y). We can state this as an ACL2s property: (property (x :string y :string) (=> (^ (< (length x) 3) (< (length y) 3)) (not (> (length (string-append x y)) 6)))) For an SMT solver to be able to report that this statement is UNSAT, it needs to understand how length and string-append relate to each other. If we ask our DP implementation from Homework 4 whether the propositional skeleton is satisfiable, it will report that it is SAT because it doesn't reason about length, <, >, string-append, etc. As you'll learn, we can ask Z3 to check the satisfiability of this statement using the following query (after setting up dependencies): (z3-assert (x :string y :string) (and (< (str.len x) 3) (< (str.len y) 3) (> (str.len (str.++ x y)) 6))) (check-sat) Z3 reports :UNSAT. Let's get started by first going through some setup instructions for the Z3-Lisp API. |# #| ===================================== = Z3 Setup = ===================================== You first need to install Z3 onto your system. Many package managers offer a prepackaged version of Z3, so it is likely easiest to install Z3 using your package manager rather than building it from source. If you're on macOS, Homebrew provides prebuilt Z3 packages as well. If using Windows Subsystem for Linux to run ACL2s, you should install Z3 into WSL rather than in "regular" Windows. Depending on your operating system, you may also need to install a "z3-dev" package. On Ubuntu, this package is called `libz3-dev`. You will also need a working C compiler to use the interface. On Ubuntu, the `build-essential` package should include everything you need, though it is fairly large and contains some unneeded software. One could also try just installing `gcc` or `clang`. After getting Z3 installed, you should be able to run it through the command line. To test this, execute `z3 --version` in your terminal and verify that it reports something along the lines of `Z3 version 4.8.12 - 64 bit` (your version or architecture may be different, that's OK). Now, we can install the Lisp-Z3 interface. Start by downloading the Lisp-Z3 Interface, gzipped tar file from the course webpage (right next to the HWK 6 link). This file should be extracted into your Quicklisp local-projects directory, which typically is located at ~/quicklisp/local-projects. I will refer to this directory as . After extracting the tar file, you should have a single folder, named lisp-z3, in . To test that everything has been installed properly, start SBCL and run the commands inside of the /lisp-z3/examples/basic.lisp file. If no errors occur, then you're all set and can proceed with the homework. If you run into any issues, ask questions on Piazza. |# ;; Exit out of ACL2s into raw Lisp :q (load "~/quicklisp/setup.lisp") (ql:quickload :lisp-z3) (defpackage :hwk6 (:use :cl :z3)) (in-package :hwk6) ;; Before we do anything, we must start Z3. (solver-init) ;; =========================== ;; Basics ;; =========================== ;; To use Z3, one adds one or more assertions to Z3 and then uses ;; (check-sat) to ask Z3 to perform a satisfiability check. ;; Let's try something simple first. ;; We want to know if the formula `x ^ y` is satisfiable. Let's add it ;; to Z3's stack: (z3-assert (x :bool y :bool) (and x y)) ;; We can see the contents of the solver using print-solver. ;;(print-solver) ;; Now, we can ask Z3 to check satisfiability: (check-sat) ;; We get an assignment: ((X T) (Y T)). This indicates that the set of ;; assertions we've added to Z3's stack is satisfiable, and provides a ;; satisfying assignment. ;; Note that Z3 still contains the stack of assertions; if we call ;; check-sat again, we'll get another satisfying assignment. (check-sat) ;; In this case the satisfying assignment is the same (since there is ;; only one distinct satisfying assignment for the formula `x ^ y`), ;; but in general the assignment may be different. ;; To clear the set of assignments, we can use `(solver-reset)`. (solver-reset) ;; ========================= ;; The Assertion Stack ;; ========================= ;; Sometimes, it may be useful to be able to remove just a subset of ;; assertions instead of resetting all of them. Z3 supports this with ;; the concept of scopes. ;; When a scope `S` is created, Z3 saves the set of assertions that ;; exist at that time. When `S` is popped, Z3 will return its set of ;; assertions to its state at the time `S` was created. ;; Let's see an example. ;; Create an initial scope that we can return to when we want an empty ;; set of assertions (solver-push) (z3-assert (x :bool y :int) (and x (>= y 5))) ;; This is SAT. (check-sat) ;; There is currently 1 assertion. (print-solver) ;; Let's create another scope, one that contains the above assertion. (solver-push) ;; We'll add an assertion that forces x to be false. (z3-assert (x :bool) (not x)) ;; Now the set of assertions is UNSAT! (check-sat) ;; There are now 2 assertions. (print-solver) ;; Let's pop off the top scope. This will remove the assertion we just ;; added. (solver-pop) ;; As expected, check-sat returns a satisfying assignment again. (check-sat) ;; We're back to the same set of assertions that we had when we ran ;; (solver-push) the second time. (print-solver) ;; We can pop back to the empty set of assertions that we had after we ;; reset the solver. (solver-pop) (print-solver) ;; ================== ;; Sorts ;; ================== ;; Z3 supports many variable types, which it calls "sorts". ;; We've already seen booleans and integers. (solver-push) (z3-assert (x :bool y :int z :string) (and x (> y 5) (= (str.len z) 3))) (check-sat) ;; Z3 also supports sequence types, including strings. (solver-push) (z3-assert (x (:seq :int) y :string) (and (> (seq.len x) 3) (> (str.len y) 3))) (check-sat) (solver-pop) ;; Here's another example showing more of the sequence operators. (solver-push) (z3-assert (x (:seq :int) y (:seq :int) z (:seq :int)) (and (> (seq.len x) 3) (> (seq.len y) 1) ;; x contains the subsequence consisting of the 0th element of y ;; this is equivalent to saying that x contains the 0th element of y (seq.contains x (seq.at y 0)) ;; z equals the concatenation of x and y (= z (seq.++ x y)))) (check-sat) (solver-pop) ;; You can define enumeration sorts as follows: (register-enum-sort :my-sort (a b c)) ;; this sort consists of one of the three values a, b, and c. ;; Now you can use this sort in assertions! (solver-push) (z3-assert (x :my-sort y :my-sort) (and (not (= x y)) ;; To represent an element of an enum, you need to use ;; `enumval` as shown here. (not (= x (enumval :my-sort a))) (not (= y (enumval :my-sort b))))) (check-sat) (solver-pop) #| Note that operations that may cause exceptions in other languages (like division by zero) are underspecified in Z3. This means that Z3 treats `(/ x 0)` as an uninterpreted function that it may assign any value to. This can lead to unexpected behavior if you're not careful. For example, Z3 reports that the following is satisfiable, since it can assign `x` and `y` different values, and has the flexibility to have division by 0 for the value of `x` return 3, and division by 0 for the value of `y` return 4. |# (solver-push) (z3-assert (x :int y :int) (and (= (/ x 0) 3) (= (/ y 0) 4))) (check-sat) (solver-pop) ;; There are many more operators and a few more sorts supported by Z3 ;; and the lisp-z3 interface. See the operators.md file in ;; /lisp-z3 for more information. The operator ;; documentation is also available on the course website (right next ;; to the HWK 6 link). Feel free to ask on Piazza if anything is ;; unclear. ;; One final note: sometimes, `check-sat` may not return an assignment ;; for some of the input variables provided to `z3-assert`. This often ;; is because Z3 is able to determine that the value of those ;; variables does not affect the satisfiability of the set of ;; assertions being checked, so it returns a partial assignment. If ;; you get a partial assignment, then all possible ways of extending ;; the partial assignment to total assignments are also assignments. ;; ========================== ;; Q1 ;; ========================== ;; 15 pts (3 pts each) ;; ;; For each of the following statements, encode the statement into a ;; SMT problem that Z3 can handle using `z3-assert` and report whether ;; the statement is satisfiable or not. ;; ;; As noted above, the list of operators supported by the Lisp-Z3 ;; interface is available in HTML format on the course website , ;; as well as in Markdown format in ;; /lisp-z3/operators.md. ;; 1a: ;; x, y, and z are boolean variables. ;; if x is true, then both y and z are true. ;; if y is true, then x is not true and z is not true. ;; if z is false, then y is false (solver-push) (z3-assert (...) ...) (check-sat) (solver-pop) ;; 1b: ;; x,y,z,p and q are all string variables. ;; the concatenation of x y and z is equal to the concatenation of p and q ;; all of the strings have at least length 2 ;; y starts with "ab" ;; p ends with "ba" (solver-push) (z3-assert (...) ...) (check-sat) (solver-pop) ;; 1c: ;; x is a sequence of booleans ;; y is an integer variable ;; y is between 0 and 32 inclusive ;; x has length equal to y ;; if x has at least one element and the element is true, then ;; the length of x is even. Otherwise, the length of x is odd. (solver-push) (z3-assert (...) ...) (check-sat) (solver-pop) ;; Now, we'll have some fun by encoding logic puzzles as SMT problems. ;; 1d: ;; (adapted from "What is the Name of This Book?" by Raymond Smullyan) ;; An island is inhabited by "knights" who always tell the truth, and ;; "knaves" who always lie. A stranger comes across three inhabitants ;; of this island standing together (Alice, Bob, and Clara) and asks ;; Alice "How many knights are among you?". Alice answers ;; indistinctly, and the stranger then asks Bob what Alice said. Bob ;; responds "Alice said there is one knight among us." Clara ;; interjects, saying "Don't believe Bob, he's lying!" ;; Is Bob a knight or a knave? Is Clara a knight or a knave? (solver-push) (z3-assert (...) ...) (check-sat) (solver-pop) ;; 1e: ;; (adapted from from "My best puzzles in logic and reasoning" by ;; Hubert Phillips, now public domain) #| Mr. Fireman, Mr. Guard, and Mr. Driver are (not necessarily respectively) the fireman, guard, and driver of an express train. Exactly one of the following statements is true: - Mr. Driver is not the guard - Mr. Fireman is not the driver - Mr. Driver is the driver - Mr. Fireman is not the guard Is the above set of constraints consistent? If so, who has what job? (hint: an enumeration sort might be helpful here) |# (solver-push) (z3-assert (...) ...) (check-sat) (solver-pop) ;; =========================== ;; Generating Constraints ;; =========================== ;; ;; It can get tedious to manually generate the constraints that encode ;; a particular problem. Since constraints are written using ;; S-expressions, we can use Lisp to generate constraints ;; programmatically. ;; Let's take a look at a very simple problem: we want to use Z3 to ;; find a normal semimagic square of order 3. A non-trivial semimagic ;; square of order `n` is an `n` x `n` grid of integers between 1 and ;; n^2 inclusive such that all of the rows and columns sum to the same ;; value. Since we did not specify that the magic square is ;; non-trivial, more than one cell in the square may have the same ;; value. ;; First, let's think about the constraints that we will need to ;; generate for this problem. We will likely want to encode each ;; square in the grid as an integer variable, and then add constraints ;; that state that the sums of the integer variables for each row and ;; column all equal the same value. ;; Let's consider an order-2 semimagic square. For this square, we ;; need 4 integer variables. I'll call them X0, X1, X2, and X3. They ;; need to have values between 1 and 4 inclusive, e.g. the following ;; conjunction must hold: #| (and (> X0 0) (<= X0 4) (> X1 0) (<= X1 4) (> X2 0) (<= X2 4) (> X3 0) (<= X3 4)) |# ;; Our square will look like this: #| +---------+ | X0 | X1 | +---------+ | X2 | X3 | +---------+ |# ;; Now, we need to encode that the sums of the rows and columns are ;; the same. I'll introduce another integer variable, S, to represent ;; the sum of the rows and columns. ;; The constraints are: ;; (= (+ X0 X1) S) ;; row 0 ;; (= (+ X2 X3) S) ;; row 1 ;; (= (+ X0 X2) S) ;; col 0 ;; (= (+ X1 X3) S) ;; col 1 ;; To write this in a form that `z3-assert` can understand, we need to ;; take the conjunction of the row and column constraints, and ;; generate a list defining each variable and its sort. We also need ;; to add in the constraints on the range of each variable. ;; In this case, we would generate: (z3-assert (X0 :int X1 :int X2 :int X3 :int S :int) (and (> X0 0) (<= X0 4) ;; x0 is within the appropriate range (> X1 0) (<= X1 4) ;; ditto for x1 (> X2 0) (<= X2 4) (> X3 0) (<= X3 4) (= (+ X0 X1) S) ;; row 0 (= (+ X2 X3) S) ;; row 1 (= (+ X0 X2) S) ;; col 0 (= (+ X1 X3) S))) ;; col 1 ;; We can use Z3 to determine if any such magic square exists: (check-sat) ;; Indeed, one exists: all of the squares are 1. Boring, but it works. (solver-reset) ;; Now, let's generate those constraints for an order-3 semimagic ;; square programmatically. ;; When dealing with a grid of variables, it is often useful to have a ;; way of transforming a pair of a row and column indices into the ;; variable at that location. I'll define such a function below. (defun get-3x3-magic-square-var (row col) ;; See the Common Lisp HyperSpec for more information about any ;; Common Lisp function. ;; For example, the documentation for `concatenate` can be found at ;; http://clhs.lisp.se/Body/f_concat.htm ;; You can also ask SBCL for documentation for a function ;; by running (describe #') in the REPL. ;; e.g. (describe #'concatenate) (intern (concatenate 'string "X" (write-to-string (+ col (* row 3)))))) ;; ;; This should give us the variable for the first cell, X0 (get-3x3-magic-square-var 0 0) ;; Don't worry if it prints out :INTERNAL afterwards - `intern` ;; actually returns multiple values (see the HyperSpec for more info) ;; Now, let's define a function that will generate the constraint that ;; states that the sum of a particular row should be equal to some variable. (defun 3x3-magic-square-row-sum (row sum-var) ;; I'm going to use the loop macro here. This is a very powerful ;; macro that allows us to avoid writing helper functions just to ;; perform basic loops. ;; See the HyperSpec and ;; https://gigamonkeys.com/book/loop-for-black-belts.html for more ;; information. ;; We want to first generate the variables corresponding to each ;; cell in this row. (let ((row-squares (loop for col below 3 collect (get-3x3-magic-square-var row col)))) ;; Then, we want to say that the sum of the squares is equal to ;; whatever the sum-var is. `(= ,sum-var (+ . ,row-squares)))) ;; Just as a sanity check, let's generate the constraint for the first row. (3x3-magic-square-row-sum 0 'S) ;; great, exactly as we expected. ;; Now, let's define a similar function for columns. (defun 3x3-magic-square-col-sum (col sum-var) (let ((col-squares (loop for row below 3 collect (get-3x3-magic-square-var row col)))) `(= ,sum-var (+ . ,col-squares)))) ;; Another sanity check... (3x3-magic-square-col-sum 0 'S) ;; looks good. ;; Now, let's put it all together. We want to generate the constraints ;; for all of the rows and all of the columns and take the conjunction ;; of them. (defun 3x3-magic-square-row-col-constraints (sum-var) (let ((row-constraints (loop for row below 3 collect (3x3-magic-square-row-sum row sum-var))) (col-constraints (loop for col below 3 collect (3x3-magic-square-col-sum col sum-var)))) ;; ,@ splices a list into an S-expression. e.g. `(1 ,@(list 2 3)) = '(1 2 3) `(and ,@row-constraints ,@col-constraints))) ;; Great, this is a conjunction of equalities, which is what we expect. (3x3-magic-square-row-col-constraints 'S) ;; Finally, we need to generate the list of variables and their sorts. (defun 3x3-magic-square-var-specs (sum-var) (let ((cell-vars (loop for row below 3 append (loop for col below 3 append `(,(get-3x3-magic-square-var row col) :int))))) `(,sum-var :int ,@cell-vars))) (3x3-magic-square-var-specs 'S) ;; We also need to assert that all of the values are between 1 and 9. (defun 3x3-magic-square-vars-between-1-and-9 () (cons 'and (loop for row below 3 append (loop for col below 3 append `((>= ,(get-3x3-magic-square-var row col) 1) (<= ,(get-3x3-magic-square-var row col) 9)))))) ;; Now, we just need to pass this information into `z3-assert`. ;; `z3-assert` is just a macro that calls `z3-assert-fn` on its quoted ;; input. All this means is that we can skip some shenanigans with ;; backquote and just pass the constraints and variable specifications ;; directly into `z3-assert-fn`. (solver-push) (z3-assert-fn (3x3-magic-square-var-specs 'S) (3x3-magic-square-row-col-constraints 'S)) (z3-assert-fn (3x3-magic-square-var-specs 'S) (3x3-magic-square-vars-between-1-and-9)) (check-sat) ;; We get our satisfying assignment, still boring (all 1s) but correct. (solver-pop) ;; You'll expand upon this in Q2 below. ;; ========================== ;; Q2 ;; ========================== ;; 25 pts ;; Use Z3 to find a normal non-trivial magic square of order 3. ;; You should use a similar approach to that shown above to ;; programmatically generate the constraints and pass them into ;; `z3-assert-fn`. ;; A magic square is a semimagic square that also satisfies the ;; property that all diagonals also sum to the same value as all of ;; the rows and columns. ;; A non-trivial magic square is a magic square such that no two cells ;; have the same value. ... ;; ========================== ;; Q3 ;; ========================== ;; 30 pts ;; ;; Develop a Sudoku solver that uses an approach similar to that ;; described above to use Z3 to generate solutions to a given starting ;; board. The top-level function should be called `solve-sudoku`, and ;; should take a starting board as an argument. The format of the ;; starting board will be defined below. `solve-sudoku` should return ;; either 'UNSAT (if no valid Sudoku board also satisfies the starting ;; board) or an assignment (a list of 2-element lists, similar to a ;; let binding, where the first element is the variable name and the ;; second is the assignment) that represents a filled-in Sudoku board ;; that satisfies the starting board's assignments. ;; ;; A valid Sudoku board is a 3x3 grid of 3x3 boxes. The 9 cells in ;; each box must all be integers from 1 to 9 inclusive, and must all ;; be different. Every row and column of the 9x9 Sudoku grid must ;; contain every integer from 1 to 9 inclusive exactly once. ;; ;; You will first use a bit-blasting encoding for this problem, ;; similar to the approach that I used to generate the example Sudoku ;; problems that I evaluated your DP algorithms using. What I mean by ;; this is that each Sudoku square will be represented by 9 variables, ;; one for each possible value it may have. ;; ;; A starting board consists of a standard 3x3 Sudoku board with only ;; a subset of cells having specified values. We will use _ to denote ;; unspecified values. The starting board will be represented by a ;; list of 81 elements, where each element can be an integer between 1 ;; and 9 inclusive or _. ;; ;; I have provided the function that generates a variable ;; corresponding to the Sudoku cell at a given row and column having a ;; particular value. If you use this function, you can (defun sudoku-cell-var (row col val) (intern (concatenate "X" (write-to-string (+ col (* row 9))) "_" (write-to-string val)))) ;; I have provided some utilities for pretty-printing Sudoku solutions ;; below. (defun assoc-equal (x a) (assoc x a :test #'equal)) (defun get-square-value (soln row col) (block outer (loop for i from 1 to 9 do (when (and (cdr (assoc-equal (sudoku-cell-var row col i) soln)) (cadr (assoc-equal (sudoku-cell-var row col i) soln))) (return-from outer i))) nil)) (defun pretty-print-sudoku-solution (n soln) (loop for row below 9 do (progn (terpri) (loop for col below 9 do (progn (format t "~A " (get-square-value soln row col)) (when (equal (mod col 3) 2) (print " ")))) (when (equal (mod row 3) 2) (terpri))))) ;; Here's an example starting board. It has a unique solution. (defconstant *sudoku-example-board* '(7 _ _ _ 1 _ _ _ _ _ 1 _ _ _ 3 7 _ 8 _ 5 3 _ _ _ _ _ 4 5 _ 9 3 _ _ _ _ 2 4 _ 1 2 6 _ 3 7 _ _ _ 7 _ 8 5 9 4 _ 2 7 _ _ 9 4 _ 3 _ 8 _ _ 5 _ 1 _ 6 _ _ 3 _ _ _ 2 4 5 _)) ;; Here's its solution. #| 7 4 8 9 1 6 5 2 3 6 1 2 4 5 3 7 9 8 9 5 3 7 2 8 6 1 4 5 6 9 3 4 7 1 8 2 4 8 1 2 6 9 3 7 5 3 2 7 1 8 5 9 4 6 2 7 5 6 9 4 8 3 1 8 9 4 5 3 1 2 6 7 1 3 6 8 7 2 4 5 9 |# ... (defun solve-sudoku (input-grid) ...) ;; ========================== ;; Q4 ;; ========================== ;; 30 pts ;; ;; 4a. (15 pts) ;; ;; Experiment with a different encoding for Sudoku cells. For example, ;; you could use integers to represent each square, or enumeration ;; sorts. You should define `solve-sudoku-alternate` below to behave ;; like `solve-sudoku` as described in Q3, except that it must use a ;; different encoding to represent the value of each Sudoku cell. ;; ;; You can continue to use the `pretty-print-sudoku-solution` function ;; I provided if you redefine `get-square-value` to work with your ;; variable encoding. ... (defun solve-sudoku-alternate (input-grid) ...) ;; 4b. (15 pts) ;; ;; Compare the performance of `solve-sudoku` and ;; `solve-sudoku-alternate`. Come up with the hardest Sudoku grid you ;; can find for each solver and explain why you think it is hard. ;; ;; Note that Z3 uses a variant of DPLL called DPLL(T) for solving SMT ;; problems. ;; ;; You may find it useful to see internal statistics that Z3 collects ;; during SMT solving. These statistics are cumulative, so you should ;; re-initialize Z3 before each query that you want to measure. ;; These statistics can be printed by calling `(z3::get-solver-stats)`. ;; ;; Unfortunately there is no single resource that describes what all ;; of the returned statistics means, but some statistics of note are: ;; - :conflicts: the number of conflicts found during DPLL ;; - :decisions: the number of DPLL decisions made ;; - :propagations: the number of times unit propagation was performed ;; - :restarts: the number of times that Z3 decided to restart DPLL ;; from the beginning, retaining learned conflict clauses (recall ;; nonchronological backtracking) ;; The hardest Sudoku board for your `solve-sudoku` implementation. (defconstant *hardest-sudoku-board* '( ... )) ;; The hardest Sudoku board for your `solve-sudoku-alternate` ;; implementation. (defconstant *hardest-sudoku-board-alternate* '( ... )) ;; ========================== ;; Extra Credit ;; ========================== ;; 25pts each ;; ========================== ;; E1 ;; ========================== ;; ;; Extend your Sudoku solver to support arbitrary-size Sudoku ;; puzzles. For n>3, you should continue to use numbers for cells ;; values greater than 9, rather than e.g. using hexadecimal values. ;; ========================== ;; E2 ;; ========================== ;; ;; Develop a solver for another logic puzzle by encoding it as ;; Z3 assertions using the Lisp-Z3 interface. ;; Any of the logic puzzles here are allowed: https://www.chiark.greenend.org.uk/~sgtatham/puzzles/ ;; Anything else needs to be OK'ed by Drew or Pete.