Your SlideShare is downloading. ×
0
pptx - Psuedo Random Generator for Halfspaces
pptx - Psuedo Random Generator for Halfspaces
pptx - Psuedo Random Generator for Halfspaces
pptx - Psuedo Random Generator for Halfspaces
pptx - Psuedo Random Generator for Halfspaces
pptx - Psuedo Random Generator for Halfspaces
pptx - Psuedo Random Generator for Halfspaces
pptx - Psuedo Random Generator for Halfspaces
pptx - Psuedo Random Generator for Halfspaces
pptx - Psuedo Random Generator for Halfspaces
pptx - Psuedo Random Generator for Halfspaces
pptx - Psuedo Random Generator for Halfspaces
pptx - Psuedo Random Generator for Halfspaces
pptx - Psuedo Random Generator for Halfspaces
pptx - Psuedo Random Generator for Halfspaces
pptx - Psuedo Random Generator for Halfspaces
pptx - Psuedo Random Generator for Halfspaces
pptx - Psuedo Random Generator for Halfspaces
pptx - Psuedo Random Generator for Halfspaces
pptx - Psuedo Random Generator for Halfspaces
pptx - Psuedo Random Generator for Halfspaces
pptx - Psuedo Random Generator for Halfspaces
pptx - Psuedo Random Generator for Halfspaces
pptx - Psuedo Random Generator for Halfspaces
pptx - Psuedo Random Generator for Halfspaces
pptx - Psuedo Random Generator for Halfspaces
pptx - Psuedo Random Generator for Halfspaces
pptx - Psuedo Random Generator for Halfspaces
pptx - Psuedo Random Generator for Halfspaces
pptx - Psuedo Random Generator for Halfspaces
pptx - Psuedo Random Generator for Halfspaces
pptx - Psuedo Random Generator for Halfspaces
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

pptx - Psuedo Random Generator for Halfspaces

240

Published on

1 Comment
0 Likes
Statistics
Notes
  • Be the first to like this

No Downloads
Views
Total Views
240
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
4
Comments
1
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Design randomized algorithm, how to generate randomness?The first question we can ask our self is This is pretty hard…The second eaisier
  • Draw a dnf
  • Transcript

    • 1. Yi Wu (CMU)<br />Joint work with <br />ParikshitGopalan (MSR SVC) <br />Ryan O’Donnell (CMU)<br />David Zuckerman (UT Austin)<br />Pseudorandom Generators for Halfspaces<br />TexPoint fonts used in EMF. <br />Read the TexPoint manual before you delete this box.: AAAAA<br />
    • 2. Outline<br />Introduction<br />Pseudorandom Generators<br />Halfspaces<br />Pseudorandom Generators for Halfspaces<br />Our Result<br />Proof<br />Conclusion<br />2<br />
    • 3. Deterministic Algorithm<br />Program<br />Input<br />Output<br />The algorithm deterministically outputs the correct result.<br />3<br />
    • 4. Randomized Algorithm<br />Program<br />Input<br />Output<br />Random Bits.<br />The algorithm outputs the correct result with high probability.<br />4<br />
    • 5. Primality testing <br />ST-connectivity<br />Order statistics <br />Searching<br />Polynomial and matrix identity verification<br />Interactive proof systems<br />Faster algorithms for linear programming<br />Rounding linear program solutions to integer<br />Minimum spanning trees <br />shortest paths minimum cuts<br />Counting and enumeration<br />Matrix permanent <br />Counting combinatorial structures<br />Primality testing <br />ST-connectivity<br />Order statistics <br />Searching<br />Polynomial and matrix identity verification<br />Interactive proof systems<br />Faster algorithms for linear programming<br />Rounding linear program solutions to integer<br />Minimum spanning trees <br />shortest paths minimum cuts<br />Counting and enumeration<br />Matrix permanent <br />Counting combinatorial structures<br />Primality testing <br />ST-connectivity<br />Order statistics <br />Searching<br />Polynomial and matrix identity verification<br />Interactive proof systems<br />Faster algorithms for linear programming<br />Rounding linear program solutions to integer<br />Minimum spanning trees <br />shortest paths minimum cuts<br />Counting and enumeration<br />Matrix permanent <br />Counting combinatorial structures<br />Primality testing <br />ST-connectivity<br />Order statistics <br />Searching<br />Polynomial and matrix identity verification<br />Interactive proof systems<br />Faster algorithms for linear programming<br />Rounding linear program solutions to integer<br />Minimum spanning trees <br />shortest paths minimum cuts<br />Counting and enumeration<br />Matrix permanent <br />Counting combinatorial structures<br />Randomized Algorithms<br />5<br />
    • 6. Is Randomness Necessary?<br />Open Problem: <br />Can we simulate every randomized polynomial time algorithm by a deterministic polynomial time algorithm (the “BPP P” cojecture)? <br />Derandomization of randomized algorithms.<br />Primality testing [AKS]<br />ST-connectivity [Reingold]<br />Quadratic residues [?]<br />6<br />
    • 7. How to generate randomness?<br />Question: How togenerate randomness for every randomized algorithm?<br />Simpler Question: How to generate “pseudorandomness” for some class of programs?<br />7<br />
    • 8. Pseudorandom Generator (PRG)<br />Both program Answer Yes/No with almost the same probability<br />Yes /No<br />Yes/ No<br />Input<br />Program <br />Input<br />Program<br />n “pseudorandom” bit <br />PRG <br />Quality of the PRG: number of seed<br />n random bit <br />Seed<br />k<<n random bit <br />8<br />
    • 9. Why study PRGs?<br />Algorithmic Applications<br />When k = log (n), we can derandomize the algorithm in polynomial time.<br />Streaming Algorithm.<br />Complexity Theoretic Implications<br />Lower Bound of Circuit Class.<br />Learning Theory.<br />9<br />
    • 10. PRG for Classes of Program<br />Space Bounded Program [Nis92]<br />Constant-depth circuits [Nis91, Baz07, Bra09] <br />Halfspaces[DGJSV09, MZ09]<br />10<br />
    • 11. Outline<br />Introduction<br />Pseudorandom Generators<br />Halfspaces<br />Pseudorandom Generators for Halfspaces<br />Our Result<br />Proof<br />Conclusion<br />11<br />
    • 12. Halfspaces<br />-<br />-<br />-<br />-<br />-<br />-<br />+<br />+<br />+<br />-<br />-<br />+<br />+<br />+<br />-<br />+<br />+<br />-<br />-<br />+<br />+<br />+<br />+<br />Halfspaces: Boolean functions h:Rn -> {-1,1} of the form <br />h(x) = sgn(w1x1+…+wnxn- θ)<br /> where w1,…, wn,θ R. <br /><ul><li>Well-studied in complexity theory
    • 13. Widely used in Machine Learning: Perceptron, Winnow, boosting, Support Vector Machines, Lasso, Liner Regression.</li></ul>12<br />
    • 14. Product Distribution<br />For halfspace h(x), x is sampled from some product distribution; i.e., each xi is independently sampled from distribution Di .<br /> For example, each Dican be <br />Uniform distribution on {-1,1}<br />Uniform distribution on [-1,1]<br />Gaussian Distribution<br />13<br />
    • 15. Index <br />Introduction<br />Pseudorandom Generators<br />Halfspaces<br />Pseudorandom Generators for Halfspaces<br />Main Result<br />Proof<br />Conclusion<br />14<br />
    • 16. PRG for halfspaces<br />Both program Answer Yes/No with almost the same probability<br />Yes/No<br />Yes/No<br />h(x) = sign(w1x1+…+wnxn-θ)<br />h(x) = sign(w1x1+…+wnxn-θ)<br />Pseudorandom Variable<br /> x1, x2 …xn<br />PRG <br />x1, x2 …xnfrom some product distribution<br />k<<n random bit <br />15<br />
    • 17. Geometric Interpretation, PRG for uniform distribution over [-1,1]2<br />16<br />
    • 18. Geometric Interpretation, PRG for uniform distribution over [-1,1]2<br />Total Number of points = poly(dim)<br />Number of points in the halfspace is proportional to area.<br />17<br />
    • 19. Application to Machine Learning<br />-<br />-<br />-<br />-<br />-<br />-<br />+<br />+<br />+<br />-<br />-<br />+<br />+<br />+<br />-<br />+<br />+<br />-<br />-<br />+<br />+<br />+<br />+<br />How many testing points is it enough to estimate the accuracy of the N dimensional linear classifier?<br />Good PRG implies we only need deterministically check the accuracy on a set of poly(N) points!<br />18<br />
    • 20. Other Theoretical Applications<br />Discrepancy Set for Convex Polytopes<br />Circuit Lower bound on functions of halfspaces<br />Counting the Solution of Knapsacks<br />19<br />
    • 21. Outline <br />Introduction<br />Pseudorandom Generator<br />Halfspace<br />Pseudorandom Generators for Halfspaces<br />Our Results<br />Proof<br />Conclusion<br />20<br />
    • 22. Previous Result<br /> [DiGoJaSeVi,MeZu] PRG For Halfspace over uniform distribution on boolean cube ({-1,1}n) with seed length O(log n).<br />21<br />
    • 23. Our Results:Arbitrary Product Distributions<br />PRG for halfspaces under arbitrary product distribution over Rnwith the same seed length.<br />Only requirement: E[xi4] is a constant.<br />Gaussian Distribution<br />Uniform distribution on the solid cube.<br />Uniform distribution on the hypercube.<br />Biased distribution on the hypercube.<br />Almost any “natural distribution”<br />22<br />
    • 24. Our Results Functions of k-Halfspaces<br />PRG for the intersections of k-halfspaces with seed length k log (n). <br />PRG for arbitrary functions of k-halfspaces with seed length k2 log (n). <br />23<br />
    • 25. Outline<br />Introduction<br />Pseudorandom Generator<br />Halfspace<br />Pseudorandom Generators for Halfspaces<br />Our Result<br />Proof<br />Conclusion<br />24<br />
    • 26. Key Observation: Dichotomy of Halfspaces<br />Under product distributions , every halfspace is close to one of the following: <br />“Dictator” (halfspaces depending on very few variables, e.g. f(x) = sgn(x1)) <br />“majority”(no variables has too much weight, e.g. f(x) = sgn(x1+x2+x3+…+xn).<br />25<br />
    • 27. Dichotomy of weight distribution<br />Weights decreasing fast (Geometrically)<br />Weights are stable after certain index.<br />26<br />
    • 28. Weights Decrease fast (Geometrically)<br />Intuition: for sign(2n x1 + 2n-1 x2+ 2n-2 x3 +…xn)<br />If each xi is from {-1,1} , it is just sign(x1). <br />27<br />
    • 29. Weights are stable<br />Intuition: for sign(100 x1 + x2 + x3+…xn) <br />Then by for every fixing of x1, it is a majority on the rest of the variables.<br />28<br />
    • 30. Our PRG for Halfspace (Rough)<br />Randomly hashing all the coordinate into groups. <br />Use 4-wise independent distribution within each group. <br />If it is “Dictator-like”: All the important variables are in different groups.<br />If it is “Majority-like”<br />(x1 + x2 +.. xn ) is close to Gaussian. 4-wise independent Distribution (somehow) can handle Gaussian.<br />29<br />
    • 31. Outline<br />Introduction<br />Pseudorandom Generator<br />Halfspace<br />Pseudorandom Generators for Halfspaces<br />Our Result<br />Proof<br />Conclusion<br />30<br />
    • 32. Conclusion<br />We construct PRG for halfspaces under arbitrary product distribution and functions of k halfspaces with small seed length.<br />Future Work<br />Building PRG for larger classes of program; e.g., Polynomial Threshold function (SVM with polynomial kernel).<br />31<br />
    • 33. 32<br />

    ×