1. The document discusses dimension reduction techniques based on the geometry of dually flat spaces, including least angle regression (LARS) and the LASSO. 2. Bisector regression is introduced as an alternative to LARS and the LASSO for problems such as generalized linear models, Gaussian graphical models, and contingency tables. 3. Experimental results on real datasets show that bisector regression performs as well or better than LARS and the LASSO in terms of criteria like the AIC.

1. Dimension Reduction Based on
the Geometry of Dually Flat Spaces

2. • 2002.04 – 2006.03
• 2006.04 – 2012.03
24 5 28 2

3. 1.
2.
3.
4.
5.
24 5 28 3

4. 1.
2.
3.
4.
5.
24 5 28 4

5. •
¾
–
–
–
9
–
24 5 28 5

6. •
¾
–
–
•
¾
–
24 5 28 6

7. ‡
–
•
–
–
→
24 5 28 7

8. ‡
¾
– ridge LASSO
• LASSO L1
–
• LARS (Least Angle Regression)
–
– LASSO
24 5 28 8

9. LARS
• LARS (Least Angle Regression)
– Efron, Hastie, Johnstone and Tibshirani: Least Angle
Regression (with discussion). Annals of Statistics,
vol. 32 (2004), pp. 407-499.
–
–
–
–
9
9
‹ LASSO
24 5 28 9

10. 24 5 28 10

11. 24 5 28 11

12. 24 5 28 12

13. .
→ ≥
.
24 5 28 13

14. LARS
[Efron, et al. (2004)]
24 5 28 14

15. LASSO
[Efron, et al. (2004)]
24 5 28 15

16. Stagewise
[Efron, et al. (2004)]
24 5 28 16

17. LARS
24 5 28 17

18. ‡
–
–
–
•
–
–
24 5 28 18

19. •
•
•
– e- m-
• e- e-
–
• m- m-
–
24 5 28 19

20. 24 5 28 20

21. 24 5 28 21

22. • LARS
–
•
• LARS LASSO
–
LARS LASSO
24 5 28 22

23. 1.
2.
3.
4.
5.
24 5 28 23

24. • Bisector Regression
Least Angle Regression
• BRGLM
– Bisector Regression for Generalized Linear Models
• BRGGM
– Bisector Regression for Gaussian Graphical Models
• BRCT ←
– Bisector Regression for Contingency Tables
24 5 28 24

25. 24 5 28 25

26. 24 5 28 26

27. .
→ ≥
.
24 5 28 27

28. •
24 5 28 28

29. •
•
24 5 28 29

30. •
•
–
–
24 5 28 30

31. 1.
2.
3.
4.
5.
24 5 28 31

32. „
•
• Bisector regression
24 5 28 32

33. •
24 5 28 33

34. • e-
• m-
24 5 28 34

35. • covariance matrix
⊧ ∽ ∨⊾≩≪ ∩ ∨⊧∩≩≪ ∽ ⊾≩≪
• Concentration matrix
⊧ ⊡∱ ∽ ∨⊾ ≩≪ ∩ ⊡⊧⊡∱⊢≩≪ ∽ ⊾≩≪
24 5 28 35

36. •
•
24 5 28 36

37. e-
m-
24 5 28 37

38. Kullback-Leibler
•
• Kullback-Leibler
–
24 5 28 38

39. m-
e- m-
m-
24 5 28 39

40. Bisector Regression
•
–
–
1.
2.
3.
24 5 28 40

41. Bisector Regression GGM
1.
2.
3.
24 5 28 41

42. 24 5 28 42

43. 24 5 28 43

44. •
•
–
24 5 28 44

45. • GGM ≢
⊧ ≩≪
⊩ ⊡∱ :
⊪
≍ ∨⊤∩ ∽ ⊧ ≪ ⊾ ∽ ∰ ∨∨≩∻ ≪ ∩ ∶∲ ⊤∩
≢ ≢
∨⊧∩≩≪ ∽ ∨⊧≍≌≅∩≩≪ ∨ ≩∻ ≪ ∩ ∲ ⊤
≢
∨⊧⊡∱∩≩≪ ∽ ∰ ∨≩∻ ≪ ∩ ∶∲ ⊤
≢
⊧≍≌≅
24 5 28 45

46. BRGGM
24 5 28 46

47. Bisector Regression
•
–
–
–
•
–
24 5 28 47

48. Bisector Regression
•
–
• LASSO
24 5 28 48

49. LARS
• LARS
• BR
→
24 5 28 49

50. LARS
24 5 28 50

51. BRGGM
24 5 28 51

52. 24 5 28 52

53. 24 5 28 53

54. 1.
2.
3.
4.
5.
24 5 28 54

55. •
• South Africa Heart Disease (SAHD) dataset
• R ElemStatLearn
•
–
•
24 5 28 55

56. BRGLM Park and Hastie (2007)
24 5 28 56

57. GLM
•
• Training, Test
– 100 10
• AIC
• 1000
BRGLM Park and
Hastie(2007)
0.30 0.30
(0.050) (0.054)
24 5 28 57

58. •
•
• R SIN package
24 5 28 58

59. 1
2 5
3 4
1
2 5
3 4
1: , 2: , 3: , 4: , 5:
24 5 28 59

60. LASSOLin (2007)]
[Yuan and
1
2 5
3 4
1
2 5
3 4
1: , 2: , 3: , 4: , 5:
24 5 28 60

61. • BRGGM graphical LASSO [Friedman, et al. (2007)]
– Graphical LASSO R glasso
• 1000 trials
– Training, test: 1000 observations
• AIC
•
24 5 28 61

62. ≭≯≤≥≬ ≶≡≲≩≡≢≬≥ ≥≤≧≥ ≂≒≇≇≍ ≧≲≡≰≨≩≣≡≬ ≌≁≓≓≏
∨≩∩ ∵ ∳ ∱∮∰∱∱ ∨∰∮∰∲∳∩ ∱∮∰∶∵ ∨∰∮∰∲∴∩
∨≩≩∩ ∵ ∴ ∱∮∰∱∴ ∨∰∮∰∲∳∩ ∱∮∰∶∸ ∨∰∮∰∲∴∩
∨≩≩≩∩ ∶ ∷ ∱∮∲∲∸ ∨∰∮∰∲∶∩ ∱∮∲∹∱ ∨∰∮∰∲∷∩
∨≩≶∩ ∷ ∱∰ ∱∮∴∴∰ ∨∰∮∰∲∷∩ ∱∮∵∱∵ ∨∰∮∰∲∹∩
24 5 28 62

63. • BRGGM
–
• 100 trials
– 100 observations
≤ ≡ ≴≡≳≥≴ ≶≡ ≲≩≡≢ ≬≥ ≥≤ ≧ ≥ ≴≲≵ ≥ ≭ ≯ ≤ ≥≬
∨≡∩ ∴ ∳ ∱∰∰∯∱∰∰
∨≢ ∩ ∵ ∵ ∹∹∯∱∰∰
∨≣∩ ∶ ∸ ∹∹∯∱∰∰
∨≤ ∩ ∱∰ ∲∲ ∱∲∯∱∰∰
24 5 28 63

64. 1.
2.
3.
4.
5.
24 5 28 64

65. •
–
•
–
•
¾
¾
24 5 28 65

66. 24 5 28 66