This document contains a summary of 3 papers on deep residual networks and squeeze-and-excitation networks: 1. Kaiming He et al. "Deep Residual Learning for Image Recognition" which introduced residual networks for image recognition. 2. Andreas Veit et al. "Residual Networks Behave Like Ensembles of Relatively Shallow Networks" which analyzed how residual networks behave like ensembles. 3. Jie Hu et al. "Squeeze-and-Excitation Networks" which introduced squeeze-and-excitation blocks to help convolutional networks learn channel dependencies. The document also references the PyTorch ResNet implementation and provides URLs to the first and third papers. It contains non-English

1. 7 E 8 G A T N
I S @
0/21 43

2. b / 0GB I P Q
SEPPBN
0GB I P Q
EEP q i h :TPDK
DPP / MEEP %@KI GB I P Q
( @K PNE QPEK o v p g ku )-t
) (- ()cn
d
591 4 l ra m d )t
7 B 1 EPK 3BI A :NBAE@PEK ( ( P (.(-PB I
K ,% 2NK B BA
j ez
bsy BP@ w / DPP / I P Q GB % EIAK%@KI

3. a
L N D
..7 . 525 4. 6 3 6 1
P
. . P
a a N

4. I
tG
4103.5/
ü
ü
ü s r
m G
ü _ s
ü _
8 6 ) ( )
28:97 ( )
8 6 (
8 6 ) (
( 8 6 ) (
) 8 6 ( ) )
8 6 ) )( (
ü r
ü Sa
ü
ü n
T
E m G
G
tG I K r m G m e
uN T uN G
m G
uN

5. / 4985 .7367 3 6 / 8 3 7 .7367 3 6
e 1 38 r is 17 o
mP P
S mS mr
/ 4985 . P / 8 3 7 . P
0 4:8 P
r c
) ( (
v
v
7 2
7 :3
7 2
7 2
7 2
7 2
7 2
ü n
/ 4985 . o
u _
ü L
ü n ta
/ 8 3 7 . b l T
dK

6. a
L N D
..7 . 525 4. 6 3 6 1
P
. . P
a a N

7. p
L s U 8 6 8 ihms
n p 9 p g
ja n /2 4 2 ,
p o p , -
4
L t d
m :
2
2
7 1
p e x c R 7
p R
p n
10/. 7 . 86 2 5 9 3 : 5 8 8

8. [ –
8
– 7 7 m 0
7 ] 8
5 7 6 9
7 ] 7 7
8
7
1 2
1 2
1 2
1 2
1 2
1 2
1 2

10. ( ) 7
32px
32px
2

11. ) 7 2 (
32px
32px

12. NU
P r NUt 3u 6 65 9 7 3 9 oFs
3 n P rI e v v
s r N P i v s
Tc
% ((
ac
, (
1 9 9 6 0684 9 6
2 6 1 9 9 6 .4 6 0684 9 6
.4 6 1 9 9 6 2 6 0684 9 6
g U
U
Tc ∩ 1 9 9 6
Tc ∪ 1 9 9 6
l U
Tc % (( (
ac , ( ) ,
%, %
rf s

13. t s 119601 P ,
A ocey vo g ocrc Aio i
N o c sryrca
t t
t t
/57484 2 .23084 2

14. N
P
IP
∩ 01 2 3
∪ 01 2 3
0 U g7 8 = N
,2 3 e a i 0 g
v U se o e g
01 2 3 ,2 3

15. A
a
A
/ 10 324 W
A
A s
G
A
G
8
9 A u W
k n m S
G
t
9 e

16. A g
b P b
b b
.3 043n
i L B
c l a me
b tu8
0 D

19. a
L N D
..7 . 525 4. 6 3 6 1
P
. . P
a a N

20. Dk i u H
ü g p GL
o ac a
ü D g rPafmc ec
ü F af r i H tf scn lc
t iy tsh T g eccn
ü b Ho
ecCi h

21. a
L N D
..7 . 525 4. 6 3 6 1
P
. . P
a a N

22. u
b
vg i
2A 4C8 0
c o
ü /0 b
vg
ü /4C4
D9 8 C4C
b
ü eT
2A 4C8 0 c
ü N
o t
ah
/4C4
D9 8 C4C p
sdr
u
2D5 6 0 u
ü
ü 2D5 6 0 N
o
t
vg
ü
L 38BC
m
6B vg
vg
ü 11
ü P
p n
8C6
u
ü v
u
ü
ü
ü i
t
ü
ü
A 9 C 8BB
CA4BC
2477 9 A
9 p
ü 11
ahp
l
b l
( )

23. L
u g
eA
m r lv o nnt
o g
a
/ 5 0 3 .
bD cP B
0 0 4738 0 598
L
ru
/ 1652 . bDbD

24. 5A6 59B
( ( ( ( /-- -
v (.) - / . ( - ( // )
b (/, .( (. ,-/
L
t r
ü o )( )( - cmf t
ü s , ), s )
n t P
v n t ia
u T t
l %
4A 6 9 31
% P
26 6 0D 9 6
e
g
4D7 8 31l
l
( )
, - . /

25. 2 2 A8 6 2 9
/
D D D D
p
D x
989 2 ed7 9 n7 9
p
D x
25
2559 8 4 9 8
f o
R u L
R
x t p 1. °
° g r c
m b
0 9B2 6 /
P
a
2 2 A8 6 2 9
iv
l
0A3 94 / lm
m
l
G

26. n n
A83 8A 8B
28A08B
A83 8A 8B 28A08B
A83 8A 8B 28A08B
7
A83 8A 8B 7
5 8 8BD
A83 8A 8B 78 A 8B
783 8A 8B 4A 8B3 5 8
783 8A 8B
im 00 wr n saLN 28A08B v vgu s
w u ex t w s
o
e l
1 D4B8 /
pP
.4B4 C9 8 B4B
_
Rcb
d
1C5 6 / pP de
e
pP d

27. ) h m p
h
17G7 . A BG7G CB
f
n DC9 r
h
DC9
h
f b
z
h z h z ef z e
f044 h 044 v
h A y v
ACA BG A l ACA BG A v P
7 B B 7G h Ss
7 B B 7G G D
L
v DC9 r h 7AA7 c d
7AA7 h d
87G9 L 621
DC9 h DC9 r r DC9 h
r 17G7 . A BG7G CB
G 97 p c u h
G G 7G C
h v t v
CDG A 621 . 7A a
BC A7 L c v c 97 B f
97 B ) 9 D 7B n
7B CA CG7G CB 17G7 . A BG7G CB f n
7B CA D 17G7 . A BG7G CB f n
h f h f D G CB r n
h
m p
5 7G 3/
i w
go
17G7 . A BG7G CB
e
f
5 8 9 3/m
m
( )
-

28. u r
u
.1 1 , 594 1 7
s
4 36
u
g 4 36
c
ut
c p e
m m
ih m
D
4 36 m
m
A4
19 87 5
.1 1 , 594 1 7
o
n
u
r
0 7A1 4 /
v L
.1 1 , 594 1 7
l
Pba
c
0 2873 / v cv c

29. v oc
tP ocP
(
0 3984 / P
oc
(
0 8A2 5 / P
oc
t
2 2 6 5 2 8
rP
5 47
t
l 5 47
oc
t
ocP n h
v oc P
. 1 oc g y u
t 44 24B e
i P L tP
t
)
0 3984 / P
oc
0 8A2 5 / P
oc
t
p
0 8A2 5 /
v I
2 2 6 5 2 8
m
bUa
ocP
0 3984 / v ocv oc

30. 73 7 7A
73 7 7A
73 7 7A
73 7 7A
27 07A
27 07A
9A9 7 _
9A9 7
F n u
v 9A9 7
.1 i o
g
l NL P p
s
t
r
1 9 4A7 /
a
4A4 B8 7 A4A9
bm
Rc
e
1B5 96 / ee

31. Ri
g
0 9B4 7 .
SN L
4 4 A8 7 4 9
e
_P
0A5 96 . _u _
D
73 7 7
73 7 7
73 7 7
73 7 7
17 /7
17 /7
_P
a v
2A5 9 P
l c
s _P m r l c n2A5 9 P l
c B bo n

32. g 1A5:96 0 Lg
i
a
1 9B4 7 0
L K
P
.4 4 A8 7 4 9
_S
g
1A5:96 0 Lga
a
Lg
A 7 3
/7 4 A
A 7 3
A 7 3
A 7 3
A 7 3
A 7 3
2A5 9
l c
bv 2A5 9 l c rs e
u o to m ns u

33. i 0 2763 / mLi
l
a
0 6 1 4 /
mL n
P
1 1 5849 1 6 9
i
0 2763 / mLia
a
mLi
r l cD or E cD o
u g D v v x u
r r
cD o . 347 t
eb

34. 0 6 1 4 / te L
P
c
0 6 1 4 / t
e L
.1 1 5849 1 6 9
a
b uv
0 2763 / te brte b
B
D
L
D
g
mt l o 0 6 1 4 / l L
l A
0 6 1 4 /
l o 0 6 1 4 / l Ln i l

35. a
L N D
..7 . 525 4. 6 3 6 1
P
. . P
a a N

36. 31 4 s
URL: 1, 3 Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun ”Deep Residual Learning for Image Recognition” https://arxiv.org/abs/1512.03385
2 Andreas Veit, Michael Wilber, Serge Belongie “Residual Networks Behave Like Ensembles of Relatively Shallow Networks https://arxiv.org/abs/1605.06431”
5R 2
5
t N
t st

37. 4 torchvision https://github.com/pytorch/vision/blob/master/torchvision/models/resnet.py
B
URL: 4 Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun ”Deep Residual Learning for Image Recognition” https://arxiv.org/abs/1512.03385

38. 4 torchvision https://github.com/pytorch/vision/blob/master/torchvision/models/resnet.py
B
URL: 4 Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun ”Deep Residual Learning for Image Recognition” https://arxiv.org/abs/1512.03385

39. 6 / : 3 .
3 . /6/3 . 46 / . 3 4: 3 . /6/3 3 6 3 . /6/3 3 6 6 4:

40. ot q
- S
d R x t
x t e c S
sz eE c
d R x t sz E
C R c S
x t
x t
i u R ot q N S
Jie Hu, Li Shen, Gang Sun “Squeeze-and-Excitation Networks” https://arxiv.org/abs/1709.01507

41. a
L N D
..7 . 525 4. 6 3 6 1
P
. . P
a a N

42. C
vx
t t u o x o
n
o n
o n
o n x
e N_ rt o n x
c o g e n x
b s sx f rt o n
. yo n
rt o n
la i m

43. L Y p
L p E
pb S
z
L pD
B Cb S
OE
cA
/
/
/Ba gC
D
/
m
r
b S
Y
Nh
/
.
L Y p ue Fo m dktn p si

44. a
L N D
..7 . 525 4. 6 3 6 1
P
. . P
a a N

45. EP
aA 22 (2 ) g S a c a
2 aiok pnbMf T I e W ( pm LWe baS
22 (2 ) D

46. 3 3 .
CE c U W
U ) P
2 3 2 3 2 3 2 3 2 3 2 3
P V OC M O
A Ui Gc l( ( S a ) (
2 2 . ( 3 . c l( Sengn la l(

47. !