How does Batch Normalization Help Optimization?
8 801
12
Microsoft Research329 тыс
Следующее
09.05.19 – 5 2903:02Популярные
118 дней – 6276:41
334 дня – 1 8111:44Опубликовано 8 мая 2019, 23:40
Batch normalization (BatchNorm) is a widely adopted technique that enables faster and more stable training of deep neural networks. However, despite its pervasiveness, the exact reasons for BatchNorm's effectiveness are still poorly understood.
In this talk, we take a closer look at the underpinnings of the BatchNorm's success. In particular, we examine the popular belief that the root of BatchNorm's effectiveness is due to reduction of an effect called internal covariate shift (ICS). We then explore the connection between BatchNorm, ICS, and the optimization landscape of deep neural networks.
See more at microsoft.com/en-us/research/v...
In this talk, we take a closer look at the underpinnings of the BatchNorm's success. In particular, we examine the popular belief that the root of BatchNorm's effectiveness is due to reduction of an effect called internal covariate shift (ICS). We then explore the connection between BatchNorm, ICS, and the optimization landscape of deep neural networks.
See more at microsoft.com/en-us/research/v...
Свежие видео
19 часов – 21 03510:34
21 час – 521:40
1 день – 1 132 74626:56
1 день – 157 06123:24
2 дня – 89 8498:59
Случайные видео
32 дня – 3 010 5169:36
75 дней – 17 5674:22
94 дня – 5 1880:36
123 дня – 18 5890:13
13.07.21 – 58 33712:38
01.11.16 – 468 8728:01
3 часа – 4 9801:16
6 дней – 1 466 70121:11
12 дней – 1 27511:50
12 дней – 8614:31
13 дней – 9470:21
1 час – 1120:48