开发者

Backpropagation and batch training

Backpropagation calculates dW (weight delta) per weight per pattern, so 开发者_JS百科it's straightforward how to modify weights when doing stochastic training. How do I use it for batch training, though? Simply accumluate dW over the entire training set and then apply the modfication, or is there more to it?


You can do a lot with the different gradients from the different samples. That includes higher order information (approximate 2nd derivative) or conjugate gradient or natural gradient or ... :)


Yes, just accumluate dW over the entire training set. At least that is how I coded it back in grad school...

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜