DSpace Repository

Robust Stochastic Gradient Descent with Student-t Distribution based First-order Momentum

Show simple item record

dc.contributor.author Ilboudo, Wendyam Eric Lionel
dc.contributor.author Kobayashi, Taisuke
dc.contributor.author Sugimoto, Kenji
dc.date.accessioned 2021-01-07T05:27:11Z
dc.date.available 2021-01-07T05:27:11Z
dc.date.issued 2020-12-16
dc.identifier.uri http://hdl.handle.net/10061/14204
dc.description.abstract Remarkable achievements by deep neural networks stand on the development of excellent stochastic gradient descent methods. Deep-learning-based machine learning algorithms, however, have to find patterns between observations and supervised signals, even though they may include some noise that hides the true relationship between them, more or less especially in the robotics domain. To perform well even with such noise, we expect them to be able to detect outliers and discard them when needed. We, therefore, propose a new stochastic gradient optimization method, whose robustness is directly built in the algorithm, using the robust student-t distribution as its core idea. We integrate our method to some of the latest stochastic gradient algorithms, and in particular, Adam, the popular optimizer, is modified through our method. The resultant algorithm, called t-Adam, along with the other stochastic gradient methods integrated with our core idea is shown to effectively outperform Adam and their original versions in terms of robustness against noise on diverse tasks, ranging from regression and classification to reinforcement learning problems. ja_JP
dc.language.iso en ja_JP
dc.publisher IEEE ja_JP
dc.relation.isreplacedby https://ieeexplore.ieee.org/document/9296551 ja_JP
dc.rights This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ ja_JP
dc.subject Training ja_JP
dc.subject Stochastic processes ja_JP
dc.subject Robustness ja_JP
dc.subject Estimation ja_JP
dc.subject Noise measurement ja_JP
dc.subject Proposals ja_JP
dc.subject Neural networks ja_JP
dc.title Robust Stochastic Gradient Descent with Student-t Distribution based First-order Momentum ja_JP
dc.type.nii Journal Article ja_JP
dc.contributor.alternative 小林, 泰介
dc.contributor.alternative 杉本, 謙二
dc.textversion none ja_JP
dc.identifier.eissn 2162-2388
dc.identifier.jtitle IEEE Transactions on Neural Networks and Learning Systems ja_JP
dc.identifier.spage 1 ja_JP
dc.identifier.epage 14 ja_JP
dc.relation.doi 10.1109/TNNLS.2020.3041755 ja_JP
dc.identifier.NAIST-ID 86635406 ja_JP
dc.identifier.NAIST-ID 74653270 ja_JP
dc.identifier.NAIST-ID 73292435 ja_JP

Files in this item

Files Size Format View

There are no files associated with this item.

This item appears in the following Collection(s)

Show simple item record

Search DSpace

Advanced Search


My Account