Learning in Memristive Neural Network Architectures Using Analog Backpropagation Circuits

Olga Krestinskaya; Khaled Nabil Salama; Alex James Pappachen

doi:10.1109/TCSI.2018.2866510

Learning in Memristive Neural Network Architectures Using Analog Backpropagation Circuits

Olga Krestinskaya, Khaled Nabil Salama, Alex James Pappachen

Department of Electrical and Computer Engineering

Research output: Contribution to journal › Article › peer-review

21 Citations (Scopus)

Abstract

The on-chip implementation of learning algorithms would speed up the training of neural networks in crossbar arrays. The circuit level design and implementation of a back-propagation algorithm using gradient descent operation for neural network architectures is an open problem. In this paper, we propose analog backpropagation learning circuits for various memristive learning architectures, such as deep neural network, binary neural network, multiple neural network, hierarchical temporal memory, and long short-term memory. The circuit design and verification are done using TSMC 180-nm CMOS process models and TiO₂-based memristor models. The application level validations of the system are done using XOR problem, MNIST character, and Yale face image databases.

Original language	English
Journal	IEEE Transactions on Circuits and Systems I: Regular Papers
DOIs	https://doi.org/10.1109/TCSI.2018.2866510
Publication status	Accepted/In press - Sep 19 2018

Keywords

Analog circuits
backpropagation
binary neural network
crossbar
deep neural network
hierarchical temporal memory
learning
long-short term memory
memristor
multiple neural network.

ASJC Scopus subject areas

Electrical and Electronic Engineering

Access to Document

10.1109/TCSI.2018.2866510

Fingerprint Dive into the research topics of 'Learning in Memristive Neural Network Architectures Using Analog Backpropagation Circuits'. Together they form a unique fingerprint.

Cite this

@article{0ef5fb840da74233805ab092f877a942,

title = "Learning in Memristive Neural Network Architectures Using Analog Backpropagation Circuits",

abstract = "The on-chip implementation of learning algorithms would speed up the training of neural networks in crossbar arrays. The circuit level design and implementation of a back-propagation algorithm using gradient descent operation for neural network architectures is an open problem. In this paper, we propose analog backpropagation learning circuits for various memristive learning architectures, such as deep neural network, binary neural network, multiple neural network, hierarchical temporal memory, and long short-term memory. The circuit design and verification are done using TSMC 180-nm CMOS process models and TiO₂-based memristor models. The application level validations of the system are done using XOR problem, MNIST character, and Yale face image databases.",

keywords = "Analog circuits, backpropagation, binary neural network, crossbar, deep neural network, hierarchical temporal memory, learning, long-short term memory, memristor, multiple neural network.",

author = "Olga Krestinskaya and Salama, {Khaled Nabil} and {James Pappachen}, Alex",

year = "2018",

month = sep,

day = "19",

doi = "10.1109/TCSI.2018.2866510",

language = "English",

journal = "IEEE Transactions on Circuits and Systems II: Express Briefs",

issn = "1549-8328",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - JOUR

T1 - Learning in Memristive Neural Network Architectures Using Analog Backpropagation Circuits

AU - Krestinskaya, Olga

AU - Salama, Khaled Nabil

AU - James Pappachen, Alex

PY - 2018/9/19

Y1 - 2018/9/19

N2 - The on-chip implementation of learning algorithms would speed up the training of neural networks in crossbar arrays. The circuit level design and implementation of a back-propagation algorithm using gradient descent operation for neural network architectures is an open problem. In this paper, we propose analog backpropagation learning circuits for various memristive learning architectures, such as deep neural network, binary neural network, multiple neural network, hierarchical temporal memory, and long short-term memory. The circuit design and verification are done using TSMC 180-nm CMOS process models and TiO₂-based memristor models. The application level validations of the system are done using XOR problem, MNIST character, and Yale face image databases.

AB - The on-chip implementation of learning algorithms would speed up the training of neural networks in crossbar arrays. The circuit level design and implementation of a back-propagation algorithm using gradient descent operation for neural network architectures is an open problem. In this paper, we propose analog backpropagation learning circuits for various memristive learning architectures, such as deep neural network, binary neural network, multiple neural network, hierarchical temporal memory, and long short-term memory. The circuit design and verification are done using TSMC 180-nm CMOS process models and TiO₂-based memristor models. The application level validations of the system are done using XOR problem, MNIST character, and Yale face image databases.

KW - Analog circuits

KW - backpropagation

KW - binary neural network

KW - crossbar

KW - deep neural network

KW - hierarchical temporal memory

KW - learning

KW - long-short term memory

KW - memristor

KW - multiple neural network.

UR - http://www.scopus.com/inward/record.url?scp=85053631277&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85053631277&partnerID=8YFLogxK

U2 - 10.1109/TCSI.2018.2866510

DO - 10.1109/TCSI.2018.2866510

M3 - Article

AN - SCOPUS:85053631277

JO - IEEE Transactions on Circuits and Systems II: Express Briefs

JF - IEEE Transactions on Circuits and Systems II: Express Briefs

SN - 1549-8328

ER -