Analog memory is of great importance in neuromorphic engineering as it enables scalable neural network design and energy efficient implementation of computationally expensive operations. With the advent of memristors, the realization of the analog memory became possible due to the intrinsic properties of memristors such as nanoscale size, non-volatility, and energy efficiency. In hardware implementations of neural networks, memristors store the values of synaptic weights and operate similarly to the synapses that are reinforced with the application of external stimuli. Memristors that are ideally continuum memories, currently are at the early stage of the development, which causes several issues in neuromorphic circuit design. Device level and architecture level issues force memory engineers to approach memristive memory design in different ways. In this chapter device-level problems: restricted number of resistance states, stochastic switching and architecture level problem: sneak paths will be discussed, and their state of the art solutions will be presented.