<p>We can store and retrieve specific patterns of activity in network models through synaptic plasticity mechanisms. When the synapses between cells in these models are bounded, then encoding a new pattern necessarily implies the partial erasure of previously stored ones. This overwriting or “palimpsest” property of networks with biologically constrained synapses has been studied intensively over the past 30 years. Most theoretical studies have focused on mechanisms for improving the memory capacity in such networks, which is starkly degraded through overwriting. However, there is another property of these memory systems which has not yet been fully explored. Namely, in the context of sensorydriven activity, ongoing learning can lead to the overwriting of some fraction of the synapses. This in turn leads to changes in the output of the network at any two distinct points in time, even if the input patterns have remained unchanged. This effect is reminiscent of the phenomenon of representational drift (RD), which has by now been wellestablished in the hippocampus, and other cortical areas. Recent experimental work has brought to light a number of puzzling findings regarding RD, which seem to defy simple explanation. These include the discovery that repetition rate can both reduce drift (in piriform cortex) and increase it (in hippocampus). In hippocampal place cells, RD has been shown to have differential effects on overall firing rates and spatial tuning. This suggests that there may be distinct underlying mechanisms. I will discuss how all of these findings are, in fact, consistent with the changes in activity observed in networks which store patterns through Hebbian plasticity. The fundamental assumption in such models is that memory storage is ongoing, and occurs between experimental sessions. The array of distinct and sometimes seemingly contradictory findings can be accounted for by differences in learning rates and correlations between input patterns.</p>