A striking feature of many sensory processing problems is that there appear to be many more neurons engaged in the internal representations of the signal than in its transduction. For example, humans have about 30,000 cochlear neurons, but at least a thousand times as many neurons in the auditory cortex. Such apparently redundant internal representations have sometimes been proposed as necessary to overcome neuronal noise. We instead posit that they directly subserve computations of interest. We first review how sparse overcomplete linear representations can be used for source separation, using a particularly difficult case, the HRTF cue (the differential filtering imposed on a source by its path from its origin to the cochlea) as an example. We then explore some robust and generic predictions about neuronal representations that follow from taking sparse linear representations as a model of neuronal sensory processing.