We investigate the potential of supervised machine learning to propagate a quantum system in time. While Markovian dynamics can be learned easily, given a sufficient amount of data, non-Markovian systems are non-trivial and their description requires the memory knowledge of past states. Here we analyze the feature of such memory by taking a simple 1D Heisenberg model as many-body Hamiltonian, and construct a non-Markovian description by representing the system over the single-particle reduced density matrix. The number of past states required for this representation to reproduce the time-dependent dynamics is found to grow exponentially with the number of spins and with the density of the system spectrum. Most importantly, we demonstrate that neural networks can work as time propagators at any time in the future and that they can be concatenated in time forming an autoregression. Such neural-network autoregression can be used to generate long-time and arbitrary dense time trajectories. Finally, we investigate the time resolution needed to represent the system memory. We find two regimes: For fine memory samplings the memory needed remains constant, while longer memories are required for coarse samplings, although the total number of time steps remains constant. The boundary between these two regimes is set by the period corresponding to the highest frequency in the system spectrum, demonstrating that neural network can overcome the limitation set by the Shannon-Nyquist sampling theorem.