Digital holography is an imaging technique that enables recovery of topographic 3D information about an object under investigation. In digital holography, an interference pattern is recorded on a digital camera. Therefore, quantization of the recorded hologram is an integral part of the imaging process. We study the influence of quantization error in the recorded holograms on the fidelity of both the intensity and phase of the reconstructed image. We limit our analysis to the case of lensless Fourier off-axis digital holograms. We derive a theoretical model to predict the effect of quantization noise and we validate this model using experimental results. Based on this, we also show how the resultant noise in the reconstructed image, as well as the speckle that is inherent in digital holography, can be conveniently suppressed by standard speckle reduction techniques. We show that high-quality images can be obtained from binary holograms when speckle reduction is performed.