Initially developed for image processing, Convolutional Neural Networks (CNNs) have been applied to genomic data with promising results. This primer will trace some of the history of neural networks with an eye towards the practical lessons learnt along the way. Then building on the idea of the Position Weight Matrix as a motif detector we will explore exactly what convolution means when applied to a DNA sequence. While drawing examples from computer vision and natural language processing, our focus will be on the application of CNNs to genomic data. Lastly, we will cover recent advances in CNNs including residual connections and dilated convolutions.