Efficient design of biological sequences will have a great impact across many industry and healthcare domains. However, discovering improved sequences requires solving a difficult optimization problem. Traditionally, this challenge was approached by biologists through a model-free method known as “directed evolution”, the iterative process of random mutation and selection. As the ability to build models that capture the sequence-to-function map improves, such models can be used as oracles to screen sequences before committing to experiments. In recent years, interest in better algorithms that effectively use such oracles to outperform model-free approaches has intensified. These span from standard Bayesian Optimization approaches, to regularized generative models and adaptations of reinforcement learning. This primer will compare such algorithms based on a comprehensive set of criteria that are important from both machine learning and biological perspectives.