In this primer, we will review methods for learning to optimize treatment decisions based on patient characteristics. Our focus will be on observational settings, where data is collected during routine care, e.g., via the medical record. We will first review the fundamental challenge of causal inference from observational data, and build intuition for the assumptions required to evaluate policies in this setting. We will then discuss techniques for evaluation of a specific policy, while demonstrating that quantities like the average treatment effect can be viewed from this general perspective. Finally, we will discuss how policy evaluation can be used within a general optimization framework to learn treatment policies directly from data.