DNA of the human genome is 2m long and is folded into a structure that fits in a cell nucleus. One of the central physical questions here is the question of scales: How can microscopic processes of molecular interactions of nanometer scale drive chromosomal organization at microns? Inferring principles of 3D organization of chromosomes from a range of biological data is a challenging biophysical problem. We develop a top-down approach to biophysical modeling of chromosomes. Starting with a minimal set of biologically motivated interactions we build polymer models of chromosome organization that can reproduce major features observed in Hi-C and microscopy experiments. I will present our work on modeling organization of human metaphase and interphase chromosomes.