Infectious diseases are a major burden to global health. Understanding their mechanisms and being able to predict and intervene epidemic outbreaks is an important challenge for researchers and decision makers alike. It should not be too hard either―if we include human contact patterns, the mechanisms of contagion and the typical features of the disease, we could model most infectious-disease related phenomena. Of these three components, the network epidemiology of the last decade has shown that our limited understanding of human contact patterns is probably the most important focus are for advancing infectious disease epidemiology. We will discuss what is known about human contact patterns and how to include this knowledge in epidemic modeling. First, we discuss recent work on what the epidemiologically most important temporal structures of human contacts are. We use about 80 empirical temporal network datasets, several arguably important for disease spreading, and scan the entire parameter space of disease-spreading models. By comparing to null-models, we identify important, simple temporal patterns that affect disease spreading stronger than the bursty interevent time distributions. Furthermore, we investigate how to eliminate the temporal information to make an as relevant static network as possible. After all, static network epidemiology has more methods and results than temporal network epidemiology and it for some purposes it is necessary. We find that an “exponential threshold” representation almost always the best performance, but time-sliced network (with a carefully chosen window, usually considerably different than the sampling time of the data) works almost as good. In contrast, networks of concurrent contacts do not seem to carry so important information.