This chapter offers insight into the notion of uncertainty, which shapes our daily life. It provides an overview of the development of theoretical models with the help of elementary mathematical concepts to study the theory of uncertainty. The theory is explained in a staircase manner to join the links between the fundamental concepts of probability and Bayesian estimation procedures. These concepts are presented with tables, diagrams, and pictures for the ease of understanding of the readers. Further, the applications of these concepts are explained with various real-life examples. We also use the R-software tools to present the Bayesian solutions of these examples.