This book will show how it is possible to explore the Universe not only with telescopes but also with Mathematics. Moreover, this book aims at explaining the equations governing the behavior of the stars, planets, and galaxies, which are very accessible to the average undergraduate who has studied some Mathematics in school. We will also probe into unseen black holes to determine their mass as well as investigate other intriguing celestial phenomena such as dark matter and dark energy, the expansion of the Universe, and explore via Einstein's Theory of Relativity, the very foundations of space and time itself. You will discover that both Special and General Relativity are absolutely essential for the functioning of the Global Positioning System (GPS), which illustrates how astrophysical ideas can impact everyday lives. Lastly, this book provides insight into how Mathematics allows us to see and hear how the Universe ticks. Each chapter is a front row seat for those who wish to listen to some of the most beautiful music expressed by the Universe, Mathematics.