In this talk we will discuss some of the foundations of computational mathematics. In particular, I will ask the rather fundamental questions: "Can everything be computed?" and: "In what way?" and discuss possible ways to answer them. This is done by linking some new developments in computational mathematics (in particular the Solvability Complexity Index) to some of the fundamental works of Godel, McMullen and Smale.