What is the essence of quantum theory? In the present talk I want to approach this question from a particular operationalist perspective. I take advantage of a recent convergence between operational approaches to quantum theory and axiomatic approaches to quantum field theory. Removing anything special to particular physical models, including underlying notions of space and (crucially) time, what remains is what I shall call "abstract quantum theory". This embeds into a more general framework that also includes classical theory, with classical and quantum theory representing two extremes in a spectrum of possible theories. I shall present this within a "hierarchy of abstraction", where adding structure leads to more specialized frameworks. In particular, adding topological spacetime and locality leads to the recently proposed positive formalism and recovers quantum field theory. Further rigidifying time and adding causality recovers the standard formulation of quantum theory. There are other promising specializations such as Hardy's proposal to work in op-space.