Over the last decade, the long-running endeavour to automate high-level processes in machine learning (ML) has risen to mainstream prominence. Beyond this, an even loftier goal is the pursuit of autonomy, which describes the capability of the system to independently adjust an ML solution over a lifetime of changing contexts. This monograph provides an expansive perspective on what constitutes an automated/autonomous ML system. In doing so, the authors survey developments in hyperparameter optimisation, multicomponent models, neural architecture search, automated feature engineering, meta-learning, multi-level ensembling, dynamic adaptation, multi-objective evaluation, resource constraints, flexible user involvement, and the principles of generalisation. Furthermore, they develop a conceptual framework throughout to illustrate one possible way of fusing high-level mechanisms into an autonomous ML system. This monograph lays the groundwork for students and researchers to understand the factors limiting architectural integration, without which the field of automated ML risks stifling both its technical advantages and general uptake.