Agricultural productivity depends primarily on energy, water, and land resources, which are increasingly becoming more scarce and expensive. Electricity generation with photovoltaic (PV) solar energy technology requires significant amounts of space, especially in densely populated countries, generating a societal debate about allocating land (that may have alternative uses) for deploying PV systems. Rather than dedicating vast amounts of agricultural land to be used as solar farms, PV systems are deployed in agricultural lands so that a given piece of land can be used for agriculture and energy generation (the so-called agrivoltaics). A framework based on systems thinking is essential for the design and operation of smart, sustainable agrivoltaics systems to meet the design goals or to satisfy the expectation of all stakeholders (farmers, energy developers, policymakers, and local community members that may be impacted). In this paper, we propose a system-based conceptual and design framework for smart, sustainable agrivoltaics. We also discuss the benefits of agrivoltaics and the challenges to their adoption.