My research develops theoretical foundations for reliable, structure-aware reinforcement learning and translates them into provably sample-efficient and scalable algorithms for large-scale, safety-critical cyber-physical systems. I study how imperfect but informative structure, including partial or misspecified models, decomposable dynamics, optimization geometry, and physical laws, can guide learning and decision-making under requirements on safety, feasibility, and stability. A central goal is to characterize how such structure reduces the data needed for learning and to establish rigorous guarantees on sample complexity, near-optimality, and constraint satisfaction, even when models are inaccurate and operational failures are costly. I apply this framework to scheduling and control in energy systems and AI computing infrastructure, where power supply, storage, flexible demand, and computing workloads are increasingly coupled.