During mission planning and execution, spacecraft operators must balance data collection and downlink, systems constraints, human factors, and navigation. As missions become increasingly complex and ambitious, these factors become more intricately entwined and conflicted. Deep Reinforcement Learning (DRL) is a State-of-the-Art (SOA) technique for autonomous spacecraft operations planning and autonomy. The goal of this project is to develop a simple environmental characterization training environment in the Planar-Restricted 2-Body Problem (PR2BP), establish benchmarks and heuristic baselines, and design a previously unstudied Markov Decision Process (MDP) formulation that will enable the spacecraft DRL agents to appropriately balance navigation and actuation capabilities.