Sim-to-Real Reinforcement Learning for Path-Safe Target Reaching in Robotic Harvesting of Hydroponic Lettuce

[orcid=0000-0003-2168-2840]

[orcid=0000-0002-5840-2008]

[orcid=0000-0002-8761-5387]

[orcid=0000-0001-7861-7789]

[orcid=0000-0001-6202-8680]

Abstract

Robotic harvesting requires precise manipulation to ensure accurate positioning of the end-effector, especially in a constraint workspace. This study introduces the Path-Safe Approach Proximal Policy Optimization (PSA-PPO), a target-conditioned local reaching framework designed for robotic harvesting of nutrient film technique (NFT)-hydroponic lettuce in greenhouses. The proposed PSA-PPO preserves the standard clipped PPO objective while introducing a harvesting-oriented policy formulation that integrates target-conditioned observations, corridor-aware spatial constraints, trajectory-history-based safety encoding, and task-specific success criteria for constrained robotic manipulation. PSA-PPO was trained entirely in Isaac Lab and successfully transferred to a physical UR10e robotic arm without any real-world policy fine-tuning. In a controlled physical benchmark against 150 scheduled targets, PSA-PPO achieved the highest reaching rate of 94.7%, outperforming Cartesian+TOTG (86.0%), RRTConnect (88.0%), and RRT* (80.0%), while reducing median RMS jerk by 39.8–73.0% relative to the classical planners during the final approach segment. A formulation ablation evaluated over 200 simulation targets demonstrates that path-safety and side-corridor shaping are the principal drivers of harvesting-compatible performance: removing either term substantially degrades strict success even when terminal endpoint accuracy is preserved or improved, confirming that optimization toward the endpoint alone is an insufficient objective for constrained crop-proximal manipulation. Finally, PSA-PPO was integrated with an RGB-D vision module and an end-effector and evaluated as a complete greenhouse harvesting system across 144 lettuce harvesting attempts, achieving 94.4% vision-guided target reaching and 86.1% complete harvesting success. These results position PSA-PPO as an effective simulation-trained harvesting motion controller that balances target-reaching reliability and low dynamic excitation.

††††††††

1 Introduction

Reliable manipulator motion remains a critical challenge in robotic harvesting because crop localization, target approach, and physical crop interaction must be executed within highly constrained and variable agricultural environments [28, 21]. Unlike structured industrial manipulation, crop targets vary continuously in geometry, accessibility, and are surrounded by deformable plant material and cultivation infrastructure [14, 41, 10]. These constraints become especially important in controlled-environment agriculture, where dense crop layouts restrict manipulator workspace while increasing the requirement for repeatable autonomous operation [37]. Hydroponic lettuce presents a unique challenge in terms of manipulation. The harvesting tool must approach the stem–root region, just above the cultivation infrastructure, establish the right cutting angle, sever the plant, retain the detached lettuce, and transfer it without disturbing neighboring crops or rigid cultivation equipment [39]. Early research on lettuce harvesting demonstrated the effectiveness of machine-vision-guided manipulation within structured production systems [7]. More recent studies have continued to improve crop-specific harvesting mechanisms and the interactions of end effectors, highlighting the importance of combining crop localization with subsequent robotic actions [40, 3]. For instance, the field-tested Vegebot integrated learned perception, force-feedback control, and a specialized end effector designed for iceberg lettuce harvesting [5]. However, these studies do not address how to convert a vision-derived cutting point for lettuce into a repeatable motion that complies with the geometry and clearance constraints of a complex agricultural environment.

Manipulator motion in agricultural harvesting is still predominantly generated using inverse kinematics, sampling-based motion planning, trajectory optimization, or visual servoing. Researchers have adapted these methods to crop-specific manipulation requirements. For example, Ma et al. [22] developed an improved BIT*-based planner [16] that jointly considered path quality, execution efficiency, and dynamic impact. Rong et al. [29] coordinated manipulator and lifting-joint motion to improve tomato-harvesting reachability, while Kang et al. [15] proposed a BIT*-based Time-Optimal and Path-Optimal (TOPO) planner that combines improved sampling and edge checking with path optimization to reduce planning time and generate shorter trajectories for selective broccoli harvesting. Recent studies have similarly modified Informed-RRT* [16] for smooth winter-jujube harvesting [11] and developed a sampling-based planner for high-quality motion generation in unstructured tea canopies [12]. At the system level, visual-servoing approaches [6] remain common for cucumber harvesting [25, 13]. Classical planners [34, 23, 18] therefore provides strong geometric trajectory and can achieve high terminal precision, but agricultural manipulation requires simultaneous consideration of reachability, obstacle exposure, trajectory smoothness, perception uncertainty, execution robustness, and crop-specific approach geometry [9, 41]. Additionally, these planners are vulnerable to singularities, sensitive to noise which often produce unstable motion and inefficient trajectories [19, 27].

Learning-based policy control offers a complementary approach for achieving smooth motion through learning loops between the robot agent and its environment. Proximal Policy Optimization (PPO) [31] has become a widely used method in Deep Reinforcement Learning (DRL) for continuous robotic control [35]. It uses a policy-gradient method that constrains policy updates through a clipped surrogate objective. However, this mechanism does not explicitly enforce geometric or trajectory-safety constraints; constraint-aware policy optimization remains a distinct problem [1]. For harvesting, a policy rewarded primarily for terminal target acquisition may reach the cutting point without preserving an acceptable approach direction, tool orientation, or intermediate clearance. Addressing this limitation requires a task formulation that represents the relevant spatial relationships, evaluates trajectory-level violations, and distinguishes terminal reaching from harvesting-compatible motion rather than relying on the standard PPO objective alone. A further significant challenge in DRL training is acquiring the necessary interaction experience of robot through trial and error. Policy optimization involves repeated robot motions and resets, substantial execution time, and potentially unsafe exploratory actions. These challenges are particularly critical in agricultural environments, where exploratory actions can damage delicate biological materials, harvesting infrastructure, or the manipulator itself [8, 35]. These limitations highlight the need to develop manipulation DRL policies without relying on exploratory policy optimization in a physical robot.

Simulation-based policy learning offers a fundamentally different development path. Instead of gathering the extensive interaction data required for policy optimization on a physical harvesting robot, both exploration and policy learning can be carried out in a virtual physics simulator, allowing the learned policy to be evaluated on hardware later. GPU-native simulation has made this approach increasingly feasible. For instance, NVIDIA Isaac Lab [24], a unified GPU-accelerated framework for reinforcement learning, showcases massively parallel robot-policy interactions directly on the GPU.

However, simulation-trained policies must overcome the reality gap yielded by imperfect dynamics, actuator behavior, sensing, and calibration variation. Previous research has addressed these discrepancies through visual and dynamics randomization [36, 26] and physical parameter identification [32]. For example, Subedi et al. [33] demonstrated simulation-only RL transfer for occlusion-aware plant manipulation by separating learned behavior from lower-level compliant control. Kim et al. [17] used visuomotor imitation learning for autonomous pepper harvesting in unstructured fields, while Rong et al. [30] combined position-based visual alignment with a corrective action policy for greenhouse tomato harvesting. However, there remains limited evidence for policies that simultaneously incorporate crop-specific approach geometry, trajectory-history clearance, simulation-only policy optimization, controlled physical comparison with multiple classical planners, and end-to-end harvesting in complex agricultural setups. Moreover, success in simulation alone does not guarantee physical reliability; evaluations of paired robots and policies have highlighted the importance of addressing control and observation discrepancies [20]. For effective crop-proximal harvesting, it is essential to preserve the learned control interface while also accommodating the constraints of physical calibration and execution. This research gap is crucial to greenhouse-grown lettuce because successful reaching cannot be defined solely by the final tool center point (TCP) error. The cutter must approach the stem region from a suitable horizontal direction, remain compatible with the target-height geometry, preserve the required tool orientation, limit wrist–TCP descent near the cultivation structure, and avoid undesirably abrupt local motion. Hence, this control problem is consequently one of harvesting-compatible local constrained reaching, rather than generic point-to-point motion alone.

To address these gaps, this study develops Path-Safe Approach PPO (PSA-PPO), a target-conditioned local reaching formulation for hydroponic lettuce harvesting. Rather than modifying PPO’s optimization rule, PSA-PPO reformulates the learning task through geometry-conditioned safe initialization, target- and clearance-aware observations, coupled reaching and motion-safety rewards, and trajectory-history-dependent strict capture and termination criteria for collision-avoidance harvesting-specific trajectories. The policy is trained entirely in Isaac Lab and transferred without real-world policy fine-tuning, using calibrated target transformation, a frozen Cartesian deployment correction, and an independent ROS execution and safety interface. The study then examines whether harvesting-compatible behavior learned in simulation is retained on a physical UR10e, how it compares with classical motion planners, and how it performs within a complete vision-guided greenhouse harvesting system.

The main contributions of this study are:

1. A target-conditioned path-safe local reaching formulation, named PSA-PPO, is developed for lettuce harvesting. It incorporates spatial constraints, trajectory-level safety awareness, and harvesting-specific task objectives.
2. The PSA-PPO is trained entirely in simulation, without any physical rollouts used for policy optimization or fine-tuning. The trained policy is evaluated against the identical 200 targets within the simulation, highlighting contributions such as path-safety shaping, side-corridor shaping, and motion regularization.
3. The simulation-trained policy is transferred to a physical UR10e robotic arm and is benchmarked against Cartesian+TOTG, RRTConnect, and RRT* algorithms over 150 common targets. This comparison assesses task reliability, terminal precision, target convergence, physical motion dynamics, joint tracking, and trajectory geometry, revealing the trade-offs involved in the controller.
4. PSA-PPO is integrated with an RGB-D-based lettuce vision module for target streaming and a end-effector mechanism in a complete greenhouse harvesting pipeline. Across 144 actual lettuce harvesting attempts, the system achieved 94.4% target reaching success.

2 Methodology

The proposed PSA-PPO motion controller framework integrates with a previously developed lettuce cutting-point localization perception module and cut-and-hold mechanism [4, 3], creating a complete pipeline. The policy is entirely trained in a simulation environment and then transferred to the physical robot without any fine-tuning required in the real world. Figure 1 illustrates the resulting perception-learning-deployment pipeline.

2.1 3D Target Streaming

The perception interface extended our previously developed lettuce segmentation and occluded cutting-point localization method [4] to RGB-D target streaming. The vision method provided the image-space cutting-point pixel (u_{c},v_{c}). Using depth aligned to the RGB frame, the corresponding depth Z_{c} and RGB camera intrinsics (f_{x},f_{y},c_{x},c_{y}) were used to back-project the cutting point into the camera coordinate frame. A fixed camera-frame Y-axis offset d_{y} was then applied to provide clearance from the NFT channel:

\mathbf{p}_{cut}^{cam}=\begin{bmatrix}\dfrac{(u_{c}-c_{x})Z_{c}}{f_{x}}\\[5.0pt]
\dfrac{(v_{c}-c_{y})Z_{c}}{f_{y}}-d_{y}\\[5.0pt]
Z_{c}\end{bmatrix},\qquad d_{y}=5~\mathrm{mm}

To overcome frame-to-frame variation caused by depth noise and segmentation fluctuations, cutting-point estimates were temporally associated and averaged before robot execution. Let \mathbf{s}_{t}^{(j)}=[u_{t}^{(j)},v_{t}^{(j)}]^{\top} denote the image-plane tracking center of segmented mask j at frame t, and let \hat{\mathbf{s}}_{t-1}^{(k)} denote the latest center associated with active track k. Each detection was matched to its nearest existing track:

k^{\ast}(j)=\arg\min_{k\in\mathcal{K}_{t-1}}\left\|\mathbf{s}_{t}^{(j)}-\hat{\mathbf{s}}_{t-1}^{(k)}\right\|_{2},

where the assignment was accepted only when the image-plane distance was below \tau_{p}=50 pixels; otherwise, a new track was initialized. Each active track maintained a fixed-length buffer of 3D cutting-point estimates, from which the streamed target was computed as:

\bar{\mathbf{p}}_{cut}^{(k)}=\frac{1}{N_{b}}\sum_{n=1}^{N_{b}}\mathbf{p}_{cut,n}^{(k)},\qquad N_{b}=15

The averaged point was published as a ROS PointStamped message in the camera optical frame. The buffer was cleared after publication, and tracks not observed in the current frame were removed to prevent stale targets from propagating to the harvesting controller.

2.2 Simulation-Trained Policy for Path-Safe Side-Approach Target Reaching

PSA-PPO was trained entirely in Isaac Lab [24] as a target-conditioned policy for the final local harvesting approach. Rather than learning the complete motion from a fixed global home configuration, the task was decomposed into deterministic target-conditioned pre-positioning followed by local reaching. Policy optimization was therefore restricted to the final approach region, where target closure, side-entry geometry, tool orientation, and wrist–TCP clearance are most relevant to harvesting. Figure 2 summarizes the training architecture.

Let \mathbf{p}^{T}=[p_{x}^{T},p_{y}^{T},p_{z}^{T}]^{\top} denote the target TCP position. The target-conditioned horizontal approach direction and same-height staging reference were defined as:

\mathbf{a}^{T}=\frac{[p_{x}^{T},\,p_{y}^{T},\,0]^{\top}}{\sqrt{(p_{x}^{T})^{2}+(p_{y}^{T})^{2}}},\qquad\mathbf{p}^{S}=\mathbf{p}^{T}-d_{a}\mathbf{a}^{T}

where d_{a} is the staging-to-target distance. Because \mathbf{a}^{T} has no vertical component, \mathbf{p}^{S} and \mathbf{p}^{T} have the same height.Each target was sampled before initialization, and its radius, azimuth, and height were used to deterministically select a geometry-conditioned safe-start joint configuration. The robot was therefore initialized near the corresponding local approach region and started the side approach corridor motion towards the target.

The reaching task was implemented as a finite-horizon reinforcement-learning process. The environment maintained trajectory-level variables required for reward and termination evaluation, whereas the actor received the following 42-dimensional observation at policy step t:

\begin{split}\boldsymbol{\xi}_{t}=&\left[\widetilde{\mathbf{q}}_{t},\;\widetilde{\dot{\mathbf{q}}}_{t},\;\mathbf{p}^{T},\;\mathbf{p}^{S},\;\mathbf{e}^{T}_{t},\;\mathbf{e}^{S}_{t},\;\mathbf{u}_{t-1},\;\right.\\
&\quad\quad\quad\quad\quad\quad\left.\ell_{t},\;g_{t},\;c_{t}^{a},\;c_{t}^{z},\;\mathbf{a}^{T},\;\mathbf{h}_{t}\right]\end{split}

where \widetilde{\mathbf{q}}_{t} and \widetilde{\dot{\mathbf{q}}}_{t} are the scaled joint positions and velocities, \mathbf{e}^{T}_{t}=\mathbf{p}^{T}-\mathbf{p}^{tcp}_{t} and \mathbf{e}^{S}_{t}=\mathbf{p}^{S}-\mathbf{p}^{tcp}_{t}, and \mathbf{u}_{t-1} is the previous action. The normalized corridor progress was \ell_{t}=((\mathbf{p}^{tcp}_{t}-\mathbf{p}^{S})^{\top}\mathbf{a}^{T})/d_{a}, while g_{t} indicated activation of the staging-to-target corridor. Tool orientation was represented by c_{t}^{a}=\mathbf{f}_{t}^{\top}\mathbf{a}^{T} and c_{t}^{z}=|\mathbf{f}_{t}^{\top}\mathbf{e}_{z}|, where \mathbf{f}_{t} is the tool forward axis and \mathbf{e}_{z}=[0,0,1]^{\top}. Thus, the desired side-approach orientation corresponds to c_{t}^{a}\rightarrow 1 and c_{t}^{z}\rightarrow 0. The clearance vector \mathbf{h}_{t}\in\mathbb{R}^{5} contains the three wrist-link clearances, TCP clearance, and their minimum value relative to the target-height safety plane. The resulting observation contained 42 elements.

The stochastic actor produced the normalized raw action \bar{\mathbf{u}}_{t}\in\mathbb{R}^{6}, which was clipped and temporally filtered before being converted to an incremental joint-position command:

\begin{split}\mathbf{u}_{t}=&(1-\alpha)\mathbf{u}_{t-1}+\alpha\,\operatorname{clip}\left(\bar{\mathbf{u}}_{t},-1,1\right),\\
\mathbf{q}^{cmd}_{t}&=\operatorname{clip}_{\mathbf{q}^{min},\mathbf{q}^{max}}\left(\mathbf{q}_{t}+s_{u}\mathbf{u}_{t}\right)\end{split}

where \alpha is the temporal-filter coefficient, s_{u} is the action scale, and \mathbf{q}^{min} and \mathbf{q}^{max} are the UR10e soft joint limits.

To separate target closure from corridor deviation, let \mathbf{a}^{\perp}=[-a_{y}^{T},a_{x}^{T},0]^{\top}. The target-frame errors were-

\begin{split}&e_{t}^{ax}=(\mathbf{p}^{T}-\mathbf{p}^{tcp}_{t})^{\top}\mathbf{a}^{T},\qquad e_{t}^{lat}=(\mathbf{p}^{T}-\mathbf{p}^{tcp}_{t})^{\top}\mathbf{a}^{\perp},\\
&e_{t}^{z}=p_{z}^{T}-p_{z,t}^{tcp}\end{split}

Path safety was evaluated from the three wrist links and TCP, denoted collectively by \mathcal{C}. Their minimum clearance, instantaneous below-plane violation, and maximum accumulated violation were-

\begin{split}&h_{t}^{min}=\min_{i\in\mathcal{C}}\left(z_{t}^{(i)}-p_{z}^{T}\right),\qquad b_{t}=\max(0,-h_{t}^{min}),\\
&b_{t}^{max}=\max_{0\leq\tau\leq t}b_{\tau}\end{split}

Thus, a trajectory that temporarily passed below the target-height plane could not be classified as path-safe solely from its final configuration.

The instantaneous reward combined target progress, side-approach corridor tracking, orientation, path safety, motion regularization, and terminal capture:

\mathcal{R}_{t}=\mathcal{R}^{tar}_{t}+\mathcal{R}^{cor}_{t}+\mathcal{R}^{ori}_{t}+\mathcal{R}^{safe}_{t}+\mathcal{R}^{reg}_{t}+\mathcal{B}_{t}

The target component rewarded TCP-to-target closure, axial progress, forward corridor progression, and entry into increasingly tight target neighborhoods. The corridor component penalized lateral and vertical departure, axial overshoot, and loitering near the staging region without continued target closure. The orientation component promoted alignment of the tool axis with \mathbf{a}^{T} while penalizing vertical tilt. The safety component penalized instantaneous and accumulated target-height-plane violation, and the regularization component penalized action magnitude, action-rate variation, action acceleration, and joint velocity.

A valid capture required simultaneous positional, orientation, and instantaneous-clearance conditions, whereas strict capture additionally required the complete learned path and initial safe-start configuration to satisfy the path-safety tolerance:

\begin{split}&\chi_{t}=\mathbb{I}\Big[|e_{t}^{ax}|<\varepsilon_{ax}\land|e_{t}^{lat}|<\varepsilon_{lat}\land|e_{t}^{z}|<\varepsilon_{z}\\
&\quad\quad\land\;d_{t}^{T}<\varepsilon_{d}\land c_{t}^{a}>\varepsilon_{a}\land c_{t}^{z}<\varepsilon_{v}\land h_{t}^{min}>0\Big],\\[1.0pt]
&\chi_{t}^{strict}=\chi_{t}\,\mathbb{I}\left[b_{0}\leq\varepsilon_{safe}\land b_{t}^{max}\leq\varepsilon_{safe}\right]\end{split}

where d_{t}^{T}=\|\mathbf{p}^{T}-\mathbf{p}^{tcp}_{t}\|_{2} and b_{0} is the below-plane violation at initialization. The capture reward was \mathcal{B}_{t}=c_{b1}\chi_{t}+c_{b2}\chi_{t}^{strict}, with the larger contribution assigned to path-safe capture. Successful termination required \chi_{t}^{strict}=1 for N_{s} consecutive policy steps, whereas b_{t}^{max}>\varepsilon_{hard} terminated the episode early.

The actor and critic were trained using PPO with the RSL-RL backend in Isaac Lab. Let \pi_{\theta}(\bar{\mathbf{u}}_{t}|\boldsymbol{\xi}_{t}) and V_{\varphi}(\boldsymbol{\xi}_{t}) denote the actor and critic, respectively. Generalized advantage estimation provided \widehat{A}_{t} and \widehat{G}_{t}, and the actor was updated using:

\begin{split}\mathcal{J}_{\mathrm{PPO}}(\theta)=&\mathbb{E}_{t}\left[\min\left(\rho_{t}(\theta)\widehat{A}_{t},\right.\right.\\
&\quad\quad\quad\left.\left.\operatorname{clip}\left(\rho_{t}(\theta),1-\epsilon,1+\epsilon\right)\widehat{A}_{t}\right)\right]\end{split}

where \rho_{t}(\theta)=\frac{\pi_{\theta}(\bar{\mathbf{u}}_{t}|\boldsymbol{\xi}_{t})}{\pi_{\theta_{\mathrm{old}}}(\bar{\mathbf{u}}_{t}|\boldsymbol{\xi}_{t})}. The critic was optimized by value-function regression, with entropy regularization retained during training.

Training targets were sampled over the practical front workspace using horizontal polar coordinates and area-uniform radial sampling. A hard-region mixture increased exposure to far-radius, negative-lateral, low- and high-height, and near-boundary azimuth configurations. Candidate checkpoints were subsequently evaluated on an independent random-workspace simulation set; however crop cutting and plant interaction were not used for policy optimization. Workspace bounds, safe-start settings, component definitions, activation gates, reward coefficients, capture and safety thresholds, network parameters and the terms removed in each formulation ablation are reported in Appendix A and B.

2.3 End-Effector Integration

The harvesting tool adapted our previously developed lettuce cut-and-hold end effector [3] for direct actuation by an OnRobot 2FG7 gripper. The redesigned components were modeled in SolidWorks 2026 (Dassault Systèmes, Vélizy-Villacoublay, France) and 3D-printed using fused deposition modeling (FDM) with polylactic acid (PLA) filament (Figure 3). The original 2FG7 fingers were replaced by custom brackets carrying paired holding fingers and shear blades while retaining the first-class lever cutting principle. A single gripper closing action therefore actuated the shear mechanism and simultaneously retained the detached lettuce for transfer.

2.4 ROS-Based Deployment and Harvest Sequencing

The selected PSA-PPO actor was integrated with the ROS harvesting architecture shown in Figure 4. Simulation validation used randomly generated workspace targets, whereas physical harvesting received smoothed cutting points from the RGB-D perception module (Sec. 2.1); the target-conditioned geometry and learned local-reaching formulation were otherwise retained during deployment.

At the beginning of a harvesting sequence, the UR10e remained at its home configuration while available lettuce targets were localized. Each camera-frame target \mathbf{p}_{cut}^{cam} was transformed to the robot frame and corrected for the cut-and-hold tool geometry:

\mathbf{p}^{T}=\mathcal{T}_{cr}\left(\mathbf{p}_{cut}^{cam}\right)+\Delta\mathbf{p}_{d}

where \mathcal{T}_{cr}(\cdot) denotes the calibrated rigid camera-to-robot transformation and \Delta\mathbf{p}_{d}=[\Delta x_{d},\Delta y_{d},\Delta z_{d}]^{\top} denotes the fixed deployment correction accounting for residual perception, calibration, and end-effector geometry. The correction was determined before the reported evaluation and was subsequently held fixed.

No external sequence of intermediate side-approach waypoints was generated during deployment. Instead, \mathbf{p}^{T} defined the same target-conditioned approach geometry used during training. The arm first moved to the corresponding deterministic safe-start configuration, after which PSA-PPO controlled the final local side approach; \mathbf{p}^{S} therefore remained an internal geometric reference rather than a separately commanded physical waypoint.

During inference, the actor was evaluated deterministically:

\bar{\mathbf{u}}_{t}=\mu_{\theta}\left(\boldsymbol{\xi}_{t}\right),\qquad\mathbf{q}^{ROS}_{cmd,t}=f_{map}\left(\mathbf{q}^{cmd}_{t}\right)

where \mu_{\theta}(\cdot) is the deterministic actor output, \mathbf{q}^{cmd}_{t} follows Eq. (6), and f_{map}(\cdot) maps the simulation joint ordering to the physical UR10e controller interface.

The ROS interface updated the policy state from UR10e joint feedback and published single-point JointTrajectory commands to the scaled joint-trajectory controller. For each command, feasible per-joint durations were computed from the current state, commanded displacement, and prescribed velocity and acceleration limits, and their maximum was used as the synchronized trajectory duration. Acceleration was checked before command publication and from measured joint-velocity feedback; exceeding the corresponding safety threshold replaced the pending motion with a hold command and entered a halted state. These execution safeguards operated outside the learned task objective.

After the target-reaching condition was satisfied, the harvest controller closed the OnRobot 2FG7 gripper to actuate the cut-and-hold mechanism, transferred the harvested lettuce to the collection basket, and reopened the gripper for release. When multiple targets were available, the remaining target with the smallest Euclidean distance from the current TCP position was selected; otherwise, the robot returned home.

3 Experiment

3.1 Experimental Setup

PSA-PPO training and simulation evaluation were conducted using NVIDIA Isaac Lab in Isaac Sim 5.1.0 with the RSL-RL backend. Training used GPU-parallel simulation on an Ubuntu 22.04 LTS workstation equipped with dual Intel Xeon Gold processors, 256 GB RAM, and an NVIDIA RTX A6000 GPU with 48 GB VRAM. Candidate policies were trained without physical robot interaction, and the selected actor was evaluated deterministically on a fixed set of randomly generated targets spanning the prescribed front workspace. The component-removal ablations were evaluated on the same targets, enabling paired target-wise comparison with the complete PSA-PPO formulation.

Physical experiments used an RB-VOGUI mobile platform (Robotnik Automation, Paterna, Spain) equipped with a UR10e manipulator (Universal Robots, Odense, Denmark), an OnRobot 2FG7 gripper (OnRobot, Odense, Denmark), and an Intel® RealSense™ D435i RGB-D camera mounted near the tool center point. The adapted end effector was mounted directly to the 2FG7, while an onboard Intel i7-based computer executed perception, target transformation, harvesting sequencing, deterministic PSA-PPO inference, ROS 1 communication, and gripper control. The simulation-trained actor was transferred to this platform without any fine-tuning.

Two complementary physical experiments were conducted. First, a controlled motion benchmark compared PSA-PPO with Cartesian+TOTG, RRTConnect, and RRT* over the same scheduled target locations, isolating controller behavior from perception and crop-interaction effects. Second, the complete PSA-PPO system was evaluated on greenhouse lettuce harvesting trials at the Texas A&M AgriLife Research Center in Dallas, TX, USA, where cutting-point targets were generated online by the RGB-D perception module. However, calibration was performed to map the targets between the camera and robot frames. Thus, the greenhouse experiment evaluated the complete vision-to-harvest pipeline. Planner configurations and benchmark implementation details are reported in Appendix C.

3.2 Evaluation Protocol

Evaluation was organized at three levels: simulation, controlled physical motion, and greenhouse harvesting, with the formulation ablation embedded within the simulation evaluation. Table 1 summarizes the corresponding populations and primary outcomes.

For simulation target k, the terminal TCP error was:

d_{k}^{T}=\left\|\mathbf{p}^{T}_{k}-\mathbf{p}^{tcp}_{k,f}\right\|_{2}

Let I_{k}^{\mathrm{final}} indicate satisfaction of the final capture criterion over the prescribed validation hold interval, and let I_{k}^{\mathrm{safe}} indicate satisfaction of both the safe-start and learned-path tolerances,

I_{k}^{\mathrm{safe}}=\mathbb{I}\left[b_{0,k}\leq\varepsilon_{\mathrm{safe}}\;\land\;b_{k}^{\max}\leq\varepsilon_{\mathrm{safe}}\right]

Strict success required both conditions, such that I_{k}^{\mathrm{strict}}=I_{k}^{\mathrm{final}}I_{k}^{\mathrm{safe}}. The corresponding simulation rates were:

R_{r}=\frac{100}{N_{v}}\sum_{k=1}^{N_{v}}I_{k}^{r},\quad r\in\left\{\mathrm{final},\mathrm{safe},\mathrm{strict}\right\}

For each component-removal formulation a, the paired strict-success loss relative to the complete PSA-PPO formulation was defined over the identical target set as:

\Delta R_{\mathrm{strict}}^{(a)}=\frac{100}{N_{v}}\sum_{k=1}^{N_{v}}\left(I_{k,\mathrm{full}}^{\mathrm{strict}}-I_{k,a}^{\mathrm{strict}}\right)

Target-frame errors, tool orientation, maximum below-plane violation, trajectory length, episode duration, and motion quantities were retained for the behavioral analyses reported with the ablation results.

Let \mathcal{M}=\{\mathrm{PSA\mbox{-}PPO},\mathrm{Cartesian+TOTG},\mathrm{RRTConnect},\linebreak\mathrm{RRT^{*}}\} denote the set of controllers for the controlled physical benchmark. Physical reaching for target k and controller c was defined by-

I_{k,c}^{\mathrm{reach}}=\mathbb{I}\left[d_{k,c}^{T}\leq\varepsilon_{r}\right],\quad R_{\mathrm{reach}}^{(c)}=\frac{100}{N_{b}}\sum_{k=1}^{N_{b}}I_{k,c}^{\mathrm{reach}}

where \varepsilon_{r}=0.020 m was the common physical evaluation tolerance. Continuous controller comparisons were restricted to the mutually successful target set:

\mathcal{K}_{\cap}=\bigcap_{c\in\mathcal{M}}\left\{k:I_{k,c}^{\mathrm{reach}}=1\right\},\quad N_{\cap}=\left|\mathcal{K}_{\cap}\right|

TCP transforms and joint states were recorded at 100 Hz, uniformly resampled, and smoothed using a 0.15-s moving-average window before derivative-based analysis. With a_{i}=\|\mathbf{a}_{i}\|_{2} and j_{i}=\|\mathbf{j}_{i}\|_{2}, motion intensity was quantified by-

A_{\mathrm{rms}}=\sqrt{\frac{1}{N_{a}}\sum_{i=1}^{N_{a}}a_{i}^{2}},\quad J_{\mathrm{rms}}=\sqrt{\frac{1}{N_{j}}\sum_{i=1}^{N_{j}}j_{i}^{2}}

together with the 99.5th-percentile jerk J_{99.5}. The primary local-motion comparison used the final 0.22 m of target-relative approach. Executed trajectory length and geometric detour were defined as:

L_{\mathrm{path}}=\sum_{m=1}^{M-1}\left\|\mathbf{s}_{m+1}-\mathbf{s}_{m}\right\|_{2},\quad D_{\mathrm{ratio}}=\frac{L_{\mathrm{path}}}{\left\|\mathbf{s}_{M}-\mathbf{s}_{1}\right\|_{2}}

where D_{\mathrm{ratio}}\geq 1, with values closer to unity indicating more direct motion. Endpoint error, joint-position tracking RMSE, normalized target convergence, and additional trajectory-geometry measures were evaluated from the same paired trajectories. Detailed trajectory processing, planner settings, and benchmark-integrity procedures are provided in Appendix C.

For greenhouse attempt k, harvesting progression was evaluated sequentially through reaching, cutting, retention, and release. If N_{m} denotes the number of attempts completing stage m, the cumulative and conditional stage rates were:

\begin{split}&R_{m}=\frac{N_{m}}{N_{h}}\times 100\%,\quad R_{m\mid m-1}=\frac{N_{m}}{N_{m-1}}\times 100\%\\
&m\in\left\{\mathrm{reach},\mathrm{cut},\mathrm{hold},\mathrm{release}\right\}\end{split}

where the conditional rate was applied from the cutting stage onward. Because successful release required completion of all preceding stages, overall harvesting success was:

H_{\mathrm{success}}=R_{\mathrm{release}}=\frac{N_{\mathrm{complete}}}{N_{h}}\times 100\%

Failures were assigned to the first unsuccessful harvesting stage. Paired statistical tests, bootstrap confidence intervals, multiplicity correction, and sensitivity analyses used for the simulation ablation and physical benchmark are detailed in Appendices B and C.

3.3 Results

3.3.1 Simulation Validation and Formulation Ablation

Simulation validation and component effects. The selected PSA-PPO policy satisfied the final capture criterion for 198 of 200 targets (99.0%), the path-safety criterion for 185 trajectories (92.5%), and both conditions for 184 targets, yielding 92.0% strict success. The 7-percentage-point difference between final reaching and strict success was dominated by trajectory-level clearance: among the 16 strict failures, 14 were safety-only cases, one was a capture-only case, and one failed both criteria. Thus, terminal target acquisition alone did not fully characterize the harvesting-compatible behavior captured by the strict criterion.

The component-removal results in Table 2 show that path-safety shaping had the largest association with strict performance. Removing these terms reduced final reaching from 99.0% to 89.0%, path-safe success from 92.5% to 69.5%, and strict success from 92.0% to 63.5%. On the paired target set, PSA-PPO remained strictly successful for 61 targets on which the no-path-safety formulation failed, whereas the reverse occurred for only four targets. The resulting strict-success difference was 28.5 percentage points (95% CI: 21.5–35.5 percentage points; Holm-adjusted p=1.18\times 10^{-13}). The P95 maximum below-plane violation increased from 2.11 to 26.8 mm, and the largest observed violation increased from 9.68 to 54.3 mm.

Removing the side-corridor terms produced a different trade-off. This formulation reached all 200 targets and achieved a smaller median final TCP error than PSA-PPO (25.3 versus 32.3 mm), yet path-safe and strict success both decreased to 73.0%. The paired strict-success loss was 19.0 percentage points (95% CI: 13.5–25.0 percentage points; Holm-adjusted p=8.22\times 10^{-10}). The result therefore separates terminal precision from the compound harvesting criterion: improved endpoint accuracy did not imply improved path-safe reaching.

Removal of motion regularization produced the smallest task-level effect. Final reaching remained 98.0%, while path-safe and strict success decreased to 88.5% and 87.0%, respectively. The paired strict-success loss was 5.0 percentage points (95% CI: 1.5–8.5 percentage points; Holm-adjusted p=0.0129). However, the corresponding raw-action-rate, action-acceleration, and TCP-jerk diagnostics did not increase monotonically when regularization was removed. The evaluated checkpoint therefore does not support attributing lower jerk directly to the regularization terms; instead, their measurable task-level association was a modest increase in strict success together with a different trajectory-duration trade-off.

Figure 5 provides the corresponding population-level view. In addition to the strict-success degradation, the below plane empirical cumulative distribution function (ECDF) shows that removal of the path-safety formulation substantially increased the upper tail of trajectory-level violation.

Behavioral and workspace robustness. The target-frame profiles in Figure 6 show that the component removals altered different aspects of the local approach rather than uniformly scaling trajectory quality. The clearest separation occurred in the path-safety channel, where the no-path-safety formulation produced substantially larger below-plane excursions. In contrast, the lateral and vertical profiles did not exhibit a consistent ordering across all formulations. This behavior is consistent with the side-corridor terms acting within a coupled target-closure and constraint-shaping objective, rather than as an independent fixed-band controller. All formulations reduced TCP-to-target distance toward the end of the approach, but differed in whether that convergence was achieved while satisfying the complete strict criterion.

The workspace analysis in Figure 7 shows that PSA-PPO retained comparatively high strict success across the evaluated radius, azimuth, and height bins, whereas removal of the path-safety or side-corridor terms produced larger degradation in several regions of the workspace. The motion-regularization formulation remained closer to PSA-PPO throughout most bins, consistent with its smaller aggregate strict-success difference.

Taken together, the simulation results identify path-safety shaping as the largest task-level contributor among the tested removals and show that side-corridor shaping can improve strict harvesting-compatible performance even when terminal accuracy alone improves after its removal. Detailed paired outcomes, fixed-band corridor diagnostics, orientation and motion-regularization analyses, three-dimensional workspace maps, and representative trajectories are provided in Appendix B. Because one independently trained checkpoint was evaluated per formulation, these comparisons characterize the evaluated policies over the common target set and do not quantify retraining variability.

3.3.2 PSA-PPO vs. Classical Motion Planners on the Physical Robot

Task-level reliability, terminal precision, and failure behavior. PSA-PPO achieved the highest observed target-reaching rate, successfully reaching 142 of 150 targets (94.7%), compared with 132/150 (88.0%) for RRTConnect, 129/150 (86.0%) for Cartesian+TOTG, and 120/150 (80.0%) for RRT* (Table 3). Thus, the simulation-trained policy retained high task-level reliability after transfer to the physical UR10e.

On the common-success targets, the median final TCP error was 9.78 mm for PSA-PPO, compared with 0.06 mm for Cartesian+TOTG, 1.62 mm for RRTConnect, and 1.61 mm for RRT*. The endpoint ECDF in Figure 8 shows this separation across the distribution. The classical planners therefore retained a substantial advantage in terminal positioning precision, while the PSA-PPO median remained below one-half of the 20-mm task-level tolerance. However, while the planner-based approach struggled to reach all targets, PSA-PPO outperformed it by 94.7%, achieving success while maintaining all safety criteria.

Failure diagnostics further distinguish the controllers beyond their binary success rates. PSA-PPO unsuccessful runs were concentrated more strongly in target-reaching events, whereas the planner executions exhibited a broader mixture of planning-, execution-, return-, and controller-related diagnostic events (Figure 9). Because these event categories were not mutually exclusive, the figure is interpreted independently of the unique unsuccessful-target counts in Table 3.

Local motion dynamics and target convergence. Within the final 0.22-m target-relative approach, PSA-PPO produced median RMS acceleration and jerk of 0.127 m/s2 and 11.38 m/s3, respectively, compared with 0.528 and 42.17 for Cartesian+TOTG, 0.555 and 40.74 for RRTConnect, and 0.267 and 18.91 for RRT* (Table 4). Relative to the planner medians, the PSA-PPO reductions in RMS jerk were 73.0%, 72.1%, and 39.8%. The corresponding reductions in RMS acceleration were 75.9%, 77.1%, and 52.4%. High-end transients followed the same ordering: the 99.5th-percentile jerk was 77.72 m/s3 for PSA-PPO versus 131.88, 164.77, and 127.84 m/s3, respectively. Joint-position tracking RMSE was also lowest for PSA-PPO. All PSA-PPO–planner paired contrasts for the four local-motion metrics remained significant after Holm correction (adjusted p=4.02\times 10^{-16}).

The normalized motion profiles in Figure 10 show that the lower aggregate PSA-PPO acceleration and jerk were sustained over substantial portions of the local approach rather than arising solely from isolated endpoints. The perpendicular-deviation profiles, however, overlap more strongly across controllers. Lower dynamic excitation should therefore not be interpreted as evidence that PSA-PPO followed the most geometrically direct Cartesian path.

Target-distance contraction also differed across controllers. PSA-PPO reduced the remaining target distance to 50% of its local-window entry value after a median normalized progression of 0.195, compared with 0.405 for Cartesian+TOTG, 0.370 for RRTConnect, and 0.345 for RRT* (Figure 11). PSA-PPO therefore performed a larger fraction of its target closure earlier in its own normalized local approach. This quantity characterizes convergence shape and does not represent wall-clock execution speed.

Trajectory geometry and controller trade-off. Lower PSA-PPO dynamic excitation did not correspond to shorter or more direct paths. Over the complete reconstructed approach, the median detour ratio was 1.81 for PSA-PPO, compared with 1.67 for Cartesian+TOTG, 1.68 for RRTConnect, and 1.70 for RRT*. PSA-PPO also accumulated greater total joint travel than the planner-based methods. The population-level geometry therefore favored the classical planners in path directness, while the physical motion-quality results favored PSA-PPO in acceleration, jerk, and tracking error. Detailed trajectory-geometry statistics, reliability geometry trade-offs, and matched examples of atypically expansive planner trajectories are provided in Appendix C.

Overall, the physical benchmark reveals a controller trade-off rather than uniform dominance: the classical planners achieved substantially greater terminal precision and more direct average trajectories, whereas PSA-PPO achieved the highest observed reaching rate together with markedly lower local dynamic excitation and joint-tracking error.

3.3.3 Target-Reaching Performance in Harvesting

The end-effector of the integrated system successfully reached 136 out of 144 vision-derived lettuce cutting points, achieving a target success rate of 94.4% in the greenhouse (Table 5). Cutting was completed successfully for 124 plants, which accounts for 86.1% of all attempts after reaching the target. Additionally, we noted that 13 plants were not properly secured after cutting, and 3 plants fell during transfer to the basket.

However, the conditional rates reveal that the most harvesting failures occur after target acquisition. Cutting succeeded for 124 of the 136 reached plants, resulting in a success rate of 91.2%. Retention succeeded for 111 of 124 cut plants (89.5%), and basket release succeeded for 108 of 111 retained plants, achieving a success rate of 97.3%. Therefore, the principal loss in complete-system performance occurred during physical interaction between the crop and the end-effector, rather than motion policy control.

Figure 12 shows the online target stream generated by the vision module, while Figure 13 depicts a typical execution of the resulting physical sequence: PSA-PPO performed the vision-conditioned side approach, closure of the 2FG7-actuated cutting and retention, and the retained lettuce was subsequently transferred and released into the collection basket.

The greenhouse evaluation therefore shows that the simulation-trained PSA-PPO controller retained a high reaching rate when driven by online RGB-D targets within the complete harvesting system, while the principal remaining performance losses occurred downstream during cutting and retention.

3.4 Discussion

The simulation results indicate that the main value of the PSA-PPO formulation lies in how harvesting-compatible success is structured rather than in terminal reaching alone. Removing the path-safety terms produced the largest degradation in strict success and substantially increased the tail of below-plane violations, confirming that transient trajectory behavior can remain unacceptable even when the final target is reached. The side-corridor ablation provides an equally important counter-example: it improved terminal accuracy and reached every target, yet reduced strict success considerably. Thus, optimization toward the endpoint alone can produce a numerically better reaching result while weakening the combined harvesting constraint. The motion-regularization ablation was less conclusive; although its removal reduced strict success, several action- and TCP-derivative measures did not worsen. The present evidence therefore supports path-safety and corridor shaping as important elements of the coupled task formulation, but does not justify attributing lower jerk directly to the regularization term. This task-structured use of simulation-trained control is related to recent plant-manipulation work by Subedi et al. [33], who separated structured kinematic motion from a policy learned entirely in simulation; however, their objective was occlusion removal rather than harvesting-compatible crop-proximal reaching. Moreover, because one independently trained checkpoint was evaluated per formulation, the ablation establishes differences over the common target set rather than retraining robustness.

The physical benchmark provides stronger evidence for transfer, while also revealing a clear learned-versus-planned trade-off. PSA-PPO achieved the highest reaching reliability and substantially lower local acceleration, jerk, and joint-tracking error, but the classical planners retained markedly better endpoint precision and more direct trajectories. These results are therefore complementary rather than hierarchical: PSA-PPO occupied a favorable reliability–dynamic-excitation regime, whereas the planners remained stronger in geometric precision and path efficiency. Figure 14 visualizes this trade-off directly. Recent agricultural planning work illustrates the value of that geometric objective; for example, Kang et al. [15] explicitly optimized planning time, path length, and trajectory smoothness for selective broccoli harvesting. The present results therefore do not suggest replacing geometric planning where path optimality is primary, but rather identify a different operating regime for learned crop-proximal motion. This distinction is especially relevant near crops and rigid cultivation infrastructure, where smooth execution is desirable, but lower jerk is not itself evidence of collision safety. Similarly, occasional expansive planner trajectories should not override the population-level finding that their paths were more direct on average. The measured physical dynamics must also be interpreted as properties of the deployed controller stack, because PSA-PPO inference was combined with action filtering and an independent hardware safety wrapper rather than executed as an isolated learned policy.

The jerk reduction achieved by PSA-PPO relative to the classical planners has two separable contributors that should be distinguished. First, the deployment stack applies a temporal action filter (\alpha=0.18) and an independent hardware acceleration safeguard (0.0045 rad/s^{2}) that attenuate high-frequency motion regardless of policy behavior; these are stack-level properties rather than learned ones, and their individual contribution to the observed jerk advantage cannot be isolated without a dedicated ablation. Second, Figure 10 shows that lower acceleration and jerk are sustained over the full local approach rather than at isolated endpoints, consistent with the policy having learned intrinsically smoother convergence as an emergent property of the coupled task formulation. The motion-regularization ablation does not support attributing this smoothness directly to the regularization reward terms, since their removal did not produce a monotonic increase in raw-action diagnostics (Figure 5(b)). Fully disaggregating these two contributors is left for future work.

The greenhouse experiment shifts the interpretation from controller behavior to system bottlenecks. Reaching remained high when predefined Cartesian targets were replaced by online RGB-D cutting points, suggesting that the sim-to-real policy did not undergo a large degradation after integration with perception, coordinate transformation, and greenhouse variability. The controlled and greenhouse populations are different; however, their similar reaching rates should not be treated as evidence of statistical equivalence. More importantly, incomplete harvests occurred after successful target reaching. The dominant residual losses therefore moved downstream to tool and crop interaction rather than target acquisition or basket release. This result both supports the usefulness of the learned motion controller and limits its system-level significance: improving reaching alone is unlikely to raise complete harvesting success substantially unless crop interaction is improved concurrently. Recent compliant-control work points to the same systems issue from a different direction; Wang et al. [38] used adaptive variable impedance control and force sensing to regulate physical crop interaction during dragon-fruit picking. For the present system, this suggests that contact-aware control and retention mechanics may now offer greater system-level benefit than further reduction of terminal reaching error alone. Similar stage-dependent losses have also been observed in field-tested harvesting systems [2].

Several limitations consequently define the scope of the present findings. The term “path-safe” refers to the implementation with side-corridor entry, collision avoidance with wrist–TCP target-height clearance criteria, and does not provide formal uncertainty-awareness guarantees for the complete robot or greenhouse scene. The physical study used one manipulator, one greenhouse environment, a frozen policy deployment correction, and a single trained policy for each formulation. Consequently, the reported results characterize the evaluated policies but do not quantify variability across independent training runs. Broader claims therefore require repeated training seeds, additional robot and crop configurations, and evaluation under changing canopy and infrastructure geometry. In addition, the planner baselines were evaluated only in the controlled reaching benchmark and not as complete vision-to-harvest systems, while the perception and cutting modules were inherited from earlier work rather than independently re-evaluated here. Future research will investigate hybrid planning and control architectures that combine global scene-aware planning, learned crop-proximal control, and contact-aware crop interaction. Such a hybrid design could preserve the planners’ geometric strengths while exploiting the reliability and lower local dynamic excitation observed for PSA-PPO.

4 Conclusion

This study formulated robotic lettuce harvesting as a target-conditioned, path-safe, side-corridor approach problem and developed PSA-PPO for simulation training and physical deployment on a UR10e robot. PSA-PPO demonstrated that such a harvesting-specific policy can be optimized entirely in simulation and transferred to a physical robot without real-world policy fine-tuning. The formulation ablation further showed that terminal accuracy alone is an incomplete measure of harvesting-compatible motion: path-safety and side-approach constraints materially affected strict task performance even when endpoint reaching remained high. On the physical robot, PSA-PPO achieved the highest reaching reliability (94.7%) and lower local dynamic excitation, while the classical planners retained advantages in terminal precision but remained unreliable in constrained conditions. The resulting contribution is therefore not universal controller superiority, but a distinct and practically useful operating regime for crop-proximal manipulation. Future work will integrate global geometric planning, learned crop-proximal control, explicit clearance modeling, and contact-aware feedback, while expanding validation across crop conditions and greenhouse environments.

Code Availability

The source code and supporting materials will be made available upon publication at https://github.com/bashir-tamu/PSA-PPO-Lettuce-Harvesting.

Acknowledgments

This research is partially supported by the United States Department of Agriculture (USDA)’s National Institute of Food and Agriculture (NIFA) Research Capacity Fund Hatch Program: TEX09954 (Accession No. 7002248) and Research Capacity Fund Multistate Hatch Program: TEX0-1-9916 (Accession No. 7008389). This publication is also supported by the Texas A&M University Division of Research Targeted Proposal Teams (TPT) funding program and Texas A&M AgriLife Research. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the authors and should not be construed to represent any official USDA or U.S. Government determination or policy.

References

1. [1] J. Achiam, D. Held, A. Tamar, and P. Abbeel (2017) Constrained policy optimization. In Proceedings of the 34th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 70, pp. 22–31.
2. [2] B. Arad, J. Balendonck, R. Barth, O. Ben-Shahar, Y. Edan, T. Hellström, J. Hemming, P. Kurtser, O. Ringdahl, T. Tielen, and B. van Tuijl (2020) Development of a sweet pepper harvesting robot. Journal of Field Robotics 37 (6), pp. 1027–1039. External Links: Document
3. [3] A. Bashir, Y. Majeed, and A. Zahid (2025) Development of an end-effector for robotic harvesting of hydroponic lettuce. Journal of the ASABE 68 (4), pp. 645–657. External Links: Document
4. [4] A. Bashir, Y. Wang, M. O. Ojo, and A. Zahid (2026) A vision system for occluded cutting point localization in robotic harvesting of greenhouse lettuce. IEEE Transactions on AgriFood Electronics 4 (1), pp. 157–167. External Links: Document
5. [5] S. Birrell, J. Hughes, J. Y. Cai, and F. Iida (2020) A field-tested robotic harvesting system for iceberg lettuce. Journal of Field Robotics 37 (2), pp. 225–245. External Links: Document
6. [6] F. Chaumette and S. Hutchinson (2006) Visual servo control, part i: basic approaches. IEEE Robotics & Automation Magazine 13 (4), pp. 82–90.
7. [7] S. I. Cho, S. J. Chang, Y. Kim, and K. J. An (2002) Development of a three-degrees-of-freedom robot for harvesting lettuce using machine vision and fuzzy logic control. Biosystems Engineering 82 (2), pp. 143–149. External Links: Document
8. [8] G. Dulac-Arnold, N. Levine, D. J. Mankowitz, J. Li, C. Paduraru, S. Gowal, and T. Hester (2021) Challenges of real-world reinforcement learning: definitions, benchmarks and analysis. Machine Learning 110 (9), pp. 2419–2468. External Links: Document
9. [9] L. He, Y. Sun, L. Chen, Q. Feng, Y. Li, J. Lin, Y. Qiao, and C. Zhao (2025) Advance on agricultural robot hand–eye coordination for agronomic task: a review. Engineering 51, pp. 263–279. External Links: Document
10. [10] H. A. Hernández, I. F. Mondragón, S. R. González, and L. F. Pedraza (2025) Reconfigurable agricultural robotics: control strategies, communication, and applications. Computers and Electronics in Agriculture 234, pp. 110161. External Links: Document
11. [11] A. Huang, C. Yu, J. Feng, X. Tong, A. Yorozu, A. Ohya, and Y. Hu (2025) A motion planning method for winter jujube harvesting robotic arm based on optimized informed-rrt* algorithm. Smart Agricultural Technology 10, pp. 100732. External Links: Document
12. [12] Z. Huang, D. Yang, Z. Cheng, and X. Jiang (2025) Towards autonomous premium tea harvesting: a high-efficiency and high-quality path planning based on optimized iabfmt* algorithm in unstructured canopies. Smart Agricultural Technology 12, pp. 101317. External Links: Document
13. [13] Z. Huang, C. Chen, N. Ikegaya, T. Chang, K. Ke, and Y. Chen (2026) A robotic harvesting system for occluded cucumbers using f2sa-yolov8 and hvsc. Computers and Electronics in Agriculture 246, pp. 111616. External Links: Document
14. [14] T. Jin and X. Han (2024) Robotic arms in precision agriculture: a comprehensive review of the technologies, applications, challenges, and future prospects. Computers and Electronics in Agriculture 221, pp. 108938. External Links: Document
15. [15] S. Kang, D. Li, S. Long, Y. Ye, F. Kuang, D. Du, and J. Wang (2025) Time-optimal and path-optimal motion planning for selective broccoli harvesting manipulator. Computers and Electronics in Agriculture 239, pp. 111007. External Links: Document
16. [16] S. Karaman and E. Frazzoli (2011) Sampling-based algorithms for optimal motion planning. The International Journal of Robotics Research 30 (7), pp. 846–894. External Links: Document
17. [17] C. H. Kim, A. Silwal, and G. Kantor (2025) Autonomous robotic pepper harvesting: imitation learning in unstructured agricultural environments. IEEE Robotics and Automation Letters 10 (4), pp. 3406–3413. External Links: Document
18. [18] T. Kunz and M. Stilman (2012) Time-optimal trajectory generation for path following with bounded acceleration and velocity. In Proceedings of Robotics: Science and Systems, pp. 1–8. External Links: Document
19. [19] S. Levine, P. Pastor, A. Krizhevsky, J. Ibarz, and D. Quillen (2018) Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection. The International journal of robotics research 37 (4-5), pp. 421–436. External Links: Document
20. [20] X. Li, K. Hsu, J. Gu, O. Mees, K. Pertsch, H. R. Walke, C. Fu, I. Lunawat, I. Sieh, S. Kirmani, S. Levine, J. Wu, C. Finn, H. Su, Q. Vuong, and T. Xiao (2025) Evaluating real-world robot manipulation policies in simulation. In Proceedings of the 8th Conference on Robot Learning, Proceedings of Machine Learning Research, Vol. 270, pp. 3705–3728.
21. [21] L. Liu, F. Yang, X. Liu, Y. Du, X. Li, G. Li, D. Chen, Z. Zhu, and Z. Song (2024) A review of the current status and common key technologies for agricultural field robots. Computers and Electronics in Agriculture 227, pp. 109630. External Links: Document
22. [22] P. Ma, A. Zhu, Y. Chen, Y. Tu, H. Mao, J. Song, X. Wang, S. Su, D. Li, and X. Dong (2024) Multi objective motion planning of fruit harvesting manipulator based on improved bit* algorithm. Computers and Electronics in Agriculture 227, pp. 109567. External Links: Document
23. [23] P. Malvido Fresnillo, S. Vasudevan, W. M. Mohammed, J. L. Martinez Lastra, and J. A. Perez Garcia (2023) Extending the motion planning framework—moveit with advanced manipulation functions for industrial applications. Robotics and Computer-Integrated Manufacturing 83, pp. 102559. External Links: Document
24. [24] M. Mittal et al. (2025) Isaac lab: a gpu-accelerated simulation framework for multi-modal robot learning. arXiv preprint arXiv:2511.04831.
25. [25] Y. Park, J. Seol, J. Pak, Y. Jo, C. Kim, and H. I. Son (2023) Human-centered approach for an efficient cucumber harvesting robot system: harvest ordering, visual servoing, and end-effector. Computers and Electronics in Agriculture 212, pp. 108116. External Links: Document
26. [26] X. B. Peng, M. Andrychowicz, W. Zaremba, and P. Abbeel (2018) Sim-to-real transfer of robotic control with dynamics randomization. In IEEE International Conference on Robotics and Automation, pp. 3803–3810. External Links: Document
27. [27] V. Rajendran, B. Debnath, S. Mghames, W. Mandil, S. Parsa, S. Parsons, and A. Ghalamzan-E (2024) Towards autonomous selective harvesting: a review of robot perception, robot design, motion planning and control. Journal of Field Robotics 41 (7), pp. 2247–2279. External Links: Document
28. [28] V. Rajendran, B. Debnath, S. Mghames, W. Mandil, S. Parsa, S. Parsons, and A. Ghalamzan-E. (2024) Towards autonomous selective harvesting: a review of robot perception, robot design, motion planning and control. Journal of Field Robotics 41 (7), pp. 2247–2279. External Links: Document
29. [29] J. Rong, Y. Liu, X. Li, C. Gao, P. Wang, T. Yuan, and W. Li (2025) Decoupled motion planning method for 7-dof manipulator and lifting joint in automated tomato harvesting. Computers and Electronics in Agriculture 237, pp. 110693. External Links: Document
30. [30] J. Rong, Z. Xiong, C. Xue, J. Yuan, C. Zhang, P. Wang, T. Yuan, and W. Li (2026) Field validation of vision-guided action policy generation for robotic tomato harvesting. Information Processing in Agriculture. External Links: Document
31. [31] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov (2017) Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347.
32. [32] N. Sobanbabu, G. He, T. He, Y. Yang, and G. Shi (2025) Sampling-based system identification with active exploration for legged sim2real learning. In Proceedings of the 9th Conference on Robot Learning, Proceedings of Machine Learning Research, Vol. 305, pp. 578–598. External Links: Link
33. [33] N. Subedi, H. Yang, D. K. Jha, S. Sarkar, and A. Raghunathan (2026) Find the Fruit: Zero-Shot Sim2Real RL for Occlusion-Aware Plant Manipulation. In IEEE International Conference on Robotics and Automation (ICRA), pp. 0–0. External Links: Link
34. [34] I. A. Şucan, M. Moll, and L. E. Kavraki (2012) The open motion planning library. IEEE Robotics & Automation Magazine 19 (4), pp. 72–82. External Links: Document
35. [35] C. Tang, B. Abbatematteo, J. Hu, R. Chandra, R. Martín-Martín, and P. Stone (2025) Deep reinforcement learning for robotics: a survey of real-world successes. Annual Review of Control, Robotics, and Autonomous Systems 8, pp. 153–188. External Links: Document
36. [36] J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel (2017) Domain randomization for transferring deep neural networks from simulation to the real world. In IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 23–30. External Links: Document
37. [37] E.J. van Henten, C.W. Bac, J. Hemming, and Y. Edan (2013) Robotics in protected cultivation. IFAC Proceedings Volumes 46 (18), pp. 170–177. Note: 4th IFAC Conference on Modelling and Control in Agriculture, Horticulture and Post Harvest Industry External Links: ISSN 1474-6670, Document
38. [38] Z. Wang, K. Lou, B. Zhang, Y. Gu, Q. Xu, and W. Fu (2025) Compliant picking control of dragon fruit picking robot based on adaptive variable impedance. Biosystems Engineering 252, pp. 126–143. External Links: ISSN 1537-5110, Document
39. [39] T. Yu and H. Jiang (2026) Toward sustainable urban agriculture: development of a robotic end effector for hydroponic lettuce in-situ harvesting in vertical farming. Biosystems Engineering 262, pp. 104362. External Links: ISSN 1537-5110, Document
40. [40] T. Yu and H. Jiang (2026) Toward sustainable urban agriculture: development of a robotic end effector for hydroponic lettuce in-situ harvesting in vertical farming. Biosystems Engineering 262, pp. 104362. External Links: Document
41. [41] J. Zhao, Y. Yu, G. Wu, L. Chen, and Q. Zhu (2026) Path planning for agricultural harvesting manipulators under environmental uncertainty: optimization objectives, engineering trade-offs, and learning-based adaptation. Smart Agricultural Technology 14, pp. 102418. External Links: Document

Appendix

Appendix A PSA-PPO Training and Task Configuration

This appendix reports the numerical task and optimization settings used for the final PSA-PPO policy. PSA-PPO used the standard clipped PPO update; the name denotes the harvesting-specific initialization, observation, reward, and strict-success formulation under which the policy was trained. The interaction and success thresholds are listed in Table A.1, the training workspace and target-conditioned safe-start rule in Table A.2, the implemented reward terms in Table A.3, and the PPO/network settings in Table A.5.

The nominal target-conditioned safe-start posture was constructed from the home configuration \mathbf{q}^{home} and a geometry-dependent posture \mathbf{q}^{geo},

\begin{split}&\mathbf{q}^{ss}=(1-\beta)\mathbf{q}^{home}+\beta\mathbf{q}^{geo},\\
&\mathbf{q}^{home}=[-0.24,-1.57,1.57,0,1.57,0]^{\top}\end{split}

where all joint angles are in radians and \beta was selected from the target geometry. With target azimuth \phi=\operatorname{atan2}(p_{y}^{T},p_{x}^{T}) in radians and z_{n}=\operatorname{clip}((p_{z}^{T}-0.59)/0.10,-1,1), the unblended posture was:

\mathbf{q}^{geo}=\begin{bmatrix}\operatorname{clip}(-0.10+0.70\phi,-1.15,1.15)\\
-1.18-0.10z_{n}+\delta_{s}\\
1.24-0.04z_{n}+\delta_{e}\\
0.05\\
1.57\\
0\end{bmatrix}

During training, an independent uniform perturbation in [-0.004,0.004] rad was added to each joint after construction of the nominal safe-start posture. Thus, the pre-positioning rule itself was target-determined, while a small reset perturbation was retained during policy training.

The final-pose capture predicate required the axial, lateral, vertical, and Euclidean position thresholds in Table A.1, c_{t}^{a}>\varepsilon_{a}, c_{t}^{z}<\varepsilon_{v}, and positive instantaneous wrist–TCP chain clearance. Strict capture additionally required both the initial and accumulated below-plane violations to remain within \varepsilon_{safe}. Successful termination required this strict predicate for N_{s} consecutive policy steps; an accumulated violation above \varepsilon_{hard} terminated the episode early.

Appendix B Simulation Validation and Formulation-Ablation Details

This appendix details the fixed simulation-evaluation setup and diagnostics supporting Section 3.3.1. Identical target coordinates and strict evaluation criteria were used for PSA-PPO and all component-removal baselines (Table B.1).

Binary outcomes were paired by target. Risk differences were computed as PSA-PPO minus the corresponding component-removal rate, with 95% confidence intervals from 10,000 paired bootstrap resamples. Exact McNemar tests were used for paired binary outcomes and Holm correction was applied across the three formulation contrasts within each outcome family. The target, rather than trajectory samples, was the inferential unit. Because one independently trained checkpoint was evaluated per formulation, these intervals and tests characterize variation over the evaluation targets and not retraining variability.

Figure B.1 exposes the paired strict-outcome contingencies underlying the aggregate differences in Table B.2.

The three-dimensional outcome maps in Figure B.2 retain target location and failure type simultaneously. Numerical robustness of the selected PSA-PPO checkpoint across the predefined workspace bins is given in Table B.3; the corresponding multi-policy bin comparison is reported in the main paper.

Representative target-matched trajectories selected for each component contrast are shown in Figure B.3. They are included as qualitative diagnostics and are not used to estimate population effects.

The fixed-band corridor measures in Figure B.4 were reconstructed post hoc from the recorded trajectories. They are not mathematically identical to \mathcal{R}^{cor}_{t}, which also contains continuous progress, overshoot, staging, and target-coupled terms. They are therefore reported as secondary diagnostics rather than as an isolated decomposition of the corridor reward.

Figure B.5 reports quantities aligned directly with the removed motion-regularization terms. The no-motion-regularization checkpoint does not produce a monotonic increase in these quantities, which is why the main paper does not attribute lower TCP jerk causally to \mathcal{R}^{reg}_{t}.

Tool-orientation behavior is provided in Figure B.6. Orientation shaping and the final orientation criteria were retained in all three ablations.

Appendix C Physical Controller Benchmark and Deployment Details

This appendix documents the controller configurations, measured-trajectory processing, analysis populations, and secondary diagnostics supporting the physical comparison in Section 3.3.2. Binary reaching outcomes use all scheduled targets for each controller, whereas continuous motion comparisons use the common-success target population defined below.

C.1 Physical Execution and Planner Configuration

The controlled benchmark used the same physical target definitions for PSA-PPO, Cartesian+TOTG, RRTConnect, and RRT*. The paper-level reaching criterion was a 20-mm Cartesian tolerance. The online controller monitor used a tighter 2-mm threshold for execution management; this monitor threshold was not used to define the reported task success.

PSA-PPO retained the frozen deployment correction described in Section 2.4. Table C.1 summarizes the execution and logging parameters used for PSA-PPO deployment on the UR10e. Planner targets were supplied directly to MoveIt without that policy-specific Cartesian correction. Accordingly, the benchmark compares the complete deployed controllers under common target definitions rather than claiming identical low-level command generation.

The MoveIt baselines used a common staged physical benchmark harness. For each stage, the measured current joint state was supplied as the planner start state when available, and a planned-start mismatch above 0.005 rad prevented execution. OMPL-generated trajectories were collision checked by MoveIt and then sent directly to the UR scaled joint-trajectory controller. Cartesian motion used collision-checked Cartesian interpolation followed by time-optimal trajectory generation (TOTG). The principal planner settings are listed in Table C.2.

The final RRT* batch wrapper explicitly passed planner_id = RRTstar and verified that the corresponding ROS planner configuration resolved to an RRT* implementation before execution. The earlier RRTstarkConfigDefault wrapper was superseded and is not used as the provenance for the reported RRT* configuration.

C.2 Trajectory Reconstruction and Analysis Populations

TCP transforms and joint/controller states were archived at 100 Hz for all controllers. All cross-controller kinematic and trajectory-geometry quantities reported in the paper were recomputed offline from these raw trajectory archives using one common processing pipeline; the online per-segment path/speed/acceleration/jerk estimates produced during data collection were not used for the comparative analysis because invalid online path-length estimates were observed for some RRTConnect segments. In the common offline pipeline, samples were timestamp ordered, resampled to a uniform grid, and smoothed using a 0.15-s moving-average window before derivative-based analysis. Trajectory length was evaluated from the smoothed position stream at 30 Hz, with a 2-mm Ramer–Douglas–Peucker tolerance used for geometric simplification. TCP velocity, acceleration, and jerk were obtained by central finite differences. The reported high-end jerk statistic is the 99.5th percentile rather than the numerical maximum. The final 0.22-m target-relative window was fixed from the local PSA-PPO task definition before controller comparison and was applied identically to all methods.

Binary reliability and continuous trajectory comparisons use different populations by construction. All scheduled targets contribute to binary reaching outcomes. Continuous controller comparisons condition on targets retained as successful for all four controllers, so that each paired metric compares the same physical target configuration. Table C.3 summarizes these populations.

The physical target was treated as the experimental unit. Paired continuous differences were evaluated using Wilcoxon signed-rank tests and 10,000 paired bootstrap resamples for confidence intervals. Holm correction was applied within each prespecified family of PSA-PPO–planner comparisons. Trajectory samples within a target were not treated as independent observations.

C.3 Detailed Motion and Geometry Statistics

The main paper emphasizes the final 0.22-m local region because this matches the learned crop-proximal control scope. Table C.4 shows that the same qualitative dynamic ordering is also present over the complete reconstructed approach.

Target-wise effect sizes for the final 0.22-m region are reported in Table C.5. Differences are defined as PSA-PPO minus planner, so negative values indicate a lower PSA-PPO metric. The endpoint-error ECDF is provided in the main paper; the numerical medians are retained in Table C.6 for direct lookup. Population-level trajectory geometry is reported in Table C.7 because lower dynamic excitation does not imply a shorter or straighter path.

C.4 Failure-Identity Sensitivity and Timing Semantics

The aggregate physical reaching totals were witnessed during the experiment, but the exact target identities of some additional planner failures were not independently preserved. These counts were therefore not retrospectively treated as recovered target-level ground truth. Instead, a deterministic sensitivity analysis retained the witnessed aggregate totals and selected the additional planner failure identities from originally logged-success trials using telemetry anomaly rank.

Hard diagnostic events were prioritized when tool-orientation travel reached at least 360^{\circ}, any single-joint travel reached at least 2\pi, or the wrist/elbow angular margin to the nearest multiple of \pi was at most 5^{\circ}. Remaining trials were ranked within each planner using percentile tails of trajectory geometry, orientation travel, joint excursion, acceleration, jerk, tracking error, and elapsed duration. The composite score was

S_{k}=0.60\,\bar{P}_{k}+0.25\,P_{k}^{\max}+0.15\,I_{k}^{\mathrm{hard}},

where \bar{P}_{k} and P_{k}^{\max} denote the mean and maximum within-planner diagnostic percentiles, respectively. Target ID was used to break ties. The selected identities were used only to form a sensitivity population consistent with the witnessed counts; they are not claimed as independently observed failure identities. Table C.8 summarizes the witnessed failure counts, trajectory availability, and anomaly-ranked identity selections used to construct this sensitivity population.

Timing variables from the benchmark are retained only as harness diagnostics. The PSA-PPO harness issued multiple external commands and monitored the TCP against the 2-mm execution threshold, whereas the paper-level outcome uses 20 mm. Consequently, elapsed benchmark time includes monitoring and waiting behavior and cannot be interpreted as neural-network inference latency or as intrinsic controller execution speed. The audit found that only 32 of 1080 PSA-PPO harness commands reached the 2-mm monitor threshold; 1048 did not reach that threshold before monitor completion or timeout, and all 142 paper-level successful PSA-PPO targets contained at least one such inferred 2-mm timeout. For this reason, timing is classified as benchmark-harness-dependent and is excluded from controller-speed claims.

C.5 Kinematic and Atypical-Trajectory Diagnostics

Singularity proximity was evaluated as a diagnostic rather than a causal failure label. Wrist and elbow margins were defined as the minimum absolute angular distance to a multiple of \pi for UR joint 5 and joint 3, respectively. The 5^{\circ} threshold in Figure C.1 was used only by the anomaly-ranking audit; proximity to this threshold does not demonstrate that a singularity caused a stall or failed reach.

Within the sensitivity-selected sets, Cartesian+TOTG contained no orientation-revolution, single-joint-revolution, or near-singularity hard events. The RRTConnect selected set contained two orientation-revolution events, one single-joint-revolution event, three wrist-near-singularity cases, and one elbow-near-singularity case. The RRT* selected set contained four, three, two, and one such cases, respectively. These counts describe the selected diagnostic subsets and not the N=97 population as a whole.

Figure C.2 shows the highest-priority selected planner execution together with PSA-PPO on the same target. The Cartesian case is the highest-ranked non-hard anomaly because no Cartesian hard event was present in the selected set. The RRTConnect and RRT* examples contain hard diagnostic events under the predefined audit rule. These examples illustrate tail behavior only and are not classified as collision events.