Collecting critically endangered cliff plants using a drone-based sampling manipulator

Cliffs present a unique flora that has been little studied until now mainly because of the inherent difficulties to access this unique environment, as shown in Fig. 2. The techniques currently used to access plants on steep slopes and cliffs (e.g., abseiling, helicopter) are generally dangerous, costly and time consuming. Using a small aerial manipulator to sample plants on the cliffs can represent many advantages, including safety and portability, as well as the capability of reaching otherwise inaccessible locations easily, quickly and at low cost.

However, several technical challenges make it difficult to develop suitable aerial manipulators for this task. Indeed, the sampling of plants on cliffs necessarily leads to significant collision risks, as well as contact forces and moments during sampling that can destabilize the drone. The samples collected would also need to be accessed from the side of the aerial platform²². Any weight (e.g., sampling tool, collected samples) located horizontally away from the center of mass of the drone creates large additional demands on the propulsion system of most drones. To collect specific plant parts in windy conditions (e.g., scion, flowers, seeds, etc.), precise and fast motion is required even in degraded Global Navigation Satellite System (GNSS) coverage near the cliffs. The great diversity of plant species and morphology found on cliffs, as well as the variety of targeted sections of plant, also represent a major design challenge. Finally, to maximize the adoption of this tool, it is also desirable that scientists with minimal training could use this platform. The next sections describe how these challenges were addressed through the development of the Mamba.

Suspended sampling platform

There are a multitude of configurations that could have been explored to sample plants on cliffs. Some drones have manipulators rigidly attached to their structure^20,23. However, these manipulators tend to have a limited reach to keep the center of mass within the propeller footprint and to minimize the inertia of the system. This could result in a high collision risk with the propellers in the uneven terrain found on cliffs. The contact forces created during the sampling operation also generate destabilizing moments through manipulators rigidly attached to the drone. To address these challenges, concepts involving a compliant manipulator operated from specialized drones were also explored¹⁰. Alternatively, some aerial manipulators were also passively suspended under the drone through a long rod^21,24. This keeps the drone above potential obstacles within the environment, significantly reducing the operator’s mental demand and stress while also reducing the disturbances transmitted to the drone to a downward force aligned with the rod and yaw torque. To maintain these advantages while providing better precision, some projects have developed cable suspended platforms equipped with thrusters^25,26. As these platforms do not have to counter gravity, the thrusters can be positioned to fight external disturbances more efficiently (e.g., wind, contact forces, drone movements). Existing systems however only stabilize the suspended platform close to its equilibrium point.

The chosen concept for the Mamba, illustrated at Fig. 3, consists of a suspended platform that can stabilize itself far from its natural equilibrium to provide a large workspace. The lifting drone in this system stays safely away and above from steep cliff faces, while supporting the platform and providing rough positioning in space through better GNSS coverage. The platform is suspended 10 m below the lifting drone using four attachment points to prevent pitch and roll motions. The cable also acts as a low pass filter, isolating the platform from the fast drone movements required to fight wind disturbances. The suspended platform design can then focus on fast and precise positioning, while also being tolerant to contacts during sampling. To do so, four pairs of bidirectional actuators are used to control the motion in the plane of the pendulum (i.e., x and y translation, as well as yaw). Two pairs of actuators are installed in the x-direction to provide sufficient force to reach plants as far as 4 m from the equilibrium position. This corresponds to roughly 3.3 m from the tip of the lifting drone’s propellers.

Since the Mamba is self-powered and has its own communication system, the lifting drone function is simply to lift the platform and hold it in place. This made it possible to select amongst the many commercially available products to accelerate the development of the Mamba. The DJI M300 was chosen as it comes equipped with a 360° optical obstacle avoidance vision system, an IP45 rating, and a flight time of 20 min with the Mamba attached (3.3 kg). It also advertised a four constellation GNSS receiver for better coverage around buildings, structures, and cliffs.

Precise control in winds

Winds under 20 km/h represent a gentle breeze on the Beaufort scale. At this level, the wind only moves the leaves, and not the branches, which allows for ideal sampling conditions. According to historical weather data from 2020, daily maximum winds are less than 20 km/h for 40 to 70% of the year, depending on the exact location on Kauaʻi Island (i.e., Lihuʻe International airport, as reported by the National Oceanic and Atmospheric Administration, and the Makaha Ridge Weather Station, as reported in the MesoWest database). This also implies that Kauaʻi experiences stronger winds on certain days which would make precise sampling difficult. Wind conditions are also more challenging near cliff faces, with increased turbulence and vertical airflow along the cliff.

To allow operations on most days, while providing precise positioning and fast rejection of wind disturbances, the actuators of the Mamba are oriented in the horizontal plane. This allows the actuator forces to directly affect the motion of the suspended platform. Each actuator of the Mamba consists of a pair of brushless DC motors and 23 cm propellers capable of producing 7 N of force. The motors are installed in opposite directions, are always idling at their minimum rotation speed, and are commanded to only create force in their preferred direction. This antagonistic configuration avoids the low-velocity dead zone of a brushless motor during thrust reversal. This makes it possible to quickly revert the direction of the thrust and nearly triples the bandwidth of the actuators to approximately 2.5 Hz²⁷. This configuration, however, comes at the expense of added mass and components.

The Mamba is equipped with a flight controller that includes a control system, and a state estimator. To avoid degraded GNSS coverage issues, the state estimator only uses data from a high accuracy inertial measurement unit (IMU) to estimate the attitude of the platform. This provides the relative position of the platform with respect to the drone and is sufficient for teleoperation. Three separated proportional-derivative controllers are used for each of the DOF controlled by the actuators. This control system also provides attitude-hold assistance (i.e., pitch and roll, which correspond to x and y displacements, as well as yaw). This implies that if the user does not send any commands, the suspended platform maintains its current state.

Figure 4 illustrates the stabilization accuracy of the Mamba when moving along a representative trajectory when suspended indoors from a 5.7 m cable (limited by ceiling height). This experiment confirmed that the sampling tool can maintain a position at a horizontal reach of 2.25 m with a precision of about 5 cm for 30 s. As the horizontal reach and precision are limited by the cable angular displacements (e.g., component of weight acting on the pendulum, IMU angular resolution), the resulting workspace when operating with a 10 m long cable would reach a radius of 4 m with a positioning accuracy of about 9 cm. To account for potential external disturbances like wind, the sampling tool was designed with an opening of 15 cm. This creates some margin for the pilot to align the target with the sampling mechanism. Field trials detailed below demonstrated that the Mamba actuators and controller could maintain a sufficiently stable position to sample plants in winds During the sampling phase, wind speed averaged 15.7 km/h with a standard deviation of 6.8 km/h, while wind gusts reached an average of 20.1 km/h with a standard deviation of 6.5 km/h. The maximum average wind speed recorded during sampling was 28 km/h with gusts up to 37 km/h. This represents a lower bound of the system performance, as no failure resulted from the wind conditions experienced during the trials. The a ttached Supplementary Video also demonstrates the stability of the system.

Teleoperated sampling of cliffs habitats

Plants growing on Kauaʻi cliffs exhibit a wide morphological variety. For this project, targets ranged from small herbaceous plants such as Euphorbia eleanoriae (plants < 40 cm), to the multi-headed sub-shrub Wilkesia hobdyi (plants < 1 m), the larger shrub and tree form of Hibiscadelphus distans (shrubs up to 5 m) and even the full-sized tree Flueggea neowawraea (trees up to 30 m). To accommodate these requirements, a specialized version of the DeLeaves’ tree sampling system (http://www.outreachrobotics.com) has been developed as shown in Fig. 3. This system includes a rotating saw and two grippers to grasp and hold the sample. The new system has been reduced in size to collect smaller plants. The sampling mechanism was designed to sample small parts from herbaceous plants and shrubs. Accordingly, the two grippers can securely hold branches up to 200 g. The length of the sample collected is limited to under 50 cm, depending on the sampling mechanism orientation. The distance between the two grippers is adjustable, from 9 to 15 cm, to accommodate different sample lengths. The cutting system consists of a brushless motor and a 77 mm diameter saw blade. The cutting system can cut a branch with a diameter up to 15 mm. The full cutting sequence is automated and takes less than 2.6 s, almost 3× faster than the DeLeaves sampler. This reduces disturbances on the suspended platform and allows easier sampling of small plants moving in winds. With the push of a button, the saw is accelerated. Then, the lower robotic gripper closes rapidly to capture the sample, before slowing down to bring the target sample into the saw at a controlled speed to prevent jamming. During that time, the upper gripper holds the sample in place. The upper gripper includes urethane foam covered with emery tape to conform to different sample shapes and increase grip on smooth stems. To align the cutting mechanism with samples growing in all directions from the cliff faces, a 2 DOFs actuated wrist is used to rotate the gripper in pitch and roll. The pitch range of motion goes from − 90° to 90° from vertical, while the roll range of motion goes from − 90° to 180°.

To allow for greater adaptability to each unique sampling mission, and because the samples of interest are rare and of high importance, it was decided that the Mamba would be guided by an operator through teleoperation. The precise local positioning and augmented stability described earlier, coupled with a human operator, also reduces the needs for global positioning through sophisticated perception or GNSS. To allow target identification and teleoperation, the system is equipped with a high-definition camera and a video transmitter offering a first-person view during the sampling process. The remote controller is run with a custom software package based on QGroundControl²⁸ which offers several status indicators as shown in Fig. 5.

Field trials

Field trials were conducted in October 2021. Performance validation and pilot training was performed on common species before targeting rare plants. A ll collections were made under appropriate permits for threatened and endangered plants. Bio sanitation of the cutting mechanism was completed between each collection event using 70% isopropyl alcohol to prevent spread of potential contaminants, fungi, or diseases.

Prior to the field trials, the NTBG drone survey program, with the help of the Plant Extinction Prevention Program, located and prioritized targets of interest. Once on site with the Mamba, a first reconnaissance flight was carried out using a small drone to allow botanists to confirm the specific samples to target, while also allowing the aerial manipulator operators to fully understand the environment in which the sampling mission would take place. After the reconnaissance flight was completed, the drone- Mamba system was deployed to the location of the target plant. Deployment of the system on location typically takes less than 10 min. As the targeted environments have many natural obstacles (e.g., mountains, canyons, dense vegetation, overhanging parts of the cliff), operations carry the risk of degraded GNSS signals and unseen obstacles. To mitigate these risks, the lifting drone was manually controlled for all operations with the help of the DJI M300 optical obstacle avoidance system. In the event of entanglement, there is a Mamba emergency release to prevent damage to the plant.

Operation of the drone-Mamba requires two users. The first user controls the lifting drone to move towards the target location and position the desired sample within the Mamba workspace. The second user then maneuvers the Mamba to position the plant inside its sampling tool and activates the cutting system to collect the sample. Once collected, the wrist can be reoriented to secure the sample. When multiple flights are planned during the same field day, two battery sets are used and recharged with a generator to allow for continuous operations.

The Mamba was successful in collecting 11 specimens of 5 different species, as detailed in Fig. 6. These collection events represent the first time each population has been sampled. It is important to document their location, bring the plant material into our collection and represent the genetics of these populations ex-situ in case the populations are lost. Due to the plants’ extremely remote locations, they may have never been sampled without this tool.

Samples were collected more than 1 km from the take-off location in an average of 8.5 min from take-off to landing. Batteries would typically be swapped between every other flight, both for the DJI M300 and the Mamba. As expected, the prototype was able to collect a variety of trees, shrubs and even small herbaceous plants. The Mamba was deployed from the road, after short hikes or even after helicopter drop-offs. It was used to collect samples on cliffs, on ridges and from other areas otherwise difficult to access on foot. The DJI M300 accessed 16 satellites on average during transit flights but only 9 satellites when sampling in close proximity to cliffs. Unfortunately, it was found that the DJI M300 only supports two concurrent GNSS constellations, instead of the four advertised, without the addition of a RTK base station.

One initially unexpected advantage of the Mamba that was exploited during the final field trials resides in its capacity to rapidly access different sampling sites in a single day from a single base station. The flight paths of the fieldwork conducted on November 3, 2021 are illustrated in Fig. 7, and represent a prime example of the reach enabled by the Mamba. Plant collections were made from three distinct areas that are separated by as much as 1100 m, from a single nearby high point that is more easily accessible. This capability facilitates operational planning, increases efficiency (e.g., less packing and unpacking) and significantly reduces the cost of sampling (e.g., number of helicopter flights required to access different locations).

Source: Ecology - nature.com