Developing assisted harvesting

By Geoff Lewis with photos by Trefor Ward 

First published in the June 2022 issues of NZGrower and The Orchardist.

Assisted harvesting is the next step in labour-saving

Several spin-off businesses associated with Waikato University are busy tackling the issue of labour shortages in horticulture.

In March, graduates Josh Barnett and Jono Tobias were collecting data from a rig towed behind a tractor through a field of broccoli as part of the on-going development of horticultural 'Assisted Harvesting' devices.

Barnett is the managing director of Axis7 Ltd which is an R&D company with technical expertise in robotics and machine design. He comes from a mechanical engineering background and Tobias from electrical engineering. 

Several years ago Barnett had designed an autonomous asparagus harvester also through Waikato University. That project is now in the hands of Tauranga's Robotics Plus Ltd with $5 million in funding to take it further.

With over five years’ experience working in Agri-robotics, Barnett is back in the field in a scoping study funded by a $30,000 grant from Agricultural and Marketing Research and Development Trust, Agmardt. The aim of the study is to collect data which would help in the development of a device to reduce the number of workers required to harvest broccoli as part of a solution to chronic labour shortages with Kaimai Fresh. Kaimai Fresh grows asparagus, broccoli, cauliflower and cabbages over 225ha near Matamata. While the project is in its early stages further funding support is looking promising, he says. 

Kaimai Fresh owner Matt Carnachan says labour has become a critical problem.

''To find labour we've tried everything, going through Work and Income, finding people in the local area and the RSE scheme, nothing really works. In the past with asparagus we needed a lot of labour but we have now reduced that to a team of 8 to 10.      ''Asparagus will fall back to being a niche product. Our focus is now on brassicas and a smaller amount of labour. The issue is a matter of survival for all labour-intensive crops, potatoes, carrots, onions, salad greens, everyone is looking to autonomous harvesters.''

Carnachan says Axis7's philosophy was not trying to make the leap to full automation but to work on a level of 'functional automation' - a harvester which can be towed behind a tractor and can reduce the number of workers needed rather than replace them entirely.

''Full automation becomes far more complex than a 'cut and convey' device that can be towed behind a tractor. There is a balancing act between economics, speed, accuracy and cost. ''

Barnett says the goal is to build a machine capable of replacing about 10 labourers in the Matamata operation..

''The labour issue has got to the point where produce is being left in the field, both New Zealand and overseas.''

The Axis7 data collection rig is equipped with stereo RBG (red-blue-green) cameras and a time-of-flight camera provided by Hamilton company Chronoptics, which provides expertise in 3D imaging. Using machine learning, these sensors can be used to accurately locate and determine the maturity of the produce, Barnett says.''We are fortunate to have worked on some great early-stage AgriTech projects in the past and see brassica harvesting as global problem that we can help to solve.''

Meanwhile an 'E-Bin' - a device designed to take the load off Kiwifruit pickers, is another Assisted Harvesting project underway between Waikato University's engineering faculty and industry organisation Zespri. 

Headed by engineering lecturer Nick Pickering, its aim is to enlarge the potential labour supply by creating technologies which reduce the physically-demanding aspects of Kiwifruit harvesting.

Pickering draws from 20 years’ experience across aircraft avionics and airport IT systems transformation. He had worked professionally with Zespri while GM of IT at Datacom and currently lectures at Waikato. There he discovered the capabilities of Waikato's Robotics, Automation and Sensing research group.

''When I learnt about the work of WaiRAS I approached Zespri with some innovative ideas and then co-developed their on-orchard automation strategy. Assisted Harvesting was one of the opportunities identified as a stepping-stone to full automation.

''Watching videos and researching the challenge during the October 2021 lockdown we quickly saw the demanding physical nature of the harvesting job with the majority of workers being under 30. At the same time we were hearing about unemployment in the regions. So we thought ''let's build something that expands the demographics of the picking workforce and try and kill 2 birds with one stone by creating augmented robotics that create jobs in our regions.''

Pickering says successful innovation often requires hitting the 'sweet spot' of technical feasibility, user desirability and financially viability. 

''Coming from academia we are good at the technical feasibility perspective but conscious that we don’t always know the nuances of a specific domain so we decided to take a human-centred co-design approach by working closely with different specialists. In this case, working with Garcia contracting, Agriculture Risk Management (ARM) and Zespri (initially in online workshops during the October 2021 lockdown) to continuously evolve the design to find something that meets every stakeholders needs. Sometimes the hardest innovation is coming up with the simplest product possible.''

''The most rewarding aspect of the project for me has been associated with the capability development providing R & D value for money through a hybrid team of professional researchers working alongside one of our full time masters students Graeme Chubb.  

''Graeme completed an undergraduate degree in engineering at Waikato and has demonstrated his technical, innovation and professional skills that epitomise the benefits we see in students performing project-based learning throughout their degree to solve real world problems – a capability that is key to keeping NZ Inc. competitive in the Agritech arena.''

The Zespri Assisted Harvest partner project began with testing in the Waikato lab on a fake canopy and instrumented 3D printed kiwifruit to evolve and assess mechanisms that work for end users while being gentle on the fruit. 

The first prototype of the 'E-bin' harvester is currently going through 12 orchard trials where fruit damage is being evaluated through the storage cycle and human factors including performance, usability, injury risk and fatigue are considered. 

''Within the human assisted harvesting project we’re working to bring together health, computing and engineering specialists from academia and integrate them with industry to solve real world problems.

"It's fantastic to see Zespri taking a lead here by funding the co-design projects for the good of the industry.

In the orchard the battery-powered E-Bin moves in front of the pickers. In Auto-mode it can move at about 50mm - 100mm/sec. Users slide the empty bin on the front and then slide it off when the bin is full.

''The E-Bin means pickers don't have to physically carry bags of Kiwifruit, which can weigh around 25kgs, and allows them to concentrate on harvesting. Varying orchard widths can be accommodated with adjustable side catchers on the bin.

''We are working on autonomous vehicles on the survey robot project but have kept the costs down for this through the use of auto-drives to free up the driver so he or she can concentrate on picking.

''The Driver can go faster in manual mode (tethered control) to move around the orchard or make course corrections required while picking. The key has been how low cost we can get to support a quick scale-up of the solution to get it solving the labour challenges as soon as possible.

''It’s early days but the prototype has been well received by all. The efficient augmentation of robotics into human-centred processes is often a missing link to the success of labour-use across all industries. 

''We believe the human-assisted harvesting prototype will actually create jobs in the region for a wider demographic of people to support the kiwifruit industry's expansion targets. We’re looking at it from a perspective of what can we achieve for NZ Inc. with robotics and the available labour pool,'' Pickering says.

A high proportion of robotics solutions consist of the cost of sensors, batteries, computing and navigation equipment.

''As a result we aim to build “Modular Agritech Systems for Horticulture (MAS-H)” from a hardware and software perspective. This is to ensure re-use across different domains and throughout the year to increase the ROI of technology while reducing the impact of supply chain interruptions including using common spares and regional support capability to a modular on-farm/orchard replacement strategy. It’s taking the proven design for availability principles from my aircraft design years and applying it into the world of Agritech,'' Pickering says.