Autonomous weeders for Christmas tree plantations - a feasibility study
7 Autonomous vehicle technology – a literature review
Henning Nielsen and Spyros Fountas
The Royal Veterinary and Agricultural University, Section for AgroTechnology
Autonomous vehicles (also called mobile robots and robotic vehicles) are normally categorized as developments from automatic steered vehicles and remote controlled vehicles, while the term robot usually is used for materials handling and tool operating automata mostly used in industry. Varieties of the two have found use in agriculture. An example is the experimental, robotic fruit harvesters developed years ago, which moved (autonomously) from tree to tree and picked the fruit by mean of an robotic arm. Another example is the milking robots, which are used in practical farming, milking the cows when they prefer. Human like robots ought to be regarded as an entertainment industry phenomena.
The most primitive robots, sometimes called industrial robots are just material handlers, mostly pneumatic, typically running through a process of gripping a work piece, carrying it and placing it in another place and returning to start the next cycle. True industrial robots are versatile equipments able to be set up to perform work tasks like seam welding, painting, moving a work piece and assemble components. To do this it should be reprogram able in an easy way without physical changes, have a memory and logic to be able to work independently and automatically and further have a physical structure of a fashion that allows for its use for several tasks without major restructuring (Lundquist, 1996).
Another area relevant to mention about work on robotics and autonomous vehicles is toys. The LEGO Mind stones products have by themselves contributed to the development, but the products have also been found useful as experimental modelling tools. Works like Rooker & Lund (2001) may be applicable for developing the man-machine interface for the operators setting up the work of autonomous machinery. This reference describes a programming tool for LEGO robots to be used by children for setting up autonomous robot soccer players.
Autonomous mini sub-marines designed for under-sea prospecting and search are mentioned by different sources, their relevance for the actual purpose is judged to be limited.
Going into the real life of designing autonomous vehicles, Gomi (2001) stated the priorities of hardware units to be: Battery, motor, connector, sensor, cpu.
The greatest future application of autonomous vehicles, many with robotic actions, is expected to be for domestic purposes including assistance to disabled people (Christensen, 2001; Gomi, 2001). At present most work in this area is at the pre-commercial stage. An exception is a vacuum cleaner from Electrolux. Although the research on other of these types of equipment has reached a rather high level of perfection, marketing for end users must probably wait for some time for safety reasons. However there is a great range of commercial vehicle types available as platforms for research purposes (figure 7.1). Some of these contain controllers with great computing capacity.
Figure 7.1:
Example of skid steered commercial available autonomous vehicle platform for
experiments. The unit is electric powered from batteries and fitted with basic sensors,
computers and communication equipment, which are open for addition of supplementary units.
7.1 Automatic and autonomous vehicles in agriculture, forestry and horticulture
Petersen (1985) described some possibilities for application of robots in agriculture and mentioned picking of fruit, harvesting of vegetables as well as transplanting and spraying. Research has been reported on robotic harvest of apples, grapes and oranges (Burely et al. 1990), robotic harvest of apples (Kataoka et al. 1997), robotic pruning of grapes (Lee et al. 1994) and robotic harvest of vegetables (Hilton, 1997).
Kondo & Ting (1998) describes a number of robots for fruit and vegetable harvesting. Robots for other crop growing operations are also shown. Many of the robots are mounted on vehicles, some without an operator. Apparently the described robots rank from commercial to experimental. The treatment shows the importance of sensors as part of the robots. Reports related to robotic weeding, e.g. Molto (1997) mostly consider weed sensing, in most cases using vision systems. According to Kondo & Ting (1998) the interest for robotic tractors (~autonomous tractors) in Japan is increasing.
Except for some robotic equipment mentioned above the only autonomous vehicle marketed for agriculture-related purposes seems to be lawn movers. Most well-known are two types sold by Husquarna, one solar powered another with battery loading from a mains connected servicing station that the machinery drive to when needed. These lawn movers move the grass in a random linear pattern like the abovementioned vacuum cleaner form Electrolux.
Autonomous vehicle platforms for experiments are commercially available, e.g. from Applied AI Systems, Inc. (2001)(figure 7.1). This firm offers a number of different types (called robots), mostly for indoors use; a few can be used outdoor. The problem is that the latter types are skid-steered, which is not very suitable for weeding purposes. Up to now no commercial vehicle platforms, which are easy to adapt for development of an autonomous weeder have been found.
An apparently more suitable vehicle platform has been made by Madsen & Jacobsen (2001) (Figure A.2, appendix A) as their M.Sc. thesis project. This is a robotic vehicle designed as a carrier for weeding purposes. The vehicle has four-wheel drive four-wheel steer and is designed by mechatronics principles with individual steering motors at each wheel
7.2 Evolution of automatic steering for agricultural vehicle
Experiments on ideas of removing the tractor driver or easing his steering job have already been reported from 1909 (AGRI/WP.2/69, 1962). Since then the technology has developed to make automatic steered vehicles possible. As stated in a review by Wilson (2000) the fact is that for automatic steering of agricultural vehicles of today the guidance will in most cases use GPS (Global Position System) for absolute position sensing and vision systems (image processing systems) for relative position sensing. A further element necessary for practical automatic steering are controllers based on computers with reasonable computing capacity.
The history of automatic steering of agricultural vehicles can briefly be divided in two epochs. Until about 1940 the experimens had mostly been on mechanical systems. A prominent result was furrow followers delivered as standard equipment for some tractors. Remote radio control of tractors was demonstrated in 1936. During the second world war servomechanisms were put into general use. This developed the theory of control systems.
From the 1950’es to the 1980’es a great amount of research on automatic steered agricultural machinery was performed. Reviews of this is found in the references above and in Jahns (1976) and Nielsen et al. (1976). A great part of this research considers sensor principles for guidance information. Further research considers design of the control systems, some on more generally applicable principles and mostly on detailed electronic design with minor relevance today. Liljedahl et al. (1962) applied the theory of automatic control on automatic tractor steering and by this introduced application of mathematical modelling or systems analysis into the subject.
In the 1970’es the increased application of electronic instrumentation for agricultural engineering research was crossbreed with electronic control for the advance of both topics, and at the end of the decennium the microprocessor was introduced in some systems. The decade also saw the first simple electronic monitoring instruments in practical farming.
During the 1980’es electronic equipment found more widespread use in practical farming, mostly monitoring equipment and some simple controls. In this decade also targeted work to increase the quality of farm electronics had been performed. Development and standardization of data bus systems for agricultural tractors were also started up in this period.
In the 1990’es the concept of precision farming brought position measurement (~ navigation systems) into practical farming with GPS (Global Positional System) in practice becoming the universal sensor for absolute position. To obtain sufficient accuracy error correction using differential GPS had to be used, but since the intentionally introduced error (selective availability) has been removed other means of obtaining accuracy have become possible. However, for guidance purposes the accuracy obtained by these systems is not sufficient. In stead RTK GPS (Real Time Kinematic GPS) has to be used. The RTK system extends the differential principle with corrections based on application of the GPS carrier wave phase.
The 1990’es also saw the development of image processing systems able to process relatively complicated images nearer to real time. These systems were partly developed to become research measurement tools, partly researched for post harvest processing of horticultural products (Bennedsen et al., 1996; Bennedsen & Kohsel, 1996; Bennedsen, 1997). Some of the latter have been developed into systems, which now are inserted into production lines in horticulture (Anonymous, 1997). Further research has been made on application of image processing systems for application by control of field machinery, e.g. fruit picking (Peterson & Bennedsen, 1999), weed detection (Pedersen, 2001) and guidance in row crops. The latter has been developed into control systems marketed for guidance of row crop cultivators (Bundgaard, 2001).
An updated broader overview on automatic steering of farm vehicles can be found in a thematic issue (no. 25, 2000) of Computers and Electronics in Farming.
7.3 Sensors for navigation
The primary sensors needed for navigation are systems that can provide information on absolute and relative position, vehicle absolute and relative orientation and speed. Supplementary systems may be needed for specific purposes. Many other sensor types may also be used internally in machinery and for crop sensing, e.g. tactile sensors for mechanically sensing presence of material or force from material. It is worth noting that some in general outdated guidance principles have properties, which may make them candidates for special applications.
7.3.1 Real time Kinematic global position system
As mentioned above navigation (~guidance) of automatically steered vehicle in future is today assumed mostly to be based on measurement of the absolute position based on RTK GPS (Real time kinematic global position system) and relative position determined by vision.
The RTK GPS is based on measurements of the propagation time for the radio waves from the Navstar satellites. The RTK principle is based on application on a local ground based reference station and obtains centimetre level accuracy using the carrier wave phase difference. The principles are described in a number of textbooks and articles. A short technical description of an actual system is, e.g. found in a manual from Trimble (1999). As a minimum a GPS equipment will deliver horizontal coordinates in a ground based coordinate system, but most provide also the vertical coordinate, time and additional information, e.g. about accuracy.
7.3.2 Computer vision
This technology is in many cases based on use of video cameras, often common colour cameras (giving RGB colour information) or grey level cameras. In some cases cameras with other spectral sensitivities or even more special camera types are used. The camera signals are after digitising processed computationally to extract relevant sensor information, e.g. the vehicle heading relative to a crop row. A great amount of literature exists about vision systems and image processing. A part of this is about vision for guidance purposes.
7.3.3 Supplementary sensors
Besides both the above primary navigational sensor systems reliable guidance often will depend on supplementary sensors, among others to detect the angular orientation/attitude of the vehicle. This comprises the three parameters: Heading, roll and pitch. The heading is the direction of driving expressed as an angle relative to another direction; it is primary information for control of the steering. Absolute measure for heading is the compass course. Different compass sensors are available, but the heading can also be calculated from a number of subsequent position measures when driving. Roll and pitch are the angular deviation of the vehicle’s vertical axis from the actual vertical, roll is the sideward angle and pitch is the up-down angle of the vehicle’s forward direction. Both can be measured with inclination sensors. For accurate position determination with RTK GPS roll and pitch are often needed for correction because the GPS aerial is placed at a higher level than the tool to be positioned.
In some cases information on driving distance and velocity are needed for a good control of vehicle driving. This can be supplied by an odometer. For measurement of shorter distances or as an alternative to vision, sensors based on ultrasound can be used.
7.3.4 The leader-cable principle
By this principle the vehicle follows a single cable buried below the track. The cable carries a voice frequency alternating current. The approach is commonly used for automatic transport systems in industrial plants, where the driver-less vehicles have bumpers operating safety stops to avoid damage on persons or other objects in the vehicle path. Variants of this system have been used or been tried for quite a lot of applications. A system for farm use was described by Morgan (1958) and since then more advanced experimental systems have been described, e.g. Jahns (1976,b). Bearing some resemblance to the present project, is the application for automatic lawn movers where the cable signals the limit of the area to be moved. Some automatic movers also use a leader cable for homing guidance to a servicing station. In a GPS-based system a leader cable could be used as part of an emergency subsystem.
7.3.5 Laser-based principles
The use of lasers has possibilities for high accuracy in areas where line of sight can be obtained. Using a laser beam as a very straight guidance line is used for different purposes. Lasers can be used for accurate measurement of angles in one or two dimensions by mechanically scanning the beam in a plane angle or a solid angle respectively. Further the distance can be measured by means of a modulated laser beam reflected back into the instrument with the laser. Distance measurement and angle scanning is sometimes combined in one instrument.
A commercial version of this combined principle made for accurate guidance in limited areas is described by Arnex (1995) and Søgaard (1998). The system has a laser scanner on the vehicle and uses reflective fix-points installed around the field in positions accurately surveyed. The system delivers two measurements per second for each reflector. Because of the low measurement frequency the equipment has a Kalman filter, which is a software estimator to extrapolate into the time interval between last measurement and the time when next position coordinates have been measured and calculated. Another remarkable feature is optics to broaden the laser beam so the light intensity is reduced to eye-safe level.
Laser scanners and laser distance-meters may be used as test instruments. Kalman filtering is generally applicable for real time purposes e.g. in control systems with time discrete and noisy signals. Laser scanners may also be applicable for replacements of vision systems, especially if range data are requested for 3d imaging.
7.3.6 Other principles
Other relative position sensors with possible relevance are mechanical types with a feeler arm. For instance the type used for sensing maize stalks (Kutzbach & Quick, 1999; Suggs et al., 1972), could be modified for sensing Christmas tree trunks.
Simple optical sensors sensing presence or non-presence of material have been used for many purposes, but not very many have been used in field machinery because of the risk of contamination.
Simple optical sensors with ability of discriminating between green plants and other materials have been used experimentally in the 1970’es (Palmer & Owen, 1971; Hooper et al., 1976). The special relevance is that these sensor types, instead of using the green colour used the more pronounced difference between red reflectance and near-infrared reflectance of green plant materials; this feature is also used in some modern vision-based systems.
Tillet (1991) has produced a more recent review of automatic guidance sensors for agricultural field machines.
The topic "degree of autonomy" has been discussed by Castelfranchi (2001). Degree of autonomy is approximately the same as "level of delegation". The topic brings in a lot of implications partly of rather philosophical nature.
7.4 Safety
Safety of machinery working without continuous human supervision is critical. Apart from the functional reliability the machinery must not hurt human or animals and should neither damage third persons property. For unmanned machinery working in open country the problem of possible entry by unwanted persons into the work area is serious.
The controller must handle safety problems as part of its operation. In the case of damage to the machinery or other error conditions the design of the controller and the rest of the machinery must assure a safe function with a graceful degradation.
For the matter of safety of third parties and the operator an alternative outer layer of safety must be provided. In most cases this will be performed by a hardware safety system independent of the controller. In particular the machinery must stop if children or animals are coming near, but not if the operator or another authorized person is present in a safe way.
The safety systems must comply with the rules set up by safety at work authorities. A first approach for safety of an area with working machines is fencing with safety switches stopping the machines in case the fence is opened, a solution which may be prohibitively costly in the case of weeding machinery and also a problem in relation to the work of an operator supervising the work now and then. In general if a specific machine is not covered by special rules the safety system of an actual machine will be judged to comply or not with a generalized set of rules. Danish rules possibly applicable to the safety of autonomous field machinery are at least partly covered by the rules for remote control of technical equipment (Arbejdstilsynet, 1995a) and the rules for automatic controlled machinery, industrial robots included (Arbejdstilsynet, 1995b).
An elementary set of safety demands to automatic steered agricultural vehicles have been set up by Jahns (1975).
7.5 Systems Architecture
An architecture is a description of how a system is constructed from basics and how those components fit together to form the whole (Albus, 1991).
Mobile robots, if they are to perform useful tasks and become accepted in open environments, must be autonomous: capable of acquiring information and performing tasks without programmatic intervention. Due to the complexity and intelligence of an autonomous vehicle it is necessary to incorporate systems architecture already in the design phase. As a result, the literature review is derived by the Artificial Intelligence and Robotics.
There are not many references about systems architectures for autonomous vehicles in agriculture. Nilsson (1980), states that a control system for an autonomous tractor should be decomposed into three functional elements: a sensing system, a planning system, and en execution system. The job of the sensing system is to translate raw sensor input into a world model. The job of the planner is to take the world model and a goal and generate a plan to achieve the goal. The job of the execution system is to take the plan and generate the actions it prescribes.
Later on, the sense-plan-act approach (SPA) became the dominant one in this area. The SPA approach has two significant architectural features. First, the flow of control among these components is unidirectional and linear. Second, the execution of an SPA plan is analogous to the execution of a computer program. Executing a plan or a program is easy when compared with generating one. Therefore, the intelligence of the system lies on the planner or the programmer and not on the execution mechanism (Connell, 1989).
The next step of the sense-plan-act (SPA) approach was the "subsumption" approach, applying task-dependent constraints to the subsumption layers to make SPA more efficient. The most well known example of this approach is the mobile robot called Herbett’s which was programmed to find and retrieve soda cans in an office environment (Connell, 1989).
Rzevski (1995) mentions three main types of three systems architectures for mobile robots. The hierarchies, networks and layered architectures.
A hierarchy is an architecture that consists of elements linked as "parents" and "children". Each parent can have one or more children. Each child may be a parent of other children. In this way multilevel hierarchies are constructed. Hierarchies are used whenever it is necessary to reduce the perceived complexity of a system caused by its scale (size).
In networks, in contrast to hierarchies, there are no levels of importance and all elements may be connected to each other. These architectures are used when there is a need for cooperation between units that are equal in importance but different in terms of skills or capabilities.
Layered architectures, consist of self-contained elements, called layers, each connected to a set of inputs and outputs and thus each capable of creating a system behaviour.
Moreover, from the artificial intelligence literature for mobile robots, the following are some dominant system architectures, which have been widely used in autonomous vehicles:
7.5.1 Three-layer architecture
The three-layer architecture consists of three components: a reactive feedback control mechanism, a reactive plan-execution mechanism, and a mechanism for performing time-consuming deliberative computations. These components run as separate computational processes. According to the algorithms, which are going, to be executed in the processes they should fall into three major equivalence classes.
The first one is the fast, mostly stateless reactive algorithms with hard real time bound on execution time, slow deliberative algorithms like planning, and intermediate algorithms with hard real-time bounds on execution time. Slow deliberative algorithms like planning, and intermediate algorithms, which are fairly fast, but can not provide hard real-time guarantees. "Fast" and "slow" are measured with respect to the rate of change of the environment (Gat, 1998).
7.5.2 The Saphira architecture
This architecture design consists of three central aspects. The ability to attend to another agent, to take advice about the environment and to carry out assigned tasks. All three involve complex sensing and planning operations on the part of the robot, including the use of visual tracking of humans, co-ordination of motor controls and planning. To be able to achieve these aspects, the Saphira architecture uses the concepts of co-ordination of behaviour, coherence of modelling and communication with other agents. (Konolige and Myers, 1998)
At the coherence concept, a mobile robot must have a conception of its environment that is appropriate for its tasks and consequently the more open-added the environment and the more complex the tasks, the more the mobile vehicle will have to understand and represent the environments. At this architecture, an internal model is used, the local perceptual space (LPS) which uses connected layers of interpretation to support reactivity and deliberation. The communication concept implies the ability to understand task commands as well as integrate advice about the environment or its behaviour.
7.5.3 The animate agent architecture
The aim of the animate agent architecture is to design software systems for intelligent robotic agents. Such agents need to be able to pursue a wide variety of goals and interact naturally with people when deciding which goals to achieve and how to achieve them. This type of architecture addresses these issues using a two level model for encoding robot behaviour. A lower level consisting of continuous processes that control the robot’s sensors and effectors and a higher level consisting of a reactive plan executor that selects sequences of actions and programs the lower level at run-time (Firby, et al., 1996).
The two level approaches is designed to cope with the following issues:
 | The details of the world |
| Dynamic situations |
| Contingencies, problems and opportunities |
| The control of continuous processes |
| The integration of purposive vision |
The architecture in general consists of a reactive task execution system, using a hierarchical library of discrete plans and plan steps, and a continuous control system using composable modules called skills. A two level architecture is used to allow the encoding of two quite different, but complementary, types of robot behaviour. The skill level supports the description of continuous control processes, while the task execution level supports the description of multistep plans.
7.5.4 Behavioural reactive system architecture
A behavioural reactive system architecture tightly couples perception to action without the use of intervening abstract representations or time history. This is drawn from the behaviourist school of psychology which considers that behaviour is simply a reaction to a stimulus. The reactive robotic systems have the following characteristics (Arkin, 1998):
| Behaviours serve as the basic building blocks for robotic actions |
| Use of explicit abstract representational knowledge is avoided in the generation of a response |
| Animal models of behaviour often serve as a basis for these systems |
| These systems are inherently modular from a software design perspective |
7.6 Sub conclusion
Autonomous vehicles, also called mobile robots and robotic vehicles, for use in open environments are in an early stage of development. No real autonomous machines have yet been marketed for practical purposes, but a few experimental types are being built and offered for sale for research purposes. A range of sensing systems is being developed to perceive necessary information on position and structure of surroundings. Other work has been done on processes for navigation, steering, safety precautions and other operational purposes. Various system architectures, including a range of databases and processors, have been developed to facilitate this information collection, processing and utilization in an organised way.