Cybernetic Machines. Cyberspace

Because a main aim of many cyberneticians is to under­stand biological learning, various demonstrations have involved ‘‘learning machines’’ realized either as computer programs or as special-purpose hardware. The various schemes for artificial neural nets are examples, and an earlier one was the ‘‘Homeostat’’ of Ross Ashby, which sought a stable equilibrium despite disturbances that could include alteration of its physical structure.

A num­ber of workers, starting with Grey Walter (4), made mobile robots or ‘‘tortoises’’ (land turtles) that showed remark­ably lifelike behavior from simple internal control arrangements. They could avoid obstacles and would seek ‘‘food’’ (electric power) at charging stations when ‘‘hungry.’’ The ‘‘Machina speculatrix’’ by Grey Walter did not learn, actually, but later developments implemen­ted learning in various forms.

A task that has been used in a number of studies is pole­balancing, where the pole is an inverted pendulum con­strained to pivot about a single axis and mounted on a trolley. The task is to control the trolley so that the pole does not fall and the trolley remains within a certain length of track. The input data to the learning controller are indica­tions of the position of the trolley on the track and of the angle of the pendulum, and its output is a signal to drive the trolley. In one study, the controller was made to copy the responses of a human performing the task; in others, it developed its own control policy by trial.

Learning, unless purely imitative, requires feedback of success or failure, referred to as reinforcement. The term ‘‘reinforcement learning,’’ however, has been given special significance as indicating methods that respond not only to an immediate return from actions but also to a potential return associated with the change of state of the environ­ment. A means of estimating an ultimate expected return, or value, for any state has to exist. The most favorable action is chosen to maximize the sum of the immediate return and the change in expected subsequent return. The means of evaluating states is subject, itself to modifica­tion by learning.

This extension of the meaning of ‘‘reinforcement learn­ing,’’ having some correspondence to the ‘‘dynamic pro­gramming’’ of Richard Bellman, has led to powerful learning algorithms and has been applied successfully to the pole-balancing problem as well as to writing a program that learned to play a very powerful game of backgammon.

Cyberspace. Interactions using the Internet and other channels of ready computer communication are said to occur in, and to define, cyberspace. The new environment and resulting feeling of community are real and amenable to sociological examina­tion. The prefix ‘‘cyber-’’ is applied rather indiscriminately to any entity strongly involving computer communication, so that a cafe offering its customers Internet access is termed a ‘‘cybercafe’’, the provision of bomb-making instructions on the Internet is described as ‘‘cyberterror­ism,’’ and so on.

In science fiction, such terms as ‘‘cybermen’’ have been used to refer to humans who are subject to computer control. These uses of the prefix must be depre­cated as supporting an erroneous interpretation of cyber­netics.