Next: , Previous: , Up: Introduction   [Contents][Index]


1.2 Spur-driven Behavior

A basic problem one needs to solve when creating an intelligent machine that performs automated synthesis of algorithms is discovering a way or manner in which a job is assigned to the machine. There are sophisticated approaches to solving this problem, for example, by creating specifications consisting of rules and constraints. A simpler approach is reducing an algorithm synthesis task to an optimization task. In this approach, a task resolution assessment unit assesses a result of execution of a version of a synthesized algorithm; the machine makes changes to this version to hopefully produce a version with a better assessment score. A classical approach to solving optimization tasks of this kind is genetic algorithms. A genetic algorithm would evolve a target algorithm by trial and error in multiple iterations to maximize the value of a fitness function.

Another approach to solving an algorithm synthesis task as an optimization task is an approach where the machine synthesizes an algorithm simultaneously with its execution. An executed algorithm may perform necessary interactions with an environment. The task resolution assessment unit provides continuous feedback on results of execution of the algorithm, and the machine continuously attempts to improve it. A behavior exhibited by the machine is reinforcement learning.

A machine can exhibit this behavior without keeping an explicit representation of a synthesized algorithm. QSMM maintains a representation of a synthesized algorithm as a finite automaton.

Historically, a numerical quantity specifying a component of the fitness of a synthesized algorithm is called spur in QSMM; the type of such component is called spur type. QSMM supports multiple spur types for an optimization task, although it is better to use a smaller number of spur types for the optimization to be more efficient. A spur value can be, for example, the logarithm of a probability to maximize, an energy value to minimize, or the sum of rewards or incentives given to a machine when a partially synthesized algorithm reaches some points in solving a task. For time-dependent feedback on the fitness of a synthesized algorithm, QSMM supports the optimization goal “maximize spur increment velocities” or “maximize spur decrement velocities”.

A programmer should provide a spur evaluation method that allows a machine to better understand a correlation between changes made to an algorithm being synthesized and observed changes to spur values. Developing a proper spur evaluation method can be one of the most complex tasks to solve when creating an intelligent machine using the QSMM framework.


Next: , Previous: , Up: Introduction   [Contents][Index]