A basic problem one needs to solve when creating an intelligent machine that performs automated synthesis of algorithms is discovering a way or manner in which a job is assigned to the machine. There are sophisticated approaches to solving the problem, for example, by creating specifications consisting of rules and constraints. A simpler approach is reducing an algorithm synthesis task to an optimization task. In this approach, a task resolution assessment unit evaluates a result of execution of a version of a synthesized algorithm; the machine makes changes to the version to hopefully produce a version with a better assessment score. A classical approach to solving optimization tasks of this kind is genetic algorithms. A genetic algorithm would evolve a resulting algorithm by trial and error in multiple iterations to maximize the value of a fitness function.
Another approach to solving an algorithm synthesis task as an optimization task is an approach used in QSMM where the machine synthesizes an algorithm simultaneously with its execution. The algorithm being executed can interact with an environment. The task resolution assessment unit provides continuous feedback on the results of execution of the algorithm, and the machine continuously attempts to improve it. A behavior exhibited by the machine is reinforcement learning. QSMM provides means for representing a synthesized algorithm as a finite automaton.
Historically, a numerical quantity specifying a component of an algorithm fitness value is called spur in QSMM; the type of the component is called spur type. QSMM supports multiple spur types, although it is better to use a smaller number of spur types for a more efficient optimization. For example, a spur value can be the logarithm of a probability to maximize, an energy value to minimize, or the sum of rewards or incentives given to a machine when a partially synthesized algorithm reaches some points in solving a task. For time-dependent feedback on changes to fitness value of an algorithm being synthesized, QSMM supports the optimization goal “maximize spur increment velocities”.
QSMM since version 1.17 proposes a higher-level approach to represent a synthesized algorithm as a PCFG (Probabilistic Context-Free Grammar). As a direct consequence of this, the main component of an algorithm fitness value becomes the probability of a PCFG.
In general, a programmer should provide a spur evaluation method that allows a machine to better understand a correlation between changes made to an algorithm being synthesized and observed changes to its behavior. Developing a proper spur evaluation method can be one of the most complex tasks to solve when creating an intelligent machine using QSMM.