Next: , Previous: , Up: Adaptive Probabilistic Mapping   [Contents][Index]


2.9 Switching Adaptive or Random Behavior

An actor uses a random number generator to select a signal from a set of possible signals according to their relative probabilities. The function qsmm_get_actor_sig_action performs this selection. The actor does not use the random number generator if there is only one signal with a positive relative probability—the actor simply selects this signal.

For a small actor, qsmm_get_actor_sig_action selects an output signal based on the relative probabilities of output signals previously calculated by the function qsmm_actor_calc_action_prob. When selecting an output signal of a large actor, the function qsmm_get_actor_sig_action indirectly calls itself a number of times for a small actor associated with the large actor. As a consequence of this, the large actor traverses the Huffman tree of a current action choice state and selects an output signal corresponding to one of tree leaves.

You can obtain a random number generator used by an actor—either provided when creating the actor or an instance of a default random number generator allocated automatically if not provided.

Function: qsmm_rng_t qsmm_get_actor_rng (qsmm_actor_t actor)

This function returns a random number generator used by an actor to select output signals. The function never returns NULL.

You typically use a returned random number generator to seed the generator after creating the actor. See Random Number Generators, for how to seed a random number generator and perform other operations on it.

A useful approach to determine the amount of contribution of an actor to the optimality of behavior of a system is comparing a measure of optimality for this system with the actor behaving adaptively versus a measure of optimality with the actor behaving randomly. The greater the difference is between those values the more contribution the actor makes to the overall optimality of system behavior.

Use the following functions to query the current mode of behavior of an actor and to switch the actor to random or adaptive (normal) behavior mode.

Function: int qsmm_get_actor_random (qsmm_actor_t actor)

This function returns a positive value if an actor is currently in random behavior mode or zero if the actor is in adaptive (normal) behavior mode. The function never returns negative values.

Function: void qsmm_set_actor_random (qsmm_actor_t actor, int flag)

This function switches the current mode of behavior of an actor to random or adaptive (normal) mode. If flag is non-zero, the function switches the current mode to random mode. If flag is zero, the function switches the current mode to adaptive mode.

For a small actor, random and adaptive behavior modes affect how the function qsmm_actor_calc_action_prob calculates probabilities of QSMM_PROB_AGGR or QSMM_PROB_LEARNED type.

In the random mode, qsmm_actor_calc_action_prob called for a small actor does not calculate the relative probabilities of output signals in the normal way but uses output signal weights for those relative probabilities. If the type of probabilities to calculate is QSMM_PROB_AGGR, that function additionally applies a probability profile to those weights if the probability profile exists in statistics storage for a current action choice state. Finally, the function normalizes the resulting values.

For a large actor, random and adaptive behavior modes are actually the behavior modes of a small actor associated with the large actor.

The function qsmm_actor_create initializes the current mode of behavior of a newly created actor to adaptive (normal) mode.


Next: , Previous: , Up: Adaptive Probabilistic Mapping   [Contents][Index]