Next: , Previous: , Up: Adaptive Probabilistic Mapping   [Contents][Index]


2.8 Customizing the Relative Probability Function

Efficient operation of a small or large actor, i.e. optimality of actions the actor produces, depends on the form of a function that returns the relative probability of output signal choice. It may well be true that, for different kinds of applications, different functions are more suitable. The QSMM package provides a few built-in functions the programmer may select for use by an actor. Unfortunately, the author of the package experiences problems with: (1) providing a sound theoretical basis for using those functions in the package (that basis might not exist for some functions); (2) devising functions that give better efficiency than the functions currently provided in the package. The author’s own experimenting had shown that, for some special cases, there do exist more effective functions, but he did fail to find general forms for those functions, making it possible to utilize them for wider ranges of values of actor parameters. To provide a way to solve the above problems by developers, in QSMM starting from version 1.15, a possibility to specify via the Actor API a custom relative probability function for use by an actor is supported.

Relative Probability Function Types

Use the following functions to get or set the type of a relative probability function used by an actor.

Function: enum qsmm_relprob_e qsmm_get_actor_relprob_type (qsmm_actor_t actor)

This function returns the type of a function used by an actor to calculate the relative probability of output signal choice. Argument actor specifies the handle of the actor. If the actor is the large one, then the type of a function used by a small actor associated with the large actor will be returned.

Function: void qsmm_set_actor_relprob_type (qsmm_actor_t actor, enum qsmm_relprob_e relprob_type)

This function sets the type of a function used by an actor to calculate the relative probability of output signal choice. Argument relprob_type specifies that type. Argument actor specifies the handle of the actor. If the actor is the large one, then the type of a function used by a small actor associated with the large actor will be set. When the value of relprob_type does not match any valid element of enumeration qsmm_relprob_e, behavior of the actor is undefined.

Below is a description of the enumeration that specifies supported relative probability function types.

Enumeration: qsmm_relprob_e

This enumeration specifies supported types of a function used by an actor to calculate the relative probability of output signal choice.

The following notations are used in the description of elements of this enumeration.

h

An action choice state.

z

An output signal.

ne

The number of spur types.

T

Actor temperature multiplied by some constant.

F(h,z)

The relative probability of emitting output signal z in action choice state h.

L

The mean discrete time period between an occurrence of action choice state h, when output signal z was emitted, and the subsequent occurrence of h, multiplied by a value returned by the function qsmm_get_actor_naction_per_evt. This discrete time period (with the foregoing multiplication applied) is called a discrete cycle period for short.

nout

The mean number of output signals of the actor (for more information on computing this number, see Other Parameters of an Actor).

W[i]

Spur weight for spur type i.

C(h,z,i)

The ratio of spur increment velocities. It is computed depending on the way of spur perception (see further on in this section) for spur type i.

The enumeration contains the following elements.

QSMM_RELPROB_BUILTIN1

The relative probability function is defined by the formula

relprob_builtin1.formula

where κ=2L.  The paper “An Approach to Optimal Action Generation for a System that Interacts with the Environment” available from the project homepage provides more details pertaining to the formula.

QSMM_RELPROB_BUILTIN2

The relative probability function is defined by the formula

relprob_builtin2.formula

The author has empirically found this formula when developing version 1.15 of QSMM package and making an actor generate output signals with frequencies proportional to probabilities of those output signals. The probabilities had been specified a priori, and spur increments supplied to the actor just after emitting corresponding output signals were equal to logarithms of those probabilities.

QSMM_RELPROB_BUILTIN3

[New in QSMM 1.16] The relative probability function is defined by the formula

relprob_builtin3.formula

This formula is a simplification of the formula that corresponds to relative probability function type QSMM_RELPROB_BUILTIN2 and is meant for developers who prefer beauty and/or simplicity.

QSMM_RELPROB_USER1

The relative probability function is defined by the formula

relprob_user1.formula

where f(L) is a function of type qsmm_relprob_user1_func_t supplied by a programmer via a call to qsmm_set_actor_relprob_helper (see further on in this section). If function f(L) is not supplied, then 1 will be used for it.

QSMM_RELPROB_USER2

[New in QSMM 1.16] The relative probability function is defined by the formula

relprob_user2.formula

where g is a function of type qsmm_relprob_user2_func_t supplied by a programmer via a call to qsmm_set_actor_relprob_helper (see further on in this section). Function g receives the value of z and statistics associated with h and z. These function arguments may be necessary to devise a sophisticated relative probability function.

When creating a small actor, the function qsmm_actor_create initializes the type of the relative probability function of the small actor to QSMM_RELPROB_BUILTIN1. When creating a large actor, the function qsmm_actor_create initializes the type of the relative probability function of a small actor associated with the large actor to QSMM_RELPROB_BUILTIN2.

Helper Functions

Below are descriptions of types for functions f(L) and g.

Data type: qsmm_relprob_user1_func_t

This is a type for the pointer to a helper function that can be supplied by a programmer for relative probability function type QSMM_RELPROB_USER1. The following declaration corresponds to this type:

typedef double
(*qsmm_relprob_user1_func_t)(
    qsmm_actor_t actor,
    double cycle_period,
    void *paramp
);

Argument actor holds the handle of an actor that has called the helper function. Argument cycle_period holds the value of L. Argument paramp holds a user parameter specified when setting the helper function for the actor. The helper function should return the value of f(L).

Data type: qsmm_relprob_user2_func_t

This is a type for the pointer to a helper function that should be supplied by a programmer for relative probability function type QSMM_RELPROB_USER2. The following declaration corresponds to this type:

typedef double
(*qsmm_relprob_user2_func_t)(
    qsmm_actor_t actor,
    qsmm_sig_t sig_cycle,
    const struct qsmm_state_s *state_p,
    const struct qsmm_cycle_s *cycle_p,
    const struct qsmm_sspur_s *sspur_p,
    const struct qsmm_cspur_s *cspur_p,
    void *paramp
);

Argument actor holds the handle of an actor that has called the helper function. Argument sig_cycle holds the value of z. Arguments state_p, cycle_p, sspur_p, and cspur_p hold statistics associated with h and z. See Structures for Accessing Storage, for a description of the corresponding structures. Argument paramp holds a user parameter specified when setting the helper function for the actor. The helper function should return the value of g.

To obtain working parameters of an actor, in the body of a helper function such functions as qsmm_get_actor_nsig_ctrl and qsmm_get_actor_discrete_cycle_period_mean may be called. See Other Parameters of an Actor, for a detailed description of these functions. If the need exists to use application-specific information associated with h, then the n-gram of h can be retrieved by a call to qsmm_get_actor_sig_ngram during execution of the helper function.

Use the following functions to get or set the helper function for an actor.

Function: void qsmm_get_actor_relprob_helper (qsmm_actor_t actor, void **helper_func_pp, void **helper_func_param_pp)

This function retrieves information on a helper function used by an actor when calculating the relative probability of output signal choice. Argument actor specifies the handle of the actor. If there is such helper function set for the actor, and helper_func_pp is not NULL, then the function will set *helper_func_pp equal to the pointer to the helper function. If there is no such helper function set for the actor, and helper_func_pp is not NULL, then the function will set *helper_func_pp equal to NULL. If helper_func_param_pp is not NULL, then the function will set *helper_func_param_pp equal to the user parameter of this helper function specified when setting it for the actor. If the actor is the large one, the function retrieves information on a helper function used by a small actor associated with the large actor. A particular interpretation of the pointer stored in *helper_func_pp depends on a relative probability function type set for the actor by a call to qsmm_set_actor_relprob_type.

Function: void qsmm_set_actor_relprob_helper (qsmm_actor_t actor, void *helper_func_p, void *helper_func_param_p)

This function sets the pointer to a helper function used by an actor when calculating the relative probability of output signal choice. Argument actor specifies the handle of the actor. Argument helper_func_p specifies the pointer to the helper function. Additionally, the function sets the user parameter of that helper function to helper_func_param_p. If the actor is the large one, this function sets the helper function for a small actor associated with the large actor.

A particular interpretation of pointer helper_func_p depends on a relative probability function type set for the actor by a call to qsmm_set_actor_relprob_type. Providing a pointer that does not conform with the relative probability function type will lead to undefined behavior of the library.

The value of helper_func_p can be NULL. Such NULL value means that there is no helper function set for the actor. Setting the NULL value makes a certain sense when the relative probability function type is QSMM_RELPROB_USER1.

To provide an example of using the functions qsmm_set_actor_relprob_type and qsmm_set_actor_relprob_helper, let us suppose a developer has proved that it is mathematically correct to use the function for calculating the relative probability of output signal choice like the function QSMM_RELPROB_BUILTIN2 but with the following one difference: the numerator of the exponent contains the expression L instead of expression L+1. To make an actor calculate the relative probability of output signal choice using this function, the developer may define the helper function such as:

#include <math.h>

static double
get_relprob_exp_mlt(
    qsmm_actor_t actor,
    double cycle_period,
    void *paramp
) {
    return log(qsmm_get_actor_nsig_ctrl(actor))*cycle_period/2;
}

To set this helper function for use by an actor, the developer may write these function calls:

qsmm_set_actor_relprob_type(actor,QSMM_RELPROB_USER1);
qsmm_set_actor_relprob_helper(actor,&get_relprob_exp_mlt,0);

Alternatively, since QSMM version 1.16, to use that relative probability function, the developer may write the function calls shown below. A description of function qsmm_set_actor_ktemperature will be given later in this section.

qsmm_set_actor_relprob_type(actor,QSMM_RELPROB_BUILTIN3);
qsmm_set_actor_ktemperature(actor,2);

Way of Spur Perception

Most of the relative probability function formulas given earlier in this section contain the function C(h,z,i) equal to the ratio of spur increment velocities. The method of computing C(h,z,i) depends on the way of spur perception for spur type i. The way of spur perception determines whether that ratio will be generally positive or negative for generally positive or negative spur increments supplied to the actor. Two ways of spur perception are supported: normal and inverse. The normal way of spur perception provides generally positive ratio for generally positive spur increments and generally negative ratio for generally negative spur increments. The inverse way of spur perception provides generally negative ratio for generally positive spur increments and generally positive ratio for generally negative spur increments.

The following enumeration specifies the way of perception of the spur of a particular type.

Enumeration: qsmm_spur_perception_e

This enumeration specifies the way of spur perception for a spur type.

Below are notations used in the description of elements of this enumeration. In the description of notations, a cycle of type <h,z> is the event history segment between an occurrence of action choice state h, when output signal z was emitted, and the subsequent occurrence of h. When referring to a particular cycle type, all cycles of that type are meant.

t

Discrete or continuous time used to compute spur increment velocity.

E[i]

The value of the spur of type i the actor currently accumulated.

H(h,z)[i]

The sum of increments of the spur of type i over cycle type <h,z> since the beginning of actor operation.

ω(h,z) 

The sum of periods of discrete or continuous time over cycle type <h,z> since the beginning of actor operation.

The enumeration contains the following elements.

QSMM_SPUR_PERCEPTION_NORMAL

The value of C(h,z,i) is the ratio of the spur increment velocity of a cycle type to the absolute value of the mean spur increment velocity:

percept_normal.formula

When spur increments over the event history are non-negative, the value of C(h,z,i) increases as spur increment velocity increases. When the spur increment velocity of a cycle type is positive and is equal to the mean spur increment velocity, the value of C(h,z,i) is equal to 1.

QSMM_SPUR_PERCEPTION_INVERSE

The value of C(h,z,i) is the negative ratio of the absolute value of the mean spur increment velocity to the spur increment velocity of a cycle type:

percept_inverse.formula

When spur increments over the event history are negative, the value of C(h,z,i) increases as spur increment velocity increases. When the spur increment velocity of a cycle type is negative and is equal to the mean spur increment velocity, the value of C(h,z,i) is equal to 1.

The inverse way of spur perception can be used to countervail the negative spur with the other negative spur perceived as the positive one because of the inverse way of perception specified.

Use the following functions to get or set the way of spur perception for a spur type.

Function: int qsmm_get_actor_spur_perception (qsmm_actor_t actor, int spur_type, enum qsmm_spur_perception_e *spur_perception_p)
Function: int qsmm_set_actor_spur_perception (qsmm_actor_t actor, int spur_type, enum qsmm_spur_perception_e spur_perception)

If spur_perception_p is not NULL, the function qsmm_get_actor_spur_perception sets *spur_perception_p equal to the way of spur perception by an actor. The function qsmm_set_actor_spur_perception sets the way of spur perception by an actor to spur_perception. When spur_perception does not match any valid element of enumeration qsmm_spur_perception_e, behavior of the actor is undefined.

Argument actor specifies the handle of the actor. Argument spur_type specifies the spur type and cannot be less than -1 or greater than or equal to the number of spur types returned by the function qsmm_get_actor_nspur for the actor. If the actor is the small one, specifying spur type -1 is incorrect.

If the actor is the large one, the functions get or set the way of spur perception for a corresponding spur type of a small actor associated with the large actor. Special spur type -1 of the large actor corresponds to the automatic spur of the small actor if this small actor uses the automatic spur.

On success, the functions return a non-negative value. On invalid spur_type, the functions return negative error code QSMM_ERR_INVAL. The value of spur_type is invalid in the following situations:

The function qsmm_actor_create initializes ways of spur perception for all spur types of a newly created actor to QSMM_SPUR_PERCEPTION_NORMAL.

Spur Weight

Use the following functions to get or set spur weight denoted by W[i] in the relative probability function formulas mentioned earlier in this section.

Function: int qsmm_get_actor_spur_weight (qsmm_actor_t actor, int spur_type, double *weight_p)
Function: int qsmm_set_actor_spur_weight (qsmm_actor_t actor, int spur_type, double weight)

If weight_p is not NULL, the function qsmm_get_actor_spur_weight sets *weight_p equal to the weight of a spur type of an actor. The value of *weight_p obtained in this way will always be finite. The function qsmm_set_actor_spur_weight sets the weight of a spur type of an actor to weight.

Argument actor specifies the handle of the actor. Argument spur_type specifies the spur type and cannot be less than -1 or greater than or equal to the number of spur types returned by the function qsmm_get_actor_nspur for the actor. If the actor is the small one, specifying spur type -1 is incorrect.

If the actor is the large one, the functions get or set the weight of a corresponding spur type of a small actor associated with the large actor. Special spur type -1 of the large actor corresponds to the automatic spur of the small actor if this small actor uses the automatic spur.

On success, the functions return a non-negative value. On failure, the functions return negative error code QSMM_ERR_INVAL. The following situations are failures:

The function qsmm_actor_create initializes weights of all spur types of a newly created actor to 1.

Actor Temperature

In the relative probability function formulas mentioned earlier in this section, T denotes actor temperature multiplied by some constant. Actor temperature is a concept that follows from the use of exponential forms for the relative probability function resembling formulas related to the Boltzmann distribution. In most of the relative probability function formulas, actor temperature is a divisor of every spur weight.

The perfect relative probability function is the one that provides maximum operating optimality of an actor without the need to tune up its temperature by hand. That is, for such function, default temperature equal to 1 can always be used. If the function is imperfect, its application is not intended one, or when applying the simulated annealing approach for solving a problem, use the following API functions to adjust the temperature of an actor for increasing its operating optimality.

Function: double qsmm_get_actor_ktemperature (qsmm_actor_t actor)

This function returns the temperature (multiplied by some constant) of an actor. Argument actor specifies the handle of the actor. If the actor is the large one, this function returns the temperature of a small actor associated with the large actor. The returned value is always finite and positive.

Function: int qsmm_set_actor_ktemperature (qsmm_actor_t actor, double val)

This function sets the temperature (multiplied by some constant) of an actor to val. Argument actor specifies the handle of the actor. If the actor is the large one, this function sets the temperature of a small actor associated with the large actor.

On success, the function returns a non-negative value. If val is not a finite number or is not a positive number, then negative error code QSMM_ERR_INVAL will be returned.

The function qsmm_actor_create initializes the temperature (multiplied by some constant) of a newly created actor to 1.


Next: , Previous: , Up: Adaptive Probabilistic Mapping   [Contents][Index]