Next: Customizing the Relative Probability Function, Previous: Event History N-gram, Up: Adaptive Probabilistic Mapping [Contents][Index]

To calculate probabilities of emitting output signals for an action choice state, which is stored in an actor window that holds current n-gram from the event history, the following function can be used.

- Function:
*int***qsmm_actor_calc_action_prob***(qsmm_actor_t*`actor`, int`rez1`, qsmm_sig_t`sig_beg`, qsmm_sig_t`sig_end`, enum qsmm_prob_e`prob_type`) This function fills the internal array of

`actor`with probabilities of type`prob_type`calculated for an action choice state, which is stored in an actor window that holds current n-gram from the event history. Argument`rez1`is reserved for future use and must be equal to 0.For a small actor, the contents of the internal array that holds probabilities calculated can be used to stochastically choose an optimal output signal by the function

`qsmm_get_actor_sig_action`

. For a large actor, the function`qsmm_actor_calc_action_prob`

operates much more slowly and should be used only when values of probabilities have to be obtained. Large actors support calling the function`qsmm_get_actor_sig_action`

, which returns control quickly, without prior calling the function`qsmm_actor_calc_action_prob`

.Arguments

`sig_beg`and`sig_end`can be both zero or may specify identifiers of the first signal (inclusive) and of the last signal (exclusive) of a signal segment for which the function should calculate probabilities. In the internal array that holds probabilities calculated, the probability of the first signal of that signal segment has offset`sig_beg`. Array elements within the signal segment, which do not correspond to output signals, are set to 0. If`sig_end`is 0, then the total number of actor signals will be used for the value of the last signal of the signal segment. The function will execute faster when the signal segment is shorter, so it is recommended to specify the segment as precisely as possible.The function returns a non-negative value on success or a negative error code on failure. Currently, the following error codes can be returned.

`QSMM_ERR_INVAL`

The value of

`sig_beg`is greater than or equal to a value (incremented by 1) of the last signal of the signal segment, or the value of`sig_end`is greater than the total number of signals of the actor, or the value of`prob_type`is invalid.`QSMM_ERR_NGRAM`

The contents of an actor window, which holds current n-gram from the event history, correspond to an invalid action choice state. To determine the validity of an action choice state, it is checked for accordance with allowed ranges of signal identifiers specified using the field

`range_sig_p`

of structure`qsmm_actor_desc_s`

when creating the actor.`QSMM_ERR_STORAGE`

Statistics storage failure. This could leave a large actor in indeterminate state. If the large actor is in indeterminate state, then after removing a reason of the error, the operation can be repeated, and if the function succeeds, then the actor’s state will become determinate.

`QSMM_ERR_NOMEM`

There was not enough memory to perform the operation. This could leave a large actor in indeterminate state. If the large actor is in indeterminate state, then after removing a reason of the error, the operation can be repeated, and if the function succeeds, then the actor’s state will become determinate.

A type of probabilities the function `qsmm_actor_calc_action_prob`

must calculate is specified using an enumeration described below.
The enumeration is also used in several other cases.

- Enumeration:
**qsmm_prob_e** This enumeration specifies a type of probabilities. It contains the following elements.

`QSMM_PROB_AGGR`

Aggregate probabilities based on learned and profile probabilities.

For a small actor, to calculate probabilities of type

`QSMM_PROB_AGGR`

, probabilities of types`QSMM_PROB_LEARNED`

and`QSMM_PROB_PROFILE`

(see below) are calculated, each learned probability is multiplied by a corresponding profile probability, and the resulting array of values is normalized. For a large actor, the above operation is performed by the small actor for all nodes of the Huffman tree.For a small actor, if you do not use profile probabilities, then it will be faster to calculate probabilities of type

`QSMM_PROB_LEARNED`

, which are equal to probabilities of type`QSMM_PROB_AGGR`

in this case.For a large actor, probabilities of type

`QSMM_PROB_AGGR`

are ordinarily equal to probabilities of type`QSMM_PROB_LEARNED`

, because profile probabilities typically follow only from a positional relationship of nodes of the Huffman tree (probabilities assigned to edges, which come from tree nodes to their child nodes, are usually equal), and those profile probabilities are taken into account when calculating probabilities of type`QSMM_PROB_LEARNED`

.`QSMM_PROB_LEARNED`

Learned probabilities, according to which optimal actions may be produced.

`QSMM_PROB_PROFILE`

For a small actor, profile probabilities specified a priori in statistics storage (see Structures for Accessing Storage, for information on how to pass profile probabilities to storage access functions to write them to storage). For a large actor, profile probabilities, which follow from the structure of a Huffman tree that corresponds to an action choice state.

`QSMM_PROB_FQ`

Probabilities proportional to observed frequencies of emitting output signals. Those frequencies are automatically incremented for output signals with identifiers passed to the function

`qsmm_actor_reg_sig_action`

, but the increment is actually performed by the function`qsmm_actor_reg_sig_in`

when the same action choice state occurs again in the event history.`QSMM_PROB_COUNT`

The last element of the enumeration equal to the number of supported types of probabilities.

To obtain a pointer to the internal array that holds probabilities of emitting output signals calculated by the function `qsmm_actor_calc_action_prob`

, the following function can be used.

- Function:
*double ****qsmm_get_actor_choice_sig_prob***(qsmm_actor_t*`actor`) This function returns the pointer to an internal array of an actor specified by handle

`actor`. The internal array holds output signal emitting probabilities, possibly relative ones. Using that pointer, the array can be examined or modified by an application program. This function never returns`NULL`

. The number of elements in the array is equal to the total number of signals of the actor returned by the function`qsmm_get_actor_nsig`

. The array can be filled with probabilities of desired type by the function`qsmm_actor_calc_action_prob`

.For a small actor, the content of the array is used by the function

`qsmm_get_actor_sig_action`

to stochastically generate an output signal. This means that the array may be modified by an application program to affect such stochastic generation. After such a modification, the array may contain values of positive infinity or relative probabilities that do not sum up to 1. Conversely, for a large actor, modifying the content of the array by an application program does not affect the behavior of the actor.The function

`qsmm_get_actor_choice_sig_prob`

locks the internal array of probabilities: when the internal array is represented by a sparse vector, the sparse vector is converted to an ordinary vector; after the conversion, the actor always uses ordinary vectors until the internal array of probabilities is released by the function`qsmm_actor_choice_sig_prob_release`

. Forgetting to call the function`qsmm_actor_choice_sig_prob_release`

after a call to`qsmm_get_actor_choice_sig_prob`

may dramatically decrease actor performance.

To release a locked internal array of probabilities, the following function can be used.

- Function:
*void***qsmm_actor_choice_sig_prob_release***(qsmm_actor_t*`actor`) This function releases an internal array of

`actor`that holds (possibly relative) probabilities of emitting output signals, which could have been locked by the function`qsmm_get_actor_choice_sig_prob`

. If the array is not locked, then the function will have no effect. After the release, a pointer that might have been previously obtained by the function`qsmm_get_actor_choice_sig_prob`

must not be used.

When the internal array is locked or if is called for a large actor, the function `qsmm_actor_calc_action_prob`

uses an ordinary vector for storing probabilities of emitting output signals, so elements of that ordinary vector can be accessed via a pointer returned by the function `qsmm_get_actor_choice_sig_prob`

.
When the internal array is released, the function `qsmm_actor_calc_action_prob`

called for a small actor can use an ordinary or sparse vector for storing probabilities of emitting output signals depending on properties of the probability profile and on the value of field `sparse_fill_max`

of structure `qsmm_actor_desc_s`

specified when creating the actor.

There is a way to obtain the handle of an ordinary or sparse vector that holds probabilities of emitting output signals. It is a cheap operation that does not perform locking or converting a sparse vector to an ordinary vector. The vector is returned as is, in read-only mode. See Ordinary and Sparse Vectors, for information on how to access the elements of the returned vector.

- Function:
*qsmm_vec_t***qsmm_get_actor_choice_sig_prob_vec***(qsmm_actor_t*`actor`) This function returns the handle of an ordinary or sparse vector of

`actor`, which holds (possibly relative) probabilities of emitting output signals. This function never returns`NULL`

. The vector can be filled with probabilities of desired type using the function`qsmm_actor_calc_action_prob`

. For a small actor, the contents of the vector are used by the function`qsmm_get_actor_sig_action`

to stochastically generate an output signal.

To stochastically generate an output signal, the following function can be used.

- Function:
*int***qsmm_get_actor_sig_action***(qsmm_actor_t*`actor`, int`rez1`, qsmm_sig_t`sig_beg`, qsmm_sig_t`sig_end`, qsmm_sig_t *`sig_action_p`) If an

`actor`is a small one, this function stochastically generates an output signal according to (possibly relative) probabilities stored in the internal array of the actor. That array can be filled with probabilities of desired type using the function`qsmm_actor_calc_action_prob`

. A pointer to the array is returned by the function`qsmm_get_actor_choice_sig_prob`

. A read-only view of the array, which can speed up accessing its contents, is returned by the function`qsmm_get_actor_choice_sig_prob_vec`

.If an

`actor`is a large one, this function stochastically generates an optimal output signal without using the aforementioned internal array. A Huffman tree is used to provide fast choice of an output signal (the number of those signals can be very large) by the actor according to probabilities of type`QSMM_PROB_AGGR`

, which can be computed by the function`qsmm_actor_calc_action_prob`

. However, for large actors, there is no need to call that function before calling the function`qsmm_get_actor_sig_action`

; the number of calls to the function`qsmm_actor_calc_action_prob`

should be made as little as possible, because for those actors that function operates very slowly.A generated output signal is stored in *

`sig_action_p`if`sig_action_p`is not`NULL`

. Argument`rez1`is reserved for future use and must be equal to 0.Arguments

`sig_beg`and`sig_end`can be both zero or may specify identifiers of the first signal (inclusive) and of the last signal (exclusive) of a signal segment, to which the generated output signal should belong. However, for a large actor, the signal segment must include all output signals of the actor; see Specifying Weights of Output Signals, for information on how to choose an optimal output signal from different subsets of output signals of a large actor. In the internal array of small actor, a probability of the first signal of that signal segment has offset`sig_beg`. If`sig_end`is 0, then the total number of actor signals will be used for the value of the last signal of the signal segment. The function will execute faster when the signal segment is shorter, so it is recommended to specify the segment as precisely as possible. For a small actor, the function does not check whether a signal it generates actually belongs to a set of output signals of the actor: if an element of the internal array within the signal segment has a positive value, then a corresponding signal can be potentially returned by the function. If at least one relative probability in the internal array within the signal segment has the value of positive infinity, then, for a small actor, the function will choose an output signal as a uniformly distributed random element from the set of signals to which such infinite relative probabilities correspond.To stochastically generate an output signal, the actor uses either a random number generator supplied via the field

`rng`

of structure`qsmm_actor_desc_s`

when creating the actor by the function`qsmm_actor_create`

or an instance of default random number generator allocated automatically if 0 is specified in that field. The handle of a random number generator used by an actor can be obtained via the function`qsmm_get_actor_rng`

, e.g. to seed the generator.The function returns a non-negative value on success or a negative error code on failure. Currently, the following error codes can be returned.

`QSMM_ERR_INVAL`

One of the following conditions is met:

- – the value of
`sig_beg`is greater than or equal to a value (incremented by 1) of the last signal of the signal segment; - – the value of
`sig_end`is greater than the total number of signals of the actor; - – the actor is the large one, and the value of
`sig_beg`is greater than the lowest output signal identifier of the actor; - – the actor is the large one, and the value of
`sig_end`is less than the highest output signal identifier of the actor plus 1; - – the actor is the small one and the internal array does not contain (within the signal segment) relative probabilities equal to positive infinity, but the sum of relative probabilities within the signal segment is not finite.

- – the value of
`QSMM_ERR_WEIGHT`

The actor is the small one, and a negative number or a non-finite number, which is not a positive infinity, was encountered in the internal array within the signal segment.

`QSMM_ERR_NOCHOICE`

The actor is the small one, and the internal array does not contain at least one positive element within the signal segment.

`QSMM_ERR_NGRAM`

The actor is the large one, and the contents of an actor window, which holds current n-gram from the event history, correspond to an invalid action choice state. To determine the validity of an action choice state, it is checked for accordance with allowed ranges of signal identifiers specified using the field

`range_sig_p`

of structure`qsmm_actor_desc_s`

when creating the actor.`QSMM_ERR_STORAGE`

Failure of statistics storage of a large actor. This could leave the large actor in indeterminate state.

`QSMM_ERR_NOMEM`

There was not enough memory to perform the operation by a large actor. This could leave the large actor in indeterminate state.

For a small actor, an optimal action is typically produced and registered using a sequence of function calls like this:

qsmm_sig_t sig_action=QSMM_SIG_INVALID; if (qsmm_actor_calc_action_prob(actor, 0, NSIG_IN, NSIG_IN+NSIG_OUT, QSMM_PROB_LEARNED)<0) goto Error; if (qsmm_get_actor_sig_action(actor, 0, NSIG_IN, NSIG_IN+NSIG_OUT, &sig_action)<0) goto Error; if (qsmm_actor_reg_sig_action(actor,sig_action)<0) goto Error;

However, if, in a particular case, the application program chooses on its own which output signal to emit, then calls to `qsmm_actor_calc_action_prob`

and `qsmm_get_actor_sig_action`

can be omitted, but the small actor should be still notified of an output signal actually emitted using a call to `qsmm_actor_reg_sig_action`

.

For a large actor, to produce and register an optimal action, a call to `qsmm_actor_calc_action_prob`

is redundant and only decreases performance.
The sequence of function calls looks like this:

qsmm_sig_t sig_action=QSMM_SIG_INVALID; if (qsmm_get_actor_sig_action(actor, 0, NSIG_IN, NSIG_IN+NSIG_OUT, &sig_action)<0) goto Error; if (qsmm_actor_reg_sig_action(actor,sig_action)<0) goto Error;

Large actors do not support choosing an output signal by an application program.
That is, for a large actor, it is incorrect to omit the call to `qsmm_get_actor_sig_action`

before calling `qsmm_actor_reg_sig_action`

, or to pass to the latter function a signal identifier other than that, which was returned by a preceding call to `qsmm_get_actor_sig_action`

.

In QSMM version 1.16 there is introduced a function, which retrieves a probability of the last output signal generated. The function returns control quickly even when called for a large actor and can be used to calculate the time of emitting an output signal, e.g. according to the exponential probability distribution.

- Function:
*double***qsmm_get_actor_prob_action***(qsmm_actor_t*`actor`) This function returns the probability of the last output signal generated by

`actor`in the function`qsmm_get_actor_sig_action`

. For a small actor, it is a probability that corresponds to the element of an internal array of the actor, which holds relative probabilities of emitting output signals. For a large actor, it is the probability of a path from the root of the Huffman tree to its leaf that corresponds to the output signal. The returned value is always in the range 0 to 1 (inclusive). If the function`qsmm_get_actor_sig_action`

is not yet called, then 0 will be returned.