Previous: Receiving Input Signals, Up: Repeated Sequence of Operations [Contents][Index]
Emitting an output signal includes the following steps:
qsmm_actor_calc_action_prob
the probabilities of output signals for a current action choice state.
This step is only necessary for a small actor.
qsmm_get_actor_sig_action
an output signal to emit.
For a small actor, that function uses output signal probabilities calculated in the previous step.
For a large actor, that function uses a Huffman tree corresponding to the current action choice state.
qsmm_actor_reg_sig_action
the output signal to the buffer containing an n-gram of signals from the event history (at this point—containing the current action choice state).
That function records the following information for the current action choice state: current discrete time, current continuous time, current spur values, and the output signal emitted.
In order to correctly calculate the probabilities of output signals, the actor should have a correct parameter specifying a relation between discrete time and the number of output signals emitted. Use the following functions to query or set the value of this parameter.
This function returns a number greater than 0 and less than or equal to 1 specifying the mean number of output signals emitted by an actor per one unit of discrete time passed.
This function sets to val the mean number of output signals emitted by an actor per one unit of discrete time passed.
On success, the function returns a non-negative value.
If val is not a finite number or is a number less than or equal to 0 or is a number greater than 1, the function returns negative error code QSMM_ERR_INVAL
.
The function qsmm_actor_create
initializes the mean number of output signals emitted per one unit of discrete time passed to 0.5.
Let us suppose that the field ngram_sz
of qsmm_actor_desc_s
structure passed to the function qsmm_actor_create
when creating an actor was equal to NGRAM_SZ
.
The general rules of setting the parameter specifying the mean number of output signals emitted per one unit of discrete time passed are the following:
qsmm_actor_shl_sig
and qsmm_actor_reg_sig_in
to set a current action choice state, and it solely consists of input signals of the actor, then set this parameter to 1.0/(NGRAM_SZ+1)
.
qsmm_actor_shl_sig
and qsmm_actor_reg_sig_in
to set a current action choice state, and it consists of last output signal emitted and input signals, then set this parameter to 1.0/NGRAM_SZ
.
qsmm_get_actor_sig_ngram
and qsmm_actor_reg_sig_in
to set a current action choice state, then use default parameter value 0.5.
The types of probabilities of output signals are the following.
This enumeration specifies a type of probabilities. The enumeration contains the following elements.
QSMM_PROB_AGGR
Aggregate probabilities based on learned and profile probabilities.
For a small actor, to calculate probabilities of QSMM_PROB_AGGR
type, the function qsmm_actor_calc_action_prob
calculates probabilities of QSMM_PROB_LEARNED
and QSMM_PROB_PROFILE
types (see below), multiplies each learned probability by a corresponding profile probability, and normalizes a resulting array.
For a large actor, the function qsmm_actor_calc_action_prob
called for an associated small actor performs this operation for the nodes of a Huffman tree to obtain the aggregate probabilities of tree leaves.
For a small actor, if you do not use profile probabilities, it is faster to calculate probabilities of QSMM_PROB_LEARNED
type—they are equal to probabilities of QSMM_PROB_AGGR
type in this case.
QSMM_PROB_LEARNED
For a small actor, learned probabilities for producing adaptive actions.
For a large actor, probabilities of QSMM_PROB_AGGR
type.
QSMM_PROB_PROFILE
For a small actor, profile probabilities specified a priori in statistics storage (see Structures for Accessing Storage, for how to pass profile probabilities to storage access functions to write them to storage). For a large actor, profile probabilities following from a subordination relationship of nodes of Huffman tree of an action choice state and from the probabilities of edges connecting parent tree nodes with their child nodes.
QSMM_PROB_FQ
Probabilities proportional to observed frequencies of output signals registered by the function qsmm_actor_reg_sig_action
.
For a large actor, calculating probabilities of this type is faster compared to the other types.
QSMM_PROB_COUNT
The last element of this enumeration equal to the number of supported probability types.
Use the following function to calculate the probabilities of output signals for a current action choice state.
This function fills the internal array of actor with probabilities of prob_type type calculated for an action choice state encoded by the buffer containing an n-gram of signals from the event history.
The functions qsmm_actor_shl_sig
, qsmm_actor_shr_sig
, qsmm_actor_reg_sig_in
, and qsmm_actor_reg_sig_action
access that buffer, and the function qsmm_get_actor_sig_ngram
returns a pointer to the buffer.
The argument rez1 is for future use and must be equal to 0.
For a small actor, the function qsmm_get_actor_sig_action
stochastically selects an output signal according to probabilities in that internal array.
The function qsmm_actor_calc_action_prob
is much slower for a large actor compared to a small actor;
you should call this function for a large actor only to obtain explicit probability values—large actors support calling qsmm_get_actor_sig_action
without prior calling qsmm_actor_calc_action_prob
.
The arguments sig_beg and sig_end can be both zero or may specify the identifiers of the first signal (inclusive) and last signal (exclusive) of a signal segment for calculating the probabilities. The probability of the first signal of this signal segment has offset sig_beg in the internal array. The function zeroes array elements that do not correspond to output signals. If sig_end is 0, the function uses the total number of actor signals for the identifier of last signal. The function operates faster when the signal segment is shorter, so specify the segment as precisely as possible.
The function returns a non-negative value on success or a negative error code on failure. Currently, the function can return the following error codes.
QSMM_ERR_INVAL
The value of sig_beg is greater than or equal to the identifier of last signal of the signal segment, or the value of sig_end is greater than the total number of signals of the actor, or prob_type is invalid.
QSMM_ERR_NGRAM
The buffer does not encode a valid action choice state.
To determine the validity of an action choice state, the function checks the content of the buffer for accordance with allowed ranges of signal identifiers specified by the field range_sig_p
of qsmm_actor_desc_s
structure when creating the actor.
QSMM_ERR_CALLBACK
A helper function of QSMM_RELPROB_USER1
or QSMM_RELPROB_USER2
type for calculating the relative probability of an output signal reported an error by returning NaN
.
QSMM_ERR_STORAGE
Statistics storage failure. See Getting the Reason of a Storage Failure, for how to get an error message describing the failure.
QSMM_ERR_STATS
Inconsistent statistics on an action choice state or cycle type detected.
QSMM_ERR_ILSEQ
A generated error message is not convertible to a wide string according to a current locale, or a storage redirection function reported QSMM_ERR_ILSEQ
.
QSMM_ERR_NOMEM
There was not enough memory to perform the operation.
Errors QSMM_ERR_STORAGE
, QSMM_ERR_STATS
, QSMM_ERR_ILSEQ
, and QSMM_ERR_NOMEM
can leave a large actor in inconsistent state.
If the large actor is in inconsistent state, then, if after removing a reason of an error a repeated call to this function succeeds, the actor’s state becomes consistent.
Use the following function to obtain a pointer to the internal array with the probabilities of output signals calculated by the function qsmm_actor_calc_action_prob
.
This function returns a pointer to the internal array of actor holding the relative probabilities of output signals.
This function never returns NULL
.
The number of elements in the array is equal to the total number of actor signals returned by the function qsmm_get_actor_nsig
.
The function qsmm_actor_calc_action_prob
fills the array with probabilities of a desired type.
For a small actor, the function qsmm_get_actor_sig_action
stochastically selects an output signal according to probabilities in that array.
A program can modify the array to affect this stochastic selection.
The function qsmm_get_actor_sig_action
handles the situation when the array contains elements equal to positive infinity or values that do not sum up to 1.
Conversely, modifying the array does not affect the behavior of qsmm_get_actor_sig_action
for a large actor.
The function qsmm_get_actor_choice_sig_prob
locks the internal array: if a sparse vector backs up the internal array, this function converts the sparse vector to an ordinary vector.
After this conversion, the actor keeps the internal array as an ordinary vector until the function qsmm_actor_choice_sig_prob_release
releases that internal array.
Forgetting to call qsmm_actor_choice_sig_prob_release
after a call to qsmm_get_actor_choice_sig_prob
may dramatically decrease actor performance.
Use the following function to release a locked internal array.
This function releases the internal array of actor locked by the function qsmm_get_actor_choice_sig_prob
.
That internal array holds the relative probabilities of output signals.
If qsmm_get_actor_choice_sig_prob
has not locked the internal array, qsmm_actor_choice_sig_prob_release
has no effect.
You must not use a pointer obtained by qsmm_get_actor_choice_sig_prob
after releasing the array.
When the function qsmm_get_actor_choice_sig_prob
has locked the internal array, and you call the function qsmm_actor_calc_action_prob
, or when you call it for a large actor, qsmm_actor_calc_action_prob
uses an ordinary vector for storing the probabilities of output signals, so you can access the elements of this ordinary vector via a pointer returned by qsmm_get_actor_choice_sig_prob
.
When the internal array is in released state, qsmm_actor_calc_action_prob
called for a small actor can use an ordinary or sparse vector for storing the probabilities of output signals depending on the properties of a probability profile and the value of sparse_fill_max
field of qsmm_actor_desc_s
structure specified when creating the actor.
You can obtain the handle of an ordinary or sparse vector holding the relative probabilities of output signals. It is a cheap operation that does not perform locking or converting a sparse vector to an ordinary vector. You can access a returned vector in read-only mode. See Ordinary and Sparse Vectors, for how to access vector elements.
This function returns the handle of an ordinary or sparse vector of actor holding the relative probabilities of output signals.
This function never returns NULL
.
The function qsmm_actor_calc_action_prob
fills the vector with probabilities of a desired type.
For a small actor, the function qsmm_get_actor_sig_action
stochastically selects an output signal according to probabilities in that vector.
Use the following function to stochastically select an output signal.
If actor is a small one, this function stochastically selects an output signal according to relative probabilities stored in the internal array of relative probabilities of output signals.
The function qsmm_actor_calc_action_prob
fills the internal array with probabilities of a desired type.
The function qsmm_get_actor_choice_sig_prob
returns a pointer to the elements of that array.
The function qsmm_get_actor_choice_sig_prob_vec
returns a read-only view of that array for speeding up accessing its content.
If actor is a large one, the function qsmm_get_actor_sig_action
selects an output signal for an action choice state encoded by the buffer containing an n-gram of signals from the event history without using the aforementioned internal array.
The functions qsmm_actor_shl_sig
, qsmm_actor_shr_sig
, qsmm_actor_reg_sig_in
, and qsmm_actor_reg_sig_action
access that buffer, and the function qsmm_get_actor_sig_ngram
returns a pointer to the buffer.
The function qsmm_get_actor_sig_action
uses the Huffman tree of this action choice state to perform fast choice of an output signal (the number of those signals can be very large) according to probabilities of QSMM_PROB_AGGR
type.
The function qsmm_actor_calc_action_prob
can compute those probabilities;
however, for a large actor, there is no need to call that function before calling qsmm_get_actor_sig_action
.
You should minimize the number of calls to the function qsmm_actor_calc_action_prob
to calculate probabilities of QSMM_PROB_AGGR
type for a large actor, because that function operates slowly in this mode.
The function qsmm_get_actor_sig_action
stores a selected output signal in *sig_out_p if sig_out_p is not NULL
.
The argument rez1 is for future use and must be equal to 0.
The arguments sig_beg and sig_end can be both zero or may specify the identifiers of the first signal (inclusive) and last signal (exclusive) of a segment for selecting an output signal. However, for a large actor, the signal segment must include all output signals of the actor; see Specifying Output Signal Weights, for how to adaptively select an output signal from various subsets of output signals of a large actor. For a small actor, the probability of the first signal of this signal segment has offset sig_beg in the internal array. If sig_end is 0, the function uses the total number of actor signals for the identifier of last signal. The function operates faster when the signal segment is shorter, so specify the segment as precisely as possible. For a small actor, the function does not check whether a signal it selects actually belongs to a set of output signals—if an internal array element within the signal segment is positive, then the function may select a corresponding signal. For a small actor, if at least one relative probability in the internal array within the signal segment is positive infinity, the function selects an output signal as a uniformly distributed random element from a set of all signals (within the signal segment) with relative probabilities equal to positive infinity.
To stochastically select an output signal, the function uses either a random number generator provided via the field rng
of qsmm_actor_desc_s
structure when creating the actor by the function qsmm_actor_create
or an instance of default random number generator allocated automatically if that field was NULL
.
Call the function qsmm_get_actor_rng
to obtain the handle of a random number generator used by an actor, for example, to seed the generator.
The function returns a non-negative value on success or a negative error code on failure. Currently, the function can return the following error codes.
QSMM_ERR_INVAL
One of the following conditions is true:
QSMM_ERR_WEIGHT
For a small actor, the internal array contains (within the signal segment) a negative element or a non-finite element different from positive infinity.
QSMM_ERR_INFIN
For a small actor, the internal array does not contain elements (within the signal segment) equal to positive infinity, but the sum of array elements is positive infinity.
For a large actor, the function qsmm_get_actor_sig_action
or qsmm_actor_reg_sig_in
called for an associated small actor when traversing a Huffman tree from the root node to a leaf node reported QSMM_ERR_INFIN
.
This error can leave the large actor in inconsistent state.
QSMM_ERR_NOCHOICE
For a small actor, the internal array does not contain at least one positive element (within the signal segment).
QSMM_ERR_NGRAM
For a large actor, the buffer does not encode a valid action choice state.
To determine the validity of an action choice state, the function checks the content of the buffer for accordance with allowed ranges of signal identifiers specified by the field range_sig_p
of qsmm_actor_desc_s
structure when creating the actor.
QSMM_ERR_CALLBACK
For a large actor, a helper function of QSMM_RELPROB_USER1
or QSMM_RELPROB_USER2
type for calculating the relative probability of an output signal reported an error by returning NaN
.
The function qsmm_actor_calc_action_prob
called for an associated small actor when traversing a Huffman tree from the root node to a leaf node calls this helper function.
This error can leave the large actor in inconsistent state.
QSMM_ERR_STORAGE
Failure of statistics storage of a large actor. See Getting the Reason of a Storage Failure, for how to get an error message describing the failure. This error can leave the large actor in inconsistent state.
QSMM_ERR_STATS
Inconsistent statistics on an action choice state or cycle type detected for a large actor. This error can leave the large actor in inconsistent state.
QSMM_ERR_ILSEQ
For a large actor, a generated error message is not convertible to a wide string according to a current locale, or a storage redirection function reported QSMM_ERR_ILSEQ
.
This error can leave the large actor in inconsistent state.
QSMM_ERR_NOMEM
There was not enough memory to perform the operation by a large actor. This error can leave the large actor in inconsistent state.
Use the following function to register a selected output signal.
This function registers sig as an output signal of actor emitted in an action choice state encoded by the buffer for an n-gram of signals from the event history.
The function records the following information for the action choice state:
Finally, the function shifts the buffer one signal left, appends sig to the end of that buffer, and increments by 1 discrete time tracked by the actor.
The function returns a non-negative value on success or a negative error code on failure. Currently, the function can return the following error codes.
QSMM_ERR_INVAL
Signal sig is not an output signal of the actor.
QSMM_ERR_NGRAM
The buffer does not encode a valid action choice state.
To determine the validity of an action choice state, the function checks the content of the buffer for accordance with allowed ranges of signal identifiers specified by the field range_sig_p
of qsmm_actor_desc_s
structure when creating the actor.
QSMM_ERR_STORAGE
Statistics storage failure. See Getting the Reason of a Storage Failure, for how to get an error message describing the failure.
QSMM_ERR_STATS
Inconsistent statistics on an action choice state or cycle type detected.
QSMM_ERR_ILSEQ
A statistics storage access function generated an error message but cannot convert it to a wide string according to a current locale, or a storage redirection function reported QSMM_ERR_ILSEQ
.
QSMM_ERR_NOMEM
A statistics storage access function reported out of memory error.
For a small actor, adaptively emit an output signal by a sequence of function calls like this:
int rc; if ((rc= qsmm_actor_calc_action_prob(actor, 0, NSIG_IN, NSIG_IN+NSIG_OUT, QSMM_PROB_LEARNED))<0) REPORT_ERROR(rc); qsmm_sig_t sig_out=QSMM_SIG_INVALID; if ((rc= qsmm_get_actor_sig_action(actor, 0, NSIG_IN, NSIG_IN+NSIG_OUT, &sig_out))<0) REPORT_ERROR(rc); if ((rc=qsmm_actor_reg_sig_action(actor,sig_out))<0) REPORT_ERROR(rc);
For a large actor, a call to qsmm_actor_calc_action_prob
is redundant and only decreases performance.
The sequence of function calls looks like this:
int rc; qsmm_sig_t sig_out=QSMM_SIG_INVALID; if ((rc= qsmm_get_actor_sig_action(actor, 0, NSIG_IN, NSIG_IN+NSIG_OUT, &sig_out))<0) REPORT_ERROR(rc); if ((rc=qsmm_actor_reg_sig_action(actor,sig_out))<0) REPORT_ERROR(rc);
For a small actor, you can override an output signal selected by the function qsmm_get_actor_sig_action
or omit calling that function.
That is, you can pass to the function qsmm_actor_reg_sig_action
an output signal not obtained by qsmm_get_actor_sig_action
.
This possibility does not exist for a large actor—it is incorrect to omit a call to qsmm_get_actor_sig_action
before calling qsmm_actor_reg_sig_action
or to pass to the latter function a signal different from a signal returned by a preceding call to qsmm_get_actor_sig_action
.
Note: you should not call the functions
qsmm_actor_spur_delta
(see Incrementing Spur) andqsmm_actor_time_delta
(see Incrementing Time) between a call toqsmm_actor_reg_sig_in
and a subsequent call toqsmm_actor_reg_sig_action
.
In QSMM 1.16 there appeared a function returning the probability of last output signal selected. You can use this function to implement timed emitting output signals, for example, according to the exponential probability distribution.
This function returns the probability of last output signal selected by the function qsmm_get_actor_sig_action
for an actor.
For a small actor, it is a probability corresponding to an element of internal array of relative probabilities of output signals.
For a large actor, it is the probability of a path from the root node of a Huffman tree to its leaf node corresponding to the output signal.
The returned value is always in the range 0 to 1 (inclusive).
If there was no call to qsmm_get_actor_sig_action
yet, qsmm_get_actor_prob_action
returns 0.
Previous: Receiving Input Signals, Up: Repeated Sequence of Operations [Contents][Index]