2.5.2 Emitting an Output Signal ¶

Emitting an output signal includes the following steps:

1. Calculate by the function qsmm_actor_calc_action_prob probabilities of output signals for a current action choice state. This step is only necessary for a small actor.
2. Select by the function qsmm_get_actor_sig_action an output signal to emit. For a small actor, that function uses output signal probabilities calculated in the previous step. For a large actor, that function uses an output signal choice tree for the current action choice state.
3. Register the output signal by the function qsmm_actor_reg_sig_out. That function records the following information for the current action choice state: current discrete time, current continuous time for each continuous time type, current spur value for each spur type, and the output signal emitted.

Types of probabilities of output signals are the following.

Enumeration: qsmm_prob_e ¶

This enumeration specifies a type of probabilities. The enumeration contains the following elements.

QSMM_PROB_AGGR

Aggregate probabilities based on learned and profile probabilities.

For a small actor, to calculate probabilities QSMM_PROB_AGGR, the function qsmm_actor_calc_action_prob calculates probabilities QSMM_PROB_LEARNED and QSMM_PROB_PROFILE (see below), multiplies each learned probability by a corresponding profile probability, and normalizes a resulting array. For a large actor, qsmm_actor_calc_action_prob called for an associated small actor performs this operation for the nodes of an output signal choice tree to obtain aggregate probabilities of tree leaves.

For a small actor, if you do not use profile probabilities, it is faster to calculate probabilities QSMM_PROB_LEARNED—they are equal to probabilities QSMM_PROB_AGGR in this case.

QSMM_PROB_LEARNED

For a small actor, learned probabilities for producing adaptive actions. For a large actor, probabilities QSMM_PROB_AGGR.

QSMM_PROB_PROFILE

For a small actor, profile probabilities specified a priori in statistics storage (see Structures for Accessing Storage, for how to pass profile probabilities to storage access functions to write them to storage). For a large actor, profile probabilities following from a subordination relationship of nodes of an output signal choice tree and from probabilities of edges connecting parent tree nodes with their child nodes.

QSMM_PROB_FQ

Probabilities proportional to observed frequencies of output signals registered by the function qsmm_actor_reg_sig_out. For a large actor, calculating probabilities of this type is faster compared to the other types.

QSMM_PROB_COUNT

The last element of this enumeration equal to the number of supported probability types.

Use the following function to calculate probabilities of output signals for a current action choice state.

Function: int qsmm_actor_calc_action_prob (qsmm_actor_t actor, int rez1, qsmm_sig_t sig_beg, qsmm_sig_t sig_end, enum qsmm_prob_e prob_type) ¶

This function fills an internal array of actor with probabilities prob_type calculated for an array of signal identifiers encoding a current action choice state. The function qsmm_get_actor_sig_ngram returns the pointer to a buffer containing the latter array. The argument rez1 is for future use and must be equal to 0.

For a small actor, the function qsmm_get_actor_sig_action stochastically selects an output signal according to probabilities in the internal array. The function qsmm_actor_calc_action_prob is much slower for a large actor compared to a small actor; you should call this function for a large actor only to obtain explicit probability values, as large actors support calling qsmm_get_actor_sig_action without prior calling qsmm_actor_calc_action_prob.

The arguments sig_beg and sig_end specify the identifiers of the first output signal (inclusive) and last output signal (exclusive) of a signal segment for calculating the probabilities. Offsets of output signal probabilities in the internal array are equal to output signal identifiers. If sig_end is 0, the function uses the total number of output signals for the identifier of last signal. The function operates faster when the signal segment is shorter, so specify the segment as precisely as possible.

The function returns a non-negative value on success or a negative error code on failure. Currently, the function can return the following error codes.

QSMM_ERR_INVAL

The value of sig_beg is greater than or equal to the identifier of last signal of the signal segment, or sig_end is too large, or prob_type is invalid.

QSMM_ERR_NGRAM

A buffer returnable by the function qsmm_get_actor_sig_ngram does not contain a valid array of signal identifiers encoding an action choice state. To determine array validity, the function qsmm_actor_calc_action_prob checks it for accordance with allowed ranges of signal identifiers specified by the field range_sig_p of qsmm_actor_desc_s structure when creating the actor.

QSMM_ERR_CALLBACK

A helper function QSMM_RELPROB_USER1 or QSMM_RELPROB_USER2 for calculating the relative probability of an output signal reported an error by returning NaN.

QSMM_ERR_STORAGE

Statistics storage failure. See Getting the Reason of a Storage Failure, for how to get an error message describing the failure.

QSMM_ERR_STATS

Inconsistent statistics on an action choice state or cycle type detected.

QSMM_ERR_ILSEQ

A generated error message is not convertible to a wide string according to a current locale, or a storage redirection function reported QSMM_ERR_ILSEQ.

QSMM_ERR_NOMEM

There was not enough memory to perform the operation.

Errors QSMM_ERR_STORAGE, QSMM_ERR_STATS, QSMM_ERR_ILSEQ, and QSMM_ERR_NOMEM can leave a large actor in inconsistent state. If a large actor is in inconsistent state, then, if after removing a reason of an error a repeated call to this function succeeds, the actor’s state becomes consistent.

Use the following function to obtain a pointer to the internal array containing probabilities of output signals calculated by the function qsmm_actor_calc_action_prob.

Function: double * qsmm_get_actor_choice_sig_prob (qsmm_actor_t actor) ¶

This function returns a pointer to internal array of actor holding relative probabilities of output signals. The function never returns NULL. The number of elements in the array is equal to the number of output signals returned by the function qsmm_get_actor_nsig_out. The function qsmm_actor_calc_action_prob fills the array with probabilities of a requested type.

For a small actor, the function qsmm_get_actor_sig_action stochastically selects an output signal according to probabilities in the array. A program can modify the array to affect this stochastic selection. The function qsmm_get_actor_sig_action handles the situation when the array contains elements equal to positive infinity or values that do not sum up to 1. Conversely, modifying the array does not affect the behavior of qsmm_get_actor_sig_action for a large actor.

The function qsmm_get_actor_choice_sig_prob locks the internal array: if a sparse vector backs up the internal array, this function converts the sparse vector to an ordinary vector. After this conversion, the actor keeps the internal array as an ordinary vector until the function qsmm_actor_choice_sig_prob_release releases the internal array. Forgetting to call qsmm_actor_choice_sig_prob_release after a call to qsmm_get_actor_choice_sig_prob may dramatically decrease actor performance.

Use the following function to release a locked internal array.

Function: void qsmm_actor_choice_sig_prob_release (qsmm_actor_t actor) ¶

This function releases the internal array of actor locked by the function qsmm_get_actor_choice_sig_prob. That internal array holds relative probabilities of output signals. If qsmm_get_actor_choice_sig_prob has not locked the internal array, qsmm_actor_choice_sig_prob_release is no-op. You must not use a pointer obtained by qsmm_get_actor_choice_sig_prob after releasing the array.

After locking the internal array by the function qsmm_get_actor_choice_sig_prob and subsequent calling the function qsmm_actor_calc_action_prob or when calling it for a large actor, qsmm_actor_calc_action_prob uses an ordinary vector for storing probabilities of output signals, and you can access its elements via a pointer returned by qsmm_get_actor_choice_sig_prob. When the internal array is in released state, qsmm_actor_calc_action_prob called for a small actor uses a sparse vector for storing probabilities of output signals, and you should use functions described in Ordinary and Sparse Vectors to read vector elements.

Regardless whether the vector is ordinary or sparse, you can obtain its handle and use functions described in Ordinary and Sparse Vectors to access vector elements in read-only mode. Obtaining the handle of an ordinary or sparse vector holding relative probabilities of output signals is a cheap operation that does not perform locking or converting a sparse vector to an ordinary vector.

Function: qsmm_vec_t qsmm_get_actor_choice_sig_prob_vec (qsmm_actor_t actor) ¶

This function returns the handle of an ordinary or sparse vector of actor holding relative probabilities of output signals. The function never returns NULL. The function qsmm_actor_calc_action_prob fills the vector with probabilities of a requested type. For a small actor, the function qsmm_get_actor_sig_action stochastically selects an output signal according to probabilities in the vector.

Use the following function to stochastically select an output signal.

Function: int qsmm_get_actor_sig_action (qsmm_actor_t actor, int rez1, qsmm_sig_t sig_beg, qsmm_sig_t sig_end, qsmm_sig_t *sig_out_p) ¶

If actor is a small one, this function stochastically selects an output signal according to relative probabilities stored in the internal array of relative probabilities of output signals. Offsets of output signal probabilities in the internal array are equal to output signal identifiers. The function qsmm_actor_calc_action_prob fills the internal array with probabilities of a requested type. The function qsmm_get_actor_choice_sig_prob returns a pointer to array content. The function qsmm_get_actor_choice_sig_prob_vec returns a read-only array view for speeding up accessing array content if it is sparse.

If actor is a large one, qsmm_get_actor_sig_action uses the output signal choice tree of a current action choice state to make a fast choice of an output signal according to probabilities QSMM_PROB_AGGR; for faster operation, you should omit a preceding call to qsmm_actor_calc_action_prob, as qsmm_get_actor_sig_action does not use the aforementioned internal array. The function qsmm_get_actor_sig_ngram returns the pointer to a buffer containing an array of signal identifiers encoding the current action choice state.

The function qsmm_get_actor_sig_action stores a selected output signal in *sig_out_p if sig_out_p is not NULL. The argument rez1 is for future use and must be equal to 0.

The arguments sig_beg and sig_end specify the identifiers of the first signal (inclusive) and last signal (exclusive) of a segment for selecting an output signal. For a small actor, specify the segment as precisely as possible, because the function operates faster when the signal segment is shorter. For a large actor, the signal segment must include all output signals of the actor; see Specifying Output Signal Weights, for how to adaptively select an output signal from various subsets of output signals of a large actor. If sig_end is 0, the function uses the total number of output signals for the identifier of last signal. For a small actor, if at least one relative probability in the internal array within the signal segment is positive infinity, the function selects an output signal as a uniformly distributed random element from a set of all signals (within the signal segment) with relative probabilities equal to positive infinity.

To stochastically select an output signal, the function uses either a random number generator provided via the field rng of qsmm_actor_desc_s structure when creating the actor by the function qsmm_actor_create or an instance of default random number generator allocated automatically if that field was NULL. Call the function qsmm_get_actor_rng to obtain the handle of a random number generator used by an actor, for example, to seed the generator.

The function returns a non-negative value on success or a negative error code on failure. Currently, the function can return the following error codes.

QSMM_ERR_INVAL

One of the following conditions is true:

• sig_beg is greater than or equal to the identifier of last signal of the signal segment;
• sig_end is too large;
• sig_beg is positive for a large actor;
• sig_end is positive and less than or equal to highest output signal identifier of a large actor.

QSMM_ERR_WEIGHT

For a small actor, the internal array contains (within the signal segment) a negative element or a non-finite element different from positive infinity.

QSMM_ERR_INFIN

For a small actor, the internal array does not contain elements (within the signal segment) equal to positive infinity, but the sum of array elements is positive infinity. For a large actor, the function qsmm_get_actor_sig_action or qsmm_actor_reg_ngram_in called for an associated small actor when traversing an output signal choice tree from its root node to a leaf node reported QSMM_ERR_INFIN. This error can leave a large actor in inconsistent state.

QSMM_ERR_NOCHOICE

For a small actor, the internal array does not contain at least one positive element (within the signal segment). For a large actor, weights of output signals of an associated small actor are unexpectedly zero, or an output signal choice tree does not represent a consistent probability profile.

QSMM_ERR_NGRAM

For a large actor, a buffer returnable by the function qsmm_get_actor_sig_ngram does not contain a valid array of signal identifiers encoding an action choice state. To determine array validity, the function qsmm_get_actor_sig_action checks it for accordance with allowed ranges of signal identifiers specified by the field range_sig_p of qsmm_actor_desc_s structure when creating the actor.

QSMM_ERR_CALLBACK

For a large actor, a helper function QSMM_RELPROB_USER1 or QSMM_RELPROB_USER2 for calculating the relative probability of an output signal reported an error by returning NaN. The function qsmm_actor_calc_action_prob called for an associated small actor when traversing an output signal choice tree from its root node to a leaf node calls the helper function. This error can leave a large actor in inconsistent state.

QSMM_ERR_STORAGE

A failure of statistics storage of a large actor. See Getting the Reason of a Storage Failure, for how to get an error message describing the failure. This error can leave a large actor in inconsistent state.

QSMM_ERR_STATS

Inconsistent statistics on an action choice state or cycle type detected for a large actor. This error can leave a large actor in inconsistent state.

QSMM_ERR_ILSEQ

For a large actor, a generated error message is not convertible to a wide string according to a current locale, or a storage redirection function reported QSMM_ERR_ILSEQ. This error can leave a large actor in inconsistent state.

QSMM_ERR_NOMEM

There was not enough memory to perform the operation by a large actor. This error can leave a large actor in inconsistent state.

Use the following function to register a selected output signal.

Function: int qsmm_actor_reg_sig_out (qsmm_actor_t actor, qsmm_sig_t sig) ¶

This function registers sig as an output signal of actor emitted in a current action choice state. The function qsmm_get_actor_sig_ngram returns the pointer to a buffer containing an array of signal identifiers encoding the current action choice state.

The function records the following information for the action choice state:

• discrete time of last occurrence of the action choice state in the event history equal to current discrete time tracked by the actor;
• for every continuous time type, continuous time of last occurrence of the action choice state in the event history equal to current continuous time tracked by the actor;
• current spur value for every spur type;
• the output signal emitted in the action choice state.

The function returns a non-negative value on success or a negative error code on failure. Currently, the function can return the following error codes.

QSMM_ERR_INVAL

Signal sig is greater than or equal to the number of output signals of the actor.

QSMM_ERR_NGRAM

The buffer does not contain a valid array of signal identifiers encoding an action choice state. To determine array validity, the function checks it for accordance with allowed ranges of signal identifiers specified by the field range_sig_p of qsmm_actor_desc_s structure when creating the actor.

QSMM_ERR_STORAGE

Statistics storage failure. See Getting the Reason of a Storage Failure, for how to get an error message describing the failure.

QSMM_ERR_STATS

Inconsistent statistics on an action choice state or cycle type detected.

QSMM_ERR_ILSEQ

A statistics storage access function generated an error message but could not convert it to a wide string according to a current locale, or a storage redirection function reported QSMM_ERR_ILSEQ.

QSMM_ERR_NOMEM

A statistics storage access function reported out of memory error.

For a small actor, use the following sequence of function calls to adaptively emit an output signal:

int rc;
if ((rc=
     qsmm_actor_calc_action_prob(actor,0,0,NSIG_OUT,QSMM_PROB_LEARNED))<0)
    REPORT_ERROR(rc);
qsmm_sig_t sig_out=QSMM_SIG_INVALID;
if ((rc=qsmm_get_actor_sig_action(actor,0,0,NSIG_OUT,&sig_out))<0)
    REPORT_ERROR(rc);
if ((rc=qsmm_actor_reg_sig_out(actor,sig_out))<0) REPORT_ERROR(rc);

For a large actor, a call to qsmm_actor_calc_action_prob is redundant and decreases performance. The sequence of function calls looks like this:

int rc;
qsmm_sig_t sig_out=QSMM_SIG_INVALID;
if ((rc=qsmm_get_actor_sig_action(actor,0,0,NSIG_OUT,&sig_out))<0)
    REPORT_ERROR(rc);
if ((rc=qsmm_actor_reg_sig_out(actor,sig_out))<0) REPORT_ERROR(rc);

For a small actor, you can override an output signal selected by the function qsmm_get_actor_sig_action or omit calling that function. That is, you can pass to the function qsmm_actor_reg_sig_out an output signal not obtained by qsmm_get_actor_sig_action. This is not possible for a large actor—it is incorrect to omit a call to qsmm_get_actor_sig_action before calling qsmm_actor_reg_sig_out or to pass to the latter function a signal different from a signal returned by a preceding call to qsmm_get_actor_sig_action.

Note: you should not call the functions qsmm_actor_spur_delta (see Incrementing Spur) and qsmm_actor_time_delta (see Incrementing Time) between a call to qsmm_actor_reg_sig_in and a subsequent call to qsmm_actor_reg_sig_action.

In QSMM 1.16 there appeared a function returning the probability of last output signal selected. You can use this function to implement timed emitting output signals, for example, according to the exponential probability distribution.

Function: double qsmm_get_actor_prob_action (qsmm_actor_t actor) ¶

This function returns the probability of last output signal selected by the function qsmm_get_actor_sig_action for an actor. For a small actor, it is a probability corresponding to an element in the internal array with relative probabilities of output signals. For a large actor, it is the probability of a path in an output signal choice tree from its root node to a leaf node for the output signal. A returned value is always in the range 0 to 1 (inclusive). If there was no call to qsmm_get_actor_sig_action yet, qsmm_get_actor_prob_action returns 0.