Previous: Receiving Input Signals, Up: Repeated Sequence of Operations [Contents][Index]

Emitting an output signal includes the following steps:

- Calculate by the function
`qsmm_actor_calc_action_prob`

the probabilities of output signals for a current action choice state. This step is only necessary for a small actor. - Select by the function
`qsmm_get_actor_sig_action`

an output signal to emit. For a small actor, that function uses output signal probabilities calculated in the previous step. For a large actor, that function uses a Huffman tree corresponding to the current action choice state. - Append by the function
`qsmm_actor_reg_sig_action`

the output signal to the buffer containing an n-gram of signals from the event history (at this point—containing the current action choice state). That function records the following information for the current action choice state: current discrete time, current continuous time, current spur values, and the output signal emitted.

In order to correctly calculate the probabilities of output signals, the actor should have a correct parameter specifying a relation between discrete time and the number of output signals emitted. Use the following functions to query or set the value of this parameter.

- Function:
*double***qsmm_get_actor_naction_per_evt***(qsmm_actor_t*`actor`) This function returns a number greater than 0 and less than or equal to 1 specifying the mean number of output signals emitted by an

`actor`per one unit of discrete time passed.

- Function:
*int***qsmm_set_actor_naction_per_evt***(qsmm_actor_t*`actor`, double`val`) This function sets to

`val`the mean number of output signals emitted by an`actor`per one unit of discrete time passed.On success, the function returns a non-negative value. If

`val`is not a finite number or is a number less than or equal to 0 or is a number greater than 1, the function returns negative error code`QSMM_ERR_INVAL`

.

The function `qsmm_actor_create`

initializes the mean number of output signals emitted per one unit of discrete time passed to 0.5.

Let us suppose that the field `ngram_sz`

of `qsmm_actor_desc_s`

structure passed to the function `qsmm_actor_create`

when creating an actor was equal to `NGRAM_SZ`

.
The general rules of setting the parameter specifying the mean number of output signals emitted per one unit of discrete time passed are the following:

- If you use the functions
`qsmm_actor_shl_sig`

and`qsmm_actor_reg_sig_in`

to set a current action choice state, and it solely consists of input signals of the actor, then set this parameter to`1.0/(NGRAM_SZ+1)`

. - If you use the functions
`qsmm_actor_shl_sig`

and`qsmm_actor_reg_sig_in`

to set a current action choice state, and it consists of last output signal emitted and input signals, then set this parameter to`1.0/NGRAM_SZ`

. - If you use the functions
`qsmm_get_actor_sig_ngram`

and`qsmm_actor_reg_sig_in`

to set a current action choice state, then use default parameter value 0.5. - If action choice states are n-grams of signals from the event history, and input and output signals interleave, then use default parameter value 0.5.

The types of probabilities of output signals are the following.

- Enumeration:
**qsmm_prob_e** This enumeration specifies a type of probabilities. The enumeration contains the following elements.

`QSMM_PROB_AGGR`

Aggregate probabilities based on learned and profile probabilities.

For a small actor, to calculate probabilities of

`QSMM_PROB_AGGR`

type, the function`qsmm_actor_calc_action_prob`

calculates probabilities of`QSMM_PROB_LEARNED`

and`QSMM_PROB_PROFILE`

types (see below), multiplies each learned probability by a corresponding profile probability, and normalizes a resulting array. For a large actor, the function`qsmm_actor_calc_action_prob`

called for an associated small actor performs this operation for the nodes of a Huffman tree to obtain the aggregate probabilities of tree leaves.For a small actor, if you do not use profile probabilities, it is faster to calculate probabilities of

`QSMM_PROB_LEARNED`

type—they are equal to probabilities of`QSMM_PROB_AGGR`

type in this case.`QSMM_PROB_LEARNED`

For a small actor, learned probabilities for producing adaptive actions. For a large actor, probabilities of

`QSMM_PROB_AGGR`

type.`QSMM_PROB_PROFILE`

For a small actor, profile probabilities specified a priori in statistics storage (see Structures for Accessing Storage, for how to pass profile probabilities to storage access functions to write them to storage). For a large actor, profile probabilities following from a subordination relationship of nodes of Huffman tree of an action choice state and from the probabilities of edges connecting parent tree nodes with their child nodes.

`QSMM_PROB_FQ`

Probabilities proportional to observed frequencies of output signals registered by the function

`qsmm_actor_reg_sig_action`

. For a large actor, calculating probabilities of this type is faster compared to the other types.`QSMM_PROB_COUNT`

The last element of this enumeration equal to the number of supported probability types.

Use the following function to calculate the probabilities of output signals for a current action choice state.

- Function:
*int***qsmm_actor_calc_action_prob***(qsmm_actor_t*`actor`, int`rez1`, qsmm_sig_t`sig_beg`, qsmm_sig_t`sig_end`, enum qsmm_prob_e`prob_type`) This function fills the internal array of

`actor`with probabilities of`prob_type`type calculated for an action choice state encoded by the buffer containing an n-gram of signals from the event history. The functions`qsmm_actor_shl_sig`

,`qsmm_actor_shr_sig`

,`qsmm_actor_reg_sig_in`

, and`qsmm_actor_reg_sig_action`

access that buffer, and the function`qsmm_get_actor_sig_ngram`

returns a pointer to the buffer. The argument`rez1`is for future use and must be equal to 0.For a small actor, the function

`qsmm_get_actor_sig_action`

stochastically selects an output signal according to probabilities in that internal array. The function`qsmm_actor_calc_action_prob`

is much slower for a large actor compared to a small actor; you should call this function for a large actor only to obtain explicit probability values—large actors support calling`qsmm_get_actor_sig_action`

without prior calling`qsmm_actor_calc_action_prob`

.The arguments

`sig_beg`and`sig_end`can be both zero or may specify the identifiers of the first signal (inclusive) and last signal (exclusive) of a signal segment for calculating the probabilities. The probability of the first signal of this signal segment has offset`sig_beg`in the internal array. The function zeroes array elements that do not correspond to output signals. If`sig_end`is 0, the function uses the total number of actor signals for the identifier of last signal. The function operates faster when the signal segment is shorter, so specify the segment as precisely as possible.The function returns a non-negative value on success or a negative error code on failure. Currently, the function can return the following error codes.

`QSMM_ERR_INVAL`

The value of

`sig_beg`is greater than or equal to the identifier of last signal of the signal segment, or the value of`sig_end`is greater than the total number of signals of the actor, or`prob_type`is invalid.`QSMM_ERR_NGRAM`

The buffer does not encode a valid action choice state. To determine the validity of an action choice state, the function checks the content of the buffer for accordance with allowed ranges of signal identifiers specified by the field

`range_sig_p`

of`qsmm_actor_desc_s`

structure when creating the actor.`QSMM_ERR_CALLBACK`

A helper function of

`QSMM_RELPROB_USER1`

or`QSMM_RELPROB_USER2`

type for calculating the relative probability of an output signal reported an error by returning`NaN`

.`QSMM_ERR_STORAGE`

Statistics storage failure. See Getting the Reason of a Storage Failure, for how to get an error message describing the failure.

`QSMM_ERR_STATS`

Inconsistent statistics on an action choice state or cycle type detected.

`QSMM_ERR_ILSEQ`

A generated error message is not convertible to a wide string according to a current locale, or a storage redirection function reported

`QSMM_ERR_ILSEQ`

.`QSMM_ERR_NOMEM`

There was not enough memory to perform the operation.

Errors

`QSMM_ERR_STORAGE`

,`QSMM_ERR_STATS`

,`QSMM_ERR_ILSEQ`

, and`QSMM_ERR_NOMEM`

can leave a large actor in inconsistent state. If the large actor is in inconsistent state, then, if after removing a reason of an error a repeated call to this function succeeds, the actor’s state becomes consistent.

Use the following function to obtain a pointer to the internal array with the probabilities of output signals calculated by the function `qsmm_actor_calc_action_prob`

.

- Function:
*double ****qsmm_get_actor_choice_sig_prob***(qsmm_actor_t*`actor`) This function returns a pointer to the internal array of

`actor`holding the relative probabilities of output signals. This function never returns`NULL`

. The number of elements in the array is equal to the total number of actor signals returned by the function`qsmm_get_actor_nsig`

. The function`qsmm_actor_calc_action_prob`

fills the array with probabilities of a desired type.For a small actor, the function

`qsmm_get_actor_sig_action`

stochastically selects an output signal according to probabilities in that array. A program can modify the array to affect this stochastic selection. The function`qsmm_get_actor_sig_action`

handles the situation when the array contains elements equal to positive infinity or values that do not sum up to 1. Conversely, modifying the array does not affect the behavior of`qsmm_get_actor_sig_action`

for a large actor.The function

`qsmm_get_actor_choice_sig_prob`

locks the internal array: if a sparse vector backs up the internal array, this function converts the sparse vector to an ordinary vector. After this conversion, the actor keeps the internal array as an ordinary vector until the function`qsmm_actor_choice_sig_prob_release`

releases that internal array. Forgetting to call`qsmm_actor_choice_sig_prob_release`

after a call to`qsmm_get_actor_choice_sig_prob`

may dramatically decrease actor performance.

Use the following function to release a locked internal array.

- Function:
*void***qsmm_actor_choice_sig_prob_release***(qsmm_actor_t*`actor`) This function releases the internal array of

`actor`locked by the function`qsmm_get_actor_choice_sig_prob`

. That internal array holds the relative probabilities of output signals. If`qsmm_get_actor_choice_sig_prob`

has not locked the internal array,`qsmm_actor_choice_sig_prob_release`

has no effect. You must not use a pointer obtained by`qsmm_get_actor_choice_sig_prob`

after releasing the array.

When the function `qsmm_get_actor_choice_sig_prob`

has locked the internal array, and you call the function `qsmm_actor_calc_action_prob`

, or when you call it for a large actor, `qsmm_actor_calc_action_prob`

uses an ordinary vector for storing the probabilities of output signals, so you can access the elements of this ordinary vector via a pointer returned by `qsmm_get_actor_choice_sig_prob`

.
When the internal array is in released state, `qsmm_actor_calc_action_prob`

called for a small actor can use an ordinary or sparse vector for storing the probabilities of output signals depending on the properties of a probability profile and the value of `sparse_fill_max`

field of `qsmm_actor_desc_s`

structure specified when creating the actor.

You can obtain the handle of an ordinary or sparse vector holding the relative probabilities of output signals. It is a cheap operation that does not perform locking or converting a sparse vector to an ordinary vector. You can access a returned vector in read-only mode. See Ordinary and Sparse Vectors, for how to access vector elements.

- Function:
*qsmm_vec_t***qsmm_get_actor_choice_sig_prob_vec***(qsmm_actor_t*`actor`) This function returns the handle of an ordinary or sparse vector of

`actor`holding the relative probabilities of output signals. This function never returns`NULL`

. The function`qsmm_actor_calc_action_prob`

fills the vector with probabilities of a desired type. For a small actor, the function`qsmm_get_actor_sig_action`

stochastically selects an output signal according to probabilities in that vector.

Use the following function to stochastically select an output signal.

- Function:
*int***qsmm_get_actor_sig_action***(qsmm_actor_t*`actor`, int`rez1`, qsmm_sig_t`sig_beg`, qsmm_sig_t`sig_end`, qsmm_sig_t *`sig_out_p`) If

`actor`is a small one, this function stochastically selects an output signal according to relative probabilities stored in the internal array of relative probabilities of output signals. The function`qsmm_actor_calc_action_prob`

fills the internal array with probabilities of a desired type. The function`qsmm_get_actor_choice_sig_prob`

returns a pointer to the elements of that array. The function`qsmm_get_actor_choice_sig_prob_vec`

returns a read-only view of that array for speeding up accessing its content.If

`actor`is a large one, the function`qsmm_get_actor_sig_action`

selects an output signal for an action choice state encoded by the buffer containing an n-gram of signals from the event history without using the aforementioned internal array. The functions`qsmm_actor_shl_sig`

,`qsmm_actor_shr_sig`

,`qsmm_actor_reg_sig_in`

, and`qsmm_actor_reg_sig_action`

access that buffer, and the function`qsmm_get_actor_sig_ngram`

returns a pointer to the buffer. The function`qsmm_get_actor_sig_action`

uses the Huffman tree of this action choice state to perform fast choice of an output signal (the number of those signals can be very large) according to probabilities of`QSMM_PROB_AGGR`

type. The function`qsmm_actor_calc_action_prob`

can compute those probabilities; however, for a large actor, there is no need to call that function before calling`qsmm_get_actor_sig_action`

. You should minimize the number of calls to the function`qsmm_actor_calc_action_prob`

to calculate probabilities of`QSMM_PROB_AGGR`

type for a large actor, because that function operates slowly in this mode.The function

`qsmm_get_actor_sig_action`

stores a selected output signal in *`sig_out_p`if`sig_out_p`is not`NULL`

. The argument`rez1`is for future use and must be equal to 0.The arguments

`sig_beg`and`sig_end`can be both zero or may specify the identifiers of the first signal (inclusive) and last signal (exclusive) of a segment for selecting an output signal. However, for a large actor, the signal segment must include all output signals of the actor; see Specifying Output Signal Weights, for how to adaptively select an output signal from various subsets of output signals of a large actor. For a small actor, the probability of the first signal of this signal segment has offset`sig_beg`in the internal array. If`sig_end`is 0, the function uses the total number of actor signals for the identifier of last signal. The function operates faster when the signal segment is shorter, so specify the segment as precisely as possible. For a small actor, the function does not check whether a signal it selects actually belongs to a set of output signals—if an internal array element within the signal segment is positive, then the function may select a corresponding signal. For a small actor, if at least one relative probability in the internal array within the signal segment is positive infinity, the function selects an output signal as a uniformly distributed random element from a set of all signals (within the signal segment) with relative probabilities equal to positive infinity.To stochastically select an output signal, the function uses either a random number generator provided via the field

`rng`

of`qsmm_actor_desc_s`

structure when creating the actor by the function`qsmm_actor_create`

or an instance of default random number generator allocated automatically if that field was`NULL`

. Call the function`qsmm_get_actor_rng`

to obtain the handle of a random number generator used by an actor, for example, to seed the generator.The function returns a non-negative value on success or a negative error code on failure. Currently, the function can return the following error codes.

`QSMM_ERR_INVAL`

One of the following conditions is true:

- –
`sig_beg`is greater than or equal to the identifier of last signal of the signal segment; - –
`sig_end`is greater than the total number of signals of the actor; - –
`sig_beg`is greater than the lowest output signal identifier of a large actor; - –
`sig_end`is less than or equal to the highest output signal identifier of a large actor.

- –
`QSMM_ERR_WEIGHT`

For a small actor, the internal array contains (within the signal segment) a negative element or a non-finite element different from positive infinity.

`QSMM_ERR_INFIN`

For a small actor, the internal array does not contain elements (within the signal segment) equal to positive infinity, but the sum of array elements is positive infinity. For a large actor, the function

`qsmm_get_actor_sig_action`

or`qsmm_actor_reg_sig_in`

called for an associated small actor when traversing a Huffman tree from the root node to a leaf node reported`QSMM_ERR_INFIN`

. This error can leave the large actor in inconsistent state.`QSMM_ERR_NOCHOICE`

For a small actor, the internal array does not contain at least one positive element (within the signal segment).

`QSMM_ERR_NGRAM`

For a large actor, the buffer does not encode a valid action choice state. To determine the validity of an action choice state, the function checks the content of the buffer for accordance with allowed ranges of signal identifiers specified by the field

`range_sig_p`

of`qsmm_actor_desc_s`

structure when creating the actor.`QSMM_ERR_CALLBACK`

For a large actor, a helper function of

`QSMM_RELPROB_USER1`

or`QSMM_RELPROB_USER2`

type for calculating the relative probability of an output signal reported an error by returning`NaN`

. The function`qsmm_actor_calc_action_prob`

called for an associated small actor when traversing a Huffman tree from the root node to a leaf node calls this helper function. This error can leave the large actor in inconsistent state.`QSMM_ERR_STORAGE`

Failure of statistics storage of a large actor. See Getting the Reason of a Storage Failure, for how to get an error message describing the failure. This error can leave the large actor in inconsistent state.

`QSMM_ERR_STATS`

Inconsistent statistics on an action choice state or cycle type detected for a large actor. This error can leave the large actor in inconsistent state.

`QSMM_ERR_ILSEQ`

For a large actor, a generated error message is not convertible to a wide string according to a current locale, or a storage redirection function reported

`QSMM_ERR_ILSEQ`

. This error can leave the large actor in inconsistent state.`QSMM_ERR_NOMEM`

There was not enough memory to perform the operation by a large actor. This error can leave the large actor in inconsistent state.

Use the following function to register a selected output signal.

- Function:
*int***qsmm_actor_reg_sig_action***(qsmm_actor_t*`actor`, qsmm_sig_t`sig`) This function registers

`sig`as an output signal of`actor`emitted in an action choice state encoded by the buffer for an n-gram of signals from the event history.The function records the following information for the action choice state:

- – discrete time of last occurrence of that action choice state in the event history equal to current discrete time tracked by the actor;
- – continuous time of last occurrence of that action choice state in the event history equal to current continuous time tracked by the actor;
- – current spur value for every spur type;
- – the output signal emitted in the action choice state.

Finally, the function shifts the buffer one signal left, appends

`sig`to the end of that buffer, and increments by 1 discrete time tracked by the actor.The function returns a non-negative value on success or a negative error code on failure. Currently, the function can return the following error codes.

`QSMM_ERR_INVAL`

Signal

`sig`is not an output signal of the actor.`QSMM_ERR_NGRAM`

The buffer does not encode a valid action choice state. To determine the validity of an action choice state, the function checks the content of the buffer for accordance with allowed ranges of signal identifiers specified by the field

`range_sig_p`

of`qsmm_actor_desc_s`

structure when creating the actor.`QSMM_ERR_STORAGE`

Statistics storage failure. See Getting the Reason of a Storage Failure, for how to get an error message describing the failure.

`QSMM_ERR_STATS`

Inconsistent statistics on an action choice state or cycle type detected.

`QSMM_ERR_ILSEQ`

A statistics storage access function generated an error message but cannot convert it to a wide string according to a current locale, or a storage redirection function reported

`QSMM_ERR_ILSEQ`

.`QSMM_ERR_NOMEM`

A statistics storage access function reported out of memory error.

For a small actor, adaptively emit an output signal by a sequence of function calls like this:

int rc; if ((rc= qsmm_actor_calc_action_prob(actor, 0, NSIG_IN, NSIG_IN+NSIG_OUT, QSMM_PROB_LEARNED))<0) REPORT_ERROR(rc); qsmm_sig_t sig_out=QSMM_SIG_INVALID; if ((rc= qsmm_get_actor_sig_action(actor, 0, NSIG_IN, NSIG_IN+NSIG_OUT, &sig_out))<0) REPORT_ERROR(rc); if ((rc=qsmm_actor_reg_sig_action(actor,sig_out))<0) REPORT_ERROR(rc);

For a large actor, a call to `qsmm_actor_calc_action_prob`

is redundant and only decreases performance.
The sequence of function calls looks like this:

int rc; qsmm_sig_t sig_out=QSMM_SIG_INVALID; if ((rc= qsmm_get_actor_sig_action(actor, 0, NSIG_IN, NSIG_IN+NSIG_OUT, &sig_out))<0) REPORT_ERROR(rc); if ((rc=qsmm_actor_reg_sig_action(actor,sig_out))<0) REPORT_ERROR(rc);

For a small actor, you can override an output signal selected by the function `qsmm_get_actor_sig_action`

or omit calling that function.
That is, you can pass to the function `qsmm_actor_reg_sig_action`

an output signal not obtained by `qsmm_get_actor_sig_action`

.
This possibility does not exist for a large actor—it is incorrect to omit a call to `qsmm_get_actor_sig_action`

before calling `qsmm_actor_reg_sig_action`

or to pass to the latter function a signal different from a signal returned by a preceding call to `qsmm_get_actor_sig_action`

.

Note:you should not call the functions`qsmm_actor_spur_delta`

(see Incrementing Spur) and`qsmm_actor_time_delta`

(see Incrementing Time) between a call to`qsmm_actor_reg_sig_in`

and a subsequent call to`qsmm_actor_reg_sig_action`

.

In QSMM 1.16 there appeared a function returning the probability of last output signal selected. You can use this function to implement timed emitting output signals, for example, according to the exponential probability distribution.

- Function:
*double***qsmm_get_actor_prob_action***(qsmm_actor_t*`actor`) This function returns the probability of last output signal selected by the function

`qsmm_get_actor_sig_action`

for an`actor`. For a small actor, it is a probability corresponding to an element of internal array of relative probabilities of output signals. For a large actor, it is the probability of a path from the root node of a Huffman tree to its leaf node corresponding to the output signal. The returned value is always in the range 0 to 1 (inclusive). If there was no call to`qsmm_get_actor_sig_action`

yet,`qsmm_get_actor_prob_action`

returns 0.

Previous: Receiving Input Signals, Up: Repeated Sequence of Operations [Contents][Index]