QSMM Programmer Manual ¶

1.1 What Is Intelligence? ¶

There are many definitions for intelligence as well as types of intelligence. Most definitions include lists of activities inherent to humans, such as reasoning, planning, solving problems, thinking abstractly, comprehending complex ideas, learning from experience, and adapting effectively to the environment. When attempting to develop intelligent machines, researchers try to make them perform a subset of activities inherent to humans with as high a level of plausibility as possible. However, from the standpoint of knowledge formalization and for a more precise formulation of research goals, it would be useful to point out the smallest subset of the activities to establish that an animate being or inanimate machine performing them is intelligent enough.

First, it is necessary to mention that intelligence implies a goal (in the “Handbook of Human Intelligence” Sternberg and Salter defined intelligence as “goal-directed adaptive behavior”). In the case of animate being, the goal could be surviving, and in the case of a machine created by humans, the goal could be solving tasks assigned by humans. Animals show their basic intelligence, for example, when they adapt to the environment, seek for food, and build nests. One can say today’s machines show their intelligence by effectively performing tasks people created the machines to perform. This is not a too distorted interpretation of intelligence if we review the book “Symbolic Logic and Intelligent Machines” of Edmund C. Berkeley published in 1959, where the author called intelligent machines electromechanical arrangements capable of solving problems that involved logic.

As animals are capable of satisfying their basic needs for living, let us assume they possess a basic level of intelligence. To better satisfy needs for living, an advanced level of intelligence would be the production and use of work tools. As it turned out, the production and use of work tools is specific not only to humans but also to chimpanzees and to a number of other animals. For example, chimpanzees are able to find stones of appropriate weights and sizes and crack nuts using them. Chimpanzees are also able to find long and thin sticks and use them to kill small rodents living in trunks of a certain species of trees and extract killed rodents for food. It is important to mention that chimpanzees not only find ready-to-use work tools in nature but can also produce them. For example, chimpanzees can construct arrangements consisting of small tree branches and put them into termite mounds to eat termites crawled on them. Another example is an experiment conducted on a pygmy chimpanzee when experimenters successfully trained him to manufacture a sharp stone tool for cutting a string to open the door of a chamber with sweets.

In a digital world supported by computers, work tools are computer programs helping solve various tasks. In a computer environment, the aforementioned advanced level of intelligence is the capability to synthesize and use computer programs and algorithms. A machine with an advanced level of intelligence should be capable of automated synthesis of algorithms in some form or fashion. Such an algorithm might have a form of an ordinary computer program, possibly containing a set of subroutines.

There does exist a higher level of intelligence—we shall call it an expert level of intelligence,—specific only to humans and not to chimpanzees. Indeed, what is an essential difference between humans’ and chimpanzees’ intelligence if they both produce and use work tools?

In the book “The Complete Idiot’s Guide to Human Prehistory,” its author Robert J. Meier briefly noted an important fact about the production and use of burins¹. Such specially sharpened stones archaeologists find during excavations corresponding to the time when, tentatively speaking, we can call creatures that have been living on Earth humans. Burins are tools created for manufacturing other tools. Chimpanzees do not make tools for making other tools. That is, an expert level of intelligence peculiar to humans is the ability to organize processing chains where one kind of work tools takes part in the production of other kinds of work tools. The established concept for this is production of the means of production.

For the computer environment, programmers develop such tools as compilers, various operating, execution, and development environments to simplify developing other computer programs. A machine with an expert level of intelligence would be capable of the automated synthesis of algorithms that the machine would execute to synthesize other algorithms to solve problems assigned by humans more efficiently. The creation of such a machine is a quite specific goal researchers could try to achieve.

1.2 Spur-Driven Behavior ¶

A basic problem one needs to solve when creating an intelligent machine that performs automated synthesis of algorithms is discovering a way or manner in which a job is assigned to the machine. There are sophisticated approaches to solving the problem, for example, by creating specifications consisting of rules and constraints. A simpler approach is reducing an algorithm synthesis task to an optimization task. In this approach, a task resolution assessment unit evaluates a result of execution of a version of a synthesized algorithm; the machine makes changes to the version to hopefully produce a version with a better assessment score. A classical approach to solving optimization tasks of this kind is genetic algorithms. A genetic algorithm would evolve a resulting algorithm by trial and error in multiple iterations to maximize the value of a fitness function.

Another approach to solving an algorithm synthesis task as an optimization task is an approach used in QSMM where the machine synthesizes an algorithm simultaneously with its execution. The algorithm being executed can interact with an environment. The task resolution assessment unit provides continuous feedback on the results of execution of the algorithm, and the machine continuously attempts to improve it. A behavior exhibited by the machine is reinforcement learning. QSMM provides means for representing a synthesized algorithm as a finite automaton.

Historically, a numerical quantity specifying a component of an algorithm fitness value is called spur in QSMM; the type of the component is called spur type. QSMM supports multiple spur types, although it is better to use a smaller number of spur types for a more efficient optimization. For example, a spur value can be the logarithm of a probability to maximize, an energy value to minimize, or the sum of rewards or incentives given to a machine when a partially synthesized algorithm reaches some points in solving a task. For time-dependent feedback on changes to fitness value of an algorithm being synthesized, QSMM supports the optimization goal “maximize spur increment velocities”.

QSMM since version 1.17 proposes a higher-level approach to represent a synthesized algorithm as a PCFG (Probabilistic Context-Free Grammar). As a direct consequence of this, the main component of an algorithm fitness value becomes the probability of a PCFG.

In general, a programmer should provide a spur evaluation method that allows a machine to better understand a correlation between changes made to an algorithm being synthesized and observed changes to its behavior. Developing a proper spur evaluation method can be one of the most complex tasks to solve when creating an intelligent machine using QSMM.

1.3 Building Blocks for Intelligent Machines ¶

Consider an artificial neuron with one input and many outputs. Activation of the input leads to activation of one of the outputs where the choice of an activated output is probabilistic. A set of such neurons is a probabilistic mapping—a function that ambiguously maps an argument from a set of possible arguments to a result from a set of possible outcomes. Such function, when invoked one time, can return one result, and when invoked another time, can return yet another result for the same argument.

If every argument of a probabilistic mapping has a corresponding set of possible outcomes with fixed probabilities, we call the probabilistic mapping a fixed probabilistic mapping.

An important concept related to the possibility of using a result of a probabilistic mapping (directly or after transforming) as an argument (or as its part) of the probabilistic mapping is state—a variable that changes its value based on its previous value.

A fixed probabilistic mapping can model a probabilistic finite automaton. An argument of this probabilistic mapping is a superposition of the previous automaton state and an input signal received. A result of the probabilistic mapping is the next automaton state (or a superposition of the next automaton state and an output signal emitted).

To establish a uniform state transition model, it is tempting to treat a set of possible states of a probabilistic mapping as a Cartesian product of sets of possible sub-states making up a full state. In this sometimes useful approach, a resulting set of states can be very large. As sub-states from different sets usually interrelate with each other, the actual number of possible states is often much smaller. They resemble perceived states—mental model states a researcher realizes when learning or programming the behavior of an entity. When using QSMM, we will tend to work with such perceived states.

A probabilistic finite automaton can have a representation in the form of a probabilistic program. In particular, such probabilistic program may be a probabilistic assembler program—an assembler program containing probabilistic jump instructions. If there are no probabilistic jump instructions in an assembler program, the finite automaton and the assembler program are deterministic.

The instruction set of a probabilistic assembler program may consist of:

1. Custom instructions for performing effective work. Every such instruction can return an outcome from a set of possible outcomes.
2. The probabilistic jump instruction. It transfers control to a custom instruction at a specific location in the program with a given probability.
3. The conditional jump instruction. It transfers control to a custom or probabilistic jump instruction at a specific location in the program on the basis of an outcome returned by a previously invoked custom instruction.
4. The simple jump instruction. It transfers control to a custom instruction at a specific location in the program unconditionally.

Locations of custom instructions in the assembler program represent the states of the probabilistic finite automaton. An argument of a fixed probabilistic mapping backing up the finite automaton is a superposition of a location of a custom instruction invoked and an outcome returned by the custom instruction. An outcome of the fixed probabilistic mapping is the location of the next custom instruction to invoke. Simple jump instructions, conditional jump instructions, and probabilistic jump instructions in the assembler program map a particular argument of the probabilistic mapping to a set of possible outcomes with fixed probabilities.

A program usually contains a set of subroutines. A probabilistic assembler program may contain custom instructions for calling subroutines and custom instructions for returning control back. A probabilistic finite automaton may represent a subroutine. Calling a subroutine means pushing a reference to a current probabilistic finite automaton and its current state to a stack and transferring control to the initial state of another probabilistic finite automaton. Returning control from the subroutine means popping the reference to the current automaton and its current state from the stack and transferring control to that state.

We can implement modulating the behavior of a probabilistic mapping by spur. Supposing a result returned by a probabilistic mapping for its specific argument somehow affects the spur, a desired behavior of the probabilistic mapping for this argument would be returning more often an outcome that leads to a greater absolute spur value (a positive value if the goal is to maximize the spur, or a negative value if the goal is to minimize the spur) or a greater spur increment or decrement velocity. We call a probabilistic mapping with this behavior an adaptive probabilistic mapping. An adaptive probabilistic mapping helps produce goal-directed adaptive behavior: return more often outcomes that bring a machine closer to a goal where the spur measures a proximity to the goal.

In QSMM, actor is an adaptive probabilistic mapping working similarly to a fixed probabilistic mapping, but instead of fixed outcome probabilities, it uses probabilities adjusted adaptively. An actor is a generic block for building intelligent machines.

If a fixed probabilistic mapping that backs up a probabilistic finite automaton becomes an adaptive probabilistic mapping, the automaton becomes an adaptive probabilistic finite automaton. Consequently, a probabilistic assembler program the automaton represents becomes an adaptive probabilistic assembler program.

Using QSMM, a researcher can develop adaptive probabilistic assembler programs producing goal-directed behavior. They are a basis for automated synthesis of algorithms.

An assembler program represents a state model, and a subroutine of an assembler program represents a state sub-model. In QSMM, node means a callable state sub-model. If a state model contains multiple nodes, we call it multinode model. A node is a probabilistic finite automaton lying behind a probabilistic assembler program. Node execution is the operation of the probabilistic finite automaton where it exhibits adaptive behavior.

1.4 Animate Machines ¶

Can an intelligent machine be conscious? This section explains a point of view where an adaptive probabilistic mapping within an intelligent machine can induce a rudiment of consciousness.

How to implement a probabilistic mapping? For its particular argument, a machine could generate probabilities of all possible outcomes. Then, the machine would randomly select a particular outcome according to the probabilities, for example, using a random number generator.

How can the machine perform this selection? Suppose, the random number generator can return two numbers, 0 or 1, with equal probabilities. A straightforward approach to utilizing that random number generator is performing a fixed number of calls to the generator to obtain a binary fractional number in the range 0 to 1. Non-overlapping segments on a line of length 1 can represent all possible outcomes, where the length of every segment is equal to the probability of a corresponding outcome. To determine an outcome, the machine finds a segment containing a point on the line at a distance equal to the binary fractional number measured from the line beginning.

For example, if three calls to the random number generator returned numbers 1, 0, and 1, the binary fractional number is 0.101. In decimal notation, it is equal to 1*2^-1+0*2^-2+1*2^-3=2^-1+2^-3=0.5+0.125=0.625. If four possible outcomes have probabilities 0.125, 0.5, 0.125, and 0.25, then the segments end at distances 0.125, 0.625, 0.75, and 1 from the beginning of the line. Assuming that points at the ends of the segments do not belong to them, number 0.625 falls in the third segment, that is, the probabilistic mapping returns the third outcome.

QSMM can use this approach to select an outcome of a probabilistic mapping according to probabilities of all possible outcomes. However, the approach has a drawback: it is hard to evaluate how much information from the random number generator the machine uses to select a particular outcome, as there can be loss of information received from the random number generator.

The following example demonstrates loss of information. If the random number generator returned numbers 1, 1, and 0, the decimal fractional number is 2^-1+2^-2=0.75. If the random number generator returned numbers 1, 1, and 1, the decimal fractional number is 2^-1+2^-2+2^-3=0.875. Both fractional numbers fall in the fourth segment, so the third number produced by the random number generator in both cases does not make any difference.

Another approach is putting more load on the random number generator to select a less probable outcome and putting less load on the random number generator to select a more probable outcome. In the extreme case, when an outcome has probability 1, and all other outcomes have probability 0, the machine selects the outcome without calling the random number generator at all.

In this approach, the machine builds a binary Huffman tree for a list of probabilities of all outcomes and then traverses the nodes of the Huffman tree from the root node to a leaf node representing a selected outcome by calling the random number generator at each traversed non-leaf node to select one of its two child nodes. This approach, however, implies rounding outcome probabilities to 2^-k, where k is the depth of a leaf node, and works better for a sufficiently large number of possible outcomes.

Generally, the greater is the probability of a leaf node, the shorter is a path to it from the root node, and, therefore, a lesser number of calls to the random number generator is necessary to select the leaf node. Thus, it is easier to select a more probable outcome, because this selection requires a lesser number of actions to perform.

Conversely, the lesser is the probability of a leaf node, the greater number of calls to the random number generator is necessary to select the leaf node. It looks like this principle is relevant to the physical environment—for a less probable event to occur, more random events have to happen.

For our example with outcome probabilities 0.125, 0.5, 0.125, and 0.25, the Huffman tree is:

Figure 1.1: Huffman tree for probabilities 0.125, 0.5, 0.125, and 0.25

For the above Huffman tree, the machine selects the most probable second outcome with probability 0.5 if a call to the random number generator returns 1. The machine selects the least probable first or third outcome with probability 0.125 if three calls to the random number generator return the sequence 0, 1, 0 or 0, 1, 1 respectively. The average number of calls to the random number generator for the Huffman tree is equal to 0.125*3+0.5*1+0.125*3+0.25*2=1.75.

For four equal outcome probabilities 0.25, the Huffman tree is:

Figure 1.2: Huffman tree for four probabilities 0.25

For the above Huffman tree, the number of calls to the random number generator to select any outcome is 2. On average, this Huffman tree puts more load on the random number generator to select an outcome compared to the Huffman tree in Figure 1.1.

To conclude, the average number of calls to the random number generator depends on a degree of difference between outcome probabilities—if some probabilities are much greater than others, the number of calls is less, and, if all probabilities are more or less equal, the number of calls is greater.

Supposing all calls to the random number generator have the same choice complexity, selecting a particular outcome of a probabilistic mapping would have a choice complexity value equal to a constant choice complexity value multiplied by the number of calls to the random number generator required to perform the selection. An average choice complexity value would be equal to a constant choice complexity value multiplied by the average number of calls to the random number generator required to select an outcome.

However, calls to the random number generator might not have equal choice complexity. Choice complexity for a stochastic act of selecting number 0 or 1 by the random number generator might depend on the number of distinguished outcomes the act has. If the outcomes are similar, the number of distinguished outcomes may be less than 2.

The distinguishability of outcomes might depend on a way how the stochastic act affects the entropy of nature. For example, when water is boiling in a kettle, water molecules move rapidly, and a possible location of a water molecule after a fixed period of time varies greatly. However, locations of water molecules in the kettle probably do not affect much—a usable outcome would be hot water in the kettle to make tea. In this case, the complexity of choice associated with a water molecule might be low. On the other hand, if the next location of a molecule were affecting a resolution of a stochastic optimization task where to search planets in space for colonization, the complexity of choice associated with the molecule could be high.

If nature is a self-sustaining environment, one can interpret the complexity of choice as the amount of change necessary to perform to sustain the environment as a result of various outcomes of a stochastic act. Supposing time is closed like space might be, sustaining the environment requires continuous effort throughout all time-space continuum.

Let us consider an adaptive probabilistic mapping. If an outcome positively correlates with a desired change in spur, and all other outcomes have a lesser correlation, the outcome has a greater probability compared to all other outcomes. If an outcome always led to a desired change in spur, and all other outcomes never led to the change, the outcome would have probability 1, and all other outcomes would have probability 0—the choice of a result of the probabilistic mapping for a specific argument would be deterministic. On the other hand, if all outcomes equally lead to a desired change in spur, they all have equal probabilities (that sum up to 1).

Suppose a machine is solving an optimization task consisting in the maximization of spur increment velocity, and the machine has learned how to achieve a constant increase in the velocity. The adaptive probabilistic mapping would have an argument where one of its corresponding outcomes has a significantly greater probability compared to probabilities of all other outcomes for the argument. This outcome would take part in reinforcing a behavior to constantly increase spur increment velocity. The machine puts a little load on the random number generator to produce a result of the adaptive probabilistic mapping for the argument, because the outcome with a significantly greater probability becomes the result most of the times.

Consider a situation that, at a particular point of time, the increment velocity has gone down, so the machine needs to change its own behavior to increase the increment velocity even more. The outcome with a significantly greater probability has now a lesser probability approximately equal to probabilities of some other outcomes for the argument. Probabilities of outcomes for other arguments may also change in a similar way. The machine engages the random number generator in a greater extent for selecting outcomes with less differing probabilities.

In other words, the machine is now in a difficult situation that means an increase of average complexity of choices the machine has to perform to increase the increment velocity. Would the machine feel the difficulty? (To answer this question, one can consider the amount of energy supplied to the random number generator.) The author considers the awareness of difficulty a rudiment of consciousness.

A computer running a pseudo-random number generator can simulate the behavior of a probabilistic mapping. Running a pseudo-random number generator is a deterministic process—the way today’s computers operate. A computer could use a random number generator implemented as a physical unit running a stochastic physical process. We could consider the unit as an interface to a (probabilistic) mapping provided by nature. A stochastic act would then be calling the mapping to obtain a result for a particular argument.

One could relate the abstract idea of good and bad to the concept of choice complexity. The good would support a certain level of choice complexity. The survival of an animate being would be preserving its ability to perform choices with a sufficient degree of complexity.

1.5 QSMM Components ¶

The QSMM library consists of the following main components:

• Actor implementation. Actor is an adaptive probabilistic mapping. An actor has a number of adjustable modes and parameters and supports limited customization of its algorithms. The Actor API makes it possible to create and destroy actors and perform various operations on them. The main operation is obtaining an outcome for a particular argument. Supplying the spur to an actor modulates causal relationships between the arguments and outcomes.
• Multinode model engine. It is an engine for running adaptive (“intelligent”) state models. The engine relies on actors to achieve adaptive behavior. A node, a state sub-model represented by a probabilistic finite automaton, interacts with an environment by means of executing assembler instructions. A developer provides a set of assembler instructions and provides their implementations as callback functions. By default, every node is a probabilistic finite automaton of uniform structure—it has all states connected in all possible ways with equal probabilities.
• Assembler. Parses a probabilistic assembler program and converts it to a probabilistic finite automaton. The assembler program defines probabilities of transitions between automaton states. The probabilities specify soft and hard constraints on automaton behavior.
• Disassembler. Converts a probabilistic finite automaton with state transition probabilities adjusted by spur modulation back to a probabilistic assembler program.

The QSMM library also includes a C implementation of functionality of STL map and multimap templates. A developer can use it to create ordered tree mapping objects in C programs without the need to rewrite them in C++.

The QSMM package includes the following main tools:

• Adaptive top-down parser. Adaptively parses a terminal symbol sequence in the top-down direction according to a template grammar specified as a set of nonterminal symbols with associated regular expressions. The parser learns a restricted (more precise) grammar by iteratively determinizing the template grammar while parsing the terminal symbol sequence multiple times.
• Adaptive bottom-up parser. Adaptively parses a list of terminal symbol sequences in the bottom-up direction by utilizing a top-down template grammar containing references to the productions of a context-free grammar inferred from a bottom-up template grammar. Both the top-down template grammar and bottom-up template grammar are sets of nonterminal symbols with associated regular expressions, where the former grammar is a transformation of the latter grammar. While parsing the list of terminal symbol sequences multiple times, the parser learns a more precise top-down grammar and, consequently, learns a PCFG (Probabilistic Context-Free Grammar) by restricting the context-free grammar inferred from the bottom-up template grammar.
• Bottom-up to top-down grammar converter. Converts a template grammar intended for parsing input terminal symbol sequences in the bottom-up direction to a template grammar intended for parsing input terminal symbol sequences in the top-down direction. The latter (top-down) template grammar can contain references to the productions of a context-free grammar inferred from the former (bottom-up) template grammar.
• Converter to a factored PCFG. Converts a template grammar specified as a set of nonterminal symbols with associated regular expressions to a PCFG for parsing terminal symbol sequences up to specified length in Viterbi style. Every nonterminal symbol of the PCFG has exact length of a terminal symbol sequence the nonterminal symbol consumes. The PCFG has multiple start nonterminal symbols for various lengths of input parse units.

1.6 Obtaining QSMM ¶

The official homepage of QSMM project:

http://qsmm.org

The package distribution is available on the project files page provided by SourceForge.net:

http://sourceforge.net/projects/qsmm/files/

1.7 Reporting Bugs and Getting Help ¶

The QSMM users mailing list is a place for discussing all things QSMM. To subscribe to the mailing list, unsubscribe from the list, view list archives, and perform other actions, use its information page:

https://lists.sourceforge.net/lists/listinfo/qsmm-users

The following types of feedback should go to the mailing list:

• Bugs. Please report bugs in QSMM and its documentation for quicker fixing them.
• New feature requests. If you would like to see a new feature in QSMM (e.g. a new API function to make it possible or simpler to write some kind of applications), then please submit a feature description.
• Questions and technical support. Because of experimental nature of this software, answers to some questions might not be as useful as one would expect.

Do not be embarrassed by a small number of messages in the list archives—be the first to report a bug, submit a feature request, or ask a question!

1.8 System Requirements ¶

System requirements include:

• operating system requirements;
• requirements to a software environment for building package programs, running tests, and building the documentation;
• a reference hardware environment.

This section does not mention usual programs and libraries for building and running programs written in C, nor it does mention standard programs commonly used in bash scripts.

Operating System—GNU/Linux ¶

QSMM version 1.19.1 supports building and using on a GNU/Linux system. The author has not tested building this package version in other environments.

Building Package Programs ¶

The author was building package programs by gcc version 13.3.

Optional but default library dependencies are gsl (GNU Scientific Library) and ncurses (console display library). The author was using the following library versions:

• gsl 2.7
• ncurses 6.5

Running Tests ¶

The test suite supports running package programs under Valgrind (open-source memory debugger for GNU/Linux). The author was using valgrind version 3.20.

Building the Documentation ¶

Building the package documentation in various formats requires the presence of additional packages in the system. Below is their list along with versions used by the author:

• asymptote 3.05
• dvipsk 2024.03.11
• imagemagick 7.1.2
• ghostscript 10.06
• texinfo 7.2
• texlive 2024

Reference Hardware Environment ¶

The author was building the package and running its programs in the following hardware environment:

• Intel® Celeron® CPU @ 2.1 GHz, 1 core
• 1G RAM

1.9 Installation ¶

For package installation instructions, refer to the file INSTALL located in the root of the package distribution. This manual does not include them to avoid duplication. The instructions also describe building documentation files, including this manual, and running the test suite.

1.10 API Basics ¶

A C program that uses QSMM includes one or more its header files. The main header file is qsmm.h. The #include <qsmm/qsmm.h> directive includes that file in the C program.

The header file qsmm.h includes a number of other header files. The C program can include them individually. For example, the header file sig.h defines datatypes and macros for signals.

Object handles reference objects of various types. For example, an object handle of qsmm_actor_t type references an adaptive probabilistic mapping.

All API functions report errors in a unified manner—via a result of int type, where negative results are error codes. A multinode model supports assigning an error handler to it for simpler error checking.

The API has a function for retrieving the version of a QSMM library.

• Header Files
• Basic Datatypes and Macros
• Object Handles
• Error Handling
• Getting Library Version

#include <qsmm/qsmm.h>

#include <stdio.h>
#include <qsmm/qsmm.h>

void
print_sig_array(
    const qsmm_sig_t *sigp,
    qsmm_sig_t nsig
) {

    putchar('[');
    for (qsmm_sig_t idx=0; idx<nsig; idx++) {
        if (idx) fputs(", ",stdout);
        printf("%" QSMM_FMT_PRI_SIG,sigp[idx]);
    }
    puts("]");
}

1.11 Linking with the Library ¶

The QSMM library may have dependencies on other libraries. The configure script executed at the package configuring phase defined the dependencies. For a shared version of QSMM library, the shared library file embeds references to the dependencies, so you do not need to specify them in a link command:

gcc example.o -L/usr/local/lib -lqsmm

(If the library resides in the directory /usr/local/lib/, and that directory is not on the standard linker search path, use the option -L/usr/local/lib.)

To link with a static version of QSMM library, you need to specify libraries it depends on in a link command. For this, use linker options -l, such as -lgsl and -lm, for example:

gcc example.o -lqsmm -lgsl -lm

Alternatively, you can link using the libtool program. It uses the file libqsmm.la copied to the library directory when installing the package. That file describes all dependencies required to produce an executable linked against the QSMM library. The libtool program simplifies static linking.

1.12 Conventions for Datatypes ¶

Names of all datatypes declared in public header files of QSMM library and declared in its source code are in lowercase. Suffixes of the names correspond to C language keywords used for datatype declarations:

typedef

The ‘_t’ suffix. All names of datatypes for function pointers have the ‘_func_t’ suffix.

enum

The ‘_e’ suffix.

struct

The ‘_s’ suffix.

union

The ‘_u’ suffix.

Names of all datatypes declared in public header files of QSMM library have the ‘qsmm_’ prefix.

QSMM has conventions for the use of basic C language types and the types qsmm_sig_t and qsmm_ssig_t in various situations. The API and the package source code itself follow the conventions. C programs that use the QSMM library can also follow them.

The primary conventions are that if a datatype holds signal identifiers or numbers of signals, and such values

• never lie outside the range 0 to 32767, then you can use the type qsmm_sig_t;
• may be negative but never lie outside the range −32768 to 32767, then you can use the type qsmm_ssig_t;
• are non-negative and may exceed 32767, then you should use the type qsmm_sig_t;
• may be negative and may lie outside the range −32768 to 32767, then you should use the type qsmm_ssig_t.

For other situations, including ones when you chose not to use the type qsmm_sig_t or qsmm_ssig_t according to the first two items of primary conventions stated above, the rules are:

1. For boolean flags, use the type char or int or use specific bits in values of other numeric types.
2. For integer quantities that never lie outside the range 0 to 127, use the type char or int.
3. For integer quantities that may be negative but never lie outside the range −128 to 127, use the type signed char or int.
4. For non-negative integer quantities that may exceed 127 but never exceed 255, use the type unsigned char or int.
5. For integer quantities that may be less than −128 but never lie outside the range −32768 to 32767, use the type int.
6. For integer quantities that may exceed 255 but never lie outside the range −32768 to 32767, use the type int.
7. For non-negative integer quantities that may exceed 32767 but never exceed 65535, use the type unsigned int.
8. For non-negative integer quantities that may exceed 65535 but do not have the restriction that they must be at least 64-bit, use the type long or unsigned long.
9. For integer quantities that may be less than −32768 and may be greater than 32767 but do not have the restriction that they must be at least 64-bit, use the type long.
10. For integer quantities that must be at least 64-bit, use the type long long or unsigned long long.

For example, in QSMM:

• the type char holds unpacked boolean flags;
• the type int holds error codes and indices of spur types;
• the type long holds frequencies and the moments of discrete time.

The set of datatypes declared in the API with the ‘_t’ suffix in their names includes the datatypes qsmm_sig_t and qsmm_ssig_t, datatypes for function pointers with the ‘_func_t’ suffix, and datatypes for object handles (see Object Handles, for more information).

Datatypes declared in the API with the ‘_e’ suffix in their names are enumerations. Elements of enumerations are in uppercase.

Datatypes with the suffixes ‘_s’ and ‘_u’ are structures and unions respectively. The order of their fields depends on field datatypes and is the following:

1. Fields of basic types. The type order is the following: char, signed char, unsigned char, short, unsigned short, int, unsigned int, long, unsigned long, long long, unsigned long long, float, double, long double.
2. Fields of types defined in QSMM with the ‘_t’ suffix in their names.
3. Fields of the other types except for pointers of void * type and except for types defined using the keywords ‘enum’, ‘struct’, and ‘union’.
4. Fields of types defined using the keyword ‘enum’. Enumerations defined in QSMM go first.
5. Fields of types defined using the keyword ‘struct’. Structures defined in QSMM go first.
6. Fields of types defined using the keyword ‘union’. Unions defined in QSMM go first.
7. Fields holding pointers of void * type.

For every type name within each type category listed above, the type order is the following:

1. Non-pointer and non-array types (not applicable to the void type).
2. Arrays (but not pointers to arrays), including arrays of pointers.
3. Pointer types (except for arrays of pointers) with full names beginning with the const qualifier in the order of increasing the number of asterisks.
4. Pointer types (except for arrays of pointers) with full names that do not begin with the const qualifier in the order of increasing the number of asterisks.

2.1 Event History ¶

Occurrences of input and output signals of an actor with particular moments of time when the occurrences took place is an actor’s event history.

An example event history along with changed spur values represented as a two-dimensional chart is in Figure 2.1.

Figure 2.1: an event history example as a two-dimensional chart

In the chart, filled dots denote the events of receiving input signals, and unfilled dots denote the events of emitting output signals. Every such event is on a horizontal line denoting a particular input or output signal with an identifier printed after the line end. Every input signal and its corresponding output signal connected by an arrow are on the same vertical line, as the API does not specify time between receiving an input signal and emitting an output signal by an actor. A plot of single spur value changing over time is above the event history.

A fragment of this two-dimensional chart in gray represented as a one-dimensional plot is in Figure 2.2.

Figure 2.2: an event history fragment as a one-dimensional plot

We can consider that every event of receiving an input signal encodes an event of receiving a tuple of signals. Identifiers assigned to unique tuples will be input signal identifiers.

For example, we can replace every input signal with a triple of signals. The above event history fragment changed to show input signals replaced with signal triples is in Figure 2.3.

Figure 2.3: tuples of signals as input signals

In this figure, input signal 3 has encoded the triple <3, 0, 4>, input signal 5 has encoded the triple <1, 2, 5>, and input signal 0 has encoded the triple <5, 3, 4>.

In this way, by setting one-to-one correspondence between single-value arguments of a probabilistic mapping and n-value tuples, we convert the mapping to an n-ary probabilistic mapping.

By including the previous outcome of a probabilistic mapping in its next n-ary argument, we convert a stateless probabilistic mapping to stateful one—an outcome of a probabilistic mapping becomes its state. Using this approach, a probabilistic mapping can model a finite automaton. After the conversion, input signal 5 encodes the quadruple <5, 1, 2, 5>, and input signal 0 encodes the quadruple <3, 5, 3, 4>, where the first 5 in the first quadruple is a previous state of the probabilistic mapping, and the first 3 in the second quadruple is the next state.

An n-ary argument of an adaptive probabilistic mapping is an action choice state of an actor. An action choice state is an input list of events an actor shall respond to by choosing an action—emitting an output signal. An action choice state might be a known or guessed current system or environment state that requires generating an adaptive action.

In the past, action choice states were meant to be n-grams of events from an event history. The current view is that action choice states are not necessarily n-grams, but the terminology remains the same for compatibility. An action choice state n-gram is a signal identifier list encoding an action choice state. The tuples <3, 0, 4>, <1, 2, 5>, <5, 3, 4>, <5, 1, 2, 5>, and <3, 5, 3, 4> are action choice state n-grams used to select output signals 5, 3, and 1.

2.2 Output Signal Selection ¶

An actor emits output signals stochastically according to their probabilities the actor calculates. It collects statistics on observed consequences of emitting various output signals in various action choice states. The actor treats the observed consequences as dependent only on a specific action choice state and a specific output signal emitted in the action choice state. An event history segment for observing the consequences is a cycle—a segment between an occurrence of the action choice state in the event history and the next occurrence of the action choice state in the event history. The actor uses the collected statistics to calculate the probabilities of emitting output signals allowed in a current action choice state.

For every occurrence of an action choice state in the event history, the actor updates statistics on a cycle type—a pair comprised of the action choice state and an output signal emitted at the previous occurrence of the action choice state in the event history. The actor updates the statistics with the following parameters:

• spur increment between the previous occurrence of an action choice state and its current occurrence;
• time increment between the previous occurrence of an action choice state and its current occurrence;
• number of output signals emitted between the previous occurrence of an action choice state and its current occurrence;
• number of times the actor emitted the output signal in the action choice state.

The actor calculates probabilities of output signals to emit in a current action choice state using statistics collected for pairs comprised of the action choice state and each allowed output signal. This statistics includes the following parameters for event history segments between an occurrence of the action choice state with emitting the output signal and the next occurrence of the action choice state:

• sum of spur increments over the segments;
• total time length of the segments;
• mean number of output signals emitted in the segments.

Using the sum of spur increments over the segments and the total time length of the segments, the actor can calculate the mean velocity of spur increment over the segments. The higher the mean spur increment velocity is for segments specified by the pair comprised of an action choice state and output signal (i.e. specified by a cycle type), the greater the probability of emitting the output signal in the action choice state can be.

Let us consider a simplistic example. Suppose, an actor has recorded that emitting output signal 18 in an action choice state concluded with spur increment +7 over a period of 40 time units until the action choice state occurred the next time in the event history. Suppose, the actor has also recorded that emitting output signal 19 at another occurrence of the action choice state concluded with spur increment +6 over a period of 30 time units until the action choice state occurred again in the event history. Therefore, when emitting next output signal in the action choice state, the actor will select signal 19 with higher probability than signal 18 because spur increment velocity 6/30 for signal 19 is greater than spur increment velocity 7/40 for signal 18. When an actor emits the same output signal in the same action choice state more than once, the actor accumulates the statistics and uses mean spur increment velocity to calculate the probability of emitting an output signal.

In QSMM, the actor uses a more complex method of selecting an output signal, so the above example is a simplification that does not fully agree with practice. See Customizing the Relative Probability Function, for more information about available and user-defined functions utilizing various kinds of statistics to calculate the probability of emitting an output signal.

To improve reaction to latest tendencies in the event history, the actor may memorize statistics for time periods shorter than the entire event history.

2.3 Small and Large Actors ¶

The most time-consuming operation frequently performed by an actor is adaptive emitting an output signal. The only actor type implemented before QSMM 1.15 was small actor. A small actor performs the operation of adaptive emitting an output signal in the following steps:

1. The actor calculates relative probabilities of all output signals.
2. The actor stochastically selects an output signal according to the relative probabilities using a random number generator.

From the standpoint of computer implementation, the most time-consuming is the first step when the actor calculates relative probabilities of all output signals. For example, to adaptively select an output signal from a set of 16 output signals, a small actor has to perform 16 evaluations of a relative probability function. Figure 2.4 illustrates this situation.

Figure 2.4: emitting an output signal in an action choice state of a small actor

Time necessary for sequential execution of this calculation process is the sum of time periods consumed by the calculation of a relative probability function value for every output signal. While calculating every value can be time expensive on its own on account of performing many arithmetic operations, fetching their input values from statistics storage can add significant time overhead to the calculation process.

To speed up selecting output signals by an actor, QSMM 1.15 introduced the concept of large actor. If the weights (or relative profile probabilities) of output signals of a large actor are equal, the number of relative probability function evaluations performed by the large actor is approximately equal to the product of a logarithm of the number of output signals and a logarithm base. Large actors provide fast selection of an output signal when the number of output signals is large or even huge; the only limitation is the amount of memory available for storing the control structures of a large actor.

The adaptivity of behavior of an actor depends on a relative probability function used, that is, on a function that returns the relative probability of selecting a signal. The default function used by a large actor can provide moderate efficiency in solving certain kinds of problems, for example, the identification of a current environment state. Developers unsatisfied with results produced using that function can provide a custom function via corresponding API calls.

A large actor performs fast stochastic selection of output signals by using output signal choice trees containing nodes controlled by a small actor. The following entities correspond to every tree node:

1. An action choice state: either root one or intermediate one. The root action choice state is current action choice state of the large actor selecting an output signal for that state.
2. An array of relative probabilities of output signals typically containing a few elements. The output signals are are either the output signals of the large actor or intermediate output signals. To every intermediate output signal there corresponds an intermediate action choice state located at a deeper hierarchy level.

Figure 2.5 illustrates this tree structure. Indexed letters “s” denote intermediate action choice states, and indexed letters “a” denote intermediate output signals.

Figure 2.5: emitting an output signal in an action choice state of a large actor

A large actor performs the operation of adaptive emitting an output signal by the following algorithm:

1. Set the current intermediate action choice state equal to current action choice state of the large actor.
2. Adaptively generate an output signal for the current intermediate action choice state by a small actor associated with the large actor. This operation requires calculating relative probabilities of all output signals (typically a few ones) for the current intermediate action choice state.
3. An output signal generated in step 2 can be either an intermediate output signal or an output signal of the large actor. If the output signal is the one of the large actor, then finish.
4. Change the current intermediate action choice state to an intermediate action choice state corresponding to the intermediate output signal and go to step 2.

Thus, using a tree represented in Figure 2.5, adaptive selection of an output signal from a set of 16 output signals requires only 8 evaluations of a function returning the relative probability of a signal. A taller binary tree would provide adaptive selection of an output signal from a set of 256 output signals using 16 evaluations of the function.

Tree structure affects weights of output signals of a large actor and the speed of emitting the output signals by the large actor. The weights might be coarsely granular, as the weight of an output signal is inversely proportional to tree arity raised to power equal to the depth of a leaf node for the output signal.

Before QSMM 1.19, large actors always used n-ary Huffman trees to adaptively generate output signals. Using a Huffman tree built for a specific list of output signal weights, emitting output signals with greater weights requires the same or a smaller number of relative probability function evaluations compared to emitting output signals with lesser weights.

QSMM 1.19 has ability to control the granularity of output signal weights to satisfy a specified tolerance. On passing a positive tolerance, a large actor rearranges a Huffman tree built for a list of output signal weights by splitting leaf nodes to make the absolute value of difference between supplied weight of each output signal and its weight determined by the depth of a leaf node for the output signal less than the tolerance. In a rearranged Huffman tree multiple leaf nodes may correspond to the same output signal.

Although having much the same API, small and large actors are not fully interchangeable. There are specifics of using actors of every type.

2.4 Creating an Actor ¶

An actor handle refers to a small or large actor.

Data type: qsmm_actor_t ¶

This is a type for an actor handle. It is a pointer, so variables of this type can be NULL. The function qsmm_actor_create creates a new actor and returns its handle. The function qsmm_actor_destroy destroys an existing actor addressed by a handle. You can pass an actor handle to API functions taking an argument of qsmm_actor_t type after the creation of an actor and until its destruction.

Use the following functions to create and destroy an actor.

Function: int qsmm_actor_create (const struct qsmm_actor_desc_s *desc_p, qsmm_actor_t *actor_p) ¶

This function creates an actor using parameters in *desc_p and stores an actor handle in *actor_p.

If actor_p is NULL, the function only validates parameters in *desc_p.

The function returns a non-negative value on success or a negative error code on failure in creating an actor. Currently, the function can return the following error codes.

QSMM_ERR_INVAL

Parameters in *desc_p are invalid.

QSMM_ERR_NOMEM

There was not enough memory to create an actor.

Function: void qsmm_actor_destroy (qsmm_actor_t actor) ¶

This function destroys an actor specified by handle actor. You must not use the actor handle after actor destruction. If actor is NULL, the function has no effect.

Below there are the description of a structure passed in *desc_p to the function qsmm_actor_create and the description of enclosed structures.

Structure: qsmm_actor_desc_s ¶

This structure describes parameters for creating an actor by the function qsmm_actor_create. The structure contains the following fields.

Field: int nspur ¶

A non-negative number of spur types the actor will use. If the actor is the large one, a small actor associated with it will use the same number of spur types.

Field: int ntime ¶

[New in QSMM 1.19] A non-negative number of continuous time types the actor will use. If the actor is the large one, a small actor associated with it will use the same number of continuous time types. The function qsmm_set_actor_spur_time sets up a correspondence between spur types and time types.

Field: int ngram_sz ¶

The length of a list of signal identifiers encoding an action choice state. That length must be greater than 0.

Field: int compat ¶

The compatibility of algorithms used by the actor: 0, 1, 2, or 3. Value 0 means to use original algorithms implemented before QSMM 1.15. Value 1 supported since QSMM 1.15 means to use enhanced algorithms:

• use a simpler formula for automatic spur increment;
• in formulas for computing the relative probability of an output signal, additionally multiply mean discrete cycle period by a ratio for converting discrete time to the number of output signals emitted and, if the relative probability function type is QSMM_RELPROB_BUILTIN1, by 2.

[New in QSMM 1.17] Value 2 further improves the algorithms:

• calculate output signal probabilities of QSMM_PROB_FQ type by the function qsmm_actor_calc_action_prob based on exact (not sometimes decremented by 1) frequencies of output signals in the event history;
• for a small actor, make the function qsmm_get_actor_sig_action select a signal without using a random number generator (i.e. deterministically) if the signal is the only one signal with a positive relative probability.

[New in QSMM 1.19] Value 3 means:

• actor supports multiple continuous time types; the field ntime of qsmm_actor_desc_s structure specifies their number when creating the actor by the function qsmm_actor_create; the actor does not use the field period_sum_c of qsmm_cycle_s structure and the field tmc0 of qsmm_state_s structure;
• function qsmm_actor_create called for a small actor initializes to 1 (instead of 0.5) the multiplier for converting discrete time to the number of output signals emitted.

Creating a large actor requires setting this field to 3. The function qsmm_actor_create sets the field compat of qsmm_desc_s structure to 3 when creating a multinode model for a large actor.

Set this field to 3 in your new programs. This manual does not include outdated details about old algorithms.

Field: qsmm_sig_t nsig_in ¶

The number of input signals. Input signal identifiers are in the range 0 (inclusive) to that number (exclusive). This field must be greater than 0.

Field: qsmm_sig_t nsig_out ¶

Initial number of output signals. Output signal identifiers are in the range 0 (inclusive) to that number (exclusive). This field must be greater than 1. See Number of Output Signals, for how to obtain the current number of output signals of an actor or increase that number.

Field: qsmm_rng_t rng ¶

The handle of a random number generator the actor will use. See Random Number Generators, for how to create and destroy random number generators and perform other operations on them. If this field is NULL, the function qsmm_actor_create creates an instance of default random number generator for use by the actor until its destruction.

Field: struct qsmm_pair_sig_s * range_sig_p ¶

Ranges of signal identifiers in a list encoding an action choice state. The field ngram_sz of this structure specifies the length of the list. The actor uses the ranges to check the validity of a list of signal identifiers encoding a current action choice state in various API functions.

If the field range_sig_p is not NULL, then this field must be the pointer to an array of ngram_sz elements, where each element is a pair. The elements correspond to positions of signal identifiers in a list encoding an action choice state. The fields first and second of each pair define the minimum value and the maximum value of a corresponding signal identifier. The value of first must be less than or equal to the value of second. The value of second must be less than the number of input signals of the actor specified in the field nsig_in of this structure. If range_sig_p is NULL, this condition means that every signal in the list lies in the range 0 (inclusive) to the number (exclusive) of input signals of the actor.

Field: struct qsmm_actor_large_desc_s * large_desc_p ¶

The parameters of a large actor. A non-NULL value of this field means to create a large actor. This mode requires setting the field compat to 3.

To improve compatibility with future versions of QSMM library, zero by the function memset an instance of qsmm_actor_desc_s structure before setting its fields.

Structure: qsmm_pair_sig_s ¶

This structure holds a pair of signals, for example, a signal range. The structure contains the following fields.

Field: qsmm_sig_t first ¶

The first element of the pair. If the pair represents a signal range, the minimum value of a signal identifier.

Field: qsmm_sig_t second ¶

The second element of the pair. If the pair represents a signal range, the maximum value of a signal identifier.

Structure: qsmm_actor_large_desc_s ¶

This structure specifies the parameters of a large actor.

Field: int tree_arity ¶

The maximum number of child nodes of every node of output signal choice trees for adaptive selection of output signals by the large actor. That number must be greater than 1. You can use value 2 in most cases. It is better to use the number of output signals of a large actor equal to a positive integer power of the value of this field.

Field: double profile_tol ¶

[New in QSMM 1.19] An upper bound on absolute value of difference between the profile probability of each output signal and its probability determined by the structure of an output signal choice tree. The large actor rearranges a Huffman tree built for an action choice state by splitting leaf nodes and making multiple leaf nodes correspond to the same output signals until the absolute value becomes less than the upper bound. The field must be 0 or lie in the range FLT_EPSILON to 1. If the field is 0, the actor does not rearrange Huffman trees built for action choice states.

To improve compatibility with future versions of QSMM library, zero by the function memset an instance of qsmm_actor_large_desc_s structure before setting its fields.

Below there is sample source code for creating a small actor.

struct qsmm_actor_desc_s actor_desc;
memset(&actor_desc,0,sizeof(actor_desc));
actor_desc.nspur=1;
actor_desc.ntime=1;
actor_desc.ngram_sz=3;
actor_desc.compat=3;
actor_desc.nsig_in=6;
actor_desc.nsig_out=4;
qsmm_actor_t actor=0;
const int rc=qsmm_actor_create(&actor_desc,&actor);
if (rc<0) fprintf(stderr,"qsmm_actor_create: %s\n",qsmm_err_str(rc));

Below there is sample source code for creating a large actor.

struct qsmm_actor_desc_s actor_desc;
memset(&actor_desc,0,sizeof(actor_desc));
actor_desc.nspur=1;
actor_desc.ntime=1;
actor_desc.ngram_sz=3;
actor_desc.compat=3;
actor_desc.nsig_in=6;
actor_desc.nsig_out=4096;
struct qsmm_actor_large_desc_s large_desc;
memset(&large_desc,0,sizeof(large_desc));
large_desc.tree_arity=2;
large_desc.profile_tol=0.005;
actor_desc.large_desc_p=&large_desc;
qsmm_actor_t actor=0;

const int rc=qsmm_actor_create(&actor_desc,&actor);
if (rc<0) fprintf(stderr,"qsmm_actor_create: %s\n",qsmm_err_str(rc));

You can obtain some parameters specified when creating an actor by the following functions.

Function: int qsmm_get_actor_nspur (qsmm_actor_t actor) ¶

This function returns a non-negative integer equal to the number of spur types used by actor. The field nspur of qsmm_actor_desc_s structure passed to the function qsmm_actor_create when creating the actor specifies that number.

Function: int qsmm_get_actor_ntime (qsmm_actor_t actor) ¶

This function returns a non-negative integer equal to the number of continuous time types used by actor. The field ntime of qsmm_actor_desc_s structure passed to the function qsmm_actor_create when creating the actor specifies that number.

Function: int qsmm_get_actor_ngram_sz (qsmm_actor_t actor) ¶

This function returns a positive integer number equal to the length of a list of signal identifiers encoding an action choice state of actor. The field ngram_sz of qsmm_actor_desc_s structure passed to the function qsmm_actor_create when creating the actor specifies that length.

Function: int qsmm_get_actor_compat (qsmm_actor_t actor) ¶

This function returns a non-negative integer number indicating the compatibility of algorithms used by actor. The field compat of qsmm_actor_desc_s structure passed to the function qsmm_actor_create when creating the actor specifies the number.

Function: qsmm_sig_t qsmm_get_actor_nsig_in (qsmm_actor_t actor) ¶

This function returns the number of input signals of actor. The field nsig_in of qsmm_actor_desc_s structure passed to the function qsmm_actor_create when creating the actor specifies that number.

Function: const struct qsmm_pair_sig_s * qsmm_get_actor_range_sig (qsmm_actor_t actor) ¶

This function returns the pointer to an array in an internal structure of an actor describing allowed ranges of signal identifiers in lists encoding its action choice states. The field range_sig_p of qsmm_actor_desc_s structure passed to the function qsmm_actor_create when creating the actor specifies the content of the array. The number of elements in the array is equal to the length of a list of signal identifiers encoding an action choice state. The function qsmm_get_actor_ngram_sz returns that length. The function qsmm_get_actor_range_sig never returns NULL.

A datatype for the handle of statistics storage is qsmm_storage_t. See Statistics Storage, for more information. You can obtain the handle of statistics storage of an actor by the following function.

Function: qsmm_storage_t qsmm_get_actor_storage (qsmm_actor_t actor) ¶

This function returns the handle of storage used by actor for collecting statistics on the event history. If the actor is the large one, the storage contains only part of the statistics. Output signal choice trees within the multinode model of the large actor hold the other part of the statistics. The function never returns NULL.

A datatype for the multinode model of a large actor is qsmm_t. You can obtain the handle of the multinode model of an actor by the following function.

Function: qsmm_t qsmm_get_actor_large_model (qsmm_actor_t actor) ¶

This function returns the handle of a multinode model used by a large actor to adaptively emit output signals. If actor is a large actor, the function returns a non-NULL handle. If actor is a small actor, the function returns NULL. You can use this function to determine whether an actor is large or small one.

2.5 Repeated Sequence of Operations ¶

After the creation of an actor and until its destruction, the actor usually performs a repeated sequence of operations—receiving a tuple of input signals followed by emitting an output signal. Any number of spur and time increments can precede or follow the two aforementioned operations.

• Receiving Input Signals
• Emitting an Output Signal
• Incrementing Spur
• Incrementing Time

int rc;
if ((rc=
     qsmm_actor_calc_action_prob(actor,0,0,NSIG_OUT,QSMM_PROB_LEARNED))<0)
    REPORT_ERROR(rc);
qsmm_sig_t sig_out=QSMM_SIG_INVALID;
if ((rc=qsmm_get_actor_sig_action(actor,0,0,NSIG_OUT,&sig_out))<0)
    REPORT_ERROR(rc);
if ((rc=qsmm_actor_reg_sig_out(actor,sig_out))<0) REPORT_ERROR(rc);

int rc;
qsmm_sig_t sig_out=QSMM_SIG_INVALID;
if ((rc=qsmm_get_actor_sig_action(actor,0,0,NSIG_OUT,&sig_out))<0)
    REPORT_ERROR(rc);
if ((rc=qsmm_actor_reg_sig_out(actor,sig_out))<0) REPORT_ERROR(rc);

2.6 Customizing the Relative Probability Function ¶

The efficiency of operation of a small or large actor, that is, the optimality of output signals an actor adaptively emits, depends on the form of a function returning the relative probability of an output signal. It may well be true that, for different kinds of applications, different functions are more suitable. QSMM provides a few built-in functions a developer may select for use by an actor. Unfortunately, the author of QSMM experiences problems with: (1) providing a sound theoretical basis for using the provided built-in functions in QSMM (the basis might not exist for some functions); (2) devising functions giving better efficiency than the built-in functions currently provided in QSMM. The author’s own experimenting had shown that, for some special cases, there do exist more efficient functions, but he did fail to find general forms for the functions making it possible to utilize them for wider ranges of values of actor parameters. To provide a way to solve the mentioned problems by developers, QSMM starting from version 1.15 supports specifying a custom relative probability function.

The main parameter of a relative probability function is its type. If the type specifies a custom form for the function, the developer shall provide a helper function for this form. Common relative probability function forms depend on the number of output signals and a type of time for computing spur increment velocity, spur weight, and a way of spur perception for every spur type. All relative probability function forms depend on actor temperature.

• Relative Probability Function Types
• Helper Relative Probability Functions
• Spur Weight
• Spur Perception
• Number of Output Signals
• Actor Temperature

#include <math.h>

static double
get_relprob_exp_mlt(
    qsmm_actor_t actor,
    double cycle_period,
    void *paramp
) {
    return log(qsmm_get_actor_nsig_ctrl(actor))*cycle_period/2;
}

qsmm_set_actor_relprob_type(actor,QSMM_RELPROB_USER1);
qsmm_set_actor_relprob_helper(actor,&get_relprob_exp_mlt,0);

qsmm_set_actor_relprob_type(actor,QSMM_RELPROB_BUILTIN3);
qsmm_set_actor_ktemperature(actor,2);

2.7 Specifying Output Signal Weights ¶

The weight of an output signal of an actor is a coefficient for multiplying the probability of selecting the output signal by the actor. The coefficient is a hint—it increases or decreases that probability. To disable emitting an output signal, set its weight equal to 0.

In the simplest case, an actor uses the weight of an output signal when selecting the output signal in all action choice states. On binding the weight to a specific action choice state, the actor uses that weight when selecting the output signal in the action choice state only.

Large actors do not support setting the weight of an output signal for all action choice states at once. However, all actors support binding output signal weights to specific action choice states.

A preloaded probability profile is a list of output signal weights loaded into an actor. You can assign a preloaded probability profile to specific action choice states for setting their output signal weights.

For a small actor, you can also bind output signal weights to an action choice state by specifying a probability profile for the action choice state held in statistics storage of the small actor. See Structures for Accessing Storage, for how to pass profile probabilities to storage access functions to write them to statistics storage.

It is essential to note that by reasons described in Number of Output Signals, changing default weights of output signals of a small actor makes its behavior ill-defined in general. Therefore, you should avoid doing this. Changing the weights seems to be a correct procedure for a large actor only, because the procedure does not alter the number of choice alternatives at decision points along the pathways to output signals.

• Setting a Weight for an Output Signal
• Preloading a Probability Profile
• Assigning a Preloaded Probability Profile

2.8 Automatic Spur ¶

Automatic spur is a means for implementing the idea that an intrinsic feature of an intelligent system is adherence to the principle of minimum energy (or maximum probability). The use of automatic spur may substantially increase the optimality of output signals an actor emits. The automatic spur also makes it possible to use an actor in cases when it does not receive explicit spur increments.

Automatic spur can be useful when an actor models a finite automaton. In this case, the output signals of the actor represent the states of the finite automaton, and each action choice state is a superposition of a previous finite automaton state and an input signal received when the finite automaton was in the previous state. The automatic spur is a measure of adequacy of a model of finite automaton states to a sequence of input signals received.

In QSMM, a common approach to building a model of internal states of an entity to solve a problem is the use of a spur scheme with two spur types for inhibitory (negative) spur and excitatory (positive) spur countervailing each other. The inhibitory spur is the automatic spur with normal way of perception. The excitatory spur is specific to the problem.

If an actor uses automatic spur, it is equal to the sum of automatic spur increments calculated by the function qsmm_actor_reg_ngram_in for action choice states that have prior occurrences in the event history. For such an action choice state, qsmm_actor_reg_ngram_in knows an output signal emitted at the previous occurrence of the action choice state. Note that for the last occurrence, an output signal emitted in the action choice state is not yet known in qsmm_actor_reg_ngram_in.

The actor calculates an automatic spur increment for a pair representing the logical consequence h → z and consisting of an action choice state h and an output signal z emitted in the action choice state. The automatic spur increment is equal to the natural logarithm of the observed probability of occurrence of the pair in the event history:

This formula uses the following notations:

v^(h,z)

The number of occurrences of the pair in the event history.

t₀

The discrete time of the previous occurrence of the pair in the event history. The function qsmm_actor_reg_sig_out records that time.

If an actor uses a single spur type, and it corresponds to automatic spur, it is better to set the type of a relative probability function for the actor to QSMM_RELPROB_BUILTIN2 or QSMM_RELPROB_BUILTIN3. It is also better to use automatic spur in conjunction with the other countervailing spur that evaluates progress in solving a problem in some other way.

Use functions described below to query or set a spur type for automatic spur. By default, an actor created by the function qsmm_actor_create does not use automatic spur.

Function: int qsmm_get_actor_auto_spur_type (qsmm_actor_t actor) ¶

This function returns non-negative index of a spur type for automatic spur used by an actor or negative error code QSMM_ERR_NOTFOUND if the actor does not use automatic spur.

Function: int qsmm_set_actor_auto_spur_type (qsmm_actor_t actor, int spur_type) ¶

This function sets to spur_type the spur type for automatic spur of an actor. Spur types have zero-based indices. If spur_type is equal to −1, the actor does not use the automatic spur; this condition does not affect the use of automatic spur by a small actor associated with the large actor.

On success, the function returns a non-negative value. If spur_type is less than −1 or is greater than or equal to the number of spur types specified in the field nspur of qsmm_actor_desc_s structure when creating the actor, the function returns negative error code QSMM_ERR_INVAL.

A small actor can use at most one spur type for automatic spur. The functions qsmm_get_actor_auto_spur_type and qsmm_set_actor_auto_spur_type called for a small actor query and set the index of a spur type for the automatic spur.

A large actor can use at most two spur types for automatic spur. A small actor associated with a large actor can use the aforementioned automatic spur—increment it for action choice states occurring in the event history of the small actor. The action choice states represent nodes of output signal choice trees traversed to generate output signals by the large actor. A large actor itself can use another automatic spur—increment it for action choice states occurring in the event history of the large actor. The functions qsmm_get_actor_auto_spur_type and qsmm_set_actor_auto_spur_type called for a large actor query and set the index of a spur type for the latter automatic spur.

2.9 Switching Adaptive or Random Behavior ¶

An actor uses a random number generator to select a signal from a set of possible signals according to their relative probabilities. The function qsmm_get_actor_sig_action performs this selection. An actor does not use a random number generator if there is only one signal with a positive relative probability—the actor simply selects the signal.

For a small actor, qsmm_get_actor_sig_action selects an output signal based on probabilities of output signals previously calculated by the function qsmm_actor_calc_action_prob. When selecting an output signal of a large actor, qsmm_get_actor_sig_action indirectly calls itself a number of times for a small actor associated with the large actor to traverse output signal choice tree of a current action choice state for selecting an output signal corresponding to one of tree leaves.

You can obtain a random number generator used by an actor—either provided when creating the actor or an instance of a default random number generator allocated automatically.

Function: qsmm_rng_t qsmm_get_actor_rng (qsmm_actor_t actor) ¶

This function returns a random number generator used by an actor to select output signals. The function never returns NULL.

You typically use a returned random number generator to seed the generator after creating an actor. See Random Number Generators, for how to seed a random number generator and perform other operations on it.

A useful approach to determine the amount of contribution of an actor to the optimality of behavior of a system is comparing optimality measure for the system with the actor behaving adaptively versus optimality measure for the system with the actor behaving randomly. The greater the difference is between the values the more contribution the actor makes to the overall optimality of system behavior.

Use the following functions to query current mode of behavior of an actor and to switch an actor to random or adaptive (normal) behavior mode.

Function: int qsmm_get_actor_random (qsmm_actor_t actor) ¶

This function returns a positive value if an actor is currently in random behavior mode or 0 if the actor is in adaptive (normal) behavior mode. The function never returns negative values.

Function: void qsmm_set_actor_random (qsmm_actor_t actor, int flag) ¶

This function switches current mode of behavior of an actor to random or adaptive (normal) mode. If flag is non-zero, the function switches the current mode to random mode. If flag is zero, the function switches the current mode to adaptive mode.

For a small actor, random and adaptive behavior modes affect how the function qsmm_actor_calc_action_prob calculates probabilities with the type QSMM_PROB_AGGR or QSMM_PROB_LEARNED.

In random mode, qsmm_actor_calc_action_prob called for a small actor does not calculate relative probabilities of output signals in the normal way but uses output signal weights for the relative probabilities. If the type of probabilities to calculate is QSMM_PROB_AGGR, qsmm_actor_calc_action_prob additionally applies a probability profile to the weights if it exists in statistics storage for a current action choice state. Finally, the function normalizes resulting values.

For a large actor, random and adaptive behavior modes are actually behavior modes of a small actor associated with the large actor.

The function qsmm_actor_create initializes current mode of behavior of a newly created actor to adaptive (normal) mode.

2.10 Revising Action Choice States ¶

Revising action choice states means enumerating or removing them. You may need to enumerate action choice states of an actor to clear or update statistics on them. The following function enumerates action choice states that have the pieces of information stored in actor’s memory.

Function: int qsmm_actor_enum_ngrams_v2 (qsmm_actor_t actor, int ngram_lower_sz, int ngram_upper_sz, int range_sz, int rez1, const qsmm_sig_t *sig_ngram_lower_p, const qsmm_sig_t *sig_ngram_upper_p, const struct qsmm_pair_sig_s *range_sig_p, qsmm_enum_ngram_callback_func_t callback_func, void *paramp) ¶

This function enumerates lists of signal identifiers encoding action choice states that have the pieces of information stored in the memory of an actor. For a small actor, the action choice states have the pieces of information held in statistics storage with a handle retrievable by the function qsmm_get_actor_storage. For a large actor, the action choice states are the union of the following state sets:

• action choice states with assigned references to output signal choice trees but without model nodes yet created to store tree instances;
• action choice states with model nodes already created to store instances of output signal choice trees;
• action choice states that have the pieces of information held in statistics storage with a handle retrievable by qsmm_get_actor_storage for the large actor.

If ngram_lower_sz is positive, it specifies the length of sig_ngram_lower_p array containing a lower bound for the enumerated lists—their prefixes with length ngram_lower_sz must be greater than or equal to the lower bound. If ngram_upper_sz is positive, it specifies the length of sig_ngram_upper_p array containing an upper bound for the enumerated lists—their prefixes with length ngram_upper_sz must be less than the upper bound. If range_sz is positive, it specifies the length of range_sig_p array containing allowed ranges of signal identifiers in prefixes with length range_sz in the enumerated lists—every signal identifier in a prefix must be greater than or equal to the field first of an element of range_sig_p at the same index and must be less than or equal to the field second of the element.

The function qsmm_actor_enum_ngrams_v2 compares two lists of signal identifiers from left to right. If a signal identifier in the first list is less than or greater than a signal identifier in the second list at the same index, the first list is less than or greater than the second list respectively. If the first list is a prefix of the second list, the first list is less than the second list. If the second list is a prefix of the first list, the first list is greater than the second list. If the two lists have the same length and content, the lists are equal.

The process of enumeration is repeated calling a callback function callback_func with passing an array of signal identifiers encoding an enumerated action choice state in ascending order of the arrays. The callback function also receives a user parameter paramp. If the callback function returns a positive value, qsmm_actor_enum_ngrams_v2 continues enumeration. If the callback function returns zero, qsmm_actor_enum_ngrams_v2 terminates enumeration and reports success. If the callback function returns a negative value, qsmm_actor_enum_ngrams_v2 terminates enumeration and reports failure.

The function qsmm_actor_enum_ngrams_v2 does not support recursive calling from the callback function for the same actor.

The function returns a non-negative value on success or a negative error code on failure. Currently, the function can return the following error codes.

QSMM_ERR_INVAL

One of the following conditions is true:

• ngram_lower_sz is negative;
• ngram_lower_sz is greater than the length of a list of signal identifiers encoding an action choice state;
• ngram_lower_sz is positive, but sig_ngram_lower_p is NULL;
• ngram_upper_sz is negative;
• ngram_upper_sz is greater than the length of a list of signal identifiers encoding an action choice state;
• ngram_upper_sz is positive, but sig_ngram_upper_p is NULL;
• range_sz is negative;
• range_sz is greater than the length of a list of signal identifiers encoding an action choice state;
• range_sz is positive, but range_sig_p is NULL;
• field first in an element of range_sig_p is greater than the field second of the element;
• field second in an element of range_sig_p is greater than QSMM_SIG_MAX.

QSMM_ERR_CALLBACK

The callback function reported an error.

QSMM_ERR_NOMEM

There was not enough memory to perform the operation.

The type of a pointer to a callback function called for every enumerated action choice state is below.

Data type: qsmm_enum_ngram_callback_func_t ¶

This is a type of a callback function pointer with the following declaration:

typedef int
(*qsmm_enum_ngram_callback_func_t)(
    qsmm_actor_t actor,
    const qsmm_sig_t *sig_ngram_p,
    void *paramp
);

The function qsmm_actor_enum_ngrams_v2 calls the callback function for every enumerated action choice state of an actor. The argument sig_ngram_p specifies an array of signal identifiers encoding an enumerated action choice state. The field ngram_sz of qsmm_actor_desc_s structure passed to the function qsmm_actor_create when creating the actor specifies the length of the array. The function qsmm_get_actor_ngram_sz returns that length for a created actor. The argument paramp is a user parameter passed to qsmm_actor_enum_ngrams_v2.

The callback function shall return a positive value to continue the process of enumeration, zero to terminate the process of enumeration, or a negative value on error.

Use the following function to remove information on an action choice state from the memory of an actor.

Function: int qsmm_actor_remove_ngram (qsmm_actor_t actor, const qsmm_sig_t *sig_ngram_p) ¶

This function removes information on an action choice state encoded by an array of signal identifiers sig_ngram_p from the memory of an actor. The field ngram_sz of qsmm_actor_desc_s structure passed to the function qsmm_actor_create when creating the actor specifies the length of the array. The function qsmm_get_actor_ngram_sz returns that length for a created actor.

For a small actor, the function removes information on the action choice state held in statistics storage with a handle retrievable by the function qsmm_get_actor_storage. For a large actor, the function removes the following pieces of information if they exist:

• reference to an output signal choice tree assigned to the action choice state;
• model node holding an instance of an output signal choice tree assigned to the action choice state;
• information on the action choice state held in statistics storage with a handle retrievable by the function qsmm_get_actor_storage for the large actor.

The function returns a non-negative value on success or a negative error code on failure. Currently, the function can return the following error codes.

QSMM_ERR_NGRAM

The array sig_ngram_p is not a valid array of signal identifiers encoding an action choice state. To determine array validity, the function checks it for accordance with allowed ranges of signal identifiers specified by the field range_sig_p of qsmm_actor_desc_s structure when creating the actor.

QSMM_ERR_NOTFOUND

No information on the action choice state—nothing to remove.

QSMM_ERR_NOMEM

There was not enough memory to perform the operation for a large actor. This error can leave the large actor in inconsistent state. If after removing a reason of this error a repeated call to this function succeeds, or the function reports QSMM_ERR_NOTFOUND, the actor’s state becomes consistent.

2.11 Example of Using the Actor API ¶

In the example of using the Actor API, an agent controlled by a sample program has to find a short path to gold located in a labyrinth and then to an exit from the labyrinth. The source code of the sample program contains a labyrinth picture encoded using a subset of ASCII characters resembling pseudographics. You can change the picture to test agent behavior for various labyrinth configurations.

To find a resulting path, the agent visits the labyrinth the number of times defined by the macro NVISIT. Moving the agent to a cell designated as a labyrinth exit finishes a labyrinth visit. A custom function returning the relative probability of an output signal is the exponent of an expression equal to mean spur increment for a cycle type divided by actor temperature. The sample program uses a simulated annealing approach, where actor temperature gradually decreases with every subsequent labyrinth visit. The macro KT_0 defines initial actor temperature, and the macro KT_1 defines final temperature.

At the last labyrinth visit, the sample program shows a labyrinth picture and agent movements in it, prints an indication whether the agent took the gold, and displays current length of a visiting path. After finishing the last visit, a user can press Q to create the files labyr_0.asy and labyr_1.asy, print the number of labyrinth visits when the agent took the gold, and exit the sample program. The two files correspond to the case before taking the gold by the agent and the case after taking the gold respectively. Each file includes the file labyr.asy and represents a labyrinth picture containing the following information:

• an upward triangle indicating an entry to the labyrinth;
• one or more downward triangles indicating exits from the labyrinth;
• one or more circles indicating the gold;
• black cells indicating places never visited by the agent;
• other cells with gray scale backgrounds indicating visited places, where the backgrounds are darker for greater visiting frequencies;
• for the file labyr_0.asy, indicated by arrows learned gradients of movements to a cell with the gold;
• for the file labyr_1.asy, indicated by arrows learned gradients of movements to an exit cell after taking the gold;
• cells on the last visiting path contain thick arrows.

To process both files by Asymptote and produce the image files labyr_0.png and labyr_1.png, use the commands:

asy -f png labyr_0.asy
asy -f png labyr_1.asy

You can specify a random seed by a program argument. If the random seed is non-negative, the actor operates adaptively (normally). If the random seed is negative, the actor operates randomly. You can compare agent behavior and resulting files labyr_0.png and labyr_1.png for the two modes of program execution.

For example, after executing the command

$ qsmm-example-labyr2 1

the file labyr_0.png produced from labyr_0.asy contains the image

Figure 2.6: image labyr_0.png produced from labyr_0.asy

and the file labyr_1.png produced from labyr_1.asy contains the image

Figure 2.7: image labyr_1.png produced from labyr_1.asy

In the source code of the sample program, each labyrinth cell is a rectangle 4x3 (WxH). The picture of a labyrinth cell is represented in Figure 2.8. Adjacent cells have common border columns or rows. For adjacent cells located vertically, row 2 of an upper cell and row 0 of a lower cell are the same rows. For adjacent cells located horizontally, column 3 of a left cell and column 0 of a right cell are the same columns.

Figure 2.8: labyrinth cell

The macros MAX_X and MAX_Y define maximum zero-based coordinates of a labyrinth cell. Thus, the labyrinth has (MAX_X+1)*3+1 character columns in width and (MAX_Y+1)*2+1 character rows in height. The macros ENTRY_X and ENTRY_Y define zero-based coordinates of the entry cell of the labyrinth relative to its upper left corner.

Within a labyrinth cell, space characters denote allowed movement paths, and other characters denote disallowed movement paths. Let us denote a character in column x and row y of a labyrinth cell by <x,y>. If a cell is reachable from a labyrinth entry, then characters <1,1> and <2,1> of the cell must be spaces; small circles denote them in the figure. If the cell allows the agent to move to a left and/or right adjacent cell, then characters <0,1> and/or <3,1> respectively must be spaces; left and right arrows denote them in the figure. If the cell allows the agent to move to an upper adjacent cell, then characters <1,0> and <2,0> must be spaces; up arrows denote them in the figure. If the cell allows the agent to move to a lower adjacent cell, then characters <1,2> and <2,2> must be spaces; down arrows denote them in the figure.

If characters <0,0>, <3,0>, <0,2>, and <3,2> of a cell are the characters ‘*’, the cell contains gold. If there are multiple cells with the gold, visiting any of them means taking the gold. It is in a single copy per labyrinth visit—visiting any cell with the gold implicitly removes it from all other cells until the next labyrinth visit. If characters at the cell corners are the characters ‘#’, the cell is a labyrinth exit. A labyrinth can have more than one exit. When modifying the picture of a labyrinth, ensure proper alignment and connection of its cells and the absence of paths outside the labyrinth. Violating the mentioned rules leads to undefined program behavior.

The file samples/labyr2.c in the package distribution installed to $prefix/share/qsmm/samples/labyr2.c contains the source code of the sample program. Below is a copy of that source code. The command make builds the sample program if the configure script has configured QSMM to use the ncurses library. The command make install installs the binary of the sample program and names the installed binary as ‘qsmm-example-labyr2’ if the configure script has configured QSMM to install example programs. See the file INSTALL in the root of the package distribution for information on the configure script.

#include <config.h>

#include <assert.h>
#include <math.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>

#if defined(HAVE_CURSES_H)
#   include <curses.h>
#elif defined(HAVE_NCURSES_CURSES_H)
#   include <ncurses/curses.h>
#endif

#include <qsmm/qsmm.h>

#define NVISIT  200
#define MAX_X   14
#define MAX_Y   14
#define ENTRY_X 0
#define ENTRY_Y 14
#define NSIG_IN (MAX_X+1>MAX_Y+1?MAX_X+1:MAX_Y+1)
#define KT_0    (NVISIT*2)
#define KT_1    (1.0/KT_0)


#define CHK_FAIL(func, ...)                                               \
    do {                                                                  \
        const int rc=func(__VA_ARGS__);                                   \
        if (rc<0) {                                                       \
            fprintf(stderr, #func ": %s\n", qsmm_err_str(rc));            \
            goto Exit;                                                    \
        }                                                                 \
    }                                                                     \
    while (0)


#define ERREXIT(fmt, ...)                                                 \
    do {                                                                  \
        fprintf(stderr,fmt "\n", ## __VA_ARGS__);                         \
        goto Exit;                                                        \
    }                                                                     \
    while (0)


enum direct_e {
    DIRECT_NORTH,  // move one step up
    DIRECT_EAST,   // move one step right
    DIRECT_SOUTH,  // move one step down
    DIRECT_WEST,   // move one step left
    DIRECT_COUNT   // number of movement directions
};


static char last_path[2][MAX_Y+1][MAX_X+1];
static long fq_max[3], state_fq[3][MAX_Y+1][MAX_X+1];
static qsmm_actor_t actor=0;


// Generate an Asymptote file containing a picture of the labyrinth with
// learned movement gradients for the cases when the gold was or was not
// taken.
// Returns: 0 = success;
//         -1 = failure.

static int
create_pic(
    const char **pic_pp,
    char is_gf
) {
    char fln[]="labyr_?.asy";
    int ii, result=-1;
    fln[6]=is_gf?'1':'0';
    FILE *const filep=fopen(fln,"w");
    if (!filep) ERREXIT("%s: failed to open the file for writing",fln);
    fprintf(filep,"import labyr;\n\n");
    fprintf(filep,"set_dims(%d,%d);\n",MAX_X,MAX_Y);
    qsmm_sig_t *const sig_ngram_p=qsmm_get_actor_sig_ngram(actor);
    sig_ngram_p[0]=is_gf;
    for (int yy=0; yy<=MAX_Y; yy++)
        for (int xx=0; xx<=MAX_X; xx++) {
            if (state_fq[2][yy][xx]<1) {
                fprintf(filep,"black_box(%d,%d);\n",xx,yy);
                continue;
            }
            fprintf(filep, "fill_box(%d,%d,%.2f);\n",
                    xx, yy, (double) state_fq[(int) is_gf][yy][xx]/
                            fq_max[(int) is_gf]);
            const int row=yy*2, col=xx*3;
            for (const char *ccp="#*"; *ccp; ccp++)
                if (pic_pp[row][col]==*ccp && pic_pp[row][col+3]==*ccp &&
                    pic_pp[row+2][col+3]==*ccp && pic_pp[row+2][col]==*ccp)
                    fprintf(filep, "%s(%d,%d);\n",
                            *ccp=='#'?"exit":"gold", xx, yy);
            sig_ngram_p[1]=xx;
            sig_ngram_p[2]=yy;
            CHK_FAIL(qsmm_actor_calc_action_prob, actor,
                     0, 0, 0, QSMM_PROB_LEARNED);
            const double *const prob_p=
                qsmm_get_actor_choice_sig_prob(actor);
            double prob_max=0, ww[DIRECT_COUNT];
            for (ii=0; ii<DIRECT_COUNT; ii++) {
                const double prob=prob_p[ii];
                if (prob_max<prob) prob_max=prob;
                ww[ii]=prob;
            }
            for (ii=0; ii<DIRECT_COUNT; ii++) ww[ii]/=prob_max;
            fprintf(filep, "arrows(%s,%d,%d,%.2f,%.2f,%.2f,%.2f);\n",
                    last_path[(int) is_gf][yy][xx]?"true":"false",
                    xx, yy, ww[0], ww[1], ww[2], ww[3]);
            qsmm_actor_choice_sig_prob_release(actor);
        }
    fprintf(filep,"entry(%d,%d);\n",ENTRY_X,ENTRY_Y);
    if (ferror(filep)) ERREXIT("%s: failed to write to the file",fln);
    result=0;

Exit:
    if (filep) fclose(filep);
    return result;
}


// Make a movement in the labyrinth in a specified direction possibly
// showing the movement on the screen.
// Returns: 3 = movement resulted in exiting the labyrinth;
//          2 = movement resulted in taking the gold;
//          1 = movement made successfully;
//          0 = cannot make the movement because a new location cell of the
//              agent is not empty.

static int
opaque_maze(
    enum direct_e direct
) {

    static const char *picture[]={
        // 0  1  2  3  4  5  6  7  8  9 10 11 12 13 14
        "+--------+-----+--------------------------#..#",
        "|        |     |                             |",  // 0
        "|  *  *  |     |                          #  #",
        "|        |     |                             |",  // 1
        "|  *  *  |     |     +-----+                 |",
        "|        |     |     |     |                 |",  // 2
        "|        |     |     |     |     +-----+     |",
        "|        |     |     |     |     |     |     |",  // 3
        "|        |     |     |     |     |     |     |",
        "|        |     |     |     |     |     |     |",  // 4
        "|        +-----+     |     |     |     |     |",
        "|                    |     |     |     |     |",  // 5
        "|                    |     |     |     |     |",
        "|                    |     |     |     |     |",  // 6
        "|  +-----------+     +-----+     |     |     |",
        "|  |           |                 |     |     |",  // 7
        "|  |           |                 |     |     |",
        "|  |           |                 |     |     |",  // 8
        "|  +-----------+     +-----+     |     |     |",
        "|                    |     |     |     |     |",  // 9
        "|                    |     |     |     |     |",
        "|                    |     |     |     |     |",  // 10
        "+--------------+     |     |     +-----+     |",
        "|              |     |     |                 |",  // 11
        "|              |     |     |                 |",
        "|              |     |     |                 |",  // 12
        "+--------------+     +-----+     +-----+     |",
        "|                                |     |     |",  // 13
        "|                                |     |     |",
        "|                                |     |     |",  // 14
        "+..+-----------------------------+-----+-----+"
    };

    static char is_gf=0;
    static int path_len, visit=0, xx=-1, yy=-1;
    int ii, col, row;
    if (xx<0 || yy<0) {
        if (visit==NVISIT-1) {
            if (!initscr()) exit(2);
            noecho();
            for (row=0; row<(MAX_Y+1)*2+1; row++)
                mvaddstr(row,0,picture[row]);
        }
        xx=ENTRY_X;
        yy=ENTRY_Y;
        path_len=0;
    }
    assert(xx>=0 && xx<=MAX_X);
    assert(yy>=0 && yy<=MAX_Y);
    if (visit==NVISIT-1) last_path[(int) is_gf][yy][xx]=1;
    col=xx*3+1;
    row=yy*2+1;
    assert(picture[row][col]==' ' && picture[row][col+1]==' ');
    assert((picture[row-1][col]==' ' && picture[row-1][col+1]==' ') ||
           (picture[row-1][col]!=' ' && picture[row-1][col+1]!=' '));
    assert((picture[row+1][col]==' ' && picture[row+1][col+1]==' ') ||
           (picture[row+1][col]!=' ' && picture[row+1][col+1]!=' '));
    switch (direct) {
        case DIRECT_NORTH:
            if (picture[row-1][col]!=' ' || picture[row-1][col+1]!=' ')
                return 0;
            yy--;
            break;

        case DIRECT_EAST:
            if (picture[row][col+2]!=' ') return 0;
            xx++;
            break;

        case DIRECT_SOUTH:
            if (picture[row+1][col]!=' ' || picture[row+1][col+1]!=' ')
                return 0;
            yy++;
            break;
        case DIRECT_WEST:
            if (picture[row][col-1]!=' ') return 0;
            xx--;
            break;
        default:
            assert(0);
    }
    path_len++;
    if (visit==NVISIT-1) mvaddstr(row,col,"  ");
    col=xx*3+1;
    row=yy*2+1;
    state_fq[2][yy][xx]++;
    state_fq[(int) is_gf][yy][xx]++;
    for (ii=0; ii<3; ii++)
        if (fq_max[ii]<state_fq[ii][yy][xx])
            fq_max[ii]=state_fq[ii][yy][xx];
    if (visit==NVISIT-1) {
        char ss[64];
        const int picture_w=strlen(picture[0]);
        mvaddstr(row,col,"[]");
        sprintf(ss," Gold found: %d",is_gf);
        mvaddstr(1,picture_w+2,ss);
        sprintf(ss,"Path length: %d",path_len);
        mvaddstr(3,picture_w+2,ss);
        move((MAX_Y+1)*2+3,0);
        refresh();
        usleep(125000);
    }
    int result=1;
    for (const char *ccp="#*"; *ccp; ccp++)
        if (picture[row-1][col-1]==*ccp && picture[row-1][col+2]==*ccp &&
            picture[row+1][col-1]==*ccp && picture[row+1][col+2]==*ccp) {
            if (*ccp=='#') {
                is_gf=0;
                result=3;
                xx=-1;
                yy=-1;
                if (visit==NVISIT-1) {
                    row=(MAX_Y+1)*2+3;
                    mvaddstr(row,0,"Press [Q] to exit");
                    while (1) {
                        int key=getch();
                        if (key=='q' || key=='Q') break;
                    }
                    for (ii=0; ii<2; ii++)
                        if (create_pic(picture,ii)<0) exit(2);
                    endwin();
                }
                else visit++;
            }
            else if (!is_gf) {
                is_gf=1;
                result=2;
            }
            break;
        }
    return result;
}


// Helper function for computing the relative probability of an
// output signal.

static double
relprob_user2(
    qsmm_actor_t actor,
    qsmm_sig_t sig_cycle,
    const struct qsmm_state_s *state_p,
    const struct qsmm_cycle_s *cycle_p,
    const struct qsmm_sspval_s *sspval_p,
    const struct qsmm_cspval_s *cspval_p,
    void *paramp
) {
    const double period_sum_c=cspval_p[1].delta_sum;
    return (period_sum_c?cspval_p[0].delta_sum/period_sum_c:0)/
        qsmm_get_actor_ktemperature(actor);
}


// Find a path to gold in the labyrinth and then to its exit.

int
main(
    int argc,
    char **argv
) {
    int exit_code=1;
    struct qsmm_pair_sig_s range_sig[3];
    memset(range_sig,0,sizeof(range_sig));
    range_sig[0].second=1;
    range_sig[1].second=MAX_X;
    range_sig[2].second=MAX_Y;
    struct qsmm_actor_desc_s actor_desc;
    memset(&actor_desc,0,sizeof(actor_desc));
    actor_desc.nspur=1;
    actor_desc.ntime=1;
    actor_desc.ngram_sz=3;
    actor_desc.compat=3;
    actor_desc.nsig_in=NSIG_IN;
    actor_desc.nsig_out=DIRECT_COUNT;
    actor_desc.range_sig_p=range_sig;
    CHK_FAIL(qsmm_actor_create,&actor_desc,&actor);
    qsmm_set_actor_relprob_type(actor,QSMM_RELPROB_USER2);
    qsmm_set_actor_relprob_helper(actor,&relprob_user2,0);
    const int seed=(argc>1?atoi(argv[1]):0);
    if (seed<0) qsmm_set_actor_random(actor,1);
    qsmm_rng_seed(qsmm_get_actor_rng(actor),abs(seed));
    int n_gold_found=0;
    const double mut=pow(KT_1/KT_0,1.0/NVISIT);
    double ktemperature=KT_0;
    qsmm_sig_t *const sig_ngram_p=qsmm_get_actor_sig_ngram(actor);
    for (int visit=0; visit<NVISIT; visit++) {
        char is_gf=0;
        int xx=ENTRY_X, yy=ENTRY_Y;
        qsmm_sig_t sig_action=QSMM_SIG_INVALID;
        CHK_FAIL(qsmm_set_actor_ktemperature,actor,ktemperature);
        while (1) {
            sig_ngram_p[0]=is_gf;
            sig_ngram_p[1]=xx;
            sig_ngram_p[2]=yy;
            CHK_FAIL(qsmm_actor_reg_ngram_in,actor);
            CHK_FAIL(qsmm_actor_calc_action_prob, actor,
                     0, 0, 0, QSMM_PROB_LEARNED);
            CHK_FAIL(qsmm_get_actor_sig_action,actor,0,0,0,&sig_action);
            CHK_FAIL(qsmm_actor_reg_sig_out,actor,sig_action);
            CHK_FAIL(qsmm_actor_time_delta_v2,actor,0,0,1);
            const enum direct_e direct=sig_action;
            const int rc=opaque_maze(direct);
            switch (rc) {
                case 0:
                    continue;
                case 1:
                    break;
                case 2:
                    is_gf=1;
                    n_gold_found++;
                    break;
                case 3:
                    if (is_gf) CHK_FAIL(qsmm_actor_spur_delta,actor,0,1);
                    break;
                default:
                    assert(0);
            }
            if (rc==3) break;
            switch (direct) {
                case DIRECT_NORTH: yy--; break;
                case DIRECT_EAST:  xx++; break;
                case DIRECT_SOUTH: yy++; break;
                case DIRECT_WEST:  xx--; break;
                default: assert(0);
            }
        }
        ktemperature*=mut;
    }
    printf("\nn_gold_found %d\n",n_gold_found);
    exit_code=0;

Exit:
    qsmm_actor_destroy(actor);
    return exit_code;
}

3.1 Storage Handle ¶

A storage handle refers to a storage instance.

Data type: qsmm_storage_t ¶

This is a type for a storage handle. It is a pointer, so variables of this type can be NULL. The function qsmm_get_actor_storage returns the storage handle of an actor. You can pass a returned handle to API functions taking an argument of qsmm_storage_t type until destroying the actor.

Use the following function to query the number of cycle-summed value types supported by a storage instance referred to by a storage handle.

Function: int qsmm_get_storage_nspval (qsmm_storage_t storage) ¶

This function returns the number of cycle-summed value types supported by storage. That number is the sum of the number of spur types and the number of continuous time types supported by an actor owning storage. The indices of cycle-summed value types for spur types precede the indices of cycle-summed value types for continuous time types.

The field nspur of qsmm_actor_desc_s structure passed when creating an actor specifies the number of supported spur types. The function qsmm_get_actor_nspur returns that number for a created actor. The field ntime of qsmm_actor_desc_s structure passed when creating an actor specifies the number of supported continuous time types. The function qsmm_get_actor_ntime returns that number for a created actor.

A value returned by the function is always non-negative.

3.2 Structures for Accessing Storage ¶

The following structure holds the condition of an action choice state.

Structure: qsmm_state_s ¶

This structure holds the condition of an action choice state h (see Relative Probability Function Types, for formulas involving h). The structure contains the following fields.

Field: long tmd0 ¶

The discrete time of last occurrence of a cycle of <h,z> type in the event history, where the field sig_cycle_next of this structure specifies z. If the field tmd0 is non-negative, and sig_cycle_next is not QSMM_SIG_INVALID, tmd0 indicates the discrete time of last emitting output signal z in action choice state h. The function qsmm_actor_reg_sig_out records that discrete time. The default value of this field is −1.

Field: double nsig_ctrl ¶

The nominal number of output signals for the probability profile of this action choice state. See Number of Output Signals, for the explanation of the concept of the nominal number of output signals for an action choice state.

If nsig_ctrl is positive, it specifies custom nominal number of output signals. If the field nsig_pos of this structure is greater than 1, nsig_ctrl must be greater than 1 and less than or equal to nsig_pos. If nsig_pos is negative, nsig_ctrl must be equal to 1. The field nsig_ctrl must not be positive if nsig_pos is 0.

If nsig_ctrl is 0, this condition means to use implicit nominal number of output signals equal to the number of cycle directions with positive profile probabilities held in nsig_pos.

The default value of this field is 0.

Field: qsmm_sig_t sig_cycle_next ¶

An output signal indicating the direction of last cycle started at this action choice state and not yet processed. If the function qsmm_actor_reg_sig_out did not register a cycle started at this action choice state, or the function qsmm_actor_reg_ngram_in has already processed the cycle, then QSMM_SIG_INVALID. The default value is QSMM_SIG_INVALID.

Field: qsmm_ssig_t nsig_pos ¶

This field contains information on the number of cycle types <h,x> (i.e. starting at this action choice state h) with positive profile probabilities assigned, where x is a possible cycle direction. Meanings of various values of this field are below.

0

The action choice state does not have a probability profile specified.

i>1

The number i of cycle directions with positive profile probabilities assigned.

i<0

A single cycle direction has a positive profile probability assigned, and −i−1 is equal to identifier x of an output signal only allowed by the probability profile.

1

Disallowed.

This field is read-only when using the functions qsmm_set_storage_state_stats and qsmm_get_storage_state_stats: qsmm_set_storage_state_stats ignores the field, and qsmm_get_storage_state_stats calculates its value based on the number of cycle directions with positive profile probabilities specified in the field profile of qsmm_cycle_s structure for cycle types starting at this action choice state.

If storage does not contain the condition of the action choice state, and storage has a set redirection function for retrieving the initial condition of an action choice state, qsmm_get_storage_state_stats calls the redirection function, and it should set nsig_pos to a correct value. See Specifying Redirection Functions, for more information on the redirection function.

The condition of an action choice state includes an array of instances of a structure holding an initial cycle-summed value recorded by the function qsmm_actor_reg_sig_out for the last cycle started at the action choice state. The number of elements in the array is equal to the number of cycle-summed value types supported by storage. The array begins with initial cycle-summed values for spur followed by initial cycle-summed values for continuous time.

The structure contains only one field but is a structure to simplify associating other information with an action choice state and cycle-summed value type for research purposes. When storage prepares a structure instance for first use, it initializes the instance with zeroes by the function memset, so fields you may have added will initially be zero.

The structure description is below.

Structure: qsmm_sspval_s ¶

This structure holds condition associated with an action choice state and cycle-summed value type. The structure contains the following field.

Field: double val0 ¶

The current value of spur or continuous time recorded by the function qsmm_actor_reg_sig_out for the last cycle started at the action choice state when registering the emission of an output signal in the action choice state.

The following structure holds statistics on a cycle type.

Structure: qsmm_cycle_s ¶

This structure holds statistics on a cycle type—a pair comprised of an action choice state and a cycle direction (i.e. an output signal emittable in the action choice state). In Spur Perception, the notation <h,z> is a cycle type for action choice state h and cycle direction z. The structure contains the following fields.

Field: long fq ¶

The number of registered cycles of <h,z> type.

Field: long period_sum_d ¶

The total length of registered cycles of <h,z> type measured in discrete time. In Spur Perception, the notation ω^(h,z)  is that total length measured in discrete or continuous time.

Field: double profile ¶

A profile probability—weight to multiply the relative probability of output signal z selectable in action choice state h. When using a probability profile, the sum of profile probabilities of all cycle types for action choice state h must be equal to 1. When not using a probability profile, the field must be equal to 0.

The statistics on a cycle type includes an array of instances of a structure holding an accumulated cycle-summed value incremented on registering a cycle of this type by the function qsmm_actor_reg_ngram_in. The number of elements in the array is equal to the number of cycle-summed value types supported by storage. The array begins with accumulated cycle-summed values for spur followed by accumulated cycle-summed values for continuous time.

The structure contains only one field but is a structure to simplify associating other information with a cycle type and cycle-summed value type for research purposes. When storage prepares a structure instance for first use, it initializes the instance with zeroes by the function memset, so fields you may have added will initially be zero.

The structure description is below.

Structure: qsmm_cspval_s ¶

This structure holds statistics associated with cycle type <h,z> and a cycle-summed value type. The structure contains the following field.

Field: double delta_sum ¶

The sum of increments of spur or continuous time over cycles of <h,z> type registered by the function qsmm_actor_reg_ngram_in. Each spur increment is the difference between a current spur or continuous time value at a moment of calling qsmm_actor_reg_ngram_in and a spur or continuous time value recorded at a cycle start and stored in the field val0 of qsmm_sspval_s structure.

3.3 Retrieving, Storing, and Removing Statistics ¶

Use the following functions to retrieve or store the condition of an action choice state.

Function: int qsmm_get_storage_state_stats (qsmm_storage_t storage, int ngram_sz, const qsmm_sig_t *sig_ngram_p, struct qsmm_state_s *state_p, struct qsmm_sspval_s *sspval_p) ¶

This function retrieves from storage the condition of an action choice state encoded by an array of signal identifiers sig_ngram_p holding ngram_sz elements. If state_p is not NULL, the function copies the condition to *state_p. If sspval_p is not NULL, the function copies conditions for all cycle-summed value types associated with the action choice state to an array addressed by sspval_p. The array must be capable of holding the number of elements returned by the function qsmm_get_storage_nspval.

If storage does not contain the condition of the action choice state, and storage has a set redirection function for retrieving the initial condition of an action choice state, qsmm_get_storage_state_stats calls the redirection function to obtain the initial condition of the action choice state. When validating the field sig_cycle_next of qsmm_state_s structure obtained by the redirection function, qsmm_get_storage_state_stats may also call a redirection function for retrieving the initial statistics of a cycle type. See Specifying Redirection Functions, for more information on the redirection functions.

If storage contained the condition of the action choice state, or storage did not contain the condition, but the redirection function has retrieved it, qsmm_get_storage_state_stats returns a positive value. If storage did not contain the condition, and the redirection function has not retrieved it, qsmm_get_storage_state_stats returns 0 and the constant default condition. On failure, qsmm_get_storage_state_stats returns a negative error code. Currently, this function can return the following error codes.

QSMM_ERR_INVAL

The storage does not support length ngram_sz for arrays of signal identifiers encoding action choice states.

QSMM_ERR_NGRAM

The storage does not support holding information on an action choice state encoded by the array sig_ngram_p. A set of action choice states supported by storage is a set of action choice states supported by an actor owning the storage.

QSMM_ERR_STORAGE

The storage did not contain the condition of the action choice state, and a redirection function called to obtain the initial condition of the action choice state has reported QSMM_ERR_STORAGE or an unexpected error code, or the redirection function succeeded, but the function qsmm_get_storage_cycle_stats called to validate the field sig_cycle_next of qsmm_state_s structure obtained by that redirection function has reported QSMM_ERR_STORAGE.

See Getting the Reason of a Storage Failure, for how to get an error message describing the failure.

QSMM_ERR_STATS

A redirection function for retrieving the initial condition of an action choice state reported QSMM_ERR_STATS, or condition retrieved by the redirection function was inconsistent.

QSMM_ERR_ILSEQ

A redirection function for retrieving the initial condition of an action choice state or the initial statistics of a cycle type reported QSMM_ERR_ILSEQ, or a generated error message was not convertible to a wide string according to a current locale.

QSMM_ERR_NOMEM

There was not enough memory to perform the operation.

Function: int qsmm_set_storage_state_stats (qsmm_storage_t storage, int ngram_sz, const qsmm_sig_t *sig_ngram_p, const struct qsmm_state_s *state_p, const struct qsmm_sspval_s *sspval_p) ¶

This function writes to storage the condition of an action choice state encoded by an array of signal identifiers sig_ngram_p holding ngram_sz elements. If state_p is not NULL, the function copies the condition from *state_p. If sspval_p is not NULL, the function copies conditions for all cycle-summed value types associated with the action choice state from an array addressed by sspval_p. The function qsmm_get_storage_nspval returns the number of elements of sspval_p array copied.

The function qsmm_set_storage_state_stats may call a redirection function for obtaining the initial condition of the action choice state when adding information on it to storage. The function qsmm_set_storage_state_stats may also call a redirection function for obtaining the initial statistics of a cycle type when validating the field sig_cycle_next of qsmm_state_s structure. See Specifying Redirection Functions, for more information on the redirection functions.

The function qsmm_set_storage_state_stats returns a non-negative value on success or a negative error code on failure. Currently, the function can return the following error codes.

QSMM_ERR_INVAL

The storage does not support length ngram_sz for arrays of signal identifiers encoding action choice states.

QSMM_ERR_NGRAM

The storage does not support holding information on an action choice state encoded by the array sig_ngram_p. A set of action choice states supported by storage is a set of action choice states supported by an actor owning the storage.

QSMM_ERR_STORAGE

The function qsmm_get_storage_cycle_stats called to validate the field sig_cycle_next of *state_p or validate the field sig_cycle_next of an instance of qsmm_state_s structure obtained by a redirection function for retrieving the initial condition of an action choice state reported QSMM_ERR_STORAGE, or the redirection function reported QSMM_ERR_STORAGE or an unexpected error code.

See Getting the Reason of a Storage Failure, for how to get an error message describing the failure.

QSMM_ERR_STATS

Inconsistent condition in *state_p or an element of sspval_p array, or a redirection function for retrieving the initial condition of an action choice state reported QSMM_ERR_STATS, or condition retrieved by the redirection function was inconsistent.

Condition in qsmm_state_s is inconsistent in the following cases:

• nsig_ctrl is not finite;
• nsig_ctrl is negative;
• the action choice state has a single cycle direction allowed by a probability profile, but nsig_ctrl is not 0 and 1;
• the action choice state has multiple cycle directions allowed by a probability profile, and nsig_ctrl is not 0 but is less than or equal to 1 or is greater than the number of cycle directions allowed by the probability profile;
• sig_cycle_next is not QSMM_SIG_INVALID but also not an output signal of an actor owning the storage;
• sig_cycle_next is not QSMM_SIG_INVALID, and tmd0 is negative;
• sig_cycle_next is not QSMM_SIG_INVALID, the action choice state has a probability profile assigned, and the function qsmm_get_storage_cycle_stats called to check the presence of statistics on a cycle type for the action choice state and cycle direction sig_cycle_next returned 0 or reported QSMM_ERR_STATS.

Condition in an element of sspval_p array is inconsistent if the field val0 of the element is not finite.

QSMM_ERR_ILSEQ

A redirection function for retrieving the initial condition of an action choice state or the initial statistics of a cycle type reported QSMM_ERR_ILSEQ, or a generated error message was not convertible to a wide string according to a current locale.

QSMM_ERR_NOMEM

There was not enough memory to perform the operation.

To update the condition of an action choice state, you first need to retrieve existing condition by the function qsmm_get_storage_state_stats, then change some fields, and, finally, store the condition by the function qsmm_set_storage_state_stats. For example, you can reset the field tmd in the condition of an action choice state using a block of code like this:

int rc;
struct qsmm_state_s state;
if ((rc=
     qsmm_get_storage_state_stats(storage, ngram_sz, sig_ngram_p,
                                  &state, 0))<0)
    REPORT_ERROR(rc);
state.tmd=-1;

if ((rc=
     qsmm_set_storage_state_stats(storage, ngram_sz, sig_ngram_p,
                                  &state, 0))<0)
    REPORT_ERROR(rc);

To satisfy validity checks performed by the function qsmm_set_storage_state_stats for the fields nsig_ctrl and sig_cycle_next of qsmm_state_s structure, a call to that function should go after a series of calls to the function qsmm_set_storage_cycle_stats described further on in this section. This is necessary because the validity checks depend on the value of nsig_pos field automatically calculated based on previous calls to qsmm_set_storage_cycle_stats. If nsig_pos is non-zero, sig_cycle_next should be QSMM_SIG_INVALID or specify the direction of a cycle type with statistics held in storage.

Use the following functions to retrieve or store statistics on a cycle type.

Function: int qsmm_get_storage_cycle_stats (qsmm_storage_t storage, int ngram_sz, const qsmm_sig_t *sig_ngram_p, qsmm_sig_t sig_cycle, struct qsmm_cycle_s *cycle_p, struct qsmm_cspval_s *cspval_p) ¶

This function retrieves statistics on a cycle type from storage. An action choice state and cycle direction sig_cycle specify the cycle type. An array of signal identifiers sig_ngram_p holding ngram_sz elements encodes the action choice state.

If cycle_p is not NULL, the function copies the statistics to *cycle_p. If cspval_p is not NULL, the function copies statistics on all cycle-summed value types associated with the cycle type to an array addressed by cspval_p. The array must be capable of holding the number of elements returned by the function qsmm_get_storage_nspval.

If storage does not contain statistics on the cycle type, and storage has a set redirection function for retrieving the initial statistics of a cycle type, qsmm_get_storage_cycle_stats calls the redirection function to obtain initial statistics on the cycle type. See Specifying Redirection Functions, for more information on the redirection function.

If storage contained statistics on the cycle type, or storage did not contain the statistics, but the redirection function has retrieved it, qsmm_get_storage_cycle_stats returns a positive value. If storage did not contain the statistics, and the redirection function has not retrieved it, qsmm_get_storage_cycle_stats returns 0 and zero statistics. On failure, qsmm_get_storage_cycle_stats returns a negative error code. Currently, this function can return the following error codes.

QSMM_ERR_INVAL

The storage does not support length ngram_sz for arrays of signal identifiers encoding action choice states, or sig_cycle is not an output signal of an actor owning the storage.

QSMM_ERR_NGRAM

The storage does not support holding information on an action choice state encoded by the array sig_ngram_p. A set of action choice states supported by storage is a set of action choice states supported by an actor owning the storage.

QSMM_ERR_STORAGE

The storage did not contain statistics on the cycle type, and a redirection function called to obtain initial statistics on the cycle type has reported QSMM_ERR_STORAGE or an unexpected error code. See Getting the Reason of a Storage Failure, for how to get an error message describing the failure.

QSMM_ERR_STATS

A redirection function for retrieving the initial statistics of a cycle type reported QSMM_ERR_STATS, or statistics retrieved by the redirection function was inconsistent.

QSMM_ERR_ILSEQ

A redirection function for retrieving the initial statistics of a cycle type reported QSMM_ERR_ILSEQ, or a generated error message was not convertible to a wide string according to a current locale.

QSMM_ERR_NOMEM

There was not enough memory to perform the operation.

Function: int qsmm_set_storage_cycle_stats (qsmm_storage_t storage, int ngram_sz, const qsmm_sig_t *sig_ngram_p, qsmm_sig_t sig_cycle, const struct qsmm_cycle_s *cycle_p, const struct qsmm_cspval_s *cspval_p) ¶

This function writes statistics on a cycle type to storage. An action choice state and cycle direction sig_cycle specify the cycle type. An array of signal identifiers sig_ngram_p holding ngram_sz elements encodes the action choice state.

If cycle_p is not NULL, the function copies the statistics from *cycle_p. If cspval_p is not NULL, the function copies statistics for all cycle-summed value types associated with the cycle type from an array addressed by cspval_p. The function qsmm_get_storage_nspval returns the number of elements of cspval_p array copied.

The function qsmm_set_storage_cycle_stats calls a redirection function for the interception of updating statistics on a cycle type if storage has the redirection function set. See Specifying Redirection Functions, for more information on the redirection function. The function qsmm_set_storage_cycle_stats may call redirection functions for obtaining the initial condition of the action choice state or the initial statistics of the cycle type when adding information on it to storage. See Specifying Redirection Functions, for more information on the redirection functions.

The function qsmm_set_storage_cycle_stats returns a non-negative value on success or a negative error code on failure. Currently, the function can return the following error codes.

QSMM_ERR_INVAL

The storage does not support length ngram_sz for arrays of signal identifiers encoding action choice states, or sig_cycle is not an output signal of an actor owning the storage.

QSMM_ERR_NGRAM

The storage does not support holding information on an action choice state encoded by the array sig_ngram_p. A set of action choice states supported by storage is a set of action choice states supported by an actor owning the storage.

QSMM_ERR_STORAGE

A storage redirection function reported QSMM_ERR_STORAGE or an unexpected error code. See Getting the Reason of a Storage Failure, for how to get an error message describing the failure.

QSMM_ERR_STATS

Inconsistent statistics in *cycle_p or an element of cspval_p array, or a storage redirection function reported QSMM_ERR_STATS, or statistics obtained from a storage redirection function was inconsistent.

Statistics in qsmm_cycle_s is inconsistent in the following cases:

• fq is negative;
• period_sum_d is negative;
• profile is not finite;
• profile is negative;
• profile is greater than 1;
• fq is zero, but period_sum_d is positive.

Statistics in an element of cspval_p array is inconsistent if the field delta_sum of the element is not finite.

QSMM_ERR_ILSEQ

A storage redirection function reported QSMM_ERR_ILSEQ, or a generated error message was not convertible to a wide string according to a current locale.

QSMM_ERR_NOMEM

There was not enough memory to perform the operation.

To update statistics on a cycle type, you first need to retrieve existing statistics by the function qsmm_get_storage_cycle_stats, then change some fields, and, finally, store the statistics by the function qsmm_set_storage_cycle_stats. For example, you can update a profile probability² for a cycle type to a new value profile using a block of code like this:

int rc;
struct qsmm_cycle_s cycle;
if ((rc=
     qsmm_get_storage_cycle_stats(storage, ngram_sz, sig_ngram_p,
                                  sig_cycle, &cycle, 0))<0)
    REPORT_ERROR(rc);
cycle.profile=profile;

if ((rc=
     qsmm_set_storage_cycle_stats(storage, ngram_sz, sig_ngram_p,
                                  sig_cycle, &cycle, 0))<0)
    REPORT_ERROR(rc);

At present, an actor uses storage to hold statistics on action choice states encoded by arrays of signal identifiers with a fixed number of elements. The argument ngram_sz of Storage API functions must be equal to this fixed length. In the future, storage may hold statistics on action choice states encoded by arrays of signal identifiers with varying lengths, so that argument may have multiple valid values.

Use the following function to remove information on an action choice state from storage.

Function: int qsmm_storage_remove_state (qsmm_storage_t storage, int ngram_sz, const qsmm_sig_t *sig_ngram_p) ¶

This function removes from storage information on an action choice state encoded by an array of signal identifiers sig_ngram_p holding ngram_sz elements. The information includes the condition of the action choice state and statistics on all types of cycles starting at the action choice state.

The function returns a non-negative value on success or a negative error code on failure. Currently, the function can return the following error codes.

QSMM_ERR_INVAL

The storage does not support length ngram_sz for arrays of signal identifiers encoding action choice states.

QSMM_ERR_NGRAM

The storage does not support holding information on an action choice state encoded by the array sig_ngram_p. A set of action choice states supported by storage is a set of action choice states supported by an actor owning the storage.

QSMM_ERR_NOTFOUND

Information on the action choice state not found in the storage—nothing to remove.

3.4 Enumerating Action Choice States and Cycle Types ¶

Use the following function to enumerate action choice states that have the pieces of information held in storage.

Function: int qsmm_storage_enum_states_v2 (qsmm_storage_t storage, int ngram_lower_sz, int ngram_upper_sz, int range_sz, int rez1, const qsmm_sig_t *sig_ngram_lower_p, const qsmm_sig_t *sig_ngram_upper_p, const struct qsmm_pair_sig_s *range_sig_p, qsmm_enum_state_callback_func_t callback_func, void *paramp) ¶

This function enumerates lists of signal identifiers encoding action choice states that have the pieces of information held in storage.

If ngram_lower_sz is positive, it specifies the length of sig_ngram_lower_p array containing a lower bound for the enumerated lists—their prefixes with length ngram_lower_sz must be greater than or equal to the lower bound. If ngram_upper_sz is positive, it specifies the length of sig_ngram_upper_p array containing an upper bound for the enumerated lists—their prefixes with length ngram_upper_sz must be less than the upper bound. If range_sz is positive, it specifies the length of range_sig_p array containing allowed ranges of signal identifiers in prefixes with length range_sz in the enumerated lists—every signal identifier in a prefix must be greater than or equal to the field first of an element of range_sig_p at the same index and must be less than or equal to the field second of the element.

The function compares two lists of signal identifiers from left to right. If a signal identifier in the first list is less than or greater than a signal identifier in the second list at the same index, the first list is less than or greater than the second list respectively. If the first list is a prefix of the second list, the first list is less than the second list. If the second list is a prefix of the first list, the first list is greater than the second list. If the two lists have the same length and content, the lists are equal.

The process of enumeration is repeated calling a callback function callback_func with passing an array of signal identifiers encoding an enumerated action choice state in ascending order of the arrays. The callback function also receives a user parameter paramp. If the callback function returns a positive value, qsmm_storage_enum_states_v2 continues enumeration. If the callback function returns zero, qsmm_storage_enum_states_v2 terminates enumeration and reports success. If the callback function returns a negative value, qsmm_storage_enum_states_v2 terminates enumeration and reports failure.

The function qsmm_storage_enum_states_v2 does not support recursive calling from the callback function for storage.

The function returns a non-negative value on success or a negative error code on failure. Currently, the function can return the following error codes.

QSMM_ERR_INVAL

One of the following conditions is true:

• ngram_lower_sz is negative;
• ngram_lower_sz is greater than the length of a list of signal identifiers encoding an action choice state;
• ngram_lower_sz is positive, but sig_ngram_lower_p is NULL;
• ngram_upper_sz is negative;
• ngram_upper_sz is greater than the length of a list of signal identifiers encoding an action choice state;
• ngram_upper_sz is positive, but sig_ngram_upper_p is NULL;
• range_sz is negative;
• range_sz is greater than the length of a list of signal identifiers encoding an action choice state;
• range_sz is positive, but range_sig_p is NULL;
• field first in an element of range_sig_p is greater than the field second of the element;
• field second in an element of range_sig_p is greater than QSMM_SIG_MAX.

QSMM_ERR_CALLBACK

The callback function reported an error.

The type of a pointer to a callback function called for every enumerated action choice state is below.

Data type: qsmm_enum_state_callback_func_t ¶

This is a type of a callback function pointer with the following declaration:

typedef int
(*qsmm_enum_state_callback_func_t)(
    qsmm_storage_t storage,
    int ngram_sz,
    const qsmm_sig_t *sig_ngram_p,
    void *paramp
);

The function qsmm_storage_enum_states_v2 calls the callback function for every enumerated action choice state of storage. The argument sig_ngram_p specifies an array of signal identifiers encoding an enumerated action choice state. The argument ngram_sz specifies the length of the array. The argument paramp is a user parameter passed to qsmm_storage_enum_states_v2.

The callback function shall return a positive value to continue the process of enumeration, zero to terminate the process of enumeration, or a negative value on error.

Use a function described below to enumerate types of cycles starting at an action choice state. The action choice state and a cycle direction make up an enumerated cycle type.

Function: int qsmm_get_storage_cycle_next (qsmm_storage_t storage, int ngram_sz, const qsmm_sig_t *sig_ngram_p, qsmm_sig_t *sig_cycle_next_p) ¶

This function retrieves the next cycle type that has a piece of information held in storage. An action choice state and cycle direction specify the cycle type. An array of signal identifiers sig_ngram_p holding ngram_sz elements encodes the action choice state.

If *sig_cycle_next_p is QSMM_SIG_INVALID, then on successful function completion, *sig_cycle_next_p will contain the first cycle direction. It has a minimum value among existing cycle directions. If *sig_cycle_next_p is not QSMM_SIG_INVALID, then on successful function completion, *sig_cycle_next_p will contain the next cycle direction greater than a cycle direction specified in *sig_cycle_next_p when calling the function. If the first or next cycle direction does not exist, then on successful function completion, *sig_cycle_next_p will be QSMM_SIG_INVALID.

If storage has a set redirection function for retrieving the next cycle type of an action choice state, qsmm_get_storage_cycle_next calls the redirection function to obtain the next cycle type. If the redirection function reports that it has not retrieved the next cycle type, qsmm_get_storage_cycle_next proceeds as if storage does not have the redirection function set. See Specifying Redirection Functions, for more information on the redirection function.

The function returns a non-negative value on success or a negative error code on failure. Currently, the function can return the following error codes.

QSMM_ERR_INVAL

The storage does not support length ngram_sz for arrays of signal identifiers encoding action choice states.

QSMM_ERR_NGRAM

The storage does not support holding information on an action choice state encoded by the array sig_ngram_p. A set of action choice states supported by storage is equal to a set of action choice states supported by an actor owning the storage.

QSMM_ERR_STORAGE

The storage redirection function reported QSMM_ERR_STORAGE or an unexpected error code. See Getting the Reason of a Storage Failure, for how to get an error message describing the failure.

QSMM_ERR_STATS

The storage redirection function reported QSMM_ERR_STATS, or a cycle direction obtained by that redirection function was not QSMM_SIG_INVALID and was greater than or equal to the number of output signals of an actor owning the storage or was less than or equal to *sig_cycle_next_p if it was not QSMM_SIG_INVALID.

QSMM_ERR_ILSEQ

The storage redirection function reported QSMM_ERR_ILSEQ, or a generated error message was not convertible to a wide string according to a current locale.

QSMM_ERR_NOMEM

There was not enough memory to perform the operation.

3.5 Specifying Redirection Functions ¶

Before first use, Storage API functions initialize by the function memset with zeroes the instances of qsmm_state_s, qsmm_sspval_s, qsmm_cycle_s, and qsmm_cspval_s structures holding conditions for action choice states and statistics on cycle types. For every instance of qsmm_state_s structure, Storage API functions also perform the following assignments: set the field tmd0 to −1 and the field sig_cycle_next to QSMM_SIG_INVALID.

An application program can provide redirection functions called by Storage API functions just after the aforementioned initialization. The redirection functions can perform application-specific initialization. Storage API functions will call the redirection functions on accessing condition for an action choice state or statistics on a cycle type when storage does not contain requested information about the action choice state or cycle type.

Storage redirection functions performing application-specific initialization may set the parameters of a probability profile for an action choice state. They may also call Storage API functions for other action choice states, for example, to copy the parameters of a probability profile from another action choice state.

Data type: qsmm_get_state_stats_func_t ¶

This is a type for the pointer to a redirection function for application-specific initialization of condition of an action choice state. The type has the following declaration:

typedef int
(*qsmm_get_state_stats_func_t)(
    qsmm_storage_t storage,
    int ngram_sz,
    const qsmm_sig_t *sig_ngram_p,
    struct qsmm_state_s *state_p,
    struct qsmm_sspval_s *sspval_p,
    void *paramp
);

The argument storage is a storage handle with the redirection function set. On calling the function, *state_p and the array sspval_p contain preinitialized values. The function can set *state_p and/or the elements of sspval_p array to application-specific initial condition of an action choice state. An array of signal identifiers sig_ngram_p holding ngram_sz elements encodes the action choice state. The elements of sspval_p array contain initial cycle-summed values. The function qsmm_get_storage_nspval returns the number of elements in the array. The argument paramp is a user parameter specified when setting the redirection function for storage.

If the function has performed the redirection and possibly modified preinitialized values in *state_p or the array sspval_p, the function shall return a positive value. If the function has not performed the redirection and has left the preinitialized values intact, the function shall return 0. On failure, the function shall return one of the following negative error codes.

QSMM_ERR_STORAGE

Storage failure. Before reporting this error, the function shall add to the storage message list at least one message describing the reason of the failure. See Getting the Reason of a Storage Failure, for more information on the storage message list.

QSMM_ERR_STATS

Inconsistent statistics on an action choice state or cycle type detected.

QSMM_ERR_ILSEQ

Unable to convert a multibyte string to a wide string or vice versa according to a current locale.

QSMM_ERR_NOMEM

There was not enough memory to perform the operation.

Use the following functions to retrieve or set a redirection function for application-specific initialization of condition of an action choice state.

Function: int qsmm_get_storage_state_stats_redir (qsmm_storage_t storage, qsmm_get_state_stats_func_t *get_state_stats_func_p, void **param_pp) ¶

This function retrieves a previously set redirection function for application-specific initialization of conditions of action choice states in storage. If get_state_stats_func_p is not NULL, qsmm_get_storage_state_stats_redir sets *get_state_stats_func_p to a pointer to the redirection function or to NULL if storage does not have the redirection function assigned. If param_pp is not NULL, qsmm_get_storage_state_stats_redir sets *param_pp to a user parameter of that redirection function specified when assigning it to storage. The function qsmm_get_storage_state_stats_redir returns a non-negative value.

Function: int qsmm_set_storage_state_stats_redir (qsmm_storage_t storage, qsmm_get_state_stats_func_t get_state_stats_func, void *paramp) ¶

This function sets a redirection function for application-specific initialization of conditions of action choice states in storage. The argument get_state_stats_func specifies a pointer to the redirection function. The argument paramp specifies the user parameter of the redirection function. If get_state_stats_func is NULL, storage will not use a redirection function for application-specific initialization of conditions of action choice states. The function qsmm_set_storage_state_stats_redir returns a non-negative value.

Data type: qsmm_get_cycle_stats_func_t ¶

This is a type for the pointer to a redirection function for application-specific initialization of statistics on a cycle type. The pointer type has the following declaration:

typedef int
(*qsmm_get_cycle_stats_func_t)(
    qsmm_storage_t storage,
    int ngram_sz,
    const qsmm_sig_t *sig_ngram_p,
    qsmm_sig_t sig_cycle,
    struct qsmm_cycle_s *cycle_p,
    struct qsmm_cspval_s *cspval_p,
    void *paramp
);

The argument storage is a storage handle with the redirection function set. On calling the function, *cycle_p and the array cspval_p contain preinitialized values. The function can set *cycle_p and/or the elements of cspval_p array to application-specific initial statistics on a cycle type. An action choice state and output signal sig_cycle indicating a cycle direction specify the cycle type. An array of signal identifiers sig_ngram_p holding ngram_sz elements encodes the action choice state. The elements of cspval_p array contain accumulated cycle-summed values. The function qsmm_get_storage_nspval returns the number of elements in the array. The argument paramp is a user parameter specified when setting the redirection function for storage.

If the function has performed the redirection and possibly modified preinitialized values in *cycle_p or the array cspval_p, the function shall return a positive value. If the function has not performed the redirection and has left the preinitialized values intact, the function shall return 0. On failure, the function shall return one of the following negative error codes.

QSMM_ERR_STORAGE

Storage failure. Before reporting this error, the function shall add to the storage message list at least one message describing the reason of the failure. See Getting the Reason of a Storage Failure, for more information on the storage message list.

QSMM_ERR_STATS

Inconsistent statistics on an action choice state or cycle type detected.

QSMM_ERR_ILSEQ

Unable to convert a multibyte string to a wide string or vice versa according to a current locale.

QSMM_ERR_NOMEM

There was not enough memory to perform the operation.

Use the following functions to retrieve or set a redirection function for application-specific initialization of statistics on a cycle type.

Function: int qsmm_get_storage_cycle_stats_redir (qsmm_storage_t storage, qsmm_get_cycle_stats_func_t *get_cycle_stats_func_p, void **param_pp) ¶

This function retrieves a previously set redirection function for application-specific initialization of statistics on cycle types in storage. If get_cycle_stats_func_p is not NULL, qsmm_get_storage_cycle_stats_redir sets *get_cycle_stats_func_p to a pointer to the redirection function or to NULL if storage does not have the redirection function assigned. If param_pp is not NULL, qsmm_get_storage_cycle_stats_redir sets *param_pp to a user parameter of that redirection function specified when assigning it to storage. The function qsmm_get_storage_cycle_stats_redir returns a non-negative value.

Function: int qsmm_set_storage_cycle_stats_redir (qsmm_storage_t storage, qsmm_get_cycle_stats_func_t get_cycle_stats_func, void *paramp) ¶

This function sets a redirection function for application-specific initialization of statistics on cycle types in storage. The argument get_cycle_stats_func specifies a pointer to the redirection function. The argument paramp specifies the user parameter of the redirection function. If get_cycle_stats_func is NULL, storage will not use a redirection function for application-specific initialization of statistics on cycle types. The function qsmm_set_storage_cycle_stats_redir returns a non-negative value.

If an application sets a redirection function for custom initialization of statistics on cycle types, the application may need to provide a redirection function for retrieving the next cycle type for an action choice state. The latter redirection function enables correct enumeration of cycle types with initial statistics provided by the former redirection function. Enumerating cycle types that do not yet have information held in storage is necessary for generating probability profiles of action choice states on demand, that is, on read access to information about action choice states and their cycle types before the first write access to the information.

Data type: qsmm_get_cycle_next_func_t ¶

This is a type for the pointer to a redirection function for retrieving the next cycle type for an action choice state. The pointer type has the following declaration:

typedef int
(*qsmm_get_cycle_next_func_t)(
    qsmm_storage_t storage,
    int ngram_sz,
    const qsmm_sig_t *sig_ngram_p,
    qsmm_sig_t *sig_cycle_next_p,
    void *paramp
);

The argument storage is a storage handle with the redirection function assigned passed to the function qsmm_get_storage_cycle_next. An array of signal identifiers sig_ngram_p holding ngram_sz elements encodes the action choice state. The argument paramp is a user parameter specified when setting the redirection function for storage.

If *sig_cycle_next_p is QSMM_SIG_INVALID, then on successfully performed redirection, *sig_cycle_next_p shall contain a minimum output signal that along with the action choice state specifies the first cycle type for the action choice state. If *sig_cycle_next_p is not QSMM_SIG_INVALID, then on successfully performed redirection, *sig_cycle_next_p shall contain the next output signal greater than a signal specified in *sig_cycle_next_p when calling the redirection function; that output signal along with the action choice state shall specify the next cycle type for the action choice state. If the first or next cycle type does not exist, then on successful function completion, *sig_cycle_next_p shall be QSMM_SIG_INVALID.

If the function has performed the redirection and possibly updated *sig_cycle_next_p, the function shall return a positive value. If the function has not performed the redirection and has left *sig_cycle_next_p intact, the function shall return 0; in this case, qsmm_get_storage_cycle_next retrieves the next cycle type without the redirection. On failure, the redirection function shall return one of the following negative error codes.

QSMM_ERR_STORAGE

Storage failure. Before reporting this error, the function shall add to the storage message list at least one message describing the reason of the failure. See Getting the Reason of a Storage Failure, for more information on the storage message list.

QSMM_ERR_STATS

Inconsistent statistics on an action choice state or cycle type detected.

QSMM_ERR_ILSEQ

Unable to convert a multibyte string to a wide string or vice versa according to a current locale.

QSMM_ERR_NOMEM

There was not enough memory to perform the operation.

Use the following functions to retrieve or set a redirection function for obtaining the next cycle type for an action choice state.

Function: int qsmm_get_storage_cycle_next_redir (qsmm_storage_t storage, qsmm_get_cycle_next_func_t *get_cycle_next_func_p, void **param_pp) ¶

This function retrieves a previously set redirection function for obtaining the next cycle type that has a piece of information held in storage. If get_cycle_next_func_p is not NULL, qsmm_get_storage_cycle_next_redir sets *get_cycle_next_func_p to a pointer to the redirection function or to NULL if storage does not have the redirection function assigned. If param_pp is not NULL, qsmm_get_storage_cycle_next_redir sets *param_pp to a user parameter of that redirection function specified when assigning it to storage. The function qsmm_get_storage_cycle_next_redir returns a non-negative value.

Function: int qsmm_set_storage_cycle_next_redir (qsmm_storage_t storage, qsmm_get_cycle_next_func_t get_cycle_next_func, void *paramp) ¶

This function sets a redirection function for obtaining the next cycle type that has a piece of information held in storage. The argument get_cycle_next_func specifies a pointer to the redirection function. The argument paramp specifies the user parameter of the redirection function. If get_cycle_next_func is NULL, storage will not use a redirection function for obtaining the next cycle type. The function qsmm_set_storage_cycle_next_redir returns a non-negative value.

An application program can provide a redirection function for intercepting updates of statistics on cycle types to organize keeping the statistics only for latest events in the event history rather than for the entire event history. Keeping statistics for a tail of the event history might make a state model less stable thereby improving the response of a system to latest tendencies in the event history. By making a state model less stable, a system might also try more state model variations and then select a variation that occurs more frequently.

The function qsmm_set_storage_cycle_stats calls a storage redirection function for intercepting updates of statistics on a cycle type if storage has the redirection function set. See Example of Using the Storage API, for an example redirection function for intercepting updates of statistics on a cycle type, an example helper function for calculating the relative probability of an output signal, and a function that sets the two example functions for a large actor.

The type of a pointer to the redirection function is below.

Data type: qsmm_update_cycle_stats_func_t ¶

This is a type for the pointer to a redirection function for intercepting updates of statistics on cycle types. The pointer type has the following declaration:

typedef int
(*qsmm_update_cycle_stats_func_t)(
    qsmm_storage_t storage,
    int ngram_sz,
    const qsmm_sig_t *sig_ngram_p,
    qsmm_sig_t sig_cycle,
    const struct qsmm_cycle_s *cycle_new_p,
    struct qsmm_cycle_s *cycle_result_p,
    const struct qsmm_cspval_s *cspval_new_p,
    struct qsmm_cspval_s *cspval_result_p,
    void *paramp
);

The argument storage is a storage handle with the redirection function assigned passed to the function qsmm_set_storage_cycle_stats. An action choice state and output signal sig_cycle indicating a cycle direction specify a cycle type. An array of signal identifiers sig_ngram_p holding ngram_sz elements encodes the action choice state.

New statistics on the cycle type is in *cycle_new_p and an array cspval_new_p. On calling the function, current statistics on the cycle type is in *cycle_result_p and an array cspval_result_p. The arrays cspval_new_p and cspval_result_p hold the number of elements returned by the function qsmm_get_storage_nspval. The redirection function can compute the difference between *cycle_new_p and *cycle_result_p and between the elements of cspval_new_p and cspval_result_p arrays.

To use *cycle_result_p and the array cspval_result_p as new statistics on the cycle type, the function shall return a positive value; in this case, the function can change *cycle_result_p and the array cspval_result_p in its own way. To replace *cycle_result_p with *cycle_new_p, replace the elements of cspval_result_p with the elements of cspval_new_p, and use *cycle_result_p and the array cspval_result_p as new statistics on the cycle type, the function shall return 0. On failure, the function shall return one of the following negative error codes.

QSMM_ERR_STORAGE

Storage failure. Before reporting this error, the function shall add to the storage message list at least one message describing the reason of the failure. See Getting the Reason of a Storage Failure, for more information on the storage message list.

QSMM_ERR_STATS

Inconsistent statistics on an action choice state or cycle type detected.

QSMM_ERR_ILSEQ

Unable to convert a multibyte string to a wide string or vice versa according to a current locale.

QSMM_ERR_NOMEM

There was not enough memory to perform the operation.

Use the following functions to retrieve or set a redirection function for intercepting updates of statistics on cycle types.

Function: int qsmm_get_storage_cycle_update_hook (qsmm_storage_t storage, qsmm_update_cycle_stats_func_t *update_cycle_stats_func_p, void **param_pp) ¶

This function retrieves a previously set redirection function for intercepting updates of statistics on cycle types by the function qsmm_set_storage_cycle_stats called for storage. If update_cycle_stats_func_p is not NULL, qsmm_get_storage_cycle_update_hook sets *update_cycle_stats_func_p to a pointer to the redirection function or to NULL if storage does not have the redirection function assigned. If param_pp is not NULL, qsmm_get_storage_cycle_update_hook sets *param_pp to a user parameter of that redirection function specified when assigning it to storage. The function qsmm_get_storage_cycle_update_hook returns a non-negative value.

Function: int qsmm_set_storage_cycle_update_hook (qsmm_storage_t storage, qsmm_update_cycle_stats_func_t update_cycle_stats_func, void *paramp) ¶

This function sets a redirection function for intercepting updates of statistics on cycle types by the function qsmm_set_storage_cycle_stats called for storage. The argument update_cycle_stats_func specifies a pointer to the redirection function. The argument paramp specifies the user parameter of the redirection function. If update_cycle_stats_func is NULL, storage will not use a redirection function for intercepting updates of statistics on cycle types. The function qsmm_set_storage_cycle_update_hook returns a non-negative value.

3.6 Getting the Reason of a Storage Failure ¶

When a Storage API function returns error code QSMM_ERR_STORAGE, the function clears a storage message list and adds to it at least one error message describing a failure.

A handle of qsmm_msglist_t type represents a message list. See Messages and Message Lists, for how to work with message lists. Use the following function to obtain a message list associated with a storage instance.

Function: qsmm_msglist_t qsmm_get_storage_msglist (qsmm_storage_t storage) ¶

This function returns the handle of a message list associated with storage. The function never returns NULL.

To dump to stderr the message list of storage in an application program with an executable named prg_name, use lines of code like these:

const int rc=qsmm_msglist_dump(qsmm_get_storage_msglist(storage),
                               prg_name, 0, 0, stderr);
if (rc<0) REPORT_ERROR(rc);

If an Actor API function returns error code QSMM_ERR_STORAGE, and the actor is the small one, the message list of a storage instance owned by the actor contains at least one error message describing a storage failure. Use the function qsmm_get_actor_storage to get the handle of a storage instance owned by the small actor.

If an Actor API function returns error code QSMM_ERR_STORAGE, and the actor is the large one, then a failed storage instance can be the storage instance of the actor itself or the storage instance of an associated small actor representing the environment state identification engine or the storage instance of an associated small actor representing the instruction emitting engine. Provided that all storage message lists were empty before calling the Actor API function, one of them would be non-empty and would contain a message describing the failure.

The following function returns the handle of failed storage instance of a small or large actor or NULL if the storage message lists are empty:

qsmm_storage_t
get_err_storage(
    qsmm_actor_t actor
) {
    qsmm_storage_t storage=qsmm_get_actor_storage(actor);
    if (qsmm_get_msglist_sz(qsmm_get_storage_msglist(storage))) return storage;
    const qsmm_t qsmm_large=qsmm_get_actor_large_model(actor);
    if (!qsmm_large) return 0;
    const qsmm_actpair_t actpair=qsmm_get_actpair(qsmm_large);
    if ((storage=
         get_err_storage(qsmm_get_actpair_actor(actpair,QSMM_ENGINE_ENV))))
        return storage;
    if ((storage=
         get_err_storage(qsmm_get_actpair_actor(actpair,QSMM_ENGINE_IEE))))
        return storage;
    return 0;
}

3.7 Example of Using the Storage API ¶

As mentioned in Specifying Redirection Functions, achieving acceptable optimality of emitting output signals by an actor may require it to keep statistics for a tail of the event history rather than for the entire event history. If an actor learns a state model, keeping statistics for a tail of the event history makes a current state model less stable allowing the actor to try more state model variations. After trying state model variations with counting frequencies of state transitions performed, an application program can partially determinize a current state model by disallowing some less frequent state transitions and continue state model training. A state model fully determinized at the end of iterative partial determinization becomes a learned deterministic state model.

The file samples/cyc_stats_upd_hook.c in the package distribution installed to $prefix/share/qsmm/samples/cyc_stats_upd_hook.c contains:

• an example redirection function cycle_stats_update_hook for intercepting updates of statistics on a cycle type to implement keeping cycle type statistics for an event history tail;
• an example helper function relprob_user2 for calculating the relative probability of an output signal based on cycle type statistics for an event history tail;
• the function setup_actor setting the functions cycle_stats_update_hook and relprob_user2 for a large actor.

All functions use an allocated instance of hook_param_s structure initially zeroed and passed to setup_actor. This section includes the source code of the three functions and associated macros and data structures.

To organize storing statistics on cycle types for a tail of the event history of a small actor associated with the large actor, cycle_stats_update_hook keeps a queue of records held in the instances of histev_cycle_s structure, where every record contains the following information about an update of statistics on a cycle type:

step

virtual time of the update designated as “modeling step”

spur

spur accumulated by the actor

tmc

continuous time of the update

sig_ngram

signal array encoding an action choice state that along with a cycle direction sig_cycle specifies the cycle type

sig_cycle

cycle direction that along with an action choice state encoded by sig_ngram specifies the cycle type

fq_delta

increment to the field fq of qsmm_cycle_s structure equal to a difference between a new value and old value passed to the redirection function

period_sum_d_delta

increment to the field period_sum_d of qsmm_cycle_s structure equal to a difference between a new value and old value passed to the redirection function

period_sum_c_delta

increments to continuous time in the field delta_sum in the elements of array of qsmm_cspval_s structures for continuous time types equal to differences between new values and old values passed to the redirection function

spur_delta_sum

increments to spur in the field delta_sum in the elements of array of qsmm_cspval_s structures for spur types equal to differences between new values and old values passed to the redirection function

The redirection function does the following:

1. Prevents recursive calls to itself when calling the function qsmm_set_storage_cycle_stats to decrement statistics on cycle types.
2. Removes too old records from the queue with decrementing statistics on cycle types specified by removed records and with decrementing spur accumulated by the actor according to the removed records. A removed record contains differences to statistics on the cycle type and differences to spur tracked by the actor relative to a previously added record. After removing the old records, storage contains statistics on cycle types for an event history tail, and the actor holds current spur values accumulated for the event history tail.
3. Adds a new record to the queue.

An application program should keep the fields step_histev and nstep_histev in the instance of hook_param_s structure up-to-date while performing state model training. The field step_histev holds a modeling step index equal, for example, to the number of nondeterministic state transitions performed. The field nstep_histev holds the length of an event history tail in modeling steps. Calculating the length of an event history tail to keep statistics is out of scope of this section.

#include <assert.h>
#include <math.h>
#include <stdlib.h>
#include <string.h>

#include <qsmm/qsmm.h>


#define HISTEV_CYCLE_NSPUR    2  // number of spur types
#define HISTEV_CYCLE_NTIME    2  // number of continuous time types

#define HISTEV_CYCLE_NGRAM_SZ 4
    // length of action choice state n-grams of a small actor associated
    // with a large actor


#define GET_NSTEP_HISTEV(hook_param_p)                                    \
    ((long) ((hook_param_p)->nstep_histev+0.5))

#define ERREXIT(fmt, ...)                                                 \
    do {                                                                  \
        fprintf(stderr,fmt "\n", ## __VA_ARGS__);                         \
        goto Exit;                                                        \
    }                                                                     \
    while (0)

#define ERR_NOMEM()  ERREXIT("out of memory")


#define CHK_FAIL(func, ...)                                               \
    do {                                                                  \
        const int rc=func(__VA_ARGS__);                                   \
        if (rc<0) ERREXIT(#func ": %s", qsmm_err_str(rc));                \
    }                                                                     \
    while (0)


struct histev_cycle_s {

    long                  step;
        // modeling step

    long                  fq_delta;
        // increment of cycle type frequency

    long                  period_sum_d_delta;
        // increment of the sum of discrete time periods

    double                spur[HISTEV_CYCLE_NSPUR];
        // actor spur:
        // [ii] = spur with type `ii'

    double                tmc[HISTEV_CYCLE_NTIME];
        // continuous time:
        // [ii] = continuous time with type `ii'

    double                period_sum_c_delta[HISTEV_CYCLE_NTIME];
        // increments of the sums of continuous time periods:
        // [ii] = increment of the sum of periods of continuous
        //        time with type `ii'

    double                spur_delta_sum[HISTEV_CYCLE_NSPUR];
        // increments of the sums of spur increments:
        // [ii] = increment of the sum of spur increments for
        //        spur type `ii'

    qsmm_sig_t            sig_cycle;
        // cycle direction

    qsmm_sig_t            sig_ngram[HISTEV_CYCLE_NGRAM_SZ];
        // action choice state n-gram

    struct histev_cycle_s *nextp;
        // next event in the list;
        // special value:
        // ==NULL - this event is the last event in the list
};

struct hook_param_s {

    char                  in_cycle_update_hook;
        // cycle update hook execution indicator:
        // !=0 - cycle statistics update hook is being executed;
        // ==0 - not executed

    long                  step_histev;
        // modeling step

    double                nstep_histev;
        // number of modeling steps within the cycle event history window

    qsmm_actor_t          actor_small;
        // reference to a small actor corresponding to the environment
        // state identification engine

    struct histev_cycle_s *histev_cycle_head_p;
        // oldest event in the history of last cycle events;
        // special value:
        // ==NULL - history is empty

    struct histev_cycle_s *histev_cycle_tail_p;
        // newest event in the history of last cycle events;
        // special value:
        // ==NULL - history is empty
};


// The hook for calling on updating cycle type statistics.
// Returns: 0 = success; no return on failure.

static int
cycle_stats_update_hook(
    qsmm_storage_t storage,
    int ngram_sz,
    const qsmm_sig_t *sig_ngram_p,             // [ngram_sz]
    qsmm_sig_t sig_cycle,
    const struct qsmm_cycle_s *cycle_new_p,
    struct qsmm_cycle_s *cycle_result_p,
    const struct qsmm_cspval_s *cspval_new_p,  // [nspval]
    struct qsmm_cspval_s *cspval_result_p,     // [nspval]
    void *paramp
) {
    int result=-1;
    struct histev_cycle_s *histev_new_p=0;
    struct hook_param_s *const hook_param_p=paramp;
    if (hook_param_p->in_cycle_update_hook) {
        result=0;
        goto Exit;
    }
    hook_param_p->in_cycle_update_hook=1;
    const long step_beg=
        hook_param_p->step_histev-GET_NSTEP_HISTEV(hook_param_p);
    const qsmm_actor_t actor_small=hook_param_p->actor_small;
    const int nspur=qsmm_get_actor_nspur(actor_small),
        ntime=qsmm_get_actor_ntime(actor_small),
        nspval=qsmm_get_storage_nspval(storage);
    int ispur, itime;
    struct histev_cycle_s *histev_old_p;
    assert(ngram_sz==HISTEV_CYCLE_NGRAM_SZ);
    assert(nspur==HISTEV_CYCLE_NSPUR);
    assert(ntime==HISTEV_CYCLE_NTIME);
    assert(nspur+ntime==nspval);
    while ((histev_old_p=hook_param_p->histev_cycle_head_p) &&
           histev_old_p->step<step_beg) {
        const double *const spur_delta_sum_p=histev_old_p->spur_delta_sum;
        const qsmm_sig_t sig_cycle_old=histev_old_p->sig_cycle,
            *const sig_ngram_old_p=histev_old_p->sig_ngram;
        struct qsmm_cycle_s cycle;
        struct qsmm_cspval_s cspval[nspval];
        CHK_FAIL(qsmm_get_storage_cycle_stats, storage, ngram_sz,
                 sig_ngram_old_p, sig_cycle_old, &cycle, cspval);
        cycle.fq-=histev_old_p->fq_delta;
        cycle.period_sum_d-=histev_old_p->period_sum_d_delta;
        assert(cycle.fq>=0);
        assert(cycle.period_sum_d>=0);
        for (ispur=0; ispur<nspur; ispur++)
            cspval[ispur].delta_sum-=spur_delta_sum_p[ispur];
        for (itime=0; itime<ntime; itime++) {
            cspval[nspur+itime].delta_sum-=
                histev_old_p->period_sum_c_delta[itime];
            assert(cspval[nspur+itime].delta_sum>=0);
        }
        CHK_FAIL(qsmm_set_storage_cycle_stats, storage, ngram_sz,
                 sig_ngram_old_p, sig_cycle_old, &cycle, cspval);
        hook_param_p->histev_cycle_head_p=histev_old_p->nextp;
        free(histev_old_p);
    }
    if (!histev_old_p) hook_param_p->histev_cycle_tail_p=0;
    if (!(histev_new_p=calloc(1,sizeof(*histev_new_p)))) ERR_NOMEM();
    histev_new_p->step=hook_param_p->step_histev;
    histev_new_p->sig_cycle=sig_cycle;
    if (cycle_new_p) {
        histev_new_p->fq_delta=cycle_new_p->fq-cycle_result_p->fq;
        histev_new_p->period_sum_d_delta=cycle_new_p->period_sum_d-
                                         cycle_result_p->period_sum_d;
        assert(histev_new_p->fq_delta>=0);
        assert(histev_new_p->period_sum_d_delta>=0);
    }
    for (ispur=0; ispur<nspur; ispur++) {
        CHK_FAIL(qsmm_get_actor_spur, actor_small, ispur,
                 histev_new_p->spur+ispur);
        histev_new_p->spur_delta_sum[ispur]=
            cspval_new_p?cspval_new_p[ispur].delta_sum-
                         cspval_result_p[ispur].delta_sum:0;
    }
    for (itime=0; itime<ntime; itime++) {
        CHK_FAIL(qsmm_get_actor_time,
                 actor_small, itime, histev_new_p->tmc+itime);
        histev_new_p->period_sum_c_delta[itime]=
            cspval_new_p?cspval_new_p[nspur+itime].delta_sum-
                         cspval_result_p[nspur+itime].delta_sum:0;
        assert(histev_new_p->period_sum_c_delta[itime]>=0);
    }
    memmove(histev_new_p->sig_ngram, sig_ngram_p,
            ngram_sz*sizeof(*sig_ngram_p));
    if (hook_param_p->histev_cycle_tail_p) {
        assert(!hook_param_p->histev_cycle_tail_p->nextp);
        hook_param_p->histev_cycle_tail_p->nextp=histev_new_p;
    }
    else {
        assert(!hook_param_p->histev_cycle_head_p);
        hook_param_p->histev_cycle_head_p=histev_new_p;
    }
    hook_param_p->histev_cycle_tail_p=histev_new_p;
    histev_new_p=0;
    hook_param_p->in_cycle_update_hook=0;
    result=0;

Exit:
    if (histev_new_p) free(histev_new_p);
    if (result<0) exit(1);
    return result;
}

// The helper function for computing the relative probability of an
// output signal.

static double
relprob_user2(
    qsmm_actor_t actor,
    qsmm_sig_t sig_cycle,
    const struct qsmm_state_s *state_p,
    const struct qsmm_cycle_s *cycle_p,
    const struct qsmm_sspval_s *sspval_p,
    const struct qsmm_cspval_s *cspval_p,
    void *paramp
) {

    const long fq=cycle_p->fq;
    double result=0/0.0;
    if (fq<1) {
        result=0;
        goto Exit;
    }
    const struct histev_cycle_s *const histev_head_p=
        ((struct hook_param_s *) paramp)->histev_cycle_head_p;
    const int nspur=qsmm_get_actor_nspur(actor);
    double exp_arg=0;
    assert(histev_head_p);
    for (int ispur=0; ispur<nspur; ispur++) {
        int itime=-1;
        CHK_FAIL(qsmm_get_actor_spur_time,actor,ispur,&itime);
        assert(itime>=0);
        double spur_val=0, time_val=0;
        CHK_FAIL(qsmm_get_actor_spur,actor,ispur,&spur_val);
        CHK_FAIL(qsmm_get_actor_time,actor,itime,&time_val);
        spur_val-=histev_head_p->spur[ispur];
        time_val-=histev_head_p->tmc[itime];
        const double spur_delta_mean=time_val>0?spur_val/time_val:0;
        double spur_weight=0;
        CHK_FAIL(qsmm_get_actor_spur_weight,actor,ispur,&spur_weight);
        const double num=cspval_p[ispur].delta_sum,
            den=cspval_p[nspur+itime].delta_sum*fabs(spur_delta_mean);
        enum qsmm_spur_perception_e spur_perception;
        CHK_FAIL(qsmm_get_actor_spur_perception, actor,
                 ispur, &spur_perception);
        switch (spur_perception) {
            case QSMM_SPUR_PERCEPTION_NORMAL:
                if (den) exp_arg+=spur_weight*num/den;
                break;
            case QSMM_SPUR_PERCEPTION_INVERSE:
                if (num) exp_arg+=-spur_weight*den/num;
                break;
        }
    }
    assert(cycle_p->period_sum_d>0);
    exp_arg*=log(qsmm_get_actor_nsig_ctrl(actor))*cycle_p->period_sum_d/fq;
    result=exp_arg;

Exit:
    return result;
}


// Set up a hook for updating cycle type statistics and a helper function
// for computing the relative probability of an output signal for a
// large actor.
// Returns: 0 = success;
//         -1 = failure.

int
setup_actor(
    qsmm_actor_t actor_large,
    struct hook_param_s *hook_param_p
) {
    int result=-1;
    qsmm_set_actor_relprob_type(actor_large,QSMM_RELPROB_USER2);
    qsmm_set_actor_relprob_helper(actor_large,&relprob_user2,hook_param_p);
    const qsmm_actor_t actor_small=
        qsmm_get_actpair_actor(
            qsmm_get_actpair(qsmm_get_actor_large_model(actor_large)),
            QSMM_ENGINE_ENV);
    CHK_FAIL(qsmm_set_storage_cycle_update_hook,
             qsmm_get_actor_storage(actor_small),
             &cycle_stats_update_hook, hook_param_p);
    hook_param_p->actor_small=actor_small;
    hook_param_p->step_histev=0;
    result=0;

Exit:
    return result;
}

4.1 Principle of Operation ¶

A schematic diagram of multinode model implementation is in Figure 4.1.

An environment state identification engine, instruction emitting engine, and instruction execution environment are main model components taking part in model execution.

The environment state identification engine is a small or large actor keeping an adaptive state model of an environment and identifying node states according to the model. This identification is a bidirectional process: the current state of a node depends on the state of the environment and needs identification; conversely, the node acts on the environment, and, therefore, the state of the environment depends on an identified node state. For a single-node model, its node corresponds to the entire environment. For a multinode model, a node corresponds to part of the environment.

Along with the adaptive state model, the environment state identification engine keeps the identifier of a currently executed node, the index of its previous state, the identifier of the last instruction or the last instruction sequence emitted by the instruction emitting engine, the outcome of that instruction or instruction sequence, an optional array of identifiers of look-ahead signals (see Setting Look-Ahead Signals), and, as an actor, it keeps current values of spur and continuous time. For a single-node model, the identifier of a currently executed node is always equal to 0. The structure of tuples encoding action choice states of the environment state identification engine is in Figure 4.2.

The environment state identification engine identifies the current state of a node based on its identifier, the previous state of the node, the identifier of the last instruction or the last instruction sequence emitted, the outcome of that instruction or instruction sequence, and optional look-ahead signals. The environment state identification engine sends the identifier of a currently executed node and the index of its identified state to the instruction emitting engine. The latter engine is a small or large actor keeping an adaptive model of emitting various assembler instructions and their sequences in various node states. As an actor, the instruction emitting engine keeps current values of spur and continuous time, and they can be different from current values of spur and continuous time for the environment state identification engine.

The structure of tuples encoding action choice states of the instruction emitting engine is in Figure 4.3. The instruction emitting engine selects an output signal encoding an assembler instruction or a sequence of assembler instructions. The identifier of a currently executed node and a current node state received from the environment state identification engine take part in selecting the output signal.

The instruction emitting engine sends a selected assembler instruction or a selected sequence of assembler instructions to the instruction execution environment for performing effective work and to the environment state identification engine for identifying the next node state. That instruction execution environment represents everything else in the multinode model, interacts with application logic or incorporates it, and is aware of all parameters of the environment state identification engine and instruction emitting engine. An assembler instruction or a sequence of assembler instructions returns an instruction outcome after execution. The instruction execution environment sends the outcome to the environment state identification engine for identifying the next node state.

The instruction execution environment keeps a node call stack for the execution of nodes to be similar to the execution of program subroutines (functions or procedures) calling each other and returning control to caller subroutines. An executed assembler instruction can push the identifier of a currently executed node and the index of its current state to the node call stack and transfer control to another node with resetting the current node state to the initial node state corresponding to the beginning of execution of a subroutine. The assembler instruction can also return control to a previously executed node by popping the identifier of the node and the index of its current state from the stack.

An assembler instruction can change look-ahead signals taking part in identifying the next node state and increment current values of spur and continuous time tracked by the environment state identification engine and instruction emitting engine.

Figure 4.1: multinode model implementation

Figure 4.2: action choice state of environment state identification engine

Figure 4.3: action choice state of instruction emitting engine

A helpful feature of a multinode model is ability to convert internal data stored in its environment state identification engine and instruction emitting engine to a representation in the form of an assembler program (see Assembler Programs). One can think about this feature as automatic synthesis of an assembler program solving an assigned task.

An important point the developer should realize is that automatic program synthesis may work only if there exists a steady state model solving an assigned task. Steady state model is a model where states have predictable input from an environment.

Below there is a list of concepts for enabling the automatic synthesis of assembler programs.

Instruction meta-class ¶

Represents an instruction name without taking into account optional instruction parameters following the instruction name after a whitespace character. The name of an instruction meta-class is the name of an instruction without including instruction parameters. Example:

move

This instruction meta-class might represent an instruction that moves an agent in an environment one step in a specific direction. The name of the instruction meta-class does not include a movement direction.

Individual instruction class ¶

Represents an instruction name optionally followed by instruction parameters. Distinct strings, where every string consists of an instruction name and instruction parameters in canonical form (see Setting Text Instruction Parameters), identify distinct individual instruction classes. Example:

move north

This instruction class might represent an instruction that moves an agent in the environment one step in the north direction.

Instruction class sequence ¶

A sequence of individual instruction classes. Example:

move west
move west
move south

This instruction class sequence might move an agent two steps in the west direction and then one step in the south direction.

Instruction outcome ¶

An individual instruction class has a specific number of instruction outcomes. The outcome of a sequence of instructions is the outcome of the last instruction in the sequence.

The environment state identification engine uses the outcome of an emitted instruction or an emitted sequence of instructions to identify the next node state. The instruction emitting engine uses the next node state to select the next instruction or the next sequence of instructions.

For example, the instruction ‘move north’ as well as other movement instructions could return a bitmask of four bits (i.e. an integer number in the range 0 to 15) after performing a move. A bit of the bitmask equal to 0 could indicate that there was an obstacle in a corresponding direction, thus giving a piece of information to the agent about the configuration of an environment. It might also be useful to return special outcome 16 if there was an obstacle in a direction specified by an emitted instruction, and the obstacle disallowed the agent to move.

Instruction class set ¶

Represents a set containing individual instruction classes and sequences of instruction classes along with their properties and is a class for nodes. For example, the instruction class set ‘walker’ intended for investigating an environment might contain the individual instruction classes

move north
move east
move south
move west

the instruction class sequence

move west
move west
move south

and the instruction class sequence

move north
move east

A node belonging to this node class might represent an agent investigating an environment.

Instruction instance ¶

An instance of an instruction class in an assembler program. For example, in the assembler program fragment

        move north
        joe  14, return_back
        move north
        ...
return_back:
        move south
        ...

there are two instances of ‘move north’ instruction class.

4.2 Creating a Multinode Model ¶

A handle of qsmm_t type refers to a multinode model. The function qsmm_create creates a multinode model according to specified initial parameters and returns its handle. The function qsmm_destroy destroys a multinode model specified by its handle. A number of functions fetch initial model parameters after creating a model. The handle of a multinode model can have a number of arbitrary pointers associated with it.

After creating an empty model, an application program should register instruction meta-classes and instruction class sets defining assembler instruction sets for model nodes. Instruction meta-classes specify instruction-centric model behavior. Instruction class sets specify node-centric model behavior. An instruction meta-class is an entity for an assembler instruction name. An individual instruction class belongs to an instruction meta-class, is a member of an instruction class set, and represents an assembler instruction with specific parameters. Every instruction meta-class requires providing an event handler function. An instruction class set can have an event handler function too.

After registering one or more instruction class sets, an application program should create nodes belonging to node classes represented by the instruction class sets. Every node has a numeric identifier, a state transition matrix maintained by the environment state identification engine, and an action emission matrix maintained by the instruction emitting engine. Both engines exist in a model instance scope holding model execution parameters. After creating a model instance, it may be necessary to override default parameters of actors representing the engines.

• Creating a Handle
• Defining Instruction Meta-Classes
• Defining Instruction Class Sets
• Node Parameters
• Creating Nodes
• Creating the Model Instance
• Associating Parameters with a Model

static QSMM_INSTR_META_CLASS(move);

static QSMM_INSTR_META_CLASS(move) {
    ...
}

enum direct_e {
    DIRECT_NORTH,  // move one step up
    DIRECT_EAST,   // move one step right
    DIRECT_SOUTH,  // move one step down
    DIRECT_WEST,   // move one step left
    DIRECT_COUNT   // number of movement directions
};

...

static QSMM_INSTR_META_CLASS(move) {
    enum direct_e direct=DIRECT_COUNT;
    if (QSMM_HAS_INSTR_CLASS(mehcall->evt))
        qsmm_get_mehcall_instr_param_bin(
            mehcall, sizeof(direct), 0, &direct);
    ...
}