Next: , Up: Example Programs   [Contents][Index]


7.1 optact

This program demonstrates a behavior of an actor when it emits more frequently output signals immediately followed by greater mean spur increments. However, the program does not reveal a generic aspect of actor’s behavior when the frequency of emitting an output signal also depends on spur increments that were made at later time after emitting the output signal.

In the example program, the actor randomly receives one of signal vectors <3, 0, 4>, <1, 2, 5>, <5, 3, 3>, <2, 4, 0>, and <4, 2, 1>, which represent action choice states shown in Figure 2.1 (and one additional action choice state), and emits one of four output signals. The total number of signal vectors received and output signals emitted is defined by the macro NSTEP (that has value 40000).

For each action choice state vector, fixed probabilities are assigned to four possible output signals. According to those probabilities, the actor receives spur increment 1 when it emits corresponding output signals. The expected actor behavior is to emit more frequently output signals, to which greater probabilities correspond.

The researcher may want to change the problem statement and to make an actor emit output signals with frequencies proportional to the fixed probabilities. In this case, instead of making spur increment 1 according to a fixed probability, spur increment equal to the logarithm of the fixed probability should be made. There are also other specifics that should be considered when solving this another kind of problem. See a program described in optact-p, where the problem is solved. For example, run that program using command line

$ ./optact-p -i1 -a -n40000 --dump-prob=learnt 5 4

At the end of its run, the optact program prints a table with the numbers of output signals emitted in each action choice state, i.e. with frequencies of those output signals. The frequencies are printed near fixed probabilities of output signals specified in the program source text. The program also dumps the contents of statistics storage of the actor.

A random seed can be specified by a program argument. If the random seed is non-negative, then the actor will operate normally. If the random seed is negative, then the actor will generate output signals completely randomly. You could compare the resulting frequencies of output signals for these two modes of program execution.

Below is sample program output for the completely random mode of operation of the program.

$ ./optact -1
< 3 0 4 > : 0.2/1975 0.4/2004 0.3/2010 0.1/1917
< 1 2 5 > : 0.4/2062 0.1/2071 0.3/1902 0.2/1966
< 5 3 3 > : 0.2/2010 0.2/2031 0.2/2026 0.4/2000
< 2 4 0 > : 0.1/1993 0.3/2000 0.5/2032 0.1/2017
< 4 2 1 > : 0.3/2052 0.3/1924 0.2/2000 0.2/2008

State < 1 2 5 >: sig_cycle_next=7, tmd0=159975, tmc0=119982, s0=10097
  sig_next=6: fq=2062, pd_s=40584, pc_s=30438, sd_s=2903, ave=0.0953742
  sig_next=7: fq=2070, pd_s=40996, pc_s=30747, sd_s=2283, ave=0.0742511
  sig_next=8: fq=1902, pd_s=39252, pc_s=29439, sd_s=2646, ave=0.0898808
  sig_next=9: fq=1966, pd_s=39132, pc_s=29349, sd_s=2264, ave=0.0771406

State < 2 4 0 >: sig_cycle_next=7, tmd0=159967, tmc0=119976, s0=10097
  sig_next=6: fq=1993, pd_s=40632, pc_s=30474, sd_s=2262, ave=0.0742272
  sig_next=7: fq=1999, pd_s=39536, pc_s=29652, sd_s=2664, ave=0.0898422
  sig_next=8: fq=2032, pd_s=39988, pc_s=29991, sd_s=2968, ave=0.098963
  sig_next=9: fq=2017, pd_s=39788, pc_s=29841, sd_s=2199, ave=0.0736906

State < 3 0 4 >: sig_cycle_next=7, tmd0=159983, tmc0=119988, s0=10098
  sig_next=6: fq=1975, pd_s=39632, pc_s=29724, sd_s=2374, ave=0.0798681
  sig_next=7: fq=2003, pd_s=41184, pc_s=30888, sd_s=2894, ave=0.0936933
  sig_next=8: fq=2010, pd_s=39808, pc_s=29856, sd_s=2638, ave=0.0883574
  sig_next=9: fq=1917, pd_s=39340, pc_s=29505, sd_s=2189, ave=0.0741908

State < 4 2 1 >: sig_cycle_next=6, tmd0=159999, tmc0=120000, s0=10098
  sig_next=6: fq=2051, pd_s=41372, pc_s=31029, sd_s=2719, ave=0.0876277
  sig_next=7: fq=1924, pd_s=38692, pc_s=29019, sd_s=2526, ave=0.0870464
  sig_next=8: fq=2000, pd_s=41208, pc_s=30906, sd_s=2460, ave=0.0795962
  sig_next=9: fq=2008, pd_s=38720, pc_s=29040, sd_s=2393, ave=0.0824036

State < 5 3 3 >: sig_cycle_next=9, tmd0=159979, tmc0=119985, s0=10097
  sig_next=6: fq=2010, pd_s=39676, pc_s=29757, sd_s=2380, ave=0.0799812
  sig_next=7: fq=2031, pd_s=40344, pc_s=30258, sd_s=2421, ave=0.0800119
  sig_next=8: fq=2026, pd_s=40760, pc_s=30570, sd_s=2477, ave=0.0810272
  sig_next=9: fq=1999, pd_s=39196, pc_s=29397, sd_s=2819, ave=0.0958941

Below is sample program output for the normal mode of operation of the program.

$ ./optact 1
< 3 0 4 > : 0.2/1551 0.4/3253 0.3/2295 0.1/807
< 1 2 5 > : 0.4/3297 0.1/1333 0.3/2136 0.2/1235
< 5 3 3 > : 0.2/1797 0.2/1674 0.2/1321 0.4/3275
< 2 4 0 > : 0.1/1223 0.3/2031 0.5/3781 0.1/1007
< 4 2 1 > : 0.3/2506 0.3/2095 0.2/1624 0.2/1759

State < 1 2 5 >: sig_cycle_next=6, tmd0=159975, tmc0=119982, s0=11849
  sig_next=6: fq=3296, pd_s=64696, pc_s=48522, sd_s=5176, ave=0.106673
  sig_next=7: fq=1333, pd_s=26876, pc_s=20157, sd_s=1741, ave=0.086372
  sig_next=8: fq=2136, pd_s=44124, pc_s=33093, sd_s=3327, ave=0.100535
  sig_next=9: fq=1235, pd_s=24268, pc_s=18201, sd_s=1604, ave=0.088127

State < 2 4 0 >: sig_cycle_next=8, tmd0=159967, tmc0=119976, s0=11848
  sig_next=6: fq=1223, pd_s=26192, pc_s=19644, sd_s=1641, ave=0.083537
  sig_next=7: fq=2031, pd_s=39532, pc_s=29649, sd_s=2855, ave=0.0962933
  sig_next=8: fq=3780, pd_s=74580, pc_s=55935, sd_s=6164, ave=0.110199
  sig_next=9: fq=1007, pd_s=19640, pc_s=14730, sd_s=1185, ave=0.0804481

State < 3 0 4 >: sig_cycle_next=7, tmd0=159983, tmc0=119988, s0=11850
  sig_next=6: fq=1551, pd_s=30904, pc_s=23178, sd_s=2102, ave=0.0906894
  sig_next=7: fq=3252, pd_s=67292, pc_s=50469, sd_s=5375, ave=0.106501
  sig_next=8: fq=2295, pd_s=46132, pc_s=34599, sd_s=3396, ave=0.0981531
  sig_next=9: fq= 807, pd_s=15636, pc_s=11727, sd_s= 975, ave=0.0831415

State < 4 2 1 >: sig_cycle_next=6, tmd0=159999, tmc0=120000, s0=11850
  sig_next=6: fq=2505, pd_s=50620, pc_s=37965, sd_s=3852, ave=0.101462
  sig_next=7: fq=2095, pd_s=41828, pc_s=31371, sd_s=3172, ave=0.101112
  sig_next=8: fq=1624, pd_s=33220, pc_s=24915, sd_s=2354, ave=0.0944812
  sig_next=9: fq=1759, pd_s=34324, pc_s=25743, sd_s=2472, ave=0.0960261

State < 5 3 3 >: sig_cycle_next=9, tmd0=159979, tmc0=119985, s0=11849
  sig_next=6: fq=1797, pd_s=34976, pc_s=26232, sd_s=2448, ave=0.0933211
  sig_next=7: fq=1674, pd_s=34064, pc_s=25548, sd_s=2386, ave=0.0933928
  sig_next=8: fq=1321, pd_s=25768, pc_s=19326, sd_s=1796, ave=0.0929318
  sig_next=9: fq=3274, pd_s=65168, pc_s=48876, sd_s=5219, ave=0.10678

Next: , Up: Example Programs   [Contents][Index]