===================================
Soar 9.3.4 Release Notes, June 2014
===================================

This release of Soar continues the 9.3 line. It includes the ability to
include search control knowledge into chunks, a new form of integrated
Tcl support, improved episodic and semantic memory and a large
assortment of bug fixes and user interface enhancements. All learning
mechanisms are disabled by default.

Soar can be downloaded by following the download link on the Soar home
page at:

http://sitemaker.umich.edu/soar

Soar releases include source code, demo programs, and a number of
applications that serve as examples of how to interface Soar to an
external environment. There is support for integrating Soar with C++,
Java, Tcl and Python applications. Many tools that aid in development
of Soar programs are also available. The newly expanded download
section of the wiki allows you to browse and download all of the
different distributions, tools and agents.

[Help and Contact information]

You can find many helpful resources on the Soar wiki at:

https://code.google.com/p/soar/wiki/Home?tm=6

To contact the Soar group, you may join and post to one of our mailing
lists:

For general Soar-related discussion and announcements:

soar-group@lists.sourceforge.net

For more technical developer discussion:

soar-sml-list@lists.sourceforge.net

Also, please do not hesitate to file bugs on our issue tracker:

http://code.google.com/p/soar/issues/list

To avoid redundant entries, please search for duplicate issues first.

===========================
Important Changes for 9.3.4
===========================

--------
Chunking
--------

(1) Soar can now include search control knowledge in chunks.

This is done by keeping track of all *relevant* search control rules
that produced preferences that influenced the selection of the operator
that produced a result. This set of preferences, called the
context-dependent preference set (CDPS), will be backtraced through,
possibly producing additional conditions in a chunk. This setting is
off by default but can be enabled by 'learn --desirability-prefs' or
'learn -p'. This new feature can allow chunking to be used in situations
that would previously produce over-general chunks. See the Soar Manual
for more information.

----------------------------
Episodic and Semantic Memory
----------------------------

(1) Epmem search now uses less memory and computational resources.

(2) Default epmem storage trigger is now each decision cycle instead of
after output generation.

(3) Soar now uses a new schema for both databases. This means old
databases can no longer be loaded. A utility will soon be provided to
convert from the old schema to the new one.

(4) Users can now specify mathematical constraints to their smem
retrieval cues. See the manual for more information.

(5) Both systems now have a new append-database setting that determines
whether Soar will erase the database when starting or re-initializing
via init-soar.

- Databases are now closed upon init-soar. With append-mode on,
this will have no effect on memories.

- Soar will now warn users that initializing Soar can clear their
memories. Only shown when changing epmem/smem settings such that
they are storing to a file with append mode off.

- Soar will now notify users that there was no change to epmem/smem
memories when reinitializing if append mode is on.

(6) Both systems now handle database settings with more flexibility.
Users may change path/database/append-database settings arbitrarily
until they get what they'd like. It will not effect any changes until
you step Soar or do something that requires that memory system.

(7) Users can now switch from database to memory mode on the fly without
losing your data. (The opposite will not work though.)

(8) Soar will now no longer automatically switch to database mode the
first time that you set the path.

(9) SMem and epmem now have options for --on, --off, -e, -d, --enable,
--disable parameters to epmem and smem to make interface consistent with
other systems.

(10) Soar now has an 'epmem --init' command. This will close and
re-initialize the database like 'smem --init'.

--------------
User Interface
--------------

(1) TclSoarLib

- Soar now includes a dynamically loaded module that alters the Soar
command line interface to allow you to use Tcl code on the command
line and within your agent. For example, users can use Tcl
variables and functions directly within their soar code or make tcl
function calls on the RHS of rules.

- This mode is enabled via a new Soar command 'cli'. For example,
to load the new Tcl module, users can enter a 'cli tcl on' command,
and Soar will look for a DLL in the same directory called TclSoarLib
and load it if found. Similarly, using an argument of Python will
try to load PythonSoarLib, if it exists. (which doesn't)

- The full release notes for TclSoarLib are at the end of this document.

(2) Soar will now give feedback on many commands and errors that it did
not before.

(3) Soar will now give more informative messages to various command line
errors instead of just always saying "invalid attribute".

(4) Commands now display their settings using a consistent display theme
that should be much clearer.

----------------------
Reinforcement Learning
----------------------

(1) Reinforcement learning now has a new trace mechanism to help users
understand why operators are selected. When enabled, all proposed
operators at all goal levels will be recorded in the trace, along with
their names, other attribute-value pairs, and transition probabilities
derived from their numeric preferences. The format in which the trace is
printed is designed to be used by the program dot, which is part of the
Graphviz suite. See the help entry for rl for more information.

(2) Reinforcement learning now has a new experimental delta-bar-delta
mode based on the work in Richard S. Sutton's paper "Adapting Bias by
Gradient Descent: An Incremental Version of Delta-Bar-Delta". Delta Bar
Delta is implemented in Soar RL as a decay mode. It changes the way all
the rules in the eligibility trace get their values updated. The key
idea is that the agent uses meta parameters to keep track of how much a
rule's RL value has been updated recently, and if a rule gets updates in
the same direction multiple times in a row then subsequent updates in
the same direction will have more effect. In some sense, DBD acts like
momentum for the learning rate.

(3) Users can now record a log file that contains every update to
productions' reinforcement learning values with the 'update-log-path'
setting. For example, 'rl --set update-log-path rl_log.txt' would start
recording a log to the file rl_log.txt

(4) The Soar 'preferences' command now prints the actual probabilities
for operators based on the current selection method. Soar previously
always assumed softmax.

---------
Bug Fixes
---------

(1) Fixed issue 105 in which a rule with an empty RHS caused a crash in
Soar.

(2) Fixed issue 127 Soar no longer inaccurately classifies certain rule
firings as operator proposals. Instead of just looking for an attribute
named operator in a rhs action, it checks to see if that id element
points to a real state and hence is a real operator proposal. If so, it
gives i-support to all of the rhs make actions of that rule firing.

(3) Fixed issue 130 in which Soar would exit with the error "Wanted to
create a GDS for a WME level different from the instantiation
level.....Big problems....exiting".

(4) Fixed issue 132 in which memory corruption could occur when wma and
epmem are both active.

(5) Fixed issue 133 in which extremely low decay rates caused Soar to
crash.

(6) Fixed bug that resulted in epmem recording breaking after init-soar.
Soar will now properly close and re-initialize epmem.

(7) Fixed bug in epmem retrieval that could result in failed queries.

(8) Fixed bug in smem that caused Soar to crash when attempting to print
from smem after the settings pointing it to the correct database were
configured but before it was first used. It will now connect to the
database before printing.

(9) Fixed a segfault in smem, triggered when a previously unhashed value
appeared in a neg-query.

(10) Fixed bug in which LTIs with i-supported children were not properly
forgotten.

(11) Fixed bug in cli that caused crashes on certain commands like an
empty echo command.

(12) Fixed bug in cli identifier printing shortcut. (You can againg type
"S1" to print S1 instead of print S1)

(13) Fixed bug in which data was not being written to epmem/smem
databases when using the cli. It now shuts down and deletes kernel,
allowing proper clean-up.

(14) Fixed bug in which Python callbacks silently ignored exceptions.

(15) Fixed memory leak in Eaters when a wme is replaced.

(16) Fixed bug due to dubious logic in ParseXML that could cause working
memory to grow unnecessarily large.

(17) Fixed a bug in which very large messages could not be passed via an
SML filter callback.

-------------------------
Building From Source Code
-------------------------

(1) The debug build is now the default. Build script now uses --opt for
an optimized build instead of --dbg.

(2) Added option for static build (--static).

(3) Build script for iOS updated for latest version.

(4) Tcl SWIG interface updated and fully functional with latest version
of Soar and the latest version of ActiveTcl (8.6).

(5) Can now build C# SWIG wrappers.

(6) Building soar without single compilation units (--no-scu) now works
again.

(7) On OSX, build process now uses clang/clang++.

(8) Now has LSB support, which can facilitate compilation on certain
*nix system.

==========
Known Bugs
==========

(1) On some platforms, the SoarJavaDebugger may crash when exiting.

See https://code.google.com/p/soar/issues/list for the current list of issues
and feature requests.

======================
Other Changes in 9.3.4
======================

--------------
User Interface
--------------

(1) Soar has a new C-based command line interface for Soar that supports
multiple agents (called mcli).

- Adds four new commands to manipulate agents: list, create, delete,
switch

- If more than one agent exists, the CLI will enter multi-agent
mode, in which the CLI will print the output of all agents
simulataneously, with each of their trace messages prefixed with the
agent's color-coded name.

- Caveats: Currently color does not show up on windows machines. On
linux and macs, colors are configured to look best on a dark
background color.

(2) Preferences command will now indicate whether a WME has different
support at different levels, as well as whether the WME is linked to a
state but has no support there.

(3) When verbose mode is on, Soar will print the filename of each file
being sourced as it sources them.

(4) Soar now properly explains that a rule has bad RHS when it either it
creates a variable that is not tested in a positive condition on the LHS
(negative conditions don't count) or it passes an unbound variable to a
rhs function.

(5) The cli now prints the port used and what type of kernel it's
creating.

---------------
Episodic Memory
---------------

(1) Soar now only prints out new king episode for an epmem search if a
new episode has actually been crowned king (considered best).

(2) Epmem search now uses most constraining values for ^before and
^after (smallest for before, largest for after)

(3) 'epmem --stats' now shows how many ^prev and ^retrieve commands have
been made

(4) Statistics for previous epmem queries once again available.

--------
Chunking
--------

(1) The settings 'enable-through-local-negations' and
'disable-through-local-negations' have been renamed to be more succinct
and consistent with other settings. They are now 'local-negations' and
'no-local-negations'. The short version is still '-n' and '-N'.

(2) After changing chunking settings, Soar will give feedback of all
current settings.

---------
Bug Fixes
---------

(1) Fixed bug that occurred when epmem is queried with an LTI, which was
promoted (stored into smem), in the middle of its most recent interval.
Soar now sets the priority to its promotion time instead of the interval
start time.

(2) Fixed bug that resulted in epmem activation count not being properly
tracked.

(3) Fixed bug in epmem storage bug that would occur when it attempted to
recurse through a Y-shaped linkage between wme's.

(4) Fixed bug in smem where Soar would give users no feedback that
certain smem -add commands were bad (for example smem -add {fdsfds})

(5) Fixed working memory activation bug due to the fact that the test
for the o-support set of a WME is not sufficiently tight. In particular,
if one of the tested WMEs has i-support, but *that i-support is removed
at the same time as a rule that depends on it fires*, the result of that
rule will have that i-supported WME in it's o-set.

(6) Fixed bug in which a newline would not be printed when sourcing a
soar file in which a soar command is issued immediately after a
production is loaded in that same file.

(7) Fixed bug in default rules for A* search, when start == goal. The
change here is that the total-estimated-cost should be updated when
estimated-cost != 0, and final-cost should be updated when
estimated-cost == 0. This is changed from the original code where the
total-estimated-cost is updated if path-cost != 0, which would fail if
start == goal, where path-cost == estimated-cost == total-estimated-cost
== final-cost == 0. The change makes sense when you think about A* ()
when the estimated cost is 0, that means you've reached the desired
state, so the final cost is known. Otherwise, the total cost is still
an estimate, so the final cost is not stated and it's only a total
*estimated* cost.

(8) Fixed bug with calls to output handlers that occurred when command
WMEs are deleted.

(9) Fixed some bugs that resulted in Soar not adding a new line when it
should have. Handling of newlines is more consistent. (but still not
perfect.)

(10) Fixed bug in which Soar would not return an error message despite a
command's parameters being invalid.

(11) Fixed bug in Eaters and TankSoar when attempting to update an
output's status twice.

(12) Fixed bug caused by newline issues in jar manifest files. If a
manifest file doesn't have a terminating newline, certain system setups
will fail to read the last line, and the resulting jar file will be
missing that line.

(13) Fixed bug detecting correct python library on *nix platforms.

(14) Fixed compiling bugs because of inconsistent mac/windows/linux
linefeeds.

(15) Fixed compile error in GDS debug code.

-------------------------
Building From Source Code
-------------------------

(1) Java SWIG source generation improved.

(2) Scons no longer traverse hidden directories looking for targets to
build

(3) Fixed bug in which scons was not properly rebuilding sml_java when
it should have.

(4) Scons will no longer try to build java targets when swig is not
installed

(5) Fixed bug in which it would not properly copy files into directories
that starts with a '.'

(6) Project/Solution generation for MS Visual Studio now works better

(7) Kernel now has doxygen setup files for automatic code help
generation

(8) Added --python option to scons for different python versions, which
takes a path to the python executable. Scons will then use it to
determine the appropriate python libraries for building the SML client.

------------
Code Changes
------------
(This section is really only really relevant to people who modify or interface
to the source code.)

(1) Significant refactoring and reorganization of epmem/smem code for
clarity

(2) Migrated to new episodic memory and semantic memory database schemas

- Designed to be much easier to understand. Database relations are
now strict.

- Eliminated the temporal hash table. Epmem will now look up
constants from four symbol lookup tables instead.

- In general, epmem is now more similar to smem database, and both
use same schema versioning representation.

(3) Lots of new debugging facilities

- SQL tracing

- More debug messages in general

- Many debug messages now print out things using the same function
rather than an assortment of different ways (cout, printf,
callbacks, etc.)

(4) Added new CLI utility functions to support more consistent printing

(5) Added new Soar Instance class that acts as a consistent hub across
various SML configurations. It is a singleton object that corresponds to
an instance of the soar dll, as opposed to an agent or kernel. Added
initially to support the TclSoarLib dll, but could also be used for
other things as well.

(6) An internal output manager to simplify and consolidate the various
way that different portions of the Soar code produce output. It also
allows us to re-direct output dynamically to other listeners (or
multiple ones), for example a database.

(7) Removed code for various experimental modes created by previous
graduate students.

(8) Scons updated and hacked to support Visual Studio 2013 until
official version does.

(9) CppUnit, PCRE removed. STLSoft replaced with simpler timer
implementation.

(10) All support of obsolete unary parallel and binary parallel
preferences now removed

(11) TestSoarPerformance now has some #defines to allow you to set a
different test agent, change the number of trials and control the amount
of information printed.

-------------------------------------

=====================
TclSoarLib
Author: Mazin Assanie
5/27/14
=====================

- Seamlessly turns Soar prompt into a Tcl prompt with a single command.
- Uses a combination of C++ and Tcl code.
- Processes commands in embedded Tcl interpreters, which calls Soar commands
as needed.
- Productions can make Tcl calls by writing “(exec tcl | <Tcl code> |)”
clauses
on the RHS of rules. Soar symbols and variables can be include in RHS item.
- Very simple installation process.
- The user must simply have ActiveTcl 8.6 installed and the TclSoarlib files
in the same directory as the latest Soar dll.
- Provides Tcl capabilities to both local and remote clients, including the
java-based C-Soar debugger.
- Processes Tcl commands from both the Soar command line and any files sourced.
- Each agent has its own Tcl interpreter.
- Tested on OSX Maverick, Windows 7, and Ubuntu 12.04.

=====
Usage
=====

To enable:

From any soar prompt enter the command:

% cli Tcl on

To disable:

% cli Tcl off

===================
Binary Installation
===================

1. Install ActiveTcl 8.6+ (http://www.activestate.com/activetcl/downloads)
- We recommend that you use the default install location.

2. Download the SoarSuite binary for your platform from the Soar wiki
- https://code.google.com/p/soar/wiki/Downloads
- TclSoarLib is automatically included in the SoarSuite download.

=========================
Building from Source Code
=========================

1. Make sure ActiveTcl 8.6 is installed.
- http://www.activestate.com/activetcl/downloads
- We recommend that you use the default install location.

2. Download the SoarSuite source code from the wiki or check out the
SVN repository
- https://code.google.com/p/soar/wiki/Downloads
- Follow the standard build instructions for your platform until you get to
the build step

3. When you get to the compilation step, specify either the “all” or the
“tclsoarlib” build target.

For example, on Windows:

% build all

or

% build tclsoarlib

==========
Known Bugs
==========

1. Turning off tcl mode causes a crash on some systems. That option is
currently disabled as a result.

=======
Caveats
=======

1. You cannot turn tcl on in a sourced file that has tcl code in it

Soar cannot deal with such files because it does not load the tcl interpreter
until the source command completes. Future versions can remedy this issue. A
more immediate solution would be to have TclSoarLib automatically loaded on
Soar launch, if it exists.

2. Currently uses two aliasing mechanism

Because the module wraps known soar commands, aliases are problematic. For now
TclSoarLib follows previous implementations and switches to its own alias
system. When Tcl mode is turned on, aliases defined in alias.Tcl are used.
Soar aliases previously defined will no longer be recognized until Tcl mode
is turned off. Future versions can remedy this issue.

3. Only one RHS Tcl call will produce output.

Soar rhs commands “write” (and even something like “cmd echo”) will always
work. But only the last Tcl function called can produce output, for example a
“puts” command or a custom Tcl proc that produces output as a side effect.
That said, all rhs Tcl calls do get executed, so they will do what they are
supposed to do, including perhaps writing output to a file. The problem is
that the print output just doesn’t get redirected to the right place, despite
being produced.

As a workaround, a user can make sure both that there is only one Tcl call
which needs to produce output and that it comes after any other Tcl RHS
actions.

Future versions can remedy this issue, but it may require a different
implementation for rhs output.

4. Does not support Tk code

Tk is a widget toolkit that many Tcl programs use to provide a GUI, for
example the old Soar TSI debugger. Future versions can remedy this issue.

5. Tcl code that tries to do low-level Soar functions may or may not work.

Creating and deleting a kernel will certainly not work. But other things like
creating an agent, other sml calls, other swig calls may work fine. This
caveat is inherent to the design of Tcl as a plug-in without a main event loop.

6. Tcl code that requires a Tcl event loop may or may not work

One example is the Tcl “after” command. Future versions can remedy this issue.

---------------

Questions: Contact Mazin Assanie (mazina@umich.edu)