This project augments the Blocks-World Hierarchical Agent with look-ahead state evaluation. The description of the original agent applies to this one. The main difference is that look-ahead is performed in the middle of the three problem spaces that it uses.
Hierarchical task composition via subgoaling
Internally simulates external environment including an i/o link
This project contains a version of blocks world that is formulated for hierarchical task decomposition. It involves three levels of problem spaces. There is sufficient evaluation knowledge so that there is no search/uncertainty at every level. The top level has a single operator: move-block, which moves a block (moving-block) to a destination. The destination can be the top of another block or the table.
The next level consists of two operators: pick-up and put-down and they arise in...
agent incorporates both operator subgoaling/means ends analysis with reinforcement learning. All search control knowledge (operator evaluation rules) are removed from blocks-world-operator-subgoaling and instead there are RL rules supplemented with rules to compute reward, both in the top state and the substate. Implemented for four blocks.
This agent is a modified version of the simple blocks-world agent that uses means-ends analysis and operator subgoaling (first used in General Problem Solver (GPS)).
Means-ends analysis involves proposing operators that can achieve part of the goal. Thus, some operators will be proposed even if they do not apply to the current state. If an operator is selected that can not apply, an operator no-change impasse arises. In that substate, the goal is to achieve a state in which the impassed...
This agent contains a version of blocks-world that involves one level of problem spaces and look-ahead but with two important extensions. It demonstrates how RL-rules can be learned by chunking and then updated in the future. The advantage over simple lookahead is that it doesn't lock on to the one path found during look-ahead after chunking. It will still do some exploration.