Research ideas for sensorimotor grounding experiments.
Originally these were signpost topics that I was meant to expand upon. I wrote them down here so I wouldn’t forget them.
Simple grounded agent with sensor encoding and simple effector in toy environment
- encoder with defined intervals for each bit
- an effector that can modify the sensor input distribution (change average?)
Little prince toy world as applied to an object model
- create a simple graph model of an object with feature-locations connected by traversable edges
Signposts and trails for sensorimotor representation generation
- posts are landmarks
- trails are refined paths between them
- grid modules are reusable and disposable sources of change
- encoded representations are features at locations
- actions are the transitions between points in the effector space
- “action-shapes” are the object models
- remapping of grid cells, translating the object, or rotating the object are the same task?
- object models encode the relationship between signposts… how?
- how to represent motor commands that are transformable?
- how to apply motor commands to grid modules?
- how to retune the grid modules to reflect varying levels of granularity in spatial understanding?
- vector addition, how to extract a new transition between signposts A and C, if we know A→B and B→C?
Retinotopic map can be used for saccade targets
- literature provides model of converting WTA neuron to rate-coded tonic command to motor neurons controlling eye muscles
Read literature to understand conversion from motor command to population code and vice versa.
- Efferent commands and afferent commands.
- Retinotopic map is an code example, but not sure on bidirectional conversion
Types of Grounding
according to Milliere
Referential Grounding
- Linking word to things in the world
- This definition is confusing since it seems redundant, nor does it give an explanation of how such a metaphysical task is accomplished. It seems like this is just what people assume grounding is already and the below are the various ways to accomplish it
Sensorimotor Grounding
- Linking words to experience
- Relating internal representation to each other
Relational Grounding
- Establishing relationships between words in a language
Communicative Grounding
- Dynamic process of establishing common ground in a conversation
Epistemic Grounding
- Linking words to knowledge base or dictionary definitions