Made with Clojure and Gorilla REPL
View this worksheet on GitHub

fxai

Sequence replay in HTM

Nov 2015 - first published.

Apr 2016 - revised to use synapses not cells for TP.

Sep 2016 - updated to new Comportex API.

In which I demonstrate replay of a sequence from its static pooled representation, via feedback. This is a proof-of-concept. Even in this simple example, some interesting questions arise: How should a pooled representation be constructed? When should it be reset? What learning process can allow feedback to induce replay? Can we pool a novel sequence? Can we replay a novel sequence?

The approach I settled on is to reset temporal pooling upon bursting (surprise), as Jeff Hawkins proposed. At the point of surprise, feedback learning occurs on all cells from the prior well-predicted sequence. Since this feedback learning is delayed, it captures following context as well as prior context. Standard HTM can reliably replay only fully-learned sequences. Replay of novel sequences is possible with a different learning algorithm at the cost of some confusion of representations.

Feedback?

In HTM theory, perceptions (and actions) feed forward up the hierarchy, while expectations feed back down, altering those perceptions and actions. Sort of. In fact it's all just neurons connecting to each other, and the utility of feedback connections from higher layers mostly comes from their temporal pooling: they change more slowly and so represent broader context or concepts.

This feedback helps us to disambiguate our perceptions, but there's more. Just as temporal pooling can represent a whole sequence as a static set of cells in a higher layer, that static representation can be used to replay the original sequence. Or any sequence consistent with it. An example Jeff gives in his book:

For me to physically move from my living room to my kitchen, all my brain has to do is mentally switch from the invariant representation of my living room to the invariant representation of my kitchen. This switch causes a complex unfolding of sequences. The process of generating the sequence of predictions of what I will see, feel, and hear while walking from the living room to the kitchen also generates the sequence of motor commands that makes me walk from my living room to my kitchen and move my eyes as I do so. [...] Thinking, predicting, and doing are all part of the same unfolding of sequences moving down the cortical hierarchy.

-- Hawkins and Blakeslee (2004). On Intelligence. p.158

I'm going to ignore the motor aspect here, and simply work on replaying, or unfolding, a sensory sequence.

Setup

Excuse me while I prepare to run some experiments. I'm defining a namespace and pulling in the Comportex HTM library and Sanity visualisations.

(ns htm-seq-replay
  (:require [org.numenta.sanity.comportex.launchpad :refer [start-runner stop-all-runners]]
            [org.numenta.sanity.comportex.notebook :refer [viz]]
            [gorilla-plot.core :as plot]
            [gorilla-repl.table :as table]
            [gorilla-repl.html :refer [html-view]]
            [org.nfrac.comportex.core :as cx]
            [org.nfrac.comportex.encoders :as e]
            [org.nfrac.comportex.layer :as layer]
            [org.nfrac.comportex.util :as util :refer [round]]
            [org.nfrac.comportex.repl]
            [clojure.set :as set]
            [clojure.string :as str]
            [clojure.pprint :refer [pprint]]
            [clojure.stacktrace :refer :all]))
nil

Here are the parameter values specifying a HTM model with two layers.

(def params
  {:column-dimensions [1000]
   :depth 5
   :distal {:max-segments 5
            :max-synapse-count 24
            :new-synapse-count 12
            :stimulus-threshold 9
            :learn-threshold 7
            :perm-connected 0.20
            ;; initially connected
            :perm-init 0.25}
   :apical {:max-segments 5
            :max-synapse-count 24
            :new-synapse-count 12
            :stimulus-threshold 9
            :learn-threshold 7
            :perm-connected 0.20
            :perm-init 0.25
            ;; we'll do this later, not on every step
            :learn? false}
   ;; no "prediction assistance"
   :distal-vs-proximal-weight 0.0
   :stable-activation-steps 10})


(def higher-level-params
  {:column-dimensions [2000]
   ;; one cell per column (ignore higher level sequences)
   :depth 1
   :activation-level 0.05
   :ff-init-frac 0.1
   :proximal {;; no proximal learning: a fixed feed-forward mapping
              :learn? false}})


(def encoder (e/unique-encoder [1000] 50))

(def two-layer-model
  (cx/network {:layer-a (layer/layer-of-cells params)
               :layer-b (layer/layer-of-cells higher-level-params)}
              {:input [[] encoder]}
              (cx/add-feedback-deps
               {:ff-deps {:layer-a [:input]
                          :layer-b [:layer-a]}})))

#'htm-seq-replay/two-layer-model

In the first published version of this work, cells in a higher layer remained active for some time if they were activated by well-predicted cells below. The number of active cells increased (linearly) in the higher layer over the course of a sequence. This pooled representation was simply the union of cells from each step.

Now the process of temporal pooling is different. Following this change to Comportex, it is the synapses from well-predicted cells which remain active for some time, instead of their target cells in the higher layer; specifically it was only those target cells which were immediately activated. This leaves open the possibility for completely different interpretations to be preferred over time as more data arrives; we are not stuck with the initial partial representation.

Furthermore, the number of active cells in the higher layer now is constant, instead of increasing over time during pooling.

It is interesting that this seems to work just as well for replaying a sequence from its pooled representation.

Input data

Keeping things simple, we'll use sequences of words, where each word gets a unique random encoding. Sentences will be preceded by a special symbol ">" so that later we can use that to trigger predictions starting from the beginning of a sentence. In reality this might correspond to some kind of attention-based signal or oscillation.

(def test-text
  "The rat the cat bit squealed.")

(def train-text
  "
The rat died.
The mouse died.
The mouse squealed.
The bird died.
The bird squawked.
The bird the cat bit squawked.
The bird the cat chased flew.
The cat the bird bit ran.
The cat ran.
The mouse the cat chased ran.
The mouse the cat bit squealed.
The mouse the cat bit died.
The rat the cat chased ran.
")
#'htm-seq-replay/train-text
(defn words-in-sentences
  "Takes a block of text and returns a sequence of sentences, each a vector of words."
  [text*]
  (let [text (str/lower-case (str/trim text*))]
    (->> (str/split text #"[^\w]*\.+[^\w]*")
         (mapv #(str/split % #"[^\w']+")))))

(def START ">")
(def STOP ".")
(defn delimit [sentence] (concat [START] sentence [STOP]))

(def train-sentences (map delimit (words-in-sentences train-text)))
(def test-sentence (delimit (first (words-in-sentences test-text))))

test-sentence
(">" "the" "rat" "the" "cat" "bit" "squealed" ".")

Learning

Now we have a fresh HTM model and some sequences of words.

Watch the HTM as it is exposed to a sentence for the first time. This is mostly just an initial sanity check and to get used to looking at these diagrams. We feed in "> the rat the cat ." and observe the set of active columns in each step over time, from left to right. The columns are sorted by most recent activation, for visual clarity. I've highlighted a single column from the second occurrence of "the".

(def sentence (delimit ["the" "rat" "the" "cat"]))
(prn sentence)

(def steps
  (rest (reductions cx/htm-step two-layer-model sentence)))

(viz steps {:distal-synapses {:permanences false}
            :drawing {:max-height-px 500}})
(">" "the" "rat" "the" "cat" ".")

653214
input
the
cells and distal / apical dendrite segments
input
163 of 1000 bits
layer-a
98 of 1000 cols
layer-b
98 of 2000 cols

Apical vs distal: from smeared-out to orderly

It is important that inputs are distinguished by context; in this case, that the two occurrences of "the" are distinguished. Originally I had allowed apical feedback by itself to put cells into a predictive state (not pictured). That caused the latter "the" to be predicted, from a matching subset of the pooled feedback, and therefore chosen as identical to the former. Confusion. Bad.

The approach I've taken is to have apical excitation be subordinate to lateral excitation: it has no effect unless there is a sufficient lateral prediction to combine it with. Then the two sources are combined additively. So if we have a large feedback context, it does not all present itself simultaneously, smeared-out, rather most of it is latent. It must be unfolded in an orderly fashion. Another way to say this is that replay needs to be cued.

However, there is a subtlety here. Perhaps we should allow a partial bias to cells with apical excitation alone (i.e. matching the feedback context), before a lateral transition has been learned. This could generate partially-overlapping representations for the same stimulus appearing in similar contexts. Which sounds good. But I don't know whether this is desirable in general, or whether it could be made to work reliably.

Full context learning

If apical segments learn immediately on every time step, as lateral distal segments do, there is a problem: the first words of a sentence are less well predicted than later words. In our test sentence for example, it replays as ("the", "cat/mouse/bird/rat", ...) The reason is that each of these have learned on the same subset of prior context (">", "the") and they all match it equally on replay.

Marcus Lewis made the suggestion to delay apical learning until the end of a sequence. Then all the previously activated cells learn the same final set of feedback bits. In the example above, activated "rat" cells would grow feedback connections to (a sample of) the full context from ("> the rat the cat bit squealed ."). The result is that all steps of a sequence can be replayed from its full pooled context without confusion, as we'll see.

When exactly should this delayed apical learning take place? At first I had it at the end of each sentence, applying to all representative (column-winner) cells from the sentence. But that is flawed when a sentence is not yet fully learned: after bursting, if the cell representation collapses to some previously learned sequence, then we have steps of that previous sequence erroneously connecting to feedback from this new one. That might create extra confusion during replay. So instead, we'll use bursting as the trigger for apical learning.

(defn htm-step-delay-apical
  "Takes a step, records winner cells. If we reach the end of a
  predicted sequence (i.e. hit a surprise), apical learning takes
  place, as if delayed, on the accumulated winner cells."
  [htm inval]
  (let [lo-lyr-path [:layers :layer-a]
        hi-lyr-path [:layers :layer-b]
        old-lo-lyr (get-in htm lo-lyr-path)
        nhtm (cx/htm-step htm inval)
        lo-lyr (get-in nhtm lo-lyr-path)
        hi-lyr (get-in nhtm hi-lyr-path)
        lo-info (cx/layer-state lo-lyr)
        reset? (>= (count (:bursting-columns lo-info))
                   (* 0.5 (count (:active-columns lo-info))))
        delayed-apical (get-in htm (into lo-lyr-path [:active-state :delayed-apical]) ())
        learn-now? (and reset? (>= (count delayed-apical) 3))]
    (cond->
      nhtm
      ;; hit end of a predicted sequence.
      ;; learn on apical dendrites of delayed cells, using prior feedback
      learn-now?
      (update-in lo-lyr-path
                 (fn [lyr]
                   (let [delayed-apical-all (reduce conj delayed-apical)]
                    (-> lyr
                       (assoc :apical-state (:apical-state old-lo-lyr))
                       (layer/layer-learn-apical (keys delayed-apical-all) delayed-apical-all)
                       (assoc :apical-state (:apical-state lyr))))))
      reset?
      (assoc-in (into lo-lyr-path [:active-state :delayed-apical])
                ())
      ;; ...and clear accumulated delayed cells.
      ;; within a predicted sequence, record cells to delay apical learning
      (not reset?)
      (assoc-in (into lo-lyr-path [:active-state :delayed-apical])
                (conj delayed-apical
                      (get-in lo-lyr [:learn-state :winner-seg :apical])))
      ;; clear temporal pooling after learning if forced break.
      reset?
      (cx/break :syns)
      reset?
      (cx/break :fb))))


(defn one-exposure
  [htm invals]
  (reduce htm-step-delay-apical
          (cx/break htm :tm) invals))

(defn steps-from-n-exposures
  [htm invals n]
  (loop [htm htm
         steps []
         i n]
    (if (zero? i)
      steps
      (let [news (rest (reductions htm-step-delay-apical
                                   (cx/break htm :tm) invals))]
        (recur (last news)
               (into steps news)
               (dec i))))))
#'htm-seq-replay/steps-from-n-exposures

The novelty question

A sequence can't be learned from a single exposure with standard HTM learning algorithms. In particular when learning a new sequence which shares a common subsequence with one previously learned, every step of that subsequence must be relearned in a separate exposure. (Or multiple exposures, depending on the learning rate).

It is possible to build up a higher layer representation of bursting steps, but they can't be replayed while the lower layer lacks its lateral connections.

When I presented this project at the Numenta Community Meetup, I had modified the learning algorithm to learn every step of a novel sequence immediately. This worked by growing new synapses to bursting columns from their following winner cells (specifically, even if those cells were predicted by bursting, which is not standard HTM). However, the major drawback of that approach is that new sequences end up reusing cells from shared subsequences of previous sequences, resulting in confusion. So I abandoned it.

Perhaps there is some middle way. It does seem remarkably non-intelligent to have to re-learn common subsequences, painfully, in every new context. Or indeed we might want common subsequences to be abstracted by temporal pooling to a single step in a higher level sequence. Somehow.

Go

Let's train our model with the sequence of words in each training sentence, presenting each sentence 3 times.

(def trained-model
  (->
    (reduce one-exposure
            two-layer-model
            (apply concat (repeat 3 train-sentences)))
    (cx/break :tm)))
#'htm-seq-replay/trained-model

Building on this trained model, feed in the test sentence so it can be learned in exactly the same way. The only difference is that we are leaving it with the pooled representation intact in its higher layer; we'll use that for replay later.

(def test-model
  (reduce one-exposure
          trained-model
          (repeat 3 test-sentence)))
#'htm-seq-replay/test-model

Let's watch that happening - the trained model taking in the test sentence in 3 repeated exposures. We can discover quite a lot:

(def steps
  (steps-from-n-exposures trained-model test-sentence 3))

(prn test-sentence)

(viz steps
     {:distal-synapses {:permanences false}
      :drawing {:draw-steps 25, :h-space-px 35, :bit-w-px 2, :col-d-px 4, :col-shrink 1.0, :max-height-px 800}})
(">" "the" "rat" "the" "cat" "bit" "squealed" ".")

282281279278277276275274273272271270269268267266265264263262261260259280
input
bit
cells and distal / apical dendrite segments
input
263 of 1000 bits
layer-a
197 of 1000 cols
layer-b
197 of 2000 cols
(pprint
  (map (fn [htm]
         [(:input-value htm)
          (->> htm :layers :layer-a :tp-state :stable-cells-buffer (map count))])
       steps))
([">" ()]
 ["the" (20)]
 ["rat" (20 20)]
 ["the" (20 20 20)]
 ["cat" (20 20 20 19)]
 ["bit" (20 20 20 19 12)]
 ["squealed" (20 20 20 19 12 20)]
 ["." (20 20 20 19 12 20 16)]
 [">" ()]
 ["the" (20)]
 ["rat" (20 20)]
 ["the" (20 20 20)]
 ["cat" (20 20 20 20)]
 ["bit" (20 20 20 20 18)]
 ["squealed" (20 20 20 20 18 19)]
 ["." (20 20 20 20 18 19 20)]
 [">" ()]
 ["the" (20)]
 ["rat" (20 20)]
 ["the" (20 20 20)]
 ["cat" (20 20 20 20)]
 ["bit" (20 20 20 20 20)]
 ["squealed" (20 20 20 20 20 20)]
 ["." (20 20 20 20 20 20 20)])

nil

Note the progression of learning in the lower layer, where the amount of bursting decreases with each exposure.

At the same time, temporal pooling in the higher layer continues as well-predicted cells below keep their synapse connections active.

The highlighted cell has a matching lateral (distal) dendrite and a partially-matching apical dendrite to the higher layer. At the end of the current sequence - at the next point of bursting - this apical dendrite will learn against the higher layer's final state.

Replay using iterative predictions

Let's start with a naïve approach to replay. Keeping feedback from the higher layer constant, we'll feed in the start symbol (">"), then take the strongest predicted input value -- a decoded word -- to feed in as sensory input. And iterate.

We'll only consider cells that were predicted by both lateral and apical excitation.

Predicted cells are decoded into previously-seen words by following their proximal synapses back to the encoded input sense, then matching against the history of known words. Finally these are filtered to meet a certain strength, by number of matching synapses ("votes").

For reference:

  • votes-frac - proportion of all the predicting connected synapses which represent the given value.
  • bit-coverage - proportion of the bits of the given value which are connected to a predictive cell (at least one cell).
  • votes-per-bit - average number of predicting connected synapses per bit, over all bits of the given value.
(defn % [x] (symbol (str (long (* x 100)) "%")))

(defn predictions-table
  [predictions]
  (table/table-view (map (juxt :value
                               (comp % :votes-frac)
                               (comp % :bit-coverage)
                               (comp #(round % 1) :votes-per-bit))
                     predictions)
                    :columns '[value votes-frac bit-coverage votes-per-bit]))

(defn consistent-predictive
  [layer]
  (let [dpc (get-in layer [:distal-state :pred-cells])
        apc (get-in layer [:apical-state :pred-cells])
        both (set/intersection dpc apc)]
    (if (>= (count both) 10) both dpc)))

(defn- cells->columns
  [cells]
  (sequence (comp (map first) (distinct)) cells))

(defn feed-predictions-gen
  [model & {:keys [steps?]}]
  (println "top layer active cells:"
           (count (-> model (cx/layer-seq) (last) (cx/layer-state) :active-cells)))
  (loop [inval START
         htm* model
         out []
         i 0]
    (let [htm (-> htm*
                  (cx/htm-sense inval nil)
                  (cx/htm-activate)
                  ;; keep higher layer constant - replace with original
                  (assoc-in [:layers :layer-b] (get-in model [:layers :layer-b]))
                  (cx/htm-depolarise))
          n-pc (count (-> htm (cx/layer-seq) (first) (cx/layer-state) :predictive-cells))
          preds (cx/predictions htm :input 10 {:get-columns (comp cells->columns
                                                                  consistent-predictive)})
          top-pred (first preds)]
      (if (and top-pred (< i 16))
        (recur (:value top-pred)
               htm
               (conj out (if steps? htm [preds n-pc]))
               (inc i))
        ;; no predictions
        out))))
#'htm-seq-replay/feed-predictions-gen

So here is the iterative generation process with feedback from the pooled test sentence.

(def gen-preds
  (feed-predictions-gen (-> test-model (cx/break :tm))))

(prn test-sentence)

(table/table-view (map (fn [i [preds n-pc]]
                         [i n-pc
                          (->> preds
                               (filter #(> (:votes-per-bit %) 2.0))
                               (predictions-table))])
                       (range) gen-preds)
                  :columns '[i pred-cells predictions])
top layer active cells: 100
(">" "the" "rat" "the" "cat" "bit" "squealed" ".")

ipred-cellspredictions


020
valuevotes-fracbit-coveragevotes-per-bit


"the"100%100%6.2


178
valuevotes-fracbit-coveragevotes-per-bit


"rat"76%100%5.7


240
valuevotes-fracbit-coveragevotes-per-bit


"the"85%100%6.3


320
valuevotes-fracbit-coveragevotes-per-bit


"cat"100%100%5.9


425
valuevotes-fracbit-coveragevotes-per-bit


"bit"95%100%6.0


539
valuevotes-fracbit-coveragevotes-per-bit


"squealed"91%100%5.6


620
valuevotes-fracbit-coveragevotes-per-bit


"."86%100%5.5

OK, that worked.

For comparison, without feedback:

(def baseline-gen-preds
  (feed-predictions-gen
    ;; take a step with no input to clear the higher layer active cells,
    ;; because we are forcing it to be frozen in this state; avoid feedback.
    (-> test-model (cx/break :syns) (cx/htm-step nil))))

(table/table-view (map (fn [i [preds n-pc]]
                         [i n-pc
                          (->> preds
                               (filter #(> (:votes-per-bit %) 2.0))
                               (predictions-table))])
                       (range) baseline-gen-preds)
                  :columns '[i pred-cells predictions])
top layer active cells: 0

ipred-cellspredictions


020
valuevotes-fracbit-coveragevotes-per-bit


"the"100%100%6.2


178
valuevotes-fracbit-coveragevotes-per-bit


"mouse"30%100%7.5


"cat"30%100%7.5


"bird"30%100%7.4


"rat"29%100%7.2


260
valuevotes-fracbit-coveragevotes-per-bit


"the"40%100%7.3


"squealed"35%100%6.4


"died"34%100%6.3


320
valuevotes-fracbit-coveragevotes-per-bit


"cat"100%100%6.1


439
valuevotes-fracbit-coveragevotes-per-bit


"chased"51%100%6.6


"bit"51%100%6.6


514
valuevotes-fracbit-coveragevotes-per-bit


"ran"96%100%4.1


614
valuevotes-fracbit-coveragevotes-per-bit


"."87%100%3.9

Replay using spontaneous activation

It is gross to feed in a decoded prediction as sensory input. A more natural and hopefully flexible approach is to activate predicted cells directly. I call this spontaneous activation: cells becoming active from distal/apical excitation alone, without proximal excitation. It is still the usual k-winners-take-all competitive selection. Once we select some cells to become active, they generate more lateral distal excitation (predictions), and so on; we iterate.

Just for display, each step we take the active cells and decode those into words, the same way as for predictive cells before.

(def sponparams
  {:distal {:learn? false, :stimulus-threshold 5}
   :apical {:learn? false, :stimulus-threshold 5}
   :proximal {:learn? false}
   :spontaneous-activation? true
   :distal-vs-proximal-weight 0.5})

(defn spontaneous-gen
  [model & {:keys [steps?]}]
  (println "top layer active cells:"
           (count (-> model (cx/layer-seq) (last) (cx/layer-state) :active-cells)))
  (loop [htm* (-> model
                  (update-in [:layers :layer-a :params]
                             util/deep-merge sponparams))
         out []
         i 0]
    (let [inval (if (zero? i) START nil)
          htm (-> htm*
                  (cx/htm-sense inval nil)
                  (cx/htm-activate)
                  ;; keep higher layer constant - replace with original
                  (assoc-in [:layers :layer-b] (get-in model [:layers :layer-b]))
                  (cx/htm-depolarise))
          decodings (cx/predictions htm :input 10 {:get-columns (comp :active-columns
                                                                      cx/layer-state)})
          lyr (get-in htm [:layers :layer-a])
          info (cx/layer-state lyr)
          n-pc (count (:predictive-cells info))
          n-ac (count (:active-cells info))]
      (if (< i 12)
        (recur htm
               (conj out (if steps? htm [decodings n-pc n-ac]))
               (inc i))
        ;; stop
        out))))

#'htm-seq-replay/spontaneous-gen
(def spgen-preds
  (spontaneous-gen (-> test-model (cx/break :tm))))

(table/table-view
  (keep-indexed
    (fn [i [decods n-pc n-ac]]
        (when (pos? n-ac)
          [i n-pc n-ac
           (->> decods
                (filter #(> (:votes-per-bit %) 2.0))
                (predictions-table))]))
    spgen-preds)
  :columns '[i pred active decodings])
top layer active cells: 100

ipredactivedecodings


020100
valuevotes-fracbit-coveragevotes-per-bit


">"95%100%6.2


18020
valuevotes-fracbit-coveragevotes-per-bit


"the"100%100%6.2


24020
valuevotes-fracbit-coveragevotes-per-bit


"rat"86%100%5.2


33720
valuevotes-fracbit-coveragevotes-per-bit


"the"88%100%5.6


44620
valuevotes-fracbit-coveragevotes-per-bit


"cat"100%100%6.1


56020
valuevotes-fracbit-coveragevotes-per-bit


"bit"96%100%6.3


62120
valuevotes-fracbit-coveragevotes-per-bit


"squealed"91%100%5.3


7020
valuevotes-fracbit-coveragevotes-per-bit


"."87%100%5.8

Here are the activated columns lined up for comparison between the two generation approaches.

(def gen-steps
  (feed-predictions-gen (-> test-model (cx/break :tm)) :steps? true))

(def spgen-steps
  (spontaneous-gen (-> test-model (cx/break :tm)) :steps? true))

(viz (concat spgen-steps gen-steps)
     {:distal-synapses {:permanences false}
      :drawing {:draw-steps 25, :h-space-px 35, :bit-w-px 2, :col-d-px 4, :col-shrink 1.0, :max-height-px 600}})
top layer active cells: 100
top layer active cells: 100

288287286285284283294293292291290289288287286285284283289
input
squealed
input
197 of 1000 bits
layer-a
147 of 1000 cols
layer-b
147 of 2000 cols

Woohoo, it worked!

For comparison, the same process without any feedback (without any active cells in the higher layer):

(def spgen-decod-steps
  (spontaneous-gen
    ;; take a step with no input to clear the higher layer active cells,
    ;; because we are forcing it to be frozen in this state; avoid feedback.
    (-> test-model (cx/break :syns) (cx/htm-step nil))))

(table/table-view
  (keep-indexed
    (fn [i [decods n-pc n-ac]]
        (when (pos? n-ac)
          [i n-pc n-ac
           (->> decods
                (filter #(> (:votes-per-bit %) 2.0))
                (predictions-table))]))
    spgen-decod-steps)
  :columns '[i pred active decodings])
top layer active cells: 0

ipredactivedecodings


020100
valuevotes-fracbit-coveragevotes-per-bit


">"95%100%6.2


18020
valuevotes-fracbit-coveragevotes-per-bit


"the"100%100%6.2


21820
valuevotes-fracbit-coveragevotes-per-bit


"mouse"38%96%2.5


3218
valuevotes-fracbit-coveragevotes-per-bit


402
valuevotes-fracbit-coveragevotes-per-bit

Progression of learning

Recall that we trained the HTM with 3 repeats of each sentence. It's interesting to look at how well a sentence can be replayed when it is not yet fully learned. As before we attempt to replay from the state at the end of the sentence; if there was bursting part way through then the higher layer will only reflect the part after bursting.

(def more-test-texts
  ["The rat the cat bit squealed."
   "The mouse the cat chased ran." ;; already learned in training
   "The mouse the cat chased ran very fast ."
   "The very fast mouse the cat chased ran."
   "The mouse ran."
   "The cat chased the bird."])


(defn spon-gen-sentence
  [htm]
  (for [decod-step (spontaneous-gen htm)
        :let [[decods* _] decod-step
              decods (->> decods*
                          (filter #(> (:votes-per-bit % 0) 2.0))
                          (remove nil?))]
        :when (seq decods)]
    (->> decods
         (map :value)
         (interpose "/")
         (apply str))))

(for [text more-test-texts]
  (let [sentence (delimit (first (words-in-sentences text)))
        stages-gens
        (->> (rest (iterate #(one-exposure % sentence) trained-model))
             (map spon-gen-sentence)
             (take 8)) ;; limit in case never fully learned
        learned-in-n (count (take-while #(not= % sentence) stages-gens))]
   (html-view
     (apply
       str
       (str "<h4>" text "</h4>")
       (concat
         ["<ol>"]
         (->> (take (inc learned-in-n) stages-gens)
              (map #(str "<li>" (apply str (interpose " " %)) "</li>")))
         ["</ol>"])))))


(
The rat the cat bit squealed.
  1. > the rat the cat bit died/squealed .
  2. > the rat the cat bit squealed .
 
The mouse the cat chased ran.
  1. > the mouse the cat chased/bit died .
  2. > the mouse the cat chased ran .
 
The mouse the cat chased ran very fast .
  1. > the cat/mouse
  2. > the mouse the cat chased ran very/. fast .
  3. > the mouse the cat chased ran very fast .
 
The very fast mouse the cat chased ran.
  1. > the mouse the cat chased/bit squealed/died .
  2. > the cat/bird ran
  3. > the
  4. > the very/cat fast mouse the cat chased ran .
  5. > the
  6. > the very fast mouse the cat chased ran .
 
The mouse ran.
  1. > the mouse the cat chased/bit died
  2. > the mouse the cat bit died .
  3. > the mouse the cat bit died/squealed .
  4. > the mouse the cat bit died .
  5. > the mouse the/ran cat/. bit died/squealed .
  6. > the mouse ran/the .
  7. > the mouse ran .
 
The cat chased the bird.
  1. > the bird the cat chased/bit flew/ran .
  2. > the bird/mouse the
  3. > the cat/bird ran/chased .
  4. > the cat chased the bird .
)

Next steps

Music. Pure abstract sequences of notes. One interesting aspect is the lack of such obvious reset points as we have in sentences.

Relative timing and time-stepping signals. I'm thinking of global oscillations (brain waves) entrained to sensory input frequencies. This could be an approach to the previous point, as a way to decide on higher level time steps.

Multiple senses. After coordinated learning, try perceiving one sense and replaying into another sense.

Generalisation?

Acknowledgments

By Felix Andrews.

This work grew out of initial conversations with Rob Freeman.

Marcus Lewis helped me to think about a lot of these ideas and reviewed this post. He also enabled this work by building Sanity.

And of course, everything here is inspired by Jeff Hawkins.

Reference info

(-> test-model :layers :layer-a cx/params sort pprint)
(str (new java.util.Date))
([:activation-level 0.02]
 [:adjust-overlap-duty-ratio 0]
 [:adjust-overlap-every 300]
 [:apical
  {:perm-connected 0.2,
   :perm-punish 0.002,
   :max-synapse-count 24,
   :max-segments 5,
   :perm-init 0.25,
   :new-synapse-count 12,
   :stimulus-threshold 9,
   :punish? true,
   :learn? false,
   :perm-dec 0.01,
   :learn-threshold 7,
   :perm-inc 0.05,
   :perm-stable-inc 0.05}]
 [:apical-bias-frac 0.0]
 [:boost-active-duty-ratio 0]
 [:boost-active-every 100]
 [:column-dimensions [1000]]
 [:depth 5]
 [:distal
  {:perm-connected 0.2,
   :perm-punish 0.002,
   :max-synapse-count 24,
   :max-segments 5,
   :perm-init 0.25,
   :new-synapse-count 12,
   :stimulus-threshold 9,
   :punish? true,
   :learn? true,
   :perm-dec 0.01,
   :learn-threshold 7,
   :perm-inc 0.05,
   :perm-stable-inc 0.05}]
 [:distal-vs-proximal-weight 0.0]
 [:dominance-margin 4]
 [:duty-cycle-period 1000]
 [:ff-init-frac 0.25]
 [:ff-perm-init [0.1 0.25]]
 [:ff-potential-radius 1.0]
 [:float-overlap-duty-ratio 0]
 [:float-overlap-duty-ratio-hi 20.0]
 [:float-overlap-every 100]
 [:ilateral
  {:perm-connected 0.5,
   :max-synapse-count 22,
   :max-segments 1,
   :perm-init 0.08,
   :new-synapse-count 12,
   :stimulus-threshold 1,
   :learn? false,
   :perm-dec 0.01,
   :perm-inc 0.08}]
 [:inh-radius-every 1000]
 [:inh-radius-scale 1.0]
 [:inhibition-base-distance 1]
 [:lateral-synapses? true]
 [:max-boost 1.5]
 [:proximal
  {:perm-connected 0.2,
   :perm-punish 0.002,
   :max-synapse-count 300,
   :max-segments 1,
   :perm-init 0.25,
   :new-synapse-count 12,
   :stimulus-threshold 2,
   :punish? false,
   :grow? false,
   :learn? true,
   :perm-dec 0.01,
   :learn-threshold 7,
   :perm-inc 0.04,
   :perm-stable-inc 0.15}]
 [:random-seed 42]
 [:spatial-pooling :standard]
 [:spontaneous-activation? false]
 [:stable-activation-steps 10]
 [:temporal-pooling :standard]
 [:transition-similarity 1.0])

"Fri Sep 23 16:45:34 HKT 2016"
(print-cause-trace *e)



Looking forward to hearing your thoughts directly or on the HTM Forum.

–Felix