At any given level <math>L</math> in the LHMM a sequence of <math>T_L</math> observation symbols
<math>\mathbf{o}_L=\{o_1, o_2, ...\dots, o_{T_L}\}</math> can be used to classify the input into one of <math>K_L</math> classes, where each class corresponds to each of the <math>K_L</math> HMMs at level <math>L</math>. This classification can then be used to generate a new observation for the level <math>L-1</math> HMMs. At the lowest layer, i.e. level <math>N</math>, primitive observation symbols <math>\mathbf{o}_p=\{o_1, o_2, ...\dots, o_{T_p}\}</math> would be generated directly from observations of the modeled process. For example in a trajectory tracking task the primitive observation symbols would originate from the quantized sensor values. Thus at each layer in the LHMM the observations originate from the classification of the underlying layer, except for the lowest layer where the observation symbols originate from measurements of the observed process.
It is not necessary to run all levels at the same time granularity. For example it is possible to use windowing at any level in the structure so that the classification takes the average of several classifications into consideration before passing the results up the layers of the LHMM.
