<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE article  PUBLIC "-//NLM//DTD Journal Publishing DTD v3.0 20080202//EN" "http://dtd.nlm.nih.gov/publishing/3.0/journalpublishing3.dtd"><article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" dtd-version="3.0" xml:lang="en" article-type="research article"><front><journal-meta><journal-id journal-id-type="publisher-id">ENG</journal-id><journal-title-group><journal-title>Engineering</journal-title></journal-title-group><issn pub-type="epub">1947-3931</issn><publisher><publisher-name>Scientific Research Publishing</publisher-name></publisher></journal-meta><article-meta><article-id pub-id-type="doi">10.4236/eng.2016.89055</article-id><article-id pub-id-type="publisher-id">ENG-70590</article-id><article-categories><subj-group subj-group-type="heading"><subject>Articles</subject></subj-group><subj-group subj-group-type="Discipline-v2"><subject>Engineering</subject></subj-group></article-categories><title-group><article-title>
 
 
  Phoneme Sequence Modeling in the Context of Speech Signal Recognition in Language “Baoule”
 
</article-title></title-group><contrib-group><contrib contrib-type="author" xlink:type="simple"><name name-style="western"><surname>Hyacinthe</surname><given-names>Konan</given-names></name><xref ref-type="aff" rid="aff1"><sup>1</sup></xref></contrib><contrib contrib-type="author" xlink:type="simple"><name name-style="western"><surname>Etienne</surname><given-names>Soro</given-names></name><xref ref-type="aff" rid="aff1"><sup>1</sup></xref></contrib><contrib contrib-type="author" xlink:type="simple"><name name-style="western"><surname>Olivier</surname><given-names>Asseu</given-names></name><xref ref-type="aff" rid="aff1"><sup>1</sup></xref></contrib><contrib contrib-type="author" xlink:type="simple"><name name-style="western"><surname>Bi</surname><given-names>Tra Goore</given-names></name><xref ref-type="aff" rid="aff2"><sup>2</sup></xref></contrib><contrib contrib-type="author" xlink:type="simple"><name name-style="western"><surname>Raymond</surname><given-names>Gbegbe</given-names></name><xref ref-type="aff" rid="aff2"><sup>2</sup></xref></contrib></contrib-group><aff id="aff1"><addr-line>Ecole Supérieure Africaine des Technologies d’Information et de Communication (ESATIC), Abidjan, C?te d’Ivoire </addr-line></aff><aff id="aff2"><addr-line>Institut National Polytechnique Félix Houphou?t Boigny (INP-HB), Yamoussoukro, C?te d’Ivoire </addr-line></aff><pub-date pub-type="epub"><day>02</day><month>09</month><year>2016</year></pub-date><volume>08</volume><issue>09</issue><fpage>597</fpage><lpage>617</lpage><history><date date-type="received"><day>August</day>	<month>9,</month>	<year>2016</year></date><date date-type="rev-recd"><day>Accepted:</day>	<month>September</month>	<year>10,</year>	</date><date date-type="accepted"><day>September</day>	<month>14,</month>	<year>2016</year></date></history><permissions><copyright-statement>&#169; Copyright  2014 by authors and Scientific Research Publishing Inc. </copyright-statement><copyright-year>2014</copyright-year><license><license-p>This work is licensed under the Creative Commons Attribution International License (CC BY). http://creativecommons.org/licenses/by/4.0/</license-p></license></permissions><abstract><p>
 
 
  This paper presents the recognition of “Baoule” spoken sentences, a language of C?te d’Ivoire. Several formalisms allow the modelling of an automatic speech recognition system. The one we used to realize our system is based on Hidden Markov Models (HMM) discreet. Our goal in this article is to present a system for the recognition of the Baoule word. We present three classical problems and develop different algorithms able to resolve them. We then execute these algorithms with concrete examples. 
 
</p></abstract><kwd-group><kwd>HMM</kwd><kwd> MATLAB</kwd><kwd> Language Model</kwd><kwd> Acoustic Model</kwd><kwd> Recognition Automatic Speech</kwd></kwd-group></article-meta></front><body><sec id="s1"><title>1. Introduction</title><p>The speech recognition by machine has long been a research topic that fascinates the public and remains a challenge for specialists, and it has continued since then to be at the heart of much research. The progress of new information and communications technology has helped accelerate this research. In our first article, we presented a method to separate phonemes contained in a speech signal.</p><p>In this article we propose to identify a flow of words often uttered in a more or less background noise. This task is made difficult not only by the deformations induced by the use of a microphone but also by a series of factors inherent in human language, homonyms; local accents; the habits of language; the speed differences between the speakers; the imperfections of a microphone, etc. For our human ear, these factors do not usually represent difficulties. Our brain juggles these deformations of speech by taking into account, almost unconsciously, nonverbal and contextual elements that allow us to eliminate ambiguities. It is only by taking into account these elements that are external to the voice itself that voice recognition software will be able to achieve a high level of reliability. Today, speech recognition softwares that work best are all based on a probabilistic approach. The aim of speech recognition is to reconstruct a sequence of words M from a recorded acoustic signal A. In the statistical approach, we will consider all the consequences of M words that could match the signal A. In this set of possible word sequences, we will then choose the one (M) which is the most likely to maximize the P(M/A) probability that M is the correct interpretation of A, we note:</p><disp-formula id="scirp.70590-formula539"><graphic  xlink:href="http://html.scirp.org/file/4-8102669x2.png"  xlink:type="simple"/></disp-formula><p>This equation is the key to the probabilistic approach to speech recognition. Indeed, the first term P(A/M) is the probability of observing the acoustic signal A if the M sequence of words is pronounced: it is a purely acoustic problem; the second term P(M) is the probability that this is the result of M words that is actually stated: it is a linguistic problem. The above equation thus teaches us that we can split the speech recognition problem into two independent parts: we will model the acoustic aspects separately and language problems. In the literature, we usually speak of orthogonality between the ACOUSTIC MODELS and LANGUAGE. The succession of possible words that is obtained must be refined and validated by the word patterns and language. The acoustic model can take into account the acoustic and phonetic constraints in a sound or group of sounds. On our part, we have chosen the WORD as decision unit. By integrating also a Markov modeling, which has higher levels of language, it becomes possible to achieve a pronounced phrases discretely recognition system (i.e. in single word).</p></sec><sec id="s2"><title>2. The Speech Signals</title><sec id="s2_1"><title>2.1. Characteristics of the Speech Signal</title><p>PAR is a difficult problem, mainly due to the specific material to interpret: the voice signal. The speech acoustic signal has characteristics that make complex interpretation.</p><p>Redundancy: the acoustic signal carries much more information than necessary, which explains its resistance to noise. Of analytical techniques were implemented to extract relevant information without too degrading it.</p><p>Variability: the acoustic signal is highly variable from one speaker to the other (gender, age, etc.) but also for a given speaker (emotional state, fatigue, etc.), which makes very difficult the recognition problem speaker’s independent speech.</p><p>Continuity: the acoustic signal is continuous and contextual effects of sound on elementary visions are considerable.</p></sec><sec id="s2_2"><title>2.2. Processing of the Speech Signal</title><p>By speech processing we mean the processing of the information contained in the speech signal. The objective is the transmission or recording of this signal, or its synthesis or recognition. The speech processing is now a fundamental component of the engineering sciences. Located at the intersection of digital signal processing and language processing (that is to say, symbolic data processing), this scientific discipline has known since the 60s a rapid expansion, linked to the development of means and telecommunications techniques. The special importance of speech processing in this broader context is explained by the privileged position of the word as an information vector in our human society.</p></sec></sec><sec id="s3"><title>3. System Overview</title><sec id="s3_1"><title>3.1. The Acoustic Model</title><p>The ACOUSTIC MODEL (<xref ref-type="fig" rid="fig1">Figure 1</xref>) reflects the acoustic realization of each modeled element (phoneme, silence, noise, etc.). It is based on the concept of phonemes. Phonemes can be considered as the basic sound units in verbal language. The first stage of speech recognition is to recognize a set of phonemes in words flow. Statistical realization of acoustic parameters of each phone is represented by a Markov model Cache (HMM: Hidden Markov Model). Each phoneme is typically represented by 2 or 3 states and density multigaussienne (GMM: Gaussian Mixture Model) is associated with each state. See <xref ref-type="fig" rid="fig2">Figure 2</xref> below.</p><p>The speech signal (picked up using a microphone) is first digitized: it is sampled by a Fourier transformation which calculates the energy levels of the signal in bands of 25 milliseconds, which strips overlap in 10 milliseconds time.</p><p>The result is compared with prototypes stored in computer memory in both a standard dictionary and a speaker’s own dictionary. This dictionary is constructed by initially sessions dictation standard texts that the speaker must make before effectively use the software. This own dictionary is regularly enriched by self learning during the software uses. It is interesting to note that thus constituted voiceprint is relatively stable for a given speaker and little influenced by external factors such as stress, colds, etc. (<xref ref-type="fig" rid="fig3">Figure 3</xref>).</p></sec><sec id="s3_2"><title>3.2. The Language Model</title><p>It is generally divided into two part linked to language: a syntactic part and a semantic game. When ACOUSTIC MODEL has identified at best phonemes “heard”, we still look the most likely message M corresponding thereto, that is to say, the probability P(M) defined above. It is the role of syntactic and semantic models. See <xref ref-type="fig" rid="fig4">Figure 4</xref> below.</p><p>From the set of phonemes from the ACOUSTIC MODEL The SYNTACTIC MODEL will assemble phonemes into words. This work is also based on a dictionary and gram-</p><fig id="fig1"  position="float"><label><xref ref-type="fig" rid="fig1">Figure 1</xref></label><caption><title> System of speech recognition</title></caption><graphic mimetype="image"   position="float"  xlink:type="simple"  xlink:href="http://html.scirp.org/file/4-8102669x3.png"/></fig><fig-group id="fig2"><label><xref ref-type="fig" rid="fig2">Figure 2</xref></label><caption><title> The part of the system using the acoustic model.</title></caption><fig id ="fig2_1"><label></label><graphic mimetype="image"   position="float"  xlink:type="simple"  xlink:href="http://html.scirp.org/file/4-8102669x5.png"/></fig></fig-group><fig-group id="fig3"><label><xref ref-type="fig" rid="fig3">Figure 3</xref></label><caption><title> Acoustic model (A phoneme is modeled as a sequence of acoustic vector).</title></caption><fig id ="fig3_1"><label></label><graphic mimetype="image"   position="float"  xlink:type="simple"  xlink:href="http://html.scirp.org/file/4-8102669x7.png"/></fig><fig id ="fig3_2"><label></label><graphic mimetype="image"   position="float"  xlink:type="simple"  xlink:href="http://html.scirp.org/file/4-8102669x6.png"/></fig></fig-group><fig id="fig4"  position="float"><label><xref ref-type="fig" rid="fig4">Figure 4</xref></label><caption><title> The part of the system using the semantic model</title></caption><graphic mimetype="image"   position="float"  xlink:type="simple"  xlink:href="http://html.scirp.org/file/4-8102669x8.png"/></fig><p>mar standards (The language “Baoule” has one) as well as a dictionary and a grammar own speaker; these reflect the “habits” of the speaker and is continuously enriched. Then SEMANTIC MODEL seeks to optimize the identification of the message by analyzing the context of the words and while basing on both its own common language semantics and on cleanning the speaker semantics (a style). This modeling is usually built from the analysis of sequences of words from a large textual corpus. This clean semantics will be enriched as you use the software. Most softwares also allow enriching the analysis of texts that reflect the stylistic habits of the speaker. These two modules work together and it is easy to conceive that there is a feedback between them.</p><p>Initially, the dictionary associated with these two modules were based on fixed syntax language models, that is to say, modeled on a grammar defined by a rigid set of rules (this is not the case in most African languages including the “Baoule” language).</p><p>Then, the voice recognition software has evolved into the use of local probabilistic models: recognition no longer performs at a word but at a series of words, called n-gram where n is the length words in a sequence .The statistics of these models are obtained from standard texts and may be enriched gradually. See <xref ref-type="fig" rid="fig5">Figure 5</xref> below.</p><p>Here too, Hidden Markov Models are those currently used to describe the probabilistic aspects. the most advanced software tend to combine the advantages of statistical models and fixed syntax models in what is called the “probabilistic grammars”, the idea being to derive from fixed grammars of probabilities that can be combined with those of a probabilistic model. In recent approaches, it becomes difficult to distinguish the syntactic model of the semantic model and we rather speak of a single language model.</p></sec></sec><sec id="s4"><title>4. Hidden Markov Model Discrete Time</title><sec id="s4_1"><title>4.1. Overview and Features</title><sec id="s4_1_1"><title>4.1.1. Fundamentals</title><p>Hidden Markov Models (HMM) were introduced by Baum and his collaborators in the 60s and the 70s [<xref ref-type="bibr" rid="scirp.70590-ref1">1</xref>] . This model is closely related to Probabilistic Automata (PAs) [<xref ref-type="bibr" rid="scirp.70590-ref2">2</xref>] . A probabilistic automaton is defined by a structure composed of states and transitions, and a set of probability distribution on transitions. Each transition is associated with a symbol of a finite alphabet. This symbol is generated every time the transition is taken. An HMM is also defined by a structure consisting of states and transitions and by</p><fig id="fig5"  position="float"><label><xref ref-type="fig" rid="fig5">Figure 5</xref></label><caption><title> Semantic model</title></caption><graphic mimetype="image"   position="float"  xlink:type="simple"  xlink:href="http://html.scirp.org/file/4-8102669x9.png"/></fig><p>a set of probability distribution over the transitions. The essential difference is that the IPs symbol generation is performed on the states, and not on transitions. In addition, is associated with each symbol, not a state, but a probability distribution of the symbols of the alphabet.</p><p>HMMs are used to model the observation sequences. These observations may be discrete (e.g., characters from a finite alphabet) or continuous (the frequency of a signal, a temperature, etc.). The first area in which the HMMs have been applied is the speech processing in early 1970 [<xref ref-type="bibr" rid="scirp.70590-ref3">3</xref>] [<xref ref-type="bibr" rid="scirp.70590-ref4">4</xref>] . In this area, the HMM will rapidly become the reference model, and most of the techniques for using and implementing HMM have been developed in the context of these applications. These techniques were then applied and adapted successfully to the problem of recognition of handwritten texts [<xref ref-type="bibr" rid="scirp.70590-ref5">5</xref>] [<xref ref-type="bibr" rid="scirp.70590-ref6">6</xref>] and analysis of biological sequences [<xref ref-type="bibr" rid="scirp.70590-ref7">7</xref>] [<xref ref-type="bibr" rid="scirp.70590-ref8">8</xref>] . Theorems, rating and proposals that follow are largely from [<xref ref-type="bibr" rid="scirp.70590-ref9">9</xref>] .</p></sec><sec id="s4_1_2"><title>4.1.2. Characteristics of HMM</title><p>A sequence <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x10.png" xlink:type="simple"/></inline-formula> of random variables with values in a finite set E is a Markov chain if the following property holds (Markov property):</p><p><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x11.png" xlink:type="simple"/></inline-formula>for any time k and any suite <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x12.png" xlink:type="simple"/></inline-formula></p><p>Note that notion generalizes the notion of deterministic dynamical system (finite state machine recurrent sequence, or ordinary differential equation): the probability distribution of the present state <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x13.png" xlink:type="simple"/></inline-formula> depends only on the immediate past state<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x14.png" xlink:type="simple"/></inline-formula>.</p><p>A Markov chain <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x15.png" xlink:type="simple"/></inline-formula> is entirely characterized by the data</p><p>・ the original legislation<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x16.png" xlink:type="simple"/></inline-formula>; <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x17.png" xlink:type="simple"/></inline-formula>for all <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x18.png" xlink:type="simple"/></inline-formula></p><p>・ and the transition matrix<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x19.png" xlink:type="simple"/></inline-formula>; <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x20.png" xlink:type="simple"/></inline-formula>for all <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x20.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x21.png" xlink:type="simple"/></inline-formula></p><p>supposedly independent of time k (homogeneous Markov chain).</p><p>Knowing the transition probabilities that exist between two succesive times is enough to globally characterize a Markov chain.</p><p>Proposal</p><p><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x22.png" xlink:type="simple"/></inline-formula>is a probability on E, and <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x22.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x23.png" xlink:type="simple"/></inline-formula> a Markov matrix E</p><p>The probability distribution of the Markov chain <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x24.png" xlink:type="simple"/></inline-formula> of <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x24.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x25.png" xlink:type="simple"/></inline-formula> original legislation and <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x24.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x25.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x26.png" xlink:type="simple"/></inline-formula> transition matrix is given by</p><p><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x27.png" xlink:type="simple"/></inline-formula>for any time k, and any suite <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x27.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x28.png" xlink:type="simple"/></inline-formula></p><p>In this model the suite is not observed directly after<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x29.png" xlink:type="simple"/></inline-formula>, but observations are available <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x29.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x30.png" xlink:type="simple"/></inline-formula> with values in a finite space O (if symbolic) or <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x29.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x30.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x31.png" xlink:type="simple"/></inline-formula> (digital case), collected through a channel without memory, that is to say, conditionally to <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x29.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x30.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x31.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x32.png" xlink:type="simple"/></inline-formula> states.</p><p>i. the observations <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x33.png" xlink:type="simple"/></inline-formula> are mutually independent, and</p><p>ii. each observation <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x34.png" xlink:type="simple"/></inline-formula> depends only on the <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x34.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x35.png" xlink:type="simple"/></inline-formula> at the same time</p><p>This property is expressed as follows:</p><p><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x36.png" xlink:type="simple"/></inline-formula>for any result, <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x36.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x37.png" xlink:type="simple"/></inline-formula>, and every sequence <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x36.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x37.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x38.png" xlink:type="simple"/></inline-formula></p><p>Example</p><p>Assume that the observations <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x39.png" xlink:type="simple"/></inline-formula> are connected with states <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x39.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x40.png" xlink:type="simple"/></inline-formula> follows <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x39.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x40.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x41.png" xlink:type="simple"/></inline-formula> where the sequence <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x39.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x40.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x41.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x42.png" xlink:type="simple"/></inline-formula> is a Gaussian white noise dimension, with zero mean and covariance matrix R reversible, independent of the Markov chain <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x39.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x40.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x41.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x42.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x43.png" xlink:type="simple"/></inline-formula> function h defined on E with values in <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x39.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x40.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x41.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x42.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x43.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x44.png" xlink:type="simple"/></inline-formula> is characterized by the data of a finite family <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x39.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x40.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x41.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x42.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x43.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x44.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x45.png" xlink:type="simple"/></inline-formula> vectors of<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x39.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x40.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x41.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x42.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x43.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x44.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x45.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x46.png" xlink:type="simple"/></inline-formula>, and was</p><disp-formula id="scirp.70590-formula540"><graphic  xlink:href="http://html.scirp.org/file/4-8102669x47.png"  xlink:type="simple"/></disp-formula><p>conditionally to<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x48.png" xlink:type="simple"/></inline-formula>, random vectors <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x48.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x49.png" xlink:type="simple"/></inline-formula> are mutually independent, and each <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x48.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x49.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x50.png" xlink:type="simple"/></inline-formula> is a Gaussian random vector of dimension d, medium <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x48.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x49.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x50.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x51.png" xlink:type="simple"/></inline-formula> and R covariance matrix so that no memory channel property is verified.</p><p>A hidden Markov model <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x52.png" xlink:type="simple"/></inline-formula> is fully characterized by the particular</p><p>The original legislation<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x53.png" xlink:type="simple"/></inline-formula>; <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x53.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x54.png" xlink:type="simple"/></inline-formula>for all <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x53.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x54.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x55.png" xlink:type="simple"/></inline-formula></p><p>The transition matrix<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x56.png" xlink:type="simple"/></inline-formula>; <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x56.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x57.png" xlink:type="simple"/></inline-formula>for all <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x56.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x57.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x58.png" xlink:type="simple"/></inline-formula> andemission densities<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x56.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x57.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x58.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x59.png" xlink:type="simple"/></inline-formula>; <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x56.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x57.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x58.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x59.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x60.png" xlink:type="simple"/></inline-formula>for all <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x56.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x57.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x58.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x59.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x60.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x61.png" xlink:type="simple"/></inline-formula> for any and all<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x56.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x57.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x58.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x59.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x60.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x61.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x62.png" xlink:type="simple"/></inline-formula>.</p><p>So just a local data (transition probabilities between two successive times, and densities of issue at a time) comprehensively characterizes a hidden Markov model, example: for K = 3, it comes:</p><disp-formula id="scirp.70590-formula541"><graphic  xlink:href="http://html.scirp.org/file/4-8102669x63.png"  xlink:type="simple"/></disp-formula><p>Proposal: The probability distribution of the hidden Markov model <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x64.png" xlink:type="simple"/></inline-formula> initial <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x64.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x65.png" xlink:type="simple"/></inline-formula> law of <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x64.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x65.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x66.png" xlink:type="simple"/></inline-formula> transition matrix, and g emission densities, is given by <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x64.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x65.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x66.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x67.png" xlink:type="simple"/></inline-formula></p><p>for all time k following<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x68.png" xlink:type="simple"/></inline-formula>, and every sequence <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x68.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x69.png" xlink:type="simple"/></inline-formula> is denoted by<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x68.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x69.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x70.png" xlink:type="simple"/></inline-formula>, the parameters characteristic of the model, and we focus on three issues:</p><p>Problem No. 1: Evaluate the model: it comes to efficiently compute the probability distribution of the following observations <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x71.png" xlink:type="simple"/></inline-formula> (or likelihood function) according to the parameters of the model M. The answer to this problem is provided by the forward Baum equation.</p><p>Problem No. 2: Identify the model: given a series of observations<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x72.png" xlink:type="simple"/></inline-formula>, this is to calculate the maximum likelihood estimator for the unknown parameters of the model M. The answer to this problem is provided by the re-estimation formulas of Baum-Welch, defining an iterative algorithm to maximize the likelihood function.</p><p>Problem No. 3: Estimate the condition of the system: given a sequence of obser- vations<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x73.png" xlink:type="simple"/></inline-formula>, it is to estimated recursively the state <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x73.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x74.png" xlink:type="simple"/></inline-formula> (filtering Song), or a good estimate <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x73.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x74.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x75.png" xlink:type="simple"/></inline-formula> intermediate state for <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x73.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x74.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x75.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x76.png" xlink:type="simple"/></inline-formula> (smoothing Song), or an overall estimate of the sequence of states<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x73.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x74.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x75.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x76.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x77.png" xlink:type="simple"/></inline-formula>, for a given model M. the response to first two problems is provided by the forward and backward equations Baum, which calculate the conditional probability distribution of <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x73.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x74.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x75.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x76.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x77.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x78.png" xlink:type="simple"/></inline-formula> state given observations<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x73.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x74.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x75.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x76.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x77.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x78.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x79.png" xlink:type="simple"/></inline-formula>.</p><p>The answer to the last problem is provided by a dynamic programming algorithm, the Viterbi algorithm, which maximizes the conditional probability distribution of the sequence of states <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x80.png" xlink:type="simple"/></inline-formula> given observations<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x80.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x81.png" xlink:type="simple"/></inline-formula>.</p></sec></sec><sec id="s4_2"><title>4.2. Equations Forward/Backward Baum</title><p>We first present a first method (basic but inefficient) to calculate the probability dis- tribution of observations<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x82.png" xlink:type="simple"/></inline-formula>.</p><p>Proposal: The probability distribution of observations <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x83.png" xlink:type="simple"/></inline-formula> is given (in the digital case) by</p><p><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x84.png" xlink:type="simple"/></inline-formula>for any sequence<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x84.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x85.png" xlink:type="simple"/></inline-formula>.</p><p>Note that elementary method provides a first expression for the conditional probabi- lity distribution of the sequence of states <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x86.png" xlink:type="simple"/></inline-formula> given observations <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x86.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x87.png" xlink:type="simple"/></inline-formula> (in digital case):</p><disp-formula id="scirp.70590-formula542"><graphic  xlink:href="http://html.scirp.org/file/4-8102669x88.png"  xlink:type="simple"/></disp-formula><p>and the likelihood of the model (obtained using the following observations <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x89.png" xlink:type="simple"/></inline-formula> in place of dummy variables):</p><disp-formula id="scirp.70590-formula543"><graphic  xlink:href="http://html.scirp.org/file/4-8102669x90.png"  xlink:type="simple"/></disp-formula><p>we deduce the following identities:</p><p><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x91.png" xlink:type="simple"/></inline-formula>.</p><p>Note the number of operations required to calculate the probability distribution of observations <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x92.png" xlink:type="simple"/></inline-formula> from this basic method is significant for each possible path <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x92.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x93.png" xlink:type="simple"/></inline-formula> of the Markov chain, you must compute the product of <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x92.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x93.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x94.png" xlink:type="simple"/></inline-formula> words, and there is <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x92.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x93.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x94.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x95.png" xlink:type="simple"/></inline-formula> different possible paths the total number of elementary operations (additions and multiplications) thus made is of the order of <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x92.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x93.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x94.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x95.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x96.png" xlink:type="simple"/></inline-formula> the number is growing exponentially with the number n of observations. we define the forward <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x92.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x93.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x94.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x95.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x96.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x97.png" xlink:type="simple"/></inline-formula> (seen as a row vector) by <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x92.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x93.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x94.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x95.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x96.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x97.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x98.png" xlink:type="simple"/></inline-formula> for all<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x92.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x93.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x94.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x95.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x96.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x97.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x98.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x99.png" xlink:type="simple"/></inline-formula>.</p><p>Note the forward variable used to calculate the conditional probability distribution of</p><p>the present state <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x100.png" xlink:type="simple"/></inline-formula> given observations<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x100.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x101.png" xlink:type="simple"/></inline-formula>: <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x100.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x101.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x102.png" xlink:type="simple"/></inline-formula></p><p>for all<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x103.png" xlink:type="simple"/></inline-formula>. (In this sense, <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x103.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x104.png" xlink:type="simple"/></inline-formula>is a distribution of non-normalized probability), and the normalization constant <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x103.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x104.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x105.png" xlink:type="simple"/></inline-formula> is interpreted as the likelihood of the model given observations<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x103.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x104.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x105.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x106.png" xlink:type="simple"/></inline-formula>.</p><p>Theorem: The sequence <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x107.png" xlink:type="simple"/></inline-formula> satisfies the following recurrence equation:</p><p><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x108.png" xlink:type="simple"/></inline-formula>for all <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x108.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x109.png" xlink:type="simple"/></inline-formula> with the initial condition <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x108.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x109.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x110.png" xlink:type="simple"/></inline-formula> for any<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x108.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x109.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x110.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x111.png" xlink:type="simple"/></inline-formula>.</p><p>Note this statement result component-by-component can also be made for the variable forward view as a row vector <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x112.png" xlink:type="simple"/></inline-formula> and<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x112.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x113.png" xlink:type="simple"/></inline-formula>.</p><p>Note the recursive calculation of the variable forward <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x114.png" xlink:type="simple"/></inline-formula> involves only the product matrix/vector, and to calculate more efficiently the probability distribution of obser- vations <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x114.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x115.png" xlink:type="simple"/></inline-formula> simply <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x114.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x115.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x116.png" xlink:type="simple"/></inline-formula> elementary operations (additions and multi- plications) to move from time k to time <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x114.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x115.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x116.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x117.png" xlink:type="simple"/></inline-formula> the total number of elementary operations to be performed is thus of the order of: <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x114.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x115.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x116.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x117.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x118.png" xlink:type="simple"/></inline-formula>this number grows only linearly with the number n of observations.</p><p>Digital implementation: Instead of first solving the equation for the forward non-standardized version of the conditional distribution, defined at any time k as <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x119.png" xlink:type="simple"/></inline-formula> for all <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x119.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x120.png" xlink:type="simple"/></inline-formula> and then deduct the normalization constant (likelihood) and the normalized version of the conditional distribution (filter)</p><p><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x121.png" xlink:type="simple"/></inline-formula>and <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x121.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x122.png" xlink:type="simple"/></inline-formula></p><p>It is more efficient, on a digital point of view, spread directly log-likelihood and filter.</p><p>Proposal: Following <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x123.png" xlink:type="simple"/></inline-formula> Verie the following recurrent equation:</p><p><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x124.png" xlink:type="simple"/></inline-formula>for all <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x124.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x125.png" xlink:type="simple"/></inline-formula> with the initial condition</p><p><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x126.png" xlink:type="simple"/></inline-formula>for any<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x126.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x127.png" xlink:type="simple"/></inline-formula>.</p><p>where the normalization constants are defined by <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x128.png" xlink:type="simple"/></inline-formula> and<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x128.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x129.png" xlink:type="simple"/></inline-formula>.</p><p>Note this result statement component-by-component may also be formulated for the</p><p>normalized forward variable seen as a row vector <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x130.png" xlink:type="simple"/></inline-formula> and <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x130.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x131.png" xlink:type="simple"/></inline-formula></p><p>where the normalization constants are defined by <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x132.png" xlink:type="simple"/></inline-formula> and<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x132.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x133.png" xlink:type="simple"/></inline-formula>.</p><p>Note: Following <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x134.png" xlink:type="simple"/></inline-formula> truth the following recurrent equation:</p><p><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x135.png" xlink:type="simple"/></inline-formula>with the initial condition <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x135.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x136.png" xlink:type="simple"/></inline-formula> and iterating log<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x135.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x136.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x137.png" xlink:type="simple"/></inline-formula>. For all intermediate time k, less than the final instant n, is defined <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x135.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x136.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x137.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x138.png" xlink:type="simple"/></inline-formula> for all<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x135.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x136.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x137.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x138.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x139.png" xlink:type="simple"/></inline-formula>.</p><p>Note: That variable allows to calculate the conditional probability distribution of the</p><p>present state <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x140.png" xlink:type="simple"/></inline-formula> knowing all comments<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x140.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x141.png" xlink:type="simple"/></inline-formula>, <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x140.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x141.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x142.png" xlink:type="simple"/></inline-formula></p><p>for all <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x143.png" xlink:type="simple"/></inline-formula> with the normalization constant<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x143.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x144.png" xlink:type="simple"/></inline-formula>.</p><p>Note: Fix the state at time k allows a break between the past up to time <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x145.png" xlink:type="simple"/></inline-formula> and the future from time<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x145.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x146.png" xlink:type="simple"/></inline-formula>. This justifies the introduction of the variable backward <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x145.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x146.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x147.png" xlink:type="simple"/></inline-formula> (seen as a column vector) and defined as:</p><p><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x148.png" xlink:type="simple"/></inline-formula>for any <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x148.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x149.png" xlink:type="simple"/></inline-formula></p><p>and in particular <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x150.png" xlink:type="simple"/></inline-formula> for all <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x150.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x151.png" xlink:type="simple"/></inline-formula> with this definition, is obtained <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x150.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x151.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x152.png" xlink:type="simple"/></inline-formula> for all<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x150.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x151.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x152.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x153.png" xlink:type="simple"/></inline-formula>.</p><p>Note: Conditionally <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x154.png" xlink:type="simple"/></inline-formula> the X_ suite <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x154.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x155.png" xlink:type="simple"/></inline-formula> to come hidden states is a Markov chain, from initial law <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x154.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x155.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x156.png" xlink:type="simple"/></inline-formula> (line i the <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x154.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x155.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x156.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x157.png" xlink:type="simple"/></inline-formula> matrix), that is to say that <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x154.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x155.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x156.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x157.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x158.png" xlink:type="simple"/></inline-formula> for all <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x154.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x155.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x156.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x157.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x158.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x159.png" xlink:type="simple"/></inline-formula> and <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x154.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x155.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x156.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x157.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x158.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x159.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x160.png" xlink:type="simple"/></inline-formula> transition matrix it follows that the backward variable can be interpreted as the likelihood of the model derived from the <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x154.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x155.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x156.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x157.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x158.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x159.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x160.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x161.png" xlink:type="simple"/></inline-formula> state at time k given observations<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x154.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x155.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x156.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x157.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x158.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x159.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x160.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x161.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x162.png" xlink:type="simple"/></inline-formula>.</p><p>Theorem:</p><p>After <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x163.png" xlink:type="simple"/></inline-formula> Verie recurrent retrograde following equation:</p><p><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x164.png" xlink:type="simple"/></inline-formula>for all <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x164.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x165.png" xlink:type="simple"/></inline-formula> with the initial condition: <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x164.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x165.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x166.png" xlink:type="simple"/></inline-formula>for all<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x164.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x165.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x166.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x167.png" xlink:type="simple"/></inline-formula>.</p><p>Note: This result statement component-by-component can also be formulated for the backward view variable as a column vector <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x168.png" xlink:type="simple"/></inline-formula> and<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x168.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x169.png" xlink:type="simple"/></inline-formula>.</p><p>Proposal: the forward and backward equations are dual to one another:</p><p><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x170.png" xlink:type="simple"/></inline-formula>not dependent of the time in question</p><p>Proposal: For the distribution of conditional probability of transition <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x171.png" xlink:type="simple"/></inline-formula> at an intermediate time given observations <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x171.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x172.png" xlink:type="simple"/></inline-formula> until the final moment is given by:</p><p><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x173.png" xlink:type="simple"/></inline-formula>for all <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x173.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x174.png" xlink:type="simple"/></inline-formula></p><p>By summing for all <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x175.png" xlink:type="simple"/></inline-formula> and using the equation backward, or by summing for all i ∈ E and using the forward equation, we find the following results in terms of product component-by-component variables forward and backward.</p><p>Corollary: the conditional probability distribution of the present state <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x176.png" xlink:type="simple"/></inline-formula> knowing</p><p>all comments <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x177.png" xlink:type="simple"/></inline-formula> is given by <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x177.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x178.png" xlink:type="simple"/></inline-formula> with the definition</p><p><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x179.png" xlink:type="simple"/></inline-formula>for all<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x179.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x180.png" xlink:type="simple"/></inline-formula>.</p><p>Note: Verie one that constant Standards</p><disp-formula id="scirp.70590-formula544"><graphic  xlink:href="http://html.scirp.org/file/4-8102669x181.png"  xlink:type="simple"/></disp-formula><p>and <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x182.png" xlink:type="simple"/></inline-formula></p><p>do not depend on the time in question, and are interpreted as the likelihood of the model given observations<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x183.png" xlink:type="simple"/></inline-formula>. instead of first solve the backward and forward equation equation separately, and to successively deduct the non-normalized version of the conditional distribution, defined at any instant k as <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x183.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x184.png" xlink:type="simple"/></inline-formula> for all <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x183.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x184.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x185.png" xlink:type="simple"/></inline-formula></p><p>then the normalized version of the conditional distribution (smoother)</p><p><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x186.png" xlink:type="simple"/></inline-formula>.</p><p>It is more efficient on a digital point of view, spread directly log-likelihood and filter,</p><p>then spread the variable defned at any time k as <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x187.png" xlink:type="simple"/></inline-formula> for any<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x187.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x188.png" xlink:type="simple"/></inline-formula>.</p><p>Note: That with normalization of the backward variable, the conditional probability distribution of <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x189.png" xlink:type="simple"/></inline-formula> state given observations <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x189.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x190.png" xlink:type="simple"/></inline-formula> is expressed as</p><p><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x191.png" xlink:type="simple"/></inline-formula>for all<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x191.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x192.png" xlink:type="simple"/></inline-formula>.</p><p>Proposal: Following <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x193.png" xlink:type="simple"/></inline-formula> Verie recurrent retrograde following equation:</p><p><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x194.png" xlink:type="simple"/></inline-formula>for all<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x194.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x195.png" xlink:type="simple"/></inline-formula>, with the initial condition: <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x194.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x195.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x196.png" xlink:type="simple"/></inline-formula>for all</p><p><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x197.png" xlink:type="simple"/></inline-formula>where the normalization constants are those already defined for the normali- zation of the variable forward.</p><p>Note: This result statement component-by-component can also be formulated for</p><p>backward standardized variable viewed as a column vector <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x198.png" xlink:type="simple"/></inline-formula> and</p><p><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x199.png" xlink:type="simple"/></inline-formula>where the normalization constants are those already defined for the normalization of the variable forward.</p><p>Note: It is noted that <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x200.png" xlink:type="simple"/></inline-formula> for all <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x200.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x201.png" xlink:type="simple"/></inline-formula></p><p>And postponing this identity in the expressions obtained above, we Verie that the conditional probability distribution of the transition <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x202.png" xlink:type="simple"/></inline-formula> given observations <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x202.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x203.png" xlink:type="simple"/></inline-formula> is expressed as</p><p><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x204.png" xlink:type="simple"/></inline-formula>for<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x204.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x205.png" xlink:type="simple"/></inline-formula>.</p></sec><sec id="s4_3"><title>4.3. Viterbi Algorithm</title><p>Forward and backward variables used to calculate the conditional probability distribution of the state this<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x206.png" xlink:type="simple"/></inline-formula>, or <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x206.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x207.png" xlink:type="simple"/></inline-formula> state at an intermediate moment, given observations</p><p><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x208.png" xlink:type="simple"/></inline-formula>defined by <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x208.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x209.png" xlink:type="simple"/></inline-formula> for all<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x208.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x209.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x210.png" xlink:type="simple"/></inline-formula>, and</p><p><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x211.png" xlink:type="simple"/></inline-formula>for any <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x211.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x212.png" xlink:type="simple"/></inline-formula> respectively, where the normalization con-</p><p>stant <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x213.png" xlink:type="simple"/></inline-formula> does not depend on the time in question, and interprets as the likelihood of the model given observations<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x213.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x214.png" xlink:type="simple"/></inline-formula>. it is not necessary to calculate the conditional average, but can be used however the estimator of maximum a posteriori, which minimizes the likelihood of the estimation error given observations <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x213.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x214.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x215.png" xlink:type="simple"/></inline-formula> and defined for the present state</p><p><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x216.png" xlink:type="simple"/></inline-formula>and for the state to an inter-</p><p>mediate time by <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x217.png" xlink:type="simple"/></inline-formula> it may hap-</p><p>pen that the sequence <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x218.png" xlink:type="simple"/></inline-formula> generated is inconsistent with the model, in the following sense: it can happen that is obtained <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x218.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x219.png" xlink:type="simple"/></inline-formula> and <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x218.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x219.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x220.png" xlink:type="simple"/></inline-formula> for two successive times, while <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x218.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x219.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x220.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x221.png" xlink:type="simple"/></inline-formula> for the same pair<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x218.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x219.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x220.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x221.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x222.png" xlink:type="simple"/></inline-formula>, which meant that the transition from state i to state j is just impossible for the model for this reason, rather it uses another estimator, called trajectoriel maximum a posteriori estimator, defined by<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x218.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x219.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x220.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x221.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x222.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x223.png" xlink:type="simple"/></inline-formula>.</p><p>And minimizes the probability of the estimation error of the sequence of hidden states given observations <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x224.png" xlink:type="simple"/></inline-formula> it is of course not possible to perform this maximization exhaustive manner, listing all <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x224.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x225.png" xlink:type="simple"/></inline-formula> possible trajectories: the efficient calculation of this estimator is provided by a dynamic programming algorithm called Viterbi algorithm.</p></sec><sec id="s4_4"><title>4.4. Re-Estimation Formulas Baum-Welch</title><p>So far, the focus was on the estimation of a hidden condition or because of successive hidden states, from a series of observations and for a given model. The goal here is to identifier the model, that is to say, to estimate the parameters of the model characteristics, from a series of observations, and the approach taken is that of estimation maximum likelihood.</p><p>In the digital case, we look at the case of the Gaussian emission densities characterized by the data of finite <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x226.png" xlink:type="simple"/></inline-formula> <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x226.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x227.png" xlink:type="simple"/></inline-formula> vectors and of finite Family <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x226.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x227.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x228.png" xlink:type="simple"/></inline-formula> matrices invertible covariance, that is to say:</p><disp-formula id="scirp.70590-formula545"><graphic  xlink:href="http://html.scirp.org/file/4-8102669x229.png"  xlink:type="simple"/></disp-formula><p>The likelihood function of the model <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x230.png" xlink:type="simple"/></inline-formula> admits expression</p><disp-formula id="scirp.70590-formula546"><graphic  xlink:href="http://html.scirp.org/file/4-8102669x231.png"  xlink:type="simple"/></disp-formula><p>obtained with the basic method, and we will study an iterative algorithm to maximize <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x232.png" xlink:type="simple"/></inline-formula> likelihood function with respect to the parameters <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x232.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x233.png" xlink:type="simple"/></inline-formula> model of either <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x232.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x233.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x234.png" xlink:type="simple"/></inline-formula> another model, for which the likelihood function takes the value</p><disp-formula id="scirp.70590-formula547"><graphic  xlink:href="http://html.scirp.org/file/4-8102669x235.png"  xlink:type="simple"/></disp-formula><p>the (log) likelihood ratio between the <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x236.png" xlink:type="simple"/></inline-formula> and the <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x236.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x237.png" xlink:type="simple"/></inline-formula> is reduced by</p><disp-formula id="scirp.70590-formula548"><graphic  xlink:href="http://html.scirp.org/file/4-8102669x238.png"  xlink:type="simple"/></disp-formula><p>which vanishes when the model <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x239.png" xlink:type="simple"/></inline-formula> coincides with the model<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x239.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x240.png" xlink:type="simple"/></inline-formula>.</p><p>Maximize <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x241.png" xlink:type="simple"/></inline-formula> compared with parameters <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x241.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x242.png" xlink:type="simple"/></inline-formula> of the model <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x241.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x242.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x243.png" xlink:type="simple"/></inline-formula> thus ensures that the likelihood of the model which achieved maximum <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x241.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x242.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x243.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x244.png" xlink:type="simple"/></inline-formula> will be greater than the likelihood <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x241.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x242.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x243.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x244.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x245.png" xlink:type="simple"/></inline-formula> current model <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x241.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x242.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x243.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x244.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x245.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x246.png" xlink:type="simple"/></inline-formula> re-formulas Baum-Welch -Estimated allow explicitly find the parameters of the new model based on parameters <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x241.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x242.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x243.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x244.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x245.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x246.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x247.png" xlink:type="simple"/></inline-formula> of the current model <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x241.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x242.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x243.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x244.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x245.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x246.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x247.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x248.png" xlink:type="simple"/></inline-formula> by repeating this procedure, we construct a sequence of increasing likelihood models, and ideally this sequence converges to a model that reaches the maximum likelihood function.</p><p>Theorem</p><p>In the digital case with densities of Gaussian issue, the iterative algorithm for esti- mating the maximum likelihood of the model parameters from the observations<inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x249.png" xlink:type="simple"/></inline-formula>, is given by explicit formulas re-estimate</p><p><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x250.png" xlink:type="simple"/></inline-formula>and <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x250.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x251.png" xlink:type="simple"/></inline-formula> and</p><p><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x252.png" xlink:type="simple"/></inline-formula>and <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x252.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x253.png" xlink:type="simple"/></inline-formula></p><p>for all <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x254.png" xlink:type="simple"/></inline-formula> where the two sequences <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x254.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x255.png" xlink:type="simple"/></inline-formula> and <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x254.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x255.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x256.png" xlink:type="simple"/></inline-formula> are the standard equations of forward and backward solutions respectively for values <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x254.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x255.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x256.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x257.png" xlink:type="simple"/></inline-formula> parameters.</p><p>Note: Concretely, if <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x258.png" xlink:type="simple"/></inline-formula> denotes the current model in step <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x258.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x259.png" xlink:type="simple"/></inline-formula> of the algorithm, then for values <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x258.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x259.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x260.png" xlink:type="simple"/></inline-formula> the para- meters are calculated standardized solutions <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x258.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x259.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x260.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x261.png" xlink:type="simple"/></inline-formula> and <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x258.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x259.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x260.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x261.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x262.png" xlink:type="simple"/></inline-formula> of equations forward and backward respectively the parameters <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x258.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x259.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x260.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x261.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x262.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x263.png" xlink:type="simple"/></inline-formula> is calculated using the formulas to re-estimate what defines the new model <inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x258.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x259.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x260.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x261.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x262.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x263.png" xlink:type="simple"/></inline-formula><inline-formula><inline-graphic xlink:href="http://html.scirp.org/file/4-8102669x264.png" xlink:type="simple"/></inline-formula> to s next step of the algorithm.</p></sec></sec><sec id="s5"><title>5. Implementation</title><p>Our model is based on acoustic signal parameters. These parameters are obtained by calculating cepstral coefficients according to a Mel scale (MFCC Mel Frequency Cepstral Coefficients). Statistical realization of acoustic parameters of each phoneme is represented by a Hidden Markov model. Each phoneme is typically represented by 2 or 3 states, and multigaussienne density (GMM: Gaussian Mixture Model) is associated with each state. GMM densities with a large number of components designed to address multiple sources of variability that are affecting the speech signals (sex and age of the speaker, accent, noise).</p><p>For example: With the following data: Number of States (K = 2); π = [0.95 0.05; 0.05 0.95]; h = [−1 1]; σ<sup>2</sup> = [3 3]; υ = [0.5 0.5]; we have the <xref ref-type="fig" rid="fig6">Figure 6</xref> below.</p><p>A robust speech recognition system combines accuracy of identification with the ability to filter noise and adapt to other acoustical conditions such as speech and emphasis of the speaker. The design of a robust speech recognition algorithm is a complex task which requires detailed knowledge of signal processing and statistical modeling. Most speech recognition systems are classified as isolated or continuous. The isolated</p><fig id="fig6"  position="float"><label><xref ref-type="fig" rid="fig6">Figure 6</xref></label><caption><title> Graphic representation of the Hidden Markov Model</title></caption><graphic mimetype="image"   position="float"  xlink:type="simple"  xlink:href="http://html.scirp.org/file/4-8102669x265.png"/></fig><p>word recognition requires a short pause between each spoken word, while the speech recognition does not continue. Speech recognition systems can be classified as a dependent or speaker-independent. Speaker dependent system recognizes only the word of the voice of a particular speaker, while an independent speaker system can recognize any voice.</p><p>The implementation presented here uses features integrated into MATLAB and related products to develop the recognition algorithm. There are two main steps in the recognition of isolated words:</p><p>・ a learning phase and</p><p>・ a test phase.</p><p>The learning phase teaches the system by building its dictionary, an acoustic model for each word that the system has to recognize. In our example, the dictionary includes the numbers “zero” to “nine” in “Baoule” language. The test phase uses acoustic models of these numbers to recognize isolated words using a classification algorithm. We start with the the speech signal acquisition, and then we end with its analysis.</p><sec id="s5_1"><title>5.1. Speech Signal Acquisition</title><p>During the learning phase, it is necessary to record the repeated statements of each digit in the dictionary. For example, we repeat the word “nnou” (which means five in “Baoule” language) many times with a pause between each statement. That word will be saved in the file 'cinq.wav'. Using the following MATLAB code with a sound card standard PC, we capture ten seconds of speech from a microphone to 8000 samples per second. We obtained y that is a matrix of 8000 rows and one column. This approach works well for training data.</p></sec><sec id="s5_2"><title>5.2. Acquired Speech Signal Analysis</title><p>We first develop a word-detection algorithm that separates each word of ambient noise. We then obtain an acoustic model that provides a strong representation of each word in the stage of learning. Finally, we select an appropriate classification algorithm for testing.</p><sec id="s5_2_1"><title>5.2.1. The Development of a Word-Detection Algorithm</title><p>The word-detection algorithm continuously reads 160 samples frames from the data of “speech”. To detect single digits, we use a combination of the signal energy and have zero crossing for each speech frame.</p><p>The signal energy works well to detect sound signals, while the zero-crossing numbers work well for detecting non-voice signals. The calculation of these measures is simple using mathematical operators and MATLAB basic logic. To avoid identifying the ambient noise of speech, we assume that each individual word will last at least 25 milliseconds. In <xref ref-type="fig" rid="fig7">Figure 7</xref> below, we plot the speech signal “five” and the power of short duration and zero crossing measurement.</p><fig id="fig7"  position="float"><label><xref ref-type="fig" rid="fig7">Figure 7</xref></label><caption><title> Speech signal “five” and the power of short duration and zero crossing measurement</title></caption><graphic mimetype="image"   position="float"  xlink:type="simple"  xlink:href="http://html.scirp.org/file/4-8102669x266.png"/></fig></sec><sec id="s5_2_2"><title>5.2.2. Development of the Acoustic Model</title><p>A good acoustic model should be derived from the word of features that allow the system to distinguish different words in the dictionary. We know that different sounds are produced by varying the shape of the human vocal tract, and these different sounds can each have different frequencies. To investigate the frequency characteristics, we examine the density estimates Spectral Power (CSP) various spoken digits. Since the human vocal tract can be modeled as a filter on all poles, we use the parametric spectral estimation technique Yule-Walker of the window Signal Processing Toolbox to calculate the DSP. After importing a statement of a single digit in the variable “word” we use the MATLAB code below to view the DSP estimate: here there is the speech signal that we have acquired (<xref ref-type="fig" rid="fig8">Figure 8</xref>).</p><p>Because the Yule-Walker algorithm adapts a linear prediction filter model autoregression to the signal, you must supply an order of this filter. We select an arbitrary value of 12, which is typical for voice applications.</p><p><xref ref-type="fig" rid="fig9">Figure 9</xref> shows the PSD estimate of three different expressions of the words “one” and “two”. We can see the tops of the PSD remain consistent for a particular number, but differ from one figure to another. This means that we can draw the acoustic models in our system from the spectral characteristics.</p><p>A set of spectral characteristics commonly used in voice applications because of its robustness is Mel Frequency Cepstral Coefficients (MFCC). MFCC give a measure of</p><fig id="fig8"  position="float"><label><xref ref-type="fig" rid="fig8">Figure 8</xref></label><caption><title> Estimate of the PSD (Yule Walker) the word “five”</title></caption><graphic mimetype="image"   position="float"  xlink:type="simple"  xlink:href="http://html.scirp.org/file/4-8102669x267.png"/></fig><fig-group id="fig9"><label><xref ref-type="fig" rid="fig9">Figure 9</xref></label><caption><title> (a) Estimating the PSD (Yule Walker) in three different expressions of the word “one.”; (b) estimating the PSD (Yule Walker) in three different expressions of the word “two”.</title></caption><fig id ="fig9_1"><label>(b)</label><graphic mimetype="image"   position="float"  xlink:type="simple"  xlink:href="http://html.scirp.org/file/4-8102669x268.png"/></fig><fig id ="fig9_2"><label></label><graphic mimetype="image"   position="float"  xlink:type="simple"  xlink:href="http://html.scirp.org/file/4-8102669x269.png"/></fig></fig-group><p>the energy in overlapping boxes frequency of a deformed spectrum by (Mel) Frequency scale 1.</p><p>In the short term, the floor can be considered as stationary, MFCC characteristics of the vectors are calculated for each speech frame detected. Using many statements of a number and by combining all of the feature vectors, we can estimate a multidimensional probability density function (PDF) vectors to a specific figure. Repeating this process for each digit, the acoustic model is obtained for each digit. During the test phase, we extract the MFCC vectors figure test and use a probabilistic measure to determine the number of the source with the maximum likelihood.</p><p><xref ref-type="fig" rid="fig1">Figure 1</xref>0 shows the distribution of the first dimension of MFCC feature vectors ex-</p><fig-group id="fig10"><label><xref ref-type="fig" rid="fig1">Figure 1</xref>0</label><caption><title> (a) The distribution of the first dimension of MFCC feature vectors for the digit “one.”; (b) Overlay estimated Gaussian components (red) and all Gaussian mixture model (green) for distribution in (4a).</title></caption><fig id ="fig10_1"><label>(b)</label><graphic mimetype="image"   position="float"  xlink:type="simple"  xlink:href="http://html.scirp.org/file/4-8102669x270.png"/></fig><fig id ="fig10_2"><label></label><graphic mimetype="image"   position="float"  xlink:type="simple"  xlink:href="http://html.scirp.org/file/4-8102669x271.png"/></fig></fig-group><p>tracted from the training data for the digit “one.” We could use dfittool in StatisticsToolbox adapt to a PDF, but the distribution seems quite arbitrary, and standard distributions do not provide a good fit.</p><p>One solution is to adjust a mixture of Gaussian model (GMM), a sum of weighted Gaussian (<xref ref-type="fig" rid="fig1">Figure 1</xref>0(b)). The total density of Gaussian mixture is set by the weight of the mixture, the mean vectors and covariance matrices from all densities of the components. For the recognition of isolated digits, each digit is represented by the parameters of the GMM.</p><p>To estimate the parameters of a GMM for a set of MFCC feature vectors extracted from the figure when learning, we use an expectation maximization (EM) iterative algorithm for maximum likelihood (ML) estimation. Given some MFCC training data in MFCC train data variable (equal to five here), we use the GMM distribution Statistics Toolbox function for estimating GMM parameters. This function is all that is needed to perform the EM iterative calculations.</p></sec><sec id="s5_2_3"><title>5.2.3. Selecting a Classification Algorithm</title><p>After estimating a GMM for each digit, we have a dictionary for use in the testing phase. Given some test speech, we extracted again MFCC feature vectors of each frame of the detected word. The goal is to find the model numbers of the maximum a posteriori probability for all the long delivery tests, which reduces to maximize the value of log-likelihood.</p><p>Given a GMM model (equal to model here) model numbers and some feature vectors tests test data (equal to five here), the log-likelihood value is easily calculated using the post office in Statistics Toolbox: [P, log_like] = later (model, five); we repeat this calculation using the model of each digit. The test speech is classified as revenues at the MGM produce the maximum log-likelihood.</p></sec></sec></sec><sec id="s6"><title>6. Conclusions</title><p>In this article we presented an overview of HMM: their applications and conventional algorithms used in the literature, the generation probability calculation algorithms in a sequence by an HMM, the path search algorithm optimum, and the drive algorithms.</p><p>The speech signal is a complex form drowned in the noise. Its learning is part of complex intelligent activity [<xref ref-type="bibr" rid="scirp.70590-ref10">10</xref>] . By learning a starting model, we will build gradually an effective model for each of the phonemes of the “Baoule” language.</p><p>Note finally that HMMs have established themselves as the reference model for solving certain types of problems in many application areas, whether in speech recognition, modeling of biological sequences or for the extraction of information from textual data. However other formalisms such as neural networks can be used to improve the modeling. Our future work will focus on the modeling of the linguistic aspect of the “Baoule” language.</p></sec><sec id="s7"><title>Cite this paper</title><p>Konan, H., Soro, E., Asseu, O., Goore, B.T. and Gbegbe, R. (2016) Phoneme Sequence Modeling in the Context of Speech Signal Recognition in Language “Baoule”. Engineering, 8, 597-617. http://dx.doi.org/10.4236/eng.2016.89055</p></sec><sec id="s8"><title>ANNEXES</title><p>function [valeur,ante,dens] = Viterbi(X,A,p,m,sigma2,K,T)</p><p>begin</p><p>% densite d'emission</p><p>dens = ones(T,K);</p><p>dens =</p><p>exp(-0.5*(X'*ones(1,K)-ones(T,1)*m).^2./(ones(T,1)*sigma2))./sqrt(ones(T,1)*sigma2);</p><p>% fonction valeur</p><p>valeur = ones(T,K);</p><p>valeur(1,:) = p.*dens(1,:);</p><p>for t=2:T</p><p>[c,I] = max((ones(K,1)*valeur(t-1,:)).*A,[<xref ref-type="bibr" rid="scirp.70590-ref"></xref>],2);</p><p>valeur(t,:) = c'.*dens(t,:);</p><p>valeur(t,:) = valeur(t,:)/max(valeur(t,:));</p><p>ante(t,:) = I;</p><p>end</p><p>end</p><p>function [alpha,beta,dens,ll] = ForwardBackward(X,A,p,m,sigma2,K,T)</p><p>% densite d'emission</p><p>dens = ones(T,K);</p><p>dens =</p><p>exp(-0.5*(X'*ones(1,K)-ones(T,1)*m).^2./(ones(T,1)*sigma2))./sqrt(ones(T,1)*sigma2);</p><p>% variable forward</p><p>alpha = ones(T,K);</p><p>alpha(1,:) = p.*dens(1,:);</p><p>c(1) = sum(alpha(1,:));</p><p>alpha(1,:) = alpha(1,:)/c(1);</p><p>for t=2:T</p><p>alpha(t,:) = alpha(t-1,:)*A;</p><p>alpha(t,:) = alpha(t,:).*dens(t,:);</p><p>c(t) = sum(alpha(t,:));</p><p>alpha(t,:) = alpha(t,:)/c(t);</p><p>end</p><p>ll = cumsum(log(c));</p><p>% variable backward</p><p>beta = ones(K,T);</p><p>for t=T-1:-1:1</p><p>beta(:,t) = beta(:,t+1).*(dens(t+1,:))';</p><p>beta(:,t) = A*beta(:,t);</p><p>beta(:,t) = beta(:,t)/(alpha(t,:)*beta(:,t));</p><p>end</p><p>function [X,Y] = gen(A,p,m,sigma2,T)</p><p>begin</p><p>sigma = sqrt(sigma2);</p><p>Y(1) = multinomiale(p);</p><p>for t=2:T</p><p>q = A(Y(t-1),:);</p><p>Y(t) = multinomiale(q);</p><p>end</p><p>w = randn(1,T);</p><p>for t=1:T</p><p>moyenne = m(Y(t));</p><p>ecart_type = sigma(Y(t));</p><p>X(t) = moyenne+ecart_type*w(t);</p><p>end</p><p>end</p><p>function Px = stpower(x,N)</p><p>begin</p><p>M = length(x);</p><p>Px = zeros(M,1);</p><p>Px(1:N) = x(1:N)'*x(1:N)/N;</p><p>for m=(N+1):M</p><p>Px(m) = Px(m-1) + (x(m)^2 - x(m-N)^2)/N;</p><p>end</p><p>end</p><p>function Zx = stzerocross(x,N)</p><p>begin</p><p>M = length(x);</p><p>Zx = zeros(M,1);</p><p>Zx(1:N+1) = sum(abs(sign(x(2:N+1)) - sign(x(1:N))))/(2*N);</p><p>for (m=(N+2):M)</p><p>Zx(m) = Zx(m-1) + (abs(sign(x(m)) - sign(x(m-1))) ...</p><p>- abs(sign(x(m-N)) - sign(x(m-N-1))))/(2*N);</p><p>end</p><p>end</p></sec></body><back><ref-list><title>References</title><ref id="scirp.70590-ref1"><label>1</label><mixed-citation publication-type="other" xlink:type="simple">Baum, L.E., Petrie, T., Soules, G. and Weiss, N. (1970) A Maximization Technique Occurring in Statistical Analysis of Probabilistic Functions in Markov Chains. The Annals of Mathematical Statistics, 41, 164-171. http://dx.doi.org/10.1214/aoms/1177697196</mixed-citation></ref><ref id="scirp.70590-ref2"><label>2</label><mixed-citation publication-type="other" xlink:type="simple">Casacuberta, F. (1990) Some Relations among Stochastic Finite State Networks Used in Automatic Speech Recognition. IEEE Ttransactions on patern Analysis and Machine Intelligence, 12, 691-695. http://dx.doi.org/10.1109/34.56212</mixed-citation></ref><ref id="scirp.70590-ref3"><label>3</label><mixed-citation publication-type="other" xlink:type="simple">Lekoundji, J.-B.V. (2014) Modèles De Markov Cach&amp;eacute;s. Universit&amp;eacute; Du Qu&amp;eacute;bec à Montr&amp;eacute;al.</mixed-citation></ref><ref id="scirp.70590-ref4"><label>4</label><mixed-citation publication-type="other" xlink:type="simple">Rabiner, L.R. (1989) A Tutorial on Hidden Markov Models and Selected Application in Speech Recognition. Proceedings of the IEEE, 77, 257-286.  
http://dx.doi.org/10.1109/5.18626</mixed-citation></ref><ref id="scirp.70590-ref5"><label>5</label><mixed-citation publication-type="other" xlink:type="simple">Kundu, A., He, Y. and Bahl, P. (1988) Recognition of Handwritten Word: First and Second Order Hiden Markov Model Based Approach. IEEE Computer Society Conference on Computer Vision Pattern Recognition, CVPR’88, Ann Arbor, 5-9 June 1988, 457-462.  
http://dx.doi.org/10.1109/CVPR.1988.196275</mixed-citation></ref><ref id="scirp.70590-ref6"><label>6</label><mixed-citation publication-type="other" xlink:type="simple">Schenkel, M., Guyon, I. and Henderson, D. (1994) On-Line Cursive Script Recognition Using Time Delay Neural Networks and Hidden Markov Models. Proceedings of 1994 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP’94, Adelaide, 19-22 April 1994, II-637-II-640. http://dx.doi.org/10.1109/icassp.1994.389575</mixed-citation></ref><ref id="scirp.70590-ref7"><label>7</label><mixed-citation publication-type="other" xlink:type="simple">Haussler, D., Krogh, A., Mian, S. and Sjolander, K. (1992) Protein Modeling Using Hidden Markov Models: Analysis of Globins. Technical Report UCSC-CRL-92-23.</mixed-citation></ref><ref id="scirp.70590-ref8"><label>8</label><mixed-citation publication-type="other" xlink:type="simple">Durbin, R., Eddy, S., Krogh, A. and Mitchison, G. (1998) Biological Sequence Analysis Probabilistic Models of Proteins and Nucleic Acids. Cambridge University Press, Cambridge. http://dx.doi.org/10.1017/CBO9780511790492</mixed-citation></ref><ref id="scirp.70590-ref9"><label>9</label><mixed-citation publication-type="other" xlink:type="simple">In Franois Le Gland (2014-2015) T&amp;eacute;l&amp;eacute;com Bretagne Module F4B101A Traitement Statistique de l’Information. http://www.irisa.fr/aspi/legland/telecom-bretagne/</mixed-citation></ref><ref id="scirp.70590-ref10"><label>10</label><mixed-citation publication-type="other" xlink:type="simple">Corge, C. (1975) El&amp;eacute;ment d’informatique. Informatique et demarche de l’esprit. Librairie Larousse, Paris.</mixed-citation></ref></ref-list></back></article>