EM algorithm - E-step notation The 2019 Stack Overflow Developer Survey Results Are Inhow does expectation maximization work?Combining multiple posterior distributionsExpectation maximization modelingWhy does $p(X;|;Y) = displaystylefracp(Z,X;P(Z;$?Questions about Bayesian inferenceEstimating errors from optimization? (Genetic algorithm or otherwise)How to optimize the log likelihood to obtain parameters for the maximum likelihood estimate?Expectation Maximization Algorithm with latent variableVector-update form of Hill function for on-line fitting of modelIs it possible to express the posterior of the function of a parameter in terms of the posterior of the parameter?

Are USB sockets on wall outlets live all the time, even when the switch is off?

How to make payment on the internet without leaving a money trail?

Confusion about non-derivable continuous functions

If a poisoned arrow's piercing damage is reduced to 0, do you still get poisoned?

JSON.serialize: is it possible to suppress null values of a map?

Why can Shazam do this?

Deadlock Graph and Interpretation, solution to avoid

Why don't Unix/Linux systems traverse through directories until they find the required version of a linked library?

Is domain driven design an anti-SQL pattern?

What is this 4-propeller plane?

Manuscript was "unsubmitted" because the manuscript was deposited in Arxiv Preprints

Are there any other methods to apply to solving simultaneous equations?

Should I use my personal or workplace e-mail when registering to external websites for work purpose?

How come people say “Would of”?

Could JWST stay at L2 "forever"?

Why is Grand Jury testimony secret?

Is three citations per paragraph excessive for undergraduate research paper?

What is the motivation for a law requiring 2 parties to consent for recording a conversation

Inline version of a function returns different value then non-inline version

Idiomatic way to prevent slicing?

How are circuits which use complex ICs normally simulated?

Does a dangling wire really electrocute me if I'm standing in water?

Is "plugging out" electronic devices an American expression?

What is the best strategy for white in this position?



EM algorithm - E-step notation



The 2019 Stack Overflow Developer Survey Results Are Inhow does expectation maximization work?Combining multiple posterior distributionsExpectation maximization modelingWhy does $p(X;|;Y) = displaystylefracp(Z,X;P(Z;$?Questions about Bayesian inferenceEstimating errors from optimization? (Genetic algorithm or otherwise)How to optimize the log likelihood to obtain parameters for the maximum likelihood estimate?Expectation Maximization Algorithm with latent variableVector-update form of Hill function for on-line fitting of modelIs it possible to express the posterior of the function of a parameter in terms of the posterior of the parameter?










1












$begingroup$


I think I understand the gits of Expectation-Maximization algorithm and its altering nature, but I am puzzled by the notation. Lets see the following examples:



  1. in Stanford notes , the E-step is simply stated as posterior probability of latent variable $z$:

$$Q_i(z^(i)) := p(z^(i)|z^(i);theta)$$



where $z^(i)$ is latent variable sample, $x^(i)$ is observed data, $theta$ are the parameters maximized in M-step.



  1. in Original paper from 1977 the E-step looks as follows:

$$t^(p) = Ebig[ t(x)|y,Theta^(p) big]$$



where I believe the $y$ is observe variable, $x$ is latent variable, $Theta^(p)$ are model parameters used in M-step. To me, this looks like:



$$E_xbig[ p(x|y,Theta)big]$$



where the $x,y,Theta$ is the same as in point 2.



I appologize for introducing 2 notations, one in point 1. another in point 2. but I am trying to keep it consistent with the linked papers.




Question



The point of E-step is to obtain such values of latent variables, that they maximize the observation of complete data, given the current model parameters $theta$ or $Theta^(p)$. Then my question is, how do I formally get these values from the presented E-steps ?



I mean, what/where do I calculate in $Q_i(z^(i)) := p(z^(i)|z^(i);theta)$ ? Because it is just a definition of posterior distribution, there is no maximization, no operation to be done.



The second one $E_xbig[ p(x|y,Theta)big]$ is a bit more intuitive, because I am calculating an expectation of distributions (I think $t(x)$ is distribution of latent variable $x$). That means, I am looking for such values of $x$, that are expected -> gives maximum probability of realizing/happening.



Can someone formally show (and explain in layman's terms), how to obtain the values of the latent variables from the equations of E-step ?










share|cite|improve this question









$endgroup$
















    1












    $begingroup$


    I think I understand the gits of Expectation-Maximization algorithm and its altering nature, but I am puzzled by the notation. Lets see the following examples:



    1. in Stanford notes , the E-step is simply stated as posterior probability of latent variable $z$:

    $$Q_i(z^(i)) := p(z^(i)|z^(i);theta)$$



    where $z^(i)$ is latent variable sample, $x^(i)$ is observed data, $theta$ are the parameters maximized in M-step.



    1. in Original paper from 1977 the E-step looks as follows:

    $$t^(p) = Ebig[ t(x)|y,Theta^(p) big]$$



    where I believe the $y$ is observe variable, $x$ is latent variable, $Theta^(p)$ are model parameters used in M-step. To me, this looks like:



    $$E_xbig[ p(x|y,Theta)big]$$



    where the $x,y,Theta$ is the same as in point 2.



    I appologize for introducing 2 notations, one in point 1. another in point 2. but I am trying to keep it consistent with the linked papers.




    Question



    The point of E-step is to obtain such values of latent variables, that they maximize the observation of complete data, given the current model parameters $theta$ or $Theta^(p)$. Then my question is, how do I formally get these values from the presented E-steps ?



    I mean, what/where do I calculate in $Q_i(z^(i)) := p(z^(i)|z^(i);theta)$ ? Because it is just a definition of posterior distribution, there is no maximization, no operation to be done.



    The second one $E_xbig[ p(x|y,Theta)big]$ is a bit more intuitive, because I am calculating an expectation of distributions (I think $t(x)$ is distribution of latent variable $x$). That means, I am looking for such values of $x$, that are expected -> gives maximum probability of realizing/happening.



    Can someone formally show (and explain in layman's terms), how to obtain the values of the latent variables from the equations of E-step ?










    share|cite|improve this question









    $endgroup$














      1












      1








      1





      $begingroup$


      I think I understand the gits of Expectation-Maximization algorithm and its altering nature, but I am puzzled by the notation. Lets see the following examples:



      1. in Stanford notes , the E-step is simply stated as posterior probability of latent variable $z$:

      $$Q_i(z^(i)) := p(z^(i)|z^(i);theta)$$



      where $z^(i)$ is latent variable sample, $x^(i)$ is observed data, $theta$ are the parameters maximized in M-step.



      1. in Original paper from 1977 the E-step looks as follows:

      $$t^(p) = Ebig[ t(x)|y,Theta^(p) big]$$



      where I believe the $y$ is observe variable, $x$ is latent variable, $Theta^(p)$ are model parameters used in M-step. To me, this looks like:



      $$E_xbig[ p(x|y,Theta)big]$$



      where the $x,y,Theta$ is the same as in point 2.



      I appologize for introducing 2 notations, one in point 1. another in point 2. but I am trying to keep it consistent with the linked papers.




      Question



      The point of E-step is to obtain such values of latent variables, that they maximize the observation of complete data, given the current model parameters $theta$ or $Theta^(p)$. Then my question is, how do I formally get these values from the presented E-steps ?



      I mean, what/where do I calculate in $Q_i(z^(i)) := p(z^(i)|z^(i);theta)$ ? Because it is just a definition of posterior distribution, there is no maximization, no operation to be done.



      The second one $E_xbig[ p(x|y,Theta)big]$ is a bit more intuitive, because I am calculating an expectation of distributions (I think $t(x)$ is distribution of latent variable $x$). That means, I am looking for such values of $x$, that are expected -> gives maximum probability of realizing/happening.



      Can someone formally show (and explain in layman's terms), how to obtain the values of the latent variables from the equations of E-step ?










      share|cite|improve this question









      $endgroup$




      I think I understand the gits of Expectation-Maximization algorithm and its altering nature, but I am puzzled by the notation. Lets see the following examples:



      1. in Stanford notes , the E-step is simply stated as posterior probability of latent variable $z$:

      $$Q_i(z^(i)) := p(z^(i)|z^(i);theta)$$



      where $z^(i)$ is latent variable sample, $x^(i)$ is observed data, $theta$ are the parameters maximized in M-step.



      1. in Original paper from 1977 the E-step looks as follows:

      $$t^(p) = Ebig[ t(x)|y,Theta^(p) big]$$



      where I believe the $y$ is observe variable, $x$ is latent variable, $Theta^(p)$ are model parameters used in M-step. To me, this looks like:



      $$E_xbig[ p(x|y,Theta)big]$$



      where the $x,y,Theta$ is the same as in point 2.



      I appologize for introducing 2 notations, one in point 1. another in point 2. but I am trying to keep it consistent with the linked papers.




      Question



      The point of E-step is to obtain such values of latent variables, that they maximize the observation of complete data, given the current model parameters $theta$ or $Theta^(p)$. Then my question is, how do I formally get these values from the presented E-steps ?



      I mean, what/where do I calculate in $Q_i(z^(i)) := p(z^(i)|z^(i);theta)$ ? Because it is just a definition of posterior distribution, there is no maximization, no operation to be done.



      The second one $E_xbig[ p(x|y,Theta)big]$ is a bit more intuitive, because I am calculating an expectation of distributions (I think $t(x)$ is distribution of latent variable $x$). That means, I am looking for such values of $x$, that are expected -> gives maximum probability of realizing/happening.



      Can someone formally show (and explain in layman's terms), how to obtain the values of the latent variables from the equations of E-step ?







      statistics optimization machine-learning expected-value






      share|cite|improve this question













      share|cite|improve this question











      share|cite|improve this question




      share|cite|improve this question










      asked Mar 30 at 11:13









      Martin GMartin G

      618




      618




















          0






          active

          oldest

          votes












          Your Answer





          StackExchange.ifUsing("editor", function ()
          return StackExchange.using("mathjaxEditing", function ()
          StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
          StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
          );
          );
          , "mathjax-editing");

          StackExchange.ready(function()
          var channelOptions =
          tags: "".split(" "),
          id: "69"
          ;
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function()
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled)
          StackExchange.using("snippets", function()
          createEditor();
          );

          else
          createEditor();

          );

          function createEditor()
          StackExchange.prepareEditor(
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader:
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          ,
          noCode: true, onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          );



          );













          draft saved

          draft discarded


















          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3168183%2fem-algorithm-e-step-notation%23new-answer', 'question_page');

          );

          Post as a guest















          Required, but never shown

























          0






          active

          oldest

          votes








          0






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes















          draft saved

          draft discarded
















































          Thanks for contributing an answer to Mathematics Stack Exchange!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid


          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.

          Use MathJax to format equations. MathJax reference.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3168183%2fem-algorithm-e-step-notation%23new-answer', 'question_page');

          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          Triangular numbers and gcdProving sum of a set is $0 pmod n$ if $n$ is odd, or $fracn2 pmod n$ if $n$ is even?Is greatest common divisor of two numbers really their smallest linear combination?GCD, LCM RelationshipProve a set of nonnegative integers with greatest common divisor 1 and closed under addition has all but finite many nonnegative integers.all pairs of a and b in an equation containing gcdTriangular Numbers Modulo $k$ - Hit All Values?Understanding the Existence and Uniqueness of the GCDGCD and LCM with logical symbolsThe greatest common divisor of two positive integers less than 100 is equal to 3. Their least common multiple is twelve times one of the integers.Suppose that for all integers $x$, $x|a$ and $x|b$ if and only if $x|c$. Then $c = gcd(a,b)$Which is the gcd of 2 numbers which are multiplied and the result is 600000?

          Ingelân Ynhâld Etymology | Geografy | Skiednis | Polityk en bestjoer | Ekonomy | Demografy | Kultuer | Klimaat | Sjoch ek | Keppelings om utens | Boarnen, noaten en referinsjes Navigaasjemenuwww.gov.ukOffisjele webside fan it regear fan it Feriene KeninkrykOffisjele webside fan it Britske FerkearsburoNederlânsktalige ynformaasje fan it Britske FerkearsburoOffisjele webside fan English Heritage, de organisaasje dy't him ynset foar it behâld fan it Ingelske kultuergoedYnwennertallen fan alle Britske stêden út 'e folkstelling fan 2011Notes en References, op dizze sideEngland

          Հադիս Բովանդակություն Անվանում և նշանակություն | Դասակարգում | Աղբյուրներ | Նավարկման ցանկ