EM algorithm - E-step notation The 2019 Stack Overflow Developer Survey Results Are Inhow does expectation maximization work?Combining multiple posterior distributionsExpectation maximization modelingWhy does $p(X;|;Y) = displaystylefracp(Z,X;P(Z;$?Questions about Bayesian inferenceEstimating errors from optimization? (Genetic algorithm or otherwise)How to optimize the log likelihood to obtain parameters for the maximum likelihood estimate?Expectation Maximization Algorithm with latent variableVector-update form of Hill function for on-line fitting of modelIs it possible to express the posterior of the function of a parameter in terms of the posterior of the parameter?
Are USB sockets on wall outlets live all the time, even when the switch is off?
How to make payment on the internet without leaving a money trail?
Confusion about non-derivable continuous functions
If a poisoned arrow's piercing damage is reduced to 0, do you still get poisoned?
JSON.serialize: is it possible to suppress null values of a map?
Why can Shazam do this?
Deadlock Graph and Interpretation, solution to avoid
Why don't Unix/Linux systems traverse through directories until they find the required version of a linked library?
Is domain driven design an anti-SQL pattern?
What is this 4-propeller plane?
Manuscript was "unsubmitted" because the manuscript was deposited in Arxiv Preprints
Are there any other methods to apply to solving simultaneous equations?
Should I use my personal or workplace e-mail when registering to external websites for work purpose?
How come people say “Would of”?
Could JWST stay at L2 "forever"?
Why is Grand Jury testimony secret?
Is three citations per paragraph excessive for undergraduate research paper?
What is the motivation for a law requiring 2 parties to consent for recording a conversation
Inline version of a function returns different value then non-inline version
Idiomatic way to prevent slicing?
How are circuits which use complex ICs normally simulated?
Does a dangling wire really electrocute me if I'm standing in water?
Is "plugging out" electronic devices an American expression?
What is the best strategy for white in this position?
EM algorithm - E-step notation
The 2019 Stack Overflow Developer Survey Results Are Inhow does expectation maximization work?Combining multiple posterior distributionsExpectation maximization modelingWhy does $p(X;|;Y) = displaystylefracp(Z,X;P(Z;$?Questions about Bayesian inferenceEstimating errors from optimization? (Genetic algorithm or otherwise)How to optimize the log likelihood to obtain parameters for the maximum likelihood estimate?Expectation Maximization Algorithm with latent variableVector-update form of Hill function for on-line fitting of modelIs it possible to express the posterior of the function of a parameter in terms of the posterior of the parameter?
$begingroup$
I think I understand the gits of Expectation-Maximization algorithm and its altering nature, but I am puzzled by the notation. Lets see the following examples:
- in Stanford notes , the E-step is simply stated as posterior probability of latent variable $z$:
$$Q_i(z^(i)) := p(z^(i)|z^(i);theta)$$
where $z^(i)$ is latent variable sample, $x^(i)$ is observed data, $theta$ are the parameters maximized in M-step.
- in Original paper from 1977 the E-step looks as follows:
$$t^(p) = Ebig[ t(x)|y,Theta^(p) big]$$
where I believe the $y$ is observe variable, $x$ is latent variable, $Theta^(p)$ are model parameters used in M-step. To me, this looks like:
$$E_xbig[ p(x|y,Theta)big]$$
where the $x,y,Theta$ is the same as in point 2.
I appologize for introducing 2 notations, one in point 1. another in point 2. but I am trying to keep it consistent with the linked papers.
Question
The point of E-step is to obtain such values of latent variables, that they maximize the observation of complete data, given the current model parameters $theta$ or $Theta^(p)$. Then my question is, how do I formally get these values from the presented E-steps ?
I mean, what/where do I calculate in $Q_i(z^(i)) := p(z^(i)|z^(i);theta)$ ? Because it is just a definition of posterior distribution, there is no maximization, no operation to be done.
The second one $E_xbig[ p(x|y,Theta)big]$ is a bit more intuitive, because I am calculating an expectation of distributions (I think $t(x)$ is distribution of latent variable $x$). That means, I am looking for such values of $x$, that are expected -> gives maximum probability of realizing/happening.
Can someone formally show (and explain in layman's terms), how to obtain the values of the latent variables from the equations of E-step ?
statistics optimization machine-learning expected-value
$endgroup$
add a comment |
$begingroup$
I think I understand the gits of Expectation-Maximization algorithm and its altering nature, but I am puzzled by the notation. Lets see the following examples:
- in Stanford notes , the E-step is simply stated as posterior probability of latent variable $z$:
$$Q_i(z^(i)) := p(z^(i)|z^(i);theta)$$
where $z^(i)$ is latent variable sample, $x^(i)$ is observed data, $theta$ are the parameters maximized in M-step.
- in Original paper from 1977 the E-step looks as follows:
$$t^(p) = Ebig[ t(x)|y,Theta^(p) big]$$
where I believe the $y$ is observe variable, $x$ is latent variable, $Theta^(p)$ are model parameters used in M-step. To me, this looks like:
$$E_xbig[ p(x|y,Theta)big]$$
where the $x,y,Theta$ is the same as in point 2.
I appologize for introducing 2 notations, one in point 1. another in point 2. but I am trying to keep it consistent with the linked papers.
Question
The point of E-step is to obtain such values of latent variables, that they maximize the observation of complete data, given the current model parameters $theta$ or $Theta^(p)$. Then my question is, how do I formally get these values from the presented E-steps ?
I mean, what/where do I calculate in $Q_i(z^(i)) := p(z^(i)|z^(i);theta)$ ? Because it is just a definition of posterior distribution, there is no maximization, no operation to be done.
The second one $E_xbig[ p(x|y,Theta)big]$ is a bit more intuitive, because I am calculating an expectation of distributions (I think $t(x)$ is distribution of latent variable $x$). That means, I am looking for such values of $x$, that are expected -> gives maximum probability of realizing/happening.
Can someone formally show (and explain in layman's terms), how to obtain the values of the latent variables from the equations of E-step ?
statistics optimization machine-learning expected-value
$endgroup$
add a comment |
$begingroup$
I think I understand the gits of Expectation-Maximization algorithm and its altering nature, but I am puzzled by the notation. Lets see the following examples:
- in Stanford notes , the E-step is simply stated as posterior probability of latent variable $z$:
$$Q_i(z^(i)) := p(z^(i)|z^(i);theta)$$
where $z^(i)$ is latent variable sample, $x^(i)$ is observed data, $theta$ are the parameters maximized in M-step.
- in Original paper from 1977 the E-step looks as follows:
$$t^(p) = Ebig[ t(x)|y,Theta^(p) big]$$
where I believe the $y$ is observe variable, $x$ is latent variable, $Theta^(p)$ are model parameters used in M-step. To me, this looks like:
$$E_xbig[ p(x|y,Theta)big]$$
where the $x,y,Theta$ is the same as in point 2.
I appologize for introducing 2 notations, one in point 1. another in point 2. but I am trying to keep it consistent with the linked papers.
Question
The point of E-step is to obtain such values of latent variables, that they maximize the observation of complete data, given the current model parameters $theta$ or $Theta^(p)$. Then my question is, how do I formally get these values from the presented E-steps ?
I mean, what/where do I calculate in $Q_i(z^(i)) := p(z^(i)|z^(i);theta)$ ? Because it is just a definition of posterior distribution, there is no maximization, no operation to be done.
The second one $E_xbig[ p(x|y,Theta)big]$ is a bit more intuitive, because I am calculating an expectation of distributions (I think $t(x)$ is distribution of latent variable $x$). That means, I am looking for such values of $x$, that are expected -> gives maximum probability of realizing/happening.
Can someone formally show (and explain in layman's terms), how to obtain the values of the latent variables from the equations of E-step ?
statistics optimization machine-learning expected-value
$endgroup$
I think I understand the gits of Expectation-Maximization algorithm and its altering nature, but I am puzzled by the notation. Lets see the following examples:
- in Stanford notes , the E-step is simply stated as posterior probability of latent variable $z$:
$$Q_i(z^(i)) := p(z^(i)|z^(i);theta)$$
where $z^(i)$ is latent variable sample, $x^(i)$ is observed data, $theta$ are the parameters maximized in M-step.
- in Original paper from 1977 the E-step looks as follows:
$$t^(p) = Ebig[ t(x)|y,Theta^(p) big]$$
where I believe the $y$ is observe variable, $x$ is latent variable, $Theta^(p)$ are model parameters used in M-step. To me, this looks like:
$$E_xbig[ p(x|y,Theta)big]$$
where the $x,y,Theta$ is the same as in point 2.
I appologize for introducing 2 notations, one in point 1. another in point 2. but I am trying to keep it consistent with the linked papers.
Question
The point of E-step is to obtain such values of latent variables, that they maximize the observation of complete data, given the current model parameters $theta$ or $Theta^(p)$. Then my question is, how do I formally get these values from the presented E-steps ?
I mean, what/where do I calculate in $Q_i(z^(i)) := p(z^(i)|z^(i);theta)$ ? Because it is just a definition of posterior distribution, there is no maximization, no operation to be done.
The second one $E_xbig[ p(x|y,Theta)big]$ is a bit more intuitive, because I am calculating an expectation of distributions (I think $t(x)$ is distribution of latent variable $x$). That means, I am looking for such values of $x$, that are expected -> gives maximum probability of realizing/happening.
Can someone formally show (and explain in layman's terms), how to obtain the values of the latent variables from the equations of E-step ?
statistics optimization machine-learning expected-value
statistics optimization machine-learning expected-value
asked Mar 30 at 11:13
Martin GMartin G
618
618
add a comment |
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function ()
return StackExchange.using("mathjaxEditing", function ()
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
);
);
, "mathjax-editing");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "69"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3168183%2fem-algorithm-e-step-notation%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3168183%2fem-algorithm-e-step-notation%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown