Derivative of a function of matrix Announcing the arrival of Valued Associate #679: Cesar Manara Planned maintenance scheduled April 23, 2019 at 23:30 UTC (7:30pm US/Eastern)Matrix-by-matrix derivative formulaMatrix Calculus Partial DerivativePartial derivative of a function of a matrixDerivative of trace of inverse matrix?How to get the derivative of a matrix function?Derivative where the variable is a matrixDerivative of a vector with respect to a vectorFinding the matrix derivative of $X^-1$ with respect to $X$Matrix multiplication and differentiationDerivative of a products with matrix exponentialSecond Partial Derivative Test for a Matrix Valued Function
JImage - Set generated image quality
What should one know about term logic before studying propositional and predicate logic?
draw a pulley system
Found this skink in my tomato plant bucket. Is he trapped? Or could he leave if he wanted?
How does Billy Russo acquire his 'Jigsaw' mask?
How to get a flat-head nail out of a piece of wood?
"Destructive power" carried by a B-52?
Combining list in a Cartesian product format with addition operation?
New Order #6: Easter Egg
Who got some sun on the voyage?
Magento 2 - Add additional attributes in register
Magento 2 Editing phtml files in Production Mode
Was the pager message from Nick Fury to Captain Marvel unnecessary?
How to show a density matrix is in a pure/mixed state?
Can gravitational waves pass through a black hole?
How to ask rejected full-time candidates to apply to teach individual courses?
How do I find my Spellcasting Ability for my D&D character?
Is a copyright notice with a non-existent name be invalid?
Table formatting with tabularx?
Reflections in a Square
IC on Digikey is 5x more expensive than board containing same IC on Alibaba: How?
How did 'ликвиди́ровать' semantically shift to mean 'abolish' and 'destroy, kill'?
Who's the Giant Batman in the back of this dark knights metal Batman picture?
Proving that any solution to the differential equation of an oscillator can be written as a sum of sinusoids.
Derivative of a function of matrix
Announcing the arrival of Valued Associate #679: Cesar Manara
Planned maintenance scheduled April 23, 2019 at 23:30 UTC (7:30pm US/Eastern)Matrix-by-matrix derivative formulaMatrix Calculus Partial DerivativePartial derivative of a function of a matrixDerivative of trace of inverse matrix?How to get the derivative of a matrix function?Derivative where the variable is a matrixDerivative of a vector with respect to a vectorFinding the matrix derivative of $X^-1$ with respect to $X$Matrix multiplication and differentiationDerivative of a products with matrix exponentialSecond Partial Derivative Test for a Matrix Valued Function
$begingroup$
I am trying to derive the gradient of the function $f(X) = AXZ + XZX^TXZ$ where $A,X,Z in R^n times n$ with respect to $X$ matrix. I read a post Matrix-by-matrix derivative formula about matrix derivate, but I am not able to follow it. In my case $fracpartial f(X)partial X)$ would be a tensor, but If I try to use the formula given in the post I would get a matrix. How should I process to get the partial derivate?
matrices partial-derivative matrix-calculus
asked Apr 2 at 17:49
Dushyant SahooDushyant Sahoo
818
$endgroup$
add a comment |
$begingroup$
I am trying to derive the gradient of the function $f(X) = AXZ + XZX^TXZ$ where $A,X,Z in R^n times n$ with respect to $X$ matrix. I read a post Matrix-by-matrix derivative formula about matrix derivate, but I am not able to follow it. In my case $fracpartial f(X)partial X)$ would be a tensor, but If I try to use the formula given in the post I would get a matrix. How should I process to get the partial derivate?
matrices partial-derivative matrix-calculus
asked Apr 2 at 17:49
Dushyant SahooDushyant Sahoo
818
$endgroup$
$begingroup$
Yes, that's correct that you would end up with 4th order Tensor. So, you can employ Kronecker product, and just vectorize accordingly.
$endgroup$
– user550103
Apr 2 at 18:29
$begingroup$
Just out of curiosity, where does this second term comes from ? And in what field does it makes sense ?
$endgroup$
– P. Quinton
Apr 2 at 18:30
$begingroup$
@P.Quinton Actually the above function is the gradient of some other complex function
$endgroup$
– Dushyant Sahoo
Apr 2 at 20:20
add a comment |
$begingroup$
I am trying to derive the gradient of the function $f(X) = AXZ + XZX^TXZ$ where $A,X,Z in R^n times n$ with respect to $X$ matrix. I read a post Matrix-by-matrix derivative formula about matrix derivate, but I am not able to follow it. In my case $fracpartial f(X)partial X)$ would be a tensor, but If I try to use the formula given in the post I would get a matrix. How should I process to get the partial derivate?
matrices partial-derivative matrix-calculus
asked Apr 2 at 17:49
Dushyant SahooDushyant Sahoo
818
$endgroup$
I am trying to derive the gradient of the function $f(X) = AXZ + XZX^TXZ$ where $A,X,Z in R^n times n$ with respect to $X$ matrix. I read a post Matrix-by-matrix derivative formula about matrix derivate, but I am not able to follow it. In my case $fracpartial f(X)partial X)$ would be a tensor, but If I try to use the formula given in the post I would get a matrix. How should I process to get the partial derivate?
matrices partial-derivative matrix-calculus
matrices partial-derivative matrix-calculus
asked Apr 2 at 17:49
Dushyant SahooDushyant Sahoo
818
asked Apr 2 at 17:49
Dushyant SahooDushyant Sahoo
818
asked Apr 2 at 17:49
Dushyant SahooDushyant Sahoo
818
asked Apr 2 at 17:49
Dushyant SahooDushyant Sahoo
818
asked Apr 2 at 17:49
Dushyant SahooDushyant Sahoo
818
818
$begingroup$
Yes, that's correct that you would end up with 4th order Tensor. So, you can employ Kronecker product, and just vectorize accordingly.
$endgroup$
– user550103
Apr 2 at 18:29
$begingroup$
Just out of curiosity, where does this second term comes from ? And in what field does it makes sense ?
$endgroup$
– P. Quinton
Apr 2 at 18:30
$begingroup$
@P.Quinton Actually the above function is the gradient of some other complex function
$endgroup$
– Dushyant Sahoo
Apr 2 at 20:20
add a comment |
$begingroup$
Yes, that's correct that you would end up with 4th order Tensor. So, you can employ Kronecker product, and just vectorize accordingly.
$endgroup$
– user550103
Apr 2 at 18:29
$begingroup$
Just out of curiosity, where does this second term comes from ? And in what field does it makes sense ?
$endgroup$
– P. Quinton
Apr 2 at 18:30
$begingroup$
@P.Quinton Actually the above function is the gradient of some other complex function
$endgroup$
– Dushyant Sahoo
Apr 2 at 20:20
$begingroup$
Yes, that's correct that you would end up with 4th order Tensor. So, you can employ Kronecker product, and just vectorize accordingly.
$endgroup$
– user550103
Apr 2 at 18:29
$begingroup$
Yes, that's correct that you would end up with 4th order Tensor. So, you can employ Kronecker product, and just vectorize accordingly.
$endgroup$
– user550103
Apr 2 at 18:29
$begingroup$
Just out of curiosity, where does this second term comes from ? And in what field does it makes sense ?
$endgroup$
– P. Quinton
Apr 2 at 18:30
$begingroup$
Just out of curiosity, where does this second term comes from ? And in what field does it makes sense ?
$endgroup$
– P. Quinton
Apr 2 at 18:30
$begingroup$
@P.Quinton Actually the above function is the gradient of some other complex function
$endgroup$
– Dushyant Sahoo
Apr 2 at 20:20
$begingroup$
@P.Quinton Actually the above function is the gradient of some other complex function
$endgroup$
– Dushyant Sahoo
Apr 2 at 20:20
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
Let
beginalign
F := f(X) = AXZ + XZX^TXZ .
endalign
Now take the differential, then vectorize, and thereafter obtain the gradient.
beginalign
dF &= A dX Z + dX Z X^T X Z + X Z dX^T X Z + X Z X^T dX Z \ \
Longleftrightarrow rm vecleft( dF right) &= rm vecleft(A dX Zright) + rm vecleft(dX Z underbraceX^T X Z_ right) \ &+ rm vecleft(X Z dX^T X Z right) + rm vecleft(X Z X^T dX Z right) \ \
Longleftrightarrow rm vecleft( dF right) &= left(Z^T otimes A right) rm vecleft( dX right) + left(left(X^T X Zright)^T Z^T otimes Iright) rm vecleft( dXright) \
&+ left(left(X Zright)^T otimes left( XZright)right) underbracerm vec left( dX^Tright)_= K rm vecleft( dXright) + left(Z^T otimes left( XZX^Tright)right) rm vecleft( dXright) \ \
Rightarrow fracpartial f(X)dX = fracpartial rm vecleft( Fright)rm vecleft( dXright) &= left(Z^T otimes A right) + left(left(X^T X Zright)^T Z^T otimes Iright) \ &+ left(left(X Zright)^T otimes left( XZright)right) K + left(Z^T otimes left( XZX^Tright)right) ,
endalign
where $K$ is the commutation matrix for the Kronecker products.
answered Apr 2 at 19:04
user550103user550103
8331415
$endgroup$
1
$begingroup$
The $K$ matrix is missing in the final result.
$endgroup$
– greg
Apr 2 at 21:32
$begingroup$
thank you, greg! I have fixed it now.
$endgroup$
– user550103
Apr 3 at 4:05
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "69"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3172171%2fderivative-of-a-function-of-matrix%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
Let
beginalign
F := f(X) = AXZ + XZX^TXZ .
endalign
Now take the differential, then vectorize, and thereafter obtain the gradient.
beginalign
dF &= A dX Z + dX Z X^T X Z + X Z dX^T X Z + X Z X^T dX Z \ \
Longleftrightarrow rm vecleft( dF right) &= rm vecleft(A dX Zright) + rm vecleft(dX Z underbraceX^T X Z_ right) \ &+ rm vecleft(X Z dX^T X Z right) + rm vecleft(X Z X^T dX Z right) \ \
Longleftrightarrow rm vecleft( dF right) &= left(Z^T otimes A right) rm vecleft( dX right) + left(left(X^T X Zright)^T Z^T otimes Iright) rm vecleft( dXright) \
&+ left(left(X Zright)^T otimes left( XZright)right) underbracerm vec left( dX^Tright)_= K rm vecleft( dXright) + left(Z^T otimes left( XZX^Tright)right) rm vecleft( dXright) \ \
Rightarrow fracpartial f(X)dX = fracpartial rm vecleft( Fright)rm vecleft( dXright) &= left(Z^T otimes A right) + left(left(X^T X Zright)^T Z^T otimes Iright) \ &+ left(left(X Zright)^T otimes left( XZright)right) K + left(Z^T otimes left( XZX^Tright)right) ,
endalign
where $K$ is the commutation matrix for the Kronecker products.
answered Apr 2 at 19:04
user550103user550103
8331415
$endgroup$
1
$begingroup$
The $K$ matrix is missing in the final result.
$endgroup$
– greg
Apr 2 at 21:32
$begingroup$
thank you, greg! I have fixed it now.
$endgroup$
– user550103
Apr 3 at 4:05
add a comment |
$begingroup$
Let
beginalign
F := f(X) = AXZ + XZX^TXZ .
endalign
Now take the differential, then vectorize, and thereafter obtain the gradient.
beginalign
dF &= A dX Z + dX Z X^T X Z + X Z dX^T X Z + X Z X^T dX Z \ \
Longleftrightarrow rm vecleft( dF right) &= rm vecleft(A dX Zright) + rm vecleft(dX Z underbraceX^T X Z_ right) \ &+ rm vecleft(X Z dX^T X Z right) + rm vecleft(X Z X^T dX Z right) \ \
Longleftrightarrow rm vecleft( dF right) &= left(Z^T otimes A right) rm vecleft( dX right) + left(left(X^T X Zright)^T Z^T otimes Iright) rm vecleft( dXright) \
&+ left(left(X Zright)^T otimes left( XZright)right) underbracerm vec left( dX^Tright)_= K rm vecleft( dXright) + left(Z^T otimes left( XZX^Tright)right) rm vecleft( dXright) \ \
Rightarrow fracpartial f(X)dX = fracpartial rm vecleft( Fright)rm vecleft( dXright) &= left(Z^T otimes A right) + left(left(X^T X Zright)^T Z^T otimes Iright) \ &+ left(left(X Zright)^T otimes left( XZright)right) K + left(Z^T otimes left( XZX^Tright)right) ,
endalign
where $K$ is the commutation matrix for the Kronecker products.
answered Apr 2 at 19:04
user550103user550103
8331415
$endgroup$
1
$begingroup$
The $K$ matrix is missing in the final result.
$endgroup$
– greg
Apr 2 at 21:32
$begingroup$
thank you, greg! I have fixed it now.
$endgroup$
– user550103
Apr 3 at 4:05
add a comment |
$begingroup$
Let
beginalign
F := f(X) = AXZ + XZX^TXZ .
endalign
Now take the differential, then vectorize, and thereafter obtain the gradient.
beginalign
dF &= A dX Z + dX Z X^T X Z + X Z dX^T X Z + X Z X^T dX Z \ \
Longleftrightarrow rm vecleft( dF right) &= rm vecleft(A dX Zright) + rm vecleft(dX Z underbraceX^T X Z_ right) \ &+ rm vecleft(X Z dX^T X Z right) + rm vecleft(X Z X^T dX Z right) \ \
Longleftrightarrow rm vecleft( dF right) &= left(Z^T otimes A right) rm vecleft( dX right) + left(left(X^T X Zright)^T Z^T otimes Iright) rm vecleft( dXright) \
&+ left(left(X Zright)^T otimes left( XZright)right) underbracerm vec left( dX^Tright)_= K rm vecleft( dXright) + left(Z^T otimes left( XZX^Tright)right) rm vecleft( dXright) \ \
Rightarrow fracpartial f(X)dX = fracpartial rm vecleft( Fright)rm vecleft( dXright) &= left(Z^T otimes A right) + left(left(X^T X Zright)^T Z^T otimes Iright) \ &+ left(left(X Zright)^T otimes left( XZright)right) K + left(Z^T otimes left( XZX^Tright)right) ,
endalign
where $K$ is the commutation matrix for the Kronecker products.
answered Apr 2 at 19:04
user550103user550103
8331415
$endgroup$
Let
beginalign
F := f(X) = AXZ + XZX^TXZ .
endalign
Now take the differential, then vectorize, and thereafter obtain the gradient.
beginalign
dF &= A dX Z + dX Z X^T X Z + X Z dX^T X Z + X Z X^T dX Z \ \
Longleftrightarrow rm vecleft( dF right) &= rm vecleft(A dX Zright) + rm vecleft(dX Z underbraceX^T X Z_ right) \ &+ rm vecleft(X Z dX^T X Z right) + rm vecleft(X Z X^T dX Z right) \ \
Longleftrightarrow rm vecleft( dF right) &= left(Z^T otimes A right) rm vecleft( dX right) + left(left(X^T X Zright)^T Z^T otimes Iright) rm vecleft( dXright) \
&+ left(left(X Zright)^T otimes left( XZright)right) underbracerm vec left( dX^Tright)_= K rm vecleft( dXright) + left(Z^T otimes left( XZX^Tright)right) rm vecleft( dXright) \ \
Rightarrow fracpartial f(X)dX = fracpartial rm vecleft( Fright)rm vecleft( dXright) &= left(Z^T otimes A right) + left(left(X^T X Zright)^T Z^T otimes Iright) \ &+ left(left(X Zright)^T otimes left( XZright)right) K + left(Z^T otimes left( XZX^Tright)right) ,
endalign
where $K$ is the commutation matrix for the Kronecker products.
answered Apr 2 at 19:04
user550103user550103
8331415
edited Apr 3 at 4:22
answered Apr 2 at 19:04
user550103user550103
8331415
answered Apr 2 at 19:04
user550103user550103
8331415
answered Apr 2 at 19:04
user550103user550103
8331415
8331415
1
$begingroup$
The $K$ matrix is missing in the final result.
$endgroup$
– greg
Apr 2 at 21:32
$begingroup$
thank you, greg! I have fixed it now.
$endgroup$
– user550103
Apr 3 at 4:05
add a comment |
1
$begingroup$
The $K$ matrix is missing in the final result.
$endgroup$
– greg
Apr 2 at 21:32
$begingroup$
thank you, greg! I have fixed it now.
$endgroup$
– user550103
Apr 3 at 4:05
1
1
$begingroup$
The $K$ matrix is missing in the final result.
$endgroup$
– greg
Apr 2 at 21:32
$begingroup$
The $K$ matrix is missing in the final result.
$endgroup$
– greg
Apr 2 at 21:32
$begingroup$
thank you, greg! I have fixed it now.
$endgroup$
– user550103
Apr 3 at 4:05
$begingroup$
thank you, greg! I have fixed it now.
$endgroup$
– user550103
Apr 3 at 4:05
add a comment |
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3172171%2fderivative-of-a-function-of-matrix%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
$begingroup$
Yes, that's correct that you would end up with 4th order Tensor. So, you can employ Kronecker product, and just vectorize accordingly.
$endgroup$
– user550103
Apr 2 at 18:29
$begingroup$
Just out of curiosity, where does this second term comes from ? And in what field does it makes sense ?
$endgroup$
– P. Quinton
Apr 2 at 18:30
$begingroup$
@P.Quinton Actually the above function is the gradient of some other complex function
$endgroup$
– Dushyant Sahoo
Apr 2 at 20:20