Tensor product
From Academic Kids

In mathematics, the tensor product, denoted by <math>\otimes<math>, may be applied in different contexts to vectors, matrices, tensors, vector spaces, algebras and modules. In each case the significance of the symbol is the same: the most general bilinear operation.
A representative case is the Kronecker multiplication of any two rectangular arrays, considered as matrices.
Example:
<math>\begin{bmatrix}a_1 & a_2 & a_3\end{bmatrix} \otimes \begin{bmatrix}b_1 \\ b_2 \\ b_3 \\ b_4\end{bmatrix} = \begin{bmatrix}a_1b_1 & a_2b_1 & a_3b_1 \\ a_1b_2 & a_2b_2 & a_3b_2 \\ a_1b_3 & a_2b_3 & a_3b_3 \\ a_1b_4 & a_2b_4 & a_3b_4\end{bmatrix}<math>
Resultant rank = 2, resultant dimension = 12.
Here rank denotes the number of requisite indices, while dimension counts the number of degrees of freedom in the resulting array.
Contents 
Tensor product of two tensors
There is a general formula for the product of two (or more) tensors
 <math>V \otimes U = V_{\left[ i_1,i_2,i_3,...i_n\right] }U_{\left[ j_1,j_2,j_3,...j_m\right] }<math>.
We are assuming here orthogonal tensors, with no distinction of covariant and contravariant indices, for simplicity.
The parameters introduced above work out like this:
 <math>rank( U \otimes V )=rank(U)+rank(V)<math>
 <math>dim( U \otimes V )=dim(U) \cdot dim(V)<math>
See also: Tensorclassical
Kronecker product of two matrices
Main article: Kronecker product.
With matrices this is usually called the Kronecker product, a term used to make clear that the result has a particular block structure imposed upon it, in which each element of the first matrix is replaced by the second matrix, scaled by that element. For matrices <math>U<math> and <math>V<math> this is:
 <math>U \otimes V
= \begin{bmatrix} u_{11}V & u_{12}V & \cdots \\ u_{21}V & u_{22}V \\ \vdots & & \ddots \end{bmatrix} = \begin{bmatrix} u_{11}v_{11} & u_{11}v_{12} & \cdots & u_{12}v_{11} & u_{12}v_{12} & \cdots \\ u_{11}v_{21} & u_{11}v_{22} & & u_{12}v_{21} & u_{12}v_{22} \\ \vdots & & \ddots \\ u_{21}v_{11} & u_{21}v_{12} \\ u_{21}v_{21} & u_{21}v_{22} \\ \vdots \end{bmatrix}<math>.
Tensor product of multilinear maps
Given multilinear maps <math>f(x_1,...x_k)<math> and <math>g(x_1,... x_m)<math> their tensor product is the multilinear function
 <math> (f \otimes g) (x_1,...x_{k+m})=f(x_1,...x_k)g(x_{k+1},... x_{k+m})<math>
Tensor product of vector spaces
The tensor product <math>V \otimes W<math> of two vector spaces V and W has a formal definition by the method of generators and relations. The equivalence class under these relations (given below) of <math>(v,w)<math> is called a tensor and is denoted by <math>v \otimes w<math>. By construction, one can prove several identities between tensors and form an algebra of tensors.
To construct <math>V \otimes W<math>, take the vector space generated by <math>V \times W<math> and apply (factor out the subspace generated by) the following multilinear relations:
 <math>(v_1+v_2)\otimes w=v_1\otimes w<math><math>+v_2\otimes w<math>
 <math>v\otimes (w_1+w_2)=v\otimes w_1+v\otimes w_2<math>
 <math>cv\otimes w=v\otimes cw=c(v\otimes w)<math>
where <math>v,v_i,w,w_i<math> are vectors from the appropriate spaces, and <math>c<math> is from the underlying field.
We can then derive the identity <math>0v\otimes w=v\otimes 0w=0(v\otimes w)=0<math>, the zero in <math>V \otimes W<math>.
The resulting tensor product <math>V \otimes W<math> is itself a vector space, which can be verified by directly checking the vector space axioms. Given bases <math>\{v_i\}<math> and <math>\{w_i\}<math> for V and W respectively, the tensors of the form <math>v_i \otimes w_j<math> forms a basis for <math>V \otimes W<math>. The dimension of the tensor product therefore is the product of dimensions of the original spaces; for instance <math>\mathbb{R}^m \otimes \mathbb{R}^n<math> will have dimension <math>mn<math>.
Tensor product for computer programmers
If a, b, and c, are rankone tensors (i.e. onedimensional arrays), with indices i,j,k, respectively, then the tensor product of them is a rankthree tensor(i.e. threedimensional array):
for( int i = 0; i < i_dim; i++) for( int j = 0; j < j_dim; j++) for( int k = 0; k < k_dim; k++) result[i][j][k] = a[i]*b[j]*c[k];
If a is a ranktwo tensor and b is a rankone tensor, with indices i & j, and k, respectively, then the tensor product of them is a rankthree tensor:
for( int i = 0; i < i_dim; i++) for( int j = 0; j < j_dim; j++) for( int k = 0; k < k_dim; k++) result[i][j][k] = a[i][j]*b[k];
Universal property of tensor product
The space of all bilinear maps from <math>V\times W<math> to <math>\mathbb R<math> is naturally isomorphic to the space of all linear maps from <math>V \otimes W<math> to <math>\mathbb R<math>. This is built into the construction; <math>V\otimes W<math> has only the relations that are necessary to ensure that a homomorphism from <math>V\otimes W<math> to <math>\mathbb R<math> will be bilinear.
The tensor product in fact satisfies the universal property of being a fibered coproduct.
Tensor product of Hilbert spaces
The tensor product of two Hilbert spaces is another Hilbert space, which is defined as described below.
Definition
Let H_{1} and H_{2} be two Hilbert spaces with inner products <·,·>_{1} and <·,·>_{2}, respectively. Construct the tensor product of H_{1} and H_{2} as vector spaces as explained above. We can turn this vector space tensor product into an inner product space by defining
 <math> \langle\phi_1\otimes\phi_2,\psi_1\otimes\psi_2\rangle = \langle\phi_1,\psi_1\rangle_1 \, \langle\phi_2,\psi_2\rangle_2 \quad \mbox{for all } \phi_1,\psi_1 \in H_1 \mbox{ and } \phi_2,\psi_2 \in H_2 <math>
and extending by linearity. Finally, take the completion under this inner product. The result is the tensor product of H_{1} and H_{2} as Hilbert spaces.
Properties
If H_{1} and H_{2} have orthonormal bases {φ_{k}} and {ψ_{l}}, respectively, then {φ_{k} ⊗ ψ_{l}} is an orthonormal basis for H_{1} ⊗ H_{2}.
Examples and applications
The following examples show how tensor products arise naturally.
Given two measure spaces X and Y, with measures μ and ν respectively, one may look at L^{2}(X × Y), the space of functions on X × Y that are square integrable with respect to the product measure μ × ν. If f is a square integrable function on X, and g is a square integrable function on Y, then we can define a function h on X × Y by h(x,y) = f(x) g(y). The definition of the product measure ensures that all functions of this form are square integrable, so this defines a bilinear mapping L^{2}(X) × L^{2}(Y) → L^{2}(X × Y). Linear combinations of functions of the form f(x) g(y) are also in L^{2}(X × Y). It turns out that the set of linear combinations is in fact dense in L^{2}(X × Y), if L^{2}(X) and L^{2}(Y) are separable. This shows that L^{2}(X) ⊗ L^{2}(Y) is isomorphic to L^{2}(X × Y), and it also explains why we need to take the completion in the construction of the Hilbert space tensor product.
Similarly, we can show that L^{2}(X; H), denoting the space of square integrable functions X → H, is isomorphic to L^{2}(X) ⊗ H if this space is separable. The isomorphism maps f(x) ⊗ φ ∈ L^{2}(X) ⊗ H to f(x)φ ∈ L^{2}(X; H). We can combine this with the previous example and conclude that L^{2}(X) ⊗ L^{2}(Y) and L^{2}(X × Y) are both isomorphic to L^{2}(X; L^{2}(Y)).
Tensor products of Hilbert spaces arise often in quantum mechanics. If some particle is described by the Hilbert space H_{1}, and another particle is described by H_{2}, then the system consisting of both particles is described by the tensor product of H_{1} and H_{2}. For example, the state space of a quantum harmonic oscillator is L^{2}(R), so the state space of two oscillators is L^{2}(R) ⊗ L^{2}(R), which is isomorphic to L^{2}(R^{2}). Therefore, the twoparticle system is described by wave functions of the form φ(x_{1}, x_{2}). A more intricate example is provided by the Fock spaces, which describe a variable number of particles.
Abstract construction
The universal property of the tensor product mentioned above extends to more general situations in abstract algebra. It allows the study of bilinear or multilinear operations via linear operations.
In general, we start with a ring R, not necessarily commutative. Let M_{R} be a right Rmodule and _{R}N be a left Rmodule. We will denote the Cartesian product of M and N, that is, the product as sets not modules, by M×N. For any abelian group Z, we define:
 Bilin_{R}(M,N;Z) = the set of φ : M × N → Z such that
 for all m, m′ in M, for all n in N, φ(m + m′,n) = φ(m,n) + φ(m′,n),
 for all m in M, for all n, n′ in N, φ(m,n + n′) = φ(m,n) + φ(m,n′),
 for all m in M, for all n in N, for all r in R, φ(m r,n) = φ(m,r n)}
Observe that
 The set Bilin_{R}(M,N;Z) is an abelian group under addition; i.e., if φ, ψ are in Bilin_{R}(M,N;Z), then φ + ψ is in Bilin_{R}(M,N;Z).
 The map Z → Bilin_{R}(M,N;Z) is a functor from the category of abelian groups to the category of sets.
The tensor product M⊗_{R}N is defined to be a representing object for the functor Z → Bilin_{R}(M,N;Z). This is equivalent to the universal mapping property given above.
To construct the tensor product, we proceed in exactly the same way as for vector spaces; the same construction carries over without any changes.
This construction is unrelated to the direct product of M and N. The direct product need not even be defined when R is not commutative. Even if it is, the only map M×N→Z which is both linear and bilinear is the zero map!
It is possible to generalize the definition to a tensor product of any number of spaces. For example, the universal property of M_{1}⊗M_{2}⊗M_{3} is that every trilinear map on M_{1}×M_{2}×M_{3}→Z corresponds to a unique linear map M_{1}⊗M_{2}⊗M_{3}→Z. The binary tensor product is associative: (M_{1}⊗M_{2})⊗M_{3} is naturally isomorphic to M_{1}⊗(M_{2}⊗M_{3}). The tensor product of all three may therefore be identified with either of those.
The tensor product, as defined, is an abelian group, not an Rmodule. In general, it is impossible to put an Rmodule structure on the tensor product. However, if M is an (S,R)bimodule, then M⊗N is a left Smodule, and similarly, if N is an (R,T)bimodule, then M⊗N is a right Tmodule. If M and N each have bimodule structures as above, then M⊗N is an (S,T)bimodule. In particular, if R is a commutative ring, then M⊗N will always be an Rmodule.
If {m_{i}}_{i∈I} and {n_{j}}_{j∈J} are generating sets for M and N, respectively, then {m_{i}⊗n_{j}}_{i∈I,j∈J} will be a generating set for M⊗N. Because the tensor product is left exact, not exact, this may not be a minimal generating set, even if the original generating sets are minimal. However, if the tensor products are taken over a field, then we are in the case of vector spaces as above, and if the two given generating sets are bases, we will get a basis for M⊗N.
If S and T are commutative Ralgebras, then S⊗T will be a commutative Ralgebra as well, with the multiplication map defined by (m_{1}⊗m_{2})(n_{1}⊗n_{2})=(m_{1}n_{2}⊗m_{2}n_{2}) and extended by linearity. In this setting, the tensor product become a fibered coproduct in the category of Ralgebras. Note that any commutative ring is a Zalgebra, so we may always take M⊗_{Z}N.
Relation with the dual space
Note that the space <math> (V \otimes W)^\star<math> (the dual space of <math>V \otimes W<math> containing all linear functionals on that space) corresponds naturally to the space of all bilinear functionals on <math>V \times W<math>. In other words, every bilinear functional is a functional on the tensor product, and vice versa. There is a natural isomorphism between <math> V^\star \otimes W^\star <math> and <math>(V \otimes W)^\star<math>. So, the tensors of the linear functionals are bilinear functionals. This gives us a new way to look at the space of bilinear functionals, as a tensor product itself.
Types of tensors, e.g., alternating
Linear subspaces of the bilinear operators (or in general, multilinear operators) determine natural quotient spaces of the tensor space, which are frequently useful. See wedge product for the first major example. Another would be the treatment of algebraic forms as symmetric tensors.
Over more general rings
It is also possible to generalize the definition to tensor products of modules over the same ring. If the ring is noncommutative, we'll need to be careful about distinguishing right modules and left modules. We will write _{R}M for a left module, and M_{R} for a right module. If a module M has both a left module structure over a ring R and a right module structure over a ring S, and in addition for every m in M, r in R and s in S we have r(ms) = (rm)s, then we will say M is a bimodule, and will denote it by _{R}M_{S}. Note that every leftmodule is a bimodule with Z acting by mn = m + m + ... + m as the right ring, and vice versa.
When defining the tensor product, we need to be careful about the ring: most modules can be considered as modules over several different rings or over the same ring with a different actions of the ring on the module elements.
The most general form of the tensor product definition is as follows: let M_{R} and _{R}N be a right and a left module, respectively. Their tensor product over R is an abelian group P together with an Rbilinear operator T: M × N → P such that for every Rbilinear operator B: M × N → O there is a unique group homomorphism L: P → O such that L o T = B. P need not be a module over R. However, if _{S1}M_{R} is an S_{1}Rbimodule, then there is a unique left S_{1}module structure on P which is compatible with T. Similarly, if _{R}M_{S2} is an RS_{2}bimodule, then there is a unique right S_{2}module structure on P which is compatible with T. If M and N are both bimodules, then P is also a bimodule, again in a unique way. (P, T) are unique up to a unique isomorphism, and are called the "tensor product" of M and N.
If R is a ring, _{R}M is a left Rmodule, and the commutator rssr of any two elements r and s of R is in the annihilator of M, then we can make M into a right R module by setting mr = rm. Note that in this situation the action of R on M factors through an action of the commutative ring R/Z(R), R modulo its center. In this case the tensor product of M with itself over R is again an Rmodule. If M and N are both Rmodules satisfying this condition, then their tensor product is again an Rmodule. This is a very common technique in commutative algebra.
Example
Consider the rational numbers Q and the integers modulo n Z_{n}. Both can be considered as modules over the integers, Z. Let B: Q × Z_{n} → M be a Zbilinear operator. Then B(q, i) = B(q/n, ni) = B(q/n, 0) = 0, so every bilinear operator is identically zero. Therefore, if we define P to be the trivial module, and T to be the zero bilinear function, then we see that the properties for the tensor product are satisfied. Therefore, the tensor product of Q and Z_{n} is {0}.he:מכפלה טנזורית de:Tensorprodukt pl:Iloczyn tensorowy