Table Of Contents

• Solutions By Variation - Solving Non-Linear Equations
• Variance/Covariance Matrices
• Propagation Of Variance
  

• A lead into non-linear least squares (network adjustment)
• Linearisation
• By calculus
• By numerical methods
• Some simple examples
  

back to top

Non-Linear Equations:

• Example: f(x) = (x - 3)³ + e^x is non-linear in x
• Observation equations for survey network adjustment are generally non-linear in terms of the coordinates.

back to top

Linearising Non-Linear Equations

• The Taylor expansion:

  for f(x) = l

  l = f(x0) + (	df	)x0 Dx + (	d2f	)x0 Dx2 + higher order terms
  	dx		dx2

• Ignoring second and higher order terms gives equation that is linear inDx:

  Dx =	l - f(x0)
  	(	df	) x0
  		dx

• To solve this equation several things are required:

  • f(x) must be differentiated
  • the value of x that satisfies l (x₀) must be reasonably estimated
  • the solution updates x: x₀ = x₀ + Dx and must be iterated to account for the neglect of higher order terms from the Taylor series.
  • Note the similarities with TDVC - initial coordinate estimates & iteration

back to top

Example

f(x) = (x - 3)³ + e^x = 15

Estimate: x₀ = 2, f(x₀) = 0.6389

Differentiate: f'(x) = 3(x - 3)² + e^x

Solve:

Dx =	15 - f(x0)	=	15 - (x0 - 3)3 + ex0	= 0.829
	f'(x0)		3(x0 - 3)2 + ex0

Next estimate: x₀ = x₀ + Dx = 2.829
Now, f(x₀) = 16.923, continue procedure until f(x₀) is sufficiently close to 15

• Importance of initial estimate.

Note

Because the solution must be iterated due to ignorance of higher order terms from the Taylor expansion, the computation of the derivatives does not need to be all that accurate - this allows some cheating to take place. Note that the reduced observation l - f(x) must be accurately computed. This applies to Least Squares adjustment as well.

back to top

• Fundamentals
• Expected value
• Mean
• Variance
• Covariance
• Standard error / standard deviation
• Rules for expected values
• Definition of a covariance matrix
• Correlation coefficient
• Weight matrices
• With uncorrelated observations

back to top

Fundamentals

• The notation E(x) means the expected value of x
• The mean of a variate is its expected value: m_x = E(x)
• Some handy rules for expected values:
• E(kx) = kE(x) where k is non-stochastic (does not depend on chance)
• E(x) + E(y) + E(z) + … = E(x + y + z + …)

back to top

Definition Of A Covariance Matrix

• If x is a vector: x = [ x₁ x₂ . . . x_n ] its covariance matrix C_x is defined as:

  C_x = E [(x - m)(x - m)^t]

   

	x1 - m1
= E (	x2 - m2	[x1 - m1, x2 - m2 xn - mn] )
	xn - mn

E{(x1 - m1)2}	E{(x1 - m1)2(x2 - m2)2}		E{(x1 - m1)2(xn - mn)2}
= E{(x2 - m2)2(x1 - m1)2}	E{(x2 - m2)2}		E{(x2 - m2)2(x2 - m2)2}
E{(xn - mn)2(x1 - m1)2}	E{(xn - mn)2(x2 - m2)2}		E{(xn - mn)2}

• The definition of variance of : x_i : s²_xi = E [( x_i - m_i )²]

• The square root of the variance of x (= s_xi) is the standard error or standard deviation of x

• The definition of covariance of x_i and x_j : s_xixj = E [( x_i - m_i )( x_i - m_j ) ]

• Using these definitions:

  Cx =	s2x1	sx1x2		sx1xn
  	sx1x2	s2x2		sx2xn
  	sxnx1	sxnx2		s2xn

back to top

Correlation Coefficient

• The coefficient of correlation is used to express the strength of dependence of one variable on another:

  rij =	sxiyj
  	sxi sxj

  r has a range [-1, 1], where +1 indicates total correlation, and 0 indicates nor correlation at all.

back to top

Weight Matrices

• As part of a least squares solution a weight matrix of observations can be used to indicate some observations are more or less precise than others, and some observations are correlated with others. A weight matrix of observations (P₁) is the inverse of the covariance matrix of the observations:

  P₁ = C^-1₁

• In some circumstances C₁ is diagonal (or assumed to be diagonal) since this makes the formation of the least squares solution simpler - e.g. forming P₁ is simple.

back to top

• Derivation of equation for propagating variance
• Applications to simple problems - Examples:
• Correlation between directions and angles (other implications)
• Distance intersections
• Propagation of variance applied to least squares
• Example with intersection geometry

back to top

The Law Of Propagation Of Variance

• Let y = Ax
• y, x are vectors, A a non-stochastic matrix (we know or measure x, and know the relationship between x and y)
• C_x is the covariance matrix of x (known from the measurement process to stick to the surveying emphasis).

• Using the relationship between mean and expected value (shown previously):

  m_y = E(y) = E(Ax) = AE(x) = Am_x

• Using the definition of the covariance matrix (also shown previously)

  C_y = E{(y - m_y)(y - m_y)^t}

       = E{(Ax - Am_x)(Ax - Am_x)^t}

       = AE{(x - m_x)(x - m_x)^t}A^t

       = AC_xA^t

• Thus if A, x and C_x are known, y and C_y can be computed:

  y = Ax

  C_y = AC_xA^t (propagation of variance)

back to top

Example 1 - Angles From Measured Directions

Measured: 3 directions measured with equal precision s_d (not correlated) to give 2 angles.

Equation:	a1 a1	=	-1   1   0 0   -1   1	d1	say y = Ax
				d2
				d3

Precisions:	Cx = s2d I
	Cy =	-1   1   0 0   -1   1	s2d   0   0   -1   0 0   s2d   0   1   -1 0   0   s2d   0   1	=	2s2d  -s2d -s2d  2s2d

Notes:

• The standard deviation of the angles is 2s_d (root 2 worse than the directions)

• The correlation coefficient (see previous) is	- s2d	=	- 1	, quite large.
  	2s2d		2

• Some survey network adjustment programs use angles and ignore the correlation, the general claim being that in a network with strong geometry the ignorance of the correlation has negligible effect on the end result. Other programs use either angles with correlations, or directions (no reduction to angles at all) their claim being the correlation between angles is significant and should not be ignored.

back to top

Example 2 - Distance Intersections

back to top

Propagation Of Variance And Least Squares

• The equation for observations:

  Bx = l + v

  B - design matrix

  x - unknown parameters (increments to the assumed coordinates)

  l - observations

  v - residuals

  C_l = P^-1_l - Covariance matrix of the observations

  C_x - Covariance matrix of the parameters

  ^ - least squares estimate of the parameter beneath

• Without to much regard for detail at this stage, the solution via least squares is:

  x = (B^tP_lB)^-1B^tP_ll

• C by propagation of variance is:

  C = [(B^tP_lB)^-1B^tP_l] C_l [(B^tP_lB)^-1B^tP_l]

  C = [(B^tP_lB)^-1B^t] [P_lB (B^tP_lB)^-1]

  C = (B^tP_lB)^-1 [B^tP_lB ] [B^tP_lB]^-1

  C = (B^tP_lB)^-1

• The precision of the parameter estimate is the inverse of the normal matrix. The formation and inversion of the normal matrix (B^tP_lB) is all that is required to perform network simulations (as in TDVC), note that the actual observations are not required, only the approximate coordinates of the network stations.

back to top

Example - Intersection Geometry

back to top