Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Some thoughts about sigma and the normal distribution Sometimes you find figures like figure 1 below where sigma () is indicated as a straight line from the mean to a point on the curve (usually called the inflexion point. See also figure 2 and 3). Questions that arise are e.g. ’what does it mean’ or ’is it of any importance’ or ’do I need this information’ etc. Maybe it is an unimportant piece of information for the practical analysist and only a mathematical entity? If you ask the person that designed the figure you will get answers like the ’inflexion point is where the curve changes directions’ or ’where the second derivative changes signs’. Most likely will the practical analysist shake his head meaning that he has not become any wiser. If we start with the formula that is the mother of all 2 (variance) for all the different distributions that we can think of, we will see this: 2 E( X ) 2 This is read as the ’expected squared deviation from the mean’. If we use mathematical tools and develop this formula we will end with expressions for for all the distribution that we need when studying problems in our environment (see other Ing-Stat documents for details). The practical analysist is usually happy with these end results. A remark (I). It seems unnecessary to decorate the text with words or expressions that lack meaning for the practical analysist. Also, figure 1 incorrectly ties hard to the normal distribution while the truth is that we find in all the distributions. (There is at least one pathological exemption. See the literature for information of the Cauchy distribution.) Figure 1. In a normal distribution N(0, 1) the standard deviation is sometimes indicated as the distance from the mean to the point of inflexion. This is true but of doubtful value for anyone that is not an mathematical expert. Sometimes this is described as where the curve changes directions. It is easy to see that on a correctly drawn normal distribution the point is not easy to find. It is also defined as where the second derivative changes signs. Figure 2 and 3 show the first and second derivatives respectively. The standard normal distribution The height of the normal curve 0.4 0.3 The inflexion point 0.2 0.1 0.0 -3 -2 -1 0 Values of the varible X 1 2 3 A remark (II). When we talk about the areas under the normal curve e.g. the area (probability) to the left of - we usually use the expression the normal distribution as to indicate how the area is distributed over the x-axis. When we talk about the very curve we often use the expression the normal density function. A remark (III). It is rather surprising that people seems to accept the normal distribution without too much questioning. If you walk in to a workshop and say that there is a positive probability that the deviations are infinitely large, people would laugh at you. However, this is what the normal distribution states. But on the other hand this reminds us that the normal distribution is only a model and not the reality. The thought of the inifinite tails was hard to swallow when developing the statistical theory for an error distribution more than 200 years ago. Then some of the scientists used a certain limit for the deviations. However, this lead to very complicated distributions. © Ing-Stat – statistics for the industry www.ing-stat.nu Rev D . 2009-12-05 . 1(3) Figure 2. This curve is the so-called first derivative of the normal distribution or rather the normal density function in figure 1. To the practical analysist this is a completely useless curve. 1st derivative of the standard normal distribution 0.2 1st derivative 0.1 (The first derivative of a function is zero where the original function, here the normal density function in figure 1, has a maximum (or a minimum). This fact seems to be supported by diagram to the left as the curve is 0 on the y-axis at the mean on the x-axis.) 0.0 -0.1 -0.2 -3 -2 -1 0 Values of the varible X 1 2 3 Figure 3. This curve is the so-called first derivative of the first derivative of the normal density function. This is usually called the second derivative of the normal distribution. It is obvious, as previously mentioned, that this curve changes signs (i.e. passes 0) at exactly ±1 sigma. Again, to the practical analysist this is a completely useless information. 2nd derivative of the standard normal distribution 0.2 0.1 2nd derivative 0.0 -0.1 -0.2 -0.3 -0.4 -3 -2 -1 0 Values of the varible X 1 2 3 # These lines creates the diagrams set c1 -3:3/0.005 end let k1 = -1 let k2 = 1 let k3 = 0.3989*E()**((-(-k1)**2)/2) let c2 = 0.3989*E()**((-c1**2)/2) plot c2*c1; wtitle 'The normal curve'; marker k1 k3; marker k2 k3; text k2 k3 ' The inflexion point'; tsize 0.9; line 0 k3 1 k3; lestyle 2 2; title 'The standard normal distribution'; tsize 1.1; grid 1; grid 2; data 0.10 0.97 0.10 0.90; etype 0; axlabel 1 'Values of the varible X'; tsize 0.9; axlabel 2 'The height of the normal curve'; tsize 0.9; connect; size 2; Scale 1; © Ing-Stat – statistics for the industry www.ing-stat.nu Rev D . 2009-12-05 . 2(3) tsize 0.6; HDisplay 0 0 0 0; Scale 2; min 0; tsize 0.6; HDisplay 0 0 0 0; nodt. # --------------------------------------------------- First derivative let c3 = -0.3989*E()**((-c1**2)/2)*c1 plot c3*c1; wtitle '1st derivative '; title '1st derivative of the standard normal distribution'; tsize 1.1; grid 1; grid 2; data 0.10 0.97 0.10 0.90; etype 0; axlabel 1 'Values of the varible X'; tsize 0.9; axlabel 2 '1st derivative'; tsize 0.9; connect; size 2; Scale 1; tsize 0.6; HDisplay 0 0 0 0; Scale 2; tsize 0.6; HDisplay 0 0 0 0; nodt. # --------------------------------------------------- Second derivative let c4 = -0.3989*E()**((-c1**2)/2) + 0.3989*E()**((-c1**2)/2)*c1**2 plot c4*c1; wtitle '2nd derivative '; title '2nd derivative of the standard normal distribution'; tsize 1.1; marker -1 0; marker 1 0; grid 1; grid 2; data 0.10 0.97 0.10 0.90; etype 0; axlabel 1 'Values of the varible X'; tsize 0.9; axlabel 2 '2nd derivative'; tsize 0.9; connect; size 2; Scale 1; tsize 0.6; HDisplay 0 0 0 0; Scale 2; tsize 0.6; HDisplay 0 0 0 0; nodt. © Ing-Stat – statistics for the industry www.ing-stat.nu Rev D . 2009-12-05 . 3(3)