The Maximum Entropy Method for Reconstructing Density Distriubtions

A summary of using the maximum entropy principle from information theory to reconstruct a PDF (or other distributions) given a finite number of moments. This is quite useful especially when using the method of moments in population balances or related stochastic transport processes. The document also includes some results from my code for reconstructing some bimodal and trimodal Gaussian distributions, beta, and log-normal distributions.

Download PDF

Cite as:
Saad, T. "The Maximum Entropy Method for Reconstructing Density Distriubtions". Weblog entry from Please Make A Note. http://pleasemakeanote.blogspot.com/2011/06/maximum-entropy-method-for.html

The Mathematics of f/stop Aperture Numbers

A sequence is a set of numbers that can be constructed using a formula known as a recurrence relation.  The most obvious of these is the set of natural numbers (integers). How? Start with any integer (positive or negative). To get the next integer, simply add 1!

Some sequences are very obvious to decipher, while others require more mathematical manipulation, such as the Fibonacci sequence. Sequences often show up in pure mathematics, number theory, and computer science. [More about sequences].

One of the most widely used sequences is the f-number or f-stop (f/stop) series of numbers in photography. No matter where you stand as a photographer, you will be faced with these numbers. Often times, you will have to rely on your calculations to determine how many stops there are between two stop numbers. You will have to memorize them or just rely on your camera - unless you know the mathematics behind these numbers.

In this article, I will explain the method I use to remember the f-stop sequence. All that is needed is to remember the first two numbers. I will first quickly present the method so that you don't have to read this entire article. I will then present the mathematical formalism for the way f-stop numbers are constructed.

How to Remember the f/stop Numbers

I will start with the most common f-stop numbers. These are given by the following set of numbers

Let us write these in the following way

Looking at each row separately, you will quickly notice that these form what is called a geometric sequence. A geometric sequence is a set of ordered numbers, in which any number is obtained by multiplying the previous number by a constant value. This constant value is known as the common ratio. Guess what the common ratio (in our case) is? It is 2 (see proof below). Here's another graphic for that

As you can see, you only need to remember the first two f-numbers, i.e. 1 and 1.4. Separate them into two sets: the even set (first row) and the odd set (bottom row). Then construct the entire f-stop range just by multiplying by 2.

Remarks:

1. Note that each set presents jumps in two stops, not one stop. f/1.4 lets in four times more light than f/2.8. Being multiplied by two should emphasize that fact - for f/1.4, the lens diameter is twice as much as that for f/2.8.  (As will be explained below, to go one full stop at a time, you'd have to multiply by Sqrt(2)~1.4, e.g. 1.4*1.4 ~ 2 and so on...)
2. Looking at the odd set (bottom row), you can notice that 5.6x2 = 11.2, not 11. So why do we choose 11? To the best of my knowledge, it is just a convention to keep the numbers easy to remember. The actual f/stop used by the lens is 11.312).

As will be shown below, if we start by f/1.0 as the smallest possible f/stop, the next full stop is 1.0xSqrt(2) = 1.414. This is the first f/stop that corresponds to the bottom row. Now
1.414x2 = 2.828
2.828x2 = 5.686
5.686x2 = 11.312
11.312x2 = 22.624
22.624x2 = 45.248
and so on...

Mathematical Analysis

For those of you who are mathematically inclined,  the analysis that follows provides the rationale behind the construction of the stop number.

In photography, the lens aperture is that opening in the lens (or on the camera body) that determines the amount of light that is to be admitted to the light sensitive medium (film or CCD ...). The surface area of this opening can be adjusted by the use of a diaphragm. The action of closing or opening the diaphragm is called stopping down the lens (whether full or half). We define the f-stop number as

In science & engineering, S is referred to as a dimensionless number, meaning that it does not have any units associated with it. In our case, since the focal length and the diameter both describe a length (m, cm, mm...), their ratio is dimensionless because the units can be simplified (just like simplification of numbers).

The advantage of using a dimensionless quantity is that any results drawn form an experiment on a specific device (lens in this case) will equally apply to any other device with the same dimensionless number. For example, you've probably heard of the Mach number "M". The Mach number is a dimensionless quantity used in aerodynamics and describes how fast an object is moving in a medium (air) compared to the speed of sound in that medium. It is probably the most popular dimensionless number on the planet! (I think the f-stop number should be added to the list of dimensionless number). Now any experiment carried out on a model jet with M = 3 for example, will illustrate exactly what happens when the real jet is flying at M = 3 in the atmosphere (shockwave structure, pressure and temperature distributions ...).

Here's an example in photography. The amount of light REACHES THE SENSOR using a 28 mm lens with S = 2 is exactly the same as that of an 80 mm lens with S = 2, although the diameters of both apertures are different (if all factors that affect image brightness are held constant). (If you can't prove this for yourself, let me know and I'll write up my proof).

This dimensionless number is a very useful tool for determining properties of lenses (and therefore the light coming through) without referring to diameters or any other data.

In cameras, when we set the f/stop number, we are essentially setting the Diameter of the lens aperture. This can be computed by knowledge of the focal length. Therefore, for a given stop S, the diameter opening of your lens is

For example, a lens set at a focal length of 70mm and a stop number of 5.6 has an aperture diameter of

Note that S is inversely proportional to D which explains why as the stop number increases, less light enters the camera since D decreases (f/32 lets in less light than f/16).

Let us now compute the stop number (S) required to let in Twice as much light, for a given lens set at a fixed focal length.

We start by computing the diameter that will let in twice the amount of light. This is equivalent to saying that the aperture surface area has to be twice as much to let in twice as much light. For example, at the same focal length, an aperture with a surface area of 10 mm^2 will let in twice as much light as an aperture with a surface area of 5 mm^2. This is how it looks mathematically

We now set

or

finally

In other words, for an aperture to let in twice as much light, its diameter must increase by approximately 41%.

Now that we have a relation between the diameters of both apertures,  we can use the f/stop equation to deduce the recurence relation between the stop numbers. This is done as follows:

Thus, to let in half as much light, we multiply the previous stop number by the square root of two ~ 1.4. Alternatively, to let in twice as much light, we multiply by the reciprocal of the square root of two ~ 0.7. In general, for a stop number Sn, we have

where S{n+1} is the stop number that lets in HALF as much light as Sn, while S{n-1} is the stop number that lets in TWICE as much light as Sn. For example, if we are a stop number of Sn = 5.6, we have

In its present form, our recurrence formula depends on a fixed stop number. It would be useful if we can write our recurrence formula based on some initial reference stop number S0. For this, we do the following

With this general recurrence formula, we can calculate the stop number at any given number of stops from a starting number S0. For example, if our lens is set at S0 = 5.6 and we want to calculate the stop number corresponding to 3 stops (i.e. letting in 3 times as much LESS light), we have

This means that n is in fact the number of stops from S0. It is simply a counter of stops.

Smallest Stop Number

The question now arises as to what is the smallest stop number that a lens can achieve and how difficult is it to manufacture such lenses. I do not have any experience with lens manufacturing (although I have been contemplating learning that skill lately), but the mathematics could give us a hint. Looking at the equation for the aperture diameter

one could argue that the longer the focal lens of a camera, the more difficult it is to achieve small stop number. Here is why. Let's say that you have a 50mm lens, for a stop number S=1, this means that the aperture diameter is equal to the focal length, i.e. 50 mm. Threfore, the actual diameter of your lens MUST be at least 50mm! If you add the barrel and the internal mechanisms, the actual diameter will be even larger than 50mm.

Look at the Nikon 50mm F1.4D lens for example. Its maximum aperture diameter is D = 50mm/1.4 = 35.7mm while the lens' actual diameter is a whopping 64.5mm!

Let's take another extreme. For a lens with a focal length of 500mm, a stop number of 1 means that the aperture diameter is 500mm! Imagine carrying a lens that's half a meter in diameter (of 1.65 ft!). These  become impractical (unless you're dealing with a telescope). For example, the maximum aperture diameter for the Nikon 600mm f4.0 is D = 600/2 = 150mm. The lens has a diameter of 165mm and weight of about 5kg!

Bottom line is that the longer the focal length of a lens, the more difficult it is to manufacture it with wide apertures. That is also why fast lenses are very expensive! And that's why fast TELEPHOTO lenses are even more expensive! There are important design considerations to take into account in that case...

It is however convenient to choose S = 1 as the smallest stop number and start from there. In this case, our recurrence formula becomes

and here's how the famous f/stop numbers are generated:

Remember, each full stop lets in twice as much or half as less light. The above equation is for reducing the aperture size, i.e. letting in less light. (the converse recurrence relation can be easily derived).

Intermediate Stops

Now what about half stops, one-third stops or one-fourth stops? How are these numbers constructed? If a full stop lets in half as much light, does a half stop let in 75% light? Let's look at that.

Keeping in mind that the f-stop sequence is a geometric sequence (multiplicative), any value sought within an interval has to obey the rules of a geometric sequence. Let us insert a "partial stop" in the middle of a full stop interval

or

but, we know that

form which we can computer P, the common ratio for the half-stop sequence

The same principles applies for deriving one-third, one-fourth,... one-mth stops (divide the interval into m sub intervals). In general, for a one-mth stop increment, we will have

For example, for a 1/3 stop increment from S0=1, we have

There is an even easier way to derive non-integer stop formulas. Using

we notice that there is no restriction on "n" being an integer. As discussed previously, n is simply a stop counter. So, for example, if you want 1/3 stops, you simply substitute n = 1/3. If you want 2.4 stops, use n = 2.4 etc...

Number of Stops Between Two Stop Numbers

One can derive an equation for the number of stops between two stop numbers using the formula

By taking the natural logarithm of both sides of the equation, we get

For example, to calculate the number of stops between f/22 and f/1.4, we set

so that f/1.4 lets is 8 stops away from f/22 and lets in ~ 256 times more light than f/22! (at the same focal length. At other focal lengthes, the amount of light that reaches the sensor is ~ 256 times more).

Voila!

Cite as:
Saad, T. "The Mathematics of f/stop Aperture Numbers". Weblog entry from Please Make A Note. http://pleasemakeanote.blogspot.com/2010/10/mathematics-of-fstop-aperture-numbers.html

Inexact Differentials

In a previous post, I discussed the proper techniques to integrate an exact total differential. The major point to be drawn from exact differentials is that their parent function is independent of the path of integration. For example, the work done by gravity is independent of the path taken. It only depends on the end points of the path. This has to do with the fact that the gravitational force can be expressed as the gradient of a scalar. We call this type of force a conservative force field.

In general, many physical processes cannot be represented by conservative fields and therefore, their total differentials are inexact. One can think of the total differential as the a small increment taken on an arbitrary path. A very popular example of an inexact field is the work (and subsequently heat) in thermodynamics.

The work done by or on a system is in general dependent on the path taken. It is a summation of infinitesimal steps along the path. In contrast, the internal energy of the system is independent of the path taken. This has to do with the macrostates of a system. A macrostate of a system is a state where external parameters are specified. These include volume, temperature, pressure, mean total energy.

Then, for the mean energy U, the total differential is simply the difference between two known macrostates (remember, that the energy is specified for a macrostate). In contrast, the work done cannot, in general, be written as the difference between two well defined quantities. You can find more details on this in Prof. Richard Fitzpatrick's online textbook on thermodynamics.

So how do we integrate inexact differentials? Simple. If the path is known then the integration can be carried out along that path!

However, we will now show that if the inexact differential is multiplied by some function of the independent variables, one can construct an exact differential. To show this, I will follow the exposition given by Prof. Richard Fitzpatrick (http://farside.ph.utexas.edu/teaching/sm1/lectures/node36.html).

Consider the inexact differential equation

where I have used the symbol \delta to denote an inexact differential. An immediate consequence is that

Furthermore, the integral of F over a closed path is not equal to zero

To make further headway, let us consider the solution of

or

Dividing by H dx, we get

This equation describes the slope of some set of curves at every point in the x-y plane. These curves can be written as

where c is a constant labeling parameter. Think of this a set of controur lines for \Gamma. Note that Gamma is a function of (x,y), the constant on the RHS merely says that Gamma is constant on a given contour line. We now form the total differential of \Gamma

Now we want to connect the total differential of Gamma to the ratio dy/dx. To achieve this, we divide the previous equation by dx

upon substitution of dy/dx, we get

or

then

where sigma(x,y) is an arbitrary function of the independent variables. Then

Upon substitution into the original inexact differential, we have

therefore

and thus, by multiplying the inexact differential by a proper factor, one arrives at an exact differential. If this factor exists, it is called an integrating factor (its reciprocal in fact is the integrating factor). Such a factor may not exist in higher dimensions however.

In thermodynamics, for a reversible process, the entropy is written as

Note that the total differential of Q is inexact. But when dividing it by the temperature, one arrives to an exact differential. In this case, the temperature is an integrating factor and the total differential of entropy is exact.

Voila!

Cite as:
Saad, T. "Inexact Differentials". Weblog entry from Please Make A Note. http://pleasemakeanote.blogspot.com/2010/07/inexact-differentials.html

How Euler Derived the Momentum Equations

In this post, I will present Euler's derivation of the momentum balance equations for a fluid flow. Please note that I will use modern notation with a few modifications to the original derivation to clarify certain issues. Nonetheless, the core of Euler's derivation is maintained.

Considering two dimensional flows with axial and transverse velocities denoted by u(x,y,t) and v(x,y,t), respectively, Euler first forms the total differentials of these fields

Next, dividing both equations by dt, we obtain

but

then, upon substitution, we recover

One can infer that Euler is heading towards Newton's law. The previous expressions represent the acceleration of a fluid element. As to the forces, Euler identifies pressure, friction, and gravity. For the time being, he abandons friction and focuses on pressure and gravity.

For gravity, the force is in the transverse direction and is given by

where

is the mass of the fluid element.

For the pressure, he considers a rectangular fluid element as shown below.

Euler then assumes that the pressure at point L is p and deduces the values at the corners of the control volume. The net force acting on each side is calculated by taking the average pressure at the vertices defining that side and multiplying it by the area. This is shown in the figure below

Then, the net forces in the axial and transverse directions are given by

and

Now, by applying Newton's law, we have

or, by projecting in both directions, we have

and

Upon expansion, we get

and

Finally, we get the Euler equations

and

Voila!

[Previous: Continuity Eq. in Spherical Coordinates]

[Table of Contents]

[Next: Momentum Eqs. in Cartesian Coordinates]

Cite as:
Saad, T. "How Euler Derived the Momentum Equations". Weblog entry from Please Make A Note. http://pleasemakeanote.blogspot.com/2010/07/how-euler-derived-momentum-equations.html