Earthshine blog

The noise model is improving rapidly. Below is a simulation of a lunar image (right) from a synthetic moon provided by Peter; and the corresponding real image (left).

Images are equalised histograms with the same scaling of log(flux).

See www.astro.utu.fi/~cflynn/logbetter.jpg

Until now I had been using Gaussian sampling for the noise, but this fails badly when only a few photons are expected in a given pixel — while this is not an issue for most of the moon (ES>10 counts/pixel in typical exposures, and BS is of course very bright) — it matters a lot when modeling the halo far from the moon.

Replaced the Gaussian noise model with a combination of a Poisson model for low count rates and Gaussian for high count rates (>25 counts per pixel). The Poisson code is both very slow and inaccurate at high count rates, so a combined model was necessary.

The Poisson code comes from here:

http://www.netlib.org/random/zufall.f
Credit : W.P. Petersen, IPS, ETH Zuerich.
(I should check it’s public code).

More on the method to get a Poisson sampled count in regions where the expected number of photons is < 1 to follow later!

MORE stuff on this:

There is a timing hit doing Poisson noise, it’s quite
long winded.

For 30 summed images the total run time is about 40 seconds
on my laptop with Poisson noise. for 30 images with Gaussian
noise, the run time is about 2 seconds. The FFTs only take about
1 second of this. So there is presently a huge performance hit. I think this can be greatly improved.

Important note: 30 summed images using Gaussian noise look
very similar to 30 summed images using Poisson. This is
because 30 images of Gaussian sampling simulate Poisson
sampling rather well. It’s very different though when talking
about a SINGLE frame:

ABOVE : GAUSSIAN noise (left) versus POISSON (right): for a simulated lunar image in a single 30 ms frame. The very low light levels are much better modeled in the Poisson frame on the right, where the flux level drops to a few photons per pixel — and very much better when the level drops well below a photon per pixel.

Gaussian (right) versus Poisson noise (left) — but for 30 coadded 30 ms images. The differences cannot now be seen by eye, since mathematically the samplings are now more or less the same thing.

I have now written my own subroutine to do Poisson noise, and it seems to reproduce very well the results of the code acquired from http://www.netlib.org/random/. It’s slightly slower though; thought I could improve on the run speed easily, but I was wrong!

Need to still check for small systematic differences between expected mean number of photons and the numbers actually returned by the Poisson routine, since we need to worry about this at the 0.1 percent level. More on that later.

We can go far by analysing various synthetic images – but real-world images are different and come with problems we have to solve and adapt to. I suggest we use this blog category to add things we have found. Now and then I will collect submitted items onto a Master List Of Real World Problems (MLORWP!).

This is the MLORWP I can think of right now:

1) Real images are not centred in the image frame – take this into account when using synthetic images for analysis of some sort – the synthetic images are all right in the middle of the frame.

2) Real images have slight variations in scale and rotation due to slippage etc in the hardware. While we can always measure what the problem is we need to make sure that e.g. synthetic images are generated for the same conditions (e.g. image rotation – the CCD is mor eor less free to rotate!).

3) Use of FFT methods to convolve images carry some consequences – one Chris put his finger on is the centering of objects by the very act of folding: We need to make sure that when we model real images the resulting synthetic image is offset by the right amount. Henriette’s Python project with Kalle Åström at Lund U could come in handy here.

4) The noise in real images is probably higher (never lower) than that given by Poisson’s distribution.