Blog Image

Earthshine blog

"Earthshine blog"

A blog about a telescopic system at the Mauna Loa Observatory on Hawaii to determine terrestrial albedo by earthshine observations. Feasible thanks to sheer determination.

Bachelor’s Thesis based on the Andreas Mogensen Moon images

Post-Obs scattered-light rem., Student projects, Uncategorised Posted on Jun 28, 2024 12:46

Mathilde Saltoft Schou worked on the ISS images of the New Moon obtained by astronaut Andreas Mogensen. She based her Bachelor’s Thesis on this work. The thesis will be available here.

Effect of aligning multiple ISS images without (left) also rotating the images, and with (right).



ISS experiment

Uncategorised Posted on Aug 22, 2023 17:29

Starting August 2023 the ISS will have onboard the danish astronaut Andreas Mogensen who will perform several experiments for danish scientists specifically.

We suggested that the impact of scattered light from the lunar bright side onto the dark side could be investigated in an experiment based on comparing images of the Moon taken from space with simultaneous images taken from the ground.

https://ufm.dk/forskning-og-innovation/rumomradet/aktuelt/nyheder/2023/tag-et-billede-af-nymanen-for-dmi2019s-huginn-forsog-earthshine

https://www.esa.int/Science_Exploration/Human_and_Robotic_Exploration/Illuminating_Earth_s_shine



Entangled States blog, about the ISS earthshine experiment

Uncategorised Posted on Aug 22, 2023 16:51

Old friend, and Bishop, Nick Knisely talks about the ISS earthshine experiment on his blog: https://entangledstates.org/2023/08/22/calling-all-citizen-scientists-lunar-photos-needed-for-climate-research/

Thanks Nick!



What next, for Earthshine observations?

Links to sites and software, Observation Resources, Optical design, Post-Obs scattered-light rem. Posted on Jul 21, 2021 10:53

We now take the next steps, after building the Mauna Loa system and operating it: into Space with NASA!

The reason is simply that, from space we can avoid the variability due to observing through an atmosphere and do not need a global network of earth-bound telescopes – one suitable instrument will do it all from orbit.

Even from NOAA’s Mauna Loa Observatory (MLO) — one of the best observatories anywhere on Earth — we could easily see variability in our results due mainly to very thin high-altitude clouds. Extinction at MLO is low of course, but variable. Sure, the ‘bad seeing nights’ can be eliminated by detecting the variability each night, but then you end up with not very many good data!

When we noticed a small Earth-Observing student satellite ( @flying_laptop on Twitter) from University of Stuttgart — we asked if they would try to catch some images of the Moon for us — and they did! The images were interesting and taught us the importance of optimized optical design — optimization for the sake of driving down ‘scattered light’ (really, a phrase covering aperture diffraction as well as various internal scattering processes). We worked with the students and reported on what we found at the 2019 annual European Geophysical Union meeting in Vienna, in the “Earth radiation budget, radiative forcing and climate change” session that Martin Wild, and others, organize each year. See our poster here.

Building on that experience, we are now looking at more ways to go into space, and also to improve on the earthshine instruments we can orbit.

One such effort is with the SAIL (Space and Atmospheric Instrumentation Lab) at Embry-Riddle Aeronautical University in Daytona Beach, Florida. With three friends there we are putting together a NASA Instrument Incubator Proposal (IIP) for development of an optical system optimized for the task at hand: observing high-contrast targets with quite extreme requirements for performance.

In our present ground-based approach we are never really observing the Moon without a contribution from the atmosphere, and must resort to various ‘subtraction schemes’ to get rid of either a ‘halo around the Moon’ (which is light scattered along the path to the image sensor) or a ‘flat sky level’ which can be due to such things as airglow, or the Moon-light scattering up from the ground onto particles in the atmosphere (this is not a ‘halo contribution’). Both kinds of contributions have to be removed before the faint earthshine can be used for terrestrial albedo studies. From space, both of these contributions would be omitted automatically, leaving only a faint contribution due to aperture diffraction — our goal is therefore to study how to build a telescope that has as little diffraction as possible.

With our group of experts in optics and satellite payloads at Embry-Riddle we are considering refractive optics, advanced sensors and ‘baffling’ to minimize unwanted light reaching the image sensor. Our IIP proposal is being submitted this week!

We have tried NASA proposals before, and this is the second try, building on reviews of our first attempt. The opportunities — the proposal call aims — vary, so emphasis is different this time: First time we focused on the sensors, now we are working on the optics.



2015 total lunar eclipse analysed

Showcase images and animations Posted on Oct 10, 2015 10:21

The recent total lunar eclipse gave an opportunity to apply photometric methods. It is possible that the light from the Sun, passing through the Earth’s atmosphere is coloured by processes that ultimately also have a climate effect on Earth. Recent volcanic eruptions may cause the eclipse to appear more red than usual.

Can this be measured accurately?

The Internet was overflowing with images and image sequences of the eclipse. Among them this composite image by K. Lewis (www.photosbykev.com):

Using the method described in the paper by Park et al, we can convert the RGB values from the image file above to Johnson B and V colours (the possibilities are limited to these two bands – more can be done with the NIR filter taken out of the camera).

We extracted the mean and median B-V colour from the sub-frames above, and plot these values in a sequence, starting from upper left and ending in lower right frame:

We see the reddening of the Moon as it is eclipsed. The dotted horizontal line is a nominal value for B-V=0.92 based on literature values for the Full Moon, and the measured values from the composite photo were adjusted up by about 0.6 in B-V to roughly coincide with the 0.92 level. We see that the eclipse was redder than the Full Moon by about 2.5 in B-V.

The jagged outline of the curve, and the evident jumps in brightness in the main photo, seem to indicate that the detector used was not linear – this could be because the images were obtained not only on the linear part of the CMOS devices’ transfer curve.

What are published B-V values from eclipses, studied with professional equipment and attention paid to conservation of linearity? Figure 4 of the paper “Wideband Photoelectric Photometry Of The Jan. 20/21, 2000 Lunar Eclipse”, by Schmude, et al, 1999, in International Amateur-Professional Photoelectric Photometry Communications, Vol. 76, p.75+
(http://adsabs.harvard.edu/abs/1999IAPPP..76…75S)
shows a change in B-V of about 2.5 magnitudes during an eclipse – like above.

We obtained other images of the 2015 eclipse and also converted the RGB values to B-V, but found eclipse B-V very different from the above. Attention to obtaining Full Moon images just before and after the eclipse takes place is required, as is some attention for exposure levels, trying always to capture the various stages of the eclipse at mid-sensitivity range of the device used.

Also important is that there are gradients in the shadow of the Earth on the Moon and the geometry of the eclipse and the times of observation – and the location on the lunar surface used – all influence the results obtained.



Trial animation

Showcase images and animations Posted on Aug 16, 2015 10:29

This is a trial video of all our good V-band images. On the left is the linear image, on the right a somewhat histogram equalized image. Want to embedd this video, but here is a link until further notice.



Some thoughts on the DSCOVR Moon/Earth picture

From flux to Albedo Posted on Aug 07, 2015 11:34

NASA recently released a set of images showing the Moon crossing the disc of Earth, as seen from the DSCOVR satellite orbiting Earth in the L1 Lagrangian point. From that point the satellite can keep constant watch on Earth, with the Earth disc always fully lit. NASA image, from the DSCOVR satellite.

This image is interesting because it brings together so many different pieces of information, and possibilities. With the Moon an unchanging object (surface cratering might change on timescales of 100s of millions of years) in the same frame as Earth it becomes possible to very accurately calibrate photometry of the Earth.

Why do we need good photometry of the Earth? Well, weather satellites already give us a flood of images of Earth, but the images – especially from older satellites – are mainly suited for weather forecasting – “Here is a cloud mass, and it is moving this way, so we think it will rain or be cloudy at such and such a place a little later”. This sort of information is of enormous value to society on a day-to-day planning basis, but for climate research the data have to be absolutely calibrated – in other words, it is no good if the pretty pictures of Earth are taken with different instruments that have different sensitivities, or with instruments that slowly alter their properties. The resulting data would not be inter-comparable.

Climate research needs to have data for Earth that tells the story of what happens over time – this is the difference between ‘climate’ and ‘weather’: Weather is tomorrows’ events, but climate is the mean of what things were like 20, 50 or 1000 years ago. If changes in climate can be detected, then we can start thinking about what causes the change. A lot is known already, but every bit of new, reliable information can be put to good use.

Looking at the DSCOVR images we note the cloud masses on Earth – if you have seen the animated film showing the Moon passing across the disc you will realize that the Earth is turning as we watch – so new cloud masses are being brought into view while others pass beyond the limb of the Earth causing variability in the average brightness of Earth. Link to the animation is here.

Using image-analysis software it is possible to pick out the parts of the image that are Earth and the parts that are Moon, and look at the statistics of the numbers. The images are ‘RGB’ images and each RGB-image contains one image for the red light, one for the green light and one for the blue light. Because the Moon is in the image there is the huge benefit that even if the camera settings or properties varied slightly in time then the ratio between Moon and Earth brightnesses in the three colour bands would be independent of these errors or slow changes.

So, with these wonderful properties of the NASA image, how useful is it, in reality?
The amount of clouds on Earth relate directly to the ‘radiative balance’ that drives climate. More clouds mean less light get into the climate system to warm it up. Therefore, variations in amount of clouds will alter the temperature. We all know it is cooler when a cloud draws in front of the Sun, but over larger scales and longer times, how much will mean temperature change for a given change in cloud cover?

Estimates indicate that a 1% change in global mean cloud cover will change global mean temperature by 1 degree C – roughly (could be half that, could be twice that), so if we could be sure to measure mean cloud cover to an accuracy of 1/10th of a percent we could expect to have data that gave us important information in the clouds-vs-temperature question.

In the NASA image the variations in the clouds seen, due to rotation, amount to some 3% in the short segment of images provided. The images are taken about 15 minutes apart from the DSCOVR satellite so in that short film (containing 12 images with a full Moon disc in view) we cover almost a quarter turn of the Earth. Aha, so we now realize that the amount of clouds on Earth varies by several percent from hemisphere to hemisphere and probably day to day. Next day the amount of variability is some other number and some cloud masses have dissolved while new ones have been formed – so on a daily basis cloud variability on Earth is high. Uncertainty is therefore high, as to the actual number of clouds present – weather forecasting may care where the clouds are and how many there are, but climate investigations need more precise (less scattered) data.

If we average many images of Earth the variability would average out and the uncertainty go down. How many pictures of Earth, like the ones here, would you need to have for the average cloudiness to be of use in the clouds-vs-temperature question?
The answer turns out to be that with about 1000 independent images the average would have an uncertainty of 0.1%.

How do we get ‘independent’ images? The phrase refers to ‘statistical independence’ and has to do with how long you must wait for a fresh cloud-scape to be present over the visible disc of the Earth. Weather systems can last several days so you can perhaps get an independent image of Earth every 2 or 3 days.

It therefore looks like data gathered over a decade could produce data for cloudiness that could be used for investigations into climate change mechanisms via the cloud-mechanism. This answer holds equally well for ordinary satellite data except these data have the added problem of instrument drift.

Of course, the Moon only crosses the Earth’s disc as seen from the DSCOVR satellite every so often – you certainly do not get a sequence of images like the above every day – or Month. So is it useless to use these data for the climate-change purpose then?

Well, all satellites have a basic problem – they do not last long in deep space due to radiation damage from cosmic ray particles and outbursts from the Sun, so they get replaced every several years or so. This will probably also have to be done with the DSCOVR satellite, if there is funding for it. The point I make here is that the presence of the Full Moon in the image frame every so often allows a re-calibration of the DSCOVR satellite instruments that give unprecedented accuracy. Geostationary satellites are between the Moon and Earth and do not capture the Moon in-frame very often, and when they do it is as a small object peeking out behind the Earth in a corner of the picture. These chance-occurrences are, however, used nowadays to calibrate satellite instruments, by EUMETSAT and others. Classically, satellites were calibrated on bright white surfaces on Earth, such as salt deserts – this gives about 1% precision and accuracy, while the lunar methods are able to reach a few tenths of a percent accuracy.

Another comment that can be made based on the NASA image is that the Moon appears dark against the Earth. In principle this is well understood because the lunar soil is dark while Earth is full of bright clouds, icecaps and a scattering atmosphere. Desert are also very bright, but forests are dark, as is the cloud-free ocean. With the images at hand we can readily measure how bright the Moon is relative to the Earth in each of the R, G and B colour-bands. The ratio should give the ratio of albedos (reflectivities) of Moon and Earth, because the geometry is the same – Sun is behind the DSCOVR satellite and both Earth and Moon are ‘Full’ at the time the images were taken. I did this and find a ratio about 2 or 3 times higher than the standard ratio expected. The Moons’ albedo is in the range 0.05 to 0.10 while Earth is near 0.3 so the ratio should be 1/6 to 1/3, roughly, but actually comes out as 0.8.

Why is this? I think there may be two reasons – first of all the back, usually hidden, side of the Moon is brighter than the front because there are more mountains and they are bright. The front has lunar Mare and they are darker than the mountains. Second, we are seeing the Moon under Full Moon conditions here and albedo, or reflectivity, depends on the angle of viewing, just as any surface you look along. Near vertical incidence the lunar soil appears brighter than at other phases. This is known as the ‘opposition surge’ and explains why the Full Moon, as seen also from Earth, appears brighter than at other times – not just because a larger fraction of the Moon is illuminated then but because a lit region simply reflects more of the incident light when the incidence and viewing angle is 0 degrees – straight down and straight back out again.

Why does albedo depend on the viewing angle? Any rough surface has shadows behind boulders and mountains and only when the angle of incidence is the same as the angle of reflection will the shadows be hidden from the observer – looking at a lit and rough surface from another angle and the shadows can be seen, leading to a lowered average reflectivity. The lunar also has small beads of glass and crystalline material which contribute to the enhanced reflectivity during Full Moon. Is there no ‘opposition surge’ for Earth? After all the geometry for Earth and Moon is the same here so a surge for the Moon might also cause brightening of Earth? I don’t know yet – but note the pretty ‘sunglint’ seen in the centre of Earth during the animation.

The images used here are not ‘science grade’ but have been optimized for viewing online so transformation of colours have been applied in order to yield a pleasing and realistic image – perhaps the scaling has caused an offset in the brightness ratios, explaining the large Moon/Earth albedo ratio. The raw science data will be released by NASA later, and consists of 10 very narrow band-pass filters.

We might pause to wonder why all this effort is worthwhile? The answer is simple and has to do with reproducibility of observations – we need to have many estimates of all properties of the climate system in order to understand it fully, and winnow out the erroneous data – a point made brilliantly by Ruth Mottram in her excellent blog sternaparadisaea.net.

As they say on Mythbusters “If it’s not reproducible, it ain’t Science!”

A very clever alternative to watching Earth from Space in order to determine its reflectivity, is to stay on Earth, or near it …, and watch Earth from there – by using Earthshine observations! And the rest of this blog is about our attempts to do just that.



Effect of flatfields

Data reduction issues Posted on Jun 05, 2015 12:10

We have generated ‘super flatfields’ by considering all the flat fields we have – we have many exposures through each filter taken on sky, the dome inside and the hohlraum lamp. While the patterns are the same for the three types (mainly a roof-tile like diagonal pattern’ we have dips in the corners of especially IRCUT and VE2 (and VE1). It is therefore not obvious which flat field is the ‘correct’ one. Assuming that the ones with the least vertical relief must be the ones closest to reality we have adopted one superflat for each filter.

We need to investigate if there are other ways of testing which FF is best – and we have the possibility of ‘scamp’ and ‘sextractor’ to generate background fields, which we assume are some sort of smoothed residual after sources are removed. Using frames from NGC6633 we generate these background fields from ‘raw’ frames and ‘bias-subtracted and flat-divided’ cluster images.

A sample of the background fields, shown as contour plots, for one science frame is shown here:

The upper field is for ‘raw’ and the lower for ‘bias-subtracted and flat-divided’. The contours seem to have a mean near 18 for both – the contours evidently are not the same. The differences between the two backgrounds amount to 0.1 – 0.2 counts.

We should center the backgrounds on sky-coordinates and average and see if the fields are consistent given method (we assume so). TBD.

If the patterns are stable we have learned that applying bias-subtraction and flat-division can alter the levels by a few tenths of a count. This may be important for photometry of the DS. Presumably it is better to correct for the flatfield than not to, but that depends on whether the flat-processed fields are consistent. Perhaps warp can be used to generate a stacked field, and its background?

Added later:
Doing just that we get the average background frame for ‘raw’ and ‘bias-subtracted, flat-divided’ processing:

The relief in that field (took the most-overlapping 512×512 region) is from +0.1 to -0.02 — so some averaging has occurred, but there is still a one-tenth of a count effect of applying the flat (in, B, and when 14 images could be stacked).

So, we cannot rule out that applying flats have a large effect compared to DS photometry – i.e. 0.1 out of 1 is 10% and at best 0.1 out of 10 is 1%.

The effect is regional inside the frame and so we can well have an effect on the limited-region fittings we perform on the DS.

Might want to consider what the effect on the BS-halo, evaluated at the DS, is if the BS also has flatting-effects brought to it.

Basically, applying flats are necessary as long as they do not bias results due to bad production of the flat-field itself.



Two-way smear from overexposure

Data reduction issues Posted on Apr 27, 2015 10:41

Here is an example of a two-way smear. The red area is the overexposed region, and it has bloomed along vertical columns in the image. To the right of this bloom is a smear extending hundreds of pixels. What is that?

Read-out direction is top-down.

What is causing this smear to the right? Is it related to the ‘depression of the sky in a broad band left-right’ that we see in some images?

The intensity of the smear is at the 500 count level – i.e. it acts just like part of the halo from the BS. This has, surely some consequences for data reductions? Do images that have not bloomed have this halo? Is this why the bright edge of the BS-halo is always hard to model?

Can we see if the fitting of the BS halo is easier in images that have been modestly exposed, compared to those exposed right up to 50.000 or 55.000 counts?



Extinction two ways

Data reduction issues Posted on Jan 13, 2015 15:34

Atmospheric extinction is caused by absorbtion and scattering of light.

Usually photometry is performed on images of sources by collecting the flux from an area near the source and relating it to the airmass of that particular observing moment. In a Langley plot the relationship between magnitudes and airmass will be an (almost) straight line, and the extinction coefficient is the slope of that line.

As light passes through more and more atmosphere there will be more absorbtion as well as more scattering. Both remove intensity from the direct beam. Absorbtion just takes the light away (at least at optical wavelengths) while scattering redistributes the light near the source. If the method of measuring light from the source therefore allows re-capture of the light scattered – for instance by allowing for large collection areas in the image – then the effect of scattering will be reduced and a smaller extinction coefficient should resultr, as compared to the extinction calculated when scattering is allowed to remnove light from the calculation.

Thus, when considering an extended source like the Moon we have an opportunity to test the magnitude of scattering vs absorbtion by explicitly collecting all light near the Moon (for instance, by summing the flux in the whole image frame) and comparing to what we find when light is collected only from a small patch on the lunar bright side.

We have done this for 5 wavelength bands and a score of observing nights where conditions were right. We measured flux in each image in two ways – either as the total flux in the image frame or as the flux inside a small patch on the lunar brightside defined by selenographic longitude and latitude. Langley plots were constructed and slopes measured using regression, along with error estimates of the slopes.

Results are plotted here:

Langley extinction for whole-image fluxes plotted against extinctions determined from a small bright-side patch in each image.

If the image is rotated and inspected (sorry about that) we see that for the five filters extinction determinations often agree, despite the expectation that extinction determined from the small patch (allowin scattered light to escape) would be larger than the extinction determined from whole image. Only a few points deviate significantly from the diagonal line.

This implies that scattering at MLO was very low, compared to absorbtion – as scattering includes Rayleigh scattering thge plot for B should be considered carefully.

In a few cases the plots allows us to identify ‘bad nights’ when the extinction determinations do not follow the diagonal.



Earthshine on Moon seen from the ISS

Showcase images and animations Posted on Dec 11, 2014 09:45

From the International Space Station (ISS) the Moon can of course be seen, and has been photographed with hand-held cameras. Someone took an image of the Moon with exposure settings chosen to highlight the earthshine. Below, such an image is discussed:

At the bottom the jpeg image is shown. A line has been drawn on the image. The slice of the image is plotted at the top. The R channel in the RGB file was used – G and B are similar but there is a higher sky-level than in R. More details on this image is available at http://eol.jsc.nasa.gov/scripts/sseop/photo.pl?mission=ISS028&roll=E&frame=20073#

On the plot we clearly see the dark sky above the Moon, the entry onto the lunar disk, the extensive dark side, the bright side and the transit of the bright atmospheric layer and then entry onto the dark Earth below.

We note that the bright side is saturated as the intensity reaches 255.

We do not know whether astronauts on the ISS have available high-dynamic range cameras. We think the above image is snapped with a standard 8-bit camera.

Using 16-bit handheld CCD cameras – such as a Hasselblad fitted with a 16-bit digital back – images of the earthshine could be obtained that showed bright side and dark side at the same time without bright side saturation allowing the DS/BS ratio to be calculated. That ratio is proportional to terrestrial albedo.

Such images could be of educational value and could highlight the role that earthshine studies can have in climate change research.

Added later:

Since the atmospheric profile is quite uniform horizontally in the above image, we estimate the mean atmospheric profile and subtract it from the slice across the Moon, thus revealing the profile across the Moon, without atmospheric effects:


The top panel shows, in black, the profile across Moon plus atmosphere; the red graph shows the profile of the atmosphere adjacent to the Moon; the bottom panel shows the difference.

We note that the atmosphere-free profile in the bottom panel above shows almost no ‘halo’ from the bright side – unlike our own images, taken from MLO, through the atmosphere.

That piece of information is quite interesting: As we understand it, our own images have a contribution to a halo from the optics of our telescope along with a variable halo due to the atmosphere scattering. From the ISS a camera, with a lens, was also used and one wonders why there is no sign of this optics-halo? Is the ISS camera optics vastly better than ours? This is not likely, since, probably, any old hand-held camera was used. On the other hand, modern lens-coatings (such as in the Canon ‘Dragon-eye coating’ lenses) have high standards and are designed to suppress lens-scattering and internal multiple reflections.

So, we wonder if the ISS has some special lenses at hand?

We note that the above image is for an extreme ‘almost new Moon’ situation with little flux from the bright side to scatter and interfere – we should see if more ISS images of the Moon, showing earthshine, at phases more like 1/4 are available? If those images also do not have any appreciable halo due to optics we might be on to something.

Note that the image of the Moon is probably taken through a clear spherical dome or window mounted on the side of the ISS – what are the implications of this for observing the Moon through a dome, here on Earth? Is distance between dome surface and camera lens a factor?

Notes:
1) Extinction not accounted for – this may cause the ‘slope to the left’ of the DS seen above.
2) The image was taken in August 2011 using a Nikon 3DS camera. That is a 14-bit camera taking .NEF frames. We have this frame from NASA now.



Internal variability

Post-Obs scattered-light rem. Posted on Sep 22, 2014 10:31

For a given stack of 100 images, how large is the uncertainty on derived albedo?

We investigate this using one of the ‘good’ stacks of 100 images for the V band. We call the stack ‘good’ because it seems that repeated fits to it with slightly different settings of the fitting algorithm give similar results.

We split the stack into 33 3-image averages, 16 6-image averages, 8 12-image averages, 4 25-image averages, and 2 50-image averages. We also fit the 100-image average several times. We fit each of these multi-image averages and plot the results:
– so the conclusion is that flux constancy varies.
Top left: log-log plot of how the RMSE depends on number of frames in each average. the RMSE is calculated in the areas used for the fitting of the model to the observed image. The fitted line has slope minus a third. Top right: log-log plot of how the estimated uncertainty of the Albedo-fit (given my routine LMFIT) depends on number of frames averaged. The fitted line has slope minus a third. Middle left: log-log plot of how RMSE and Albedo uncertainty depend on each other. The line has slope near 0.9. Middle right: Fitted albedo against number of frames. The error bars are the formal estimates of the albedo fit uncertainty (given by LMFIT). Bottom left: How the standard deviation of the Albedo determinations (in middle right plot) depends on number of frames averaged. To avoid ‘overplotting’ we have offset N slightly by adding small random numbers but still maintain a clusteringa round the actual N value.

We see some interesting things in this evaluation of ‘intra-stack variability’: The uncertainty (both RMSE and Albedo uncertainty) ought to drop as one over the square root of number of frames averaged, if the noise causing the scatter is normally distributed and independent and there were no biases. As it is, the errors drop with a minus 1/3 slope indicating that something is holding back error-averaging. We also see some hint in this in the middle right panel: albedo seems to depend on how many frames we average – the frames were also averaged sequentially (3-averages formed from frames 1,2,3 then 4,5,6 and so on). We can test if the order of averaging is important by doing some bootstrapping and randomizing (TBD) but there is also the possibility that increased averaging allows some subtle biasing factor to emerge. Finally, we see that intrinsic error (i.e. that not due to choice of image stack but due to factors like noise within a given stack) can be brought below 0.2% when averaging just 20 frames. We also see that the formal estimates of albedo uncertainty are not that far from the scatter of the albedo determinations themselves (but might be off by a factor of 2 or so).

So, in summary, we have found that scatter can be brought down to the few-tenths of a percent level by averaging and using our fitting method and observed images. [Wahey! That is actually a major point for the Big Paper! Rejoice!] We find that there is some sign of bias causing a monotonic evolution of the fitted albedo with increased averaging, as well as a decrease in scatter that does not drop as expected for averaged independent noise.

We can test if the order of averaging is important by bootstrapping, and this will be done. It could help rule out effects due to some time-dependence of the properties of the images in the stack.

CONTINUED LATER:

We have now tested some more ideas: We test effect of alignment as well as effect of the order in which the images are selected for averaging inside the 100-image stack.

Upper left: red line and black points show the old result – sequentially selected frames are averaged, without alignment. Green points and line – the images are selected non-sequentially as in ‘bootstrap with replacement’ and are also not aligned. We see that the slopes of the lines are the same – about -1/3 but that the bootstrapped results have about 25% higher error than the sequentially selected.

This gave the idea that ‘something changes’ along the stack which results in increased error. It could be that the Moon drifts slowly across the sky so that mis-alignment is more evident in sums of frames selected far apart as opposed to in sequence. There could also be ‘other time dependent’ things going on – such as various CCD-related effects.

Upper right: to test the drift hypothesis above we next aligned the selected frames when bootstrapping. As we see there is no effect of doing this! So drift is not the problem – or is it? Alignment consists of sub-pixel shifts that may induce variance-changes due to non-conservative interpolation in the image frame. We have considered this in this blog (see discussion and further blog references HERE). Since the green data look the same in upper left and upper right we conclude that alignment (i.e. possible flux non-conservation) cannot be the issue.

Bottom right: This shows that the scatter on bootstrapped albedo determinations are now larger than for sequential access, and that levels between 0.2 and 0.3% can be reached, at best.

So why is it ‘bad’ to pick the images in random order, rather than sequentially?

Speculation department: If it turns out that alignment simply isn’t the issue then what? We have the shutter and we have the bias. In the above we assume that shutter works fine and that the bias is the same in all frames. We test here the shutter, by looking at the total flux of each image in the 100-image stack:
Top panel shows percentage deviation of the total flux in each bias-subtracted frame relative to the average of the whole stack. The standard deviation is 0.46%. Flux is not constant from frame to frame. Why? Second frame shows the maximum flux in each frame after smoothing. The two graphs are very similar. The last frame shows the mean of a dark corner in each image. The graph does not look like the other two – so it is unlikely that strong sky variations or changes in bias caused the changes in total flux.

We guess that the changes in flux is due to either changes in atmospheric transmission or shutter variability.

But how could flux in-constancy cause the problems we see with increased errors when bootstrapping? Apart from a few big dips in the total flux, most points seem to be randomly scattered wrt the neighbours. In that case sequential access as opposed to random access of image sequence would not matter, would it?

Hmmm.

There is the ‘wobble’ of the image as seen in a film we have on this blog – waves of ‘something’ wash across the lunar disc causing small transmission changes. Sequential images may then be more alike than images selected far apart, and alignment, being whole-image shift based, would not repair the problem due to ‘wobble’. A test suggests itself – look at fits to single images in a stack and the scatter these have. Then consider whether that is more or less than the scatter seen between n-frame averages of the two types.

CONTINUED EVEN LATER:

It was tested whether albedo fit scatter increased or decreased when integer pixel shifts and sub-pixel shifts were used: It turns out that sub_pixel shifts is better than integer pixel shifts in most cases, but that a straight sum (i.e. un-aligned) is best! NB: This is the case for the present choice of 100-image stacks. YMMD.



MM ears, revisited

Post-Obs scattered-light rem. Posted on Sep 10, 2014 09:30

Below, we have investiagted the phenomenon of ‘Mickey Mouse ears’ – i.e. the tendency for theoretical lunar images to have too bright cusps so that the residual image ends up with a bipolar structure ressembling Mickey Mouse’s ears. We have investigated what happens to the ears if a semi-empirical BS model image is used, based on scaling and clip-selecting the BS from the observed image. We showed that the ears went away.

Now we investigate what happens to the quality of the fit – for our goal is to derive terrestrial albedos with minimal error (both scatter and bias; bias we can live with as long as it is constant; scatter is a pain).

We selected 10 images that we fit with both semi-empirical BSs and theoretical BSs and tabulated the values for fit-parameters as well as some of their formal fit errors (output from LMFIT).

In this plot we show the changes, in percent, of some key parametrs and their errors, in the sense that the purely theoretical result is subtracted from the semi-empirical result:

Change, in %, of some key fitting parameters. (semi-empirical – theoretical)/(½*(semi+theor))*100.

A: albedo, is larger by several percent in the semi-empirical (S) fit, than in the purely theoretical (T) fit.

dA: error on albedo reported by the fitting routine, is mainly smaller in S compared to T.

alfa: the power to which the PSF is raised is less by several % in S compared to T. This implies a broader PSF in the final fit in S.

ped: the pedestal added to the whole image has both positive and negative changes.

dX: the shift of the model image in X direction needed to align model and observed image. Mainly smaller in S than in T.

cf: core factor on the PSF – has larger values in S than in T. So the PSF peak is ‘pulled up’ in S relative to the wings.

contrast: lunar albedo map contrast, has smaller values in S than in T. This is consistent with the ‘pulled up’ core which ‘copies’ detail better.

RMSE: the root mean square error in the area of the image used for minimization, is mainly smaller in S than in T.

So, what have we learned? The two formal estimates of errors – dA and RMSE are both smaller in S than in T, so S would appear to be a better way to go.

Taken together – the smaller errors (on A, and RMSE itself) in S compared to T is encouraging, but we really do not know how much closer to the ‘true value’ for A we are, just that we are probably closer – at least in terms of scatter, not necessarily in terms of bias.

Using the semi-empirical method compared to the fully synthetic method adds very little in terms of computational overhead, so we will use S henceforth.

We note that the width of the distribution of albedos in units of the mean is 8 percent smaller in S than in T. The S method gives tighter distributions of albedo. Of course, the intrinsic (geophysical) scatter in albedo is present in this number too.



Mickey Mouse ears.

Post-Obs scattered-light rem. Posted on Sep 08, 2014 11:30

It seems our synethtic model images of the Moon have a distribution across the lunar disc that does not match observations. When we fit such an image to the ‘band’ near the photo-equator and subtract the fitted image from the observed image we get a residuals image with ‘Mickey Mouse ears’ – i.e. the residuals in intensity are bi-polar.

We are testing a semi-empirical method where the BS of the synthetic image is replaced by the observed BS, scaled to the right value. When such an image is fitted to the observed image we get no Mickey Mouse ears problem:

Left image: Residuals after a well-fitting synthetic image has been subtracted from the observed. Note the polar nature of the residuals – the cusps are clearly too bright while the area between the cusps is too dark. Right image: residuals when a well-fitting image, based on the observed BS, is subtracted. There is very little large-scale structure now.

The two fits deliver terrestrial albedo as one of the fitting parameters.
For the left image (i.e. the entirely synethtic model image) we get A=0.37298, for the semi-empirical image we get A=0.36311, that is a difference of 2.7 %, which is considerable, given our science goals of 0.1%.

Whether the fit has been improved where it counts is a different matter. The RMSE for the two fits are identical (RMSE=0.026).

The large change in fitted albedo is a (regrettable) feature of our fitting method – change something a little bit and the fits change by several percent.

We suggest that a meaningful test of fit quality will be to fit the ensemble of images with the two methods above and see if the scatter is less in one than in the other. TBD.



Reflectances

Data reduction issues Posted on Aug 30, 2014 11:43

We have commented earlier that for an otherwise well-fitted model image the ‘horns’ or cusps of the fitted image appear relatively too bright compared to middle-of-the-disc pixels. We investigate this now.

For an observed image and the corresponding best-fitted model image we extract certain pixels defined by the incoming and outgoing angle wrt to local normal on the lunar disc and compare the intensities in the observations to those in the model image. Here is an example:
The upper panel shows the observed image with certain pixels picked out (the even dark band along the terminator). These pixels all see the Sun under angles between 55 and 65 degrees wrt the local normal. In the lower panel we show the intensities of those pixels in the observed image (black) and the model image (red) plotted against the angle between local normal and the direction towards the Earth.

We see that the model ‘dips’ for large angles (i.e. when the direction towards Earth is almost paralell with the local surface) while the model values (red) are unwaveringly high. We also notice that the model values start low at low angles and rise with a modest plateau starting at 75 degrees or so.

If we repeat the above for other sun-angle bands we see the same pattern – model is ‘flat’, observations ‘dip’ when the angle to Earth is large, leaving the cusps always brighter in the model than in the observations.



Saturn is brighter than Earthshine

From flux to Albedo Posted on Aug 11, 2014 10:29

This is an image of the Moon occulting Saturn. Saturn is evidently brighter than Earthshine – however, this is a phase near Full Moon so the Earth is near New, and thus faint. Still, eh?
From APOD. Image by Carlos Di Nallo.



Solar halo profile Aug 1 2014

Exploring the PSF Posted on Aug 01, 2014 13:36

With a 1-degree occulter I took an image of the Sun. A slice across that image, through the darkest spot on the occulter gives this profile, plotting from darkest spot and outwards:

A log-log plot of the radial profile of the occulted Sun, on August 1 2014, using f22 and 1/500 s exposure, and a 1-degree occulter. Pixel scale is 0.04 deg/pixel, so 1 degree is at 25 pixels and the 200 pixels are at 8 degrees. The image is from the R-plane, G and B are not shown.

The occulter is the dark part at left stopping near pixel 20. The drop-off after pixel 200 is some roof or something like that, while the curve between pixels 20 and 200 are what the halo around the Sun looks like. The CCD is 12-bit so saturation starts near 4000, and non-linear response, perhaps, before that. However, we see clearly that the dropoff due to scatter in the optics is faster than dropoff of halo intensity. This implies that optical scatter plays a smaller role in defining the halo than does atmospheric scattering.

Note that this is with a multi-element SIGMA  telelens – albeit on a rather hazy day.
Of interest for our own little earthshine telescope is to repeat the above on some crystal-clear day (hopefully approximating conditions on Mauna Loa) and see if the optics drop-off is also faster than atmospherics then.

On Mauna Loa, of course, the effects of optics are what they are, independent of altitude while the atmosphere is MUCH clearer there – and there is less of it than in Frederiksberg.



Pedestal is strange

Post-Obs scattered-light rem. Posted on May 23, 2014 14:37

In our fits of model-images to observed images we fit a model with several parameters. One of them is a ‘sky pedestal’ which is a constant that is added to the model image before fitting in order to accomodate sky brightness that is not due to the scattered-light halo.

We include this term because we figure that the sky might be uniformly bright for several reasons:

1) moonlight could be reflected from the ground up to the sky and back into the telescope
2) airglow could brighten the sky
3) unresolved stars and zodiacal lights shine from the sky behind the Moon
4) light could be scattered inside the telescope and contribute a more or less uniform ‘haze’

We show here the magnitude plot of that pedestal term against lunar phase for our 5 filters:

Error: axis labels have been switched!

In order to interpret this plot you should know that the pedestal term is ADDED to the model image, so a positive value of the pedestal implies that the model is representing a sky brightness, and that a negative value of the pedestal implies that the model needs to subtract something before it can fit the observation well.

Notice that the pedestal term is negative for all but the VE2 filter. That is, only in VE2 is a well fitting model one that has a sky brightness contribution. For the remaning filters the model needs a subtracted term for the fit to be good.

Notice also that the VE2 pedestal is about 4-6 times larger than the other filter’s, in absolute terms.

Notice also that the term goes to zero for large phases (i.e. near New Moon).

So – turning to speculation now – what is going on?

Only VE2 behaves as expected: As the Moon gets brighter more light has to be removed. This could be Moonlight scattered from the ground, onto the sky, and back to the telescope [See the paper by Bernstein, R. a. R. ∼A. “The Optical Extragalactic Background Light: Revisions and Further Comments.” The Astrophysical Journal 666, 663–673 (2007). They model the moonlight reflected from the ground near a telescope.]. Or light scattered inside the telescope – but not airglow (why would it depend on lunar phase?). Nor can it be zodiacal light because although it depends on lunar phase when you observe the Moon in particular it has the wrong phase-dependence – towards Full Moon the ZL near the Moon is low; towards New Moon the ZL near the Moon is stronger (because the Moon is nearer to the Sun and the ZL is strongest near the Moon).

So – the pedestal for VE2 behaves as we expect widely scattered light from the Moon should behave.

By the way, let us call this light ‘ADL’ from now on – ambient diffuse light, to tell it apart from the light scattered in the halo, which has a strong profile with distance from the Moon.

But for the other filters the pedestal behaviour is opposite!

I think this could be a sign of the fitting process having a problem – the method has to specify negative pedestals for something else to work. That could perhaps also be the case for VE2 but to a smaller extent than the conventionally expected scattering: if it is bigger for VE2 a contribution could still be taken out leaving a positive pedestal for that filter.

Do we have any reasons to believe in stronger ADL for the VE2 wavelengths than the rest? The VE2 filter is of a different nature than the rest – perhaps it scatters more light? This would match the phase-behaviour.

What could the role of airglow be? While the zodiacal light and the galactic background and stars come from behind the Moon and are blocked by this, the airglow is all over the image. What happens when we fit such images with just one pedestal term? Can the procedure go wrong in the observed way?

What do we know about airglow? Does it depend on wavelength?

WORKING HYPOTHESIS:

Let us assume that: In the VE2 filter there is a lot of scattering, and the method of fitting deals with this by specifying a large positive pedestal. For reasons not yet understood, there is a tendency to make the pedestal a little too small – but this is not seen in VE2 because the scattering is so large. In the other filters the scattering is less and the error comes through as a small negative pedestal.

PLAN:

Work on understanding why the pedestal needs to be a small negative number for the other filters. Perhaps it is a consequence of some of the choices of frozen parameters in the model. We do freeze ‘core factor’ and ‘rlimit’, for instance. Optimally, the factors that need to be frozen should be done so at values that do not induce strange problems such as the present one.

Secondarily, find out if the small negative pedestal influences albedo determinations significantly. With the DS intensity being proportional to albedo we can see that the small negative pedestal values directly correspond to albedo biases in the 0.2% range.



Bulk extinction coefficients

Data reduction issues Posted on May 23, 2014 13:12

In this entry we estimated extinction coefficients for all data from each filter – that is, we did not study each night individually, but took all B, V etc data seperately.

We found, at that time:

B 0.15 mag/airmass
V 0.10 mag/airmass
VE1 0.08 mag/airmass
VE2 0.06 mag/airmass
IRCUT 0.12 mag/airmass

Since then we have eliminated some observations that we now know had problems of one sort or the other, and have the opportunity to re-estimate extinction coefficients. We now find:

B 0.18 mag/airmass
V 0.11 mag/airmass
VE1 0.06 mag/airmass
VE2 0.09 mag/airmass
IRCUT 0.05 mag/airmass

kB is 0.03 higher; kV 0.01; kVE1 0.02 lower; kVE2 is 0.03 higher and kIRCUT is 0.07 lower. The changes of +/- 0.02 are as expected given Chris’ analysis of extinction from single nights, but the change in kIRCUT is large – however, it is now more in line with kVE1: the two filters are almost identical, so that is a step in the right direction.

We found these extinction coefficients by plotting extinction-corrected flux against lunar phase and fitting a third-order polynomial. For trial values of the extinction coefficient



Large halo

Data reduction issues Posted on May 17, 2014 12:23

In some of our images sky conditions were such that the halo obviously was ‘too large’ for good results to be expected. We seek to eliminate these images now.

One simple measure of a halo that is “too large” is by looking on the BS and finding those images where the halo drops slower than in other same-phase images. We calculate and index of halo-narrowness based on the ratio of brightness 20 pixels beyond the BS edge, and the maximum brightness in the image. We call this the Q20 index.

On nights with ‘large halos’ we also often see a bright sky quite far away from the lunar disc, on the DS part of the sky. We quantify this by the mean brightness of the 10×10 box in the lower DS corner of the bias-subtracted image.

We plot now the phase vs Q20 and the sky brightness vs Q20:

Each row is for one of the 5 filters, left column is Q20 against the illuminated fraction of the lunar disc (i.e. phase). Right column shows sky brightness on the DS vs Q20. Q20 is the ratio of the BS intensity 20 pixels from BS edge on the sky and maximum brightness of the image.

We see that Q20 is quite constant against lunar phase, implying that we are correctly normalizing the halo. We see only a small drop in Q20 for (perhaps) V and VE2 filters as phase becomes very small.

We see some outliers where Q20 is high compared to most of the data at same phase.

We see (right column) that sky brightness and Q20 are related – for large values of both, but otherwise Q20 is a constant.

In fitting images to models – discussed abundantly elsewhere in this blog – we realize the need to concentrate mainly on images with as little halo as possible, so we wish to eliminate the images with a large Q20 value. Where shall we set the cutoff?

On the plots the dashed line shows 1.3 times the median Q20. We (arbitrarily) choose to get rid of all images with Q20 above this line.

We also note that Q20 is largest for VE2 and smallest for B – light is scattered further the longer the wavelength is. Does this tell us anything about the scattering mechanism? I’d venture that if the scattering was basedon the Rayleigh mechanism we should see more scattering in th eblue – wonder how Mie scattering goes? At least some of the scattering is in the optics and is fixed and unrelated to the atmsopheric effects – perhaps that can be confirmed by considering some optics?

What of the sky brightness?

We see that VE2 and V have higher sky brightness values than the other filters – VE2 so more than V. Most B, VE1 and IRCUT images have sky brightness below 1 (count), while VE2 has mos between 1 and 10. V has most between 0.,1 and 1 butr a fair fraction above 1.

For VE2 we spculate that the sky brightness is realted to long-wavelength flux from the sky – this could be from water vapour – so we suggest that VE2 sky brightness is compared to MLO weather data for humidity. Since the H2O may be high up in the atmsophere and not a ground level where the hygrometer is, this may not be easy. Satellite imagery for the nights in question could be brought to play on this issue. [Student Project!].

We are uncertain why V should have sky brightness issue if nearby B and VE1/IRCUT do not.

We arbitrarily suggest to also eliminate all images with sky brightness above 2 counts (10 for VE2).

Removing such images from ‘Chris list of best images’ we are left with 494 in number, and the list itself is available on request.



Weather conditions

Met sensor Posted on May 16, 2014 09:50

At the MLO, weather is monitored by automatic instruments. The data are available here http://www.esrl.noaa.gov/gmd/dv/data/index.php?site=mlo.

For our observing times we extract all available met data and inspect them:

Left: (top) Vertical temperature gradient (T2m – Ttop) in degrees C against fraction of Julian Day (fJD). fJD=0 at midnight, so each night’s observing starts at 0.8 or so and proceeds to the right, wrapping around at midnight. T2m and Ttop are temperatures measured on the MLO met tower at 2 m above ground level and at tower top (much more than 10 m up!). (middle) Relative Humidity in % against fJD. Ignore bottom panel.
Right: (top) Histogram of vertical temperature gradient for all observing times (black) and for every hour of 2012 (red). (bottom) Histogram for RH.

We note that the vertical temperature gradient for our observing times almost always are negative – that it is warmer higher up than near the ground – this implies vertical stability. The opposite condition certainly occurs during 2012 but does not correspond to observing times – probably because conditions were bad (convectively unstable or cloudy?). We also see that the gradient becomes less negative through the night implying cooling and equilibration of temperatures. Changes in RH are small. RH during observations show a skewed distribution relative to what was available – so something about observing picks out not the driest nights.

We shall see if the few nights with large RH (say, above 40%) tend to correspond to particularly bad conditions, by inspecting observed images and checking for ‘drifting wiggles’ and plain old ‘large halo’.



Observed minus Model

Showcase images and animations Posted on Apr 25, 2014 20:26


An example of what our fit residuals look like. Note the structure around the BS (right, on the lunar disc). Also the straight-edge structure on the BS sky … do we have internal reflections going on here?

Structure on the DS is in the tenths of counts range.

Here is the same image, but now in % of the local value … don’t scream! We have +/- 50% errors on the BS – but less elsewhere …



101 albedo determinations

Post-Obs scattered-light rem. Posted on Apr 24, 2014 09:43

If you fit each image in a 100-image stack with our model, and plot the histogram of albedos, you get this:

The dashed line shows the fitted V-band albedo of the 101st image, which is the average image made up of the aligned and averaged 100 images in the stack.

In fitting the above 100 images we found residuals that were ‘wavy’ – probably the waves of ‘something’ that is seen in the movie here. The residuals look like this:

If you fit the 100-image average image, you get this:


The residuals are much smaller – but also ‘wiggly’. The RMSE of the 100-image average fit is about 3.7 times smaller than the single-image fit’s. Can this, and the width of the histogram (1.8%), be used to form a scaling argument so that the uncertainty on the 100-image fit can be deduced?

I’d like to think that

error_fit_100_average = SD_histogram*(RMSE_100/RMSE_1)

which would give us an uncertainty on the fit of the 100-image average of 0.5%whichis quite a lot. In contrast, the standard deviation of the mean, SD_m=SD/sqrt(N-1), is 1/10th of the 1.8%, or 0.18%. This is more in keeping with the experience we have had with fitting ideal images with added noise.

The advantage of the fit of the 100 individual images is that you get the histogram and thus a measure of the actual uncertainty – when you fit the 100-average image you just get the fit – formal uncertainty estimates depend on knowing (independently) the noise on each pixel. One way around is to consider Monte Carlo Markov Chain fitting using the Metropolis-Hastings algorithm – this iterates the fit in a Bayesian framework and generates histograms of all fitted parameters, but is very very slow because MANY iterations are needed. Another way would be to bootstrap the single images, somehow.

—–

Here are the similar histograms for the same night from V, B and VE2. Again, the dashed line is the albedo found in the 100-image average image, while the histogram is made up of 100 single-image determinations:

We now have 3 stacks for which the mean of the histogram and the albedo of the mean image are very close – this encourages us to think that the uncertainty of the determination based on the mean image is similar to the standard deviation of the mean for the histogram. Let us test this by fitting even more stacks and their component images along with the mean image. It would be a way towards using the mean-image determinations as the ‘real thing’ and designate the variations we see from stack to stack as geophysical and not based in analysis method bias.



Drifting … clear sky

Showcase images and animations Posted on Apr 23, 2014 09:26

Here is a movie of what 100 images in a stack, on a VERY CLEAR NIGHT, looks like if you look at frame-to-frame changes, and histogram equalize the images:

//www.youtube.com/embed/g2uyeH5X96I
If you monitor 4 patches of 20×20 pixels each (one on the dark side of the Moon, one on the dark side sky, one on the bright side of the Moon and one on the bright side sky), you get these mean series:


There is some food for thought here. Consider the DS patch – it varies by tenths of counts (this is mean over a 20×20 patch) – the mean of that patch is typically several counts – so we are looking at several percent to ten percent changes with time here. Of course, we do not fit individual images, but only the stack average.

Notice how BS sky patch is dropping in intensity in this sequence – this is consistent with BS patch getting brighter – less light is being scattered to sky here.

If the sequence of images is scaled to the total flux of one of the images in the sequence we get a small drop in BS patch variability – it goes from 0.22% to 0.12%. SO either the shutter is variable and causing this, or the total amount of light in the 1×1 degree frame is variable, for other resons: scattering or extinction may remove light from the beam entering the telescope. But certainly, the light in the small 20×20 BS patch varies on its own even in flux-normalized images, so non-shutter variability is present.



Non-Poissonianity

Real World Problems Posted on Apr 21, 2014 09:58

If we take a stack of 100 images, calculate the total counts in each frame and tabulate this, and then take the list of totals and compare the mean of the list to its variance we find that variance is MUCH more than the mean. This should not be the case if the totals were Poisson-distributed.

We find that variance is 100s to 1000s times bigger than the mean of the list.

This could possibly be due to shutter variations – the shutter is … less than fantastic … after all.

Let us inspect the problem and see if it was less at the beginning of operations at MLO. Perhaps wear made it worse as time went on? We did collect 50000 images or so.

Perhaps atmospheric transmission changes on short time scales is this big? Each of the 100 images in a stack took about a second to capture. Does sky transparency change by a lot over that timescale, on a 1×1 degree field?



Experiments in fitting

Post-Obs scattered-light rem. Posted on Apr 20, 2014 09:00

In this entry below we suggested 4 things to try to improve the accuracy and precision of our fitting method. A fifth idea is that byt fitting small, sharp-edged regions on the DS we are allowing small shifts in the image to ‘pull’ thge fit. A similar comment applies to the effects of atmsopheric turbulence. We have elsewhere shown that the lunar disc ‘wiggles’ when you look at an animation of 100 images. Some regions (5-10 arcminutes in diameter) shift sideways by what looks like a large fraction of a pixel, relative to other parts. If we use small patches to fit on quality will suffer if we fit across a wiggly patch’s edges.

Using larger patches might therefore be an idea, and we test that here, while keeping all other thinsg fixed. That is, we use the exact same pixels as before but instead of concatenating 11 row-averaged bands we average the 11 bands into one profile. Again, we use the set of 10 ideal images to which realistic noise had been added.

We get:

mean albedo: 0.28031400
std. dev. of 10 fits:0.00021
SD expressed as percentage of mean: 0.08 %

Compare this to what we found before, using concatenated strips:

mean albedo: 0.28051
std. dev. of 10 fits: 0.00030
SD expressed as percentage of mean: 0.11%

It would seem we have about 30% less spread in the values found. As before the known fixed albedo of this ideal image was near 0.2808, so we have moved slightly away from this, and the bias is now about 2 SDs.

The use of averaging the 11 bands into one (they follow the contour of the lunar disc) does seems to ensure that flux is not lost outside a narrow band. We still use radial bins (5 pixels wide) so flux can be moved from bin to bin that way. We hesitate to make larger bins, as the degrees of freedom (surely?) would suffer. We have about 28 radial bins.

So, we have less scatter but more bias! Not sure this is a way it’s worth to go. Alsao recall that thgese are ideal images – we are not sure what happens with real images. One day we might fit all images using the concatenated and averaged versions of the fitting method and see which has the smaller scatter.

We jave more suggestions from last blog entry to try:

1) Use more sky pixels
2) Use more DS pixels
3) Use BS edge
4) Average several images

4. We can always do and is independent of other method choices. 3 is scary, so I will try 1 and 2 next.



Two experiments

Post-Obs scattered-light rem. Posted on Apr 18, 2014 15:41

We conduct two experiments, to asses the abilities of the fit-a-model-image method.

Experiment 1: Same image, with differing noise.
=============================

We pick an ideal image, generated from JD2455945, and to this one image we apply a PSF convolution, and 10 times generate images with varying Poisson noise. We then fit these 10 images, and look at the distribution of results.

We find

mean albedo: 0.28051
std. dev. of 10 fits: 0.00030
SD expressed as percentage of mean: 0.11%

The actual value for albedo used to generate the ideal image was 0.28083594, which is 1 SD above the mean value found.

So, we can (again) confirm that in principle, we can achieve fits with small bias and small errors, as per our original science goal.

Experiment 2: Same image with noise, different fit starting guesses.
=========================================

We pick now one of the noisy artificial images and refit it 10 times – each tiome starting at a random guess near the supposed best guess. We pick our vatiation froma normal distribution with amplitude 0.03 and multiply the fixed starting guesses by 1+0.03*N(0,1) for the 8 starting parameters (there are 8 – 7 before – we added shift om image along y-axis also). These guesses were then allowed to converge and we collected the results. We found:

mean albedo=0.280183
S.D.=1.4181365e-05
SD in % of mean=0.0051

We see that the sactter is very small, i.e. that convergence is almost always to the same value. However, there is a bias in that the known albedo (see above) was not found.

The bias is (0.280183-0.28083594)=0.0006529 which is 0.23% of the correct value.

From this we lkearn that
a) convergence may not always be dependent on starting point – the same best solution will be arrived at, and that
b) the noise added to the image itself will have enough influence, even if small (i.e. realistic for averaging 100 images in a stack, and coadding 20 ‘lines’ in each of 11 ‘bands’) to cause a bias in this perfect-case example, where noise is only Poissonian.

The bias is a mere 0.2% of the correct value which we could lkive with given our science goal of 0.1, I guess, but reality adds more problems to our images than just Poisson noise. These effects appear to give us a typical spread of 1% – about 10 times what we saw in experiment 1, above.

As it is, we are using 1/10 of the pixels in the image for our fits so we have at most a factor of 10 more pixels to work on. This could in principle give sus a factor f 3 reduction in the uncertainty, if all pixels are equally effective at driving the solution. If all went well, our 1% errors coul dbecome 0.3% errors which woul dbe acceptable (just), in the absence of a bias.

What can we do along this line?

1) Many of the DS sky pixels are not being put to use – they coul dhelp set the sky-level of the fitted model better.
2) Not all DS disc pixels are being put to use, but there is a limit how far onto the disc we can go before BS halo starts being a too importnat factor, but perhaps we could double the number of pixels.
3) We only use the BS disc and BS sky to calculate the total flux which is used to ensure same flux in model and observation. We might be able to also fit some of the edge near the BS or just try to match the level of the BS sky near the disc – but the levels are much higher and our cirrent use of errors based on differences between absolute levels would become dominated by the BS pixels in driving the fit.
4) We can use more images from the same night – at least inside some period of time where the geometry of Earth and Moon does not change significantly – perhaps half an hour or so. In that time some 10-30 images could be gathered in reality. If several filters are wanted then perhaps 10 images in each filter (in practise it seems we could reach that on some nights, inside a half hour) and then repeat for other filters.

In summary, if erros were due to Poisson noise onkly, we could hope to double the number of DS disc pixels, use a lot more of the DS sky pixels, and perhaps work towards 10 images in each filter inside each half hour, giving us, potentially, the required improvements in uncertainty.

We can work on this!

There is a “however”, however: All the noise is not Poissonian – there are issues having to do with:

a) Bias subtraction – and airglow – and zodical lights: letting the ‘pedestal’ solve for these may be a bad idea – perhaps we should fix the pedestal at predicted levels in each image? This needs some wor, since we are sensitive to errors there too!
b) The atmospheric turbulence causes ‘wiggle’ in the images,. described on this blog, and this is a factor in driving the error in the fits.
c) The model we fit is not faultless either – the BDRF is a simple one and we see hints of some azimuthal-angle problem ain fitting along the edge of the disc. Model couild be improved.
d) Other parameters in the model might be fixed too – such as the ‘core factor’. Experiments are still needed to see if this is a good idea.



Status of model-fitting

Post-Obs scattered-light rem. Posted on Apr 17, 2014 11:12

After exhaustive (-ing?) tests of fitting the same image over and over again under varying conditions, we now turn to fitting all images in two colours from one night. We take B and V images from JD2456045.

We have before fitted ‘lines’ going from DS sky onto the DS disc itself, and then moved on to another position along the DS edge. Thsi helped us see that fits along the edge could vary systematically, and we have speculated that it was due to the BRDF-model used had some azimuthal angle-dependence that was different from reality.

Instead of getting several fits along the edge and combining these, we now choose to combine lines and make one fit, thus allowing the errors to balance out somewhat.

A typical fit, for 11 lines, looks like this:

The upper panel shows 11 concatenated ‘lines’ going from 50 pixels off the DS edge to 70 pixels onto the disc. The white curve is observations and the red is the model. The lower panel shows the difference between the two at ‘best fit’.

We have 8 parameters in the model that together describe such things as the terrestrial albedo (the main goal of the exercise) and parameters that adjust the PSF, as well as the relative alignment of model and observed image.

In the above, the residuals are small, buts how systematic behaviour near the ends – going along the sequence of the zig-zags we are essentially sweeping along the edge of the DS – so something is not well at extremes of the sequence, but near the middle the residuals are very small.

We smooth the concatenated lines (and corresponding model line) before fitting – this helps reduce a problem we had before, namely, that repeated fits of the same image starting at slightly diffeent (1% different) starting guesses could finish up at slightly different ‘best fits’.

For all images in B and V for one night we can compatre the various quantities determined in the fitting:

From top to bottom, left to right: Albedos in B and V images against time of day (not really Julian Day, but fraction of day since that JD started); alfa of PSF wings against time; PSF core radius (here held fixed at 3 pixels); pedestal against time; image shifts in X and Y; lunar albedo contrast factor; PSF core factor; and RMSE against time. Black crosses are B-band, while green crosses are for the V-band.

Here we see a clear difference in B and V albedo, with some scatter. The PSF is behaving diffrently in B and V (core factor, pedestal and wing alfa). Lunar albedo amp contrast factor is unaffected by B and V. RMSE is bigger for B than for V. Image shifts are near 1 pixel in each idrection.

The above choice of a fixed core radius was prompted by noting that leaving it free to vary inside the range 2-10 would cause the fit to pin the core radius to the lower limit in V and the uppe rlimit in B. We argued that some of the factors core radius/core factor/ andpedestal were degenerate and chose to fix the core radius at a compromise value. We note the extreme values of the core factor parameter for B, compared to V. In essence, a large core factor implies that most of the flux is in the core of the PSF and little is in the wings. It is hard to say if this is what we expect for B vs V – the colour-dependence of light-scattering in optics (and atmosphere) is not understood by us. Yet.

The observations in B and V were not obtained at the same times (because we use filter-wheels) so it is not easy to combine the B and V albedo determinations. Some of the scatter seems to be correlated (first points in left top panel above), but the rest do not. The means and standard deviations and standard deviations of the means, and the SD in percent of the mean, for B and V are:

B: 0.3608 0.005 0.0012 1.3%
V: 0.2902 0.002 0.0006 0.8%
mean SD SD_m SD/mean*100

So we have scatter on albedo determinations of near 1%, like BBSO. Formal errors on the determination of albedo for a single image depend on being able to estimate the uncertainty of the data. As we clearly have some systematics going on (plot at top of this entry) this is not possible.

For the night JD 2455945 (the subject of our paper) we have fewer good images, but get is:

B: 0.37007500 0.00020615528 0.00011902381 0.055706352
V: 0.28093333 0.00023094011 0.00016329932 0.082204595
mean SD SD_m SD/mean*100

The B and V here are ALBEDOS, not magnitudes. Converted to magnitudes the Balbed/Valbedo ratios on the two nights have Johnson B-V colours that differ from one night to the other by 0.06 mags.

Thus we are told that one night’s earthlight was 0.06 mags bluer than the other. This is several times our estimated observing precision.

Is that a lot or a little? Our EGU 2014 poster tells us which …



Error Budget revisited

Post-Obs scattered-light rem. Posted on Apr 12, 2014 17:03

We have been looking at the ‘pedestal’ which is a part of the model we fit to our observations. With a black sky and perfect bias and dark-current removal, the pedestal should be 0, and the fitted model should have a halo that sloped off from the Moon and perfectly explained all brightness seen on the sky.

In reality, there is background galactic light, zodiacal light, airglow and any terrestrial light (Moonshine reflected from the ground!!), to allow for. To this end we have operated with this pedestal in the model, which was a vertical offset of the model. In looking at fits we notice that the pedestal is degenerate with the halo-parameters in such a way that they tend to lie along a line when a lot of fits are compared, each started at slightly different points – even for the same image repeatedly fit. The use of this pedestal therefore leeds to a bias in the determined albedo. Which is not good.

We therefore have tried to evaluate expressions for the additional sky brightness so that the pedestal could be fixed at an assigned, physics-based, value and not interfere with the fit itself.

In doing that we had reason to revisit and elaborate on Henriette’s error budget analysis. In particular, we performed the analysis allowing for error terms in DS and BS fluxes, dark currents, and exposure times. We introduced the uncertainty on the superbias factor and see now that it is the dominating factor when area-means are considered.

For area-means of 20×20 points, the error due to the superbias factor is as large as the error due to Poisson statistics on the DS. For smaller areas than 20×20 the DS Poisson error dominates.

Since levels of sky brightness are of the same order as the contribution due to factor uncertainty, or larger, we see that we have a problem.

We could try to estimate sky brightness better, but as we realize that fitted albedo varies, for a single image treated in different data-reduction ways (bias removal, and stacking), we need to also re-consider the use of ‘anchoring’ of the model halo in the DS sky part of the image. This is what BBSO achieve by fitting a line to the DS sky and subtracting it from the image.

Improvements in estimating sky brightness could also be pursued, but all constant contributions will disappear in ‘anchored’ procedures. The point is to not let the fit be biased by such terms.



More statistics about fitting methods

Post-Obs scattered-light rem. Posted on Apr 06, 2014 12:49

We continue the investigation into how well albedo can be determined by looking at justa single image and applying various methods to it.

We compare the ‘old’ method to a new method. The old one consists of plotting row-averaged slices that start on the DS sky and proceed onto the DS disc. We place the slice at 9 different locations up and down the edge of the lunar disc. The ‘new’ method consists of averaging the whole image (and fitted model) into one row by averaging along columns. We do two runs of the old method – varying the distance we go onto the DS, and one run of the new.

Black crosses indicate the albedo found by fitting along 9 slices ono the DS disc going from 50 columns away from the edge to 100 columns beyond that. Red crosses indicate the same but going to 50 columns onto the disc. Green crosses indicate results found when column averaging the whole image but starting from 9 different starting guesses of the fitting parameters. The dashed and dotted lines give means and standard deviation of the mean for all trials.

We see that the three methods give a spread of values, with few repetitions. The standard deviation of the Mean is about 0.25% of the mean, but the standard deviation of the distribution is about 1% – single determinations will therefore vary a lot, while the mean over many trials can give a mean with small error.

We notice that the green crosses – which are results of identical trials in the ‘collapse the whole image into one row’ method but starting from slightly different starting guesses – all lie below the mean and have some scatter – so the new method is biased wrt the old one and the methdo itself is finding ‘best fits’ that depends on starting conditions. That the convergence is not to the same number each time show that the LMFIT method used reaches a stopping criterion. We have chosen max 500 iterations.

Neither of the above findings are reassuring – fitting method choices determine the outcome. Possibly, averaging over a large number ffo different methods would give us a result that is stable and near the truth.

In summary, we are looking at method-dependent biases of half and whole percentages, and errors of means near 0.2%.

It is not clear if the bias comes from lunar albedo-map and reflectance-model choices, or are stochastic in nature. THe effect of stochasticity is seen from the green crosses where all things are fixed, except the starting guess for the iterated model-fit.



Fit variability from a single image

Post-Obs scattered-light rem. Posted on Apr 03, 2014 08:18

We explore how well albedo can be determined from just a single image, varying the fit approaches.

A single 100-image stack of the Moon was selected, aligned (or not, see below), bias-reduced and averaged and then subjected to our model fit. The fit selects 9 21-row wide ‘strips’ cutting across the DS edge of the disc so that we have 50 columns of sky and then 100 columns of disc to fit on. The 21 rows are averaged in observation as well as model.

Applying this method we get 9 estimates of the terrestrial albedo. We selected three methods for centering stack-images:

1) Align the images in the stack once allowing for sub-pixel shifts, then average
2) Align iteratively, refining the alignment wrt an updated reference image, then average
3) Do not align at all, just average

In this figure we show the albedo determined from the three sets:
The determinations are shown in sequence, separated by vertical dot-dashed lines – method 1,3 and 2 (in that order). The overall mean and standard deviation is shown

We see systematic deviations: all Method 1 values are below the mean, all method 3 values are above the mean and method 2 is a bit above and a bit below.

The mean of these values is 0.3047 with a standard deviation of the mean of 0.0014 (0.45% of the mean), which is not too bad! However, we could clearly do better if the method-dependency was not present. How to remove this?

Methods that shift images by non-integer-pixel amounts risk being non-conservative – i.e. the total and area-fluxes are not conserved. We have elsewhere in this blog estimated how bad the problem can be [see http://iloapp.thejll.com/blog/earthshine?Home&post=369 also see http://iloapp.thejll.com/blog/earthshine?Home&post=299]. We found that it was slight, means can only change by much less than 0.1% during a sub-pixel shift of the image.

Not aligning images at all risks comparing ‘smeared’ observations with ‘knife-sharp model images. The smearing occurs due to image-wander during exposure and can be as much as a pixel or two.

Revision of our alignment methods seem in order, given the above. Possibly shifts by whole pixels only? This is at least conservative away from edges.

Note that method 3 (the middle set) has the least internal scatter. For this set the standard deviation of the mean is just 0.2% of the mean. That is pretty strange, actually, since this is the ‘do not shift images before coadding’ method. Hmm. Then again, perhaps it is not strange, and we are simply seeing that the likely most fuzzy image has the most stable fit – as the strips select different parts of the light-and-dark lunar surface to fit on, contrast causes the fit to wander if the image is sharp. But on the third hand, our method now allows for ‘contrast scaling’. Hmmm, mark 2.

Shifting and co-adding involves two issues – the interpolation required at a given non-integer pixel shift may not be flux-conservative (our tests seem to show this is only a small problem). The shift itself may be poorly estimated. Our shifting method is based on correlation – it does not look specifically at such things as ‘are the edges still sharp’ – i.e. Sally Langfords Laplacian edge-detection method. The use of edge-information only is not clearly the best method since it is based on very few image pixels and thus suffers from noise. Perhaps a hybrid method can be envisaged – “correlations and maintaining-sharp-edges”?



Fit improvements

Post-Obs scattered-light rem. Posted on Apr 01, 2014 11:34

Since the start of this project we have worked on a method that fits suitably-convolved amd scaled theoretical images of the Moon to real observed images of same. Below we show that the method has improved, by fitting a profile from the same night with two methods from 2013 and now.

This is a pdf file showing the three plots:

As you can see, the present method leaves residuals that are structure-free – this is not the case with the older methods.

We are therefore approaching a situation where all data can be fit pretty much perfectly by a multi-parameter model.

The model uses these fitting parameters: a vertical offset representing sky brightness, a shift of the image along rows, a parameter for the width of the PSF core, a parameter that vertically scales the core inside that width, a parameter that sets the slope of the wings, a factor that scales the lunar alnbedo map contrast, and the terrestrial albedo.

Since the results still show signs of a dependency on just which part of the Moon we are fitting on, we conclude that the BRDF functions we use – Hapke 63 – are not capturing the right azimuthal-angle dependence of the reflectance. This area is now where improvements are to be found.

If we succeed we shall actually be contributing to Hapke-type work, by providing empirically validated BRDF functions.



PSF estimated from Markab compared to Dragonfly paper’s

Exploring the PSF Posted on Mar 15, 2014 09:21

In this post below we compared our canonical PSF (raised to various power) to the PSF shown in Figure 6 of the AvD paper.

We have since re-estimated /below) our PSF from radii near 0.1 arc minutes to 13 arcminutes using excellent data from multiple images of the star MARKAB. This PSF is empirical for one specific night and does not need the raising-to-a-power that the canonical one does. We plot it as is on the AvD figur, trying now to re-create the twopanels of that Figure:

Right panel shows our PSF as surface brightness in magnitudes per area.

I’d say our empirical PSF (green line) looks like it is realistically modelled by the canonical one raised to 1.8 and still beats the AvD PSF (the crosses).

We extended our PSF beyond 13 arcminutes by a third order power-law.

We have assumed a few things: AvD speak of normalized flux – we take this to imply that the PSF is divided by its VOLUME, not its area. We assume that calculating the surface brightness involves taking the total flux inside some distance from the peak and dividing it by the area.



Our own PSF re-fitted

Exploring the PSF Posted on Mar 14, 2014 17:49

We have some excellent data from observing the star Markab, in Pegasus. We have 334 images at 4 seconds exposure, and each is bordered by dark frames of the same expsoure time – thus we can subtract bias and any dark current easily.

We did this, and aligned and coadded the many images. Then we inspect the PSF by cosnidering the intensity around teh star in concentric rings:

We see that the core of the PSF is about 2.5-3 orders of magnitude above the point where the wings start. We see the wings drop off and join the sky level. The red crosses are median bin values. The blue curve is a polynomial fit, the coefficient of which are given in the table. The polynomial is found for a fit to 1/radius against PSF for the wings, and a Voigt function for the core out to about 1 arc minute.

Some points: The curve is not the PSF – because there are remnants of sky brightness – about 0.88 counts [star scaled to 50000 counts at the peak]. The PSF would be well approximated by the above polynomial without offset (first coefficient).
The part of the PSF from 1-3 arc minutes is roughly linear, as if a simple 1/r ^alfa power-law applies. This is not the case when we consider the above.

We might revisit the extended halo we see in lunar images with the above insight into the effect of a sky level.



Dragonfly-eye paper

Exploring the PSF Posted on Mar 14, 2014 10:37

An interesting paper has appeared on Arxiv:

pdf

This paper discusses the use of novel lens coatings in commercial grade lenses (Canon) and DSLR cameras to build a multi-lens, co-additive system. They discuss the PSF they have in their Figures 6 and 8 (8 is about ghosts).

We can compare the PSF we believe we have to theirs:

Here we have used the same axis-scaling as Figure 6 of AvD. The three curves are our normalized PSF profiles for three settings of the ‘power’. These are 1.8, 1.6, and 1.4 – the largest at the bottom. Remember that the ‘canonical PSF’ we use already has a power slope of close to 1.7 so with the extra powers above we reach 3.06, 2.72 and 2.38, respectively. We know that the power of 3 is the limit for the wings of a diffraction profile, so use of values of ‘power’ near 1.8 implies we think we are diffraction limited – in the wings. Actual fits to observed images tends to give powers nearer to 1.7 or a total power-law exponent near 2.9. The points on the plot above are taken (by eye) from Fig 6 of AvD.

They seem to have not only a broader PSF than us, but the ‘wings’ appear to be linear in lin-log: that is, their wide halo is not a power law as we infer, but is rather an exponential.

On the face of it, our optics are ‘better’ than theirs, but hold on:

1) we do not actually measure all of our PSF – we infer that a power-law tail is in order; we only have actual profile measurements from point sources such as stars and Jupiter (almost a point source) inside several arc minutes. The rest of the wings are inferred from how the halo around the Moon looks at distance.

2) In their Figure 8 an image of Venus and a star is seen – and a ghost of Venus. The authors state that “the ghost contains only 0.025% of Venus’ light”. I think that number is so small that our requirements of 0.1% accuracies in photometry would be met. Note that ghost and the PSF are unrelated – the PSF wings describe light scattered while ghosts are reflections off optical element surfaces. The paper says that it looks like not all lens elements were coated with the novel coating.

What do we get from this?

I would like to investigate the PSF we have some more – this can be done with repeated images of a point light source in the lab: this would clarify the non-atmospheric part of the PSF. Despite current problems with FWs we may be able to pull this off with what we have.

I would like to understand where the (inevitable) ghosts are in our system – did Ahmed park them on the Moon itself? If they are same size as the Moon and well-centered they may not cause any damage since they merely replicate the image information (if same size and in focus).

It seems the AvD system has very weak ghosts but larger scattering than us (if we understood our PSF wings correctly) – if the weak ghosts are due to these new lens coatings then they are of interest to us in possible future designs of the system. If the scattering really is as high as it looks, compared to our (guesstimated PSF wings) then we have a better system than them, period. Mette commented that these coatings can cause scattering – wonder if the AvD people chose the coatings to lower ghost intensities or to get rid of scattered light? They do not say so directly, but do compare to large telescope PSFs (their Fig 6), and find less scattered light in their own system.

We need to understand how they measure their PSF into the wings.



JD2456717

Observing log Posted on Feb 28, 2014 14:07

Tested a script while looking through the telescope. Script was set up to switch filters and take sequences of images.

Everything seems to be badly scrambled! Colours of filters do not match what is requested, and the sequences of images received are incorrect – for instance, when asking for 10 images at 2 seconds each I get about 3 images at 2 seconds and the rest are rushed through rapidly.

Shutter opens at strange times after a sequence.

It seems like the system is badly shaken up now. I think some of the above behaviour may explain some of the odd image-sequences we had at MLO, but the majority from MLO were good: now it all seems random.

My guess is that the LabView software has been jumbled somehow – it is easily done since the coding is not deterministic as text code is – it is all little coloured lines and boxes. It is easy to pick something up and drop it down in the wrong place. A test and a fix for this might be to restore an older version of the software – for instance something after Ingemar fixed the bad coding problems we found while at MLO and before the system was hit by lightning.

Another possibility is that hardware has been damaged in such a way as to scramble/ignore software instructions. Not sure how to test for this – we used a ‘sniffer cable’ to listen to the erroneous traffic to the mount, but I am not sure what to listen in on here?

Wonder if there is a way to ‘reset the hardware’? Perhaps the FW are ‘off by one’ – some latch is not clicking home?

—————

Inside the ‘breakout box’ there are lights to watch as a script runs. There is a light on a ‘hager’ relay and it evidently lights up when the two Thorlabs filter-wheel controllers are being adressed. They have displays that show numbers and I am guessing they show the position of the FW.

While running a sequence through the 5 filters I saw the numbers change. I think I caught a rythm corresponding to several passes through the 5 filters, but I swear there was an irregularity [hard to prove afterwards as there is lots of uncomfortable crouching and waiting to do]. Also strange is that the FW displays show two sets of numbers first, these then change to another set, then the hager light goes out and you can hear the shutter start working [its progress is uneven, however, despite being asked to take 10 identical images]. Here is a sequence:

FW2:5->5,6->1,2->3,4->2,3->4,
FW1:2->1,2->1,2->1,2->1,2->1,

where “A->B” means that the FW display first showed an A which then changed to a B, after 5 seconds or so. Notice how FW1 always goes “2->1”. FW2 seems to be cycling all numbers. Notice how the start position in one pair is always 1 higher than the end position in the previous set. There is a cryptic sentence deep inside Dave’s LabView code about how Thorlabs uses one set of indices and his code another – this could be it, I guess.

I think FW1 is the ND filter and is always set the same way – in the above test I was asking for the ‘AIR’ position and I think this is what we see. I will now run a test where I ask for one of the ND filters instead and we will see if the numbers change!

—————

OK, I did that, and it seems I am right. FW1 is the ND filter. I set it to ND2.0 and the sequence 2->1 above changed to 5->6.

Should run a sequence now of just changing the ND filters.

——————

Ok, did that too. This time I used a script that always set the ‘B’ filter, but cycled through the available ND grades – that is AIR, ND0.9, ND1.0, ND1.3, ND2.0. The typical sequence of readouts from the Thorlabs controllers was:

FW2:3->2,3->2,3->2,3->2,3->2, …
FW1:6->1,2->1,2->3,4->4,5->5, …

we see how the colour filterwheel (FW2) always sets the same filter and always seems to start at the one numbered one higher. Visual inspection during the cycling revealed that the colour of the light switched between the colour of the lamp and Green – i.e. the FW2 settings actually acquired were ‘Air’ and ‘V’ – not ‘B’ as I asked for.

FW1 meanwhile seems to have cycled irregularly – 1,1,3,4,5, … (2 is missing). It was not really easy to see whether various ND grades were inserted or not, but clearly AIR was one of them since I could see colours of the light.

Note that in a sequence “A->B” a sound – such as of a filter-wheel changing position – is heard before A, and before B.

Not sure what to make of this. It could be a signals problem where feedback from the actual position of the wheel is missed or missunderstood by the software so that new or wrong commands are sent.

Is the software that directs the FW giving absolute commands or relative ones, as in
“Go to the blue filter”
vs
“go to the filter after the one you are at, which I assume is the IRCUT filter, so that you will arrive at the B filter since it comes after IRCUT”?

Why does the wheel change position twice during a ‘set the FW-wheel’ command? Makes sense only if it always went to some home position, and then proceeded from there – but clearly it is not starting from the same position each time, given the readout above. Again, could it be that the intent is to send it to home each time but this goes wrong and it doesn’t actually arrive at home, but assumes it did for the next relative command?

NEXT time, we need ABSOLUTE ENCODERS. I want a little man inside the telescope with a cell phone calling us with the ACTUAL position of each FW!

 



JD2456715

Observing log Posted on Feb 26, 2014 14:02

Manually testing the shutters and FWs.

We disconnected the CCD camera so we could look through the telescope along the optical axis, at a lamp. We ran the camera, so that the internal shutter opened and closed regularly, and then manually changed the color and ND filters.

Strange behaviour was observed: Setting the color of the filter was not a simple affair – the colour would be a bit random, and the ND filter would reset – apparently by itself – after a few seconds.

We will try to run scripts next, to see if the command of the devices from the scripts are more consequential and whether the ND resets in the middle of a sequence. Our experience from masses of observations from MLO (before the lightning struck) is that stacks of images are all OK as long as the first image is all right.



What colour is the Earth, then?

Real World Problems Posted on Feb 13, 2014 19:16

An artist, Jeremy Sharma, in Singapore, has asked us what Pantone colour code the earthshine colour corresponds to. He heard of us through the Guardian story.

I can see two ways to answer him: One is via a conversion from colour temperature to Pantone code (or, equivalently, hexadecimal RGB code), another is to take a realistic image of Earth and averaging the RG and B channels.

I cannot, yet, find a conversion from colour temperature to Pantone code or RGB intensities, but the temperature to use is given by B-V=0.44 which is near 6400K, I think.

For these images of Earth:

http://news.bbc.co.uk/2/hi/8547114.stm

I can take the average of the R G and B channels of both images – omitting the black sky around earth.

the RGB averages are:

89.5127 92.4462 115.971

or, rounded to nearest integer

90 92 116

The hexadecimal equivalent of this triplet is #595C73
(use IDL ‘Z’ format to work that out …)

On this page

http://goffgrafix.com/pantone-rgb-100.php

(and links thereon) it is possible to find Pantone colour codes and the equivalent hexadecimal code. The above value does not appear on these pages – but a tool at

http://goffgrafix.com/colortester.php

allows you to type in the above hexadecimal number and see the corresponding colour as shown on a computer terminal.

The above colour can be ‘brightened’ or ‘darkened’ by multiplying each of the RG and B values by some number (same for all three) and then converting to hex code. In doing this avoid saturating any of the three (they must be smaller than 255).

I did this so that a lighter rendition of the same colour would appear, and the result is

#BAC0F0

which you can also type into that page and see (they allow showing of two such colours next to each other).

Real earthshine is thousands of times fainter than moonshine so I doubt you can ‘see’ the colour if you make the numbers realistic (in fact, RG and B would be 0 in such an image since the numbers are integers and cannot be smaller than 1 without being 0). The two above colours have the same tones but differ in brightness.

I expect that a dye maker or a paint shop with a smart machine can mix your colour according to the hexadecimal code, instead of the Pantone colour code.



Albedo 2 ways

From flux to Albedo Posted on Feb 12, 2014 09:34

For our ‘splendid night’ JD2455945 we can have two estimates of the albedo of Earth:

One estimate is provided by the ‘profile fitting’ where albedo is directly determined as a model parameter of the Sun-Earth-Moon system, and another estimate is given by the B-V determination we have from the cancelling halos.

We get:

From profile fitting albedoB/albedoV: 1.217 +/- 0.003
From B-V: albedoB/albedoV: 1.202 +/- 0.005

The error on profile fitting results is propagated from the standard deviation of several determinations of albedo in B and V and thus is ‘internal error’.
The error on the second estimate is taken from the paper and is ‘internal’ also – the dependency on error of the solar B-V value drops out.

The two estimates are similar but, given errors, formally different.

The fitting result is based on 6 fits in total – three in the B image and three in the V image.



B-V and cloudiness variations

From flux to Albedo Posted on Feb 09, 2014 20:20

The amount of clouds on Earth changes from day to day, and as clouds reflect sunlight (with B-V=0.64) more clouds cause B-V of the earthlight to drift towards the solar value, while less clouds shows the Earth bluer. How much do we expect the terrestrial B-V colour to change day to day on the basis of changing cloud masses?

Inspection of GOES West full-disc satellite images of Earth shows that the local noon (i.e. disc is fully illuminated) brightness variations due to changing clouds amount to about 3.2% of the mean brightness. As the Earth below is dark (Pacific Ocean) almost all of the brightness is due to clouds so we are not too far off by saying that the amount of clouds varies by 3.2% around its mean value on a daily basis. With Earth on average being almost 67% cloudy we can re-use the Stam models of expected terrestrial spectra to see what the expected changes in B-V is due to cloud variations.

We find that B-V will vary, with a standard deviation near 0.005 around the observed value of B-V=0.44.

This is smaller than the total observational error we have. We have shown that the theoretical Poison-noise limited uncertainty would be 0.005 in B-V, but we cannot observe that well. Yet.

So – we should not expect to see values for B-V very different from 0.44, which helps explains why we get the same value as Franklin’s mean value using just a single (but precise) observation.

We may be able to see 2 and 3 sigma deviations in the cloud cover, however.

The purpose of our telescope was never to observe daily cloud variations – in the long term we hope to be able to qualify that we can set limits to climate-change induced changes in albedo.

Also, the above is just an investigation into whether we can use COLOUR changes to quantify albedo changes – we still have to quantify how well our direct-photometry measurements can see albedo changes. Paper II.



Brighter-Fatter effect

From flux to Albedo Posted on Feb 09, 2014 13:33

In an interesting paper by Antilogus, et al subtle effects of electron interactions inside the CCD material itself are discussed, which could be a tool for understanding and perhaps quantifying CCD nonlinearity.

We have tested our own Andor BU987 CCD camera for these effects and see them.

Plots of row- or column-neighbour means vs correlations. Top row: first panel: full set of values of means vs column-correlations
– we note the strong non-linearities that set in at about 50.000, and
some outliers.

Top row: second panel: for just the range of mean values below the onset
of non-linearity a robust regression is performed (red line).

Bottom row: same as top row but now for row-correlations.

We note the general correspondence to results in the ‘brighter-fatter’
paper: rows are more strongly correlated with their neighbours than are
columns – the ratio of slopes is about 2.0.

Can this be used quantitatively to correct for non-linearity in our CCD? The camera linearity was already tested for linearity while in Lund. See Figure 8 in the report.



Shutter/FW failure – 2456693.2

Observing log Posted on Feb 04, 2014 16:31

Testing the system on a lamp-illuminated flat field. With a script, sequences of images are taken with commands to change filters. Kinetic sequences.

There is obviously some problem with either the shutter or the setting of the FW, because while some images are well exposed at more or less recognizable exposure times (we use a different lamp from the hohlraum we had on MLO) some are simply almost all black at those same exposure times.

To test if it is the shutter that is failing to open, or the FW that does not set, we must eliminate one thing at the time – first let us run a script with exposure times of varying lengths, and simply listen for how long the shutter is open.

If it seems the shutter is always open as long as we ask it to be, we move on to the FW – perhaps we can open the telescope box and simply look at whether it turns (pant a white spot on the side!).



New superbias

Bias and Flat fields Posted on Feb 03, 2014 12:47

Now that the camera is up and running again, a series of exposures have been taken to test the bias field.

Below is the old and new compared, and the difference in percent:

The bias level has changed by 1.3% or so, and the slope of the new field is different from the old – there is also some, smaller, change in curvature. The change in mean level is not so important since we scale the superbias to the current bias mean value anyway, but the change in slope is important. The relative level from up-slope to down-slope is something like 0.1% which is the adopted science goal for our project.

Further testing of the bias field will be performed as we proceed and if the above pattern is stable we shall simply start using a new superbias for currently acquired science frames. If studies show that the bias goes up and down and changes slope willy-nilly, as it were, then we have more to worry about!



More progress … and yet …

Observing log Posted on Jan 23, 2014 11:24

As stated before, scripts can now be run and images gathered. We have solved a minor problem with disc-access that suddenly popped up. Still not sure what this is due to, but can now save images from automatic scripts. This clears the way for a row of lab tests that will start now.



Scripts running

Observing log Posted on Jan 21, 2014 12:40

It was possible to start a simple observing script today: Mount was not attached so nothing moved except shutters and FWs (and the SKE is still not working) – files were obtained but it turned out that something with the setup of permissions on the PXI has changed so images could not be written to the ‘NAS’ drive (a network drive in the instrument rack). Am seeking help from DMI IT department for this.

If that starts working we can run tests on bias fields at least, and later on flat fields (with a screen of some sort in the distance) and then focus tests and all this.



Hoijemakers

Links to sites and software Posted on Jan 21, 2014 08:43

Henriette sent us this:

Jens Hoijemakers in Leiden wrote a thesis about a telescope you could put on the Moon and monitor Earth with – an earthlight telescope!

He has a YouTube video: http://www.youtube.com/watch?v=AcofvAjVT7Y

His thesis is here:

And here is a paper.

Thanks Henriette!



Press mentions

Links to sites and software Posted on Jan 19, 2014 14:01

Our paper, and the story in The Guardian seems to be generating quite a lot of activity on the net. Here is a collection of what I can find:

Original Arxiv.org paper – Jan 9 2014:
http://arxiv.org/abs/1401.1994

From Jan 10 2014:

The Guardian story:
http://www.theguardian.com/science/2014/jan/10/dark-side-moon-turquoise-astronomers

The Economic Times of India – :
http://articles.economictimes.indiatimes.com/2014-01-13/news/46150068_1_dark-side-moon-earth

The Raw Story:
http://www.rawstory.com/rs/2014/01/10/hawaiian-telescope-discovered-the-dark-side-of-the-moon-is-turquoise/

and so on …

http://www.arihantbooks.com/NewsEvents/487/Science-Buzz/Moon-s-dark-side-is-turquoise-in-colour_-claim-experts

CNET: http://news.cnet.com/8301-17852_3-57617095-71/brain-damage-the-dark-side-of-the-moon-is-turquoise/

Several stories captured by a news ‘bot: http://www.newsfiber.com/p/s/h?v=EBXOI5VPyVXw%3D+aivNYXh2b1I%3D

A search on Google gives a lot:
http://goo.gl/NG7049

This search: http://bit.ly/1f2a3fM gave 746 hits on Jan 21 2014.



More on Stam Earth model spectra

From flux to Albedo Posted on Jan 18, 2014 10:33

In this entry we started exploring the Stam 2008 set of model Earth spectra. That entry was based on the ‘land only’ set of Stam models. Ocean-only models are also provided, so we can mix these with the land models to get more realistic Earth spectra. In Stam ocean models the albedo is assumed to be 0 apart from a specular contribution, so all colours are dominated by the Rayleigh scattering and a bit of sunlight reflected at the glint point. In land-only models the vegetation red edge has a large contribution, so we expect some changes when a land-ocean mix is introduced. We assume that Earth is 26 % land surface in what follows.

As before we calculate the spectrum of a range of cloudyness percentages. We find that:

B-V for Stam cloud free forest model is: 0.19
B-V for Stam cloudy forest model is : 0.54

B-V for Stam cloud-free ocean model is : -0.23
B-V for Stam cloudy ocean model is : 0.53

Note how similar the 100% cloud cases are for land-only and ocean-only, as expected. There is a small effect of the surface below the clouds (which is one of the features of the Stam set of models – others have ignored this effect).

Note also how blue a cloud-free ocean only Earth would be – this is, I expect, the B-V of our own blue skies on a cloud-free day, as we have discussed elsewhere. I think we should try again to measure blue-sky B-V with our DSLR calibrations.

As before we estimate slopes of the B-V vs cloud % relationship.

Change in % cloud cover per mmag change in observed B-V: 0.31 %pts/mmag
Change in % cloud cover for total observed error in B-V: +/- 6.3 %pts
Change in % cloud cover for internal observed error in B-V: +/- 1.6 %pts

The total B-V errors we have are +/- 0.02, mainly due to uncertainties in the Solar B-V as pointed out by Chris. If we leave all measurements relative to the fixed (but somewhat unknown solar B-V) we have internal errors of just +/- 0.005 mags and that implies we could determine Earth’s cloud cover to within something like +/- 1.6%-points on the basis of observed colour alone. Not bad!

With a technique for observing B-V on all nights we could thus complement direct albedo-determinations (found with edge-fitting for instance). The colour-method has drawbacks and benefits relative to the direct albedo determination in that a difference is used instead of an absolute fit.

This is a point towards why we need to understand how the halos are formed and just why they cancelled on that night!

It is also fuel for fixing the SKE because with an SKE more images could be obtained where the remnant halos in B and V almost cancelled on the DS.

With the CCD working again, we have some hopes of understanding the SKE problems, as well as performing some very careful lab observations with halo due to optics only.



CCD is back

Observing log Posted on Jan 17, 2014 16:25

We managed to get actual images from the CCD camera today. Hans will tell the story of what was wrong.

So – testing all systems together, next!



Colour model of Earth

From flux to Albedo Posted on Jan 16, 2014 15:21

Stam (A&A 482, pp.989-1007, 2008) has provided spectra of model Earths. From these we can construct B-V colours and compare to our observations.

Using the F0 and F1 model coefficients from Stam (cloud free and 100% cloudy, respectively) we combine these linearly as a function of cloud-cover fraction, and apply Johnson B and V filter transmission curves, and calculate B-V.

We check the method’s correctness on the Wehrli Solar spectrum (we get B-V=0.64) and for Vega (we get B-V=-0.014). Both are very close to the accepted values, so we trust the method for other spectra.

We generate a set of models going from cloud free to 100% cloudy, and get:

Is it just my eyes or is this a really lousy rendition? I wish the blog software allowed for better pictures! Anyway, what we (almost) see is that the observed B-V for earthlight corresponds to about 40% clouds on Earth, with our observational uncertainty of +/- 0.02 mags corresponding to quite a wide range of cloudiness: +/- 8%.

Colours are thus not the way to determine cloudiness on Earth!

Conversely – the variations in cloudiness on Earth is considerable from day to day (at some phase angles +/-20%) so we should be able to use colour variations as an indicator of cloud variations, along with the direct albedo measurements.

The Stam model is based on assuming a Lambertian forest surface; Earth seen at phase angle 90 degrees, and polarization ignored.

Our (considerable) errors on observed colour is due to the conversion from earthshine colours to earthlight colours and the problems of taking the effect of the lunar surface into account. Our observational error on earthshine colour is 0.005 mags. This would allow us to determine Earth’s cloud cover to about 2%-points from B-V colours alone.

A telescope in space (or on the Moon) looking directly at Earth could have colour errors that were even smaller.

The model has to account for orientation of Earth and so on, of course. This is not the case in the Stam model.



… and Dutch Radio!

Observing log Posted on Jan 15, 2014 15:36

‘de Kennis van nu’. The program will be available online, from the web-link http://www.wetenschap24.nl/programmas/de-kennis-van-nu.html



DMI news item

Observing log Posted on Jan 14, 2014 16:34

Our paper also made it to an online DMI news item.
http://bit.ly/1eBN5K2



In The Guardian online version

Links to sites and software Posted on Jan 11, 2014 08:20

Only a few hours after posting our A&A accepted manuscript on arxiv.org:
http://arxiv.org/abs/1401.1994
we were called up by Ian Sample, a Journalist from The Guardian newspaper, who had noticed the catchy title of the paper – he interviewed, and then he wrote this:
http://www.theguardian.com/science/2014/jan/10/dark-side-moon-turquoise-astronomers

A&A editors have asked us if we would like to have our Figure 1 on the cover of A&A and I said ‘YES!’. They could not promise they would use it – but let’s see.



Status JD2456645

Observing log Posted on Dec 18, 2013 10:30

Sloooowly the system is coming back to us: We now have LabView Engineering Mode control of Front shutter, Iris, Mount, Filter Wheels, and now also Focus stage.

We still need to get the SKE stages working.
We need to plug in the camera and see if it can be brought under LV control.

With these things running we are closer to getting some technical data from the system, such as bias frames, all-colour flat fields (for the first time), and we could study the shape of the PSF in different filters.

Important next steps are:

a) get the CCD working and under LV control
b) Try running simple scripts again without telescope mounted

Future, ambitious, steps are:

c) assemble the actual telescope in a testing facility and run pointing tests – perhaps test long sequences of observing scripts indoors and actually debug the system so it would work under field conditions.
d) See to the never-attempted integration with a weather station



AERONET data vs our PSF profiles

Post-Obs scattered-light rem. Posted on Dec 14, 2013 16:12

At MLO the aerosol load of the atmosphere is measured and a database of hourly, daily and monthly average values for the absorption cofficient, in three optical bands, exists. Here is the link:

http://www.esrl.noaa.gov/gmd/dv/data/index.php?site=mlo&category=Aerosols&frequency=Hourly%2BAverages

The data cover the period in 2011+2012 where we observed..

Our fitting procedures have produced estimates of the power a PSF has to be raised to in order for the convolution of model images with the resulting PSF to produce a ‘good fit’ at the edge of the DS disc in lunar images.

We compare our ‘alfa’ values with AERONET absorption coefficients, and get this plot:


We used daily average AERONET values, which goes some way towards explaining why the scatter for our data (the ‘Power’ alfa) is large while AERONET seems less messy. The overplotted red and green lines are two different robust regression fits. The slope is negative implying that large absorbtion coefficients correspond to small ‘alfa’ values. This matches how we empirically undersatnd the behaviour – on bad foggy nights (presumably with lots of aerosols in the air) we got broad PSFs with small values of alfa.

Repeating the above with hourly-average data the situation does not improve,

So our first interpretation of the above is that ‘alfa’ is only a poor descriptor of the aerosol load – ‘alfa’ is really just a dummy fitting parameter for us – alfa is the value that causes a good fit at the edge of the DS disc. It describes the shape of the PSF. Apparently thisis only weakly coupled to the total amount of absorption.

We must understand the AERONET data better first.



More small steps

Observing log Posted on Dec 05, 2013 14:17

Got the FWs moving today, using very basic LabView modules. Engineering Mode does not seem to want to talk to its ‘subVIs’ any more.

For reference:

in the VI ‘set DO line’ you can select port and line combinations. So far we know that:

port0/line22 operates the relays in the Breakout Box labelled RE2 and RE4.
port0/line23 – RE3 och RE5
port0/line20 controls the Thor FWs. The two relays adjacent to the Thor controllers come alive. The FW can be selected via the pushbuttons on the boxes.

If we can get the camera alive next we could at the very least do lab work on the CCD properties – make color flat fields, study the PSF in various filters, and so on.



Small steps …

Observing log Posted on Dec 04, 2013 16:21

Today Hans was able to operate the front shutter and the front iris using LabView. The licensing problem seems to have been cleared up (it seems to have been a typo in a form somewhere – perhaps it happened at MLO, or later at DMI). The CCD camera and the Magma no longer are listed with ‘yellow triangles’ in the system – we have yet to plug it in and see it operate.

Next steps are to use same methods to get all devices running – Engineeering mode seems to be dead somehow, but all the ‘sub VIs’ are there and so far we have gotten the front shutter and iris VIs to work – can slew telescope from sub VI also.



Should Clementine be scaled to Wildey?

From flux to Albedo Posted on Nov 04, 2013 13:13

We use synthetic models of the Moon, illuminated by a model Earth, to generate fits to our observations. The properties of the theoretical models determine not only the fit itself but also the quality of the fit. Two important choices are made in generating the theoretical models – the lunar
surface albedo map has to be chosen, and the lunar reflectance model has
to be chosen.

We can choose surface albedo either directly from
the Clementine map, or as this map scaled visually to better match the
1970s albedo map by Wildey.

We can choose the reflectance models
in several ways – current models being tested are ‘the new Hapke 63’ and
the ‘Hapke-X’ reflectance model.

We test how well these choices work by considering the RMSE of the ‘edge-profile fits’
generated between the theoretical Moon models and the observed ones.
Combing the two choices above in all possible wasy we get these results:
There are four histograms here but pairwise hide behind the other. The major effect is that due to the choice of albedo map (blue vs green above) – the scaled Clementine map has 25% lower RMSE than does the un-scaled map.

The effect of reflectance model is very minor and can barely be seen

The RMSE is calculated from the profile of 20-row averaged ‘cuts’ at the dark side edge, starting 50 pixels from the edge and ending 50 pixels onto the disc. Fitting was performed vith the MPFITFUN routine in IDL, using a 4-parameter fitting model where intensity offset, horisontal pixel shift, terrestrial albedo and PSF-parameter ‘alfa’ were varied to obtain the optimal fit. RMSE was then calculated from the residuals between observed edge-profile and model same.

The Clementine map was scaled (see http://earthshine.thejll.com/#post253 ) against the Wildey map in the sense that the lower-resolution grid from the Wildey map was interpolated by the Clementine map and the scatter-plot generated. An obvious offset and slope difference existed and from the robust linear regression of one onto the other a scaling relationship for the Clementine map pixel values was generated and applied to all pixels in the Clementine map.

So, we may be able to help constrain the sort of work current space-missions do! We should not avoid to point this aspect of the work out, in the ‘big article’.

Puzzling, at first, is the almost complete absence of an effect due to the choice of reflectance model. I can see effect clearly when total flux vs phase angle is plotted, so we simply do not have a large sensitivity to reflectance at the edge of the disc. Fair enough, only a local set of pixels are involved and the flux of the model bay be quite wrong while the fit at the edge is till very nice. So we should constrain our models by fitting many different aspects of our data – both the reflectance model against observed total flux as function of phase, and the albedo map by edge-fitting.

The subject of Clementine vs Wildey has been discssed in this blog before. Here:

http://iloapp.thejll.com/blog/earthshine?Home&post=304

and

http://earthshine.thejll.com/#post253



Diffuse reflectance imaging

From flux to Albedo Posted on Oct 17, 2013 14:27

Hasinoff et al have (link) shown how images of the DS edge can be used to invert for the illuminating light field (i.e. earthshine) and receive low-resolution maps of terrestrial albedo distribution.The pdf file is here:

Some notes on this:

1) The edge pixels that can be observed from Earth are differently illuminated because the source of light is extended and at a slight angle with the observer’s position.

2) Sally Langford described a method to find the edge of the lunar disk by using the Laplacian operator on the image and in essence do edge-detection. This method will be influenced by the fact that the edge is not uniformly lit. This applies, in principle, to our edge-fitting method too – except we have so low resolution that only 1-2 pixels at the edge show the effect. We do fit these – perhaps we should not.

3) We should be able to test the method Hasinoff et al describe from purely archival images found on the net – or we could try to get our own images (with the eyepiece projection method it should be possible to get high-magnification images of the DS edge). Or with the help of some willing amateur astronomer.
Or by applying for ‘service time observations’ at some large telescope.



Phase-dependent albedo

From flux to Albedo Posted on Oct 08, 2013 14:27

In the last plot in this entry http://iloapp.thejll.com/blog/earthshine?Home&post=375 we found the rise in fitted albedo unreasonable. Let us review what we have:

1) The albedos are derived as a scaling-parameter inside the Earth model we use. This model assumes the Earth is a Lambertian sphere with uniform albedo.
2) We use Hapke 63 to describe the reflectance of the Moon, and Clementine albedos for the lunar albedo.
3) We fit the DS edge – adjusting parameters in the synthetic lunar-image model so that a good fit is found between observed DS-edge profile and model DS-edge.
4) We see a 30-40% rise in albedo as Moon approaches Full (and Earth New).
5) We see a much weaker dependence of time-of-day in albedo. From considerations of the Opposition Surge (which may be inadequately represented in Hapke 63) we expected a drop in albedo towards end of day, and a rise at the start of day. We do see weak signs of the drop at the end of the day, but at the start it is ambiguous.

5) makes me think the problem is not the Opposition Surge in Hapke 63. 4) makes me think it is the total flux of the synthetic models that depend incorrectly on lunar phase. 1) makes me think that perhaps the Earth is not very Lambertian and that the discrepancy becomes much worse, particularly towards the crescent New Earth (Full Moon).

If scattered light is incorrectly included in the fit at the edge we would find a larger fitted albedo as lunar phase approached Full Moon, as we do – but there is NO WAY the fitting method is making a 30-40% bias-like error – it would be blatantly obvious when the fits were inspected.

What’s going on? I would like to compare the observed run of total fluxes against the model-based expectation.

Tests:

First we fit the edges of realistic but synthetic lunar images (made with Chris ‘syntheticmoon.f’ code):

Here models with albedo=0.30 have been edge-fitted – just like observations. We see that the edge-fitting method is completely able to retrieve terrestrial albedo.

So, there is a problem of some sort with the total flux of synthetic models. Let us look directly at the total flux from the synthetic images:


Here are shown (black) lunar model image magnitudes (adjusted to just about the right V magnitude at Full Moon) against lunar phase [FM=0], and expressions fro the same from Allen (red: 1955 edition, orange: 1975 edition), and from the JPL ephemeris system (green). JPL is based on Allen 1975 but adjusts fro actual distance to the Moon. Only ou rmodels allow for libration – hence the East-West assymetry in the black symbols.

It seems our synthetic images have a smaller range in total flux against phase than do the Allen and JPL expressions. By 1.5-2 magnitudes for some phase ranges, relative to Full Moon flux – or Full Moon is less bright by 1.5-2 magnitudes than are the Allen expressions, relative to intermediary phases. Our Moon images, relatively speaking, lack flux as we approach Full Moon.

In normalized-image edge fitting this will cause the model expression that is fitted to have been scaled by too small a number, making the step-to-sky at the DS edge too large, so albedo should compensate during the fitting by adjusting down. We see the opposite …

Could it be that our near Full Moon albedos are all right? But they are so large – 0.4 and more at small phases: but what do we know? Perhaps we are making a relative error in estimating what the ‘answer should be’?



OBD revisited

Exploring the PSF Posted on Oct 07, 2013 14:22

In this entry: http://iloapp.thejll.com/blog/earthshine?Home&post=374 we considered a blind deconvolution method.

The author, Michael Hirsch has kindly been in touch with us and pointed out that the isoplanatic patch is small so using the OBD on instantaneous images from a typical stack, will yield speckle-like PSFs that are then smeared by the method. A smeared PSF will cause the concurrently deconvolved image to have too sharp edges! This is what we see.

We should instead either

a) use subimages the size of the isoplanatic patch and receive the PSF for that image and a deconvolved version of that image, or

b) generate a series of stack-average images and apply the OBD to that, receiving the time-average PSF and the deconvolved image.

Of course, we thought we were doing b), but should instead perform the time averaging before OBDing.

Doing a) would not solve our halo-problem as the isoplanatic patch is small and we are looking for info on 1×1 degree PSFs.

Testing to follow …

Note that night JD 2456076 is the one with most (15) 100-image stacks in all filters. You could get 30 in a row if you combined IRCUT and VE1 …



Earthshine phase angle returns

From flux to Albedo Posted on Oct 05, 2013 15:39

In this entry: http://iloapp.thejll.com/blog/earthshine?Home&post=372 we looked at the evolution of earthshine phase angle as a function of time of day. This is an important subject for us because the lunar surface reflectance function has an ‘opposition peak’ which results in the infamous Opposition Surge near Full Moon.

Our observations have sampled the peak from just under 1 degree to about 1.5 degrees. From the plot in the above post it is clear that at JD start (noon in Greenwich, midnight at MLO) we sample the peak in the direction towards the peak (large to small phase angles). At the end of the JD we are sampling away from the peak (small to large angles).

The two situations are represented by Westerly and Easterly observing directions – Moon setting and rising, respectively.

There is a consequence of this ‘daily sweep’ of the range near the OS: as our reflectance model – Hapke 63 – does NOT contain the right OS we will see a rise or fall in the fitted albedo.

On the basis of the above argument I am making a PREDICTION: when we look at the fitted albedos we should see them INCREASE at the start of the day (because phase angles are dropping and to compensate the lacking reflectance the albedo has to go UP), and at the end of the day the fitted albedo will DROP.

Next, is a plot of the V-band albedos and the time of day:

We see albedos plotted against (JD mod 1). For each day one colour has been chosen, but some colours had to be reused. Each day has sequences of 3 points – this is because three fits of the model image to the observed image is performed.

To the left we see no slope, or perhaps a little rise towards the end now and then. To the right we see a general slope downwards. As predicted.

If the reflectance was correctly representing the Opposition Surge there should be no change in albedo as the day passed. We may be able to model the OS so that we can get the slopes to go away thereby determining the OS observationally. Troublesome will be the absence of a clear slope to the left, though. Hmm.

The large difference in albedo from day to day is mainly due to a phase-dependency in the determined albedos. The variations in terrestrial albedo are more subtle. We need to first understand why the fitted albedos depend relatively strongly on lunar phase. It could be due to problems related to model-fitting (although I doubt it since we essentially fit the step-size at the edge of the DS disk and this cannot be corrupted by scattered light). It could also be due to an incorrect reflectance function representation in the model used. This would not be the same dependence as discussed above related to near-opposition angles. The phase dependence I speak of is shown in the second row of panels below (the first being a repeat of the above plot):
Above (second row of panels) each ‘strip’ is a nights observations. V-band albedos only shown. In this representation New Moon is towards the sides and Full Moon is at 0 degrees.



‘Blind Deconvolution’ of a 100-stack

Exploring the PSF Posted on Oct 04, 2013 12:48

By careful analysis of the halo around stars, Jupiter and the Moon, we have arrived at an understanding that the PSF is related to a power-function 1/r^alfa, where alfa is some number below 3.

It is also possible to estimnate the PSF using deconvolution techniques. In particular, the ‘multiframe blind deconvolution’ method produces estimates of both the deconvolved image and the PSF, without any other input than the requirement that the output image and the PSF be positive everywhere.

There is matlab software provided here: http://pixel.kyb.tuebingen.mpg.de/obd/

and a paper here: http://www.aanda.org/articles/aa/pdf/2011/07/aa13955-09.pdf

Fredrik Boberg and I – well, Fredrik – has tested the software on images pulled from a 100-stack on (nonaligned, but almost aligned) images. The output comes in the form of estimates of the PSF and estimates of the deconvolved image. The deconvolved image has an unnaturally sharp limb – but more on that somewhere else – for now, let us see the PSF!

The estimated PSF is highly non-rotationally symmetric, but does have a central peak. The ‘skirt’ wavers up and down over many orders of magnitude. A radial plot of the PSF and a surface plut looks like this:

The red line shows the 1/r³ profile. The elliptical shape of the PSF causes the broad tunnel seen above – the divergence at radii from 30 pixels and out is the ‘wavering skirt’.

The PSF does appear to generally be a 1/r³ power law, which is encouraging. It appears to be quite flat out to 2 or 3 pixels – which also generally matches what we have seen in the PSFs empirically generated from imags of stars.

Next:
Test on a synthetic image convolved by a known PSF.
pre-align the images in the stack – perhaps a slow drift causes the non-round psf. Repeat the above for several stack from different nights.
Repeat the above for stacks taken through different filters – any indication that PSFs are different?



EMCCD characteristics

Relevant papers Posted on Oct 02, 2013 20:14

This paper develops the theory of the properties of EMCCD cameras, like the Andor 897 that we have:

http://www.aanda.org/articles/aa/pdf/2012/01/aa17089-11.pdf

Very relevant if a new method to handle bias and noise were to be developed in our projects.



Earthshine phase angle

From flux to Albedo Posted on Sep 30, 2013 06:02

We use Hapke’s 1963 formulation for the reflectance function in making the synthetic models that are at the heart of converting observed earthshine intensities to terrestrial albedos. H63 has a part, at small phase angles, that correspond to the reflectances for the DS – on the BS, phase angles are large as the source is the Sun and the observer sits on Earth – but for the DS the source is the Earth. Seen from the Moon, the Earth is about 2 degrees in diameter – just what is the effective source-Moon-observer (i.e. phase) angle? For our roughly 500 observations we calculate the effective DS phase angle from geometry and a simple model of the sun-lit part of the Earth based on NCEP cloud cover, and geometry. The intensity-weighted distance between MLO and each sun-lit pixel is summed so that the effective photo-centre distance, in degrees as seen from the Moon, is arrived at. This is the plot:


Photo-centre phase angle for DS observations plotted against time of day. No correction for actual Moon-Earth distance have been performed; a standar distance of 384000 km has been used. Each linear sequence of points corresponds to an observing night, showing the evolution of phase angle as Earth rotated.

We see our observations distributed in a characteristic way: most observations early in the JD have decreasing phase angles; most observations at the end of the JD have increasing phase angles. The median phase angle is something like 1.8 degrees, with some as low as 0.5 degree and some as large as 1.6 degrees.

Since MLO is on Hawaii, opposite Greenwich in longitude, the JD change occurs for some observing nights in the middle of a set of observations – so the above plot should really be plotted from -0.5 to 0.5 instead of from 0 to 1.

We see that we are sampling H63 at reflectances near the ‘opposition peak’ and that the range of phase angles we have may sample enough that the errors H63 has in representing this part of the true reflectance function could be important in our work on reducing observed intensities to terrestrial albedos. We ought to have available several reflectance formalism so that we can see the importance during data-reductions. Hans has provided such additional formalism in the synthetic code, but is is complex to use (for me) so we must make an effort to get used to the new code, for the ‘large paper’.



Surface brightness of Earthshine

From flux to Albedo Posted on Sep 26, 2013 08:56

Since we are calculating absolute calibrated B and V magnitudes (on the ‘lucky night’ 2455945) for the DS and the BS we can convert these to surface brightnesses, for comparison with e.g. Pallé et al published work.

The formula for surface brightness is

mu = mag + 2.5*alog10(w*w*N)

where mag is the magnitude determined from a patch containing N pixels, with each pixel covering wxw arc seconds. In our case
w=6.67 arseconds/pixel and
N is 101 and 113 for the 6×6 selenographic degree patches we use (+/-3 deg). With the magnitudes for B and V, BS and DS from the paper – but using N=1, since we report average magnitudes per pixel, we get:

DS:
mu_B = 14.29 m/asec²; SD=0.06
mu_V = 13.54 m/asec²; SD=0.06

BS:
mu_B = 6.21 m/asec²; SD=0.05
mu_V = 5.31 m/asec²; SD=0.06 (all SDs are internal error estimates based on pixel bootstrapping with replacement).

BS is about 8.23 magnitudes brighter than the DS – there are published numbers for these quantities (e.g. Franklin). His Table 1 has differences of about 10 mags between DS and BS. He may be talking about magnitudes per area – not magnitudes per pixel, like we are.

Pallé et al in this paper
http://adsabs.harvard.edu/abs/2007AJ….134.1145M
give plots showing mags/asec² and for the phase we have (about -140 on their plots) they have a BS-DS difference of 8.4ish mags – so we are within 0.2 mags which seems possible, given the scatter they show in Fig 1.

Not sure I like or understand why Franklin is 1.5-2 mags different in the DIFFERENCE – would that come about when you differ by mags/pixel and mags/area?



How bad is the Drag?

Data reduction issues Posted on Sep 22, 2013 11:04

During data-reduction we found the many ‘dragged images’. These are images where the shutter probably did not close before readout started. They look like this one (histogram equalized):
The ‘drag’ is from the BS but is probably present everywhere, but not visible from the DS. The BS has a maximum near the right hand limb where the counts are near 10000. The ‘drag’ is near 240 at most. There is no gradient in the drag along the direction of the drag:

The absence of a gradient in the drag helps us understand its origins – it is the effect of constant-speed readout. The readout speed is supposedly 1e-6 s per pixel (in the image header: this may be read from the camera but could be a fixed number entered at setup ….). Reading 512×512 pixels should take 0.26 seconds. The exposure time for this image was set to 0.0274s. Being a ‘dragged image’ we have no idea if this is right.

I do not quite understand the ratio of maximum drag value to maximum BS value. If the drag/BS ratio is 240/10000 and the BS exposure time is 0.0274 s the drag must have lasted just 240/10000*0.0274 = 0.00066 s. The camera is a frame transfer system – is frame transfer that fast? Perhaps the 0.26 s readout time is a slower line-by-line operation? This page mentions ‘a few msec’ for the frame transfer: http://depts.washington.edu/keck/pg22.pdf . We have 0.66 msec, so – close.

If the shutter got into the act in mid readout the amplitude of the drag is affected by this. I think many dragged images we have seen are like the above one – dragging is seldom much worse than this.

Let us check other dragged images with higher exposures and see if the drag/BS ratio is constant or not.

The above was prompted by pondering whether we could have a ‘shutter-less’ camera. It would appear not – at least not if the drag is due to open shutter during readout. The readout would have to be 24 times faster than it is and I am not sure CCD cameras come with such fast readout. 24 MHz readout?



Is image-shifting means-conserving?

Post-Obs scattered-light rem. Posted on Sep 22, 2013 09:02

In our data-reduction methods, image-alignment is often involved. It is based on interpolation. An important questions i whether the mean is conserved during such alignment?

To test this we iterate the shift of an observed image, measuring the mean of a patch given by selenographic coordinates. We iterate many times – i.e. reuse the previously shifted image for next round – in order to get a clearer answer. Visual inspection shows the iterated image getting ‘fuzzy’ as the shifts recur. That is, image standard deviation suffers. How bad is the situation for the mean of an area?

The answer is ‘not very much’. Iterating the shifts 40 times we find thatthe per-shift loss in mean value over a realistically sized patch on the DS is -0.005 %. The mean in the patch was about 10 counts which is ‘bright’ for the DS. At other lunar phases the counts may be just 2 or 3 – still just a factor of 5 from the above.

We would never shift the same image 40 times, of course, but the iterated shifts above allows us to build up a change in the mean that can be fitted with a regression. We have learned that one shift of an image may ‘cost’ 0.005% of the mean flux. This is a lot less than the goals for accuracy we have set, which are ‘0.1%’.

Added later: an identical test, but using ROT to rotate an image bya random amount, iterated, showed that errors per rotation were almost always less then 0.01%.



Lower limit on noise levels

Post-Obs scattered-light rem. Posted on Sep 19, 2013 16:21

We are interested in just how good the method we use is when it comes to measuring B-V in small areas on the DS and the BS of observed images. In this entry we could find an error as low as 0.005 (once we stopped measuring incorrectly). Is this thelower limit?

We now use synthetic images with realistic levels of Poisson noise added to get the answer. We use ‘syntheticmoon_new.f’ from Chris Flynn and input the ideal image from JD2455814 – i.e. the almost Full Moon – and coadd 100 images, each with realistic noise added, and folded with a realistic PSF (alfa=1.7). We do this twice, with different noise seeds – convert to instrumental magnitudes, and then subtract the two images, pretending they are B and V images and call the result our ‘synthetic B-V image’.

The resulting image is then measured in the areas designated by the agreed selenographic coordinates – those used in our little paper in Table 1. The errors found are:

0.0004 in the synthetic image
0.001 in the case of the real observed images

So, something is generating extra noise for us in the observed images, increasing the noise by more than a factor of two. This could be due to many things – real images are bias subtracted and flat fielded – synthetic are not; there is no readout noise in the synthetic images; image registration does not have to be performed (but could be simulated) in the synthetic images and we have seen variance do strange things during the necessary interpolations that take place during image alignment. There may be more issues to think of here.

This was for a situation with full illumination on both patches. What happens when one is in Earthshine only?

We will simulate this, but, for now, assuming the DS is 1000 fainter than the BS we would expect the error level to be sqrt(1000) larger in a DS-BS image, or about 0.012 mags. This is close to what we see (0.015).

Added later: Ona synthetic image of our lucky night (JD2455945) we find that the lower error limit on B-V BS-DS is

0.0021
and, for the observed image (see this entry)
0.005

So, again, we have a factor of two or so more nois ein reality than the best-case synthetic world predicts.

There is potential for improving our technique! For now we should report the above in the little paper. Fine tuning bias subtraction, flat fielding and image alignment – and understanding the role of image ‘wigglyness’ in alignment issues is needed.



NCEP cloud data and B-V variations

From flux to Albedo Posted on Sep 18, 2013 15:21

In this entry we introduced a Monte-Carlo based model for the spectrum of earthlight. We are able to use the model to estimate B-V for outgoing light, given input for cloud albedo, surface albedo and so on – and cloud fraction.

From the NCEP reanalysis project we can take the global total cloud fraction product (‘tcdc’) and calculate the global mean value for different times (we pick year 2011 here) and use the values for the B-V model.

I have done this. In 2011 TCDC varied between 50 and 55 percentage points. delta(B-V) [i.e. the differences between the solar spectrum B-V and the earthlight B-V, in the model] varied from 0.090 to 0.075 at the same time. That is, earthlight became more solar coloured for larger cloud cover – good – and implies that Earth is bluer when there are fewer clouds.

The change in B-V is 0.015 mags – can we measure that at all?

In our little paper we have relevant results. Uncertainties on B and V are at the 0.005 to 0.009 mags level, so that differences are at the 0.012 level (worst case) for a B-V value – on the ‘lucky night’.

But on the Full Moon night of 2455814 we have errors of just 0.001 and 0.002 on B-V.

We need to understand why the error can be so different. We promised 0.1% accuracies at the start of this project – and seem to be able to get it with 0.001 mags error – but why not on both nights?

Obviously, the error on the DS will be larger since there are much fewer photons – but this does not help explain why the errors are similar on the ‘lucky night’.

One point is that on 2455814 we measure B-V from one image generated by alignment of the B and V images in question. On 2455945 we measure B and V in seperate images and take the difference of those means. There are ‘cancellation’ issues at play here – surface structure will add to the variance of single -band images while some of the structures cancel (particularly if the images are well centred) in difference images.

Added later: yes, there is an effect (obviously, duuuh) – if we calculate the DS-BS difference from areas in the B-V image, instead of in the B and V images seperately, and perform bootstrap sampling on the pixels involved, we get the mean over the resamplings and its standard deviation to be

-0.155 +/- 0.005

whereas the difference between <B> and <V>, and its error calculated from error propagation, is

-0.154 +/- 0.014.

That is, we have a third the uncertainty. This will be used in the paper.

The uncertainty is still a bit high, though. (5 times what we promised, or to be fair about 5/sqrt(2)=3.5 times. since the above is a difference and not a directly measured quantity – which was the thing about which we made promises!).

It could be that noise on the low-flux DS is dominating here – this remains to be seen. And we still need to understand why it is still higher than for the Full Moon night – but things are making a bit more sense now.



Use Moon in EO satellite images

Student projects Posted on Sep 14, 2013 16:43

The Moon is occasionally present in corners of images of the Earth taken by Earth Observation satellites. Use such images and the known colours of the Moon to link the filter system used by the satellites to the Johnson UBV system. Use the resulting tool to calculate B-V colours of the Earth through time. Map out seasonal variability. Test sensitivity of the result to imposed variations in e.g. ice cover (by image manipulation) or similar for desertification.



Full moon B-V colour

From flux to Albedo Posted on Sep 14, 2013 09:17

I have looked through all suitable images close to full moon (i.e. at high Sun-Earth-Moon angle) and only came up with a few relevant nights, all within 10 degrees of full moon. We were keen to make a colour map across the face for figuring out what colour range is there, and what colours are reflected under BS sunlight, rather than DS Earthlight.

JD2455814: is the best full moon data available — it’s shown on the left. We ran it through our colour pipeline and get a B-V ~ 0.9. The B and V images turn out to be slightly different sizes, which made the colour map hard to produce. Later it has turned out that this may be due to distortions introduced in the shift to align the images at all. Work is proceeding on this.

JD2455905: data taken when we were following an eclipse — Dec 09, 2011, total lunar eclipse. No non-eclipsed images, so nothing useful on the Brightside colour night (we should check the Moon is red though).

JD2456082 is also eclipsed — partially. No useful colour data as we have only eclipsed images. June 3, 2012

JD2455847 looked like a promising night — full moon at an airmass of 1.54. However, when I make a colour map (right panel above), it has a B-V of ~0.4 across the face. Can only assume that the exposure times are wrong in the headers (as has happened occasionally). One corner of the moon is slightlty eclipsed for
some odd reason in one filter only (V) — opposite Tycho. Perhaps we hit
the dome in this filter? These data should clearly not be trusted further.



What did Allen mean?

Relevant papers Posted on Sep 13, 2013 19:47

In researching material for our first paper we come across the data in Allen for the colour index of Earth: “0.2”. There is no evident source of that piece of information so in order to interpret it and use it in the paper we have to do some detective work. Here it is:

References given by Allen (various editions)
in the table that gives B-V of Earth as ‘0.2’.
==========================

The references are both in the caption of the table, and in the B-V/U-B
column of that table. The table is on page 299 of the fourth edition
by Cox.

reference numbering as in Allen 4th ed.

—————————————————————————-
1. Astronomical Almanac, 1998, USNO

In the 1997 edition there is on p. E88 a table with a dash ‘-‘ for
Earth’s B-V. For the Moon there is 0.92 without a reference.

—————————————————————————-
2. Allen 1973 – the two colour columns of the table on page 144 of this
3rd edition – gives references to Irvine and Harris (see below).

In Allen 1955 there is on p. 159 a table with ‘C’ for planets and
Earth has 0.2. In the notes it is explained that C is the ‘colour
index’ and that it is offset from the larger published value by 0.1
to 0.2 … What was ‘C’ as published and where and what is C compared
to B-V? The references on p.160 of the 1955 edition are not specific
for each piece of information, but we find as number [4] a reference to
Danjon. There are two references, it seems – 1936 and ‘Danjon, Comptes
Rendus, 227, 652, (1948)’. The 1936 is an observatory publication on
earthshine observations I am trying to find. The CR reference is about
Mercury and Venus. Using information from Wildey’s paper on Mariner 2 data
for Earth where the mean Danjon C.I. of earthshine/light, BS Moon and Sun
are given, we can estimate the linear transformation from the C.I. system
to B-V. We get, using Wildey’s data for Moon and Sun C.I. but not his
B-V data for same (using instead Holmberg et al for Sun) that

B-V = .112+.671*.(C.I.)

With 0.2 for C.I. we get B-V=0.25.

We can test the effect of using B-V for Moon = 0.92 (an often cited
value), this gives
B-V= -0.0665+.8968*(C.I.)
and C.I. = 0.2,0.3,0.4 gives B-V=0.11, 0.20, 0.29, respectively.

—————————————————————————-
3. Irvine et al 1968, AJ 73, 251, 807
Paper is on B-V for the planets – not Earth.
—————————————————————————-

4. Harris, in Vol III, (eds) Kuiper and Middlehurst, p. 272.

The Harris chapter in the book briefly discusses the photometry of Earth
– mainly in the form of a citation to Danjon’s chapter in Book II of the
series by Kuiper and Middlehurst. The Danjon phase law is given for earth,
but no comment on colours.

—————————————————————————-
Summary: The source of the ‘0.2’ for Earth in Allen is thus not source 1,
nor 3 – but 2 and 4 point at Danjon. The 1933 An. d’Obs. Strasbourg Vol
3 reference is in fact available online (now) at ADS. It is in French, and
it is long – but towards the end things hot up because a summary of color
index data for the Earthshine (Lumiere cendree) and Earthlight (Earth
itself) are discussed, although in the Rougiere system. The effective
wavelengths of this system are given in chapter 4 on p 171 and are

Rouge 606 nm
Vert 545 nm
Bleu 467 nm

so Rouge is between Johnson V and R, Vert is between Johnson B and
V, closer to V, and Bleu is between Johnson B and V, closer to B.

Of interest is a description of the areas on the Moon used – it seems
Crisium and Fecunditatis are used but only after 1928 is this fixed.

Also of interest are mean values (mean over several years) for the colour indexes,
given on
p 174 earthlight C.I. = 0.33 from -0.01 to 0.6
p 175 earthshine C.I. = 0.62
The variability is large and is said to be mainly seasonal. Note the
addendum at the very end (p. 179) where everything important seems to be
updated in a breathless paragraph! An updated value for C.I. for the Moon
by Rougiere is given as 1.10 (cited by Wildey in the Mariner 2 paper),
and the EarthSHINE C.I. is therefore updated to 0.64. There is a note
that there isn’t space to also update the earthLIGHT number but I assume
it is upped by 0.02 also. So, if Allen is referring to Danjon’s C.I.
as 0.2 and states that it is 0.1 to 0.2 lower than the published value,
we can see that Allen is talking, indeed, about the C.I. for Earth itself
– earthLIGHT – and not earthSHINE.

So, using the linear equation developed above
B-V = .112+.671*.(C.I.)
and now inserting C.I.=0.35 (with limits at +0.01 to 0.62)
we have for the earthLIGHT B-V = .35 (with limits from 0.12 to 0.53).
For an earthSHINE colour of C.I. =0.64 +/- 0.3 we get a B-V range of 0.54 +/- 0.2
—————————————————————————-
Conclusion: This spans our observation as well as Franklin’s, but with large uncertainties. Note the numerically identical B-V and C.I. values – so Allen is right in as much as replacing the ‘C’ from the 1955 edition with the ‘B-V’ of the 1973 (and on) editions.
—————————————————————————-

Data to plot

Year (B-V)ds-(B-V)bs
————-
1926-1935 0.54(+/- 0.2) to 0.85 = -0.31 to -0.11
1967 -0.17 +/- 0.05
2012 -0.16 +/- 0.02

Plotting this we have:

The Danjon data are an average of measurements from 1926 to 1935 and upper and lower end of the bars indicate the extent of seasonal variability, surmized by Danjon. The Franklin bars represent +/- 1 standard deviation based on the 12 determinates from 2 nights in his paper, and our error bars represent +/- 1 standard deviation based on scatter due to photon statistics in our measurement from a single night in two images.



Nice list of filter properties

Links to sites and software Posted on Sep 10, 2013 07:50

Here is a link to a site that lists transmission curves of many color filters – e.g. Wratten filters.

It looks like Schott RG665 is most like VE2, and might be the long-pass filter we need on a modified SIgma SD100 camera. We need it with 58mm thread, though.

Wratten #25A is available from B&H with 58mm thread but has its cutoff at 620 nm.

B&H have a 58mm Polaroid that looks ok. Transmits from 720 nm and up.



Brigthness at the limb

From flux to Albedo Posted on Sep 05, 2013 12:48

We have noted interesting behaviour of our data at the lunar limb, link here. It seems that our data show a convergence of intensity ratios, between certain symmetric points on and near the limb, towards unity. We saw the same in model data.

The model we use is based on the Hapke 1963 BRDF (i.e. reflectance) model and looks like this:

BRDF ~B(phase)*S(phase)* 1/(1+cos(e)/cos(i))

where e and i are the angles of emission and incidence. For a given instant the phase is the same everywhere (almost) so the only angular dependency that remains is the last term above. On the limb we can have various values of i but only one value of e – namely pi/2 – recall that i is the angle between local normal vector and the Sun, while e is the angle between local normal vector and the direction towards the observer.

cos(pi/2) is 0, of course, so the last term above is unity everywhere on the lunar limb. The ratio of the reflectance in two points on the limb therefore reduces to unity. The ratio of two limb intensities should therefore be equal to the ratio of the albedos at those two points.

So, there is no clue here as to why we observe, and the model gives, intensity ratios near unity along the limb. Mystery remains!

Our observation of unit intensity ratio certainly is consistent with Minnaerts reference to Schoenberg’s statement – namely that the ‘intensity along much of the bright limb is a constant value’. Intensity ratios between limb points would give unity.



Minnaert symmetries

From flux to Albedo Posted on Sep 04, 2013 13:48

In 1941 Minnaert published a paper about lunar photometry – in particular the reflectance. There is an idea in that work which we can use to view our data, and perhaps learn something.

The intensity we observe in any pixel of the lunar disk is:

Intensity ~ albedo * reflectance(i,e,alfa)

Where ‘albedo’ is a measure of the darkness of the material in the pixel and ‘i,e,alfa’ are angle of incidence, angle of emission and lunar phase, respectively.

Consider now two points A and B on the lunar disk placed such that iA=iB and eA=eB. Furthermore we have that the phase, alfa, is almost constant all over the image (alfa is the angle between a point on the Moon and the observer and the Sun – and apart from the finite size of the Moon compared to the distances to Sun and Earth this angle is the same for all points on the image of the Moon). In these two special points we thus have

reflectance(iA,eA,alfa) = reflectance (iB,eB,alfa) so that the ratio of observed intensities from A and B are simply:

I_A/I_B = albedo_A/albedo_B

By finding many pairs like A and B we could investigate how ratios of albedos across the Moon vary – or compare the ratio from observations to our map of albedo – at the present “Clementine scaled to Wildey”.

We now do just this!

From the synthetic image software code that Hans wrote we have, for every pixel on the model image, its angle of incidence and emission. We can therefore select an arbitrary point A on the BS and find its symmetric point ‘B’ also on the BS, extract the observed intensity ratio A/B as well as the same ratio for the ideal model image and tabulate these against solar zenith angle (i=SZA) and earth zenith angle (e=EZA). We have done this for 1000 randomly distributed points and show the results here:

Top panel shows observed (black) and model (red) A/B ratios plotted against EZA. Small EZA (in radians) correspond to points near the middle of the terminator on the disk while large values correspond to points closer to the limb, near the sky. Second panel shows the same plot but now using solar zenith angle (also in radians) as the x-axis – small SZA correspond to points near the sub-solar point, in this image near the limb, while large SZA correspond to points near the terminator where the Sun is low in the sky as seen from the Moon. Bottom left plot is the ideal ratio against observed ratio. Green line is the robust regression line and blue line is the diagonal. Bottom right is a small image of the Moon with points A and B plotted as white dots.

We emmidiately notice the tendency for observed ratios to be larger than ideal ratios – at least for the median to large values of the ratio: the observed ratios lie above the diagonal line in bottom left panel.

The observed ratio has a larger span not just because of ‘noise’ – careful inspection of ‘slices’ across e.g. dark Mare [not shown here] reveals that observed A/B ratios reach more widely separated extremes there than does the ideal ratio slice across the same slice.

This seems to imply that the albedo map we use is too ‘flat’ – it should be scaled so as to give more extreme darks and lights by perhaps 25%, judging from the plots.

We also have to wonder why albedo ratio is a function of EZA and SZA – nearer the terminator or middle of the disc ratios are simply higher than near the limb. As intensities are smaller there while brighter near the limb we wonder what is going on – remember that the ratio is reflectance-independent by construction! Note also that both observations and models behave in this way. It cannot be ‘nonlinearities in the camera’ since the model behaves in the same way, nor ‘scattered light in the observations’ since the scattering-free model behaves in the same way. The model is also assuming a spherical surface – no hills, boulders or crevices for shadows to hide in.

Here is a repeat of the above for a different lunar phase:


Similar features are seen here – towards small SZAs the ratios go to 1. As before the general picture is that variations observed from dark to light are larger than in the model. This reminds us that we are using a fixed albedo map in the synthetic model – we can expect different dark/light ratios in different wavelength bands. The above scalingmay sereve to achieve this?

We shoudl repeat for many examples from each band and each lunar phase to see which features are general.

There is a comment by Minnaert in, “Planets and Satellites” ed. Kuiper and MIddlerhurst, Vol III. p. 222 to the effect that the “bright limb shows the same brightness over 3/4 of its length and that this was noted by Schoenberg (1925) and this was only rarely checked” – wonder if that last remark is still true? We can certainly confirm that the albedo-ratios tend towards 1 near the limb, but we do not understand yet if these two observations are linked. Time to read Schoenberg!



FFT deconvolution of images

Post-Obs scattered-light rem. Posted on Aug 30, 2013 15:06

Our observed images are a convolution of a point-spread function (PSF) and the Object (the Moon). We know from analysis of images of various sources – the Moon itself, Jupiter, and stars, that the PSF has the character of a power law along the lines of a core with wings that follow 1/r^alfa with alfa near 2.9.

It is in principle also possible to determine the PSF by de-convolution. Clasically we would expect that

Image = Object * PSF

Where the ‘*’ implies spatial convolution. Taking FT of both sides we get

F(I) = F(O) x F(P)

where F is the forward Fourier transform, and the ‘x’ implies multiplication. Rearranging and taking inverse Fourier transforms (f) we get:

P = f(F(P)) = f(F(I)/F(O))

We have I (the image) and if we had O we could calculate the PSF P. Usually P is very noisy because of amplification of small noisy signals at high frequencies due to the division above. We can average over several estimates of P, however.

For us, this is easy because our data come in the form of 100-image stacks. Unfortunately we do not know O, the object. But we can make synthetic images of O that are highly realistic!

We have applied the above procedure on an image-stack of the Moon, for which we also have a synthetic image. We average the 100 PSFs that are generated. The radial profile from the peak looks like this (black curve):

The blue line is a power law with alfa=2.1 and the red line is a power law with slope 0.3. For this particular night we know that a single power law (plus a narrow table-lookup core) of alfa=2.6 has been estimated for this image using forward modelling techniques where we fit parts of the scattered halo at about 100-150 pixels distance from the photo-centre of the bright side of the Moon.

The two estimates of the PSF are thus quite different, and I wonder why.

Possibly it is a sign that the edge-fitting can be performed well with a large family of PSFs. We know that the fit is excellent at the edge of the disc.

I suggest that a next step could be to test both PSFs or types of PSFs and see how they perform in various situations. How close to the observed image is the ideal image once it is convolved with the above profile?



Moon halo in R, G and B

Exploring the PSF Posted on Aug 28, 2013 14:42

We have indications that the halo seen in B and V images are somewhat similar, but that VE2 images of the Moon have decidedly different halo profiles. Because the filters used in B, V and VE2 are fabricated (and function) differently we may have the situation that this causes the differences in halo profiles. B and V are ‘coloured glass’ filters while VE2 is a thin-film filter. In a DSLR detector the R,G and B filters are essentially bits of coloured semiconductor material. As a test we inspect the halo seen around the Moon in images taken with a DSLR camera. In such images the R,G and B channels are obtained at the same time at precisely the same observing conditions.

On the internet (http://bit.ly/13YZIOU) we have found a very detailed large-scale JPG (i.e. 8 bit only) 10-min exposure image of the sky near Orion containing the Moon. We submitted the image to nova.astrometry.net and received a solution back, including image scale. We took the resulting WCS-equipped image and extracted the profile of the lunar halo in R, G and B and plot these against radius from the estimated disk centre. We have subtracted a sky level for each colour, estimated by eye, and show the profile in the upper panel and repeat it in the lower panel with straight lines fitted:

The profiles are saturated out to about half a degree but after that they follow a remarkably similar shape in this log-log plot. Fitting power law functions (1/r^alfa), we may even see some sort of ‘straight line behaviour’ between radius 0.6 and just short of 2 degrees, with another linear trend taking over out until the sky-noise is reached at 4 degrees or so. The slopes are fitted-by-eye only but are -2.9 (near the canonical ‘can-never-be-steeper-than-3’) value, and -1.3.

Before assuming that ‘DSLR RGB imaging’ will solve all our filter problems, let us recall that the VE2 vs other profile differences we have found are subtle; that the above is based on an 8-bit image; that nothing is known about the image treatment performed by the author of that image, and that we do not have data similar to VE2 here – the R channel is not VE2 [ … but read more here link !]. Let us instead try to do something similar with 14-bit RAW images.

Note that the above image was obtained by tracking the sky. Apparently, tracking allowed a long exposure that gave us a wide halo to study. In our own DSLR-On-the-sky wide-field images we have failed to get results similar to the above – exposures were limited to several seconds to avoid overexposure and trailing. Our own MLO telescope images are restricted to the 1/2 degree reach from disc centre.

Note that the above essentially speaks very well for the sort of DSLR optics used – the amount of scattered light near sources must be low or we would not see so steep a PSF! Again, this may be easier to do with a wide-field lens such as used for the above image – trhings may be different with a tele-lens that allowed a closeup of the Moon. The ‘core’ of the PSF should be better investigated, if we can get some of our own images of e.g. Jupiter or bright stars – the above image is VERY wide-field and contains zillions of crowded stars, not likely to give us good point-source PSFs.



Colour woes

Data reduction issues Posted on Aug 19, 2013 12:57

In this post we commented on the problems we have with the requested and actual exposure times. It turned out NOT to be a good idea to correct the requested exposure times – certain colour problems got worse!

What happened was that B-V for the BS plotted against time of day showed a decided reddening towards the end of the Julidan day – this corresponded to the setting Moon. Airmasses were growing towards the end of the day.

When time of day and B-V were plotted we could see slopes using UNcorrected exposure times of 0.4 magnitudes/day. If we DID correct the exposure times as explained in the above link we got almost a whole magnitude per day in slope!

This got me thinking about whether the extinction corrections we perform are correct – they depend on knowing what the airmass is and knowing what the extinction coefficient is. If either, or both, are wrong we can expect colour-terms to pop up.

The airmass we have been using assumed surface pressure and 0 degrees C and 0 % relative humidity and a wavelength of 0.56 microns. More appropriate is the pressure that occurs at the MLO (we guessed it is 0.6*760 mm Hg), and a temperature of at least a few degrees C (we never observed in frost), and a relative humidity of 20%. Some of these quantities could actually be replaced by the available MLO meteorological data available – more on that later. We should also use an appropriate wavelength for B and V of 0.445 and .551 microns, for B and V respectively. However, using these better values will at most give an offset in B-V and cannot explain an airmass-dependent colour term. [or can it? gotta think about that one …]

Next we must consider the formula used for the airmass. Looking in the IDL code ‘airmass.pro’ provided by Marc W. Buie, we see it is the “cosine based formula derived by David Tholen” with an update 2003. The formula appears to be:
cz = cos(zenith[z])
am[z] = sqrt(235225.0*cz*cz + 970.0 + 1.0) – 485*cz
When we have a reference for this I shall return to the question of whether the approximate formula is good enough.

Finally there are the extinction coefficients. Since we see an increase in B-V with airmass it is either the V magnitude that is being over-corrected or the B magnitude that is being under-corrected. Experimenting, I find that kV=0.08 and kB=0.17, instead of the 0.1 and 0.15 that Chris found from standard-star observations, givea a colour-free B-V progression with time of day (at Moon-set – Moon-rise is too underrepresented to be useful here). Is it within the error margins that kV and kB should have these values? Old man Chris says he estimated the values 0.1 and 0.15 ‘by eye’ here but I think he also listed some regression results elsewhere, with errors – but I can’t find ’em!

Anyway, using the values 0.08 and 0.17 gives a mean BS B-V of 0.898 +/- 0.054 (error of mean +/- 0.004). This value is relevant when keeping the discussion about the “van den Berg value for B-V=0.92” in mind, link here. Our own average of vdB62 data gives 0.88 +/- 0.02 – within the errors we have, easily.



Requested vs acquired exposure times

Shutters Posted on Aug 18, 2013 08:22

We had a method to measure exposure times – but the method depended on an LED inside the telescope system, and it appears that the system scattered light which interfered with exposures. So we had it turned off. From data collected before the device was turned off it is possible to look at the relationship between requested and supposedly actual exposure times.
For requested exposure times below 10 ms it appears we did not get anything useful. For requested exposure times above that there is a bias which decreases with the increasing duration requested. It is simple enough to fit a regression line to the above, it is:

exptime_actual=-0.0045021465+exptime_requested*1.0004696

and use this relationship to correct all exposure times. We have tried this.

On the left, results without correction of exposure time. To the right with.

We have extracted B and V magnitudes from the set of ‘good images’. We have converted to magnitudes using Chris’ NGC6633 and M22 calibrations. For B and V images taken within ½ and hour of each other we have calculated the mean B and V values for the BS and then B-V. We plot these values against lunar phase as well as the time of day, and we see that there is a phase- and time-of-day-dependence in B-V. When we compare B-V from images where the exposure times have NOT been bias-corrected with B-V from images where the bias HAS been corrected we realize that the problems are much worse for the bias corrected images, suggesting that the bias-correction is not valid.

Hmmm.

That in turns suggests that the method for measuring exposure time is faulty. As far as we know the method was based on passing IR-light through the shutter aperture to a detector and then calculating the opening time of the exposure from data collected. We are not sure if this was based on a timing mechanism or a flux-collected method.

From our own images of a constant light source we ought to test the stability of the shutter. From the lab we have Ahmad’s laser images, and from MLO we have many stellar images, Jupiter images and also hohlraum images.



Some small insights

Relevant papers Posted on Aug 15, 2013 16:46

Minnaert 1941 paper on the ‘reciprocity principle’ and other symmetry-considerations:

pdf

Photometry of the bright side of the Moon has been carried out for at least 100 years. An important paper is the one by van den Bergh:

pdf

He observed the Moon at 5 different occassions – two of them at Full Moon and one of them at half Moon, and derived B-V colours for 12 regions, measured relative to Mare Serenitatis. He also observed standard stars and tied photometry of Mare S. to these – thus tying the B-V photometry of the 12 regions to the standards.

Here are some results and insights:

For Mare S. he found B-V = 0.876 +/- 0.022. Over the 12 regions on the Moon (20” to 40” large) he reports B-V relative to M. Ser. This offset has mean value -0.017 +/- 0.005, where the mean was weighted (by me) with the square of the mean error reported by vdB. Knowing B-V for M. S. we can say that an average over 11 regions on the Moon (I dropped one measurement as it was flagged problematic) is 0.859 +/- 0.023.

I would say that this is what vdB62 reports. From somewhere we have the notion that he reports 0.92 for B-V – I do not see that in his paper. But he does quote Harris for saying that the mean over many regions on the Moon is 0.92, but does not comment further on that.

I notice that vdB does not allow for the different phases of his observations, and I think I know why – the photometry may depend on the lunar phase, but the colour does not (at least for these phases). We should check that, as we have BS B and V images galore!

Another paper, that by Wildey and Pohn:

pdf

contains masses of UBV photometry, and seems to discuss lunar phase and corrections thereof. Should be looked at. Many tables to digitize, though. Bachelors project?

Franklin paper:

pdf


Estimates of earthlight B-V color.

Error budget Posted on Aug 11, 2013 11:13

We have some observationally based estimates of B-V for earthshine. We are working on various models of the same that would allow us to check our assumptions and set error limits.

A somewhat sophisticated model of the color (B-V) of earthshine has been made. It consists of a Monte Carlo simulation where scattering events are decided by random number generators and probabilities bazsed on albedos.

I draw photons from the wavelength distribution given by the Wehrli 1985 Solar spectrum. For each photon we allow a Rayleigh scattering event to occur with probabilities depending on the wavelength. If the photon is not Rayleigh scattered its further fate is given by factors describing the fraction of the surface covered by land (0.3), and the albedos of clouds, land (either Soil or Vegetation), and ocean. Some photons are absorbed while others are scattered into space. The scattered photons
are collected and the emergent spectrum built up. If repeated for a large number of Monte Carlo trials a smooth spectrum results. The B-V colour of this is then calculated, using Bessel transmission curves for the Johnson B and V filters. The trials are done assuming that all land is covered by either vegetation or is bare soil. Cloud albedos
are wavelength-independent, while ocean albedos are zero above 750 nm. Tables of albedos for land types and ocean is given by “Spectral signatures of soil, vegetation and water, and spectral bands of LANDSAT 7.Source: Siegmund, Menz 2005 with modifications.” found at this link:http://bit.ly/14jD3qm.

The B-V of the incident solar spectrum is calculated as a check. The expected solar value of 0.623 (+/- 0.001) is recovered, where the uncertainty is the S.D. of the mean over the trials. The B-V values of the emergent spectra are subject to stochastic noise, so a number (20, each with 2 million photons) of trials is performed and the mean of derived photometric values, with standard deviation of the mean, calculated.

The results for earthlight are:

B-V of earthlight, assuming all land is vegetation-covered:
0.5692 +/- 0.00057
where the uncertainty is the S.D. of the mean.

B-V of earthlight, assuming all land is bare soil:
0.5759 +/- 0.00068
where the uncertainty is the S.D. of the mean.

The difference is 0.0067 +/- 0.00088 with earthshine due to the soil-only model being redder than the vegetation-only model. The difference in B-V is small, but significant.

A validating test is to look at how much flux is taken out at 550 nm – we know that extinction at that wavelength is near k_V = 0.1 so about 10% of the flux is removed. Since Rayleigh scattering is symmetric in the forward and backward direction (for single scattering events) we estimate that half of the light removed from the beam is scattered into space. Thus we expect the flux of Rayleigh scattered light to be about 5% of the incoming flux. We measure the Rayleigh/Incoming flux ratio to be 0.0521 +/- 0.0003. That is, as expected, about 5% of the sunlight is scattered by Rayleigh scattering into space, a similar amount being scattered downwards and giving us the blue sky, and totalling an extinction at 550 nm of near k=0.1. This assumes all absorption is scattering due to Rayleigh. At 550 nm that is not a bad approximation.
For the B band at 445 nm we find that on average 2.3 times more flux is taken out by Rayleigh, corresponding to an extinction coefficient 2.3 times larger than in V. That is a bit steep, surely? Chris has measured k_B=0.15 from extinction data.

So, the model suggests that when you switch all land areas from vegetation only to soil only the B-V will redden by less than 0.01. The color of both modes is near B-V=0.57, with realistic land-ocean, and cloud fractions, as well as albedos. The model indicates a colour very close to the cruder model from the paper (B-V=0.56 at k=0.1). We use the exercise here to estimate that errors due to omission of land- and ocean- albedo is of the order 0.01 mags, and that B-V of earthlight is near 0.56.

Now, what did we find observationally?

We found DS B-V is 0.83. We also know that the sunlight has B-V=0.64 but appears as Moonlight at B-V=0.92. so that we can infer that one reflection on the Moon reddens light by 0.28 in B-V. The earthlight is also reddened by striking the Moon and we can infer that our observation of the DS implies earthlight that is 0.28 bluer, or B-V=0.83-0.28=0.55. That is very close to what our model suggests!

Note that all uncertainties here are governed by photon statistics –
not underlying physics or observational skills. Note also that we use
the phrase ‘earthlight’ when we mean the light scattered by Earth and
observed in space. By ‘earthshine’ we shall mean the color of the DS of
the Moon, which is illuminated by the earthlight. We have not even tried
to model the effects of hemispheric geometry, or any effects of
reflectance-behaviour on angles.

——— added later ————-

This was done under an assumption that corresponds to ‘airmass is 1
everywhere’ – this is wrong, of course, since airmass for those surface elements near the edge of the terrestrial disc is larger than
1. It turns out that if you assume that airmass goes as sec(z) then the
area-weighted effective airmass for a hemisphere viewed from infinity is
2. We repeat the above trials with twice as much Rayleigh scattering.
The colours we get now are:

Assuming all land is plants : B-V = 0.5063 +/- 0.0009
Assuming all land is bare soil: B-V = 0.5152 +/- 0.00075

The difference is: 0.0089 +/- 0.0012

The Earthlight is now bluer than under the airmass=1 assumption (good), and the difference in turning land pixels from bare soil to vegetation is still small.

If we assume complete ignorance of the soil/vegetaion mix we get:

B-V earthlight, for airmass=1 : 0.57 – 0.58
B-V earthlight, for airmass=2 : 0.51 – 0.52.

The difference between airmass=1 (not realistic) and airmass=2 (realistic) is 0.06 mags.

I conclude that the model indicates a B-V of earthlight, at the most realistic assumption of the effective or average airmass involved in Rayleigh scattering, near 0.515 with a small dependence on wheather the land is vegetation or bare soil on the order of 0.01 at most.

I should repeat the above with VE1 and VE2 filters to see what change in NDVI vegetation index we might expect.




Aurora works.

Met sensor Posted on Aug 08, 2013 09:36

We have mounted the Aurora weather sensor on the roof of DMI and found out how to run it from Linux. A plot is generated automatically and updated each minute. The link is here:

http://web.dmi.dk/solar-terrestrial/staff/thejll/metdata.html

Top frame shows sensor temperature (i.e. how hot the sensor itself is) and the radiance temperature of the sky measured inside a wide cone looking up.
Next frame shows the difference between the two – when it is large the sky is probably clear – i.e. no clouds and no humidity.
Third frame shows the amount of light reaching the sensor – so you see day and night and clouds passing in front of the Sun.
Bottom panel shows rain falling on the sensor.

Future work will now be directed towards producing and statistically validating a warning signal (‘Close The Dome!’) on the basis of these readings.

We might mount a webcam also.



Earth on JD 2455945

Showcase images and animations Posted on Aug 07, 2013 13:38

We have the unique B and V images from JD2455945.17xxx in which the ‘halo’ cancels almost perfectly, allowing us to see the colour of the DS. At the time of observation we can ask what the Earth was like. Here is the image generated from the Earth Viewer at Forumilab ( http://www.fourmilab.ch/cgi-bin/Earth ):

Above is the model image of Earth seeen from the Moon at the time /Jan 18 2012, about 1600 UTC) of our observations. Below is imagery from the GOES West satellite. There are also GOES images from 12 UTC and 18 UTC, the above one is from 15UTC and is the closest in time.

Evidently the Moon and the GOES satellite were not in line as the aspect is slightly different but we get the idea: Southern Ocean and Antarctica was white with cloud and ice, North and South America were partially cloudy, and the sunglint was on ocean off the coast of Peru, approximately.

Here is a simple model image, as if seen from the Moon at the relevant time. The image is generated from geometry and the ‘NCEP reanalysis cloud product’. You notice some similaririties in cloud cover between the satellite image above and this model image. The model only does the perspective projection of pixels onto a sphere – there is no modelling of limb brightening or anything like that.

The images does allow a simple analysis of the effects of various extreme values for land albedo, and used for two assumed wavelength bands could be used to set limits on what we expect the colour of the earthshine to be at this particular time. More to follow!



LabView code

Observing log Posted on Aug 02, 2013 12:29

Ingemar was here and looked at the LV code. We semeed to learn the following:

1) The NI Report Generation Toolkit license has run out and needs replacement.

2) The newest set of LV codes are on the NAS under ‘Earthshine Project/LabView Sou…/’

3) The Engineering mode refuses to run because of some error messages related to the AXIS – these are for the FWs, the SKE and the Dome. No matter what you want to do with the Engineering Mode it will always start by initializing all its sensors, and if one of them fails, nothing works! Since we have no dome there will indeed be a problem here!
Ingemar was able to turn off some of this (NB: set it back to as-was, later!) and indeed the code progressed to a further point (when it then ran into other problems). Front shutter was tested while the error-checking was turned off in Eng. Mod. but did not work.

Of the above 1) may not be important as long as scripts are not being read. But 3) seems pretty serious.

Questions:

Are the new licenses for the MAX correctly installed?

Can the code that refers to the dome be turned off?

Will things work again once the dome is turned off and the other AXIS cables are plugged in?

Why did front shutter also not work – it has no motor?

Added later: Some of the above is now clearer. See here. A number of the errors were related to wrong LV license number, and the new IP numbers for DMI must be set insid ethe LV code. For reasons we do not understand the COM-port numbers had changed.



Core vs. Wing

Data reduction issues Posted on Jul 24, 2013 23:56

In using the fitting of model to observations at the edge of the lunar DS we have the following situation:

It is seen that the ‘edge’ of the observed image is not ‘sharp’ – it slopes over perhaps 3-4 columns on either side of the midpoint of the jump from sky to disc. The models often have problems matching that low slope and tend to be ‘sharper’ at the edge, at ‘best fit’. We fit +/- 50 columns on either side of the edge so more pixels on the sky participate in driving the fit than does the ‘edge-pixels’.

The shape of the edge is driven by convolution with the PSF – and the PSF core dominates in setting the sharp details. The slope of the DS sky halo is given by the wings of the PSF.

There is an opposition between fitting edge and halo slope – lower alfa would broaden the PSF and ‘lift’ the halo wings, increasing the slope at the same time as ‘dulling’ the edge of the model edge. With more pixels driving the fit at the sky halo the edge is bound to be ignored – thus too large alfa’s probably are the result. The pedestal added to the model image to match sky brightness is competing with alfa’s effect on the sky halo – part of the level of the sky halo is provided by the pedestal value while part is given by the effect of alfa on the PSF itself.

Thoughts:

1) Exponentiation of the canonical PSF to power alfa may not preserve the shape of the PSF near the core in the required way.

2) More weight could be given to the ‘edge pixels’.

3) Slope of the edge is not just due to rounding by the PSF – there is also some ‘jitter’ from co-addition of 100 frames so alfa is used to account for more than the effects of the PSF.

4) The fitted pedestal is sometimes negative – it should probably always be positive due to sky brightness after bias subtraction, but the bias varies by 1 count due to the temperature effect and is scaled, as described elsewhere. The pedestal is often near -0.1 counts – are we sure that the bias has been correctly scaled to the level of 0.1 counts?



Putting the Force to the Test

Post-Obs scattered-light rem. Posted on Jul 19, 2013 12:19

Following up on this entry we test the Force method on ideal images. This should enable an understanding of the methods actual strengths and weaknesses.

We take an ideal image and fold it with two PSF’s – one with alfa=1.74 and one with 1.76. The 1.76 image is the ‘sharper’ of the two, so we now subject this image to a test where it is folded with various PSFs so that it becomes ‘less sharp’, subtract it from the 1.74 image and look at the difference, expressing it as a percentage of the known intensities (known from the ideal image it is all based on).


Sharper pdf here:

In the upper panel we see the cross-section of the ideal image (black line) and the 1.74 image (red) and the 1.76 image (blue). In the lower panel we see the percentage differences. Black curve is for the difference between the red and blue curves (errors on the DS larger tan 1% everywhere, growing fast as we approach the BS), while the green and red curves show the percentage difference between the 1.74 image and a sequence of images formed by folding the sharper, 1.76, image with PSFs using alfa=1.98, 1.96, … , 1.88. (top green curve has alfa=1.98, next down is red and has alfa=1.96, etc) The difference can, evidently, be less than 1% (for alfa=1.90) but not all over the DS at the same time – there is a tendency for the error to dive faster near the BS and in fact become negative. Near the DS edge to the sky it is possible to have errors in the 0.1-1% interval.

This test is realistic in that the PSF used is developed from real images; in that the procedure of modifying one PSF to another through the use of raising a canonical PSF to a power (alfa) is theoretically underpinned; and in that the brightness levels used in the ideal image are realistic.

The test is unrealistic in assuming that an image with one PSF can be transformed into an image with another PSF by convolution; the procedure of generating PSFs from a canonical PSF through exponentiation is probably only good in the wings of the PSFs.

The test shows that the method is not foolproof. Possibly it is better to first deconvolve an image and then forward convolve it in order to match PSFs – but this procedure needs to be tested on its own.

The test seems to show that for areas near the DS/sky rim a halo can be removed so that errors are in the few tenths of a percent range.



B minus V albedos

Post-Obs scattered-light rem. Posted on Jul 16, 2013 13:01

After some deliberations we now return to the ‘edge fitting method’ and its results. There are 19 nights on which more than 17 B and V pairs of closely spaced images exist. We consider those B and V image pairs with negative lunar phases, and look at how the B minus V (albedos, not colours!) values are distributed against lunar phase. We see this:

pdf here:

The error bars are estimated on the basis of error propagation in the B and V albedos from the edge-fitting uncertainty procedure. We have added small offsets in phase to get separation of points. The above actually contains data from 19 nights. To better see things we calculate the nightly median value Of B and V albedo, and now plot these:

pdf here:

Here, error bars are given by the standard deviation of the nightly values. Each point is labelled with the JD number. We see three night-after-night sequences: 2456073-76, 2456016-17, and 2456045-47. Given the error bars, these sequnces follow the same pattern – a rise in B minus V (albedo) as you go from new moon (left) to half moon (-90 degrees). Since the three sequences painmt the same pattern we are confindet in saying that ‘Something Is Going On Here!’.

At the moment we do not know if this is geophysics in the shape of Earth albedo changing with phase – or some aspect of the halo (which increases from left to right above) being harder to model as phase grows.

Next it may be appropriate to study what the Earth actually looked like in the above three sequences. The negative phases selected for the above plots all correspond to the Sun illuminating Earth from Western Pacific/Australia/East ASia and westwards. As time passes on a given day the Moon sinks further in the West (as seen from Hawaii) and more of Asia contributes to the earthshine. On days in a sequence, at the same local hour, less of Earth is illuminated as seen from the Moon so the contribution to earthshine drops but is more and more ‘sicle-like’. SUmmarizing:

For the negative phases selcted we expect:

a) during a single day – contribution to earthshine by continental areas increases, ocea contributions decrease,
b) same time of day but consequitive days – earthshine contribution comes from areas closer and closer to the edge of Earths disc.

I think this means that – if clouds are evenly distributed – we should see reddening of the albedos during a sequence taken on one day, and a blueing in a series of days.

We see b)! Have we seen a)? Below is a plot of the B albedo minus V albedo values for each of the days in question plotted against JD day fraction.

sharper pdf here:

We see little evidence of any up- or down-turns in this. Perhaps 2456104 shows an upturn – i.e. the opposite of what we thought we’d see. The remaining narrow sequences above seem to show a slight downward trend as the day passes – i.e. reddening. Potentially, clouds dominated these days so that little of the surface was visible?



Force method vs Edge Fitting method: The Fight

Post-Obs scattered-light rem. Posted on Jul 16, 2013 09:28

In understanding this this post, we now compare the results to those from the edge fitting method. We remind thereader that the Force method gives us the difference in B-V [magnitudes] between the BS and the DS, whil ethe dge-fitting method gives us albedos fro Earth in e.g. B and V bands. We look at the same range of phases in the two methods and repeat the relevant ploits here. First the edge-fitting method results:

Upper panel shows us B-band albedo minus V-band albedo against lunar phase (upper panel) and against the average alfa (B-alfa and V-alfa averaged).

We repeat the plot from the Force method here:

Here are the pdfs:

In the upper panels we see a slight dependence on lunar phase – B albedo minus V albedo as well as B-V [mags] do tend to rise towards Full Moon. Note that this implies opposite colours! The rise in B albedo relative to V albedo means that the Earth appears bluer as we approach Full Moon in the edge-fitting method, while larger B-V [mags] in the Force method implies a redder Earth as we approach Full Moon.

In the lower panels we see that the alfas used in the edge fitting method and the derived albedo differences have little relationship with each other, while the ‘incremental alfa’ needed in the Forcing method bears a strong relationship with B-V, as discussed before.

This seems to rule out the worst of the possibilities given in the Force posting – namely that if B and V albedos depended as strongly on alfa as B-V does then we would have a serious problem. In the present situation we are not in that position – but where are we then?

For both methods we seem to have outliers for phases less than -120 degrees. Let us take a closer look at these points. They are from night JD 2456073. The upper plot shows that the B and V albedos we could derive from edge fitting’ differs by .1 between the upper group and the lower group. From the Force method we see that B-V [mags] differs by 0.5 between these two groups.

The images involved have these names :

2456073.7452091MOON_B_AIR_DCR.fits
2456073.7472223MOON_V_AIR_DCR.fits
2456073.7555269MOON_B_AIR_DCR.fits
2456073.7658635MOON_B_AIR_DCR.fits
2456073.7758928MOON_B_AIR_DCR.fits
2456073.7781942MOON_V_AIR_DCR.fits
2456073.7862409MOON_B_AIR_DCR.fits
2456073.7882301MOON_V_AIR_DCR.fits
2456073.7964398MOON_B_AIR_DCR.fits
2456073.7983881MOON_V_AIR_DCR.fits

There are more data points for the edge-fitting method, in the plots above, than there are for the Forcing method. This is because more B and V combinations were picked tested for the former. The images picked were observed withing half an hour of each other. These images should be visually inspected.

[later:] Tried that – it gets messy: differences in the level-differences of the DSs in B and V can be spotted but do seem to depend on knowing the exposure times, and this is one of the things we do not think worked – better to trust the edge-fitting method, since it is a ‘common mode rejecting method’.



Repeated Force method

Post-Obs scattered-light rem. Posted on Jul 14, 2013 09:44

In this blog-entry we discussed a method to force halos to cancel by convolving the sharper of two images in a B/V pair, until their halos matched and could be cancelled by subtraction.

From the many possible B and V image pairs that we have, a number has been selected, and the method applied. The criterion used was that the linear slope on the DS disc should be within 1 sigma of 0. The result, for negative lunar phases, is this:

Here is a sharper pdf:

In the upper frame above, we plot the B-V difference between the BS and the DS. On average, B-V seems to be 0.4 bluer on the DS than on the BS. Given that the BS has B-V near 0.92 [Allen, 3rd ed.] we can convert the above to B-V for the DS by subtracting it from 0.92. That would give us B-V values for the DS near 0.5. Fred Franklin found less blue values since his B-V for DS was 0.17 below the BS value. That is at the lower limit of what we find.

We plot +/-1 error bars – they seem to get bigger as phase approaches Full Moon. We note that the scatter is larger than the errors – so something other than the errors we can estimate are causing the scatter.

Is there a trend in the above? By eye, it seems like there is – as phase approaches FM we have more positive ordinates – implying that the DS gets bluer with lunar phase.

The points fall in ‘columns’ corresponding to single nights – so something is causing a colour-change as the night progresses. All data are of course extinction-corrected so we doubt that the nightly colour-change is due to extinction. Is it a true color-change in earthshine?

We can compare the photometric B-V derived in the above way with B-V differences in albedo found by fitting the DS disc edge.

Now for the lower panel:

In the lower frame we plot B-V against the PSF-parameter alfa needed to bring the sharper image into the same state of blurriness as the less sharp image – so that halo cancellation could occur. There is a very clear relationship between alfa and the B-V achieved. I guess this is bad news? Nature (i.e. the B-V colour of Earth) should not depend on a PSF parameter used in the above convoluted way. Oh dear.

I still think it is worth comparing the present results with the ones based on fitting the DS edge. If we get a contradiction it may mean anything – but a strong agreement would imply that the ‘edge-fitting method’ has some gremlins hiding in it – i.e. that results are very dependent on those alfas.

It is also possibly revealing that the nightly points fall in columns [probably sequences once telescoped out to single nights – will check]. It is telling us something about things that change on an hourly basis – this includes airmass (i.e. extinction) and also the Earth-Moon-Observer aspect – so, still interesting!



Using the Force

Post-Obs scattered-light rem. Posted on Jul 11, 2013 15:47

Measuring B-V on the DS in B and V images with different PSFs: The Case For Forcing them!

We have detected some image-pairs where the B and V halo appear to cancel. We have plenty of other image-pairs where they evidently do not cancel. This is seen by structure on the sky – DS sky in particular as well as ‘slopes’ across the DS in the direction towards the terminator near the BS. We here investigate whether it is possible to ‘force’ the image with a narrower PSF to acquire a halo that resembles the broader image’s halo so that cancellation can occur.

If the differences between the PSF width-parameters (alfa) is small we expect to convolve the sharper image with a quite narrow PSF that only broadens the halo a little. We will use PSFs that are based on our ‘canonical PSF’ and simply raise it to a large number – this will generate a more and more delta-function like peak.

We identify centered pairs of B and V images that are close in time; identify the image with the narrower PSF – by using previously fitted alfa values; we then broaden the sharper image, while conserving flux, and inspect the difference between the B and V images. The results of such a trial is shown in the Figure:

Here is a less fuzzy pdf file of the above plot:

We see 15 panels showing a ‘slice’ through the disc centre, of B-V (magnitudes). Each panel has an increasing power (alfa) used to raise the canonical PSF to – at first we raise it to 1.5, then to 1.6, and so on. A smaller power will broaden the PSF a lot and thus give us ‘fuzzier’ images. We expect that when the image is too fuzzy we will see slopes in the B-V slice – just as when two observed images in a pair are subtracted and one has a broader PSF than the other. As alfa increases in the figure we reach a point where it is so large that the sharper image no longer is made appreciable more fuzzy – in the limit of large alfa it becomes an almost identical copy of the original image – hence, towards the end of the sequence of plots we essentially see what the slice across the original B-V image looked like.

In each image we notice the DS to the left and the BS to the right – both situated between the two vertical dashed lines, denoting edge of disc. We see a ‘level difference’ between the DS and the BS. We notice the slope of the DS – upwards for large alfa and downwards for smaller alfa – e.g. at alfa=1.7. Somewhere near alfa=1.8 and before alfa=1.9 the slope is horizontal.

We expect the real DS to be ‘flat’ [we need to check this against realistic models] so when alfa is near 1.85 it seems we have induced enough additional fuzziness in the sharper of the two images that the halos cancel and the level difference between DS and BS is what it is in reality, [We need to test this on images with known properties: Student Project!].

We notice also that the level difference depends on alfa, even when alfa has gone past the ‘now DS is flat’ point. This is a warning that incorrect estimation of
the limiting alfa value may give us spurious results for delta(B-V)! We estimate the slope on the DS (between vertical dotted lines) and look for the values of alfa where the DS slope is not statistically different from zero at the 3 sigma limit. We also do this for the BS. We find midpoints of the fitted DS line and the fitted BS line (red lines in uppermost plot) and take the difference bétween these, forming a sort of ‘+/- 3 sigma interval of confidence’ for the B-V difference between BS and DS. This is plotted here:

Here is the pdf:

We see thaat the generous +/-3 sigma interval allows quite a range in B-V – and also that if we do not estimate the right alfa in the above procedure – but make it too large then we will underestimate B-V.

For the 3-sigma limits we squint at the above graph and read off upper and lower B-V limits for the BS/DS difference: 0.5 to 0.7.

All of the above has been done for an average of an 11-row slice across the middle of the disc. The method could in general be extended to treat broader strips, or the whole ds – planes could be fitted instead of lines.
First, though, it is necessary to understand if the above is even correct! We need to look at artificial images of the Moon. These should have their own “B-V colours” then we fold the B and V images with seperate PSFs and try to force one to be as fuzzy as the other, and so on. The actual slope of the DS, when fitted with lines or planes could then be estimated and in fact used as the goal slopes for the above procedure. Work work work!



Wrong time stamp

Real World Problems Posted on Jul 05, 2013 15:44

We have noticed that some images have a JD time that differs from the JD implied by the time information in the FITS file header. By checking (almost) all files that have a JD in the filename against the time in the header the following list of suspect images are revealed:

2455633.8734508
2455633.8734679
2455633.8734848
2455633.8735015
2455641.6260333
2455643.4725818
2455643.4800437
2455643.4817785
2455643.4846651
2455643.4866171
2455643.4891739
2455643.4911491
2455643.5007650

This list is the JD from the filename – the value is 1 hour larger than the JD implied by the DATE field in the FITS header. None of these appear on the list of good images as determined by Chris. and his ‘tunelling’ code which inspects magnitudes vs lunar phase.



Brightness of daylight sky

Post-Obs scattered-light rem. Posted on Jul 04, 2013 12:03

In V band, extinction is about 0.1 mag/airmass.

The Sun’s apparent brightness is -26.74.

0.1 mag of extinction means that about 10% of the light is taken from the sun and spread over the sky, about 21,000 square degrees. Half of this is scattered out of the atmosphere, half down to make the blue sky.

The integrated magnitude of the sky would then be of order -26.74 + 2.5*log10(10*2) = -23.5.. i.e. 3.2 magnitudes dimmer than the Sun.

Spreading this over half the sky (21000 square degrees or 2.7E11 square arcseconds) gives a reduction in surface brightness by 2.5*log10(2.7E11) = 28.6 magnitudes.

This yields a daytime sky brightness in V band of -24.24+28.6 = 4 mag square arcsecond.

If we have any daytime exposures with the ES telescope — we could check this and get numbers in other bands as well…



Meteorological conditions and Good Seeing

Met sensor Posted on Jul 04, 2013 09:06

On JD2455945 conditions were such that the BS halo in the B and V images cancelled almost perfectly, giving us the possibility of seeing the DS colour itself. We have a link to material discussing scintillations here.

Is it possible to understand which meteorological conditions led to this unusual situation? At the MLO there is a meteorology tower and data are taken and stored for every minute. A link to the data is here.

A plot of selected parametrs for the night in question is here:

The plot shows 3 days on either side of the observation moment. A more legible pdf version is here

Note:

a) The rising pressure. The daily cycle is due to heating of the atmosphere,

b) that the observation occured towards the end of the night – temperature at ground level was dropping,

c) the vertical temperature gradient was positive (it was indeed almost the maximum seen that night) – the air was warmer higher up,

d) wind speed was relatively low,

e) relative humidity was relatively low.

We next plot ‘alfa’ (the parameter that sets the width of the PSF in our model images, found by fitting), against four of the above parameters:

There seems to be no strong pattern. Slightly, there may be an indication that low vertical T gradients allow for larger alfas, and that low relative_humidity allows for larger alfas. Ignoring the outliers at low alfa (a sign of a very poor fit or night) we look closely at the dependence on relative_humidity:

It would seem that for RH between 10 and 20% a large value of alfa is obtainable. Large lafa implies a narrow PSF. However, fo rthe nighjt JD2455945 (where the halos cancelled) we have unremarkable conditions: alfa is small (1.54), pressure is medium (680 hPa), relative humidity is medium (42%) – only wind speed is lower (3 m/s) than most other data points. Th ethree values of alfa always determined from the same image are very stable, however, differieng by 0.001 only.

Speculating wildly: Is it because alfa is small that we get halo cancellation? With a small alfa the halos widen and perhaps their ‘tails’ cancel better?

————————–

Here are some papers that discuss meteorological conditions and ‘good seeing’ conditions:

http://adsabs.harvard.edu/abs/1974Obs….94…14M
http://www.aanda.org/articles/aa/pdf/2004/30/aa0215-04.pdf



More Moon and Earthshine colours

Post-Obs scattered-light rem. Posted on Jul 03, 2013 10:29



Is it possible to ab initio calculate the expected colour of earthshine? Earthshine, in the visual band, is due to diffuse scattering from clouds and the surface of the Earth, and Rayleigh scattering from molecules in the atmosphere. Can we simply calculate the expected B-V colour of Earth, seen from space?

If we assume that the diffuse scattering on clouds preserves the colour of the light (I think this is true – clouds look ‘white’) and if we assume that clouds cover most of Earth (a fair assumption given that the oceans are dark and cover 70% of the Earth) then we can make a model of Earth’s spectrum consisting of a fraction of the solar spectrum plus the complementary fraction of light ‘blued’ by Rayleigh scattering. This model assumes no ‘true absorption’ in the visible bands (also a fair assumption; ozone may actually absorb at UV wavelengths but the solar flux at those wavelengths is small).

By considering the Rayleigh effect and imposing flux conservation we get the following B-V colours for the spectrum of Earth:

f B-V
——–
0.0 0.63
0.1 0.51
0.2 0.40
0.3 0.29
0.4 0.20

where ‘f’ is the fraction of the beam that undergoes Rayleigh scattering. We have elsewhere estimated the B-V colour of Earth (here we do not mean ‘earthshine as measured on the DS’) to be 0.49 +/- 0.02. That would seem to imply that we have a correspondence with (crude) theory if 10% of the beam undergoes Rayleigh scattering.

Since most photometrically measured extinction coefficients are in the 0.1 – 0.2 mags/airmass range it does not seem unlikely.

In meteorological observations pyranometers are sometimes used to monitor the clarity of the air during the day – they typically measure the global radiation (all wavelengths, all directions – i.e. from the Sun and from the sky) and the ‘diffuse’ radiation – i.e. only the part of the light not coming directly from the Sun. Perhaps it is possible to estimate the fraction of light removed from the incoming beam, by scattering, using such, essentially bolometric measurements?

Using a SURFRAD pyranometer data set from Boulder CO, USA, I determine the noontime ratio of Diffuse to Global radiation, and plot it for each day of a whole year:

There is a great deal of variability! This is due to clouds. On perfectly clear days I guess that the ratio of diffuse to global radiation is at a minimum since the contribution to the diffuse is only from the blue sky while the decrease of the direct is also only the Rayleigh. Clouds tend to increase the former and decrease the latter giving a larger ratio.

The incoming beam contributes to the diffuse radiation and to the global radiation. The same amount of light scatters down to Earth as scatters into space so:

Incoming = Global + Diffuse

Then D/I = D/(D+G)

Looking at the plot it is clear that near the middle of the year you
can see D/(D+G) near 0.1. This is not exactly the same as saying ‘the
bolometric extinction coefficient is 0.1’ but we are close.

Problematic is the large amount of clouds – I shall look for a station that has less clouds!

It seems that Desert Rock Arizona is less cloudy:

Notice that we are getting D/(D+G) as low as 0.07. The altitude of Desert Rock AZ is 1007 m, while the station near Boulder, above is at 1/4 the altitude (213 m). Does that explain the different minimum ratio?

At MLO they publish a transmission coefficient which is approximately the same we are trying to look at – it would seem that at MLO the transmission can be 92-94% when there is no volcanic dust: http://www.esrl.noaa.gov/gmd/grad/mloapt.html



Colour of sky at night

Links to sites and software Posted on Jul 02, 2013 15:18

There is a page that discusses the B-V colour of the sky at night: http://www2b.abc.net.au/science/k2/stn/archives/archive8/newposts/60/topic60381.shtm



Fast trends in bias?

Bias and Flat fields Posted on Jun 17, 2013 09:03

We rely on subtracting ‘smooth scaled superbias’ fields from science frames, because of the 20-minute thermostat-oscillation in the bias mean, and a wish to avoid adding noise by subtraction of noisy bias fields. We generate the scaling factor for the superbias by averaging the bias frames taken just before and after the science frame. We seem to be assuming that the bias mean does not change – or does not change irregularly – during this procedure. Let us examine that assumption.

We take most bias frames as single frames, but happen to have 50-image stacks here:

2455643.4932200DARK_DARK-50ms.fits.gz
2455643.4957035DARK_DARK-500ms.fits.gz
2455643.4942722DARK_DARK-300ms.fits.gz
2455643.4949637DARK_DARK-400ms.fits.gz

We calculate the average of each subimage in these stacks and plot the results:


The camera can take many images per second in ‘kinetic mode’ so the above sequence last something like 10 seconds each. There is evidently some slight trend during that time – at the 0.05% (or 0.15 counts) level compared to the bias mean. This may not seem much, but if the scaled superbias mean is wrong by 0.15 cts it can mean an error on the DS flux of many percent – because the DS fluxes we are using are at anything from near 1 count above sky to maybe 10 counts above sky. For a DS at 1 count the 0.15 count error is 15% while it is 1.5% for the 10-count DS. This is quite serious.

It may be the reason that the halo-removing methods based on ‘subtracting a model halo’ did not work well – they relied on the bias being correctly subtracted previously. The ‘profile fitting method’ apparently worked better and this is probably because it inherently contains a fit to the sky (and thus bias, and also any insufficiently subtracted, bias).

Bias subtraction does more than subtract a mean level, of course – there is also some structure – notably a 0.2 count raised edge along the vertical sides, stretching some 10% into the field. Some row asymmetry is also evident, along with a very slight ‘ripple’ in the column direction (Henriette’s thesis has all this).

Subtracting the scaled superbias is therefore mostly a good thing, but it should be realised that important level-errors may be present causing blind reliance on the ‘complete bias removal’ to be dangerous – better to allow for an additional small pedestal in your modelling.



Reach of halo

Post-Obs scattered-light rem. Posted on Jun 14, 2013 10:02

In considering B-V images, as here we are have to know how far the BS halo reaches onto the DS. We have looked at that before, here. We now revisit this issue. Here is a contour plot of the DS and the halo, with colours so that we can see how far the halo is likely to reach – we see it reach into the sky on the BS – how far do we think it reaches onto the DS?

Consider the yellow end of the greens, for instance – on the sky that contour lies 40-50 pixels from the BS rim – on the DS it lies adjacent to the red area which is the BS. The yellow-green contour therefore does not interfere much with most of the DS. The blue contours on the DS, howvere, are represented on the sky far away from the BS – so the blue areas on the DS may be interfered with by that part of the halo.

The above is V-band image. In B-V images we rely on much of the halo being similar in B and V and thus cancelling. The above noticeable gradient in DS brightness is absent in the B-V image.



Is there Life?

Real World Problems Posted on Jun 12, 2013 19:30

With the whole telescope system back at DMI we are working to bring it back to life. After that we will try to fix the problems that beset us.

So far (latest activity on top) we have this:

July 8 2013: All PCs on internet reachable from outside, and so is the iboot and the NAS c an be seen from various machines.

July 4 2013: ibootbar is now reachable from woof by using, in browser on woof, ‘ibootbar’. On egregious you need ‘ibootbar.usr.local’. Eigil says woof is reachable from home using woof instead of via nordlyset in ssh menu. egregious should also be thusly reachable from ssh.

July 2 2013: Eigil Pedersen has made it possible to reach the woof linux machine by giving it an IP number. It can in turn see the iBoot viaits 192 IP number. Working on the NAS.

June 24 2013: Located the Aurora cloud sensor, and downloaded software. Will try to set up on Wine/Linux machine.

June 20 2013: Working to install static IP-numbers. Connected DEC-axis motor and its limit-switch so that we do not have to rely on dubious short. Devices still do not react when acessed from LabView.

18 June 2013: We figured out that the limit switches must be connected – then the mount comes alive. LabView still freezes when you click on whatever.

June 14 2013: Wired up part of the mount head with the Dome Breakout Box. Still need to understand why LabView is talking about “no licenses” for “Report Generation Toolkit” and the “Internet Toolkit”. Where were the licenses before?

June 12 2013: Booted up the Watchdog, woof and PXI computers. Labview looks strange – some VIs are not working and then none will run?

June 11 2013: Reset all the 115V selectors to 230 V and applied power – nothing blew up!

May/June 2013: Uncrated the control rack and the telescope tube. Inventoried the other boxes.



Aberdeen

Showcase images and animations Posted on Jun 12, 2013 19:23

We help illustrate the theme of ‘simplicity’ for Aberdeen:

http://www.simplyaberdeen.com/simply_being.php#.UbiukM7CVic



Need to flatten?

Bias and Flat fields Posted on Jun 10, 2013 13:01

These are calibrated B-V images from JD2455945.17, as explained at this link.

The one on the left is generated WITH flatfielding of the B and V images used, while the one on the right is generated WITHOUT flatfielding. They are almost identical, but not quite. Here is the difference between the two:

The pattern is the expected pattern from the CCD chip. The histogram of the values is here:

So there is a difference which is distributed around 0 with S.D. 5 millimagnitudes. The effect of flattening would appear to be small – but recall that these are difference images.

Apart from various problems related to aligning images it is rather nice to work with difference images – problems tend to cancel out! However, note in the above how some of the structure on the DS in the B-V image looks like the features of the falt field – as if perhaps the flatfield we used did nothave enoughj ‘amplitude’ in the stripes.



B-V images, revisited

Post-Obs scattered-light rem. Posted on Jun 06, 2013 14:20

[Update: See note at end!]

In this post Chris showed that the night of 2455945.1xxx has halos, in B and V, that seem to be at a constant distance across the DS. This implies that the ‘gradient problem’ is absent on this night – perhaps because the night was very clear or the haloes in B and V almost identical. [Inspection of the ‘alfa’ for the profiles shows that V images had alfa in the range 1.547–1.549 while B images had alfa in the range 1.544–1.547. These values seem close but halo shape is very sensitive to alfa, so analyze first!]. It is possible to generate a B-V image for this night using the only good images for B and V (in the sense of ‘being on Chris list of good images’, as explained elsewhere). We used one B and one V image:

2455945.1776847MOON_V_AIR_DCR.fits
2455945.1760145MOON_B_AIR_DCR.fits

both bias subtracted but not flat-fielded. Exposure times were taken from the image headers.

This is the result:

The colours are of course chosen arbitrarily, to aid excitement and imagination! For this image we corrected for extinction using

Vinst(idx) = -2.5*alog10(Vim(idx)) – Vam*0.1 ; kV=0.1
Binst(idx) = -2.5*alog10(Bim(idx)) – Bam*0.15 ; kB=0.15

and calculated B and V images from the B and V instrumental images, in an iterative manner:

BminusV=Vinst*0.0+0.92 ; BminusV is an IMAGE
for iter=0,10,1 do begin
V = Vinst + 15.07 – 0.05*(BminusV)
B = Binst + 14.75 + 0.21*(BminusV)
BminusV(idx)=B(idx)-V(idx)
print,iter,mean(BminusV(idx))
endfor

Here, ‘idx’ is a pointer that selects for all pixels on the lunar disc – both DS and BS.
Vonvergence was swift and independent of initial value. Convergence was to B-V=0.925.

A plot of a ‘slice’a cross the disc above is here:

We see a BS B-V value near the dashed line at 0.92 and a DS value between 0.75 to 0.8.

The colour image reveals some structure on the DS. It seems we see a faint flatf ield pattern? This should be looked at. On the BS colour differences eem to be lunar in origin and coul dbe compared to the literature on this subject.

The deep ‘cut’ in the middle is due to some small-distance difference in the B and V halos, we think.

Until we are told otherwise we think this is the first map or image showing the colour of the Dark Side of the Moon, which is also an album by Pink Floyd.

Images in other colours may be fothcoming.

Another version of the image above, here with a legend is here:
On the night of JD2455945.1ish the Earth was iilluminated over South America and shone on the Moon. Seen from the Moon the Earth looked like this, from the Fourmilab Earth Viewer:

So, mainly Pacific Ocean but also also all of South America. Need some satellite images of this to see how the clouds were distributed!

Note added later: The (far) above is unfortunately quite sensitive to how well the images are aligned. This needs to be looked at before anything else.

Here is a nice link to images of the
colour of the Moon obtained in sunshine and with simple DSLR cameras –
i.e. jpeg files: http://www.datarescue.com/life/kepler/moontests/raw_vs_jpeg.html



Data inventory – non-lunar images

Observing log Posted on Jun 06, 2013 10:23

Here is a list of non-lunar images taken at MLO. There are stars, clusters, galaxies, and planets and asteroids. Many are suitable for studies of extinction.



Images of telescope

Showcase images and animations Posted on May 23, 2013 10:47

Here are some images of the telescope assembled – perhaps it is possible to use the images to understand which plug goes where? Sharper jpegs are in /home/pth/SCIENCEPROJECTS/EARTHSHINE/JPEGS/ – they are the ‘imgXXX.jpg’ files.

I’ve also added images from the construction in Lund – those images are in same place as above, but are called imagexxx.jpg (xxx from 332 to 347).

This link shows more images from the telescope on MLO.

Here is an image of the cabling at the back of the CCD, during MLO.



Inventory of boxes

Real World Problems Posted on May 23, 2013 09:30

Largest box:

Telescope tube plus its attached cables.
Rack of electronics.
Very long and thick bundle of cables.
Rod for Hohlraum source + foot

Smaller Wood box:

LCD monitor + VGA cable
Laser printer Samsung ML-2525
Small white box: Spare lamps + relay
Holhraum Sphere in its own box
Box of fan filters for rack
2 x Shutter boards + Edmund pack of flat soft things + a small mw laser + a filter labelled ‘IR cut’
European power supply for Axis camera
2 Axis cameras with robofucus lenses, and additional lenses and mounting brackets.
Weather box:
Rain and light sensor – DMI?
DMI anemometer
Väisala device in ‘chineese tower’ affair.
DMI IR rain sensor
Väisala humidity sensor HMT100
Cables + spare PXI parts + bracket for adjusting SKE
Uniblitz shutter # VS25ST1-100
Vincent Associates 710P Shutter Interconnect Cable
3 Thor labs FW drivers / electronics. Model FW102B. Implies that we have
three Thor labs FWs – the colour FW, the ND FW, and one more – somewhere!
Box of mixed cables – VGA + DVI + 2xUSB repeaters
2 sealed metal envelopes with “D.C.D. Panel Mount SA 4 COMP”
Tools for shutter tuning
Spare Ferrorperm Knife edges on glass.
Mixed VGA, DVI, USB cables along with 2 USB extender cables
US 115V power cables – to PC-type plugs.

Blue box:

Polar alignment scope
Point source lamp + scope
CCD camera
Magma + mounting HW
Oscilloscope
Keyboards
Dome breakout box
Mount parts:
head
mounting plate
Counter weight
base w. electronics
Power for Axis camera (perhaps it moved to Wooden box?9
A part of the PXI – probably the spare or original PC
Spare HD for PXI

Where are:
Mount handbox controller?
Aurora cloud sensor?
Various interface cards for camera to PXI?
Additional external cameras?



Reviving the system

Real World Problems Posted on May 23, 2013 08:34

Here is a collection of links useful in our attempts to power up the rack of control system, at DMI:

iBootBar : http://dataprobe.com/support/index.html



Ghost and Dragging

Optical design Posted on May 21, 2013 07:58

Here are example sof a ‘ghost’ and ‘dragging’:


The dragging is due to no shutter being used – i.e. readout was only way to terminate exposure and hence frame was illuminated as the image was shifted to the ‘hidden register’ We hav a frame-transfer camera. The image is from the test phase in Lund and no shutter was installed then.

The BS ghost is faintly vissible: It is a copy of the BS and the lower cusp is seen poking out slightly down and to the left of the real BS. This was due to the CCD camera being aligned almost perfectly along the optical axis of the system – the ghost arises as the shiny CCD surface reflects light back into the optical system and reflections are produced at all optical surfaces – the back of the second secondary lens, the front of the second secondary lens, the back of the first secondary lens, the front of the first secondary lens and the back of the primary objective and its various surfaces. At MLO the camera was tilted at an angle so that the reflection from the CCD did not go back up the system. This tilting did not appear to have any adverse effects on focus etc.

Images from lab hard disc recovered from MLO.



New blog!

To-do list Posted on May 20, 2013 12:39

We are starting a new blog over at

nextelescope.thejll.com

where issues related to how to design (and one day, hopefully, build) a better eshine telescope are considered.

The present blog will from now on only discuss how to use the data obtained from the Hawaii telescope. 

 

OK, we updated that site with an entry about a NASA proposal.



Expected albedo variability

From flux to Albedo Posted on May 06, 2013 15:42

Some of the notes discussed below, in the next several postings, are collected in this doc:

There are some considerations of how how much albedo change we expect during global warming – and the detectability of such changes are discussed.



Is Earth Lambertian?

From flux to Albedo Posted on May 06, 2013 09:17

No.

But we – and BBSO – calculate albedo by comparing the earthshine measured on the Moon to the intensity of earthshine predicted by a terrestrial model based on a uniform Lambert sphere.

We test how well this works by taking a series of GERB satellite data for several weeks across a year and extract the total flux from the whole-disk images. The MSG satellite bearing the GERB instrument floats over lon,lat=0,0 so always sees the same part of Earth. Johanne has extracted images for every fifteen minutes for several weeks in a year. We plot that (top panel, below).

We use the eshine synthetic code Hans wrote to generate Lambertian images for a full cycle of sunrise over earth. As the code is based on what the Earth looks like from the Moon we pretend that one month is like one day and thereby can extract the phase law for the Lambertian uniform-albedo Earth in order to compare it to the GERB data. We also plot that (second panel, below). [Note that the difference between view from satellite and view from Moon may be important: Sun-Earth-Viewpoint angles should be the same, and if the Sun and Moon are not at similar latitudes as Sun and Satellite we could be generating artefacts in what follows: we should see if we can use the synthetic code for satellite viewpoints. For now we ought to find dates when the Moon was at latitude 0 (like the satellite) and the Sun also same latitudes – tricky to do.]

Lastly we divide Gerb fluxes by Lambertian fluxes, correct for the fact that Geostationary orbit and the Moon are at different distances and multiply that corrected ratio by the uniform albedo used in the Lambertian models. We plot that (bottom panel, below).


We see that the albedo does not come out constant. This is not surprising since the Earth has real clouds that drift around – but that is only what gives the thickness of the thick line of points in the last plot above. The ‘wiggles’ are due to the inadequacy of the Lambertian model. Near New Earth (Full Moon: never observed) the derived albedo rises. Near Full Earth (New Moon: attractive to observe due to strong eshine, but difficult due to Moon close to Sun) the albedo is flat. At intermediary values (20% and 80% of the cycle) the Lambertian albedo is relatively high so that the derived albedo is lowered.

How can we use these insights to understand what Johanne shows in plots of how derived albedo evolves during the nights?

Our aspiration is that the above can give insight into

1) the ‘phase dependency’ we see in derived albedos when we plot all data corresponding to all phases during the morning branch – i.e. Moon setting over Western Pacific/Australia, and

2) the nightly tendency to have falling albedo through the night, for that same branch.

As for 1 the reader should look in the May 3 presentation at slide 22; as for 2 the reader shoul dlook at slide 23.

We have to figure out whther the above plots explain any of these sightings. Could the almost quadratic phase dependency seen when all data are plotted be due to the ‘dip’ near 20 and 80% of the cycle? Could the ‘nightly slopes’ be due to the same?



How variable is Nature?

Real World Problems Posted on May 05, 2013 14:31

Our efforts to measure albedo precisely and accurately are limited by the natural variability. Albedo is inherently a quite variable property of the Earth – viewed day-to-day [see presentation linked to below for details on the GERB data variability!]. Just how much does global-mean albedo vary – in the short run and over many years?

We have CERES albedo data for the period 2000 to 2012. It is given as monthly mean data for a 360×180 degree grid. We take this and calculate global-mean monthly values. Some results are here:

Data downloaded from: http://ceres.larc.nasa.gov/cmip5_data.php
these are the monthly-mean data prepared by NASA for the CMIP5 effort.

I calculate the global weighted means for the given monthly-mean values
of upward and incoming shortwave light at TOA.

The linfit slope is:

Slope: 5.4920926e-08 +/- 5.7758642e-07 in albedo units/day.

This is an INsignificant slope – per decade it would amount to 0.09 %
of the mean albedo. Using, instead of the slope, the +/- errors on the slope we get:
+/- 0.97 % per decade.

The mean we get is:

Mean albedo: 0.218214

this is small. Publications using these data say: 0.29 … there is some
problem with accounting for regions’ weights. I omit all areas where the
incoming flux at TOA is less than 12 W/m² – this helps avoid Inf’s and
NaN’s. I weight each area with the cosine of the latitude of cell middle.

Month Albedo S.D. S.D. as % of mean
1 0.228720 +/- 0.000820013 or, +/- 0.358522 %.
2 0.222061 +/- 0.000866032 or, +/- 0.389997 %.
3 0.206976 +/- 0.000856246 or, +/- 0.413694 %.
4 0.215791 +/- 0.000886397 or, +/- 0.410766 %.
5 0.219209 +/- 0.00129653 or, +/- 0.591460 %.
6 0.219878 +/- 0.00122390 or, +/- 0.556628 %.
7 0.215800 +/- 0.000900811 or, +/- 0.417429 %.
8 0.212036 +/- 0.000893605 or, +/- 0.421440 %.
9 0.203454 +/- 0.000899703 or, +/- 0.442214 %.
10 0.215973 +/- 0.00123064 or, +/- 0.569814 %.
11 0.227462 +/- 0.000582274 or, +/- 0.255988 %.
12 0.231050 +/- 0.00111927 or, +/- 0.484429 %.

What do we learn?

We learn that albedo is remarkably constant when observed by satellite. There is no discernible slope to the data but if we use the 1 sigma uncertainties on the slope as upper limits we find that per decade the albedo has changed less than 1%.

The monthly means follow an understandable annual cycle (maxima in NH and SH winters with minima in March and September). The spread around monthly means amount to 0.25 to 0.6% of the monthly mean value.

Climatologically it is an open question whether albedo ought to change with climate drift. During the observing period global mean surface temperatures have changed by about +0.1 degree C [see http://www.ncdc.noaa.gov/sotc/service/global/global-land-ocean-mntp-anom/201101-201112.png ]. This is during the ‘hesitation period’ that is much discussed presently. During other decades mean T has risen much more – but we have no albedo data from these periods.

Using the above 1 sigma upper limits on slope of 1% per decade we see that if the slope is due to changes in T then the relationship is 1% per 0.1 degree or 10% per degree. This is based on an upper limit and the true value is closer to 0% per degree.

Note that the previous argument is unrelated to the EBM based relationship that works for equilibrium climate only – there, and only there, the expected relationship is -1% per degree.

So, what does that give us? If we were to observe a larger slope we could use the data in the “satellites are getting it wrong” mode – as Pallé et al did for a while. If we measure no slope we can hope to set more stringent limits to the slope than the above satellite values do – can we determine the slope of global mean albedo to better than the 1% per decade above? In this presentation I found upper limits of 0.2% per decade based on a null hypothesis of ‘no albedo change’ and realistic observing limitations. The numbers used were based on Frida Benders CERES data, but they do not differ enormously from the present, longer, ones.

So can we reach 0.2% error per decade, observationally?

This requires a discussion of the single-frame errors we get as well as the period-mean data we can expect. More alter!



Presentation November 2012

Showcase images and animations Posted on May 05, 2013 05:36

Presentation at Swinburne (Melbourne) in November 2012



Presentation May 3 2013

Showcase images and animations Posted on May 03, 2013 12:31

This earthshine presentation was given at DMI, on May 3 2013.



B-V images

Post-Obs scattered-light rem. Posted on Apr 29, 2013 10:29

We have previously considered B-V images of the Moon. This was done with ‘raw’ images – that is, images where the halo had not been removed. Since we have the BBSO linear method implemented and since it does clean up the DS we can also calculate B-V images for the Moon based on these.

We have 55 pairs of B and V images thata re close in time (about 1 houror less apart). Using the standard star calibration relationships that Chris worked out fropm NGC6633 standard stars, we can convert images to instrumental magnitud eimages and from there to calibrated B and V magnitud eimages. We also corrected for extinction since the images were not obtained at the same time.

SInce the calibration relationships depend on B-V we have to assume some B-V values and iterate (Chris solves algebraically). The iterations converge quickly. We use onlythe brightest pixels in each image – i.e. the pixels delineating the BS – for calculating the mean B and mean V values needed to update B-V in each iteration.

The values for BS B-V that we converge to have this distribution:

The mean B-V=0.989, and the S.D.=0.019. The accepted value – e.g. Allen (4th ed), Table 12.14, gives B-V=0.92 (van den Bergh observations?). We therefore have a significant discrepancy. It should probably be noted that our values come from phases near 90 degrees, while the Allen values may be from ‘Full Moon’ conditions.

If we accept the above B-V (BS) values at face value we can continue:

The Sun has B-V=0.642 (Holmberg et al, MNRAS 2006). One reflection off the Moon reddends this value by 0.989 – 0.642 = 0.347. This value will also apply to earthshine that is observed after one reflection off the Moon, even if it is the DS. [We ignore here any colour-dependencies in the lunar surface mare vs highlands!]

If we can estimate the B-V of earthshine as seen on the lunar surface, we can work backwards to what the B-V of that light was before it struck the Moon – it will be the observed value minus 0.347.

Before trying this we need to understand to which degree the use of BBSO linear images, as opposed to ‘raw’ images, has helped us observe the true colour of the DS – has an important amount of the BS halo been removed from the DS?

We generate centered B-V images and plot the average of 20 rows across the middle of the images:

We see two panels – each panel is the result of using a fixed B image and two different V images – all three taken a short time apart. The black curve is the run of B-V values in the ‘raw’ image – that is, the image where no effort has been made to remove the BS halo. The red curve is from images cleaned with the BBSO linear method. The deep jag in the middle is the BS/DS terminator. The DS is to the left of this and the BS to the right. Since the BS is not altered by the BBSO linear method the red curve covers the black curve on the BS.

On the DS we see that cleaning the image has resulted in a slight reddening of the DS – it was ‘too blue’ in the red images.

We also see that the ‘linear gradient’ in B-V across the DS is unaltered qualitatively by the cleaning of the image. Why?

If we push on, ignoring the not-yet-understood gardient, and assume that the part of the DS closest to the sky has an un-polluted B-V value then we can calculate the colour of earthshine before it strikes the Moon, as explained above. First we extracted DS B-V values for that part of the DS disk that is to the left of 90% of the vertical columns on the disk. These values were on average 0.29 +/- 0.05 below the BS value.

If the BS value is given the canonical B-V=0.92, then we have a B-V for the DS of 0.63.

Franklin (JGR 72, no 11, p.2963-, 1967) measured B and V repeatedly on the DS. The difference between his mean B and his mean V is 0.64. We are close, but we are worried about scattered light!

Subtracting the effect of reflection once on the Moon brings us to the value for B-V of earthshine, before it strikes the Moon, that is, as it would be seen in space:

B-V_ES = 0.28.

There is one published B-V value for earthshine, based on Mariner II data in the 1960s. The paper is http://adsabs.harvard.edu/abs/1964JGR….69.4661W by Wildey. Unfortunately I cannot make head or tail of that paper!

Playing a bit more with the above, we can consider the effect of Earth on light – the Sun has B-V=0.642 when it strikes Earth. If the earthshine has B-V=0.28, then the bluing effect of Earth is 0.28-0.642 = -0.36 in B-V.



Updated ‘best list’

Real World Problems Posted on Apr 24, 2013 10:34

We update yet again the list of ‘best images’ that Chris has generated by inspection of compliance of absolute magnitudes against lunar phase.

We can removea few more images by hand inspection. We found about 10 that have ‘cable in view’ as well as various near-horizon problems. The list is here and now contains 525 images:

We note that Johanne is working her way through many images and finding ‘bad focus’ cases. Since we believe these are coincident with ‘not the right filter acquired’ cases, we shall eventually be further updating the list of best images.



Is RON stationary?

Data reduction issues Posted on Apr 17, 2013 16:52

We are using co-add mode – that is, we take stacks of 100 images and co-add them to increase the SNR. Since there is RON (read Out Noise) in each frame – about 2 ADU/pixel, tests have shown – it is important to know whether a sequence of added frames have a mean that converges.

We now test this directly on the stack of 100 images: “2456015.7742682MOON_V_AIR”. We extract a small square of size (2*w+1) around regularly spaced points covering the image plane and calculate a mean subframe (averaging along the stack direction), the average value of that mean subframe (a scalar for each of n coadded subframes), and plot these. These are shown in the PDF file:

The top panel in each page shows where the point was chosen. w was 4 so subframes are 9 pixels by 9 pixels, centred on the point. Second panel in each page shows how the mean value of the average subframes evolves, along with estimates of the +/-1 sigma error bounds (calculated based on the value the series converges to, or at least evolves to, in the last 10 steps of the series; the known RON (about 2.4 ADU/pixel in all observations, estimated from bias frames differences), the number of pixels in each subframe and the number of subframes co-added). Last panel shows the mean value of each sub-frame used (along the stack direction).

What do we see? In page 1 a point on the sky has been selected and we see a mean value that evolves inside the expected error bounds, and we see that the subframes have mean values with some spread, but no trend.

Page 11 shows a point on the DS. Nice evolution.

Page 24 is on the BS and is very hard to explain. This holds also for points ‘near the BS’.

It seems we have little to worry about on the sky and on the DS, but on the BS we see strange evolution of the running mean! We have before touched on such subjects in this blog, when we tested effects of alignment of images – perhaps this problem was hiding inside the other problem?

[added later:] Chris asked a good question. Here is a plot of the running mean of the whole frame, as frames are taken from the stack. The value is expressed as a percentage deviation from the middle value on the curve – about 1378 counts. A small drift is seen at the 0.05% level from one frame at left to all frames at right. I think this could be due to sky variations – or slight drift of the Moon inside the frame causing ‘light to fall off the edge’. If drift is the explanation we may have an answer for why the mean in a much smaller sub-frame, when near the BS, drifts so much more – image brightness gradients are being sampled in a small subframe. Experiment at top should be redone but with drifts taken out. No large drift was evident sp I guess we are learning that a small drift can be very important!

This has an implication for how we measure the albedo – if we use ‘DS patch divided by BS patch’ we run a risk . if we use ‘DS patch divided by total flux’ we are much better off.



PSF profiles from the NGC6633 images

Post-Obs scattered-light rem. Posted on Apr 07, 2013 09:38

The PSF of the VE2-band Arcturus data deviated markedly from the other bands (next to previous post). We check on this further by looking at the PSF profiles of the stars in NGC6633, which were used to calibrate the filters (in this post).

We plot the central flux normalised PSFs in B, VE1, VE2 and IRCUT versus V (shown in green in each panel). The PSF is formed from 95 different stars spread over the frame in each case: e.g. this shows the stars in the IRCUT image.

The PSFs look like this:

Same plot as above but in log-linear form:

None of the PSFs are as sharp as the V band one — they all have more
flux at greater distance from the center. Interestingly, the VE2 band
profile is not discrepant, so the conclusion is that the IRCUT data for
Arcturus may simply have been out of focus.

It would seem the lesson from this is that the PSF can vary quite a bit — but note well that this is in the core only — as this is not a test of the extended power-law wings at all, as we are only probing to a radius of about 4 pixel (~30 arcsec) with these data.



V-VE1 colour map

Post-Obs scattered-light rem. Posted on Apr 07, 2013 03:59

In the previous post, we noticed that the VE1 and IRCUT filters have very similar profiles to the V band — at least for the data we obtained pointing at Arcturus.

Here we show a colour map (V-VE1, on an arbitrary magnitude scale: properly calibrated colours will come later).


It’s clear that the deep artifacts around the edge of the moon are much smaller now, compared to the B-V colour maps we have been producing to date.

Data used:

2456015.7558321MOON_V_AIR_averaged.fits 2456015.8108611MOON_VE1_AIR_averaged.fits

This shows that the core of the PSF of the V and VE1 filters are quite similar — and the power law tails as well — not just for Arcturus but for lunar images too.



Profiles of Arcturus in different bands

Post-Obs scattered-light rem. Posted on Apr 07, 2013 00:53

This is a followup to the previous post on the V and B band PSFs.

Here is the stellar profile of Arcturus, shown in B (blue) and V (green). Clearly the B band light has a different core to the PSF than V.


Arcturus data were taken on the night of 2011-03-22. The files used are

align_stacked_2455643.4800437Arcturus-B-FILTER.fits align_stacked_2455643.4891739Arcturus-V-FILTER.fits

Oddly, the V band data are affected by what we called “shutter bounce” — a non-axisymmetric feature to the right (along crows in the CCD) of the PSF, but the B band data show no sign of such a bounce (the PSF profile can be exceedingly cleanly centered, unlike V band). No explanation for this for the moment!

Here are the other bands, all compared to V (green symbols):


PDF version here:

Here are the same profiles, this time with the data averaged into radial bins. The solid green line is the V band profile in each case.

VE1 and V are very similar — but this is what we would expect. IRCUT is not too far off V either.. the B and in particular the VE2 filters differ significantly from V. The difference between VE2 and all the other filters is huge, so we need to look into that next!



PSF in V and B

Post-Obs scattered-light rem. Posted on Apr 06, 2013 01:22

It is clear from the colour maps we have been making of the moon, that the B band PSF must be quite a bit broader than the one in V-band. We have only measured the V-band profile with any precision to date: this is a first go at the B-band profile.

The images used are these:

2455945.1776847MOON_V_AIR_averaged.fits

and

2455945.1760145MOON_B_AIR_averaged.fits

both taken at an airmass of ~1.3 – so quite high in the sky (which is good).

A wedge shaped region is defined for each, as shown in this figure by the green vectors:


The (V band) image is on a logarithmic scale, with counts/pixel shown along bottom. The red circle marks the fitted (by eye) position of the moon.

Centers and radii are :
V band center (195, 287), radius 147 pix
B band center (197, 291), radius 147 pix

Plot shows log radius (from center of moon, in pixels) versus flux (counts/pixel) in V band (green curve) and B band (blue curve).

It’s clear that the dark side of the moon (DS) is bluer than the bright side (blue curve offset by ~0.05 in the log on the DS).

Plotting distance from the edge of the bright side, we get this:


Focusing on the falloff of light off the brightside edge (coincidentally the B and V band fluxes are almost identical in the two images, so they are effectively normalised at the peak flux) — we see in the figure above that the blue light has considerably broader scattered light than the V band. (Note that the radial scale is now linear, not log).

Next plot shows the falloff of light on a log-log scale, as a function of distance from the lunar edge (less 3 pixels, in order to catch maximum flux).


Clearly blue light is more scattered than V!

Up to now, we have been using a single V-band scattering profile for the light — which is based on this technique of measuring the scattered light off the lunar edge, fitting a power law to the falloff at large distance (the power law tail of the profile, which has a slope of order -2.7 to -2.9 on very clear nights, at least in the V band).

The power law tail in B appears to have the same slope as in V — the B band light beyond ~ 10^1.5 = 30 pixels from the lunar edge out to the limit of the wedge (250 pixels) follows V band closely.

However, there is a large excess of light from the lunar edge out to about 25 pixels. This is messing up our colour maps, producing the dark (very blue) artifacts around the moon.

More work is certainly needed! A deconvolution method to extract the true PSF from full moon images, in all our bands, might be the thing to try next, rather than the “poor man’s” wedge method! (Note that in the wedge method, we correct for the fact that the moon is not a point source, we do not use the profile of the light in the wedge directly!)



For the poster at EGU: comparing methods’ results

From flux to Albedo Posted on Mar 29, 2013 15:21

[edited versions:] We have discussed many ways of extracting albedo from our data. We first considered the ‘ratio over ratio’ methods – they consisted of extracting counts from patches on areas of the scattered-light cleaned images in the DS and the BS – the ratio of DS/BS observed to DS/BS model is proportional to the albedo of the Earth. The other method, used more recently, is based on ‘profile fitting’ near the DS sky edge.

We have now arrived at comparing the 5-color albedos derived from these two methods:


Right
: Albedos for positive and negative lunar phases (Full Moon is at 0 degrees) from the DS/BS method where the “BBSO log” method has been used to remove scattered light. Left: Same but for the profile fitting method. Note different
vertical scales on axis. The same nights were considered for both
plots, but not all are present in both, due to outlier removal, etc.

We see, for both methods, a rise in the derived albedo as phases nearer Full Moon are considered. This is possibly due to effects of scattered light from the BS which has been incompletely handled by the respective methods. The values found with the two methods are quite similar – apart from increased scatter and less colour-separation in the DS/BS method. The data for the positive and negative branch of the phase diagram are not similar, in either case.

From tests shown elsewhere in this blog we do not expect the halo to interfere with the DS for large absolute lunar phases – i.e. near new Moon. The above diagrams shows lowest values for phases near 110 degrees. There is a slight increase in values larger than this – what can that be due to, if the halo is interfering less and less? Well, we must again remember that the above results are model-dependent and the Model may be adding its own fingerprint. For instance, it may be that the synthetic images we use have a phase-dependent error in their representation of lunar and/or terrestrial albedos. Note that the same synthetic models are used in the DS/BS method as in the profile-fitting method.
This question can be addressed by studies of the effect on the ‘bend’ seen above of different BDRF models.



B-V of the Moon vs. phase

Bias and Flat fields Posted on Mar 23, 2013 18:24

We show here the B-V mean value of all pixels lit by sunshine. The data have been selected for being ‘good’, so the scatter is somewhat disappointing. Best one can say is that we do not contradict the published B-V=0 .92 value.

Blue and red points correspond to different parts of the lunar cycle – before and after Full Moon – waxing and waning, whatever you want to call it.



List of possible student projects

Student projects Posted on Mar 22, 2013 09:51

We have often come across good ideas for student projects. Here is a start of a collection of projects – just links, but text can be added to explain more.

How do meteorological conditions determine seeing at the telescope?
—————————————————————————
http://iloapp.thejll.com/blog/earthshine?Home&post=343

Was the bias pattern constant?
———————————-
http://iloapp.thejll.com/blog/nextelescope?Home&post=3

Understanding the PSF:
————————–
http://iloapp.thejll.com/blog/earthshine?Home&post=313

Albedo maps and their use in modelling observations:
———————————————————-
http://iloapp.thejll.com/blog/earthshine?Home&post=304

Atmospheric turbulence studied via Moon images:
——————————————————-
http://iloapp.thejll.com/blog/earthshine?Home&post=295

Colour of earthshine – Danjons work:
—————————————-
http://iloapp.thejll.com/blog/earthshine?Home&post=280

Image analysis methods – Laplacian method:
————————————————-
http://iloapp.thejll.com/blog/earthshine?Home&post=273
http://iloapp.thejll.com/blog/earthshine?Home&post=272

Modelling Earth:
—————–
http://iloapp.thejll.com/blog/earthshine?Home&post=258



Understanding the linear slopes

Exploring the PSF Posted on Mar 22, 2013 09:30

In this post we saw that the difference between B and V (magnitude) images could have the shape of a linear slope on the DS and plateau on the BS. We are trying to recreate that using synthetic models. It is surprisingly difficult!

Using V and V images we saw that differences typically had the shape of level offsets – not slopes. In the B-V images we saw linear slopes on the BS. I thought the linear slopes originated in different PSFs in two filters – different alfa-parameters, for instance.

Well, taking a synthetic image and convolving it twice with two slightly different PSFs and converting to magnitudes and subtracting gives this:

Upper panel shows the ideal image we are using – BS to the right and the rest is DS. Bottom panel shows the difference between the image convolved with alfa=1.73 and alfa=1.72*1.02. DS is columns left of 360 – there is no linear slope. There are plenty of features on the DS above, but none ‘slope away linearly from the BS’.

A straight line in a lin-log plot corresponds to an exponential term. The difference between two Gaussians of different width is probably another Gaussian. Are we learning that the real PSF has a Gaussian term in it that varies between filters? Since V-V images did not show this behaviour the Gaussian is not manifested by the inevitable slight image alignment problems. Our model PSF is an empirical core with power-law extensions – and the above experiments show that such PSFs do not yield linear-slope differences.

Perhaps we could study the real PSF by studying difference images in a thorough way? Student project!



The halo, seen in B-V

From flux to Albedo Posted on Mar 21, 2013 10:49

In posts below we have discussed how to best investigate colour differences. Here we saw that sky brightness and exposure time problems can be detected.

Using selected good images in B and V we found the pairs that also were close in time, and generated B-V images. We noted (discussed here) that B-V on the BS is not always near the published value of 0.92, even in images selected for not having (obvious) exposure time problems. We wonder if the value 0.92 is more of a classical photometer value? That the colour of the whole BS on average is 0.92? Perhaps – but we also wonder if the reflectivity of the Moon has a phase dependence so that the B-V colour, even if a BS average, is lunar phase dependent?

Here, we choose to bring the B-V value of the BS in our selected images to 0.92 in order to study which values we get for B-V on the DS. This in an attempt to see if we can discuss what the B-V color of the DS, and therefore of Earth is.

For a range in lunar phases equivalent to illuminated fraction from 35% to 50% we plot the B-V values of a slice across the lunar disc, through the centre and 40 rows wide. We average over the rows:

We have aligned the images by the deep cut, which corresponds to the BS/DS border – the terminator. On the right of this we have the BS and on the left the DS.

We see that the BS is level. We have offset the image values so that the BSs are near 0.92 (by eye). We see that the DS has a slope. We see some level differences in these slopes but the slopes themselves are fairly similar. For one of the profiles the B-V reaches as low as 0.5ish, but there is still a slope.

On the basis of that I think we ought to say that “B-V for earthshine is at or below 0.5 in absolute terms”. Better may be to say that “the DS B-V is 0.42 below the BS B-V level.”

The B-V os sunlight is 0.656 ± 0.005 (Chris has measured a similar value of 0.64). If we also know that the sunlight on the Moon appears to us to have B-V of 0.92 then we can infer that one reflection from the Moon reddens the sunlight by (0.92-0.656)=0.264. The Sun also shines on Earth (lucky us!) and that light has its colour altered by the Earth – when it strikes the Moon and comes back to us it has B-V of 0.5 or less. Knowing what one reflection off the Moon does to the B-V color we infer that B-V of Earthshine is 0.25 or less. Franklin performed UBV photometry [with a ‘diaphragm’ of diameter near 1 arc minute – it means he observed areas on the DS, preventing the BS light to enter the photometer, but not BS-scattered light from optical elements before the photometer] on the Earthshine and found that B-V ‘for Earth’ was 0.17 below B-V for the Sun – this implies that B-V for Earth is 0.47. We have a value (from our lowest value) slightly above that.

Some notes:
1) The above slope is rather straight. The halo itself is there because the halo from B and V have not cancelled. This must be telling us something about the scattered light halos? Probably that it is linear in a lin-log plot – which is what we use using the ‘BBSOlog’ method. What would Ve1-VE2 images look like? We cannot tell since the halo is not (primarily) due to atmospheric scattering and Mette knows no rule for how scattering in lenses depends on wavelength. We have to try, before we know if Vegetation edge data can be extracted in this way.
2) We also see that we have no images without a B-V gradient across the DS. If we had had an SKE we might hope to achieve ‘clean’ B-V images. But we don’t. So we can’t.
3) All is not lost – when we fit images profiles directly we compensate for the halo and can therefore extract actual albedo data that does not depend on the presence of the halo.



And now, V – V images

From flux to Albedo Posted on Mar 20, 2013 15:13

To check consistency we now look at V minus V images, where the two V images are chosen to be closer in time than 30 minutes but not taken at the same time. We correct for extinction; we convert the raw images (bias subtracted, of course) into instrumental magnitude images by calculating the flux from the nominal exposure times and taking -2.5*log10, and then align them and subtract. We shoul dget images of 0s since the fluxes, once corrected for extinction, should show the same flux – at least on the BS where the Sun is shining. We do this and get a royal mess:

In each frame the insert shows a color-contour plot of the V-V image. The graph shows the usual slice across the middle of the image, averaging over 40 rows.

We see in upper left panel a clear offset at the terminator – i.e. the DS has different level but the BS are similar. Upper right shows a failry decent pair of images – the terminator is giving some problems and the DS as well as the BS differences are offset from 0 by a small-ish amount which coul dbe caused by an error in exposure time of some 10% or so (not unlikely). Lower left shows that while the two BSs are at teh same level then the DSs differ violently. The lower right shows a really nice example of two images agreeing.

What is going on? We hand-inspect the above images and see that in the case of the lower left image the sky level is much higher in one of the two images used, although they are observed less than 30 minutes apart. The counts inside mare Crisium are 3 and 11 in the two cases – i.e. a factor of about 4 or a magnitude difference of 1.5 – well outside the plot frame. So, from this we learn that we have a method to detect stable sky conditions! It also tells us that using images from different filters should be done with great care – even small differences in sky conditions will cause the DS to shoot off!

As it is, these images are merely bias subtracted – there is no individual correction for a ‘pedestal’ due to sky conditions. Luckily we can still use these images for albedo work in that we expressly fit a pedestal term!

What else can we learn? Well, any error in actual exposure time will influence the DS and the BS with the same factor – hence the magnitude differences plotted above will become offsets for both DS and BS – hence, upper right is consistent with ‘wrong exposure time’ in one of the images. Having assigned one filter and received another will have the same effect as an error in exposure time – if the DS and BS are altered by the same factor. I wonder if ‘wrong filter’ could camouflage as ‘wrong exposure time’? Since DS and BS have different colors I doubt it – but we should investigate this.

So, three things learned:

1) Wrong exposure time will lift or depress DS and BS by same amount.
2) More sky brightness in one image than in the other – affects DS only.
3) Image pairs like the ones used for lower right panel, above, are probably both OK.



B – V images of the Moon

From flux to Albedo Posted on Mar 20, 2013 11:09

I have located all ‘good images’ in B and V. That is, all B and V images, made from stacks of 100 images, that are on Chris’ list of ‘good images’. I have furthermore identified all pairs of B and V images that are taken less than 1 hour apart. For all these pairs I calculate a B-V image, by using Chris’ calibration of flux against known standard stars. I allow for an extinction correction based on kB=0.15 and kV=0.10. I get airmasses from the Julian date of the image, and IDL software. I plot a colour contour plot as well as a ‘slice’ across each image. The slice is made up of an average of 40 rows centred around the row that goes through the centre of the B-V image. A total of about 33 image pairs have been plotted in this way. Here is a (large) pdf file containing the plots.

There are several strange things to see. Generally we get the profiles shown here. That is – the BS is somewhat flat, while the DS slopes towards the DS sky. Oddities include images where the DS is as flat as the BS – just at another level. Is that ‘extremely good nights’? I think not. In some, the DS is much higher than the BS – that could be images in which the exposure time is incorrect due to shutter problems. Or filter-wheel problems! In many the BS is flat, but not near the 0.92 value we expect based on publications. For small differences with this ‘canonical value’ I think we could be talking about exposure time uncertainties. If we are at B-V=1.0 instead of at 0.92 we could have an 8-10% error in the exposure time. The times are short and it does not seem unlikely we have that big a problem. Sigh.

I will suggest that we ‘correct the BS value’ to 0.92 by simple shifting, and then consider how our DS B-V values look. With some ‘deselection’ of some obviously bad images we can perhaps arrive at a base set of good B-V images.

I think the result will be that few show any leveling off in the ‘slope of B.V wrt distance to BS’.

We have few images at small lunar phase (i.e. small illuminated fraction). Chris has analyzed one good pair of B and V images at small phase, but it is not in my pipeline of good images – the automatically detected centre coordinates and radius are way off (because the sicle is so small that automatic methods do not work). Our new student Johanne is inspecting these images by hand and will adjust radius and centre coordinates and then we should have a few more images to play with at small phase. These may show us if we can get far enough away from the BS that the DS slope in B-V levels off and shows the true earthshine B-V color.



The Blue Marble

From flux to Albedo Posted on Mar 18, 2013 15:47

[Note added later: we have updated this posting using extinction corrections for B and V. kB=0.15 and kV=0.1 were used, and airmass=1.67. JD was near 2456016.82]

In preparing for the EGU 2013 conference, where we will have a poster on color results from the earthshine project, I want to show a ‘B-V’ image of the Moon – since we can, and may be the first to publish such a thing.

Using Chris’ calibration of our filters against standard stars in M41 and NGC6633 I can reduce observed images in any filter to ‘magnitude images’ in the same filters. By taking the difference between the B and V image I thus arrive at a ‘B-V image’. Here is a slice across the middle of that image:

Second image added later: We now correct fore extinction:
Here, the BS is to the right of the ‘cut’ in the disk, and the DS is to the left. It appears that the BS is higher than the published value for ‘moonshine’ of 0.92 [see e.g. Allen, “Astrophysical Quantities”, 4th ed, Table 12.14]. Added later: Even after correcting for extinction the B-V is still higher than Allen’s value – we now get B-V near 0.95-1.0. Perhaps now, not so much larger than that value?

I am not quite sure about the DS. There is probably a slope because the halo in B and V are not identical and thus do not cancel. But as the BS is approached the color of the halo does approach the BS value, since that is where the light comes from. So I understand that part.

At the extreme left of the DS there is least influence from halo. The value is a bit up and down, but appears to be lower than the BS by up to 0.3. If B-V is a smaller number in one place compared to another it means that B is smaller than V in the first place, relative to the second – i.e. on the DS B is relatively smaller there than on the BS – since these are magnitudes it means we have shown that earthshine is blue – the Earth is indeed ‘a blue marble’!

Here is the B-V image itself, with cleverly chosen colours to reinforce our message: Sky is masked out, BS is to the right, DS to the left.

I’ll try to redo this for a lunar phase with less halo.



Further adventures of VE2

Data reduction issues Posted on Mar 12, 2013 16:11

We have been following the mystery of the VE2 filter. Ana Ulla asked a good question: “Do you see the same problems in images not of the Moon?”, so I checked that, using our copious collection of Altair images. I fit a Moffat profile to each well-exposed image of Altair and extracted the sky level. I plot it here as a function of which filter (0=B, 1=V, 2=VE1, 3=VE2 and 4=IRCUT) and as a function of time, for one night (JD 2455845):

In the top panel we see that VE2 (the brownish points) have two levels, and in the lower plot we see that the VE2 level switches from high to low after some time. (Bottom panel has fractional day as x-axis). We also see that the black points (B) increase their level (slightly) at the same point (0.87) where VE2 drops to the low level. Blue points (VE1) have some history like that too. V (red) is quite stable. Green is IRCUT.

If the excess in VE2 (and others) was due to a source of light, then we would hardly see a down-shift in the level. Ana’s idea was that thin cirrus could enhance the nIR band of VE2 (quite like the Johnson I band) – but that does not explain why B rises, unless that is a particular behaviour of cirrus clouds, which I doubt. I do not think anything above is consistent with a light source. I think this is what we have had to conclude from the other results presented in this blog – e.g. the inverse exposure time dependence and the inverse airmass dependence.

I am guessing when I say that I think it is something to do with electronics. The camera knows which filter is being used! (Not).



VE2 Gets Stranger

From flux to Albedo Posted on Mar 11, 2013 09:53

We have already shown a strange behaviour of the VE2 filter – or what appears to be a problem related to the use of the VE2 filter. The post is here.

Now we are able to report something even stranger (OK, just as strange then). We appear to have a relationship between the ‘VE2 pedestal’ and the airmass of the observation:

In the upper frame we see the old VE2 pedestal vs exposure time plot. In the lower frame we see that the pedestal is also a function of the airmass! If the pedestal was proportional to airmass we could, perhaps, understand the phenomenon as something showing a flux proportional to the amount of atmosphere (I am thinking of some sort of airglow here) – but NO, the pedestal height is smaller the larger the airmass is!

Is this some electronic problem related to how the telescope is oriented? Then why the inverse proportionality on exposure time too? Why not the other filters?

What is going on here??



The VE2 Mystery

Data reduction issues Posted on Mar 08, 2013 21:25

During reduction of our data it has become evident that the VE2 filter gives us certain problems. A new problem is that VE2 exposures tend to have a ‘sky component’ that is much larger than in the B,V,VE1 or IRCUT filters. Here is a plot of the relationship between exposure time and the ‘pedestal’ – i.e. the offset that has to be subtracted from the VE2 image to bring the sky level down near to 0 (in bias-subtracted images, of course).

It does seem like the pedestal height is a decreasing function of the exposure time. Why is it any function of exposure time, when the other filters do not show this behaviour?

If there was a source of light in he image frame, then its level should INcrease with exposure time, surely? Here is the plot for the VE1 pedestal:

The level is much smaller than in the VE2 case – and it does not appreciably depend on exposure time.

What is going on?



Evaluation of fit improvements

From flux to Albedo Posted on Feb 28, 2013 10:08

In this entry: http://iloapp.thejll.com/blog/earthshine?Home&post=304 we investigated the effects of using a lunar albedo map based on scaling the Clementine map to the older WIldey map vs. scaling the Clementine map so that lunar mare and highlands matched what is published in the literature. [Note: the point being that the ‘Clementine map’ we have available is just a picture file – jpeg! – so that pixel values have to be scaled to albedo values somehow.]

We were doing this fitting in the ‘new way’ which is to fit ‘profiles’ starting on the sky on the DS side and extending onto the DS itself, modelling the contribution from the scattered BS light. In doing this we saw that improved fits could be obtained if the lunar map albedo was ‘stretched’ so that dark and bright areas better matched our observations. We did this stretching by eye and were able to improve the formal fits.

We have now compared the fits on 535 images done using the ‘Clementine scaled to Wildey’ map and the ‘Clementine stretched by eye’ maps.

We have found that the RMSE (that is, the square root of the sum of the squared residuals [i.e. observed profile minus best fitted model profile]) is improved using the scale by eye’ map – in two ways: a subset of images were poorly fit using the other map; they are now much better fit, and the mean RMSE of the finally selected images is lower.

We selected ‘good fits’ on the basis of alfa (the PSF-width parameter) having to be in a narrow range and that the relative uncertainty on the fitted albedo should be below a certain limit.

Mean log10(RMSE) is now near -1.1 in units of counts/pixel. We fit a profile that is 150 pixel columns long – 50 pixels on the sky and 100 on the DS.

Mean relative fit-uncertainty on the albedo is near 1.5% when using ‘counting statistics errors’ on the observation.

We note that the VE2 images more frequently have a larger ‘pedestal’ or sky offset after bias removal than the other filters. While most filters have an offset of near 0.2 counts (+/-0.5 or thereabouts) the VE2 offset is more often near 4 or 5 (+/- 1-2 counts). What is the cause of this? The bias frames surrounding the VE2 exposures have been spot-checked and seem OK – bias is near 400. A few, otherwise perfectly all right, VE2 images have a sky offset of 10-40 counts! Observations from the same night in other filters show nothing like this – some sort of nIR ‘fog’? Does the sky emit nIR light?

Does this have a conseqeunce for the ‘tunnel selection’ of VE2 images, done by Chris? [A data-selection method designed to take into account shutter and filter-wheel problems by requiring that image total fluxes follow a known phase-curve.]

While the ‘new fitting method using stretched lunar albedo maps’ formally works best of all methods we have seen so far, it is very empirical. Later we may be able to use the method to produce a ‘best fitting lunar albedo map’ that improves on the original Clementine map. We may then also be able to compare to what the LRO people (and LLAMAS) are finding.



Effect of Wildey vs. Clementine

From flux to Albedo Posted on Feb 25, 2013 08:49

After the scattered light has been removed we still have the task of converting the observations to a terrestrial albedo. This is done, by us and the BBSO, with the use of a model for how the Moon reflects light. This involves assuming a lunar albedo.

The BBSO has a set of lunar-eclipse observations that they use to find the DS/BS patch albedo ratio – we use the Clementine map. That map is available to us only as an image file. We scaled that image so that the lunar mare and highland albedos correspond to what the literature states.

An alternative would be to scale it to match what lunar albedo maps show. The only digital lunar albedo map we have is the 1970s Wildey map, which Tom Stone gave us. In this blog: http://iloapp.thejll.com/blog/earthshine?Home&post=253
we performed a regression-based scaling of all the pixel values in the Clementine map to corresponding (or interpolated) pixel values in the Wildey map, using the two maps’ coordinate systems.

We therefore have the possibility of fitting our observations to the lunar map of Clementine scaled to Wildey (let us call that the “Wildey map” from now on) or the Clementine map (“Clementine map” – but note that both are Clementine-based!) scaled to the highlands and mares. This choice has been investigated.

We find that the contrast in the Wildey map is lower than in the Clementine map, which affects the quality of fit.


Same lunar-edge profile (white line) fitted with two different models (red line) – one uses the Clementine lunar albedo map, scaled to match lunar highlands and mare; the other is the Clementine map scaled, using every pixel, to fit the Wildey lunar albedo map. Notice the smaller contrast between highs and lows in the upper image (Clementine scaled to Wildey) compared to the lower (Clementine scaled to highlands and mare).

There is evidently a difference in the quality of the fits – neither being particularly good, failing to match the observed highs and lows. The conseqeunces for the albedo fitted is at the 3% level.

The mean difference in the terrestrial albedo determined using the two maps:
16 albedo values determined using both Clementine-scaled-to-Wildey (vertical axis, in top plot) and Clementine alone (h. axis). We see that the two albedos are near the diagonal – with only a few outliers. The histogram shows the absolute difference between the two albedo determinations – it is about 1% of the albedo value.

As long as the large spread in albedo values (from 0.23 to 0.42) is real and not due to some data-treatment bias, we have an effect from the choice of lunar albedo map that is at the level of the errors from pixel statistics (shown elsewhere in this blog).

We would like the known biases to be less than the effects due to scatter, so perhaps some work on the lunar albedo is in order? We can look at the mean values of the two maps – there was no requirement that mean albedo be conserved, and we can look at various ways of stretching the contrast, while maintaining mean value – this is a nice piece of work for a student project.

Performing a hand-adjustment of the contrast scaling, while maintaining the original Clementine map mean, we can refit the relevant profile. We get this:


Smaller RMSE, small but important change in ftted albedo.



Smallest halo

From flux to Albedo Posted on Feb 23, 2013 06:53

The best data will be those that have almost no interference from the halo at all. Those exposures, if they can be identified, will have the least scattered light on the DS and will therefore be the easiest to clean up. Here we try to identify those exposures. We simulate the effect of the halo on ideal images and measure the amount of flux added by the halo – before any attempts to remove it. We loop over relevant values of lunar phase and ‘alfa’, the PSF-width parameter. We calculate the change in flux for a region near the DS rim and express the change in percent. The contour plot shows contours at 1,2,4 etc percent change. Overplotted are red data points showing locations of real exposures.

We see that the least polluted images are found at large phase (near New Moon) and for narrow PSFs (large alfa values). This is not surprising! We see that we have several exposures taken under conditions that allow just 1,2,4 etc percent change in DS intensity.

If these images are the least polluted by BS-scattered light, we should have the most success when applying our scattered-light removal techniques to these. If they have a 10% effect due to scattered light, and we can remove 99% of that we have reduced the error induced by the BS halo to the 0.1% level.



Meteor caught in the act!

Showcase images and animations Posted on Feb 21, 2013 09:17

During processing a strange signal was found in a frame combined from 100 images. It turns out that in one image of this stack we have what appears to be a meteor or a satellite flash – or something:

The image is from Jan 17 2012: (UTC 2012-01-17T13:28:48). The exposure time was 0.009 seconds! The trail is about 3/4 of the lunar diameter in length – i.e. about 22 arc minutes. The orientation is such that it is travelling almost Due North (or South!).

What is it? Well – it is clearly between us and the Moon! If it is a meteor its height would be something like 50-100 km. The speed would then be 35 – 70 km/s. A satellite in low earth orbit has speed 8 km/s. Since the image is taken from Hawaii at UTC 13:28 it is near midnight on Hawaii – i.e. the Sun is behind the Earth and unlikely to be illuminating a LEO satellite.

An airplane flies at 800 km/hr at altitude 10 km, so the distance covered in 0.009 s is 2 m which would subtend an angle of 1 arc minute. This is no airplane – or it is much closer, in which case we should see details of the plane.

As far as I can tell there is no pronounced peak in meteorite activity in January.



6 nights

From flux to Albedo Posted on Feb 17, 2013 08:21

We select 6 nights and extract albedos in 4 bands:

Get better pdf file here:

Here are albedos determined for 6 nights. We see 4 filter bands (VE2 is still being processed). Each color represents one night. There are 3 determinations of albedo from each image – hence points come in columns of three.

We see:

1) Albedos can differ in level from night to night – e.g. blue and red in panel 1.

2) Albedos can also be almost constant during a night – e.g. purple and green points in panel 1 vs red or blue.

3) Some nights have large scatter – e.g. green vs blue

4) B albedos are higher than V and others

5) VE1 and IRCUT albedos are very similar – the filters are also very similar, so this is good for reproducibility.

6) While some albedos evolve smoothly on a given night IRCUT shows a ‘dip’ – e.g. blue crosses in B vs blue points in IRCUT

Questions of interest:

a) Why are some nights noisy? E.g. the green points. Information on airmass, alfa and bias frame statistics, and RON, are available.

b) Can the nightly evolution (i.e. slope) of the points be related to anything happening on Earth?

c) Can the change in slope in IRCUT be related to anything special?



VOM plots and ideal models

Data reduction issues Posted on Feb 12, 2013 11:41

In several posts we have considered the behaviour of variance-over-mean images (VOM). In the ideal world these should be 1 because the noise (once RON is subtracted) is Poisson-distributed. We have seen how this is hardly the case in observed images.

We now consider a test using ideal models with and without imposed image drifts. We generate 100 synthetic images with Poisson noise, and RON as well as a bias.

Below, each set of three panels show a cut across one image in the synthetic stack, then a slice of the VOM image before image alignment and then the same slice after alignment. On the left are images that were not drifted while to the right images were allowed to ‘jiggle around’ by a few pixels.

We see that VOM is 1 on the BS in un-jiggled images, but that DS and sky values fall below 1.

We see a HUGE effect on images that were allowed to drift.
Some of it we understand. Before alignment, VOM rises on structured areas of the drifted images because surface albedo variations are being introduced for a given pixel along the stack direction. The effect on the DS and sky is much less – perhaps because the Poisson noise is so large comapred to the signal variations. After alignment, VOM falls to slightly below 1 on the BS, except near the edge. On the DS and the sky, though, a large lowering is seen. So far it is not understood how this comes about.

Any strange effects seen in observed images will be all the larger since the images do not just drift but also ‘shimmy’ because of atmospheric turbulence.

The effects of aligning images to sub-pixel shifts is part of the above.

Let us learn from this that a noise model probably is hopeless to build in the presence of image shifts – despite realignment – and that sub-pixel interpolation is not a welcome added bother. We could just omit single images with the most drift and use a straight average of the remaining stack. In real images we have the option to not use stacks that have a l ot of drift – but we do not know the extent of the ‘shimmy’ for the remainder.

These realizations above have the most impact on our ability to interpret results that discuss the effect of alignment – alignment reduces some problems but probably adds some others.



Effect of alignment, II

Error budget Posted on Feb 10, 2013 13:01

In this post: http://iloapp.thejll.com/blog/earthshine?Home&post=297 we investigated the effects of alignment on the noise. We measured noise in squares on the final image and therefore had some ‘cross-talk’ from the variations due to surface structure.

Now, we consider the variance-over-mean along the stack direction – that is, for each image pixel we calculate variance and the mean and look at the ratio of these, which we call ‘VOM’. In the perfect world, this image should be 1.

We look at unaligned images, and then at aligned images. The ‘superbias’ was subtracted (but not scaled to the adjacent dark-frame average). RON of 8.3 electrons was subtracted before VOM was calculated as (SD-RON)^2/mean. Gain used was 3.8 e/ADU. We plot VOM in a slice across the final images:

Top is VOM in the unaligned image, and below is VOM in the aligned image. A lot can be mentioned: First, the surface is not constant, obviously. Second, the effect of alignment is not just a uniform lowering of the VOM that we expect (same mean, less cross-talk between pixels).

In the top image we have VOM near 0.1 (dotted line) on the DS and most of the sky. On the BS the VOM is near 10 apart from the peaks that occur at the intersection with the lunar disc edge. There variance rises because of jitter between images and the mixing of disc and sky pixels.

In the aligned image VOM is near 2 or 3 on the BS disc, higher at the peaks (so there is still an edge-effect, but less). On the DS and the sky a spatial structure has been formed, slanting inwards towards the BS.

What is going on? Effects of pixel-subshifting? What does the interpolation do? Why is it spatially dependent? Strange flux-nonconservation?

The sub-pixel shifting used is based on the IDL ‘interpolate’ function. In the manual for this function it is suggested to allow for cubic interpolation by setting a keyword “cubic=-0.5”. I did this and recalculated the above. The effect was slight.

A test was performed to see what variance is induced by the interpolation. By shifting an image by a non-integer amount of pixels in x and y and then shifting that image back, and calculating the difference, we see that using INTERPOLATE without ‘cubic’ induces 2-3 times more residual S.D. than does use of INTERPOLATE with cubic=-0.5. The interpolations lost variance compared to the original image. With cubic interpolation the loss of standard deviation relative to the mean is at the 1% level – quite a lot actually. [Note: conservation of flux not the same as conservation of variance]

Could it be some effect of bias-subtraction? VOM has dropped most in the halo near the BS. Why?



indiserver

Control Software Posted on Feb 09, 2013 15:19

In a potential upgrade of what we currently have, we might consider going to linux and freeware, for our command system.

Some devices can be operated via the indiserver concept. Devices for which drivers have been written can be commanded from essentially the linux command line. This means that scripting systems are easier to set up – for us at least.

I have been able to control our SXV-H9 CCD camera using this system. Here is a link to how it is done:

There exist indiserver drivers for gphoto2 – that means that Canon DSLR cameras, for instance, can be operated in this way.

A TrueTech filterwheel is being tested. LX200 telescopes can be controlled – i.e. mounts such as ours.



Noise model

Data reduction issues Posted on Feb 08, 2013 12:37

It would be a Good Thing if we could explain observed images in terms of a ‘noise model’ – that is, a model that explains why the signal and the noise bears a certain relationship.

We consider here a stack of 100 raw images taken in rapid succession. The difference between adjacent images should only contain noise in that any objects shown in both images should subtract. The difference between two series drawn from the same poisson distribution but independent has a variance that is twice the variance in either of the series subtracted, so we apply a factor of 1/2 to the difference image to get the variance image.

We subdivide such a variance image into smaller and smaller squares, and in each square we calculate the variance. We generate an image of the ratio between the variance image an dthe mean original image, also subdivided In the case where everything is Poissonian this image should be a surface with value 1. In reality there will be noise in this surface – and there will be strcutures seen wherever imperfect obejct subtraction took place. Here is the result:


In the three rows we results from different subdivisions – into 8×8, 4×4 and 2×2 squares. In the leftmost column is a histogram of the values of the image and to the right is plot of the profile across the middle of the image.

We see, to the right, that variance over mean (VOM) is not 1 everywhere. We are evidently picking up a good deal of variance near lunar disc edges [The image corresponding to the above is a quarter Moon with the BS to the right and DS to the left, situated in the image field center]. We see that the sky manages to have VOM near 1 and that parts of the DS does this, but that most of the BS appears to have VOM>1. Even ignoring the peaks that are due to edges we see a value for VOM near 2 or more [also seen in the rest of the 100-image sequence].

Since we used raw images we have to subtract a bias level. We subtracted 390 from all raw image pixels. We applied the ADU factor of 3.8 e/ADU to all bias-subtracted image values. We then calculated the variance and subtracted the estimated RON (estimated elsewhere at near sigma_RON=2.18 counts, or variance_RON=4.75 counts²; consistent with ANDOR camera manual “8.3 electrons”).

The subtracted bias is a little small – the observed mean bias value is nearer 397, but if we use that value we get strange effects in the images – only a relatively low value of the bias mean gives a ‘flat profile’ for the sky in a slice across the image. This is one poorly understood observation.

We also do not understand why the VOM is nicely near 1 on the DS while it fails to clearly be near 1 on the BS – both areas of the Moon have spatial structure which is bound to contribute to the variance in the difference image during slight misalignments.
That is the second poorly understood thing.

Progress towards a ‘noise model’ is therefore underway, but there is some distance to go still.

How would non-linearity in the CCD influence the expectations we have from Poisson statistics?



Effect of image alignment on result quality

Data reduction issues Posted on Feb 07, 2013 10:42

We are using stacks of 100 images. These are acquired by the CCD camera in rapid succession – 100 images can be obtained in less than a minute. During that time there may be small motions of the telescope, and the turbulence in the air, as well as the slight change in refraction due to airmass changes may cause image drifts in the stack.

In one approach we ignored the possible drift and just averaged the 100 images. In the other approach we use image alignment techniques to iteratively improve the alignment of the stack images: First we calculate the regular average, then we align all stack images against that average image, then we calculate a new average image on the basis of the alignments and re-align all images against this average and so on. This procedure turns out to converge, and after 3 iterations we can stop and save the last average image.

We would like to know the effect of doing this on result quality. We therefore generated two sets of averaged images – the first using the simple first kind above, and the second the iterative method.

We estimate ‘errors’ in the bootstrapping way. That is, we extract MHM averages (mean-half-median, as explained elsewhere) of the DS intensity on raw and cleaned-up image patches and also estimate the statistical error on these values by bootstrapping the pixels inside the patch, with replacement. This bootstrap procedure gives us a histogram og MHM values and the width of this distribution is a measure of the ‘error on the mean’. We express this error as a percentage of the mean itself, for RAW images and images cleaned with the BBSO-lin, BBSO-log and EFM methods.

we now compare the results to see the ‘effect’ of performing alignment of stack images:

PSF Alignment RAW EFM BBSOlin BBSOlog
1: one without 0.73 1.25 0.74 0.79
2: one with 0.57 1.03 0.62 0.68

3: two with 0.58 0.99 0.63 0.69

Table showing errors (in percent of the mean). Lines labelled 1: and 2: show results for ‘one-alfa PSFs’ with and without alignment. The third line shows the effect of using a ‘two-alfa PSF’ and alignment.

We see that there has been a large reduction in errors by using alignment. Raw images improved by 20%, EFM images by 17%, BBSO-lin by 16% and BBSO-log by 14%.

The effect of using a two-alfa PSF on aligned images is small – indeed, all images except EFM experience a small increase in the error (probably not significant).

The effect of alignment on single-alfa PSFs is not investigated.

We conclude that alignment is a beneficial operation. We note that EFM has the largest errors but other arguments imply EFM is the better method to use – this is related to the stronger phase-dependence seen in non-EFM images, and is discussed elsewhere.



Stability of imaging in rapid sequence

Data reduction issues Posted on Feb 06, 2013 23:10

Our standard practice to date has been to take 100 exposures of the moon, each with an exposure time of 10 to 40 milliseconds, depending on the brightness of the moon. The exposure time is set so that the bright side attains about 50,000 counts maximum, in order to get good signal but not risk overexposure (saturation of the CCD wells).

We have looked at the differences between successive images in these 100 image stacks. A good night with a fullish moon was chosen (JD2456106)

2456106.8279161MOON_V_AIR.fits

and the relative difference between the first 50 consecutive pairs of images in the stack computed, and made into a movie — here

www.astro.utu.fi/~cflynn/reldiffs.ogv

(this can be viewed with mplayer in linux)

An example of nine consecutive frames is shown below :


During the sequence of 50 exposures in the movie, the moon only moves about 1 pixel down and 1 pixel to the left, so drift in the position of the moon from one frame to the next is practically negligible.

White and black on the grey scale run from a -10% to a 10% difference with the previous image. Note that the two compared frames are normalised to the same total flux (this is typically a correction of ~1%).

We see plenty of structure in the difference image, which we interpret as being due to air cells — turbulence — above the telescope. The telescope size (3cm) and short exposure times (20 ms) mean that scintillation is very significant (because of the turbulence). In tests on bright stars with our telescope and 10 ms exposures, we were getting scintillation fluctuations in the fluxes of order 50% — very large — as one would expect (Dravins et al 1997a) (paper attached below).

Some of the structures look greatly elongated, and remind us of the “flying shadows” discussed by Dravins et al 1998 (paper attached below). To quote them : “Atmospheric turbulence causes “flying shadows” on the ground, and intensity fluctuations occur both because this pattern is carried by winds and is intrinsically changing”.

STUDENT PROJECT:
Analyse sequences of images in 100 image stacks, and make an estimate of the amount of turbulence present following Dravins et al’s work — for correlation with our fitted parameter alpha of the PSF — to see if the amount of turbulence in the frames has an effect on the PSF’s long power-law tail.

Papers:

Dravins et al 1998:


pdf

Dravins et al 1997a (the scintillation figure is on p 186):


pdf

Dravins et al 1997b:


pdf



PSF with two parameters: Better?

Data reduction issues Posted on Jan 26, 2013 17:29

In this post: http://iloapp.thejll.com/blog/earthshine?Home&post=293 I proposed to fit more elaborate PSFs than we have been using.

I present here the results of fitting a 2-parameter PSF. The previous one was characterized by an exponent alfa, and had the form

PSF ~1/R^alfa.

The new one allows two exponents – one regulating the slope of the PSF up to a cutioff point in terms of radius, and the other parameter taking over outside that point. Thus we have 2 new parameters to fit – exponent and cutoff point.

Using such a PSF and evaluating success on the parts of the image designated by the mask – and using a mask consisting of sky on both sides of the Moon, but no polar areas – i.e. a ‘belt across the image’ – we arrive at images with slight changes in the estimated halo on the DS of the Moon and larger changes in the area on teh BS-sky.

This plot summarizes what was found:

The top panel shows absolute value of residuals (obs-EFM-model) in a slice across an image after the halo has been removed (along with most of the BS, as is always the case in the EFM method). The PSF used was the new tow-parameter PSF.
The middle panel shows the result when the usual one-parameter PSF has been used.
The last panel shows the difference between the two panels, expressed as a percentage of the first one. The illegible y-axis label for the bottom panel shows 10^-2 at the bottom, next label is at 10^0
.

Note that absolute values of residuals are plotted above.

We see the DS to the left, the almost-erased BS to the right and the sky on either sides.
Clearly, the BS sky has been removed much better in the first image – i.e. with the two-parameter PSF. On the DS we see changes in the range from 1 to a few percent on the sky-ward part, rising to tens of percent near the BS.

This implies that a signifcantly different amount of scattered light has been removed using the two PSFs – but which is the better result? Judging by the RMSE per pixel on the mask in the two cases there has been a significant improvement in going from one to two parameter PSFs. The RMSE per pixel in the one-parameter PSF case is about 5 [counts/pixel], while the RMSE per pixel is 0.23 [counts/pixel] in the two-parameter case. Most of this change has evidently taken place in the BS part of the sky.

We need to know if there has been an improvement on the part of the image that matters – the DS!

The best fitted parameters were quite alike – near alfa=1.7. The best-fit cutoff point was near 30 pixels.



A Bold Proposal!

Exploring the PSF Posted on Jan 26, 2013 10:45

It seems that success for us will be linked to our ability to correctly remove scattered light from the BS. This hinges on our understanding of the scattering model – in essence, the PSF.

Currently we use a PSF that is empirical. The core is made up of values from a table generated from observations of bright stars and Jupiter. To that we link an extension, also empirical, that is based on what the far wings of the lunar halo looks like. This PSF is then ‘exponentiated’ during fitting to actual images in the EFM method.

We noted during several posts below that the values for the exponent varied little and only now and then seemed linked to the extinction. This could be an indication that most of the time (during good nights) we are limited by something fixed – such as the optics – rather than atmospheric conditions. On the other hand it could mean that the PSF is not very accurate in its basic from and that the fitting method gives up at some stage, leaving us with an exponent that is somewhat random, and therefore not linked to the atmospheric conditions.

We should also recall what happens during application of the EFM method: We have tested various forms of sky masks for this method – some that allowed fitting emphasis on both the DS and BS sky, and others that emphasized only one side. Common for the ones that focused on either just the DS sky or the BS sky was that the halo shape on the other side was not very good.

Might these things be indicating that a better PSF should be generated or a better way found to apply the fits?

I’d like to suggest the following: Perhaps the PSF has a form like

PSF ~1/R^alfa(R)

instead of the present

PSF ~1/R^const_alfa ?

I would like to try to use a piece-wise constant alfa so that the PSF is separated into radial zones, each of which has its own alfa, found by fitting.

More to be added …



More on SKEs

Optical design Posted on Jan 26, 2013 10:06

In this post: http://iloapp.thejll.com/blog/earthshine?Home&post=289
the importance of the SKE for scattered light was discussed. The images shown, though, were not shown fairly – with intensities scaled to comparable levels. I therefore extracted a line across the BBSO image and a line across our image, at right angles to the SKEs, rescaled the intensities, aligned the plots and get this:
The black curve is from our image (whichis a sum of 10 well-exposed images). The red curve shows the cut across a single BBSO image.The BBSO image has only the DS peeking our behind the SKE, while our image has the BS in full view.

We see entirely comparable ‘halos’! The BBSO image ha a more pronounced sloping ‘tail’ onto the black side of the SKE than we do, and more noise. If that sloping tail is ‘halo’ we had a better system than the BBSO!

What does the above mean? It does NOT mean that we have less ‘halo problems’ than BBSO does – because the BBSO expose their DS so that the halo from the BS is not allowed to be formed. Yes there is a similar halo from the DS on their images as there is from the BS on our images – but the halo from our BS is very much stronger than their DS halo.

When the BS halo is small – i.e. near New Moon – we have minimal effect of the BS halo.



Some results

From flux to Albedo Posted on Jan 25, 2013 16:02

We have now reduced all the good data, and have arrived at the intrinsic properties of each image. We extract now the ratio of the DS-patch to the total flux and divide this by the similar ratio extracted from the synthetic model. This ratio of ratios is the same as the ‘Lambertian sphere terrestrial albedo’ and is the quantity published by e.g. Goode et al.

We select data for airmasses less than 2 and for statistical error less than 1.5%. This error is that due to mainly pixel-selection inside the ‘patches’ on the lunar surface (one near Grimaldi, one inside Crisium). Monte Carlo bootstrap sampling of the pixels inside a lon-lat box were resampled, with replacement, and the standard deviation of the consecutive means calculated. Errors due to image alignment, improper scattered-light removal, synthetic model problems and so on, are not included.

Here is the plot of extracted albedo against lunar phase for the VE1 filter (the filter with least scatter: IRCUT is almost identical to this plot): [scroll down for discussion]

The colours designate the method used for scattered light removal: red is the EFM method, orange or yellow is the ‘linear BBSO method’, and green are the raw data. Crosses and diamonds indicate the sign of the lunar phase – crosses are for positive phase. Error bars (error due to counting noise, not image alignment etc) have been plotted over the symbols, but are all smaller than the symbols themselves.

What do we see? There is scatter and there is dependence on lunar phase.

First, let us discuss the lunar phase dependency: For small phases we are essentially getting closer to Full Moon and the scattered light is more and more of a problem. The Raw data are doubled by phase 80 degrees, showing the importance of removing the scattered light. BBSO-linear is better than no removal all while EFM is better than BBSO-linear. There is also another lunar phase dependency – notice how the albedo rises for large lunar phases. Since the halo is smallest here this is not an effect of scattered light! The albedo is a composite quantity, consisting of quantities measured in observed and modelled images. The scattered light influences mainly the model, and mainly at small lunar phases. The reflectance model soley influences the synthetic lunar image, of course – I think we are seeing the influence of the (inadequate) reflectance model at large lunar phases – the albedo rises artificially due to this.

The there is the scatter. We see ‘daily progressions’ for some connected lines of data points. For instance, by 90 and 95 degrees phase. These points represent our candidates for geophysics! I think the ‘progression’ along these lines are due to different parts of Earth )oceans, clouds, continents) rotating into view for these observing sequences.

Below is the same plot, for the B filter. Apart from more scatter we see that the EFM seems able to remove most effects of scattered light, compared to the BBSO-linear method. A stronger ‘upturn’ is possibly seen at large phase.



Best of the data – revisited

Data reduction issues Posted on Jan 23, 2013 15:24

Chris thoroughly investigated the quality of all data and arrived ta a list of the 534 best images: http://iloapp.thejll.com/blog/earthshine?Home&post=277
I have collected all those images in ‘cubes’ consisting of raw observation, EFM- and BBSO-cleaned images, images for incidence and emergence angles and the ideal synthetic image and lunar longitude and latitude images – each ‘cube’ contains centred images for a given ‘stack’ of 100 observations. We thus have 534 ‘good coadded’ images with all secondary data that belongs to that moment. Each cube is about 16 Mbyte uncompress,a nd slims down to 5-7 Mbyte upon being bzip2-ed.

I inspected the correlation of these observed stacked images with their synthetic models and found that 8 of them are poorly correlated with their synthetic models. I inspected each case and found that the problem was either focus (the observed image is blurred) or a slight alignment problem.

Omitting those images we arrive at this list of 525 best images:

These are all available as ‘cubes’, upon request.

In the future we might wish to only work with these images, since alignment and image-reversal issues have been dealt with (the observed images swap East and West upon the meridian flip).

We can set up a Mark I dataset, to be updated later as we find issues or solve the remaning (small) alignment issues.



Effect of SKE on halo

Post-Obs scattered-light rem. Posted on Jan 22, 2013 13:39

We have some images of the SKE inserted in the ray path of our telescope, during Moon observations. We also have some images of the same from the BBSO telescope, kindly lent to us by Phil Goode.


BBSO images om two separate periods about a week apart. Top row shows histogram equalized imges of the BS (left and the DS (right) with SKE superimposed over the BS. Lower row shows the same – BS short exposure on left and DS long exposure with SKE on right.

Histogram-equalized image of the Moon with SKE inserted from our own telescope. SKE runs from upper left to lower right and is opaque above that line.

Our image shows a halo around the Moon that stretches behind the SKE – hence we know that halo is formed to a large degree in the secondary optics! On the BBSO images we do not see this as clearly – either because the BBSO telescope is of a different design or because the exposures with SKE (right column of images) covers much more of the BS. Assuming the latter we understand why there is no halo behind the BBSO SKE – the BS was effectively blocked and no bright light entered the secondary optics and could not cause a halo. On our image the SKE is inserted experimentally only, leaving a large part of the BS to shine into the secondary optics.

So, we learn that the halo is not from atmosphere or primary optics alone – apparently a large fraction of it comes from the secondary optics!

We also see the importance of having an SKE. While the BBSO group uses the ‘linear BBSO method’ to remove scattered light over the DS they have a much smaller problem than us because the halo from the BS is nowhere near as strong as ours is!

We now see how terribly important the SKE is.



Natural variability in albedo

From flux to Albedo Posted on Jan 22, 2013 09:09

Following on from post we now inspect satellite images one week apart in order to understand the natural variability found in satellite images of the same area.

Upper panel: average image pixel value for sequences of MTSAT images, 1 hour apart, for almost one day in March 2013 and a week later (red curve). Image pixel values is in arbitrary units but is proportional to pixel brightness. Lower panel: difference between upper panel black and read curves, expressed as a percentage of the mean of ther ed and black curves.

The difference between the two curves is on the order of 10% and varies from 8% to 18% during one day.

This tells us two things: Albedo (or something proportional to albedo) can vary by roughly 10% over a week. Albedo can vary during one day by almost as much.

This is useful information to have when we interpret the earthshine data.

We keep in mind that the smooth variations in the black and red curves in panel one are due to the day/night cycle – not intrinsic albedo variations: but the difference between the curves and the variability in the difference tells us about albedo variations.



Satellite images

From flux to Albedo Posted on Jan 21, 2013 09:35

We can reduce our observations to a number that is equal to the ‘Lambertian albedo of Earth’ at the moment of observation. That is, for a sphere behaving like a Lambertian sphere we can find the single-scattering albedo that gives the same earthshine intensity as we observe. This takes phases and all relevant time-dependent distances into account.

In order to understand if the data we get are realistic we wish to compare to satellite images of the Earth. From work published by others [the Bender paper] we are told that the terrestrial albedo varies by many percent from pentad [avg, over 5 days] to pentad.

Many of the best observations we have have ‘sunglint coordinates’ over South-East Asia. One geostationary satellite hanging over that spot is the Japaneese ‘MTSAT’. We may be able to get data from it, but have only ready access to a Meteosat that hangs over the Indian Ocean.

From the Indian Ocean Meteosat we have extracted a series of images for a given day, in order to start to understand what sort of variability we shall expect on time-scales that are shorter than a pentad.

We have access to half-hourly images from the satellite and have 12 hours of data – 25 images. We take the average of each image and plot it:

The black line is the observed mean intensity of the whole-disc image, and the red curve is a 6th order fitted polynomial. The difference between the black and the red curve has a standard deviation of 0.6% of the mean of the black curve.

The large-scale behavior of the black curve is due to the phase – we see half a day pass as seen from the satellite so Earth changes phase from new to full to new again. Variations over and above that would be due to changes in earth’s reflectivity. There is a pronounced sunglint in the Arabian Sea near Noon.

Some pixels in the image are saturated.

What do we learn from this?
By removing the fitted curve we learn how much variability there is as a day passes. Some of this would be due to the curve not actually being a 6th order polynomial – so the variability we get from the residuals are an upper limit. Fitting a 7th order polynomial lowers the S.D. to 0.4% of the mean.

It seems that the presence of a sunglint for some of the frames (the sunglint moves on to land – Horn of Africa – in the local afternoon and does not ‘glint’ in the sea anymore) does not generate a ‘spike’ of any kind in the average brightness.

We should remember:
That average brightness is not the same as albedo since the brightness depends on albedo times a reflectance function. But most of the reflectance behavior is removed via the polynomial fit, we think.
That cloud-patterns on this scale hardly vary much.
We do not yet fully understand the processing of the satellite images – are they normalized somehow? Were they all taken with the same exposure time?

So?
On this day at least variability in the albedo was much less than 1%.
We should inspect several days of images to see if the average level differs much from day to day.
We should try to get information from MTSAT, and at higher cadence – we can observe the earthshine with minute spacing.
Try to get more technical information about the satellite images.



Wildey digital lunar atlas

Relevant papers Posted on Dec 31, 2012 09:26

This is the digital atlas of the Moon produced by Wildey which we scale the Clementine map to.



Laplacian method applied to all good observations

From flux to Albedo Posted on Dec 30, 2012 22:41

The Laplacian method has been discussed here

http://iloapp.thejll.com/blog/earthshine?Home&post=283 .

We now extend the method to include an identical analysis step applied to synthetic images generated for the moment of observations. We then take the ratio of the results from observations and the results from the models. This will eliminate the effects due to geometry and reflectance (as long as the reflectance model used for the synthetic images is correct), leaving only the effects of changes in earthshine intensity. So, this is another ‘ratio of ratios’ result. The ratios involved are, in summary:

(Laplacian signature at DS/ Laplacian signature at BS; in observations) divided by
(Laplacian signature at DS/ Laplacian signature at BS; in models).

We use only the ‘good images’ identified by Chris in this entry: http://iloapp.thejll.com/blog/earthshine?Home&post=277

We start by inspecting the dependency of (Laplacian signature at DS/ Laplacian signature at BS; in observations) on phase [attention: this is not yet the promised ratio of ratios – just a ratio! :)], in each of the available filters:

We see the morning and evening branches folded to the same side, for comparison. We see an offset between the branches in each filter, and we see the dependency on lunar phase. We next look at the same ratio but from models. Since we do not have colour-information in our models we redundantly now show plots for each filter as if the models were color-dependent – they are not: but the points available in each filter are different, of course. Recall that a model is generated for each observation:

We see a very similar pattern – dependency on phase (but steeper this time). We also notice that the branches are closer together than in the observations.

What does this mean? The branches are separated, in observations, due to different distributions of light and dark areas on the eastern and western halves of the lunar disc facing Earth. Our Laplacian method samples pixels right at the edge of the lunar disc and has evidently met areas of different albedo. The separation in observations is not reproduced in the models – this can imply that the model albedos are incorrectly distributed (i.e. mares and craters etc in the wrong places) this is somewhat unlikely as we use one of the most detailed lunar albedo maps, from the Clementine mission. However, our model does not sample colour – the map used, and thus the ratio between ‘light’ and ‘dark’ – is taken from the 750nm Clementine image [I think; must check!]. This map was stretched to match the older Wildey albedo map [see here: http://iloapp.thejll.com/blog/earthshine?Home&post=253 ] which was made in such a way that the ‘filter’ the WIldey map corresponds to is a combination [see: http://iloapp.thejll.com/blog/earthshine?Home&post=286 ] of the Johnson B and V filters. We see the most well-reproduced branch spacing in the V band observations, compared to the models. This implies we have some colour-information about the two halves of the Moon – or a tool for how to scale the lunar albedo map when different colours are to be considered.

We next inspect the ratio of the observations and the models – the ratio of ratios:


For each filter is shown the Albedo derived – it is the ‘ratio of ratios’ spoken of above and is identical to the one used in BBSO literature – it is the albedo relative to a Lambertian Earth-sized sphere. The model albedo used was 0.31 so the ‘actual’ albedo derived is the above times 0.31.

We see that there is a phase-dependence in this – particularly in one branch, with the observations being relatively brighter than the models at phases nearer Full Moon, compared to phases nearer new Moon.

Since we have seen the EFM method produce less phase-dependent albedos [see here: http://iloapp.thejll.com/blog/earthshine?Home&post=252 ] we think the Laplacian method needs further development and investigation before it can be used.

It is worth listing why the Laplacian method might be useful:

1) It does not require careful alignment of model and observation. The signal is extracted from a robustly defined location in each image.
2) It is a ‘common-mode-rejecting’ method and is not dependent on image resolution.
3) It is relatively fast.



More CPU/GPU tests

Post-Obs scattered-light rem. Posted on Dec 30, 2012 00:07

I have been looking at the use of GPUs versus CPUs for our scattered light analysis. We need to be able to convolve artificial Lunar images (outside the atmosphere) with the instrument PSF. GPUs offer a considerable speed advantage.

First look (CPU versus GPU):

Upper left panel: artificial lunar image outside the atmosphere.

This artificial image is then convolved with a 2-D Gaussian-like PSF
which has fat (powerlaw) tails, and which closely reproduces what we see
in real data.

Upper right panel: convolution using 2-D FFT code running on a CPU

Lower left panel: convolution using 2-D FFT code running on a GPU

Lower right panel: the ratio of the two methods, i.e. the ratio of the two previous panels

There is a lot of structure in there, mainly images of the lunar
crescent turning up in different places — at a level of about 0.1% of
the intensity.

IMPORTANT: the CPU code was written in double precision, whereas the GPU was in single precision.

(The above reproduces with more explanation an earlier post)

Notes: The CPU code calls the FFTW3 libraries from Fortran (Dec’s ifort compiler is used), just using the standard Fortran to C wrappers provided with FFTW3. The GPU code is in written in CUDA.

Second look (CPU only, single versus double precision):

The plot above shows the ratio of the single precision CPU versus
double precision CPU (i.e. no GPU results shown on this plot).

There is similar structure in the ratio — and at about the
same level as the GPU tests gave, i.e. discrepancies at the level of a
few x 0.1% of the intensity.

Third look (CPU in double precision, renormalisation)


In this plot
we compare CPU double precision, applied to the ideal Lunar image, and without “min/max
renormalisation”. (Min/max renormalisation means scaling the input image so that the smallest value in the frame is 0.0 and the largest value is 1.0).

The ratio panel of the two convolutions (bottom right) shows noise only, and at a
very low level — 1 part in 1E7. Highly acceptable!

Fourth look (CPU, single precision, renormalisation)


This plot shows the same as the previous one — but with single precision rather than double. The artefacts are back, at the same old level of a few x 0.1%!

Thoughts:

We might already be able to conclude from the above that double precision FFT/CPU is robust (negligible
artefacts), but that a single precision CPU, or a single precision GPU,
produces similar sized (few x 0.1%), and thus slightly worrying, artefacts.

But I need access to a double precision GPU to test this. Hope to do so next week!
The acid test will be the results of comparing double precision on a CPU to a GPU.



Laplacian revisited: A Brain-Wave

From flux to Albedo Posted on Dec 27, 2012 09:52

On this blog we have previously investigated the laplacian method of estimating earthshine intensity. [See here: http://iloapp.thejll.com/blog/earthshine?Home&post=273
and here http://iloapp.thejll.com/blog/earthshine?Home&post=272 .]

One of the limitations we realized then was that image resolution influenced the results from the Laplacian edge signature – the more the edges of the image are smeared the lower the Laplacian signature estimate will be.

We now revisit the method after realizing that the whole image is affected in the same way by resolution issues – the intensity estimate derived from the DS and BS edges will both depend on image resolution in the same way. The DS estimate is proportional to earthshine intensity, while the BS estimate is proportional to ‘moonshine’ intensity (plus earthshine – a vanishing contribution). The ratio of the two estimates will be independent of resolution – i.e. the ratio has the ‘common mode rejecting’ property.

We convolved ideal images of the Moon with the alfa=1.8 PSF and estimated both the ‘step size’ at the edges (on DS as well as BS) and the Laplacian signature. We plot them below:

The top panel shows the step size estimate against the Laplacian signature estimate on the DS only. We see an offset, with Laplacian estimate being smaller than the step size estimate. This is caused by the resolution issue – the edge is fuzzy and the Laplacian is degraded. The step size is estimated robustly from the original ideal image (i.e. before the convolution is performed) and is thus not dependent on image resolution.
The bottom panel shows the ratio of DS and BS estimates – x-axis is the step size estimator and the y-axis is the Laplacian estimator. We now see that the dependence on image resolution is gone from the Laplacian estimate. The correlation between the two estimates is good, at R=0.92, but not perfect. Noise was not added to the images. Estimates of Laplacian were performed on the line through the disc centre. The step size estimator is based on the average of 9 lines through the disc center.

[Added later:] Since we are working with synthetic models we know the actual earthshine intensity in each image, so we can compare what the actual intensity is to what is found with the ‘step’ and the ‘Laplacian signature’ methods:

In the first panel we see what we saw above – that the step and Laplacian methods are rather consistent. In the panel below we compare to the actual earthshine intensity.

Since the Laplacian signature method (and the step size method) literally express the DS/BS intensities ratio (which is only proportional to the earthshine intensity) we have different values along the x- and y-axes. The geometric factors having to do with distances and Earth’s radius are not compensated for.

The main result is that the relationship between actual earthshine intensity and the Laplacian signature method (and by extension, the step size method) are not quite proportional – there is a slight curve. In interpreting the above we should keep in mind that the Laplacian estimate of DS illumination from the edge derivatives is in a different role than the estimate of BS illumination – the former is not very geometry dependent while the latter is: That is, earthshine is due to light from a source we are sitting on doing our observations from, while the BS illumination is more angle-dependent in that the reflectance properties of thr Moon come into play to a larger degree – at the moment I am not sure whether there is an angle-dependence ‘along the BS edge of the disc’ that can cause a problem in interpretation. Will need to look at this.

Another thought: is it possible to make a calibration relation between actual earthshine intensity and the Laplacian DS/BS signature estimate?



A whiff of success

Post-Obs scattered-light rem. Posted on Dec 19, 2012 14:07

We do have many problems to contend with – but now and then we are confirmed in what we set out to do: Here is an example of the scattered-light reduction of an observed image (2456034.114etc) where the EFM Method seems to be doing well compared to BBSO linear. We compare to a synthetic model image generated for the moment of observation (i.e. the libration and geometric factors are representative of the observing situation):

Top panel: slice across the Moon at row 296 so that Grimaldi near the edge is transected. The black curve is the synthetic model unconvolved – it is in units of W/m². The red curve is the scaled and offset profile along the same row of the EFM-cleaned image; and the blue curve is the BBSO-linear cleaned image identically scaled and offset.
Middle panel: detail of the above.
Bottom panel: difference between EFM and ideal and BBSO-linear and ideal, along row 296, expressed in percent of the ideal value.

The observed data were scaled since the units are different; they were offset because the EFM model did not have 0 value on the DS sky. The BBSO-linear, being ‘anchored in the DS sky’ did have a 0 value in the sky. The same scaling factor and offset was used on EFM and BBSO, for comparison, however – hence the little offset on the DS sky at right.

What do we see? Well, the EFM-cleaned (red) line follows the synthetic model quite nicely between column numbers 325 and 370 while the blue line (BBSO-cleaned) diverges all the time. Near the DS edge (columns 370-385) the synthetic model is higher than EFM.

What does it mean? The BBSO-linear method has better removed the flux on the sky – it is designed to do that, while the EFM is designed to minimize the residuals squared in a mask on the sky around the Moon. This implies that the BBSO-linear method, in the present case, would be better than EFM on the lunar disc near the DS sky. As we move further onto the DS disc the BBSO-linear method will fail more and more since the halo is not linear with distance from the edge – the method underestimates the amount of scattered light on the disc between the edge and the BS. We do see how the blue line diverges more and more; we do see the red line cling closer to the ‘true value’ (assuming the synthetic model is ‘true’) across the disc, before it too fails nearer the halo and the BS. The behavior nearer the edge may be a consequence of how we model the synthetic images – we have to use a reflectance model to make the synthetic images – and the angles of emergence and incidence corresponding to ‘near the edge’ is not one for which the reflectance model is inherently very good. The models we use are based on the simple ‘Hapke 63’ model. We speculate that the disparity between observations and synthetic models in the columns 370-385 is due to model inadequacy.

Anyone using a reflectance model as simple as the Hapke 63 model will encounter the above problem if they try to use pixels near the lunar disc edge – the natural thing to do, if the halo is removed using the linear method is to use edge or near-edge pixels.

Hence the problem of reflectance modeling and the inadequate linear method become coupled! Our EFM method seems to be a way around this obstacle – allowing use of disc areas further from the edge where simple reflectance modelling is adequate – hence we should be getting more reliable terrestrial albedo data. One day.



Are the PSF alfas correlated?

Post-Obs scattered-light rem. Posted on Dec 19, 2012 08:50

In the EFM-method we determine the alfa values for every image. Is there a link between the alfa value for one filter and the rest in an interval of time? It is our understanding that alfa is determined by the amount of scattering in the optics plus the atmosphere. We therefore expect that on ‘bad’ nights the alfa values will tend to move in the same direction. We investigate this here.

We find all alfa values in all EFM-treated images. We sort them into half-hour bins. We calculate all the alfa values in each bin and plot the results. Below is a pdf showing all the plots between some filters, at different ‘zoom-levels’. The image shows the last zoom-level, highlighting the dense ‘clump’ of points:


There seems to be a general agreement that the alfa values are correlated – bad nights (i.e. ‘broad PSFs’) occur in all filters at the same time. Since VE1 is just about identical to IRCUT the scatter seen above means that the fitting routine is unable to make a perfect match – or that observing conditions, during the half-hour bins used, changed.

Using 15 minute bins does not improve matters:
I therefore suspect that the fitting method does not find the best fit each time.



What colour is the earthshine?

From flux to Albedo Posted on Dec 17, 2012 14:19

What B-V colour should we expect for the earthshine?

We will here estimate it by using the change in colour of Sunlight that has struck the Moon once, and the colour of Earth as estimated from spacecraft.

The Sun’s B-V is +0.650 [Allen, 1973] [Holmberg et al, MNRAS, 367, 449, 2006]

The Full Moon’s B-V is +0.85 [Lane&Irvine, AJ 1976 78, p. 267]
[vdBergh has +0.876 for Mare Serenitatis;
Allen 4.ed. table 12.16 has ‘Moon’ B-V 0.92;
Gallouet (1963) has +0.94;
Wildey & Pohn (1964), AJ vol 69, p.619 have a range of values near +0.86 to +0.87 (their work seems good and a milestone).]

The Earth’s B-V is 0.2 [Allen 3. ed, but appears based on a 1961 work – so pre-spaceage?]

The Moon’s DS B-V is 0.64 on average given data in [Franklin (1967), JGR 72, p 2963]

If Sunlight is reddened by one reflection off the Moon by 0.85-0.65=+.2 mags, then we expect earthshine, bounced once off the Moon to redden by the same amount.

If the Earth has B-V=0.2 as seen from space then seen after one reflection it ought to be redder by +0.2 or appear to us observing it from Earth at B-V=0.4. This is not what Franklin measured.

Basically, we do not yet know Earth’s B-V colour! I am making inquiries, and we shall see.

Note that Danjon did lots of colour observations of earthshine – but in the Rougier system. Wildey [JGR vol 69, p.4661+] refers to a transformation from Colour Index (“C.I.”) in the Rougier system to B-V in the Johnson system – but without giving numerical details. The method is based on transformations using the Full Moon and the Sun colours.

The transformation should be made specific and the data from Danjon placed online. Another student project!



Case study in B-V: JD2456034

Post-Obs scattered-light rem. Posted on Dec 17, 2012 11:34

We compare the B-V values on the DS of images from JD2456034: we look at images only exposed to bias-removal (‘DCR’ images) and images cleaned with EFM and images cleaned with the two variants of the BBSO method: linear and log. We show a ‘slice’ across the disc at row 256:

Top left: black is the B-V slice from the DCR images; red is the B-V values from the BBSO-linear cleaned images. The second graph in the upper left panel is the difference between the B-V values (BBSO-linear images minus cleaned image). The vertical dashed lines show the edge of the lunar disc and the start of the BS in column units.
Top right: same, but for BBSO-log method.
Bottom left: same but for EFM images, as shown elsewhere on this blog.

The results are quite different – the BBSO-linear method has given us fairly constant B-V values across the disc – they are about .05 mag below the DCR values (i.e BBSO-linear are bluer than DCR values).

The BBSO-log values seem completely unrealistic.

The EFM-cleaned values also look a bit unrealistic in that there is a spatial dependence on the magnitude of the B-V relative to DCR – in a way that looks like a remnant effect of the halos. The Delta(B-V) value changes sign across the disc.

The night JD2456034 is very close to New Moon and we know from other results that this is when the BBSO-linear method is likely to work best (the halo being small). Since there is a phase-dependency in the overall results for BBSO methods over and above what EFM shows we know we cannot universally use BBSO results. On the DS, towards the edge, the BBSO-linear method should be very good – it is ‘anchored in the sky’, unlike the present EFM method, and therefore should be unbiased near the edge. Our EFM method, at the moment, only minimizes the square of the residuals on the sky, inside a mask.

Near the sky, the DS B-V values in the BBSO-linear and EFM-cleaned images differ by about 0.04 mags. We should look at EFM methods that also ‘anchor in the sky’ and see what we get then.

The present largest worry is not the B-V offset but rather the dependence on position on the disc that the EFM method shows.

What does the literature tell us we should expect B-V to be under earthshine?

At the moment I only know of Franklin’s 1967 paper. It gives B and V values of the earthshine.

http://adsabs.harvard.edu/abs/1967JGR….72.2963F

The average B-V seems to be +0.64. Can we find any other information on B and V?



More colour maps

Showcase images and animations Posted on Dec 14, 2012 06:22

I have made four colour maps now in B-V on four different nights.


The nights are arranged like this:

JD2455938 JD2455945
JD2456015 JD2456034

A uniform scale is used for all four images, from B-V = 0.0 to 1.4. The BS light is coming out in all four images around B-V=1.0, and the ES at around B-V = 0.6-0.8,
depending on whether one is looking at highlands or lowlands (lowlands are redder).

The cause of the black dips along the BS rim is the problem that the B images are slightly larger than the V images – so even with good registration of the center of the moon (to the closet pixel), there are issues in producing these colour maps. The falloff in the halo light cannot be the same power law in V and B, because the halo changes colour — but this is still to be checked.



Best of the data — averaged frames with 100 good slices

Observing log Posted on Dec 14, 2012 05:58

I wrote a program called moonraker in fortran, which processes all our data, seeking out the best for further reduction. Peter has written something similar in IDL.

It goes through all the images and throws out ones with

1) saturated BS (> 50,000 counts in the peak of the BS)

2) underexposed BS (<10,000 counts)

3) smeared images (heavy bleeding of photons in the Y-direction on the chip, reaching the bottom 20 pixel wide strips of the frame). A mean count level of more than 50 counts in this strip triggers a flag and the frame is rejected.

4) rough check if the moon is too close to the edges of the frame. This is not yet optimal, but throws out the worst cases.

5) checks that the temperature of the CCD is OK (an entry in the fits file header tells us if the temperature of the CCD has stabilised or not)

If all 100 slices in multiple exposure frames are OK, according to the above, they are bias subtracted (using the biases on either side of the exposure, averaged appropriately for each slice, and scaled to the superbias created by Henriette) and averaged, and written to disk.

6) The magnitude of the total light in the frame (i.e. apparent magnitude) is then compared to the expected V magnitude (using the JPL model, with corrections for the east and western sides of the moon, depending on which side is illuminated by BS), taking into account the airmass of the observation and the extinction for the particular filter. This isolates for removal exposures which have been taken with an incorrectly reported filter. Finally, after extinction correction, only those frames which fall within ~ 0.2 magnitudes of the correct magnitude using the JPL model are retained.

The list of the final set of good exposures using this method is attached (all100.doc — actually an ascii text file):

There are 534 good frames, spread from nights JD2455938 to JD2456104.

A summary is as follows:

Night, number of good frames in V, B, VE1, VE2, IRCUT, comments

JD2455938 2 2 0 5 0 fullish moon, some foggy frames
JD2455940 3 1 0 2 0 fullish moon
JD2455943 3 3 0 3 1 fullish moon
JD2455944 3 1 2 1 3
JD2455945 1 1 1 2 0
JD2456000 3 0 0 1 1
JD2456002 7 0 0 0 0 V band only
JD2456003 9 0 0 0 0 V band only
JD2456004 1 0 0 1 0
JD2456005 6 0 0 0 0 V band only
JD2456006 4 0 0 0 0 V band only
JD2456014 3 0 0 0 0 V band only
JD2456015 7 10 4 6 4 1/3 moon, lots of frames
JD2456016 11 10 9 11 11 “
JD2456017 7 11 0 11 2 “
JD2456028 5 5 0 9 0
JD2456029 0 0 0 6 2 2/3 moon, lots of scattered light
JD2456030 1 0 0 3 2
JD2456032 1 0 0 4 1
JD2456033 4 1 4 2 4
JD2456035 0 0 0 1 0
JD2456045 5 5 4 4 4
JD2456046 4 8 10 11 10
JD2456047 6 5 8 6 7 stars visible in some frames, useful for PSF
JD2456061 2 4 3 6 4
JD2456062 0 0 0 2 0
JD2456063 0 0 0 5 0
JD2456064 0 0 0 3 0
JD2456073 4 7 7 10 8 some eclipsing of moon – dome issues?
JD2456074 4 2 3 7 5 scattered light haze or fog?
JD2456075 8 7 8 7 8
JD2456076 9 1 0 2 0
JD2456089 3 2 1 1 0
JD2456091 2 1 5 0 4
JD2456092 0 0 3 0 5
JD2456093 0 0 1 0 5
JD2456104 5 7 5 1 5 some foggy frames, moon a bit close to bottom corner

I’ve inspected all of these by eye and they look mostly pretty good! There are still a few bad frames here because the scattering of the halo is not selected for yet — but the number of frames with low alpha (i.e. haze or fog) is just a handful.

Some of these nights will be pretty useful for looking at colour changes in the ES over time. I am doing that next.



B-V for JD2456034 – updated

Showcase images and animations Posted on Dec 13, 2012 10:06

In this – uipdated – posting we combine more images from JD2456034 – we construct B-V images in various ways.
The discussion refers to Chris’ originalposting:
http://iloapp.thejll.com/blog/earthshine?Home&post=275

The first images we look at are not the bias-reduced images Chris uses, but ‘EFM-cleaned’ images (i.e. scattered light has been removed). Both the B and V image used were co-additions of 100 images each. The two images were centered in the image frame. We look at tow sets of B and V images – the one at the top is cleaned with one setting of the EFM method; and the one below is done with another setting of the EFM method. Some of the features to the left on the sky are the effect of ‘cyclical overlap’ from the right side when the image is shifted.


We see the DS to the left and the BS to the right. The BS halo and the BS have become undefined (i.e. ‘NaN’) because either B or V is negative here (remember, it is not a bias-reduced image with full halo in place – these are EFM-reduced images so the BS and parts of the halo are now zero or negative. In overlap almost all of the BS and the BS halo have become NaNs!

Here is a ‘slice’ across the middle of the upper image:

And here is the same slice across the lower image:
And finally, here is the slice across the un-reduced B-V image – that is, the image formed from B and V images calculated from the images that were only boas-reduced (as in Chris’ plot).
The uppermost image has a large ‘dip’ when we get near the BS and its halo remnant . In the middle image the halo has apparently been better removed. STrangely, thereis least sign of a halo in the ‘raw’ images where nothing has been done to remove the ahlo. This needs to be discussed!

We used other airmasses than Chris. In my calculations the airmasses for the two images involved are:

B_am=2.545 image is 2456034.1142920MOON_B_AIR_DCR.fits
V_am=2.477 image is 2456034.1164417MOON_V_AIR_DCR.fits

I used kB=0.15 and kV=0.10, like Chris.

I did not solve for B and V by solving two equations with two unknowns – I iterated. Convergence was fast. I iterated on the whole images.

The first of the above EFM cleanups was done with a weighting of the mask used to define the area of the sky on which to reduce the sum of squares that favored the DS part of the sky. In the second attempt above equal weight was given to the RH and LH sides.



More colour mapping

Showcase images and animations Posted on Dec 13, 2012 04:42

Following on from the previous post, we have made a colour map based on two images (V and B) taken on JD2456034

2456034.1142920MOON_B_AIR_averaged.fits
and
2456034.1164417MOON_V_AIR_averaged.fits

Both are averaged results from stacks of 100 exposures, so the S/N ratio is 10 times better compared to the images in the last post.

V band exposure time was 72.5 ms, and the B band exposure was 222 ms, for a total of 7.3 seconds in V and 22 seconds in B. The observations were at an airmass of ~ 2.3.

The colour map looks like this:


with the colour scale (B-V) running from 0.2 to 1.3.

The BS (brightside) has a colour of around 1.0, which is still a little redder than the expected value of 0.9 (for BS on a full moon) by van den Bergh (1962).

There is more scattering of the light in B than in V, which accounts for the deep valley to the right of the BS, and the excess halo light beyond it. Colours here are not reliable. Colours just beyond the concave edge of the crescent on the DS are probably also affected by differences in the scattered light profile, so should be regarded carefully (this part of the moon can be examined when the crescent is on the other side). The left half should be pretty good!

The DS (darkside) has colours ranging from about 0.6 to 0.9 — with the lowlands redder than the highlands.



Lunar colour map

Data reduction issues Posted on Dec 12, 2012 06:00

We have used B and V images of the moon from JD2455859 to make a map of the B-V colour of the reflected light from a thin crescent moon.

We use the photometric transformations derived from our open cluster observations: http://earthshine.thejll.com/#post229

The B band image of the moon was shifted to the position of the V band image, and the colour computed from the fluxes in B and V directly from the pixel fluxes.

The map below shows the result:


The scale at bottom runs from B-V=0.1 to B-V=1.5.

The darkside (DS) has a B-V colour in the range 0.6 to 0.9 — whereas the crescent has B-V ~ 1.1. The measured colour of the BS is B-V ~ 0.9 (Van den Bergh, 1962, ApJ, 67, 147), so we may be a little too red. More work needed on this.

We are seeing roughly the colour difference expected between ES and the BS — ES should be a few tenths bluer in B-V.

There is a clear residual halo in the colour map, indicating that the scattered light around the crescent falls off differently in B and V filters (as we expect).

Very interestingly, the crater pattern becomes much harder to see in the colour map. The V and B images look like this (after registration) V on the left, B on the right:


The lowlands appear to be slightly redder than the uplands in the colour map, by a few tenths of a magnitude in B-V.

More work needed on this — this is just a progress report!

Technical info:

a) Images used were:
2455859.1313477MOON_V_AIR_DCR.fits and 2455859.1365256MOON_B_AIR_DCR.fits

b) The transformations used were (airmass is 2.6 for V and 2.7 for B, extinction coefficients for V and B are 0.10 and 0.15 respectively).

Vinst = -2.5*log10(sumv/exptimev) – 2.6*0.10 ! airmass = 2.6
Binst = -2.5*log10(sumb/exptimeb) – 2.7*0.15
V = Vinst + 15.07 – 0.05*bmv
B = Binst + 14.75 + 0.21*bmv

which implies that B-V = 1.35 * (Vinst-Binst) – 0.43

(the airmasses above were incorrect in the first version of this entry, they have been corrected now)



More on Laplacian method

Post-Obs scattered-light rem. Posted on Dec 10, 2012 16:31

In this entry on the blog: http://iloapp.thejll.com/blog/earthshine?Home&post=272 we discussed the Laplacian method of estimating earthshine intensity by removing the influence of the scattered light halo. We have tested it on a good image from near New Moon (i.e. lots of bright DS, very little interfering BS). Consider this plot:

In the top two rows we see the observed image, the EFM-image and the Laplacians of these. In the second row second plot there are three additional curves, in black. The centre one shows the estimate of where the disc edge is, based on the estimates we have of disc centre and disc radius. The other two lines are parallel outliers at + and – 40 pixels from the edge. In the third row we show first the coaddition of all rows along the edge, using the estimates of the edge and the outliers as start and stop – this allows a coaddition that centres on the disc edge. Second in third row is the same thing based on the Laplacian of the observed image. We see the signature of the double derivative of an edge. In the last row we show the same as in the third row but for the EFM-cleaned image and its Laplacian.

The application of the EFM method removes most of the halo and this is consistent with the slight lowering of the slope after the ‘step’ in the first panel of fourth row, compared to first panel of third row. We extract the size of the ‘jump’ as well as the distance between the minimum and the maximum of the Laplacian edge signatures. The jump size is estimated as the difference between the mean of the first third of the curve and the last third. We collect the four estimates of the earthshine intensity – two jump heights and two estimates based on the Laplacian signature, and plot them below for 4 differently rebinned versions of the original images – unbinned; binned 2×2; binned 4×4 and binned 8×8. This is done to ‘sharpen’ the edge in case the edge is ‘fuzzy’ so that the 3-point numerical operator used in constructing the Laplacian will interact correctly with the edge.

We see color-coded graphs. The estimates of earthshine intensity based on the observed image and the EFM-cleaned image are black and blue, respectively. The estimates based on the ‘signature size’ in the Laplacian of the observed and EFM-cleaned images are red and orange, respectively.

The differences are quite large – several percent, and there is a dependence on the rebinning of the image – even in the case of the step method. The results that ‘ought’ to be the best is the jump-size one based on the EFM-cleaned one since the halo is absent and because the jump size method should have less statistical error-of-the-mean than the signature size method since more pixels are involved in estimating the former. The error-in-the-mean for the jump estimates for the unbinned image is of the order of tenths of percent – clearly dominated by systematic effects.

The above suggests that there are large method dependencies.

The ‘jump estimator’ may be victim of the slopes seen near the jump – especially in the EFM case – with such slopes the mean before and after the jump is dependent on just how you estimate the jump. Possible improvements is to extend a linear regression from the first half or third of the points, before the jump to the midpoint and do the same from the other side and calculate the ‘jump size’ as the difference between the two extrapolations. The slope is still a concern, however. It is due to the lunar surface albedo variations. The Jump size itself is a function of earthshine intensity as well as lunar surface albedo and therefore on libration, as well as the geometric distance factors. Being estimated right at the edge the reflectance is a constant, leaving lunar albedo as a variable factor (through libration). This could be taken out with knowledge of the parts of the surface of the Moon right at the edge – i.e. via modelling.

The ‘signature size’ method has different challenges – it is based on fewer pixels for one thing. It also depends on the lunar albedo – and not on reflectance, provided the edge is used (and not features on the DS, as Langford et al do). However, the method depends on knowing that the image is in good focus. Bad focus will blur the edge causing a lowering of the signature size – for area methods such as the EFM we do not depend on a sharp edge but can estimate levels from regions chosen away from the edge. Of course, we still get blurring from lunar features overlapping.

The above study did reveal dependence on binning – but if blurring was being addressed the results are not encouraged as there was no convergence.

So many things to compare! Super student project! A study of artificially blurred synthetic images could be done, and ‘step’ and ‘signature’ methods compared.



Laplacian method

Post-Obs scattered-light rem. Posted on Dec 10, 2012 11:29

In an interesting article Sally Langford, et al. [ http://adsabs.harvard.edu/abs/2009AsBio…9..305L ] describe an analysis method for earthshine images. Essentially, the Laplacian’s effect on images is to detect edges and the size of the ‘jump’ in going from e.g. sky to lunar disc in the observed image translates into an amplitude of a characteristic signature of the derivatives of an edge. This enables the large-spatial-scale features of the scattered light halo to be removed from the short-spatial-scale features of edges and craters.

Our method for removing effects of halo is to construct a forward model of the halo, based on the BS of the Moon (caught in the same image as the DS – Sally’s FOV is smaller – about 20 arc minutes; our is 60 arc minutes), and subtracting it, leaving the DS almost halo-free.

How well does the Laplacian method work on our type of images? Sally’s images were well-exposed images of the DS alone – allowing large counts – 10’s of thousands; ours are co-addition of 100 images taken so that the BS is not overexposed, leaving the signal on the DS typically near 10 counts. One image with 10,000 counts should thus have the SNR of 100 of ours.

We took one of our good images – from JD 2456095 – near New Moon – and plot the profile across the image near disc centre in the image that we obtain using the forward modelling method (named ‘EFM’ in the plot and this blog) and the same profile from the Laplacian of the image. We average 20 rows.
Inspection of the columns (approx # 100-120) where the lunar disc edge is we see a clear ‘step’ up from the sky level in the EFM-cleaned image but there is no sign, above the noise, in the Laplacian of the same image. Inspecting the whole Laplacian of the EFM image we do see a faint signature:
The above image is ‘histogram equalized’ to show the feature. Our own method – also shown when histogram-equalized is here:

We see that the halo has been only partially removed on the right.

I think we would be hard pressed to extract a signal from the Laplacian of the image. It also bothers me that the whole earthshine signal is reduced to the value of the derivatives in just edge pixels. In our own image we have hundreds of pixels to measure on – in the Laplacian we get only a signal along the edge.

We should note that the remnants of halo on the right are much less evident in the Laplacian image, suggesting that there is a kernel of a good idea in the method. For reference, the Laplacian of the raw observed image is here (histogram equalized):

It does seem as if the Laplacian helps remove a lot of the halo, but also reduces the analyzable part of the image to the DS edge.

The Langford paper mentions both smoothing images and co-adding them – the former is done at the resolution of the worst image for a sequence. The Langford paper analyses features on the disk – not the edge.

Working out how the Laplacian is best used, on realistic images of the type we have, would be a good student project!



Bad image structures

Post-Obs scattered-light rem. Posted on Dec 04, 2012 16:23

We have, below, studied the ‘structures’ or ‘bands’ that, in some images, stretch from side to side. One concern was that the structure was induced by some step in image reductions. We therefore compare raw images to EFM-cleaned images to see if the structure is present in both.

Visual inspection of histogram-equalized versions of raw and EFM versions of the images reveals the structure, although it is easier to see in the EFM image since the halo and most of the BS is gone:

Plotting the average of the first 20 columns of each image, and scaling them to each other shows that the structure better:

The black line is from the EFM-cleaned image, while the red curve is from the raw image.

This seems to rule out that the ‘structures’ are induced by image-reduction steps.

We remain suspicious of the possibility that the bright light from the BS itself somehow is causing the problem. It is not a reflection since the structure is a deficiency of counts. Only way to explain it is by some process subtracting counts from these bands – or lowering the sensitivity of the whole set of bands – perhaps some sort of non-linearity caused by strong light in one part of the CCD affecting the whole row?

We still need to map when the structures is present – we want to see if it appears in a timeline or is a function of the placement of the image in the frame. Inspecting the first 700 images of 2000 for the EFM frames shows no sudden dependency on time of this problem.

Idea: also extract the maximum counts of the image and use this as a parameter – we do have suspicions that non-linearity sets in far below 55.000 counts.

Question: Why is the structure along rows, while readout direction is along columns? What is special, in a CCD chip, about the rows?

More to follow.



Effect of disc position on halo removal

Bias and Flat fields Posted on Nov 30, 2012 14:43

In the entry below, at:
http://iloapp.thejll.com/blog/earthshine?Home&post=269
we considered the need to remove some residuals left over when a poorly-fitted halo had been removed from the observed image. Sensing that the problem has to do with asymmetry in the solution forced by the lunar disc not being well centred, we consider now the effect of lunar position in the image on the quality of the halo fit. We estimate the quality of the halo fit from the mean value in a sky-patch near the DS.

We see here on the x-axis the value of the disc centre coordinate (i.e. column number in the image) and on the y-axis the mean value of the DS sky patch. We seem to have some scatter as well as a structure that looks like an inverted parabola, for these points near y-value 0. That is – the sky-patch mean value of the DS in images where the halo has been removed with the present EFM method depends on where the lunar disc is – the further away from x0=256 (middle of image) the Moon is, the larger a residual is left on the sky after the halo is removed with the EFM method.

We need to invent a better EFM method!



More Structure …

Bias and Flat fields Posted on Nov 30, 2012 08:45

In this post:
http://iloapp.thejll.com/blog/earthshine?Home&post=268
I pointed at the unwanted structure in the sky near the lunar disc in an image that was Bias-reduced as well as had had its halo removed. Bias problems were ruled out – and left was a speculation on internal reflections in the camera system.

If we are truly left with that horrible structure on the sky we cannot just remove the halo in the present way, as the method depends on ‘fitting the sky’. If there are basins and hills in the ‘sky’ the fit will be bad – if the halo subtracted has the right shape but not the right ‘level’ then perhaps we can think of a fix: After removing the halo, but before extracting photometry from the DS disc in the image we can reference the sky adjacent to the DS and offset our disc value from there.

I tried this, by estimating the sky level on a part of the sky near the DS on an image where the halo has been subtracted, following the EFM method:

The two patches on the lunar disc are the Grimaldi and Crisium reference patches (DS and BS, respectively [although the BS value is estimated from the same pixels in the raw image]) and the large semi-rectangle on the sky next to the DS is the ‘skypatch reference area’. We calculate the average value of that patch and subtract it from the value extracted for the DS. If the error is of an offset type, rather than a slope type we have then corrected for the halo misfit.

To see the effect of this problem we estimated the DS/BS ratio in all EFM-cleaned images, for each filter, with and without referencing to the sky level:

As usual, the jpegs above are poor in quality so we also post the pdfs:

Each panel contains three columns – the rows are for filters. The first column is the DS/BS ratio as function of lunar phase (Full Moon is at phase 0). The second column is the ratio of the DS and the total RAW image flux [for reasons explained elsewhere!]. The third column shows the ‘alfa’ value derived by the EFM method.

Almost the same list of images were used for the two plots above – note the differences by inspecting the alfa plots – crosses are present in one plot but not the other one.

We see that the sequences of points for the case ‘skyptach reference level removed’ are slightly fuzzier than the sequence where the reference is not subtracted! Before we can interpret this I think we need to restrict the two plots to the same set of images – I shall do this and return …

[later]: This has now been done. For the IRCUT filter (341 data points) we show the effect of skypatch-referencing on the DS/BS ratio expressed as obs/model:

First we notice the wide span in DS/BS along both axes – this is not new: this is mainly the phase-dependence we see – this is MUCH LESS than would be seen for BBSO-method treated images, and is not the point – the point is that the spread along y for a given value of x – say near x=1 is something like 50%: That is – the effect of removing the sky-level from the DS value, in EFM-cleaned images, is to alter the DS/BS ratio by 50%. This means that the EFM-method did not do a very good job of removing the halo. It did remove a lot of the halo – but clearly not so much that a considerable effect is felt when the small remaining offset is removed.

This is important for our Science Goals: On the one hand we see the EFM method work much better than ‘BBS linear’ and ‘BBSO log’ methods in removing the effect of phase on the DS/BS ratio [shown elsewhere in this blog], on the other hand we see it does not do a sufficient job.

We should note that the BBSO linear method removes the sky level in one step (since it is a fit to the sky near the DS) but that, being linear, it does a poor job of removing the halo where it matters most – on the DS disc. The EFM method may be much better at removing a halo of approximately the right shape – but there remains a bias that is important.

The current EFM method is not ‘anchored in the sky’ – it merely seeks to minimize the residuals formed when an empirically generated halo is subtracted from the observed image – as evaluated on a masked section of the sky part of the image. There are choices made when that mask is set up, and they are:

a) The mask used above was such that sky was included on both sides of the lunar disc, set off from the disc by some 20 pixels and curtailed vertically at 0.7 radii (so that a ‘band across the lunar disc, but not including the disc’ is set up). This gave weight to both the part of the halo on the BS and the DS.

b) The mask is quite large to give access to a large number of pixels so that the effect of noise on the fitting-procedure was reduced.

The requirement in b) forces the choice in a) to be so large that any unevennesses in the sky (such as we suspect occurs due to internal reflections when the Moon is near the side of the image) has an influence on the quality of the fit, forcing the above consideration of a ‘special removal of the sky level near the DS’. To this comes the problems of allowing the BS halo to influence the fit.

We should investigate how we can improve the EFM method. Possibilities include

1) spatially weighting pixels so that ‘difficult areas’ are avoided or are given less influence on the final fit – such as the bothersome BS part of the halo.

2) use much smaller areas to fit the halo on the DS to – such as a patch near the DS patch on the lunar disc. This has to be balanced – on the one hand we get more influence of noise (which influences the BBSO method too, as it uses ‘narrow radial conical segments’ on the sky), but on the other hand we eliminate the effect of the BS halo as well as the ‘unevennesses in the sky field’.

3) One hybrid form of the EFM, tested earlier, consisted of enforcing that the model halo should be flux conserving (as now) and at the same time pass through an average point on the sky near the DS in the observed image.

We should implement a method that does not show the above sensitivity to removal of a sky reference level calculated on the already cleaned-up image. Access to a system without these internal reflections would also be nice. One significant step to take before that could be to find all images without the ‘streak’ effect that lead to the above considerations.



Structure in unwanted places

Bias and Flat fields Posted on Nov 27, 2012 15:25

A strange structure has been found in some reduced images. To the right we show a dark frame taken seconds before the image on the left. The image on the left is an EFM image – i.e. the halo from the BS has been subtracted after fitting to the sky areas of the image. We clearly see the structure in the left image – a ‘band’ stretching to the left of the DS. This structure is not present in the dark image – so it is not an artefact of bias subtraction [Of course – the bias subtraction is not performed with adjacent dark frames – noise would be added that way – rather, a scaled almost noise-free superbias is subtracted. It is like a lower-noise replica of the image on the right – i.e. no structure, just mild level of noise.]. Below the two we show a plot along a vertical axis in the two images: columns 50 to 150 were averaged and shown in the plot as the black and the red lines.

There is a very clear structure in the sky of the Moon image. Given that it is not induced by bias-subtraction it must be due to the presence of the Moon itself. We speculate that:

a) it is some optical effect – reflection – from the inside of the camera or telescope. Halo-subtraction has reduced the sky level to almost zero but a little too much has been subtracted in the ‘dark band’ and a little too little has been removed outside the band. The fitting of a rather smooth ‘halo’ from the observed image could give this effect, if the structure itself is present in the image.

b) some electronic effect is causing the rows with the very bright BS in to somehow ‘jump low’ due to some effect we do not understand. NB: The readout direction is ‘down’ – not to the left!

Note the presence of what looks like a truncated halo to the left in the image, at the frame edge: this could be an internal reflection showing the right hand side of the halo being reflected inside the camera. We saw this same effect in the animated sequence of the tau Tauri occultation – as the Moon migrated to the RHS of the image frame an ‘echo’ appeared on the left.

This therefore seems to support idea a) above. If this is the case we may have an effect on solution-quality from position of Moon in the image. We should investigate if there is a position nearer the centre that eliminates the effect, and then omit images that are too close to the edge.



Scaled Bias

Data reduction issues Posted on Nov 23, 2012 09:06

One of the reduction steps performed has to do with the scaling of the bias images due to the thermostat-induced temperature variations in the CCD chip. This temperature variation causes a 20 minute period in the mean level of the bias with an amplitude of almost 1 count – thus of importance to our attempts to analyse extremely small signal levels.

We take Bias frames on both sides of all science exposures – one just before and one just after. If we were to just subtract the average of these frames from our science image we would be adding noise to the result – we therefore need to subtract a smoother bias frame. We have constructed a ‘super bias’ frame as the average of hundreds of bias frames – it is very smooth, but probably has a level that is unrelated to the actual level in each frame.

By scaling the super bias to the mean level of the average bias frames taken at the time of the science frame we get a scaled superbias that we can subtract – it has the right level and very little noise.

We need to understand how the scaling procedure performs, so we have extracted the scaling factor from the 5000+ exposures we have.


Top frame shows the factor on the superbias as a function of the sequence number, and the bottom panel as a a function of the Julian Day of the observation.

Most factors are near 1 but some stand out. 9 files have factors above 1.09 – their Julian days (integer part) are:
2455938 (7 images) 2455940 2456032 (one on each).
A list of the 209 images with factors over 1.08 is here:

The 4 unique JDs are: 2455814 2455938 2455940 2456032, with the majority of cases on 2455814.

These images should perhaps be inspected very carefully for problems with bias.

A close-up of the factors nearest 1 looks like this:



The best of data, the worst of data ..

Data reduction issues Posted on Nov 22, 2012 10:28

Using the methods described by Chris in the entry below – i.e. at

http://iloapp.thejll.com/blog/earthshine?Home&post=265

I have also selected for the likely good data by taking extinction into account and looking for linear sequences. This resulted in a set of images that I deem ‘good’. That list can be compared to Chris’ list.

Chris has 3162 ‘good’ images, while I have 2990. The cross between the lists finds 2273 instances on both lists. This list of ‘jointly agreed good images’ is here:

Methods for selecting good images differ slightly: Chris does not yet consider the alpha value found in the EFM method – I select for alphas in a narrow range near the mode of the distribution for each filter.

With converging selection criteria the list above would expand somewhat – perhaps best to keep the criteria a little different to avoid duplication of potential errors.

The joint list contains 49 unique nights. The distribution of images over nights is:

58 2455856
363 2455857
404 2455858
33 2455859
228 2455864
65 2455865
276 2455917
173 2455923
117 2455924
11 2455938
6 2455940
15 2455943
10 2455944
5 2455945
2 2456000
3 2456001
8 2456002
9 2456003
3 2456004
6 2456005
4 2456006
5 2456014
31 2456015
54 2456016
34 2456017
3 2456027
4 2456028
16 2456029
11 2456030
10 2456031
7 2456032
15 2456034
2 2456035
26 2456045
49 2456046
27 2456047
20 2456061
2 2456062
5 2456063
3 2456064
35 2456073
21 2456074
31 2456075
11 2456076
7 2456090
12 2456091
9 2456092
6 2456093
18 2456104

where the first column gives the number for the night. Note that towards the end of the sequence there are few images per night – that is because we were realizing the necessity to observe ‘stacks’ rather than sequences of single images which results in fewer co-added images.


A histogram of these data is shown above.



Identifying the best of the data

Data reduction issues Posted on Nov 21, 2012 23:35

We have been using the apparent magnitude of the moon compared to the JPL model of its expected magnitude (V band), to check which images suffer from gross problems, such as an incorrect filter in the beam, or light losses from thin cloud, or other reasons. This follows up on previous posts (http://earthshine.thejll.com/#post256, http://earthshine.thejll.com/#post254).

This plot shows the difference between the measured apparent V magnitude of the moon and the JPL magnitude (i.e the difference between instrumental and true magnitude) as a function of airmass. There are about 5000 frames, mainly of averaged data (i.e. up to 100 images averaged together) in the total data set (as of November 2012), of which about 1000 are in the V band, and plotted here.

(a pdf version is also available).

Blue represents one side of the moon (the side with Mare Crisium), green the other side. Opposite sides of the moon have differing distributions of uplands and mare, leading to about a 0.15 magnitude offset between the reflected light and the JPL model (which implicitly assumes a uniform disc). The dotted lines show the expected trend with airmass for an adopted 0.1 magnitudes/airmass extinction — quite close to what we are seeing in the data. There is a lot of structure in this diagram!

To clarify which side of the moon is which: the next plot shows the Crisium side illuminated by sunlight:


This plot shows the Grimaldi side being illuminated.

Now we concentrate on the crescent light on the Crisium Mare side (blue points in the plot):

Concentrating on just this side of the moon, we see a very tight (extinction) relationship with airmass, with notable outliers. They are all on the dim side of the relationship — indicating loss of light relative to the JPL magnitude). The reason for most of these has been identified — most often because of thin cirrus or haze (this affects the scattered light around the bright side of the moon, which we model with a power law with slope “alpha” — these are indicated on the plot with that symbol. The moon went behind the tower and cables on the western horizon in one case “cables” and one image has low S/N. Two have “?” — no reason was found for these outliers, so they are simply dropped. This leaves us with a nice looking sequence of data for this side of the moon. I’ll look into these further as there may still be structure in there as a function of lunar phase.

The other side of the moon, the side with Grimaldi and extensive Mare, shows much less clear trends with airmass than the other side.

The dotted lines are the same as in the previous figure — they bracket the good data on the Crisium side. Strange things are happening at low airmass (z<1.2) – it turned out that most of these peculiar data are from a single night (JD2455912) as the next figure shows (background of these points shown in blue). Note the airmass scale has changed to make these points easier to see.

The entire night of JD2455912 seems to be a dud — as similar strange happenings show up in the other filters — VE1, VE2, IRCUT (but not clearly in B, as only one good B image was taken on that particular night).

Analysis of the remaining (grossly) discrepant points shows that they are often all on a single night. The next plot shows some of these problem nights:

Blue: Totally discrepant data from JD246691 : drop these.

Red: Near full moon!

Purple: data from JD2456061,2,3 (three nights). For almost all, the moon is too bright by 0.3 mag — possibly bias issues? Will look at this.

The bulk of the green points lie along the expected extinction line, with an offset relative to the other side of the moon (Crisium) of about 0.15 mag. We can bracket these data and use them for the ongoing analysis of earthshine. We look at this in the remainder of this post.

The next plot shows both sides of the moon in green and blue crosses, but for all filters (instrumental magnitude – JPL V-magnitude) as a function of airmass. No data have been dropped yet — it’s all the nights we have.

The colour of the moon in the various bands shows up nicely as an offset relative to JPL V-magnitude (e.g. B-V is of order 0.9, second panel from bottom). The Crisium side of the moon shows much cleaner results than the Grimaldi side, in all colours. The dotted lines bracket the good Grimaldi data, and are the same in all panels. Data which appears between these two lines in other bands then V, mean that the V filter was actually in the beam, not the nominated filter. There may be similar effects for all filter combinations, but for the moment we can only pick out misfirings which resulted in the V-filter being in the beam. Only a small fraction of the data seem to be so affected, which is very good.

Extinction depends on the wavelength of the filter : this is clearly seen as differing slopes in the zeropoint (ie. the difference between instrumental and true (JPL) magnitude) with airmass.

Once the grossly erroneous observations/nights are cleared out of the data, the sequence of good data as a function of airmass can be picked off the plot (we call this tunneling).

Firstly, opposite sides of the moon are reduced to the same scale, by adding 0.05 mag to the magnitudes (in all bands, B, V, VE1, VE2, IRCUT) on the Crisium side, and subtracting 0.15 mag from the Grimaldi side photometry. This brings both sides onto more or less the same sequence with airmass. We then estimate (by eye) the extinction as a function of airmass in each filter. The results are:

B 0.15 mag/airmass
V 0.10 mag/airmass
VE1 0.08 mag/airmass
VE2 0.06 mag/airmass
IRCUT 0.12 mag/airmass

Interestingly, the extinctions in IRCUT and VE1 are somewhat different, which we wouldn’t expect as the filters are ostensibly rather similar. We should look into this further. In any case, these extinctions allow us to define lines about 0.3 mag wide around each sequence for each filter and pick off the best data. The result is this diagram:

A list of these frames is here:

There are 3162 frames on this list — cut down from about 5000, so about 60% of the frames have survived the “tunneling” process. This is still a quite conservative list because I haven’t

1) checked for the Lunar centering,

2) checked for “alpha”, the scattered light parameter, except for those very obvious cases which showed up by eye.

Final plot: using just the good data, the extinction corrected apparent magnitudes at MLO are compared to the JPL model as a function of Lunar phase. The scatter is small in some of the filters — we are down to absolute photometry errors of just a ~ 0.1 mag relative to JPL, which is pretty good considering we cover a range of airmass mainly from 1 to 5, but with some of the data out to airmasses of 15!

There is a hint we can do better still, because there is still structure as a function of phase — for example, the B band magnitudes at negative phase are a bit brighter still than the positive phases — indicating we didn’t get the correction for each side of the moon quite right. There are also trends in the reflected light with phase, but this probably reflects the fact that phase and airmass are somewhat correlated (small absolute phases tend to get observed at higher airmass, because the moon starts the night closer to the horizon).



Where does the earthshine come from

From flux to Albedo Posted on Nov 05, 2012 13:14

The earthshine comes from the various parts of the Earth that are turned towards the Moon, and the Sun – all the clouds and oceans and deserts and ice-caps that are illuminated and visible from the Moon contribute to the Earthshine. Which parts contribute most?

We take a representative image of Earth, as seen from space, and investigate where the flux mainly originates.

Splitting the above JPG image into the R, G and B channels we can analyses where e.g. 10, 50 and 90 % of the light comes from. That is – we seek the pixels that contribute these fractions of the total flux, and identify them in images. Note that R,G and B refers to other wavelength intervals than the B, V VE1 VE2 and IRCUT bands we have – our B band is bluer than the ‘B’ used in JPG images.

In the three frames we see 3 rows (B, G and R, from the top) – on the left in each panel is the original R,G or B band image, while to the right are the pixels contributing to the 10, 50 and 90 percentiles of the total flux in the image. The order of the panels is: top left 90%, left bottom: 50% and top right is 10%.

In the 90% images at top left we note that the B image (top row) looks different from the R and G images – the light in the B band comes from atmospheric scattering – Rayleigh scattering, and aerosol scattering – as well as the ocean and the clouds; other bands have more of their flux coming from clouds.

Variations in the blue may therefore tell us more about the atmospheric state than do the other, redder, bands. The Rayleigh scattering is due to molecular scattering – as long as the composition of the atmosphere is the same this ought to be constant in time; but some of the blue scattering is also due to aerosols and thus we may have a tool to investigate variations in the aerosol load. The longer-wavelength bands will tell us more about the continents. All bands are quite dominated by clouds – a small cloud can reflect as much light as a larger un-varying continental area.

The above is repeated here on another image of the Earth – more realistic as it is half-Earth. Image from Apollo 8.

And here is the B,G,R images and the 90% percentiles:

Top to Bottom: B,G,R, Left: R,B or G-band image – right: 90th percentile image.

We again see that the light contributing to the blue image (top) is more diffusely distributed than in e.g. the red (bottom) case where most of the light comes from variable features like clouds. This implies that we should expect larger variability in our albedo data for the red images than the blue image.



Summarizing

Data reduction issues Posted on Nov 04, 2012 13:29

While waiting for the CCD to be fixed, I am summarizing the discussions we have had on this blog. The material should be used largely in section 6 of the manuscript we submitted, leaving the other parts of that paper – or splitting it into tywo papers. Here is the submitted manuscript as well as the start of a summary the summary is really just a collection of links to this blog.



Observatory Choice Returns

Real World Problems Posted on Oct 23, 2012 08:38

What is the effect on observational coverage if we have different numbers of observatories and observe in different ways?

Since this depends on what we mean by ‘observational coverage’ we define OC as ‘largest fraction of the time with continuous observations’. Note that this is different from, say, ‘largest fraction of days where at least one good observation was made’, OK?

For 1,2,3 … observatories chosen from the list of known observatories (in the IDL code observatory.pro) and evaluating at 15 minute intervals we get the following (non-optimal, but pretty good) results when January and July are combined:


Upper panel: The red curve shows OC for Moon above 2 airmasses and Sun lower than 5 degrees under horizon. Blue is same but for 2 airmasses. The dip at 5 in the red curve is an artifact of the search method we use – exhaustive search between 44 available observatories would be too expensive so we seach for best of 100 random picks of 1,2,3,4 … in the list of 44. Lower panel: The same, but evaluated for more stations, best of 200, and with a 10% random – uncorrelated – occurence rate of ‘bad nights’ (clouds, for instance).

We see that, compared to this we have less OC – that’s because that search was for ‘Moon above horizon, Sun below’ instead of the more realistic constraints used here.

We see that extending observations from AM 2 to AM 3 is equivalent to adding two observatories for midrange values.

We see that adding many more observatories is in the end a loosing proposition – the 7th observatory on the blue line adds nothing compared to the sixth.

Notes:
a)More exhaustive searches can be made, but takes time. This would probably smooth the curves above and also uncover slightly better solutions.
b) We have restricted the site choices to the positions of known observatories. Since most observatories are on the NH summer months (when the Moon is not as high in the Northern sky) there is a handicap.
c) The method is slow – because the altitude of Sun and Moon are evaluated from very precise routines. Simpler and faster expressions for altitude could be used – but one for each observatory would be needed.



Observatory choice

Real World Problems Posted on Oct 19, 2012 10:58

The Moon is not observable all the time from a single observatory, all year round. To get complete – or almost-complete coverage we ask: where should the earthshine telescopes in the network be placed? The plot below shows, with coloured symbols, when the Moon is observable (defined as Moon up and Sun down), for 5 observatories around the world. MLO is Mauna Loa, MSO is Mount Stromlo in Australia, LCO is Las Campanas in Chile, SAAO is South African Astronomical Observatory. The underlying curve shows the earthshine intensity for one month. The Earth was modelled as a cloud-free sphere with bright continents and dark oceans – hence the zig-zag nature of the curve. Addition of randomly placed clouds would tend to dampen the amplitude of the zig-zags by about a factor of two.

When the underlying curve is not covered by coloured symbols it means that the Moon is not observable from any observatory. We note that this happens particularly when the earthshine is intense – that is near Full Earth, which is New Moon. This is clearly because the Moon is almost never in the sky alone when it is New (it has either risen shortly before the Sun, or will set soon after the Sun). Observability is best near Full Moon (New Earth) near day 15 – but then observing earthshine accurately is very difficult.

On average a single observatory experiences the Moon above the horizon and the Sun below, 25% of the time. Given the choice of these 5 observatories the total observability depends on time of year – given in the panels.

Clouds and the need for the Moon higher in the sky cuts the observability.

The above simulation is for two months of the year – the declination of the Moon changes with the seasons so the relative contribution by each observatory changes with time.



The Daily Progression

Data reduction issues Posted on Sep 28, 2012 15:25

In this post, we noted that there is a daily progression in the obs/mod ratio of the DS/BS ratio itself (let’s call this thing ‘the relative albedo’ from now on!). We speculated as to its origin.

We have now calculated the “relative albedo” with two sets of terrestrial models – one a uniform Lambert-reflectance sphere and one a non-uniform sphere with ocean-land contrast but with a Lambert reflectance, while keeping the set of observations constant. We compare these two sets of results:

The upper panel shows the “relative albedo” for a non-uniform Earth and the lower one for a uniform Earth. The linear regression slope is printed on each line.

We note that the ‘progression’ for V has changed sign in going from a non-uniform Earth to a uniform earth. The magnitude of the slope on B has changed by 50 %.

We conclude from this that the ‘daily progression’ is due to the terrestrial modelling of surface albedo on Earth.

Or: we can state that we are seeing actual surface albedo variations during a nightly sequence of observations!

During this set of observations (night of JD 2456046) The B-band albedo of Earth decreased slightly, while the V-band albedo rose. This is the albedo of the half-Earth shown in the figures in the post here.

For fun, we plot here the B and V relative albedos for tow nights with overlapping phase intervals:

The two nights are 2456075 and 2456016. For the B progressions it looks like there is a difference, at phase -96 degrees of about 0.15 in the relative albedo, and given the scatter of the points we might be able to confidently see a difference 1/10th of that, I guesstimate. That is 1.5% at a given point in phase (that is, time).

BBSO discuss ‘1-2% per night’.

If we take the above results at face value we seem to be able to track albedo changes through a night. Averaging sucha progression, in order to get the ‘nightly mean value’ would, of course, yield an average value, and an average can be affixed with a ‘standard deviation of the mean’ which can become very small if you have enough points – and if it gives any meaning to average the points. It does not give that much meaning to average the above progressions since they change on scales longer than our observing sequences.

But anyway, we need to start understanding what BBSO does in their data-reduction. Notably – do they assume Earth is a Lambert sphere? If so, we have perhaps arrived at their “A*” quantity? If, even more, so – we now just need to observe for a few decades!

Given only 1.5 years of data we should for now focus on how we present the results, and how we get something ‘meteorological’ out of the data – as opposed to ‘climatological’. For instance, does the level-difference between nights 2456016 and 2456075 correspond to a change in albedo in the relevant areas of Earth that we can determine from Earth observation data from satellites? More clouds, perhaps? 2456016 is the upper sequence.

Let’s see! (more to follow…)



Lunar phase and azimuth

Data reduction issues Posted on Sep 27, 2012 12:49

For all our observations, we plot the lunar phase (in the convention we use) against the azimuth (degrees East of North) of that observations.

We see that most positive phases are observations East of the meridian; most negative phases are West. Only rarely has the Moon been followed past the meridian.



A look at some first results: B and V

Data reduction issues Posted on Sep 27, 2012 11:38

We consider now the DS/BS ‘ratio of ratios’ – that is, the DS/BS ratio in observations divided by the DS/BS ratio in the model images. We have used the Clementine albedo map scaled to the Wildey levels, and Hapke 1963 reflectances in the model images.

We look at the data from several nights in the phase-interval from -130 to -50 degrees (Full Moon is at 0 degrees; negative phases implies we are observing the Moon in the West (setting)):

What we see are short sequences of points – each sequence covering one night. We note how many of these nightly sequences ‘dip’ during the progression of one night. We note the smoothness of each nightly sequnce – this bodes well for our original idea that albedo could be extracted with high precision. Jumps from night to night are more troublesome! But we need to first understand the ‘progression’.

Inspection of the time-stamps on the data in a ‘progression’ shows that time moves to the right – that is the ‘ratio’ plotted starts out high at a larger negative phase and ends as a lower ratio at a less negative phase (moves left to right). Since the observations are in the West we are looking at a setting Moon, or increasing airmass along a progression.

On a single night several things happen:
1) the Moon is seen through different airmasses, and
2) the Earth turns.
[recall that the synthetic models contain all the correct distances as functions of time, so effects due to that should divide out].

1) If the effect is due to the increasing (or decreasing) air mass we need to understand how a ratio of intensities taken from two areas on the Moon, obtained from the same image, can show a dependence on airmass. Differential extinction across the lunar disc? really? We are looking at 10-30% changes in a single progression!

2) Since these observations are all at negative phase they may all represent the same configuration of Earth and Moon – i.e. we could be seeing the effect of the same parts of the Earth rotating into view through the night. From a fixed observatory there is a tendency for the same areas on Earth to reflect light to the Moon on subsequent nights – in our case either the American sector and oceans on either side, or the Austral-Asian sector.

We should not forget 1 – but let us turn to 2 for now. We can extract ‘scenarios’ for the Earth at the moments of observation corresponding to the data-points above. Inspectingh the observing scenarios could perhaps teach us what is going on.

We select the progression near -91 degrees. That is a set of observations from JD 2456047.75 to 2456046.87, or about 3 hours (the date is April 30 2012). At the start the Earth was in this configuration (crosses on the globe show the part illuminated by the Sun):

While this is the configuration at the end islike this:

That is, the Earth has rotated slightly during the observations and
more of the Asian landmass has swung into view, meaning that more of the
earthshine is coming from the Asian landsmass, than at the start.

Can we estimate the expected change in Earth’s reflectivity during this sequence? Yes. We have quite elaborate code for that, know as “Emma’s code” due to a student who worked on it in Lund. It shows the Earth from any angle at any time, with superimposed albedo maps of clouds as well as continents. A reflectance law is imposed. I think there is even some Rayleigh scattering added to give the ‘bright edge’ of the Earth (see any Google Earth image at startup, as you zoom in – the edge is bright, and that may even be realistic).

That code is quite complex and is not run in a jiffy. [We need more students!] But we can make a perspective model of the Earth using simple IDL routines. We can wrap surface albedo and cloud maps onto a sphere and view it from the angle the Moon is at, for any given time. There is no reflectance imposed – just the albedo maps and the perspective view. The two situations above look like this with cloud maps for the moment of observations (taken from the NCEP reanalysis product TCDC):


The simple average of the brightness of the two images above are 42.6 for the former and 40.6 for the latter – so it is getting dimmer by about 4% as time passes – perhaps because bright Australia (cloud-free at this time) is entering dusk. The cloud map images are available at 6 hourly intervals only, so there has not been time for the cloud image to change – it is simply rotated and other bits have become hidden by the day-night sector advancing.



How alfa relates to extinction

Post-Obs scattered-light rem. Posted on Sep 26, 2012 16:17

We have extracted the mean value of alfa (the parameter that describes how broad the PSF for a given image is) for a given night, and the corresponding value of the extinction coefficient for that night. We have only few nights where the extinction could be determined.

The plot of one vs the other looks like this:

(download and look, etc)

We see that there is a tendency for alfa to be narrowly constrained, and that the extinction has a broader distribution. In general there is no strong relationship between the values, but if we ignore outliers and the effect they have on the regression (plotted as a red line) we see a general tendency for high extinctions and small alfa values to be related: For B it is quite clear. V would be clear but for the outlier, IRCUT also seems to be clear. VE1 and VE2 are all over the place. A broad PSF is given by small values of alfa. We expect broad halos (i.e. broad PSFs) on hazy or turbid nights – nights on which extinction also should be large.

Factors that determine scatter in alfa are things like image focus and how the nonlinear fitting routine determined it should stop. Physical factors include haze and thin cloudyness on the night in question.

Factors influencing the scatter in extinction include the actual regression: we used all nights with more than 3 observations used to determine the airmass vs extinction line, and for which the determinations from 3 regression methods agreed to a S.D. of less than 0.02. The three methods were – ordinary least squares, and two ‘robust’ methods (“ladfit” and “robust_poly_fit” in IDL). While doing the actual regression it was necessary to eliminate some outliers by hand. Physical factors include haze and cludyness and whether the halo around the Moon was well captured inside the image frame.

The relatively low value of alfa for the VE2 filter is still not understood.



What the JPL model for the moon’s apparent magnitude gives us

Real World Problems Posted on Sep 24, 2012 12:09

We weren’t sure of how exactly JPL models the Moon’s brightness in HORIZONS, i.e.

http://ssd.jpl.nasa.gov/horizons.cgi

The following plot shows that JPL calculates the actual observatory-object distance in giving the Moon absolute magnitude, but does not use any albedo map of the Lunar surface — so it is symmetrical with phase on either side of new moon (say).

Lower panel : apparent magnitude of the Moon over a 12 month period (September 2010-2012) computed with Horizons, and shown in a narrow range of the illuminated fraction (40 to 50 percent). The scatter in the apparent magnitude around the trend is ~ 0.1 mag.

Most of this scatter is due to the changing distance of the moon around its orbit (between ~0.019 and 0.023 light minutes). The middle panel shows the magnitude if we correct the photometry to a standard distance (0.0215 light minutes) — this reduces the scatter to 0.03 mag. (The distance provided by Horizons includes the position of the observatory on the Earth — this can make a difference of up to ~12,000 km in the Moon-Observatory distance).

The upper panel shows the residuals in a least squares fit to the middle panel, as a function of Julian day over the 1 year sample period. This clearly shows that most of the 0.03 mag scatter in the middle panel is due to the changing Sun-Earth distance during the course of the year. Accounting for this reduces the scatter to <0.01 mag. We interpret this to mean that no surface features of the Moon are being included in the Horizons’ apparent magnitude estimate, since we expect considerably more scatter than that (but we are working on this!).



Effect of reflectance model

Post-Obs scattered-light rem. Posted on Sep 20, 2012 11:41

At the moment we are displaying our results by providing plots of ratios of the DS/BS ratio in observations, to that in models. We do this chiefly to get rid of common factors – such as solar irradiance and distance-related geometry.

What remains in such a ‘ratio of ratios’ are the effects of:

1) The Earth’s actual albedo,
2) the model’s Earth albedo,
3) Earth’s reflectance (real vs modelled),
4) lunar surface albedo’s in the reference patches (real ratio vs model ratio)
5) effects due to the choice of lunar reflectance model.

We are really only interested in 1.

2 is an assumed value so that the results we get for terrestrial albedo are relative to that choice. We use a value of 0.31.

3 is a choice – we expect that any errors made in this choice will be seen as a phase-dependency in the results, and we can therefore control or at least understand it. Earth is more Lambertian than the Moon. The Earth has edge-darkening, the Moon has very little. We use a Lambertian model for an otherwise uniform Earth.

4 is observable, but only with difficulty – you need a good total lunar eclipse, then the albedos in the two patches on the Moon – or their ratio – can be measured. BBSO has done this. We have not (yet). In the model we make a choice, based on which lunar albedo map we use. As long as it is fixed the results will be relative to that choice. Perhaps we can use published images of total lunar eclipses to extract the ratio?

5 is a choice – we have models for Lambertian reflectance, as well as the Hapke 63 model and other, as yet untested, more advanced reflectance models. We expect that incompletenesses in these models will be seen as a phase-dependency in the results.

Of the above only some could possibly induce a phase dependency: 3 and 5.

We have reduced all data using both lunar models based on the Lamertian reflectance and the Hapke 63 model. We show them next – look here for a discussion of what we are actually showing: ‘ratio of ratios’ and all that:


(as before, plot needs to be downloaded as it does not show up well on this blog).

The plot is for 5 filters with the EFM method applied. The left column is the ratio of DS/BS in obs to the same in model, while the right column is DS/total in obs relative to models, where ‘total’ means the disk-integrated brightness of all source counts (plus a few stars that we can ignore!).

The first page is for the Lambertian lunar reflectance. The last page is for the Hapke 63 reflectance.

We notice that left and right columns are quite similar. We notice that the ‘jump up’ at angles corresponding to about 40 degrees from New Moon is much smaller in the Hapke 63 model results. Inspection of the files that correspond to the individual points in the ‘jump up’ and those next to the jump, reveals that the ‘jumped up’ points have a different processing history: they are the result of coadding and averaging single image sequences, while the rest are stacks of observations that were averaged. Why these should be different is unclear, as yet.

We conclude that there was an effect of lunar reflectance model on our results – and that Hapke 63 is better than Lambert. This is not surprising as the Moon is well-known not to be Lambertian in its reflectance.

So – we are beginning to see observations constrain theory!

There is still some scatter to account for, and we shall return to this mattrer, using the estimates of nightly seeing available from the measured alfa.

There is also some ‘slope’ in the result wrt phase – so the EFM method has a success rate that is phase dependent. WHile the second-best method (BBSO logarithmic; not shown here) has some ‘upturn’ towards Full Moon (center of plot), the EFM has an even slope down towards FM. Perhaps some empirical fine-tuning of the method will remove this problem too.



Brightness of opposite sides of the Moon

Data reduction issues Posted on Sep 19, 2012 11:10

We have isolated the very best images in V band, in which the Moon is well centered and well exposed.

We measure the apparent magnitude of the moon in these images by simply measuring the total flux, and applying the standard photometry relations (i.e. http://earthshine.thejll.com/#post229).

We correct for extinction of 0.10 mag/airmass (from http://earthshine.thejll.com/#post249).

We then compare the apparent magnitude to the expected apparent V magnitude from the JPL ephemeris for the Moon (http://ssd.jpl.nasa.gov/horizons.cgi). This uses the relation quoted in Allen “Astrophysical Quantities”, which actually comes from Eqn 8 of this paper: http://adsabs.harvard.edu/abs/1991PASP..103.1033K.

The plot shows the difference in the apparent magnitude as a function of phase (new moon = 0). Blue and green show opposite sides of the moon.

There is a bit of scatter in these data, but there are two clear sequences around phases 50 to 100 showing that opposites sides of the moon differ in luminosity by about 0.1 mag. This is quite a lot less than we expected to see, viz. this post:


http://earthshine.thejll.com/#post253



Albedo maps compared

Post-Obs scattered-light rem. Posted on Sep 19, 2012 09:59

In a previous post (here) we have compared the Wildey and Clementine albedo maps. These maps are important for our synthetic modelling code since the albedo (along with reflectance assumptions and correct geometries) are the basis of constructing realistic model images, used in analysis. We can compare these two maps very directly by accessing both and plotting the mean albedo in boxes at common lon,lat positions:

Evidently, the Clementine albedos are lower than the Wildey ones by a factor of two for dark areas and by some tens of percent at brighter areas.

This may be an explanation for the discrepancy we have between observed morning/evening brightness ratios and modelled ones, which Chris Flynn put his finger on.
We seem to have [not shown, but material could be inserted] an observed difference (expressed in magnitudes) of 0.12-0.14 magnitudes at absolute phase 90 degrees between the morning and evening integrated brightness. In models, based on Clementine albedo and either Hapke 63 or Lambert reflectances the difference is more like 0.3 magnitudes. If the Wildey map is more correct than Clementine, in terms of the highlands/mare albedo, then using the ‘flatter’ Wildey albedos in the synthetic code would help on the morning/evening brightness issue by lowering that ratio in the models.

The issue with using Wildey instead of Clementine is that Clementine is a global map – Wildey only covers something like -89 to 89 degrees in longitude and something similar in latitude so there is no remedy for modelling under lunar libration. We could ‘scale’ the Clementine map, using the above relation and see what that gets us, though.

Before we do that, we should fully understand how our own synthetic code uses the Clementine map – I believe there is more to its use than merely being used as a lookup-table. Hans will be able to tell us about this.

A second-order robust polynomial fit to the above data (where points where Clem > Wild have been omitted) is:

W=0.067509410+C*0.50252315+C^2*1.3644194



Removing scattered light – 3 methods compared

Post-Obs scattered-light rem. Posted on Sep 18, 2012 10:34

Earlier we showed the performance of the linear and logarithmic BBSO method. Now we have added the EFM method.

As before, plots are not shown well on this blog, so please download this pdf file and look at it:

You see the same 5 panels on each page – one panel per filter. On the panels you see the behaviour of the ‘ratio of ratios’ against phase – that is, the DS/BS ratio in observations relative to a set of synthetic lunar model images.

First page: RAW data – i.e. no scattered light has been removed. Full Moon is at the centre of the plot so we see the effect of scattered light – the obs/model ratio is increasingly not 1 as we near FM.

Second page: the linear BBSO method has been appllied to images. We see a reduction in scattered light – points ‘move down’ towards the 1-line.

Third page: logarithmic version of BBSO method – a slight improvement is seen.

Fourth page – the new one: the EFM method – a quite large improvement is seen over the best of the others! Many of the selected points are now lying on a flat seqeunce, except points towards New Moon, and some outliers.

The EFM data has been ‘selected’ in the sense that the parameter alfa, determined from images, has been used to select ‘good cleanups’. A histogram of the detected alfa values was made and a notable peak found and only images with alfa in a narrow range were used for the above plot. Alfa between 1.67 and 1.73 were picked. The absence of VE2 points is due to this – the peak of the VE2 alfa distribution is between 1.60 and 1.66. We must investigate whether these are ‘good’ solutions. [added later: a brief visual inspection seems to imply that the 1.7 solutions are the good ones – not the majority, which for VE2, lies in the peak at 1.63. Hmm.]

Speculatively, we note the ‘turnup’ of points towards New Moon. We ahave earlier discussed that this may be due to some feature or failure of the synethtic lunar image model to correctly portray the Moon at large phase angles. On the other hand the ‘turnups’ now look less like a gradual sequence, and more like a ‘jump’ up. The ‘jumps’ occur near 40 degrees from New Moon – …. is this the ‘rainbow angle’? The jump seems large – the rainbow angle paper spoke not of factors of 2 and 3 and 4 but of percentages. On the other hand the phenomenon has never been observed, as far as we know. It would be nice to have points ‘outside’ the rainbow angle to see the jump go down again (if this is the rainbow, that is). We note here that as we get closer to New Moon the lunar sicle is narrow and it becomes increasingly important to place the ‘patch’ in which the BS brightness is measured – here is an opportunity to experiment with DS/total ratios, instead.

Work continues …



Comparing lunar albedo maps

Showcase images and animations Posted on Sep 14, 2012 14:35

We have been using the Clementine mission lunar albedo map for our work. There is an older digital map, by Wildey (The Moon, vol, 16, 1977), which we obtained access to thanks to Tom Stone of the USGS.

Here is a graphical comparison of the two:

Here they are again, now interpolated to the same 2.5×2.5 degree grid so that they can be compared numerically, along with their relative difference:
Upper left: Wildey map. Upper right: Clementine map. Lower left: the relative difference = (W-C)/C. There are some interpolation artefacts on the rhs of both maps as well as in some of the edges, brought about when the full-globe result from Clementine (orbiting lunar satellite) and one-side-only result from Wildey (ground-based telescope observations) are accessed.

The map of the relative differences reveals an overall albedo offset with Wildey being darker than Clementine. In the Mare areas differences of 40-50% are found whil ethe brighter highlands have differences in the 5-20% range. The differences in albedo is therefore not a constant but depends on albedo.

We see no obvious longitude or latitude dependence in the difference. This may be relevant to whether we are interpreting the Clementine map correctly in terms of angle-dependence in the conversion from observations to the map. Or the same convention was used in the Wildey map, produced 30 years earlier!

[added later: more here]



Removing scattered light – 2 methods compared

Post-Obs scattered-light rem. Posted on Sep 13, 2012 11:05

We evaluate the effect of two different scattered-light removing techniques on our data, comparing to no removal at all. We do this by considering the DS/BS ratio – and we look at the ratio of this ratio in observations to the same ratio in models – so it is a ratio of ratios, ok?

The graphics do not reproduce well on this blog so I put a link to a pdf file here: download it and read on.

There are 3 pages to look at, and on each page there are 5 panels – one panel per filter, one page per data set.

The first data set shows the ratio of ratios extracted from data where only a bias has been subtracted, plotted against lunar phase. Second page shows the ratio of ratios when the LINEAR BBSO method has been applied, and the last page shows the ratio of ratios when our logarithmic variant of the BBSO method has been applied.

Full moon is at phase 0.

We see that there is a scatter and there is a systematic dependence on lunar phase – the points ‘curve up’ towards full moon (the middle) and towards the edges (new moon).

Near Full Moon it becomes increasingly difficult to remove scattered light because the BS is closer and closer to the patch on the DS where photometry was extracted. The Eartshine is also weaker and weaker as you approach Full Moon because that corresponds to approaching New Earth.

Therefore a hypothesis was that the ‘curve up’ towards Full Moon was due to incompletely removed scattered light. Looking at the raw vs linear vs log methods it is evident that this ‘curve up’ is substantially reduced by application of either of the scattered-light removal methods. The linear method does well and the log method adds an increment of improvement.

I believe these plots are a demonstration that BBSO linear is not perfect – and that small improvements are possible. It is open to discussion what BBSO has done about this problem – light seems not to be completely removed in their method. How do they average their data to compensate? We should note that our image scales and observing methods (ND filters; coadd etc) are not identical.

Soon we will add a third method – the EFM!



Lunar apparent magnitude with phase

Data reduction issues Posted on Sep 11, 2012 05:22

The plot shows the apparent magnitude of the moon in V and B as a function of lunar phase (phase=0 is new moon).


We measured the flux in images in which the filter was reliably V or B and used the transformations determined from NGC6633 (i.e. http://earthshine.thejll.com/#post229) to get the apparent magnitude.

These data have been obtained for a large range of airmass (z) — from z = 1 to 10, with most of the data in the range z = 1 to 3. We derived extinctions of 0.10 mag/airmass for V and 0.17 mag/airmass for B, by comparing to the apparent V magnitude from the JPL ephemeris for the Moon (http://ssd.jpl.nasa.gov/horizons.cgi) (more about this below). The solid black line shows the V apparent magnitude as a function of
phase after extinction correction, and adjusting the zeropoint by 0.2
mag in V to fit.

Note that B is ~ 1.0 mag fainter (i.e. B-V ~ 1.0, as we’d expect).


The airmass fits are shown above : the plot is the difference between the apparent magnitude from the JPF ephermeris and our transformed instrumental V band (or B band) magnitudes, shown as a function of airmass. The two lines show 0.10 mag/airmass (V band) and 0.17 mag/airmass (B band) — they are not fits. There are some bad outliers, especially in the V band, which are probably due to the incorrect filter being in the beam.



PSF alfa estimated via EFM method

Post-Obs scattered-light rem. Posted on Sep 07, 2012 09:12

By running the computers for one week we have been able to process all observed images (both singles and those that had been coadded from stacks) with the EFM method, and thus estimated the exponent, alfa, that the basic PSF must be raised to to model the halo around the Moon. Plotting the histogram of 9000 values for alfa we get:

We see a broad distribution from about 1.3 to 2.1. Inspection of the results reveals that only the images with alfa values very close to ~1.7 are any good. The lower values correspond to hazy or even partially cloudy nights. The values higher than ~1.7 have yet to be examined.

A typical (histogram equalized) image of the residuals for one of the images with alfa near 1.71 is here:


A ‘slice’ across the image shows this:

Upper panel is the slice – black being the image, red being the fitted halo; second panel is a detail view of panel one showing the DS, vertical dashed lines showing the limits to the sky on which the halo is fitted; third panel shows the difference between the black and red curves in panel two.

We see that the EFM method has been able to fit the sky so that it is essentially just noise – even on the BS (right) side of the disc, and that the DS has been revealed for a wide area onto the disc itself, only near the terminator is there a problem with the subtraction – the residual dips down.



CCD camera rotation

Data reduction issues Posted on Sep 05, 2012 09:09

As discussed here, the CCD camera became twisted in its thread on the telescope at one point. The problem was fixed, but this means that some of our images have a slight rotation about the frame mid-point. This influences the success of subseqeunt data reduction steps: especially the steps that depend on extracting flux from specific areas on the lunar surface.

We therefore tested for the presence of a rotation angle by correlating a synthetic image made for the observing moment with each and every observed image, rotating the synthetic image until the correlation was maximum – in 1 degree steps.

We plot the detected best rotation angle as a function of image sequence number and date:

Top frame: detected image rotation angle vs sequence number, Bottom panel: angle vs observing day since start.

It certainly seems that almost all images up to number 1800 or so has a rotation angle of some -7 to -8 degrees. That seems to correspond to just a few nights near night 40-50. The detection of rotation is a bit spotty so there are also other episodes where a rotation angle other than 0 is detected – such as images 2000-2500. That more intermittent episode corresponds to a few nights near night 180, but there are a few more examples near night 220.

The CCD twist was correctd by Ben on JD2455991, and this datum is shown as a vertical dashed line in the plot above. Since this is not consistent with the angles measured we have to say that the test so far has been inconclusive!

Added later:

Actually, it was not impossible to inspect the relationship between model images and observed images visually and to confirm when an obvious image rotation was present. Partial results (note: more points than above) look like this:


Here color coding indicates in red the images that so far obviously have a rotation problem, and in blue images that show no obvious problem.

The presence of blue symbols at large rotation angle must be due to failure of the algorithm for detecting rotation! It is not an easy problem to solve – at New Moon there is precious little to correlate images on, unless the DS is used – but the presence of the halo gives problems, so that histogram equalization is not an obvious remedy.

A fixed derotation for the detected nights could be implemented – this affects some of the early observing nights where single images were taken (not stacks).

Added even later:

By manual inspection and image comparison, the following de-rotation angles were found for the JD in the beginning of our sequence:
2455856.1078938 7.711
2455857.0817247 6.881
2455858.0931277 6.881
2455859.1269613 0.000
2455864.7037639 5.352
2455865.7157216 6.116
2455886.0356274 0.000
2455905.9722493 0.000
2455912.0991783 0.000
2455917.1285750 0.000
2455923.7124300 0.000
2455924.7257543 0.000

Apart from 2455859 all angles before 2455886 were clear to find. 2455859 was hard to inspect as it is very near New Moon and almost no features were detectable.

As a working hypothesis, let us assume that all images before 2455886 must be rotated by something like 6 or 7 degrees, to bring them into good alignment with their synthetic models.



BBSO method results

Post-Obs scattered-light rem. Posted on Sep 03, 2012 15:26

The results for applying the linear BBSO scattered-light removal method to all data are further considered. Here we show data where the lunar disk is well centred and the radii extracted are between 131 and 149 (real range is 132-148) pixels. (Sorry about the image rotation!):


Rotate image and then use this caption: Results from comparing model images to observed images – models calculated for the observing moment but no halo generated. In the observations the halo has been attempted removed via the BBSO linear method. Column (1) DS counts to total counts for observation (black symbols) and same for Model in red symbols, against lunar phase; Column (2) observed DS to total ratio against airmass; (3) ratio of the DS/tot ratio for observation and models.Note that Full Moon is at 0 degrees phase.

We note that for lunar phases between New Moon and about 100 or 110 degrees the model and observed ratio of DS to total flux behave similarly. For phases closer to Full Moon the model DS/tot ratio is much smaller.

That is probably because near Full Moon the halo gets closer to the DS patch where the DS is measured: In the model no halo is present so the DS brightness is not polluted until the patch is actually part of the BS – in the observations the halo moves with the lunar terminator towards the BS and becomes harder and harder to remove so that it starts to pollute the DS – even if measured in images with some of the halo removed.

In the middle column we see a fair spread in observed DS/tot vs airmass. Most of the observations are for AM < 3, while a few have even higher airmasses. In the bulk of the observations we see no filter-to-filter consistent evolution of DS/tot ratio against airmass, and conclude that there appears to be no dependence on airmass; this is good as the whole point of simultaneous observing of DS and BS is that external factors, like extinction, affect both sides identically.

In the last column we see the ratio of the DS/tot number between observations and models. The behaviour noted from the first column, above, is evident here – the DS/tot ratio in models is much smaller than in observations for phases inside 110 degrees or so and the ratio of ratios goes up accordingly. For phases between 110 or 100 degrees and New Moon we see a more constant behaviour consistent with the observation made above that model and observations behave similarly. We see, however, a consistent pattern from filter to filter – there is some ‘curvature’ with phase. The inner part of this could be due to lingering halo effects, but the outer ‘curve up’ must be due to effects in the model, not in the observations. Near New Moon it seems that the DS/tot ratio for observations grows compared to the same ratio for models – the models’ DS is not as bright as in the observations near New Moon. We note that the DS model brightness is a function both of the reflectance description of the Moon and of the Earth.

Summary:

1) There is indication that halo-removal near Full Moon (phases between 0 and +/- 100 degrees) is increasingly incomplete.

2) There seems to be little or no effect of airmass on the DS/tot data.

3) There is some indication of a reflectance problem in the models for phases approaching New Moon – either in the Earth description or in the Moon.

Comments:

1) is good news – we thought we could only do removal up toi half Moon – the limit seem sto be 10 or 20 degrees beyond that, towards a fuller Moon. 2) is very good news – it opens up for our method to be applied at low altitudes, i.e. for small lunar phases near New Moon.
3 should be investigated by now redoing the whole reduction but with some simple and single change in the modelling – we cannot change the BRDF for Earth (it is always Lambertian), but we can change the lunar BRDF from Hapke 63 to Lambertian.

In these models the Earth was modelled as a ‘cloud free Lambert sphere’ with ocean/land contrast built into the single-scattering albedo. The Moon was modelled by the Clementine single-scattering albedo map with Hapke 1963 reflectance.



Method considerations

Post-Obs scattered-light rem. Posted on Sep 01, 2012 18:15


Residuals image of Moon minus model of BS and its halo, fitted to the sky (i.e. the EFM method). The green and blue colors are a few tenths on both sides of 0. The mask used covers the image except the disk of the Moon.

In this application of the EFM to an observed image of high quality we see that the residual sky is not even. Particularly on the BS (to the right) there are some streaks. A blob at the bottom also shows unevenness, as does the striations to the left of the DS.

This illustrates a shortcoming of the EFM. Since the PSF used is rotationally symmetric it is possible that the unevenness seen above are in the observed image – perhaps some sort of reflection effects in the optics. The EFM is a global method seeking to subtract all the scattered light with one model. The BBSO method is essentially local – it estimates the scattered light in a wedge off the disk. On the other hand the BBSO linear method almost certainly removes too little of the halo as you get nearer to the BS. The EFM method attempts to model all parts of the halo.



First results

Data reduction issues Posted on Aug 31, 2012 14:00

Here are some of the first results from applying the BBSO linear method – the reductions are slow so more will be on hand later! Sorry about the low quality in the image – there must be a way to do it better, but …

Rows 1-5 give results for each filter. Ignore columns 1,2 and 3 for now – they are diagnostic. Column 4 shows the ratio of observed terrestrial albedo to modelled terrestrial albedo as a function of lunar phase.

We pick out the DS as either Grimaldi or Crisium depending on which is in the DS. We then calculate the observed DS brightness divided by the brightness of the whole disk. We then do the same for the synthetic model, and plot the ratio of the two.

This may seem a strange quantity to plot, but consider that in the (unlickely case) that we both had perfect observations and the model was correct in all aspects, then we would see a ratio of 1.0.

If the model is somehow wrong – for instance if the phase function it is based on is unrealistic then the ratio would have a phase dependence.

As it is, we do not have perfect observations and we see a fair bit of scatter. The scatter comes about for several reasons – first of all the observations have Poisson noise – we are extracting a small 4×4 pixel patch on what is the dark side where counts are probably on the average of 5 or so. Additionally we have noise from the alignment between the actual Moon and the coordinates we have calculated from which to extract information – there is a missmatch of up to several pixels here, so for a small area like Grimaldi a few pixels error in placement brings you into the bright surrounding areas. For Crisium, which is larger, this is less of a problem.

Finally there is a still un-solved problem with synthetic models and observations apparently being off by some small amount in terms of a small rotation. This may be from the days when the CCD camera was actually physically twisted by a few degrees in its placement on the telescope. In such cases the intensities of the pixels extracted in the synethtic model and observed image are even more different.

So, some things to work on are:

1) Use larger DS patches, so that the Poisson statistics are not as much of an issue.

2) Put the patches in uniform areas on the Moon so that missalignments do not cause acquisition of contrasting areas. Inside large, even Mares or on the brighter highlands.

3) Use better estimates of disc centre and radius.

4) Figure out a way of aligning the synthetic model and the observed image better.

PS: More data are available all the time so the figure above will update now and then.

Added later:

Have implemented 1 and 2, by enlarging the area inside Crisium that is used, and using a rectangle in the highlands south of Grimaldi instead of Grimaldi itself. Also trying 3.



Linear and Log BBSO methods

Post-Obs scattered-light rem. Posted on Aug 30, 2012 12:50

The linear BBSO (i.e. as done by BBSO) method and the modification that operates on logarithmed images can be compared:

A strip, 20 rows wide, across the image was made and the rows of that strip averaged, and the average plotted. Black is the original bias-subtracted image, red is theordinary linear BBSO method and blue is the log-method. We plot the absolute value of the fluxes to avoid problems with log.

Both the linear and the log method does well on the sky at left, but there are differences on the edge-near disk. Tests on synthetic images, where we know the flux to expect on the disk, has shown that the log-method is more accurate than the linear method.

The residual mean (about 2) is unsettling, but has to do with the use of absolute values and the averaging over several rows – inspection of the images reveal values distributed around 0.



Differential refraction

Data reduction issues Posted on Aug 29, 2012 15:16

We have to determine the radius and centre of the lunar disc in order to reduce observations.

In doing that we must be aware that differential refraction causes the Moon to appear non-circular as it comes closer to the horizon. Using a formula from the Nautical Almanac Explanatory Supplement we generate the following table for 600 mmHg, 10 degrees C and 30% relative humidity:

Z d_refr am
——————
27 1″ 1.1
69 6″ 2.8
75 12″ 3.9

where z is the zenith distance in degrees, d_refr is the differential refraction in arc seconds and am is the airmass. The differential refraction is calculated over a 1-degree distance centred on the given zenith distance. Our FOV is about 1 degree wide and one pixel covers about 7″.

We thus see that the Moon is differentially refracted by less than one pixel up to about 2.6 airmasses. Two pixels are reached about 26 degrees (4 airmasses) above the horizon.

Some of our observations are certainly close to the horizon, as we have tried to observe when the lunar phase is near or less than 30 degrees (at Newish Moon).

At 30 degrees the uncertainty in our determination of disc radius and disc centre (based on a circular assumption) is thus starting to be challenged by the differential refraction.



Driving on the Moon

Data reduction issues Posted on Aug 29, 2012 14:14

In order to extract fluxes from particular areas of the Moon we need to take lunar libration into account. Based on the synthetic model code Hans wrote we can do that now.

On the left is the synthetic model. On the right is the observed image. On it are some dots – they are supposed to be inside Mare Crisium and Mare Nectaris and two other locations on the darkened side. We see that we nail Nectaris and Crisium. Doing this for every observed image we will be able to extract feature fluxes and compile DS/BS statistics.



Fluxes vs phases

Data reduction issues Posted on Aug 27, 2012 16:22

Since the setting of the colour filter (as well as the shutter) was unreliable we must find a way to detect which images are taken through which filters.

Here we plot the raw fluxes (counts divided by nominal exposure time):

Total lunar Fluxes plotted as magnitudes against the lunar phase (New Moon is at 0). Bright is up, faint is down.

The data for each image (black symbols), here expressed in magnitudes, are overplotted with a phase law (red) inspired by that in Allen “Astrophysical Quantities”, except we modify the coefficients in that and use instead:

mag=offset(ifilter)-(0.011*abs(ph)+4e-9*ph^4)

Notably the coefficient on the linear term is about half of what Allen specifies.

Particularly in VE1 we note the presence of two sequences of data. We have ‘fitted’ (by eye) the sequence that is represented on both sides of the new moon to the Allen phase law. The orphan sequence is below the adopted data suggesting that a filter with less transmission was obtained here when VE1 was requested.

For B it seems that intermittency causes some fluxes to be higher, but they do not fall in a delineated sequence so cannot be identified with a filter.

The V filter seems to have the same problem, although less so.

VE2 is also somewhat ‘broad’ in its distribution.

IRCUT shows two sequences.

The VE1 and IRCUT filters are extremely similar in transmission properties and are quite similar in the plot above, including the presence of the ‘second sequence’.

The ‘second sequence’ is quite similar in flux to the V observations, and it is consistent to say that when IRCUT and VE1 failed to be chosen V was obtained instead.

This seems to also apply to some of the outliers in VE2 and B.

We thus suggest that a working hypothesis for the failure of the filter wheel is that when it failed the V filter was selected instead.

We next proceed to eliminate the outliers in each filter so that a dataset can be defined which will allow identification of the extinction laws in each filter. With that in hand it may be possible to ‘tighten up’ the data and move towards a ‘golden dataset’ from which also DS fluxes are worth extracting.

The presence mainly on the left side of the diagrams of a ‘second sequence’ of data implies that something like a mechanical problem is behind the FW failure – because the telescope is flipped over the meridian to observe mainly in the East or the West depending on whether the Moon is rising or setting (before or after New Moon).



Synodic period of Moon

Post-Obs scattered-light rem. Posted on Aug 27, 2012 11:36

In order to precisely remove scattered light from the observed images we need to know the centre of the lunar disc in image coordinates, as well as the radius. These numbers are used by the BBSO method, while the EFM method can work without them. Extraction of fluxes from designated areas of the Moon also requires knowledge of the disc coordinates.

We have, as described elsewhere, found a fairly good way to estimate disc centre and radius, and have more than 5000 images (singles or sums of stacks) with know coordinate estimates.

We can check on the quality of these data by inspecting the time evolution of the disc radius in terms of the lunar synodic period (27.322 days).

Plotting the detected radius against observing time modolu 27.322 we get, for the 5 filters:

There seem to be some outlier groups, as well as a general scatter. The scatter is on the order of 2-3 pixels while the outliers reach 5. These outliers can be identified and the relevant images inspected.

We fit a general sine curve to the data, and get:

Filter: B
Offset Amplitude Period
141.169 7.98152 27.6367
+/- 0.0108026 0.0166443 0.00208776

Filter: VE1
140.663 7.81091 27.5796
0.0122441 0.0199136 0.00187041

Filter: V
140.647 -7.47896 27.5736
0.0413606 0.0837630 0.0111588

Filter: VE2
139.845 8.51341 27.5771
0.0245651 0.0383531 0.00435187

Filter: IRCUT
140.744 7.98575 27.6255
0.0358718 0.0596396 0.00708377

The period is not close to the expected 27.322 days. We expect this is due to a poor fit (in turn due to the outliers). We identify the outliers. 108 images are found that have radius more than +/-3 from the fitted sine curve.

Upon inspection, it turns out that not many of the identified outliers are obvious ‘bad images’. The determination of radius and disc centre is therefore somewhat deficient.



Reinstalling the SDK

Control Software Posted on Aug 25, 2012 20:50

Following the power surge and the consequent failure of the system, I installed SOLIS in case the software and drivers had been hurt by the power surge. This was apparently not a good idea – during construction of the system it was realized that ‘drivers’ from SOLIS and ‘drivers’ from the SDK are not the same and that only drivers from the latter must be present.

I therefore today de-installed (using the Windows 7 Control Panel) the Andor SOLIS package – as well as the unwisely installed Andor SDK package (both, apparently, with their provided drivers).

I then tried to run the ‘setup.exe’ present in the Andor SDK zip file – the setup offered to install itself into

C:\Program Files\ANDOR SOLIS\Drivers

which is supposedly not right – the SDK should install into

C:\Program Files\National Instruments\LabVIEW 2009\user.lib\Andor Tools

according to Ahmad. So I rebooted the system. Actually, it seems I chose ‘shutdown’ so it is not coming back up – must ask Ben to physically reboot.



Repeatability of finding center and radius of the lunar disc.

Error budget Posted on Aug 21, 2012 12:33

A required image analysis step is the determination of lunar disc centre and radius.

We currently use a hybrid method: First we fit circles to points on the BS rim (found as an image by using edge-detection image-analysis methods: SOBEL filters!) of the lunar disc image – this is done many times using different points and then the median is extracted for x0,y0 and radius. These values are then used as starting guesses for a more refined method that searches a range of possible values near the starting guesses and determines a ‘winner’ based on how well a synthetic circle matches the ‘rim image’ generated above.

We compare the statistics of the starting guesses and the final adopted values. In two separate runs on 88 different images we assemble these values:

Run 1:
x0 start guess: 311.82352 +/- 2.7447506
x0 final guess: 310.44694 +/- 2.3127377
y0 start guess: 248.49938 +/- 1.1339389
y0 final guess: 248.70556 +/- 1.0812740
radius start guess: 136.32425 +/- 0.98834739
radius final guess: 136.90833 +/- 0.44461002

Run 2:
x0 start guess: 311.78076 +/- 2.9802184
x0 final guess: 310.46767 +/- 2.3079720
y0 start guess: 248.51968 +/- 1.0747726
y0 final guess: 248.70327 +/- 1.1017349
radius start guess: 136.42162 +/- 1.2634192
radius final guess: 136.90870 +/- 0.40265642

x0 is found slightly more precisely with the final method (2.3 pixels vs 2.8).
y0 is found with similar precision in the two methods (about 1.1 pixels).
radius is found better with the final method (0.4 vs about 1 pixels).

We seem to be able to determine the centre of the lunar disc with image analysis techniques to the 1-2 pixel level, and radius to better than half a pixel.



Effect of registration errors on measuring the flux in a darkside patch

Error budget Posted on Aug 21, 2012 11:53

We have looked at the error in measuring the flux in a patch on the earthshine side if the position of the patch is uncertain by a few pixels.

A circular patch (aperture) was chosen, as shown by the green circle below:


The aperture is 31 pixels in radius, and is in a not particularly uniform luminosity area of the lunar disc.

The amount of scattered light into this area is rather small (~<10% of the flux) and has been ignored.

The flux in this aperture was computed for the correct registration (no offset in x or y) and for offsets of 1 and 2 pixels in both x and y (i.e. a rectangular grid of -2 to 2 pixels offset in x and the same in y).

Simply summing the flux in this aperture yields a 1.5% error in the flux, if the registration error is up to 2 pixels. The error reduces to 1.0%, if the registration error is just 1 pixel.

We tried rolling off the edge of the aperture with a cosine function (i.e. from 1.0 to 0.0) — it starts 6 pixels inside the edge of the aperture. We weight the flux in each pixel by this function. Pixels inside this rolloff zone are weighted with 1.0.

The improvement is rather small. For this “soft tophat” aperture, registration errors of order 2 pixels lead to a flux error of ~1.4%. This is to be compared to the error of 1.5% for the hard edged aperture. If we can achieve registration errors of 1 pixel, the soft edged aperture yields a flux error of 0.8%, compared to 1.0% for a hard edged aperture.

Next we tried a much more uniformly illuminated region of the moon:


The results are much better. In this region, a registration error of up to 2 pixels yields flux errors in the patch of order 0.2% (31 pixel hard edge aperture) and 0.15% (31 pixel soft edge aperture). This is very acceptable!

Conclusions:

Using a soft edged aperture helps ameliorate registration errors, but not as much as we had thought.

Selection of uniformly illuminated patches helps much more! Doing both is a good thing.



CCD alive after all?

Real World Problems Posted on Aug 13, 2012 15:06

The image below is taken at the IIWI computer with the CCD camera
attached to the board, sitting in a PCI slot on the IIWI. The software
used is SOLIS.


This image looks like things we have seen before. The broad stripes are reversed but that may be something in the display software settings. It is a very primitive setup – all I have managed to do is take a
picture at some long exposure time. The CCD is still attached to the
telescope, I think, so the shutters are closed and what we see is a flat+bias
frame. As it was dark in the dome when the image was taken the ‘signal’ is dark current – not light. The bands are structures in the flat
field. The spots are noise. The large spots are possibly CR hits?

I think the image proves that the camera is able to take pictures. The
noise may be due to a damaged cooler or – more probably – that the
cooler is not switched on (I don’t know how to do that) yet.

It is at least an image from the camera – so board and camera must be
OKish. Why then are there no images when the camera is attached to the PXI? One
answer could be that the PXI was damaged during the MLO power surge. I
am leaning towards that theory now.

Added later: Here is an image with cooler ON.

The minimum values is 402 – a bit high, but at least it now looks like a bias frame. I would say that there is nothing wrong with the camera or its board. So the problem must be in the PXI!



Power problems

Real World Problems Posted on Jul 10, 2012 10:50

July 10 2012: A power problem at MLO has been reported. At the moment the power is back, and all our machines – except the PXI can be reached.

If the PXI is damaged this could be the end of current efforts to use the telescope.

Hopefully access will be back soon and we can gather some more data, before the decisions we have to make in September.



Refining understanding of fluxes

Real World Problems Posted on Jul 09, 2012 15:23

Following on from the Data Summary post, we have reviewed all the good data and built an understanding of what the flux-ratios are between B and the other filters. We then use that ratio to test all observations to see if the FW was behaving as expected or showed signs of malfunction. The malfunctions often mean that all fluxes are the same, consistent with one single filter being set instead of the sequence of filters the script asks for.

Focusing then on only those nights where the FW was well behaved we extracted all total fluxes (total bias-corrected image counts divided by nominal exposure time), limited the data to airmasses less than 4, removed the lunar eclipse, and then plotted all fluxes against lunar phase:

(sorry about the strange cropping!) In different colors we see the extinction-corrected fluxes as a function of lunar phase. We only show data between 30 and 90 degrees (90=half Moon) since this is all we can acquire in co-add mode. The blue,green,red and orange symbols correspond to B,V,VE2, and VE1+IRCUT (very similar filters), respectively. The curves are 4th order polynomia fitted with a robust code that omits outliers.

We see some scatter around the lines – some of it (e.g. near 50-55 degrees is probably clouds. Some of it, in IRCUT/VE1 near 90 degrees may be ‘no filter was inserted at all’, or shutter exposure time was longer than requested.

From the fits we can generate flux-tables for use in identifying the remaining data: There are many more images available but they seem to be with unknown filters because the filter acquired and requested were simply not the same. These images may still be of use in DS/BS and albedo analysis – we just need to figure out what filter was used!

Apart from causing many lost images (i.e. we get a bias frame instead) the shutter semes to mostly work as expected when it opens at all. The Filter Wheel, however, selects random filters (as far as we presently know), or no filter at all, when it does not work.

The present work enables a data-selection filter for use in post-observational processing.

Note that the present data are not relevant for DS/BS studies as scattered light has not yet been removed. First we have to identify the images that can be further analysed!



Data summary, July 2012

Real World Problems Posted on Jul 07, 2012 19:24

This report summarises the data we have until July 2012. We have about 24 good nights of data …



Filter Wheel and/or Shutter annoy, again.

Mechanical design Posted on Jul 03, 2012 09:37


The above is the result of testing flux constancy in images taken of the Hohlraum lamp on a given night, through all filters. Since the lamp is constant and the telescope is not moved then, provided that the FW and shutter works, it should be possible to get constant fluxes for each filter. The above shows that this is hardly the case.

In the first five panels (left to right, top to bottom) we see histograms of the fluxes (cts/s) derived by opening images, subtracting the bias, and extracting the total flux as well as the exposure time and filter name from the headers. We see that in no filter is there a prevalent flux – that is, either the lamp altered its brightness or the FW never acquired the requested filter or the shutter was miles off (so that the requested exposure time was nowhere near the one we got). The bottom right plot shows the fluxes plotted in order of acquisition – since we see straight sequences I think the lamp is not the problem – but the shutter and or the FW is.

Near the top we see a special pattern – this is understood when inspecting the list of of filters and fluxes:
6712986.9 _VE2_
6714074.6 _VE2_
103523071.0 _VE2_
103529310.0 _IRCUT_
18201912.2 _IRCUT_
18202890.8 _IRCUT_
18206373.8 _IRCUT_
18204287.7 _IRCUT_
18206416.7 _IRCUT_
18205757.8 _IRCUT_
103526333.0 _IRCUT_
103539529.0 _V_
8245622.5 _V_
8245177.7 _V_
8244685.3 _V_
8247179.9 _V_
8246506.9 _V_
8247198.6 _V_
103533820.0 _V_
103570347.0 _B_
6290177.6 _B_
6288536.8 _B_

We see that when the filter supposedly changes, the flux is all strange given the otherwise regular sequence.

So: never use the last or first image in a sequence from a stack! Whether this has been the case throughout the 1.5 years of data we soon have, is to be revealed by further analysis. For now, let us simply reject all first and last images in all sequences.

Added later:

A quick look at the NGC6633 images shows the ABSENCE of the above problem:
8660536.8 _B_
8661084.9 _B_
8659963.1 _B_
8629954.5 _B_
8661203.9 _B_
8630249.1 _B_
34529656.0 _VE1_
34784961.7 _VE1_
34790632.0 _VE1_
34526064.7 _VE1_
20713556.2 _VE2_
20713411.2 _VE2_
20713019.0 _VE2_
51780586.5 _IRCUT_
51774002.0 _IRCUT_
52019051.0 _IRCUT_
51776315.5 _IRCUT_
52013538.0 _IRCUT_
51770409.5 _IRCUT_
52012276.5 _IRCUT_

The filters did NOT act strange when changed. So – more intermittency, but also a lesson: check all sequences of images for the above problem!



JD2456109 – weather log

Observing log Posted on Jul 01, 2012 12:06

Clear. Moon up.
7-9 m/s from 160 deg.
14% RH rising
7 deg C.



Instrumental zero points

Observation Resources Posted on Jul 01, 2012 03:47

Peter and Chris observed NGC6633 – an open cluster in the Galactic plane — on 26th June 2012. The aim was to calibrate the zero points and colour dependencies of all the filters.

Coordinates are RA = 18h 37m and DEC = +06 34 (J2000) — which goes more or less overhead. We observed very close to the zenith, so the airmass was very close to 1.0.

Image above shows the FOV for the IRCUT filter (which is very similar to V). About 80 well measured stars are identified in the cluster and marked with blue circles.

We got great images for all 5 filters (first time we got all 5!).

Johnson V and B data were obtained from WEBDA:

http://www.univie.ac.at/webda/cgi-bin/ocl_page.cgi?dirname=ngc6633

Exposure times were:
V 25 sec
B 34 sec
VE1 12 sec
VE2 32 sec
IRCUT 12 sec

Counts were measured in the ~ 70 identified stars in a 2.5 pixel aperture (i.e. 7*2.5 = 20 arcsec radius aperture).

Transformations to V, B, VE1, VE2 and IRCUT were derived from ~ 70 stars, where the instrumental magnitudes for each filter are of the form:

Vinst = -2.5*log10(Vcounts/exptime)
Binst = -2.5*log10(Bcounts/exptime) etc…

V = Vinst + 15.07 – 0.05*(B-V)
B = Binst + 14.75 + 0.21*(B-V)
VE1 = VE1inst + 16.30 + 0.18*(B-V)
VE2 = VE2inst + 13.88 + 1.09*(B-V)
IRCUT = IRCUTinst + 16.43 + 0.16*(B-V)

The scatter in the derived relations is
V 0.02 mag
B 0.05 mag
VE1 0.04 mag
VE2 0.06 mag
IRCUT 0.06 mag

The transformed data for the stars are shown in this plot:

The V and IRCUT filters both transform to Johnson V, and the B filter transforms to
Johnson B, with relatively small colour terms (i.e. the dependency on (B-V) of the transformation).

Followup:

Firstly we compare these relations to those derived at the last (only partially successful) attempt to calibrate the filters using M41.

The report on the M41 data is here:
http://earthshine.thejll.com/#post112

The transformations from the two clusters are:

V = 15.15 + Vinst – 0.08*(B-V) : NGC6633
V = 15.07 + Vinst – 0.05*(B-V) : M41

B = 14.46 + Binst + 0.26*(B-V) : NGC6633
B = 14.75 + Binst + 0.21*(B-V) : M41

The colour terms are quite similar, but the zeropoints differ substantially, particularly for B. Since we were unsure about the quality of the M41 data, I think these these should be disregarded. We’ll do NGC663 a few more times over the next few weeks — it’s a very well placed cluster, and the stars are quite sparse — which is very good for us.

We need a set of images as the cluster goes to much higher airmass, so we can measure the extinction coefficients for each filter. I think we have at least some of these data already from 26th June 2012.

TO DO: apply these to lunar images to measure the colour of the brightside and earthshine light.



Bad focus

Real World Problems Posted on Jun 30, 2012 08:37

Just realised that the images with bad focus are probably images where the right filter was not set. This should be checked by comparing flux, phase and focus (3 Fs ..). Could perhaps be used as a method to select only those images taken with known filter.



JD2456105

Observing log Posted on Jun 27, 2012 13:15

I do not understand this image:

The image is not overexposed (3000 counts only) so the streaks are not ‘blooming’, which occurs when the pixels are saturated (starts near 60000 counts). A streak downwards would be due to readout while shutter was open (down corresponds to the readout direction). But UP is very strange – it could happen if the telescope was moving while the Moon was being imaged, but as far as I know this is not a situation we have seen before. It could also happen if the readout direction changed – but I do not think that is possible – and in this case readout would have had to occur in both directions then.



JD2456102

Observing log Posted on Jun 24, 2012 13:52

Very clear. High wind.

Shutter failed pretty miserably – 1 image stack of 16 was useful.

Later restarted telescope and was able to calibrate the mount – this means that the shutter started working again.

Google Sky helps find the calibration position.



Met data

Met sensor Posted on Jun 21, 2012 21:42

In order to maintain a record of the observing conditions, I now make daily automatic downloads of the VYSOS graphic of the weather at MLO:
http://72.235.176.178/Weather_VYSOS5.gif
With these images stored it should be possible to later go back and see whether it was a clear night, etc.



JD2456095 – weather log

Observing log Posted on Jun 16, 2012 15:59

Hazy – VYSOS line is yellow/white. Moon rises in clouds.

8 m/s from 180 deg
RH is 15%



Looking at raw data

Real World Problems Posted on Jun 15, 2012 11:22

If we take all our data and plot the flux for each image (total counts divided by nominal exposure time) against the lunar phase (0 phase is Full Moon) at the time of observation we get plots like this:

This is for the B filter, and the top panel shows the raw flux. We see the outline of the expected ‘phase law’ with some scatter. In the second panel we see the data corrected for extinction (the data are taken at different airmasses and must be extinction-corrected). The scatter is not really reduced, so we conclude that the scatter is mainly due to something else. In the third panel we have more or less taken out the phase law (determining the phase law is a matter of research – BBSO does it with empirical methods, we are going to use the latest Hapke et al phase laws, but at the moment merely correct for the ‘geometric illumination fraction’).

The factors that cause the scatter that remains could be

a) clouds – i.e. flux is reduced, by the passage of a thin cloud, compared to data at the same phase but without clouds.

b) The shutter failed so that we do not have access to the actual exposure time – merely the one we asked for.

c) The color filter that was used was simply not the one we asked for.

d) The Moon was not centred in the frame and part of the flux is missing because the Moon is outside the edges of the picture. Lunar eclipses also enter as a problem but happen, of course, only at full moon – we have examples of this and know how to eliminate these frames (and full moon data will not be used for DS/BS work anyway).

I think that data suffering from problem a could be eliminated by some sort of image analysis that looks for an uneven sky – but it will be a tough job to separate this effect from c.

Problem b will not affect the DS/BS ratio. Remember that the above is mainly the BS we are looking at! When the shutter fails it often causes ‘dragged images’ and they can quite easily be eliminated.

Problem c is worse – we know from the problems we encountered, in trying to find the focus for each filter, that the FW does not always select the filter we ask for. Due to the lack of ‘polling’ in the system design we are victims of ‘timeout’: proceedings may halt while the system waits for a filter to be selected, but after a preset time the system simply moves on and does the next command, such as an exposure. How can we detect and eliminate the exposures that were taken through the wrong filter? (a future design should have a method to query the system about the identity of the filter in the beam: some sort of coding system?).

In the plot above there are clearly two sequences of flux values for phases between +90 and +150 with a flux-ratio of about 3.5. Is it possible to understand if this is due to selection of a different filter (or no filter at all)?

From experience we know that the flux-ratio between the B filter and no filter at all is likely to be near 10, while the ratio between B and V is like 2 or 3. So – it may not be easy to assign a ‘real filter’ to each observation on the basis of flux (and phase-information) alone.

Since DS/BS is unaffected by shutter problems but is affected by ‘wrong filter’ issues we should be able to design a data-examination method. More to follow on this.

Seperately, a delicate analysis method for the presence of drifting clouds in the image is needed in order to eliminate problem a. Suggestions are welcome.

Problem d can be handled with image analysis, and this will be implemented as part of a data-validation filter.



JD2456093 – weather log

Observing log Posted on Jun 14, 2012 16:11

clear, 40F, 15-20% RH rising, 6-8mph or 2-4 m/s from 220 degr.

Many ‘hung shutter’ frames but the ones that work are perfect.



JD2456092 – weather log

Observing log Posted on Jun 13, 2012 15:25

clear, 35 F, 30% RH, 2 mph winds.



JD2456090 – weather log

Observing log Posted on Jun 12, 2012 14:18

hazy
40 F
RH is 30% and rising
4-6 mph winds.



JD2456090 – weather log

Observing log Posted on Jun 11, 2012 15:14

9 C/50 F, 8mph/3-4 m/s, 15% RH, clear but not v. so.



Pressure sensor

Met sensor Posted on Jun 07, 2012 17:53

Arduino code for the BMP085 pressure sensor – it also reads temperature – output is T (in deegrees C) and p (in HPa). Note that this is for Arduino 1.0 only.

//Arduino 1.0+ Only
//Arduino 1.0+ Only

/*Inspired by http://bildr.org/2011/06/bmp085-arduino/

Get pressure and temperature from the BMP085.
Serial.print it out at 9600 baud to serial monitor.
*/

#include <Wire.h>

#define BMP085_ADDRESS 0x77 // I2C address of BMP085

const unsigned char OSS = 0; // Oversampling Setting

// Calibration values
int ac1;
int ac2;
int ac3;
unsigned int ac4;
unsigned int ac5;
unsigned int ac6;
int b1;
int b2;
int mb;
int mc;
int md;

// b5 is calculated in bmp085GetTemperature(…), this variable is also used in bmp085GetPressure(…)
// so …Temperature(…) must be called before …Pressure(…).
long b5;

void setup(){
Serial.begin(9600);
Wire.begin();

bmp085Calibration();
}

void loop()
{
float temperature = bmp085GetTemperature(bmp085ReadUT()); //MUST be called first
float pressure = bmp085GetPressure(bmp085ReadUP());
float atm = pressure / 101325; // “standard atmosphere”
float altitude = calcAltitude(pressure); //Uncompensated caculation – in Meters

Serial.print(temperature,2);
Serial.print(” “);
Serial.print(pressure,2);

Serial.println();//line break

delay(1000); //wait a second and get values again.
}

// Stores all of the bmp085’s calibration values into global variables
// Calibration values are required to calculate temp and pressure
// This function should be called at the beginning of the program
void bmp085Calibration()
{
ac1 = bmp085ReadInt(0xAA);
ac2 = bmp085ReadInt(0xAC);
ac3 = bmp085ReadInt(0xAE);
ac4 = bmp085ReadInt(0xB0);
ac5 = bmp085ReadInt(0xB2);
ac6 = bmp085ReadInt(0xB4);
b1 = bmp085ReadInt(0xB6);
b2 = bmp085ReadInt(0xB8);
mb = bmp085ReadInt(0xBA);
mc = bmp085ReadInt(0xBC);
md = bmp085ReadInt(0xBE);
}

// Calculate temperature in deg C
float bmp085GetTemperature(unsigned int ut){
long x1, x2;

x1 = (((long)ut – (long)ac6)*(long)ac5) >> 15;
x2 = ((long)mc << 11)/(x1 + md);
b5 = x1 + x2;

float temp = ((b5 + 8)>>4);
temp = temp /10;

return temp;
}

// Calculate pressure given up
// calibration values must be known
// b5 is also required so bmp085GetTemperature(…) must be called first.
// Value returned will be pressure in units of Pa.
long bmp085GetPressure(unsigned long up){
long x1, x2, x3, b3, b6, p;
unsigned long b4, b7;

b6 = b5 – 4000;
// Calculate B3
x1 = (b2 * (b6 * b6)>>12)>>11;
x2 = (ac2 * b6)>>11;
x3 = x1 + x2;
b3 = (((((long)ac1)*4 + x3)<<OSS) + 2)>>2;

// Calculate B4
x1 = (ac3 * b6)>>13;
x2 = (b1 * ((b6 * b6)>>12))>>16;
x3 = ((x1 + x2) + 2)>>2;
b4 = (ac4 * (unsigned long)(x3 + 32768))>>15;

b7 = ((unsigned long)(up – b3) * (50000>>OSS));
if (b7 < 0x80000000)
p = (b7<<1)/b4;
else
p = (b7/b4)<<1;

x1 = (p>>8) * (p>>8);
x1 = (x1 * 3038)>>16;
x2 = (-7357 * p)>>16;
p += (x1 + x2 + 3791)>>4;

long temp = p;
return temp;
}

// Read 1 byte from the BMP085 at ‘address’
char bmp085Read(unsigned char address)
{
unsigned char data;

Wire.beginTransmission(BMP085_ADDRESS);
Wire.write(address);
Wire.endTransmission();

Wire.requestFrom(BMP085_ADDRESS, 1);
while(!Wire.available())
;

return Wire.read();
}

// Read 2 bytes from the BMP085
// First byte will be from ‘address’
// Second byte will be from ‘address’+1
int bmp085ReadInt(unsigned char address)
{
unsigned char msb, lsb;

Wire.beginTransmission(BMP085_ADDRESS);
Wire.write(address);
Wire.endTransmission();

Wire.requestFrom(BMP085_ADDRESS, 2);
while(Wire.available()<2)
;
msb = Wire.read();
lsb = Wire.read();

return (int) msb<<8 | lsb;
}

// Read the uncompensated temperature value
unsigned int bmp085ReadUT(){
unsigned int ut;

// Write 0x2E into Register 0xF4
// This requests a temperature reading
Wire.beginTransmission(BMP085_ADDRESS);
Wire.write(0xF4);
Wire.write(0x2E);
Wire.endTransmission();

// Wait at least 4.5ms
delay(5);

// Read two bytes from registers 0xF6 and 0xF7
ut = bmp085ReadInt(0xF6);
return ut;
}

// Read the uncompensated pressure value
unsigned long bmp085ReadUP(){

unsigned char msb, lsb, xlsb;
unsigned long up = 0;

// Write 0x34+(OSS<<6) into register 0xF4
// Request a pressure reading w/ oversampling setting
Wire.beginTransmission(BMP085_ADDRESS);
Wire.write(0xF4);
Wire.write(0x34 + (OSS<<6));
Wire.endTransmission();

// Wait for conversion, delay time dependent on OSS
delay(2 + (3<<OSS));

// Read register 0xF6 (MSB), 0xF7 (LSB), and 0xF8 (XLSB)
msb = bmp085Read(0xF6);
lsb = bmp085Read(0xF7);
xlsb = bmp085Read(0xF8);

up = (((unsigned long) msb << 16) | ((unsigned long) lsb << 8) | (unsigned long) xlsb) >> (8-OSS);

return up;
}

void writeRegister(int deviceAddress, byte address, byte val) {
Wire.beginTransmission(deviceAddress); // start transmission to device
Wire.write(address); // send register address
Wire.write(val); // send value to write
Wire.endTransmission(); // end transmission
}

int readRegister(int deviceAddress, byte address){

int v;
Wire.beginTransmission(deviceAddress);
Wire.write(address); // register to read
Wire.endTransmission();

Wire.requestFrom(deviceAddress, 1); // read a byte

while(!Wire.available()) {
// waiting
}

v = Wire.read();
return v;
}

float calcAltitude(float pressure){

float A = pressure/101325;
float B = 1/5.25588;
float C = pow(A,B);
C = 1 – C;
C = C /0.0000225577;

return C;
}



Cloud Sensor

Mechanical design Posted on Jun 02, 2012 18:57

We need a cloud sensor at MLO. It can be used for at least two things – it can gather a record of sky conditions for us that we later can use in selecting the best data – and, in our wild fantasies, it could be integrated with a control system so that it warned of clouds and shut the dome.

I have started some experiments on simple sensors and data-loggers to see what can be done. This is a link to an image of the output from a sensor running in Denmark at the moment.

The system consists of a thermometer-chip (LM335a) and an IR thermometer sensor (LMX90614) both sitting on an Arduino and being read once a minute. The signal is processed further by off-line software and a the plot is generated in IDL.

The two sensors are complementary – the thermometer chip reads the ambient air temperature (actually, it measures the temperature of the chip body) and the IR sensor measures the temperature of the sky above the sensor in a 45 degree cone.

In general, the air radiates with a temperature that is the ‘air temperature’ but the presence of clouds increases the downward longwave radiation field, and the absence of clouds decreases the downward longwave forcing – this causes the readouts from the two types of thermometers to form a cloud detector via a normalized signal based on the ratio of the two. The day-night cycle can be seen in both signals and is removed by taking the ratio, while the passage of clouds is seen as wiggles.

At the moment the system is just a prototype based on a loosely wired breadboard design, but if it works it can be hardwired as a dedicated PC board and permanently interfaced to an Arduino and packaged as a weather-resistant sensor. It relies for protection now on just some cling film (IR-transparent) and I need to experiment with glass and harder plastics of various sorts. Then I will move it to DMI and operate it from the roof there, and eventually we could move it to Mauna Loa.

I see it at first as being read from a USB port on the Linux machine there, offering up the present type of plots and generating a database for archival use, and only later would we try to incorporate it into a more automatic control system, as envisaged originally.

LM335A is the left chip – flat side is facing viewer, pins are 1,2,3 from left.
LMX89614 is the right chip – pin numbers are (1 – lower left, 2 – lower right, 3 – upper right, 4 – upper left (IC is actually round). The capacitor is 0.1 muF. The lower left resistor is 1k the other two are 4.7 k.



Good Questions

Error budget Posted on Jun 01, 2012 08:20

At Henriette’s very successful Master’s thesis defense yesterday a few good questions were asked by the opponents and the audience. Some of them we were prepared for, but a few were novel:

1) A question about the quality of FFs was a little mixed up, but the essence was ‘are they good because they are almost overexposed’? Now that we know about the CCD nonlinearity which starts even at 15000 counts and is not obviously in the ‘blooming phase’ until you pass 55000 we might consider the impact of non-linearity on various quality estimates again. However, the high quality of the hohlraum FFs is due to the hohlraum being a good uniform source.

2) If we had a way to rotate the CCD we could indeed investigate field gradients. This need only be done once. As far as I know the CCD stops its rotation because it is now held by alignment of two steel pins. Let us ask Ben to comment on whether a 180 degree turn of the camera is possible in the present setup.

3) After which interval does lunar libration and sunrise on the Moon cause so many changes in the illumination of the lunar surface that stacking images is pointless? This can be investigated with our forward model system. A long time ago I tested that and found that half an hour was the limit, but this should be redone and focused on areas of the Moon – there are obvious changes near the terminator after juts a few minutes but what is the effect near the disc edge, and so on. It takes some 3-4 minutes to get two bias frames and a stack of 100 images.

Added later: A quick investigation using synthetic images of the quarter Moon suggest that the maximum change in a single pixels’ intensity occurs on the terminator and is roughly 1% per minute. Away from the terminator – in typical DS and BS pixel positions – the change in DS/BS ratio is like 0.03% per minute. This is thus an issue mainly for BBSO, who do not expose DS and BS simultaneously.

4) What would be the error due to alignment problems alone? We can investigate that with the forward model.

5) There is no WCS in the file headers. Could alignment be aided with such a system? WCS is the World Coordinate System – this is what we can find by using astrometry.net – the pointing of the telescope is found with sub arcsecond precision and written into the FITS header. We may be able to do that if there are stars in the frame – since we are using coAdd mode at the moment it seems impossible to get stars in the frame except occasionally (e.g. tau Tauri).

6) In the precision error budget independence is assumed – how much de-pendence is OK for the budget still to be valid? This is a good question about statistics fundamentals! We sought to separate the dependent and independent errors in two budgets, but in general the question can be handled with formal methods or simulations where synthetic examples are investigated.

7) Scale the bias frame better by following the cooler periodicity. This was countered by a warning: Beware that the periodicity seen may depend on camera load.
The counter was good because the nice periodicity is obtained when taking long sequences of bias-only images, right? Under observing conditions the heat load on the camera due to taking science frames may be different – and may vary depending on exposure time and number of frames taken. We can investigate this by studying bias frames from regular sequences of image-taking: are there signs of a periodic bias level in those images? We know the period and we know the time of exposure so it should be possible to perform a signal analysis. A worst-case, ‘upper limit’ analysis could be performed by simply studying how much the mean value of scaled superbiases varies from regular observing sequences: If the variability of the mean level is more than that seen in the bias-only sequences then we know that we have additional heat load on the cooler during science frame taking.



Data status May 2012

Observing log Posted on May 30, 2012 14:05

As of end of May 2012 we have about 25000 images labeled “MOON” – many of these contain 100 images each.

From these, after coadding all image-stacks, and by selecting only good data, we get 6871 single images (these are now either single observations or aligned 100-image bundles). These cover 55 separate nights from July 2011 to now.

Most of these 55 nights are observed in all filters.

None are science-grade SKE or ND images.

With an expected 200 good nights and a duty cycle of 20-25% we are not doing too badly!



JD2456076

Observing log Posted on May 29, 2012 08:41


Shutter all over the place today – dragging AND overexposure??

9-11 m/s from 120 deg. RH near 20%, 6 deg C. clarity 9/10.



JD2456075 – weather log

Observing log Posted on May 28, 2012 08:11

Hazy or foggy – quite large halo on Moon in TV cam. Difference image showing structures.

15 m/s from 160 deg, steady. 6 degrees C. RH is 30%.



JD2456074 – weather log

Observing log Posted on May 27, 2012 07:27

12-18 m/s from 140 deg. humid – RH is 70% and rising. T is 5 deg C. It is almost clear – but minutes ago small puffy clouds blew over …

Ups — more clouds. Closing down for a while now.

Observing again – must say there is some fog banks that come and go. Not one of the best nights.



JD2456073

Observing log Posted on May 26, 2012 09:45

Clear. 12 m/s from 150 deg. 48 deg F. 10% RH.

At 2456073.8ish the cables are in the frame.

Good data – butthe usual filter dragging and blanks. Especially V which is unusual.



JD2456072 – weather log

Error budget Posted on May 25, 2012 08:10

clear, RH just 5%. 15mph. 50 F.



JDS2456071 – weather log

Observing log Posted on May 25, 2012 08:09

18 m/s from 160 deg. RH 20%. 5 deg C. Clear.



Focus (4)

Real World Problems Posted on May 24, 2012 13:21

The FW problem is not resolved – but the usual work-around worked also this time: reboot everything. Power cycle. Then the 55536 error code went away, and the script to set focus and take images and save data worked as intended. This time I believe the results because the collected fluxes at the same exposure time through different filters depended on the filter in a sensible way:

Filter median flux
—————————
B 21048.9
V 215239.9
VE1 336068.9
VE2 46270.8
IRCUT 328818.2

The focus is still hard to determine from data like this, though:

It seems to me that the radius measurement may be useful, and the SD method. Not all filters can have their focus determined from the above – wider limits must be set, and I will do that to get a clearer ‘peak’ answer.

The above is the result of a single run through all filters so the occurrence of ‘double tracks’ in some places is really mystifying.

It remains unclear why the FW did not move, but it is probably related to that 55536 signal. The other error code was a reference to the dome not finished with a home seek. Not sure if that influences the ability of the FW to move??

None of this explains the previous problem we have had with VE2 alone – in a sequence of good B,V, VE1 and IRCUT VE2 was suddenly out of focus, and then back in. Hmm.



Focus (3)

Mechanical design Posted on May 23, 2012 17:28

After completing the analysis of the best focus for all filters I am completely confused. It looks like all filters have the same focus near 24000:
Of the above only VE1 is a little different. ‘double rows’ of points imply there were two sets of images exposed at different times.

Note that if the minimum radius is near the same value, and the scale factor is the same then the ‘volume’ of the star image is the same – but the exposure times were the same .. usually there is a factor of 20 less photons in the B filter compared to V … so the above test has not selected different filters. And yet the command is there in the script:

…..
SETFILTERCOLORDENSITY,VE2,AIR,,,,,,,
SETFOCUSPOSITION,FOCUS097,,,,,,,,
SHOOTKINETIC,10,5,512,512,VE2_FOCUS097,,,,
DOMEAZ,,,,,,,,,,,,,,,,,,,
SETFILTERCOLORDENSITY,IRCUT,AIR,,,,,,,
SETFOCUSPOSITION,FOCUS080,,,,,,,,
SHOOTKINETIC,10,5,512,512,IRCUT_FOCUS080,,,,
DOMEAZ,,,,,,,,,,,,,,,,,,,
………

The entire script is here:

I started noticing this ‘same exposure time in all filters’ problem some weeks ago – but it was intermittent.

I am going to ignore the new settings for focus for now and see if I get reasonable exposures next time I use the moon script.

What on EARTH is going on????



Focus (2)

Mechanical design Posted on May 23, 2012 13:34

Extending the lower limit on the search for a good focus we now have a preliminary result for the B filter:

The plots show various focus-related quantities against focus position. The first two panels shown (on lin-log and lin-lin scales) the radius of the fitted Moffat profile. The next shows the standard deviation of the image, divided by the mean value of the image, the fourth box shows the exponent of the fitted Moffat profile, and the last box shows the vertical scale (height, really) of the fitted Moffat profile.

The various properties – which should all be maximum (or minimum) at the best focus – indicate values of best focus from 24500 to 26000, with 25500 being perhaps the least uncertain.

The focus we use at the moment for the B filter is 28250 – outside the range found here. That value corresponds to almost a doubling of the width of the Moffat profile.

We do not understand how the setting can be that much off – possibilities include that something happened during transportation or that the addition of the heater has changed the focus position.

Perhaps Ahmad has an original INI file from the days in Lund and could check what value was found best a year ago?

Perhaps Torben could see Torbjörn, or whoever understands the mechanical design in detail, and ask whether the heater could be warping the optical bench? Temperatures to 60 degrees C is seen on the clip-on thermometer in the dome.

The good news is that if better focus positions can be found for all filters in the above way then we should be seeing less fuzzy images in the future – it seems we have a FWHM PSF now that is about 2 pixels.



JD2456070 – weather log

Observing log Posted on May 23, 2012 10:57

Clear. 15 m/s from 140 deg. RH is 20%. 4 deg C.



Focus

Mechanical design Posted on May 23, 2012 08:29

For some time there have been intermittent focus problems with the system – especially the VE2 filter would go in and out of focus for no discernable reason. The focus is supposedly set, in scripted mode, by referring to focus positions from a script. The currently used focus positions are:

[FOCUSPOSITIONS]
; The names made of (Color filter) + (ND filter) + (Knife Edge DF)

IRCUT_AIR_SKE = “28750”
IRCUT_ND_SKE = “29000”
B_AIR_SKE = “28250”
B_ND_SKE = “29000”
V_AIR_SKE = “30750”
V_ND_SKE = “29000”
VE1_AIR_SKE = “28750”
VE1_ND_SKE = “29000”
VE2_AIR_SKE = “24000”
VE2_ND_SKE = “29000”
AIR_AIR_SKE = “29000”
AIR_ND_SKE = “29000”

As the focus seemed more and more unreliable I defined a new set of positions in order to be able to test on stars from 28000 to 31000:

FOCUS090 = “28000”
FOCUS091 = “28100”
FOCUS092 = “28200”
FOCUS093 = “28300”
FOCUS094 = “28400”
FOCUS095 = “28500”
FOCUS096 = “28600”
FOCUS097 = “28700”
FOCUS098 = “28800”
FOCUS099 = “28900”
FOCUS100 = “29000”
FOCUS101 = “29100”
FOCUS102 = “29200”
FOCUS103 = “29300”
FOCUS104 = “29400”
FOCUS105 = “29500”
FOCUS106 = “29600”
FOCUS107 = “29700”
FOCUS108 = “29800”
FOCUS109 = “29900”
FOCUS110 = “30000”
FOCUS111 = “30100”
FOCUS112 = “30200”
FOCUS113 = “30300”
FOCUS114 = “30400”
FOCUS115 = “30500”
FOCUS116 = “30600”
FOCUS117 = “30700”
FOCUS118 = “30800”
FOCUS119 = “30900”
FOCUS120 = “31000”

I then ran, in steps of 100, over all these positions in all filters for 5-15 images in each position. I fitted a Moffat profile to the brightest source in each image and plot here the radius (sqrt(rx² + ry²)) of the profile:


[Sorry for the poor quality of that plot – a downloadable pdf of that plot is here:

The width of the distribution of points is due to repeated trials in some cases. Scattered points far from the general relationship are shutter failures.].

The result is, apparently, that all minimum-radius occurs at the leftmost side of the plot – i.e. at or below 28000 – this is not at all close to what we use from the scripts. What is going on?

Is the fitting of a profile a bad way to estimate best focus?

Is the setting of the focus from the System INI file misunderstood somehow?

The start of the script used to run these tests is here:

SETDATASOURCE,PROTOCOL,,,,,,,,
CCDCOOLERON,,,,,,,,,
SETIMAGEREF,0,0,,,,,,,,,,,,,,,,,,,
STARTCYCLE1,10000,,,,,,,,
WAITMINUTES,0.01,,,,,,,,
ENDCYCLE1,,,,,,,,,
PROTOSEGMENT1,,,,,,,,,
SETDATASOURCE,PROTOCOL,,,,,,,,
CCDCONFIG,CCDCONFIGSETUP-DEFAULT,,,,,,,,
CCDINIT,,,,,,,,,
OPENEXTSHTR,,,,,,,,,
IRISOPENCLOSE,OPEN,,,,,,,,
TRACKSIDEREAL,,,,,,,,,
SETMOUNTRADEC,15,45,00,02,18,57,,,,,,,,,,
MOVETOCOORDS,,,,,,,,,,,,,,,,,,,
STARTCYCLE1,1,,,,,,,,
DOMEAZ,,,,,,,,,,,,,,,,,,,
SETFILTERCOLORDENSITY,VE1,AIR,,,,,,,
SETFOCUSPOSITION,FOCUS090,,,,,,,,
SHOOTDARKFRAME,0.0762,1,512,512,DARK,,,,
SHOOTKINETIC,10,5,512,512,VE1_FOCUS090,,,,
SHOOTDARKFRAME,0.0762,1,512,512,DARK,,,,
DOMEAZ,,,,,,,,,,,,,,,,,,,
SETFILTERCOLORDENSITY,VE1,AIR,,,,,,,
SETFOCUSPOSITION,FOCUS091,,,,,,,,
SHOOTDARKFRAME,0.0762,1,512,512,DARK,,,,
SHOOTKINETIC,10,5,512,512,VE1_FOCUS091,,,,
SHOOTDARKFRAME,0.0762,1,512,512,DARK,,,,

etc etc and then next filter.

Anyone? Help?



JD2456069 – weather log

Observing log Posted on May 22, 2012 08:23

Windy at 17 m/s from 160 deg. Clear. RH 50%. T near 4 C.

Focus problems – again!



CCD linearity (2)

Andor camera field experiences Posted on May 18, 2012 11:42

We have seen that the relationship between signal variance and signal is not linear. This may be worrysome for us, but it is not really the non-linearity we originally feared. We would be in great trouble if signal level was not a linear function of incident intensity. In the body of data generated to show the variance-vs-level nonlinearity we also have the material to test whether level rises proportionally with exposure time. Of course, we do not really trust the shutter – it sticks and is variable at times – and we have no way to vary intensity of our lamps, but we start by inspecting the signal vs exposure time relationship in the V filter:From top left to bottom: Box 1: same as shown before – variance vs signal strength. Box 2: residuals after linear and 2nd order (red) fits have been subtracted. Box 3: signal level against nominal exposure time with 2nd order polynomial fit shown in red. Box 4: residuals from Box 3 when linear and 2nd order fits are subtracted. Box 4: same as shown before – nominal uncertainty on the parabola fit seen in Box 1.Inspecting the above plot in Box 3 we see that the signal level rises remarkably linearly as a function of nominal exposure time. Now, it could be that the exposure time is all wrong (we only have the time we ask for, not the time we actually get) but it seems unlikely that the linear relationship should arise by chance. The 2nd order coefficient shown in Box 3 is very small compared to the one in Box 1 – the signal-exposure time relationship is more linear than is the variance-signal strength relationship. It thus seems that electrons are not lost in whatever process causes the flattening out of the variance at high signal levels and it has been suggested in the literature that electrons migrate to neighbouring less-filled pixels. Thereby total signal is conserved but bright areas tend to blur into the darker neighbouring areas.If this is what happens we also have to think about the consequences of this – we must think of using DS/BS analysis methods that avoid using small bright regions for reference. This was at one point the core in the EFM method – the model of the observed image was constrained to pass through a region on the BS selected by the user – and often we chose the brightest parts near crater Tycho. This is now no longer used in the EFM – we now use a flux-conservation constraint to scale the model image. This would seem to be rather safe in the current situation.Eventually the DS/BS ratios need to be converted to albedo and this step is influenced by the choice of reference areas – something BBSO also must worry about in their reductions, but have not discussed.



CCD linearity

Andor camera field experiences Posted on May 16, 2012 14:07

Ahmad has tested the linearity of the CCD in Lund, and Henriette presents work on tests based on data taken at MLO. Since even small deviations from linearity in the CCD may be important for us we are revisiting the issue, and present here some results based on the variance-vs-level method which is nicely explained in Ahmad’s report on the Andor Camera.In the upper panel is plotted the variance against the level for all filters. Plots for the individual filters are available but the scatter is not much less. The solid line is the fitted least squares linear regression. As red is plotted the best-fitting parabola, and the dashed line is the diagonal. Beside the panel are written the coefficients of the best fitting 2nd-order function. In the middle panel is the difference between the data and the linear fit (black symbols), and between data and 2nd-order fit (red symbols).Lower panel shows the magnitude of the formally estimated uncertainty of the fitted parabola, in percent of the signal level. Dashed line shows the fiducial 0.1% error level we strive for. Variance and level was calculated from a central area of 190×190 pixels in the pairs of images. Images were bias subtracted using the ‘superbias’ we have, without scaling to surrounding dark field levels.The nonlinearity is clearly quite large since at 25000 counts a 0.1% difference is just 25 counts – small compared to the differences seen above. At 50.000 the observed difference of 1.500 corresponds to 3%. At count levels above 55.000 the relationship rapidly breaks down and saturation occurs. For observations below 55.000 we can probably correct all observed images we already have. In the future it may we best to observe only up to 52.000.Acceptable linearity may only be available up to count levels of 10- or 15.000. We have to correct our observations – but at least it can be done at any time in the future. We should now and then gather and archive a huge set of lamp images as were used above.



JD2456063/4

Observing log Posted on May 16, 2012 09:41

Trying to determine if CCD is linear by observing pairs of stars at various exposure times. Finding that focus is bad !?Is it even possible to get ‘doughnut’ images for out-of-focus stars in a refractor?RH is 50%, T is about 44 F, wind is 6-8 mph- It seems clear.Later, osberving Moon: ..54.06.. T=37ish F, RH is down to 42%, wind 2-3 mph. seemed clear when the Moon rose.



EFM software rewritten to Fortran.

Post-Obs scattered-light rem. Posted on May 15, 2012 10:33

Update:Thanks to help from Rong Fu it now seems we have a working Fortran 90 code for the EFM Method! The code is being tested against synthetic test cases and a performance report will be written. The code will be set up to run on the DMI Cray and with it we will be able to process much of the observations we have accumulated, and improve the paper significantly.The point of the new code is that it is no longer written in C (which is a bit of a mystery to us old-timers) and can now more easily be experimented with and optimized.



JD2456062/3 – weather log

Observing log Posted on May 15, 2012 10:31

at ..62.8 clear, 3 deg C, 2 m/s from 250 deg. RH 40-55%.at ..63.11 T lower, still clear, low wind, RH 60%This is a really strange night – the relative exposure times are very unusual.The factors are:B 20V 22VE1 20VE2 18IRCUT 20Especially B is very strange indeed.A lot of shutter hang going on, though.



JD2456061/2 – weather log

Observing log Posted on May 14, 2012 12:49

3 m/s from 220 deg, 4 deg C, RH 35% and rising.
Zenith clear but Moon rose behind clouds.

When observing resumed there was a slighthaze – VYSOS cloud line sloooowly moving upwards as dawn approached – but at least no obscuring clouds, as was the case at Moonrise!



JD2456060 – weather log

Observing log Posted on May 13, 2012 10:34

Clear
T about 4 C
RH down to 15% from a high hours earlier.
3 m/s winds



Drift alignment

Mechanical design Posted on May 02, 2012 15:01

Here are some data relevant for performing a better alignment of the mount. By taking pairs of images with 10 minutes between in the East, South and West, while tracking at sidereal rate, it is possible to see which way the centre-of-field drifts and from that deduce where the mount polar axis is pointing and how it should be corrected to point more closely at North. Center-of-field is determined from astrometric plate solutions using astrometry.net.

The center-of-field coordinates have been measured on JD2456049. Field 1 comes before field 2 in each pair. RA and DEC are:

E1: 18:20:11.665, +03:17:26.152
E2: 18:20:08.522, +03:17:49.439 which is 23 arc seconds more Northernly

S1: 13:54:10.407, -19:06:52.553
S2: 13:54:09.741, -19:06:43.784 which is 9 arc seconds more Northernly

W1: 09:13:41.508, -10:49:04.877
W2: 09:13:39.970, -10:49:08.317 which is 4 arc seconds more Southernly

Analysis:

When tracking due East centre-of-field drifted North
When tracking due West centre-of-field drifted South
When tracking near the meridian centre-of-field drifted North

Now, what does this imply? I am thinking it means that our Polar axis points East of true North, and is not elevated enough.

————-

later:

In order to calculate the angles that the mount needs to be turned by we need a method to analyze such positional data as we have above. I am testing various least-squares approaches in order to determine the 3D rotation matrix. After a lot of work it turns out that accurate fits are difficult to get when the number of positions is small. I generated artificial examples with known rotations and 60 pairs of points and ended up with solutions that were only within +/-1 degree of the known answer.

For the three pairs of points found above, the solutions were very small numbers, easily 0 within the +/- 1, or more, error limit expected.

At least this is consistent with a situation where the polar axis is quite well aligned already, and that a better alignment will require delicate work with the adjustment screws indeed!



JD2456046 – weather log

Observing log Posted on Apr 29, 2012 08:04

12.8 degrees C in dome at start – 45 F outside on MLO mast
9 m/s increasing at start. At xxx.86 the wind is 16 ms or 20 mph. Steady from South.
RH 15%
clear

Exposure factors for the filters are very good this night:

B 34.0
V 11.8
VE1 2.6
VE2 10.0
iIRCUT 2.6

These are factors, note xposure times. The exposure time is some number times the above. A typical IRCUT exposure time may be 0.008 s. The code we have – script_builder_MOON.pro – scales a table like the above one appropriately for the lunar phase.

Later: The above factors seem inadequate on another night with the Moon rising – B is overexposed while VE1, VE2 and IRCUT are at 12000. Hmmm.

A bit later still: Hmmm – the above problem was due to hanging shutter – but a very consistently hanging shutter! So the table is OK. B is perhaps a bit high but now V and VE2 are just right.



JD2456045 – weather log

Observing log Posted on Apr 28, 2012 08:24

12.9 C in dome at start (6045.75..)
RH 25%
0 wind
clear
Moon setting

Near ..45.86.. one of the cables from the MLO tower gets in the image.



ES as fraction of Moonshine

Relevant papers Posted on Apr 19, 2012 11:46

The Earthshine is usually a small fraction of the total light we receive from the Moon. But near new Moon the BS is very small and the DS starts to dominate – but at which phase?
Using Hans’ synthetic lunar-image code (based on Hapke 63 reflectances and thus UNDERestimating DS intensity and OVERestimating BS intensity for phases close to New) we get:

It seems the ES contributes 10% or more of the Moonlight at phases closer than 10 degrees to New Moon, and more than 1% at 30 degrees from new Moon.

Now, the SOLSTICE instrument in space has produced data for total lunar irradiance at angles all the way up to 180 (or is it just 170?) degrees from Full, so potentially they have data that contains 1% and more earthshine. The LRO data mentioned in Buratti et al 2010 does not cover to near New Moon. BBSO only gets out to 140 degrees or so (i.e. 40 from NM).

Due to the simplicity of the Hapke63 reflectance there is potential for a brighter ES and fainter BS near NM so that a larger fraction of the Moonshine is due to reflected earthshine. (the above is thus a lower limit on the fraction of Moonshine that is ES, in other words).

Is there potential for using LLAMAS data from SOLSTICE to do some clever work on Earthshine intensity?



Wishes

To-do list Posted on Apr 18, 2012 10:31

A wish-list for this system should include

‘proper polar
axis alignment’ and/or
‘automatic centering of
telescope on flattest part of hohlraum field’



4 FFs using the ND filters

Bias and Flat fields Posted on Apr 18, 2012 10:27

We inspect four flat fields taken of the Hohlraum lamp using AIR, ND1.0, ND1.3 and ND2.0 (ND0.9 was skipped as it is apparently AIR). All images were exposed so that count levels were between 10000 and 56000.

At upper left is AIR, upper right is ND1.0, then ND1.3 lower left and ND2.0 lower right.

Looking at these images there is evidently some light-leakage problem at the upper right corner, that has to do with which ND filter is used – or the lamp shifted slightly between exposures. Other experiences with the Hohlraum lamp underline how exactly positioned the telescope must be – this is difficult since small mount alignment problems apparently accumulate over time (the ‘creep problem’). The mount should therefore be freshly calibrated before use of the Hohlraum, which is not always convenient. It must furthermore be calibrated on a part of the sky near the lamp or the ‘offset problem’ will be evident.

‘offset’ and ‘creep’ are relatives, I think – something to do with the axes of the mount and the polar alignment.

A wishlist for this system might include an item about ‘proper polar axis alignment’ or perhaps an item about ‘automatic centering of telescope on flattest part of hohlraum field’.



JD2456035 – log entry

Control Software Posted on Apr 17, 2012 15:32

RH 10%
T 37 F
wind ~20 mph
Moon rising in thin clouds. Later v. nice data (past 2456035.1 for example).
Azimuth near 90.0 degrees.
Dawn started at 5:10 HST.

Note 1: File with actual rising is B frame at 2456035.0738666 – could possibly show halo of BS against the mountain, as the BS rises, despite clouds on horizon – air between mountain-side and telescope probably quite clear.
Note 2: Rose only a few minutes after ephemeris time so we almost caught it rising from sea. Other nights have been 20 m delayed due to slope of Hawaii.
Note 3: started tracking at altitude -4 degrees – so the KS allows this, at that azimuth at least.

Note 4: Changed SHUTTERCLOSINGTIME in the Andor ini file from 20 ms to 100 ms to see if this helps for the ‘dragging problem’. Later changed this back to 20 ms since I got the usual amount of ‘dragged’ frames.



JD2456003 – instrumental alfa determined

Post-Obs scattered-light rem. Posted on Apr 17, 2012 09:28

On the night 2456003 we happen to have a nice sequence of the Moon rising through a clear atmosphere and we can determine the instrumental alfa – i.e. the power of the PSF due to instrumental effects only.

Normally alfa depends on atmospheric and instrumental effects but on a steady night it is possible to gather data for several airmasses and then extrapolate to airmass zero.
The different colors are for different filters (more or less matching the wavelength – blue to red). The data were obtained from the fits of the EFM method.

We see a little bit of scatter in VE2 (red) but otherwise a remarkable agreement in intercept and slope. It would seem that the instrumental alfa is very near 1.724 for all filters. Since we have a ‘base PSF’ (with a power of 1.6) which we raise to alfa’th power to fit observed PSFs we see that the instrumental ‘real alfa’ is near 2.76. The diffraction limit is at 3. We are not saying we have a system that is diffraction limited – but any values of alfa larger than 3 would be impossible!

If we assume that the instrumental alfa is always fixed or decreases (large alfa means narrow PSF, small alfa means wide PSF) with time then measuring the alfa_instr on all good nights would allow us to perform some sort of quality check on the data for that night. Also, keeping an eye out for the largest alfa_instr ever will help us set better upper limits.

Such diagnostic testing as the above should be made a routine part of a future pipeline for reducing data.

Note: This is our second visit to night 2456003 – we have also looked at the same as the above but based on FFM, here. The value for alfa_instr is nearer 1.76 for the FFM, suggesting that there is method-dependence. However, FFM is currently our most troublesome method, of the FM methods.



JD2456034 log entry

Observing log Posted on Apr 16, 2012 15:56

Moon rising in a cloud.
50% RH.
Temp=35 F.
11 mph winds.



New method to calculate bias and FF

Bias and Flat fields Posted on Apr 15, 2012 10:43

I am testing a new method to calculate bias and flat fields. The principle is this: If several images are taken of the same scene at different exposure times then for each pixel you could plot counts against exposure time. They should lie on a straight line (if the shutter works so we can trust exposure time) with intercept at the bias value for that pixel and the slope equal to the gain. The gain and the flat field are the same thing, just scaled differently.

I did this for the V filter using AIR, N1.0, ND1.3, and ND2.0, and using the hohlraum lamp as the source. There is a ND0.9 filter setting that appears to be just AIR, so we ignore that one. For each pixel the regression of exposure time (taken as the requested exposure time – as noted in this blog the measured exposure time no longer does anything, it seems) against counts was made and the intercept and slope stored, along with uncertainties on these, were collected as well as the correlation between regressor and regressand.

More results are on the way but so far this is what we get for the flat field and its uncertainty:

On the left is the flatfield normalized to mean=1. On the right is the uncertainty in each pixel expressed as a percentage of the FF. The color bar at the bottom gives the colors for the uncertainty – i.e the uncertainties are in the range 0.19-0.28%.

The FF looks familiar – bars, spots and the dge effects. I suggest it could be compared to the ones Henriette has from analysis of a few sky sessions as well as the dome flats and some lamp flats, for the thesis.

The plot for the ‘bias’ is more puzzling:

The bias is on the left and the values are near 650-690. The uncertainty is on the right and it is in the range of 10%.

The value for the bias is odd – we are used to 394-396. The uncertainty is huge. We see the signature of ‘dust spots’ on the bias and a little of the ‘barred structure’ that we know from the FF. There seems to be some ‘cross-talk’ during the regression.

I think the large calculated bias is due to an offset in the exposure times – if the requested exposure time is larger than the actual this will cause an overestimate of the intercept – i.e. the bias. We can calculate the offset from the difference between calculated and observed bias and the gain. We have the observed gain from a large number of dark frames taken at the same time as the V images. Calculated and observed biases and the gain are available as images, and this yields the offset as an image of the same size. Taking the average of this we find 0.010 +/- 0.0008 s. That is – the exposures (at least this time) were shorter by 10ms than what we requested. The spread in the offset is due to the roughly 10% noise found in the calculated bias image, not the much more accurately determined gain image.

For this trial the requested exposure time and counts are extremely closely correlated, so the shutter was well-behaved.

The plot shows correlation between exposure times and counts for all pixels.

Does the above suggest a hybrid procedure – subtract the known bias taken
from dark frames and fit the rest, keeping the slope as the gain?

Note added later: The method depends on the source being time-invariant and spatially flat. The main interest right now is that the technique allows an estimation of the pixel.to-pixel uncertainty of the derived flat-field.
This can be compared to other methods that depend on averaging of FF exposures.



How to find a comet

Links to sites and software Posted on Apr 12, 2012 10:34

Comet ephemerids:

link



Three lunar reflectances

Post-Obs scattered-light rem. Posted on Apr 12, 2012 09:29

A central tool in both our and BBSO’s analysis methods is the understanding of the lunar reflectance since it relates the observations to fluxes from the Earth and ultimately the terrestrial albedo. BBSO has measured and determined their own lunar reflectance function. Until now we have been using the Hapke 1963 reflectance model.

Kieffer&Stone 2005 have a reflectance model suitable for phases between Full Moon and Half Moon – these are based on terrestrial ROLO observations of the Moon.

From space SOLSTICE has imaged the Moon at lunar phases to 170 degrees and they show a preliminary reflectance function.

We compare now the irradiance expected from the Moon at a range of phases given our own eshine code (i.e. based on Hapke63), and the Kieffer&Stone model as well as the LLAMAS reflectance from SOLSTICE. The curves have been scaled vertically to coincide at intermediary phase angles.


The crosses are from the Kieffer&Stone irradiance model and should only be used from 0 to 90 degrees. The triangles are from our simulation code, and the red symbols are the LLAMAS data as hand-digitized from Figure 3 of the Holsclaw et al 2010 paper. The K&S model is wavelength dependent and has been interpolated to 282nm which is the wavelength of the LLAMAS data. Our own model has no wavelength dependence.

We see some disagreement on what the slope is as a function of phase for intermediary values, and large disagreement at large phases. The Hapke63 and LLAMAS values strongly disagree at phases from 120-170 degrees.

That is the phase-range in which we currently are limited to work (since the SKE does not work), so we have an issue related to data interpretation. I think what we see here explains why the Lambertian albedo we derive for Earth for lunar phases near New Moon can be above 1, as noted elsewhere in this blog. It is because the Hapke63 model gives a lunar irradiance that may be 2-3 times greater than the observed (LLAMAS) value for relevant phases. In the FFM method we scale the DS intensity to total flux in observations and synthetic (i.e. Hapke63-based) images and with synthetic irradiance exaggerated by so much the ratio to DS flux is low in synthetic images and we have to compensate by driving Lambertian albedo up.

So, we understand stuff!

For better use of the synthetic image code we need to implement a more realistic reflectance. Since LLAMAS data are only preliminary we have to wait for them to publish a full wavelength- and libration-dependent model. In the meantime we could perhaps roughly scale the Hapke63 reflectance – or something like that?

BBSO is somewhat ahead of us here, as they have derived their own reflectance model. We do not have resources to do that currently, although we have plenty of data to do it with. It is a suitable student project (PhD) or even work for a postdoc.



Results

Post-Obs scattered-light rem. Posted on Mar 30, 2012 11:03

We have a result from night 2456003.


The 5 colored plot frames show the ratio of DS/total-flux for all filters using 4 different scattered-light reduction methods as a function of time. The two red curves are the BBSO method and our logarithmic variant of it. The two blue curves are the EFM and FFM methods. The solid curve in the frames with color is the value expected from a constant-albedo Lambert sphere. The last frame is showing the evolution of airmass with time.

The colored curves show the DS/total flux ratio divided by the mean of the BBSO method curve. The spread (+/- 10 %) is BIAS between the methods – they do not find the same answer.

Note that the slope of the observations match the expected theoretical slope.

The method with least scatter is the FFM, closely followed by the EFM. The two BBSO-like methods are not doing well, with our own logarithmic variation on the BBSO method doing least well.

Of interest is comparing VE1 to IRCUT since these filters are almost identical – the small wiggles in one set of curves do not re-appear in the other filters’ curves. This helps us understand what is geophysical signal and what is observational (and data-reduction method) noise.

VE2 is all over the place! This is probably related to the yet not understood intermittent focusing problem we have with that filter.

I find it very encouraging that the slope in the observations is so close to the theoretically predicted slope: The slope is there because during the observing period the phases of Moon and Earth slightly changed, which alters the illumination of the Moon by the Earth and Sun.

When we reduce more nights of data we will be able to begin to see if this slope is always there or changes sign with rising/setting Moon. If that is no problem we can begin to trust the signal we receive – even if the signal is biased by factors we do not yet understand.

This is a feather in our cap! We now know what nobody else knows: How much of the earthshine-based result is due to method and not geophysics. This will have consequences looking forward when enough data has accumulated and has to be interpreted in terms of climate processes.

I am sure that improvements in our analysis-methods will continue to aggregate and we may see the bias between the methods lessen.



Cable near setting Moon

Real World Problems Posted on Mar 29, 2012 09:59

The image shows a cable which the moon can move behind as it is setting at 285 degrees azimuth and 25 degrees latitude.

Arrows show the dim line which is the cable against the sky. The middle arrow shows a glint off the cable from the Moon.


The cable can whip around quite a bit in the wind. It clearly shows up in the CCD images when the Moon is behind it.



Results for 2456003

Post-Obs scattered-light rem. Posted on Mar 27, 2012 15:08

We show results allowing a comparison of the EFM and FFM methods:

All observations for night 2456003 have been reduced to DS/(total flux) for both EFM and FFM method, and a direct albedo determination is also available from the FFM. The DS/total values are of course proprtional to terrestrial albedo and are plotted as crosses (FFM) and boxes (EFM) above. The line is the direct albedo estimate from the FFM method. Values have been normalized to a scale near 100.

The spread is near 5% but the standard deviation of the mean would be near 0.5-1%. If these numbers represented all data from one night we would be in line with BBSO claims, or slightly better.

Analysis of the BBSO method on these data awaits.



Comparing EFM and FFM results

Post-Obs scattered-light rem. Posted on Mar 27, 2012 11:03

For the night 2456003 we now have both EFM and FFM results (and BBSO method results, to be analysed later).

We look at the mean difference and standard deviation of various quantities determined from the images by the two algorithms. The differences are calculated as percentages thus: 100*(FFM-EFM)/((EFM+FFM)*0.5). Data are for about 40 observations with a mix of filter repetitions – since these are differences the mixture of filters should not matter as same-filter images are compared.

alfa Mean: -0.3805 SD: 0.2027
a Mean: -66.64 SD: 0.6907
BS Mean: -0.0004525 SD: 0.0001109
TOTall Mean: -5.082 SD: 0.5449
TOTnoPed Mean: -0.01426 SD: 0.003551

DS23 Mean: 30.68 SD: 7.404
DS45 Mean: -7.415 SD: 9.557
x0 Mean: 0.9189 SD: 1.491
y0 Mean: 0.06020 SD: 0.3928
radius Mean: -11.19 SD: 1.802

The difference in offset a is imposed by the codes and can be ignored. The two totals are: TOTall=sum of pixel intensities in image, TOTnoPed=sum of image pixels minus the estimated bias and sky brightness pedestal (i.e. most close to ‘total original light in the image without sky’). The two DS values are calculated at 2/3 and 4/5ths of the radius on the DS of the disc. The difference is radius is due to user setting and is and error I can see now. Ups. More later!

We notice that disc center coordinates (x0,y0) have a bit of variance – the algorithm does not find the same lunar center and radius each time.

Fixing the codes so that radius is determined in the same way we rerun the analysis and get:

DS23 Mean: 21.70 SD: 5.806
DS45 Mean: 1.648 SD: 6.742
x0 Mean: 0.7158 SD: 1.765
y0 Mean: 0.1147 SD: 0.2926
radius Mean: 0.2838 SD: 1.524

The problem on radius is solved now. The other results are more or less the same – DS23 is significantly biased while DS45 is not (given the scatter).

What can we learn from this analysis? The error on DS and total flux is very small. DS45 is not biased, but the scatter is very large – that is, the EFM and FFM methods arrive at the same answer but both are subject to noise.



Halo size vs. alfa

Post-Obs scattered-light rem. Posted on Mar 25, 2012 18:34

The ‘width’ of the halo we see is due to scattering in the atmosphere and the telescope. With data of the Moon rising it may be possible to extract information on how much of the halo is due to atmosphere and how much is due to the telescope. We first inspect data from night 2456003.

We do two things:

1) We run the FFM method on all aligned and coadded images thus getting values for alfa and airmass, and terrestrial albedo.

2) We calculate the fractional area in each image where the intensities fall in a specific fractile. We choose the fractiles between 0.1 and 0.3 % which turns out to be mainly in the BS halo. Doing this for all images gives us fractile area, or ‘halo size’, as well as airmass.

First we look at the contour plots from each image, in the B band. We have subtracted not only a bias frame but also a fitted linear surface – fitted to the corners of the bias reduced image.

We see 9 images of the Moon (each from the aligned and coadded images available at 9 different times in the B band on the given night).

We have extracted the area (counted in number of pixels) of the fractile between 0.1% and 0.3% (heavy dashed and next contour outwards) cumulated intensity. [Quite complex statistic that: 0.1% refers to the 99.9% brightest pixels of the image. 0.3% refers to ‘99.7% brightest pixels’.] We plot this, against other information extracted:

Upper Left: fractile area vs. time (Moon was rising). We see a smaller area as the Moon rises – less light is being scattered into that intensity fractile.
Upper right: Airmass and fractile area almost proportional.
Middle Left: FFM alfa vs time – the width of the PSF depends on where the Moon is.
Middle right: albedo vs airmass – the FFM-fitted albedo depends on the airmass: Not so good!
Lower left: albedo and alfa are dependent: Not so good
Lower right: fractile area and alfa are dependent: Not so good.

The last three results are not independent, of course.

We can learn a few things from this:

a) the FFM is not (yet) able to determine albedo etc independent of image quality – this is probably due to a degeneracy in the factors of the model we use – it seems that pedestal, alfa and albedo are coupled.

b) the nice relationship between fractile area and airmass, and alfa and fractile area promises an equally nice relationship between alfa and airmass. By linear regression we find:

alfa = 1.76 – 0.0036*airmass

which suggests that alfa (in the B band) is 1.76 at zero airmass – i.e. the contribution of the telescope in the B band to the PSF is a power law with power 1.76.

For all bands we get:
———————–
alfa_0 error
———————–
B 1.757 0.0010
V 1.763 0.0014
VE1 1.757 0.0006
VE2 1.745 0.0002 (one outlier discarded)
IRC 1.758 0.0004
———————–

Within +/- 0.001 it is possible to see no real color-dependence in the alfa_0 parameter. A consistent value for almost all bands is alfa_0=1.76. The VE2 PSF may be slightly broader with alfa_0 at 1.75. VE1 and IRCUT (which are almost identical filters) have very similar alfa_0 and errors.

We have on some nights by other means found alfa near 1.8.

The problem that albedo depends on the PSF power has to do with our model definition and fitting techniques and is open to improvement by adjusting fitting weights and so on. While it is too bad we are not ‘there’ yet with the FFM we see a tool for detecting when we succeed – namely, when the albedo no longer depends on alfa, and this information will be put to use.



Results from 7 nights, EFM method

Post-Obs scattered-light rem. Posted on Mar 23, 2012 15:20

From night 2456000 to 2456007 we have excellent data. Here they are (sorry about the sideways plot) for each band and each night. I show the ratio of the DS intensity in EFM-corrected images to the total image flux. Alfa was always near 1.7 so flux is not expected to be lost off the edge of the image. We have scatter at the 1% level and up.

We have a fairly close relation between observed and expected ratio if we chose the simple Schönberg phase law (blue curve), while we seem to be contradicting the more advanced Lommel-Seeliger phase law (red curve). Hmm? Moon and Earth modeled as Lambert spheres in this argument.

The larger scatter on the first and fifth nights in some filters indicate observing problems and not geophysics!
I think issues in the EFM method are: centering of the Moon in order to extract DS intensity; and choosing the pixels to use as source. On the first night problems are due to very low earthshine at this phase.

The phases or Sun-Earth-Moon angles covered by the 7 nights extend from 97 to 34 degrees. Unfortunately skipping the interesting 42-degree angle. Except for the brevity of the observing window on the last night I see no strong indication that we are getting particularly bad data at high airmass – some of these data points are at airmass 3-7!

Now then – the above to be repeated with the BBSO and FFM methods: all for the poster at the EGU.



Moon movie with halo

Showcase images and animations Posted on Mar 23, 2012 04:34

Movie shows 9 images of the moon centered on a particular crater on the bright side.

Greyscale is “minmax” + “log” in DS9.

The halo light looks very stable around the bright side edge. Naively at least, it appears as if the determinations of alpha (power law fall off of scattered light at large angle) should be very stable.

In this time sequence one can see the terminator changing slightly — certain features brighten, others dim – – especially crater walls and mountains.

Movie is here:

http://www.astro.utu.fi/~cflynn/Vmovie.mpeg

play it with e.g. mplayer Vmovie.mpeg -loop 0
for a continuous loop.



Moonrise

Showcase images and animations Posted on Mar 21, 2012 13:55

Click.



JD2456006

Observing log Posted on Mar 19, 2012 17:12


Extraordinary night – followed the Moon from moonrise until dawn. Clearest night we have had yet. Halo almost non-existent on DS.

This shows that we may be able to get very good data very close to the Sun – but in that case extreme refraction would have to be handled. Will look into whether this can be achieved from the JPL ephemeris. Exposure time also needs to compensate for the strong extinction through 5-10 airmasses. Not sure how to do that without stopping and starting scripts.



Data inventory March 2012

Observing log Posted on Mar 19, 2012 13:43

As of March 18 2012 we have 78.000 images of the Moon, of which 51.510 are relevant for data-reduction with the current techniques at hand.

Distribution across lunar phases is seen here:



Term Independence in Error Budget

Error budget Posted on Mar 18, 2012 09:28

In modelling the error budget in terms of the relative uncertainty on an intensity

delta_I/I

we are looking at something different than the error on the terrestrial albedo

delta_albedo / albedo
or
delta_(DS/BS) / (DS/BS)

We seem to be finding large delta_I/I’s and small delta_albedo/albedo. This can be because an error on BS could be compensating an error on the DS – i.e. the errors are not independent. In writing the error budget as

(d_I/I)^2= Sum_i [ (d_a(i)/a(i))^2],

where the a(i)’s are terms in the error budget we have made the assumption that the a(i)’s suffer errors that are independent so that various cross-terms in the derivatives are zero in the mean.

In model-fitting, such as ours, where some of the factors to be fitted are de-pendent (alfa and offset is an example) the error budget should be written differently, and we should look at this. Henriette’s estimate of d_DS/DS remains correct, I think, but it should be understood that some other terms appear in the full d_albedo/albedo error budget, which cancel. An example of this is seen in the post here.



PSF smooths

Error budget Posted on Mar 17, 2012 08:09

Henriette has shown that there is a bias in the EFM method (on one synthetic image corresponding to one lunar phase with one setting of alfa).

To understand this – and to ultimately understand how to avoid the problem – we perform some tests on synthetic images.

First I compare the intensity of pixels in two images – one image is a synthetic image and the other is one generated by folding with a PSF with alfa=1.6 and 1.8). No noise is added. Plotting all pairs of pixels from the two images against each other we get:


In the upper left panel we compare pixel intensities in the original synthetic image with the same pixels’ intensities in the 1.6-folded image. To the right in the upper row we see results for 1.8. We note that bright pixels have moved to the right side of the red line indicating they have been weakened in intensity; we note that pixels with originally very low intensities have increased and we note that intermediary pixels have suffered both fates. The panel to the left in the middle row shows the percentage changes in pixels from image 1 and 2 in the row above and we see, as expected, that folding with 1.8 lessens the intensity less than if using 1.6 and this is of course because the 1.8 PSF is narrower than the 1.6 one and thus influences fewer pixels at the further parts of the DS. In the rightmost image in the middle row, and the last image at the bottom, we repeated panels i and 2 but now on a log scale to more clearly show the faint end of the sequences. We have also coloured in red those pixels that in image 3 suffered a larger change in intensity than 1.000%; these are all pixels at the faint end – probably pixels on the DS right next to the BS which had their intensities enormously increased when the halo spread out.

The above result was for single pixels. We next investigate what happens to the mean intensities in squares placed across the image at various places and compared between the two images.


Boxes that were 7×7 were placed randomly all over the images and mean intensities extracted. Upper row shows the original intensity vs. that found in the folded image, for alfa=1.6 and 1.8. Bottom row shows the changes of those boxes’ intensities in percent of original value (except 0 values). Red dots indicate negative values folded up to be shown in this log-axis display. We see what we expect – large means suffer the least; low values suffer the most.

The above results help us understand what happens to intensities in folded images. We also understand that if the halo is not completely removed from the DS then the DS may be left with a positive residual flux. We also clearly understand that if the BS is used as the reference than it will always be an underestimate of the true BS intensity in real observed images – by on the order of 1%.

Since we will always only have observed images to estimate the DS/BS ratio from we will always be making a small mistake in using the observed BS intensity (in addition to whatever error we make in using ‘corrected’ DS intensities). What is the change in error on the BS if we have an alfa=1.6 and an alfa=1.8 PSF?

The answer is – about 80% – that is if the BS was in error (compared to the correct value) by 1 percentage point at alfa=1.6 then it may well be in error by about 0.5 percentage point (compared to what is true) when alfa is 1.8. That is – the influence of nightly variations in the PSF (say from 1.6 to 1.8, which is realistic) may cause the DS/BS ratio to vary by 1/2 % solely by the influence via the BS – never mind what problems we may have on the DS under such seeing changes.

As far as I know the BBSO does not address this problem. It is not a problem that is resolved by ‘extinction corrections’ since these always apply to all pixels in an image equally (apart from the field size issue).

Does using ‘the total image flux’ work? Chris has in this blog analyzed what happens to the total flux when an image is convolved with PSFs of various alfas. We revisit those results now:


For a large set of random values of lunar phases a synthetic image of the Moon was generated. It was folded with a PSF with a random value for alfa between 1.5 and 1.85. The total flux of the original image and the folded image was then calculated and the difference converted toa percentage of the original total flux, and plotted above. Even if the PSF is flux-conserving the flux is not conserved for small values of alfa (broad PSFs) because the flux in the halo starts reaching the edge of the 512×512 image frame and is lost. The results above show that for all lunar phases the flux is conserved inside the +/- 0.1% limits for alfa larger than 1.57 or thereabouts. Such a small value corresponds to a fairly bad night so as long as we stay to good nights we can count on the total flux in the image being conserved.



Stdev and bias in EFM

Error budget Posted on Mar 16, 2012 15:35

I have worked with the EFM method for removing scattered light, testing it on a synthetic moon image from night JD 2455923.

The “true” scattered light is represented by an image:
S_true = observed – pedestal – usethisidealimage
The best guess scattered light is represented by the end result from the fit minus the pedestal:
S_efm = trialim – pedestal

I have investigated the mean value of S in the 3 boxes shown below for 100 S_true and corresponding S_efm images. It is the same synthetic image that is the basis of all the “observed” images, but they have slightly different noise added when they are convolved with the PSF.The results:

It is clear that we have significant bias. Not all the stray light is removed with the EFM method. On the positive side, the standard deviations of the fits are small. The counts here can be compared to a typical DS signal of 5 counts.



Error on albedo vs. error on DS

Post-Obs scattered-light rem. Posted on Mar 13, 2012 12:38

When we use the BBSO or EFM methods for removing scattered light we end up with a number for the DS flux – usually in a box on the DS.

When we apply the FFM we end up with a number for the terrestrial albedo – based on all pixels in the image.

The precisions of the two approaches is not the same. In a test on synthetic images with added, realistic, noise, and realistic variability in the offset and PSF-parameter alfa we find that the

S.D. of the determined albedo is 0.12%, while the
mean error on DS intensity is 1.2 %.

This difference was seen in the Figures of the paper we submitted. The present results are a re-calculation of those simulations – it looked too good to be true in the paper, but I can conform that it is indeed the case. I speculate that it is due chiefly to more pixels being used to determine the albedo in FFM than the DS flux in the EFM.

Further calculation of the albedo from DS intensity would carry the 10-times larger error forward to the albedo!



Error on determination of linearly changing albedo

Error budget Posted on Mar 11, 2012 12:41

I have run simulations like Peter’s of our ability to recover a changing albedo with time.

The sims are over a period of a decade with 100 observations of (global) albedo per year (randomly chosen nights). 1000 sims were run and the slope of the albedo change with time measured using least squares from the monthly means. One sim was for no change in the average albedo with time, the other for a linearly incrasing albedo with a slope of 0.1 percent of the albedo per year — i.e. mean yearly albedo increases from 0.300 to 0.303 over a decade.

It’s important to start each sim on a different day of the year — if not, there is a bias in the slope caused by the seasonal pattern being the same in all sims.

The plot shows the histograms of the slope determinations.


Black histogram: no change to the mean albedo with time. Green histogram: change of 1 percent over a decade – i.e. 0.1% per year.

The natural variation in the albedos was set at 1 percent in addition to the seasonal variations.

The histograms are centered on the correct slope — but the width of the histograms is quite broad and backs up Peter’s sims of last week and reported in the blog.

For the green curve the mean of the 1000 simulations is 0.100 (i.e. an excellent match to the input albedo rise of 0.1 %/year) and the scatter is 0.025.

This setup (10 years, 100 albedo measurements per year, 1% natural variation in the albedo) gives a 4-sigma detection of a rising albedo of 0.1%/year



Albedo variations in CERES data

Error budget Posted on Mar 09, 2012 10:25

What is the variability in global-mean daily-average values of albedo?

We need this number in order to argue for the levels of precision we are striving for: A main counter-argument to very high instrumetal and method precision could be ‘the natural variability is much higher, so you need not work so hard’. A much worse counter-argument would be ‘you must have much higher precision to catch the very small variations expected’. The worst is probably ‘natural variability is much higher than all climate change variations of interest so you will never be able to say anything interesting using your – or satellite – albedo data’.

We estimate what the daily global-mean albedo variability is. We necessarily have to do this from satellite data, and use the CERES data product (click on the link).

Downloading 1×1 degree daily-average CERES albedo for ‘all sky’ conditions and averaging all data points not flagged as ‘bad data’ we get a daily series for several years with seasonal variations – like the Figure 1 in Bender et al. Extracting and removing the seasonal cycle, and removing all obvious glitches in the data we get the daily albedo anomaly series. We calculate the standard deviation of this series and express it as a percentage of mean albedo.

We find that the CERES-based global-mean albedo varies by 1% from day to day (1SD is 1.07% of the mean).

We did not weight the data with a cosine law towards the poles, but the effect of this is minor. The 1SD on albedo is therefore near 0.003.

We simulated daily observations of albedo data drawn from a population with the Bender et al seasonal cycle and daily noise added with the 1% SD from above. This is an experiment in sampling a constant albedo. We simulated 10, 20 and 30 years of data. We fitted straight lines to the data using 365 points a year (unrealistic) or 100 points a year (realistic) and expressed the change in albedo, according to the straight-line fit, at the end of the 10, 20 or 30 years period as a perecentage of the mean albedo.

The plot shows histograms from 1000 realisations of the above process. To the left are results for 365 samples a year. To the right for 100 samples a year, and each row is for 10, 20 and 30 years of data, respectively. The histograms show what the chances are of observing an albedo slope of a certain value when the noise is realistic and the actual underlying albedo is constant. For 10 years of data and 100 observations per year the chances of observing an albedo slope between -0.35% and +0.35%, when the slope is really 0, are thus about 68 in 100 (expressed that way to avoid using “%” twice in the same sentence … ). For 20 years of data the same odds apply to the +/- 0.24% range. 30 is – 0.2%.

These results differ from those in the previous post on this subject (below) by the value used for the standard deviation of the variability. In that post we had used the values for scatter in Bender, and they are valid for monthly means and are about 3 times smaller than what we used here. As sqrt(30) is about 5.5 we learn that daily albedo values are not independent – there are about 3 days between independent data points for global-mean albedo!

What have we learned? If our instrument and methods can really give us 1 datapoint each night with 0.1% precision then we are slightly on the luxury side – natural variability will falsely show us slopes of +/- 3 times that number in 10 years of data, and even for 30 years of data we are still a factor of 2 better than we ‘need to be’.

However, if the instrument and methods are not quite up to the 0.1% level we have aimed for (a realistic situation, actually … ) then we are not being over-perfectionists.

Likewise, reported and discussed changes in albedo slope are quite large. Bender et al show in their Figure 1 a change in observed albedo, between the CERES data and the ERBE data of 4% over about 15 years. Nobody believes this change is real – it is due to instrument calibration issues – but the numbers serve to show what we are up against: We should ask, can we rule out 4% change in albedo over 15 years with our earthshine data? The answer would seem to be – yes, we will be able to verify changes in global-mean albedo to the +/- 0.35% level with 10 years of data.

So we are doing all right with our idea!

We should repeat the above with an analysis of what can be done with 1, 2 etc instruments; or a global network of instruments with the occassional dropout due to realistic bad weather, and so on.



JD2455996

Observing log Posted on Mar 09, 2012 10:05

Hazy – ring around the Moon.



measuring linearly changing albedo

Error budget Posted on Mar 09, 2012 05:41

Simulation of 100 albedo measurements per year of a linearly rising
albedo of 1 percent per decade. The simulation is based on the mean monthly global albedo estimates over several decades from a suite of models as in Bender et al . 2006 (Tellus, 58A, 320) figure 3.

grey crosses : individual measurements

green line — monthly averages

red circles — annual averages

the black lines are the fits to the monthly averages — they both come out with the correct slope – so it does work!


the two panels show the effect of 1 percent (lower) and 0.1 percent scatter (upper) on
the individual albedo measurements due to natural variability.

the 0.1% case allows the underlying model to be seen clearly.

1% scatter in the daily albedo makes things look a deal noisier — but the right slope comes out and one can get a pretty good seasonal pattern
from the monthly means and better stull by folding the data into a year.

we are looking further into what daily variations naturally arise — the Bender paper is not clear on this.



comparison of alignment procedures

Error budget Posted on Mar 08, 2012 18:30

I have compared two different methods for aligning images: Chae’s IDL procedure and Kalle’s / my center of mass Python procedure.

The example given here is the result for a stack of 11 images obtained within 0.7 minutes on JD2455864 in the V-filter, and it is representative for many other similar stacks.

I have aligned the stack with each of the two methods, determined the standard deviation of the stack and divided this with the coadded image of the stack. Finally the image has been multiplied with 100, so that it directly shows the relative error on the raw observation in percent. The images are basically the dO/O in the error budget equations.

The left images are aligned with Chae’s method and the right images are aligned with the center of mass method. The only difference between top and bottom images is the scaling. The top ones are shown on a square root squale that emphasizes the bright side of the Moon, and the bottom images are shown on a histogram equalized scale that emphasizes the dark side of the Moon.

The center of mass method out-performs Chae’s method for both the bright and the dark side. This is especially true for the earthshine. Both methods have difficulties with the bright limb.

The difference between the two methods is not so much the method for calculating the required offset. Instead it is the interpolation technique used in the subpixel move that matters. Chae’s method (shift_sub) uses bilinear interpolation, whereas the CM method uses exponential functions as the interpolating functions. With the CM method there is visible blurring of the lunar features, and it is this smoothing that lowers the standard deviation of the individual pixels.

In the form the methods have now, I would recommend Chae’s alignment whenever we wish to show a nicely centered pretty Moon image and the CM method whenever we wish to determine the intensity of a box of pixels on the Moon. Ofcourse we can play with the parameters of both methods and this might increase or decrease the blurring.



Wild Slew is back!

Observing log Posted on Mar 07, 2012 08:21

On JD2455993 I tried to do lamp flats since the night was cloudy and humid. Started the LAMPTEST script, the system started slewing towards the hohlraum lamp – and kept going towards the floor. KS stopped it before it hit anything.

Why on earth are we back to the ‘old ways’? This is the sort of behavior we had before the ‘dont send commands during slewing’ problem was solved by Ingemar.

Because the system has been well behaved for a long time I did not have the cable sniffer running so we have no record of what went on.

Any ideas of automatic operations while the system still does this must be ruled out!

Will now reboot everything, calibrate (on another night), and see if the problem reappears.

We can only hope that someone had been moving the telescope about by hand, as that would explain what happened.

T=7,4 C in dome, RH=70%.



Can it be done, we ask!

Error budget Posted on Mar 06, 2012 14:36

Our instrument design and data reduction goals have been driven by a 0.1% error requirement.

In light of considerable natural variability from year to year and week to week in earths albedo we ask which limits on achievable accuracy the natural variability set?

We illuminate this problem by modelling natural albedo variability, based on Figure 3 in Bender et al (2006). The Figure gives an annual cycle and the 1 S.D. variability limits around this cycle.

We use that data to generate 10, 20 and 30 years of daily simulated albedo values. We fit a straight line to the entire dataset as well as to a dataset made up of 100 annual samples (like realistic observing conditions – one data point per night). We calculate the change in albedo over the period simulated and express this change in percent of the average albedo. We do that for the daily sample and for the 100-times-a-year sample. We next show the histograms of these accumulated errors. To the left in each row is the histogram of errors in albedo for the ‘all-nights period’ and to the right is the histogram for the ‘100 annual samples’.

The S.D. of these errors are:

years all days 100 days
10 0.102 0.184
20 0.077 0.132
33 0.060 0.105

The above suggest that natural variability will impose an uncertainty on estimates of global albedo at about 0.1% if just 10 years of nightly data are analysed, but that 30 years of data are needed if only 100 samples a year are made.

Any errors our measurements have above the 0.1% level will thus only make the problem worse.

The above analysis assumes global coverage of our data; all sampling effects due to lunar phases being are ignored by the random selection of 100 point per year.

Overall the analysis suggests that we may be able to achieve climate-change related data of interest in a decade as long as we do not allow errors in our observations or data-reduction much above the 0.1% level.
On shorter time-scales the high accuracy of our method will allow us to track such phenomena as the daily or weekly changes in clouds masses as well as more subtle changes such as seasonal changes. A volcanic eruption would be yum-yum!



Arithmetic solution for albedo

Post-Obs scattered-light rem. Posted on Mar 05, 2012 15:37

“Eureqa” is software that searches for regression formulae that fit given data in the most parsimonious way – it not only tests which regressors are the best, but also searches a large space of possible functional forms. We have used Eureqa to hunt for solutions for SSA (i.e. albedo) in sets of data extracted from a large number of synthetic images.

A training set was generated from the averages in 64 equal-size boxes laid out over synthetic lunar images.
Each synthetic image was generated for the same JD, but each had its own realization of Poisson noise. One or 30 of these frames were averaged, and a set of 1000 such images were generated with random values of alfa, pedestal and albedo sampled from uniform distributions.

The distribution limits are:
alfa 1.5 to 1.85
ped 0.0 to 50.0
albedo 0.1 to 0.9

Eureqa was run until no further progress seemed possible.

Results:

formula from 30 frame averages:
“SSA = (199,9*V29 – 199,3*V48)/
(V53 + V47*V47 + 399,8*V29*V47*V47*V47 – V1*V29 – V1*V47 – 398,5*V47*V47*V47*V48)”

formula from single frames:
“SSA = 60,55*V28 + 171,8*V11*V28 – 0,006181 – 60,62*V32 – 159,8*V40*V40”

The VXX refer to the mean of the XXth box on the lunar image – counting from the corner and along rows, then columns.

Statistics 30 frames 1 frame
————————————————————–
“R^2 Goodness of Fit” 0.9999703 0.99935872
“Correlation Coefficient” 0.99998515 0.99967931
“Maximum Error” 0.0055585436 0.027577193
“Mean Squared Error” 1.6123362e-6 3.5070792e-5
“Mean Absolute Error” 0.00093508038 0.0043065976
“Albedo error” 0.32% 1.4%
————————————————————–

We see that averaging 30 frames gives better results than using single frames, as expected: We see the maximum error better by a factor of 5, the mean square error by 22, the mean absolute error by 5. Thus, the error in the fit scales about as you would expect – by the
SNR.

We see albedo errors of 0.3 – 1.4%. These are calculated using the Mean Absolute Error (i.e. best case choice – not conservative). These are all higher than our science goal of 0.1% so we must use more frames.

Discussion.

How would one use a system based on the above formulae?

Each formula is trained on centered images from a given point in time. It can be used only on images also centered and with the same image scale and from the same time. A training set can be generated for any point in time, and it is possible to ensure that the image scale is the same as for the observed image. Observed images may not be centered, however. It is probably more difficult to shift the observed image, because of edge
effects, than it is to determine the shift and then shift all synthetic images equivalently and train the formula on the shifted synthetic images.

While the use of a simple formula such as the one above would be extremely fast, it does not appear to offer an advantage in terms of error over our other methods.

The method is somewhat similar to Chris’ principal components method, but governs itself in finding the optimum boxes – i.e. areas on the lunar image to use.

The method quickly finds similar-quality solutions if started from a fresh set of images, but the formulae may differ in choice of detected variables. This implies that the possible-solution space has not been sufficiently sampled by the 1000 image training-set.

Generating larger training sets is possible – generating 1000 takes about 20 minutes, though – so it is not realistic to generate sets that are e.g. 100 times larger.

Training the formula takes 10-20 minutes using 1000 images and 2 CPUs on an ordinary PC workstation. Perhaps there is scope for much larger sets using the CRAY and multiple processors?



JD2455991

Observing log Posted on Mar 04, 2012 09:29

Moon in clouds. RH 85%. No observing.



JD2455990

Observing log Posted on Mar 03, 2012 17:04

A slight haze at MLO.



JD2455988

Observing log Posted on Mar 02, 2012 08:15

Trying to observe Moon. Many thin clouds, but it may clear up later and then we will not waste time on setup. T=7.4 C. RH near 40%, varying.

Shutter works irregularly at first, then regularly – as if it has some dirt or oil in it that needs to be rubbed around until it is warm? Now and then there is ‘dragging’ but, as always, mainly in one filter – when a new filter is chosen the dragging may go away, or reappear in another filter.

Dome needed a fresh ‘find home’ – wonder why? I left it in a working condition? Dome now follows telescope.

Moon in frame but not well centred. Why? It is just a few days since I calibrated the mount on a target in the same part of the sky? ‘Slow creep’ of alignment seems to be a feature of this system.

More clouds now. Shutting down. Need a cloud sensor up and running!



SVD on images

Post-Obs scattered-light rem. Posted on Mar 01, 2012 10:41

Our analysis of images is hampered by their numbers and size. Many of the pixels in each image represent sky or otherwise similar areas: In other words – the ‘degrees of freedom’ of the image is probably less (much less) than the number of pixels in the image. In order to explore the potential of methods that operate not in the image plane but in various transform planes I investigated just how many singular values are needed to satisfactorily reconstruct a typical lunar image from an SVD – a singular value decomposition. First I just tried to reconstruct an image completely, using all 512 singular values:

At upper left is the original (equalized) image. At upper right is the reconstructed image, and at lower left is the difference image. At 512 singular values the RMSE in % was very low – 0.00219 %. The RMSE fell below 0.1% at 420 singular values. The usual plot of the singular values sorted by size shows a ‘break’ near 100 singular values, suggesting that by using more than 100 SVs it is mainly noise that is being reconstructed. Inspection of the spatial reconstruction error as function of increasing number of SVs showed that at 80 SVs the added effort goes mainly into reconstructing the sky – i.e. noise.



JD2455987

Observing log Posted on Mar 01, 2012 08:12

Ring around the Moon! No observations! Doing lampflats instead. T=7.3 C in dome.



JD2455983

Observing log Posted on Feb 26, 2012 10:42

Trying to observe M41. Temperature in dome 6.4 C at start. Very light haze.

Shutter giving problems: So far, all shots are bias frames – except one that is 0’s in one third … as if the camera died?

At one point, when this happened, some LabView panels having to do with Andor configuration and Shutdown moved to the front on the logmein display: I think this means they had been activated – I had done nothing to cause this – so perhaps it means that the system did it having sensed something was wrong?

Abandoned M41 – then tried the hohlraum source. No images at all recorded now.

Rebooted everything.



JD2455982

Observing log Posted on Feb 25, 2012 13:14

Mount re-calibrated after Ben’s fix of the Kill Switch ring. Can now reach -30 degrees declination.



Bunch of papers on earthshine

Relevant papers Posted on Feb 06, 2012 10:49

2004: The BBSO group’s Science paper:

http://adsabs.harvard.edu/cgi-bin/nph-data_query?bibcode=2004Sci…304.1299P&db_key=AST&link_type=ABSTRACT&high=4d491396e418829

2005 Controversial multi-dataset compilation:http://adsabs.harvard.edu/abs/2005GeoRL..3221702P

2006: Frida Bender et al at MISU gets in the act:

http://adsabs.harvard.edu/cgi-bin/nph-data_query?bibcode=2006TellA..58..320B&db_key=AST&link_type=ABSTRACT

http://adsabs.harvard.edu/cgi-bin/nph-data_query?bibcode=2006GeoRL..3315812B&db_key=AST&link_type=ABSTRACT

2006: Palle has a shot back ati Bender:

http://adsabs.harvard.edu/cgi-bin/nph-data_query?bibcode=2006GeoRL..3315813P&db_key=AST&link_type=ABSTRACT

2007:

Langley/NASA is not amused:

http://adsabs.harvard.edu/cgi-bin/nph-data_query?bibcode=2007JCli…20..575L&db_key=PHY&link_type=ABSTRACT

Langley revises their methods:http://adsabs.harvard.edu/cgi-bin/nph-data_query?bibcode=2007GeoRL..3403704L&db_key=AST&link_type=ABSTRACT

BBSO shoots back:

http://adsabs.harvard.edu/cgi-bin/nph-data_query?bibcode=2007JASTP..69.1556G&db_key=AST&link_type=ABSTRACT

2009:

BBSO rumbles some more:

http://adsabs.harvard.edu/cgi-bin/nph-data_query?bibcode=2009JGRD..11400D03P&db_key=AST&link_type=ABSTRACT

And so does NASA:

http://adsabs.harvard.edu/cgi-bin/nph-data_query?bibcode=2009JGRD..11411109L&db_key=PHY&link_type=ABSTRACT

I stumbled across these, which look interesting:

http://adsabs.harvard.edu/cgi-bin/nph-data_query?bibcode=2009ApJ…700.1428O&db_key=AST&link_type=ABSTRACT



FFM Revised

Post-Obs scattered-light rem. Posted on Feb 04, 2012 20:48

I have re-fitted the synthetic images with the FFM method, but this time used weighted least squares to perform the fitting. The weight was 1/sigma in each pixel where sigma is the Poisson SD. The previous bias that was seen for single images appears to be gone now. The scatter for single images is also much smaller.



FFM scores even Higher

Post-Obs scattered-light rem. Posted on Feb 03, 2012 08:40

Here are the results of a test of the FFM method similar to the previous
one (below) but now based on 100-image stacks. The improvements are
appreciable: There is almost no bias (5e-5 or 0.02%) in the determination of albedo (top panel), the scatter is less (SD=0.00034 or 0.11%) and the range of useful lunar phases seems to extend to almost +/-120 degrees. The mean bias is 5 times less than the SD so in effect we have no bias now and are determining albedo to within our science goal. Dotted line shows +/-1% from true albedo value in top panel.

Note that the error bars are unrealistic – they are merely the formal values that the fitting method gives under the assumption that all data are statistically independent. We can get empirical error bars instead by doing some Monte Carlo or bootstrapping on a few of the results.

Test for alfa=1.6 remains to be done. More later!



FFM scores High

Post-Obs scattered-light rem. Posted on Feb 02, 2012 13:49

For a full month of single images (alfa=1.6) we show how well the FFM does at recovering the terrestrial albedo from the noisy synthetic images.

The crosses show the solutions for the single scattering albedo used in the Earth-component of the model that generates the synthetic lunar images. The dashed line shows the true value (0.297). Bars on the points show the LM-estimated 1 SD. The two dotted lines show the 1% upper and lower levels from the true value. New Moon is at phase 0. The simulations were performed on single realistic image calculated using alfa=1.6.

The solutions are clearly biased from to the true value and the bias is significant at the several sigma level, but the bias is small compared to other methods’, such as our own implementation of the BBSO method. Solutions are inside the +/-1 % limits for phases roughly between -90 and +90 degrees fromNew Moon. BBSO quotes 1% levels of uncertainty on albedo for a single night. We get 1/3 % in single images.

If the realistic limits on lunar observability is +/- 40 degrees from New Moon we have significantly extended the range available with the FFM method, compared to the logBBSO method applied to stacks of 500 images which was seen to be more than 1% biased outside about +/- 45 degrees!

Here is a 3-panel figure showing additonally the detected values for offset and alfa:

We see some breakdown of the method near New Moon – but NM cannot be observed anyway so that is OK.
I think that the bias in alfa is causing the bias in albedo – so if we could fix that …

Similar plot for alfa=1.8 (incomplete – it crashed! Gahh!)



A success

Post-Obs scattered-light rem. Posted on Feb 01, 2012 17:05

The Full Forward Method for determining not only earthshine intensity but also terrestrial albedo has been tested on, so far, a singe synthetic lunar image corresponding to a few days after New Moon.

By minimizing on the difference between the ‘observed image’ and trial models from Hans’ synthetic image generator over the whole image frame, for three values of PSF width, we can retrieve the terrestrial albedo to within 0.07% for alfa=1.6, 1.7 and 1.8.

This is below our science requirement, and gives confidence!

The model must be tested for images corresponding to a whole month now. The calculation is time-consuming since it is not programmed for the DMI Cray yet, so meanwhile the method runs in a mixture of Fortran and IDL on a workstation.

The fitting method can handle offsets in the image so to some extent an incorrect bias subtraction should be subsumed by this analysis method.

The method relies on Chris’ insight that a library of lunar images for a given phase but different terrestrial albedos can be generated from the linear sum of an ‘albedo is 0’ image and an ‘albedo is 1’ image. The weighting function between these extremes is simply the albedo and is one of the fitting parameters.

Testing remains also for how dependent we are on knowing the PSF – i.e. not just the exponent but also things like non-power-law shape.

This result brings to an end many weeks of worries I had that we would not in time be able to show that we have a method that is good, and now – after some more calculations overnight – we should be on line to write the results section of our paper!

Wahooo!



If only!

Relevant papers Posted on Jan 29, 2012 11:38

Just for fun: this in yesterday’s “The Age”, from an article originally in the Guardian:


“Albert Einstein with an equation for the density of the Milky Way.”

Oh how we pine for an equation for Earth’s albedo!



More analysis of scattered light

Post-Obs scattered-light rem. Posted on Jan 29, 2012 01:48

Some more analysis of scattered light.

I take three synthetic lunar images —

1) a thin crescent with no ES, i.e. BS only

2) ES only

3) BS + ES


Raw images shown on the left — with scattered light on the right. (All images shown at the same display levels, making the halo hard to see!)

The albedo adopted was 0.30 for the ES on (2) and (3).

All three are generated with the correct flux scaling for the components
i.e. image (3) is the sum of (1) and (2).

These three images are convolved with the PSF with alpha=1.8.
No noise is included in the final output images.

I then compute the amount of light which has scattered off the
original synthetic images and onto new pixels. (A list of all pixels which
had non-zero counts in them in the original images was kept and the
counts in these pixels in the convolved images compared to the total
counts in the convolved images. The normalisation of the convolution is set
to the total counts in the original images).

While doing this, I checked how much light scatters off the edge of the 512×512
frames and into the larger (3*512)x(3*512) frames used for the FFT. This turns out
to be ~0.04%, a good deal less than our target accuracy (0.1%)) for alpha=1.8, so this is not a major source of error.

The amount of off-moon scattered light is as follows

1) BS only : 4.2 percent
2) ES only : 0.58 percent
3) BS and ES : 3.9 percent

let’s call this the “halo light”.

these numbers highlight why getting the BS to ES normalisation correct
when generating forward scattering fits to data is important!

We want to measure the DS to BS ratio to an accuracy of 0.1 percent, so the approximately 4 percent of halo light has to be gotten right when figuring out the amount of BS light. The difference between BS only halo light and BS+ES halo light is of order 0.3 percent, 3 times more than our desired accuracy. This shows that the ES makes a non-negligible contribution to the halo light.

Those numbers were for alpha=1.8. For alpha=1.6, I get

1) BS only : 6.9 percent
2) ES only : 1.58 percent
3) BS and ES : 6.0 percent

Now the ES is making a very significant contribution to the halo light — 16 times more than the accuracy with which we would like to know the BS light.

The light lost off the 512×512 frame is small at 0.04% – half of our target error.
However, for alpha = 1.6, this lost light increases to 0.3 percent of the total.

Lost light off 512×512 array for centered thin crescent Moon

alpha lostlight
2.0 0.004 %
1.8 0.04 %
1.7 0.11 %
1.6 0.32 %

It looks like we need to take into account lost light off the 512×512 array
when we are doing the fitting for the BS, especially if the images aren’t well centered, as is presently an issue!

More added

In response to Peter’s comments:

We have both done tests on artificial data which suggest that we are able to measure alpha with an accuracy of +/- 0.01. I figured out the corresponding missing light off the edges of the image for this uncertainty in alpha:

alpha lostlight
1.80 +/- 0.01 0.038 -/+ 0.004 %
1.70 +/- 0.01 0.11 -/+ 0.01 %
1.60 +/- 0.01 0.32 -/+ 0.03 %

So if we really have alpha=1.8 and can measure it with an accuracy of 0.01, we are pretty safe — at least for this thin crescent which is well centered.

Peter asked: what if extinction and alpha are correlated? Great idea! We should certainly look into this!

I had another idea — what about if we made longer exposures of the moon which would saturate the moon but sample the halo light well all the way to the edge of the frame? Could we then measure alpha with appropriate accuracy where it counts — at the moment we are merely extrapolating a not very well sampled halo beyond the edges of the frame to estimate the missing light (in 30 ms exposures, the halo light is very close to the bias level at the edges of the frame, so is not well measured out there, and a bias error could lead to systematics in the estimate of alpha, perhaps larger than the error estimate adopted above of 0.01, which was obtained for rather idealised synthetic data.

The behaviour of the scattered light around the moon beyond the edges of the frame is still very much an open question — we didn’t acquire the data at the last full moon to settle this, but just got more questions!

More still: position of the Moon

Tested the effects of the positioning of the moon using the same setup as above

For 1 pixel shift of Moon in the x-axis, I got negligible changes in the amount of lost light.

For a 10 pixel (circa 70 arcseconds) shift (to the left along the X-axis for a crescent moon which is illuminated on the left side):

alpha lostlight
2.0 0.005 %
1.8 0.04 %
1.7 0.12 %
1.6 0.33 %

The changes to the fractional lost light are very small. So the good news here is that we are not very sensitive to the precise position of the moon if it’s near the center.



The Test Returns

Post-Obs scattered-light rem. Posted on Jan 28, 2012 17:38

In order to better understand the various data-reduction methods, I test here a modified forward model. It consists of a test on synthetic images – this allows us to test what the error is relative to the ‘truth image’.

A synthetic ‘observed image’ is generated as the convolution of an ideal image with Poisson noise and an intensity offset as in a pedestal. This pedestal models the effect of small error sin bias subtraction.

Trial images are next generated and compared to the observed image – when a good match is found the winner model image is declared. We estimate errors in a box on the DS. We use a grid search in alfa for the PSF and in offsets. We CALCULATE a factor that scales the intensity so that once the offset and the PSF are chosen the resulting image is scaled so that the total flux is the same as in the observed image.

We used 0 bias or offset and we used 0.01 as the offset. We used alfa =1.6, 1.7 and 1.8. We allowed loops that bracketed these values.

The test was based on the RMSE between the observed image and the trial images. For the winning model image the error between the known ideal image intensity inside the box and the intensity inside the same box in the offset and scaled (but unconvoluted) image was calculated.

We found errors in the 0.0 to 0.75% range, seemingly independent of the value of alfa or offset.

The above was for one lunar phase near new Moon. We repeated the test on an image with a phase larger (like quarter moon). We found errors from 0 yo 1.85% with a tendency to be large for small alfa – i.e. broad PSFs.

The above shoul dbe repeated for all phases.

The results are similar to what we have found for the EFM method – but for that method results are available for all phases, so – … (more later)

On the above we note that the best solution found seemed to have biases in the alfas and the offset – nevertheless the best solution was fairly good, as shown.

So: the above test was witha method that unlike the EFM did not constrain the trial images to ‘fit through the sky’, but it did, like the EFM, constrain the total flux.



The Testing of Methods

Post-Obs scattered-light rem. Posted on Jan 26, 2012 08:07

Here are some results from a sensitivity-tst of one of our scattered-light-removal techniques. The technique is a forward convolution method but is not the EFM method, but shares elements from that method.

The EFM method consists of these steps:

0) An observed image is at hand.

1) From the observed image an idealized ‘source’ is constructed from all the bright pixels of the image.

2) A model observed image is calculated on the basis of trial values of alfa (the power-law exponent in the PSF) and the source image in 1 by FFT convolution

3) The model image is now shifted and scaled in intensity so that the halo matches the sky part on the DS and so that the brightest part on the BS disc matches.

4) the RMSE between the model image and the observed image is calculated on an area of the image that includes large parts of the sky around the Moon.

5) The trial image that has the smallest RMSE ‘wins’

6) That winning image is subtracted from the observed image. Since the model image did not contain any DS pixels (see step 1) the difference between the observed image and the model image should be the DS.

That was the EFM method. The method I test now is based on the above but omits step 3. In the EFM method step 3 was included in order to ensure flux conservation while making the model ‘fit the sky’ (this is reminiscent of the BBSO method where the sky if fitted ensuring that the model is anchored in the actual sky level thus making the residuals there 0).

We show two plots. Each plot has 3 panels – they are:
Panel 1: The Moon image being analysed.
Panel 2: A slice through the lunar disc with three graphs – the red one shows the known solution (the tests here are performed on synthetic images where a halo is applied with a convolution, using aflfa=1.8, to simulate effects of atmosphere and optics); the black one is the ‘observed’ image in step 0 above; the blue curve is from the image in step 2.
Panel 3: The black curve is the ideal image showing the structure on the DS, the thick green curve is the difference between the blue and the black curve in Panel 2 – i.e. it should be the model estimate of the DS.
Panel 4: This is the percentage difference between the two curves in panel 3.

The difference between the panels on the left and the right is that on the left a small value has been subtracted from the image in step 0 to simulate an uncertainty in bias subtraction; on the right no such value has been subtracted. The value subtracted on the left is 0.01.

A typical uncertainty in bias level may be a number of that magnitude – the consequence is an error in the estimate of the DS flux of 5-10 %.

Henriette’s estimates of how well we can subtract bias levels can be used here to introduce more realistic values of the error (0.01 may be too large, we hope).

The above shows the sensitivity in a scattered-light-removal technique to small errors in bias subtraction. This was NOT the EFM method which anchors itself in the DS sky, but a similar analysis is required for that method before we can present it in a paper.

Note how the slopes of the DS sky in the above plots do not match, despite the same alfa being used in the generation of the test image and the model image. Remember that in a lin-log plot like this a power law is not linear. Inspection of the blue curve in panels 2 above shows that it does indeed curve – the different slopes on the DS sky may be due to the proximity of the DS flux itself (which is also convolved and contributes to the DS sky halo) in the black curve. In other words: the halo on the DS sky in a real image may not be entirely due to the scattered BS, which we have assumed in the above.

This suggests that we should test methods based on convolving entire synthetic images of the Moon for the modelling part – make up the model of more than just the BS as in step 1. If we make up the models of BS+DS images we will have to enter the intensity of the DS, or directly, the terrestrial albedo, as a parameter in the grid search.

This is actually a feature of ‘Chris Method’ that we have seen in this blog.

This is our next step, but first we want to know if the above simpler method (just loop over values of alfa, not alfa and albedo) can be shown to work better if we do as we actually do in the EFM method – namely, we ‘force a fit’ on the DS sky. Since the preliminary trials of the EFM has shown up some not-understood problems of their own we are still working on the matter.



JD2455949

Observing log Posted on Jan 22, 2012 17:21

Ran scripts to observe Saturn and Mars. Strange problems with script not writing to disc. Rebooted and all was OK.

T 4C outside, 6.7 inside. 10 m/s, 40% RH. Clear.



Conservation of flux

Exploring the PSF Posted on Jan 22, 2012 13:37

During convolution of synthetic images with various PSFs we need to consider if flux is conserved. We calculate such images for a range of alfas and show here the difference in percent of the total flux of the image relative to the ideal image.

Percentage-change PSF-file
——————————
-1.6768524 % out1p4.fits
-0.060953825 % out1p6.fits
0.014277556 % out1p8.fits
0.027423956 % out1p9.fits
——————————

That is, in an image convolved by the broad PSF with alfa=1.4 1.67% of the flux is lost, while only 0.03% of the flux is changed when convolving with a PSF with alfa=1.9 (the narrowest possible).

The loss (and gain) in flux is not understood yet – but is perhaps related to clipping (in the case of the broad PSF) of halo when the images are padded 3×3 in order to perform the FFT, or roundoff (narrow PSF – many values are rounded below 1 to 0).



Effect of a PSF on the lunar disc

Exploring the PSF Posted on Jan 22, 2012 13:17

To show the effect of progressively wider PSFs smearing the ideal image, we apply the standard PSF with alfa set to 1.4, 1.6, 1.8 and 1,9 (close to the limit), and plot a ‘slice’ across the discs:

Top panel: In red the counts of the ideal image across the lunar disc, along with slices of smeared images for alfa=1.4 (top), 1.6, 1.8, and 1.9 (lowest).
Bottom panel: The absolute-value percentage difference between the slices in the top panel and the ideal image slice.

In the first panel we see that indeed the halo is approximately linear on the DS part of the sky.

We see in the bottom panel that the DS (columns 150-390, approximately) is strongly affected by the smearing by the PSF for values of alfa we have seen on even ‘good nights’ – alfa~1.6. Occasionally we have seen alfa~1.8 and even there the DS is strongly affected at the extreme edge of the disc, away from the BS, by approximately 10%.



Testing stability of Poisson-noise ‘addition’

Post-Obs scattered-light rem. Posted on Jan 22, 2012 11:11

In our tests of various scattered-light removal techniques we are using synthetic images to which various realistic things have been done – smearing by a PSF and ‘adding’ Poisson noise, etc.

Poisson noise should not be ‘added’ because this will increase the mean value of a region of an image. Instead, Poissonian noise should be drawn from the Poisson distribution with a mean at the value of the image pixel under consideration.

We have two codes to generate Poisson noise in images and we test these next. We take one synthetic (i.e. noise-free) image and define a region on it. In that region we calculaet the mean. We then generate Poisson-noisy images using our IDL code and a Fortran code. These are different in that the IDL code writes out floating-point images while the Fortran code writes out integer images.

In a loop over 1000 trials we generate noisy images from the original image and each tine calculate the mean in the predefined region. We set aside the percentage difference between the IDL code and the ideal image and the Fortran code and the ideal image.

Then we plot these percentage differences:

The two distributions are symmetric about 0 and practically identical. From this we conclude that both codes ‘add’ Poisson noise to the image correctly and that the writing-out of the integer resulting image in the case of the Fortran code does not cause any rundoff problems, even if the region inside of which we tested had a small mean value (about 3.9 pcounts). The regiion was 19×19 pixels large.

We do note that the spread, although similar, is quite large – SD is 4 and 6 counts respectively. This implies, unsurprisingly, that the number of boxes of size 19×19 that must be averaged over is in the hundreds if the standard error of the mean is to be at our science goal of 0.1%. We knew that – this is just a reminder …

Final note: the Fortran code uses a fixed seed to generate random numbers. A modification was needed to read in a random seed instead, from the command line. This is supplied when the code is called by the user.



JD2455944

Observing log Posted on Jan 18, 2012 11:38

Observing asteroid EROS since it is close to Earth. Trying for a movie of the Moon rising behind the flank of Mauna Loa.

45% RH (rising), some indication of clouds, T near 3 C outside, winds 4 m/s 220 deg.



Moon halo

To-do list Posted on Jan 15, 2012 14:32

In order to better understand the empirical description of the Moon halo we need more deep images of the Moon itself. We have studies of point sources that show the halo but need a connected set of images from one night that allows tracing of the Moon halo out far, and then a set of images of the sky that show the halo but without the Moon in the image. We almost did this for one night but could improve the data in the following way: Image where Moon is in the frame should be deeper which probably means overexposing the CCD or we should get MANY shorter exposures and then stack them. We also need longer exposures from the non-Moon part of the scan to go as far out as possible.

SWARP may be useful for generating one connected picture of the Moon Halo, suitable for study and display in papers.



Ellipticity

To-do list Posted on Jan 15, 2012 14:28

We know, from observations of the M7 cluster, that the stellar images are not round across the field. This may because some of the optical components are not at right angles to the optical axos. I think that the color filters are angled slightly. We would like to map the ellipticity better and need more images – they should be rich in stars from side to side. Deep images of the Milky Way may be what we need.



JD2455940

Observing log Posted on Jan 13, 2012 17:37

T fell to -3 degrees C outside. Inside the dome it was as low as 2.7 degrees C. About 30% of images were all right. RH very low. Wind near 8m/s.

Hit kill switch again at Alt 75 degrees near meridian. Mount calibration may be lost now – but seems to be less than a few pixels.



Target drift

Mechanical design Posted on Jan 13, 2012 17:35

In a sequence of observations of Mars (using Sidereal tracking) we sought to learn if there is a periodic error in the tracking. We plot the pixel coordinates of Mars in the session covering several hours:


What is most evident is that Mars drifted relative to the frame – but this may be the actual motion of Mars, although it seems a lot (Mars moves in RA mostly, not DEC – i.e. Y). This can be checked against the position of the stars in the frame – more later.



JD2455938

Observing log Posted on Jan 12, 2012 11:38

Moon observing with script Moonb.csv, which – wait for it – AUTOMATICALLY adjusts the dome position! Whohoo! AND the exposure time!

T near 4 C in dome so many dragged frames. RH v. low. Winds near 5 mph. Halo around Moon seems large.

Good to listen to the shutters etc via a Skype call from the telescope. Shutter is LOUD, when it works.

Mount is re-calibrated following yesterdays chrash into the kill-switch.

Image still jumps, but not as much as before – it seems to be at only the 5-10 pixel level now.



Scattered light from Moon measured at MLO

Post-Obs scattered-light rem. Posted on Jan 11, 2012 09:36

Peter and Chris took images of the sky near the moon, going outwards from the Moon in 0.5 degree steps, in order to measure the powerlaw falloff of light using the MLO telescope.

Data taken on night of 2012-01-08.

We got good data for 4 positions, 1.0,1.5, 2.0 and 2.5 degrees from the moon,

WCS coordinates for the frames were found using astrometry.net. This is a great site!

Stars in the fields were found by searching Simbad for objects brighter than 10th magnitude. A small program was written to locate these stars and do the photometry based on the magnitude equations determined previously for the CCD, and their J2000 coordinates. This is quite nicely automated now. Bias frames were subtracted from each of the images and the magnitudes and sky flux of the known stars in the field computed. The magnitudes were compared to the known magnitudes and came out very nicely.

The images were taken at
POS3 : 06:27:24.313 +22:22:27.07
POS4 : 06:27:25.994 +22:51:53.62
POS5 : 06:27:23.910 +23:22:22.01
POS6 : 06:27:26.336 +23:51:34.44

Some example images with stars of known magnitudes:
POS6:

POS 3:


The Moon during all of this was at 06 27 36 +21 50 28.

Distances from the center of the moon were computed for each star, and the sky flux next to that star plotted as a function of Lunar distance on a log-log plot.


PDF version of this plot:

Upper plot shows the magnitudes of the stars found in all four fields versus their correct magnitude. Lines is the 1:1 relation.

Lower plot shows the brightess of the sky versus the distance from the Lunar center in log(arcsec).

Line is a powerlaw fit, and has a slope of -0.6.

This is very curious. We were expecting -1.5 to -2 or so. What is this extra light?

The surface brightness is about 16 mag/sqarcsec in the inner most position at 1 degree from the moon, dropping to about 17.5 mag/sqarcsec at 2.5 degrees from the moon. This is a very slow drop in the brightness. The Moon was in the Milky Way plane, but the luminosity of the Milky Way is about 20-21 mag/square arcsec. So it’s not the Milky Way we are seeing.

For the present this is a cosmic puzzle!

UPDATE

Improved coordinates of the Moon were computed from the times of the exposures by Peter and a better plot results. Here it is:

Slope of the halo is now -0.9. That’s a bit better. More work to be done on this at the next full moon.



Coordinates of bad pixels

Bias and Flat fields Posted on Jan 08, 2012 12:24



2455933

Observing log Posted on Jan 07, 2012 10:31

Testing focus and testing the script MOONb.csv that should automatically set exposure time (software sets it automatically as it builds the script).

T 4 deg C, wind 5 m/s, RH 20%. Almost Full Moon. Slewing went well. Moon nicely centred.



Bad Pixels

Bias and Flat fields Posted on Jan 06, 2012 21:59

Our CCD has a number of bad pixels with lower than average sensitivity. They are a result of imperfect manufactoring, and most of them are collected in small groups. Their numbers and location are stable, and almost all of them are completely removed with even a mediocre flatfield.

But the most severe of the bad pixels, seem to have different strength in different master flatfields, and therefore they show up in diminished form in difference images between two master flatfields. The worst pixels may be 6-7% different from one master to the next, and therefore these pixels should be avoided. Luckily most of the bad pixels are close to the edge of the CCD. There is one semi-severe group in the central part of the CCD. I have seen examples of 1.7% variation in a couple of pixels from this group.

The figure is the twilight master flatfield in the IRCUT- filter from session JD 2455827, shown in power-scale to emphasize the small dark speck that are the bad pixels. The three most severe groups of bad pixels are circled in green, and the two second worst ones are circled in blue. These five groups of pixels should be avoided if possible – just to be on the safe side.



2455924: results from a good night

Post-Obs scattered-light rem. Posted on Jan 03, 2012 15:42

On the night JD245524 conditions were good: Moon was setting in clear dry skies and CoAdd mode was possible. We obtained images in all bands, and present the reduction to DS/BS ratios using the EFM method here.

Scaled superbias was subtracted from all images. No flat-fields were applied. Images that were poorly centred in the frame were removed. Images for which the estimated PSF exponent was above 2 and all negative values of DS intensity were removed. On the remaining 171 images DS/BS ratios were extracted using two patches on the DS disc – near the rim and further from the rim: (2/3 and 4/5 times radius away from the centre of the disc).

We plot the PSF exponent determined and the B,V,VE1,VE2 and IRCUT bands and DS/BS ratios below:

We see that the exponents of the PSF (upper frame) are quite stable for each filter, but that the VE2 filter has a smaller best value (i.e. PSF is ‘thicker’ in VE2).

For the two patches on the DS disc (close to rim middle panel; further from rim bottom panel) we see quite stable DS/BS ratio values but that the value does depend on where the DS patch is located. The V filter is represented only by data near fractional JD = 0.77. We see a large scatter there: this was when the Moon passed directly behind the MLO antenna, which interfered with the images and extraction of the photometry. The considerable scatter near 0.77 should therefore be discregarded – the data should be ignored at that point. This also removes all V data from futher consideration.

For the remaining data we look at stability of mean values.
VE1 mean: 0.0083 +/- SD 0.0006 Zm: 77.8037
VE2 mean: 0.0073 +/- SD 0.0005 Zm: 67.6726
B mean: 0.0109 +/- SD 0.0006 Zm: 127.6579
V omitted……………………………………….
IRCUT mean: 0.0081 +/- SD 0.0005 Zm: 108.8674

Here, SD is the standard deviation of the data for a band while Zm is the ratio of the listed mean and the listed SD divided by the sqrt of the number of points minus 1 – i.e. the ratio of the mean and the SD of the mean. The number of points in the different bands are 33, 25, 60, 11 and 51 for VE1, VE2, B, V and IRCUT respectively.

Now, these SNRs need to be MUCH bigger to reach our Science Goals of SNR=1000. If the errors are randomly distributed and independent we need 60 times more data-points in the best case (B).

We need to discuss why we have so few data :

a) There are only about 150 good data points (i.e. images). We actually had 210 observations. This number was reduced by the need for images witha well-centred lunar image: This requirement comes from needing similar images to give the EFM method a level basis of data to work on – if the Moon is sometimes in one side and sometims in the other side of the image frame the weight given to thedata during the location of the best fitting PSF varies and leads to systematic effects – i.e. bias. So by centering the Moon better we can have more data to work with.

b) By far the largest factor limiting the number of images is, at present, the faulty shutter: We may have taken more than 1000 images on that night, but most of them failed because the shutter stayed shut. This is despite some improvement received when the Mylar blanket was installed.

c) The amount of data is also dependent on the time interval available: it necessarily starts at sundown for these types of CoAdd images and stretches until the Moon sets – perhaps 2-3 hours at best. The present interval is only 1.2 hours long. If time was not wasted by fiddling with the dome control time could be saved.

d) As we observe in a cycle with equal numbers of images assigned to each filter we are limited by the length of the longest exposures. Change of this strategy could give us a larger number in some filters. Often it is the B filter that takes the longest to get by a factor of almost 10, compared to the broader and redder filters.

The above are the main reasons for the limited number of images we get on even a good night. In order theya re:

1) Wasted time due to shutter failing.

2) Limited observability of Moon in CoAdd mode.

3) Spending too much time on long exposures in B rather than getting many short ones in VE1 etc.

4) Wasted time due to dome and precise telescope pointing issues.

Of these there may be little to do presently about #1, and nothing to do about #2, but #3 should be considered and experimented with and #4 may be possible to do something about soon. I estimate that if 1 could be adressed we could have 5-10 times more data, while #3 and 4 could give us 50-100% more data if adressed.

Thus it seems we will likely not get the 60 times more data we estimate may be needed to reach SNR of 1000 by averaging short-exposure CoAdd mode images alone. This means we really need the Lund mode operating, and the SKEs. Perhaps that is where the investment of time is most valuable?

………….added later:

Here is the above plot with the ‘antenna interference’ points removed:

Note how well the VE1 and IRCUT (same filter, almost) line up.



MOON table for 2012

Observation Resources Posted on Jan 01, 2012 10:39

The table gives times and positions when the Moon is suitable for observing from MLO. The columns are: JD, SEM angle, illumination fraction, Moon altitude, Sun altitude and UTC date. An asterisk at the end indicates the special SEM angle near 42 degrees. PDF table – click on the icon:



New category

Observation Resources Posted on Jan 01, 2012 10:34

Let us collect various tables, plots and links under the new category so that it becomes easier to find them when observing sessions need to be planned.



Data inventory as of end of 2011

Observing log Posted on Dec 31, 2011 12:14

Here is a histogram showing the distribution of our observations across Sun-Earth-Moon angle (i.e. lunar phase on 0 to 180 degree interval). The huge spike near angle 50 is from the tau Tauri observations.



Link to Torben’s images from MLO

Showcase images and animations Posted on Dec 30, 2011 20:50

Torben took these pictures at MLO:

Click here.



Improved periodic bias plot

Bias and Flat fields Posted on Dec 30, 2011 20:21

I have improved the plot showing the periodic bias level for multiple areas on the CCD. I have added arbitrary offsets to separate the different datasets from each other, and I have included a small figure showing the location of the five 16×16 areas on the CCD.



The Earthshine Telescope at MLO

Showcase images and animations Posted on Dec 30, 2011 19:22



Wind histogram for MLO

Showcase images and animations Posted on Dec 30, 2011 19:19



Local map of MLO

Showcase images and animations Posted on Dec 30, 2011 19:15

Our telescope is in the Groundwinds building:



MLO on a map of Hawaii

Showcase images and animations Posted on Dec 30, 2011 19:11



Image of our shed at MLO

Showcase images and animations Posted on Dec 30, 2011 19:08



Telescope spectral report

Optical design Posted on Dec 30, 2011 09:38

This is Ahmad and Rodrigo’s report on the spectral properties of the telescope:



Laser versus LED

Post-Obs scattered-light rem. Posted on Dec 30, 2011 04:16

I looked at a bright white LED instead of a bright green laser.

Imaged it with the Canon 35 mm camera as before.

The result is exactly the same scattered light power law around the source.

Did this to check that nothing funky is going on with using a laser as a point source!

I added a scattering plane to the setup – a translucent tupperware lid was placed directly in front of the camera lens. Imaged the LED with and without the scattering plane. The lid scattered light very nicely (left) versus naked LED (right)!


The red curve shows the scattered light profile, the black curve the naked LED. Light is scattered on all scales in this experiment. Can one have a regime where scattering in the optics is dominant and elsewhere where atmospheric scattering is dominant? Not sure at all about this — my feeling is if the atmospheric scattering has a shallower power law index than the optics scattering, then it will always win. Could scattering be scale dependent?

The power law index for the naked LED is -2.6, for the scattered version it’s -1.5. This latter value is very similar to what we get for the moon in the 35 mm camera shooting through air! The tupperware lid simulates the Earth’s atmosphere, presumably by accident.

Scattering through camera optics, air and tupperware lids all produce power law profiles. Is that not interesting? Scattered light follows a power law under a wide range of conditions!



Lamps on the Antenna

Exploring the PSF Posted on Dec 29, 2011 15:19

As the night was fairly clear, I put the telescope on the lamps on the Antenna (Alt/Az: 21*34’/256*25′).

As the several lamps on the Antenna seem distinct it is at least not an ‘extremely foggy night’. I took V-band exposures at about 1 second.

Then I extracted the profile from the 25 coadded images. Only the quadrant below and to the right of the lower of the two sources above was used:

To about 20 pixels we see the actual lamp (i.e. the glass enclosure and filament). From about 30 pixels to short of 100 pixels we see the halo dropoff. The red line is a 1/r^(2.8) PSF.

This is contrary to what Chris found using the Moon and the occulting balcony! Unless the halo we see above is built up in the few hundred feet between the lamp and the telescope it must be due to the optics in the telescope. We cannot rule out that there was some fog, but the size of the exponent (2.8) indicates a ‘clear night’, I think – or is that circular thinking?

Anyway – it is not impossible that both optics and ‘air’ scatter in the same way.

Wonder if we can detect any examples where there is one halo from the optics and another from the atmosphere?

The above is not an occulting experiment.



Laser PSF in Canon 35 mm

Exploring the PSF Posted on Dec 29, 2011 12:54

As a follow up to the knife edge imaging of the moon with the Canon 35
mm camera, I took images of a laserpointer shining into the lens.
Laserpointer was set on with tape and shone into the camera lens from a
distance of about 5 meters. Camera was on a good tripod. The laser
pointer was aligned to be pointing into the lens by watching to see it
emerge from the back of the camera out the viewfinder and onto a wall
behind the camera. Eyes were kept well away from everything!
Exposure times of a few hundredths to a few thousandths of a second gave good halos.


Laser was pretty well centered. There is a weak secondary image off to lower right. The main image is saturated in the core out to about 10 pixels radius.

The laser has a strong core surrounded by a quite uniform intensity “platform”, after which the light falls away like a power law.

The radial profile of the laser (blue) compared to the moon (green) is shown below.


The laser pointer’s profile falls off faster than the moon — but not very much faster.

If we assume that the laser is a good point source, then there is scattering in the lens/CCD combination in the camera which is seen as a power law at large R.

Slope of the lunar halo is about -2.0, whereas the slope of the laser pointer halo is about -2.6. Getting close to the diffraction limit of -3!

The lunar profile is then interpreted as a combination of both scattering in the camera and scattering in the atmosphere. Its slope is shallower — so atmospheric scattering totally dominates at large R (scale both powerlaw falloffs to the same flux at log(R)=2 or 100 pixels to see this — atmospheric scatter would then dominate internal lens scatter by about an order of magnitude out at R=1000). Not sure what’s happening close in, as the laser pointer has a strong uniform intensity “platform” of light around the core — easily seen by eye just by pointing it at a piece of paper. A point like source of light, like a street lamp seen at a large distance might be a way around this problem.

I also imaged the laser pointer projected onto a white wall — long exposure times of many seconds — and got the same result — similar core and platform and a halo with the same slope.

All very interesting !



JD2455924

Observing log Posted on Dec 29, 2011 09:23

At sundown T=8.8 wind dropping, 10-14 m/s, RH about 20%. 2 hours later outside T was 5 degeres C while dome T was still 7.

Shutter failing often – and ‘open while reading out’ occurring. Again we see sets of images failing. Of course not related to the physical filter itself – but then why e.g. all VE1 images failing while adjacent images OK?

No clouds. halo starting to reach DS edge towards sky.

Noted that an antenna gets in the way of the Moon during some Moonsets.



Knife edge due to a house

Post-Obs scattered-light rem. Posted on Dec 28, 2011 14:48

Observed the full moon with a Canon 35 mm camera, eclipsed by a “sharp edge” (the front balcony of our house) and then moved the camera so that the Moon appeared from behind the edge and imaged it again. The moon was about 45 degrees above the horizon on a very clear sky. Sector was chosen to avoid the gum trees seen lower left smiley

Image above is the UNECLIPSED moon close to but detached from the edge of the house. The radial profile was measured in the inner region of the sector shown, with the outer region defining the background sky for subtraction.

Image above shows the light from the moon, now placed just behind the roof’s edge. The radial profile was measured in a similar manner as before.

The idea is to test if the scattered light far from the moon is the same in both cases, i.e. is mainly due to the atmosphere.

The result is that it does indeed seem to be due to the atmosphere — the light falls off as a log-log power law exactly the same and, for two images with the same exposure time, the falloff has the same surface brightness on the sky.

The falloff at large R (> log(R)=2.1, or about 125 pixels (1 pixel is about 1 arcminute) is exactly the same in the two cases, going out to more than 1000 arcmin (15 degrees). There is some funny behaviour near the knife edge, with some missing light in the eclipsed profile relative to the uneclipsed one, but it was very difficult to estimate where the center of the moon was behind the knife edge — since the moon is eclipsed — and the behaviour is quite sensitive to the moon’s position. Tried to figure out where the moon was using stars in the background, but the field distortions in the 35mm lens were too severe to model with a handful of stars.

If we try this again it would be better to leave the camera fixed and wait for the
Earth to turn and eclipse the moon, rather than moving the camera. A sequence of images as the moon approaches the knife edge might make modeling where the moon ends up behind it easier.



JD2455923

Observing log Posted on Dec 28, 2011 05:47

Catching the setting crescent Moon. T=8.4 C in the dome. Got some nice Moon images – almost no scattered light!

Then observed Uranus, but images were either saturated or misfired. T=4.7 C now.

Now M41. Getting very few frames. T=4 C and the wind is strong at gusting to 16/17 m/s.



Colour filters

Optical design Posted on Dec 27, 2011 14:52

The filter transmission as measured by Rodrigo for the B, V, VE1 and VE2 filter up to 800 nm and the IRCUT filter for a much wider range as given by the manufacturer.



M41 and photometric calibration of filters

Post-Obs scattered-light rem. Posted on Dec 27, 2011 03:50

UPDATE: The measurements on this post have now been superseded but more reliable data from NGC6633. See blog entry for July 1st, 2012!

On 23/12/2011 Peter and Chris observed M41, an open cluster in the Milky Way plane at 06:46, -20:40. It was a very clear night and we got good images (although off center) in all bands. Image below is a centered B band frame.


Stars identified from the WEBDA database for this cluster are shown below:


Data for these stars are here:

http://www.univie.ac.at/webda/cgi-bin/frame_list.cgi?ngc2287

The transformation to standard V and B are rather good. There is a small
colour dependence in the V filter (-0.08 mag/mag) and quite a large one in the B filter (+0.28 mag/mag). The scatter is quite low in the V transformation — 0.02 mag, and a little larger in B — 0.05 mag.

Transformations

Instrumental magnitudes convert to true magnitudes as follows:

* compute instrumental magnitudes
V0 = -2.5*log10(ADU/sec)
B0 = -2.5*log10(ADU/sec)

* instrumental colour
(B-V)_0 = B0-V0

* transform instrumental colour to Johnson B-V
B-V = -1.056293 + 1.529783*(B-V)_0

* transformation to Johnson V (scatter is a very small 0.02 mag)
V = 15.15 + V0 – 0.08*(B-V)

* transformation to Johnson B (scatter is quite large at 0.05 mag)
B = 14.46 + B0 + 0.26*(B-V)

These are for a (rather small) 2.5 pixel aperture – designed to avoid crowding by nearby stars. The correction for the aperture size is ~0.1 mag brighter.

There are no standard magnitudes for VE1, VE2 or IRCUT. I have searched for
similarities with V, B or I, since these are the bands we have external magnitudes for.

The following definition of VE1 magnitude gives a very close 1:1 match between V and VE1:

VE1 = 16.34 – 2.5*log10(ADUs/sec) + 0.06*(B-V) (scatter 0.04 mag)

so VE1 is functionally quite similar to V.

On the other hand, VE2 is very similar to I. With this definition for the VE2 magnitude:

VE2 = 14.22 – 2.5*log10(ADUs/sec)

one gets very close to a 1:1 relation with I, with a scatter of only 0.03 mag. No colour term either — so VE2 this is an excellent match to I.

We didn’t get any good frames in the IRCUT filter. We’ll have to try that some other time!

Summary : Our V and B are much like their standard Johnson counterparts.
Our VE1 is functionally more or less the same as Johnson V, and VE2 is very close to Cousins I band. The latter seems a bit odd. I haven’t seen the filter curves, so they’ll be fascinating in this context. What is the IRCUT filter going to show? More to follow.



One Clear Night Returns

Post-Obs scattered-light rem. Posted on Dec 25, 2011 10:24

Applying also the logarithmic BBSO method to the rainbow images, we get the following results:

We see much more scatter in the early part of the sequence, compared to the ordinary BBSO method. This is both the part where we had the lowest fluxes (shortest exposure times) and (we presume) a proximity to the ‘rainbow phase’ – so are we sure if we see scatter due to rainbowing or just low exposures?

As earlier we expect that scatter rises when erroneous additive levels are removed – but do we have a way to decide if ‘too much’ has been removed? The log method is able to ‘remove too much’ while the linear method is more likely to underestimate how much is to be removed.

The larger total/DS ratios in the early part indicate lower Earthshine intensity. Remember that the rainbow is supposed to be brighter than ‘non-rainbow’ Eartshine.

It is probably realistic to assume that problems have arisen in the early part of the sequence due to low exposure times, not because of low actual fluxes.

Shall we consider co-adding frames into bins – every connected set, for instance?



One Clear Night

Post-Obs scattered-light rem. Posted on Dec 23, 2011 22:36

The Moon was observed going through the ‘Rainbow angle’ on Dec 21 2011. Observation started at Moonrise and stopped at sunrise a few hours later. All filters were in use and good exposures were obtained with minimal dropouts due to shutter stick.

Bias was subtracted and the scattered light removed using the BBSO method. The total counts for a 23×23 pixel box on the DS was extracted in each image as was the image total flux. The ratio of the total flux to DS box is plotted in the figures above in the upper panels. DS box and total are plotted below. To the left are the results when scattered light has been removed; to the right when it hasn’t.

The step up in total flux near the start is due to adjustment of the exposure time used.

The increase in scatter that appears to occur when scattered light is removed may be quite realistic as the total/DS ratio will be more stable when the DS total contains an (erroneous) contribution from as-yet unremoved bias or sky levels. When these contributions are removed the true scatter of the DS data becomes evident. Note that, e.g. the cyan points lie on a straighter sequence in the corrected image than in the uncorrected implying variability in the scattered light, which was quite realistic for that night.

The observed downward trend in the total/DS ratio is consistent with a gradual increase in earthshine intensity.

The sequence of observations started as the Rainbow angle had already been met. The Moon moved from red to blue side of rainbow.

We next add the expected evolution of the Earthshine intensity for this night. We use the synthetic model and generate models for the ‘cloud-free’ case – i.e. The Earth has no clouds but does have surfaces with different albedo: land, ice, water – and the ‘uniform’ case whjere Earth is just a Lambert sphere with a phase function.

The solid line shows the evolution of Earthshine intensity for a Lambert sphere; the dashed line shows for a cloud-free Earth. The two vertical dotted lines delineate the observing window. The large dip correspnds to the (cloud-free) Pacific ocean.

We see that Earthshine is predicted to increase during the observing session if the phase-function rules – but decrease if the surface albedo rules.

At the observing time the Moon was at -19 degrees declination so the sunglint was also southernly (Sun was near -23 degrees in decl). The sunglint position does not provide all the earthshine but is at the centre of the area that contributes. For the observing time we see the sunglint moving off South America and into the South-Eastern Pacific:

We can use the modelling code to make a map of the areas that contributed to the earthshine at the moment of observation.


This map corresponds to UTC 1 hour on that day. As the sunglint moves further West the dark Pacific comes to dominate the average albedo.

It is quite possible thatthe Pacific was not cloud free at these times and therefore helped to increase the average albedo. From http://goes.gsfc.nasa.gov/goeswest/fulldisk/vis/ we can download full disc images from the GOES West satellite at 3 hour intervals.

USA is at upper right. South-Eastern apcific at lower right – Sun is just rising on Hawaii ner athe middle of the image. While therea re some large swirls of clouds in the South Pacific it does not seem to be enough to cause the growth in average albedo observed. A quantiative calculation would have to be performed.

Summary: We have observed a decrease in the total(DS ratio which implies a rise in the DS intensity – i.e. increasing Earthshine. We have shown that Earth’s phase function was growing at the time and thus contributing more light to the DS of the Moon, but we see that the rotation of the Earth is bringing darker areas into view thus counteracting the phase-function tendency.



ND0.9 filters do nothing

Optical design Posted on Dec 21, 2011 20:43

On the lamp in the dome we can test the densities of the various ND filters by calculating and comparing the observed fluxes with and without the filters. So far I have tested most of the ND0.9 filters. They appear to be blanks – or have not rotated into position.

Filter Without ND With ND0.9
B 3511 3512
V 26500 26500
VE1 210700 209700
VE2 268200 273000
IRCUT 196050 no data

The numbers are in counts/second and are based on the measured mean counts in the various frames and the exposure time MEASURED and reported in the FITS header.



When fog falls yea verily upon the mountain

Observing log Posted on Dec 21, 2011 15:26

A skycam image showing where there is fog on the mountain. The navigation beacons are hazed out (in the NNW) so there’s fog at ground level. Close up please!



Noise model – Poisson versus Gaussian noise estimator

Post-Obs scattered-light rem. Posted on Dec 14, 2011 14:14

The noise model is improving rapidly. Below is a simulation of a lunar image (right) from a synthetic moon provided by Peter; and the corresponding real image (left).


Images are equalised histograms with the same scaling of log(flux).

See www.astro.utu.fi/~cflynn/logbetter.jpg

Until now I had been using Gaussian sampling for the noise, but this fails badly when only a few photons are expected in a given pixel — while this is not an issue for most of the moon (ES>10 counts/pixel in typical exposures, and BS is of course very bright) — it matters a lot when modeling the halo far from the moon.

Replaced the Gaussian noise model with a combination of a Poisson model for low count rates and Gaussian for high count rates (>25 counts per pixel). The Poisson code is both very slow and inaccurate at high count rates, so a combined model was necessary.

The Poisson code comes from here:

http://www.netlib.org/random/zufall.f
Credit : W.P. Petersen, IPS, ETH Zuerich.
(I should check it’s public code).

More on the method to get a Poisson sampled count in regions where the expected number of photons is < 1 to follow later!

MORE stuff on this:

There is a timing hit doing Poisson noise, it’s quite
long winded.

For 30 summed images the total run time is about 40 seconds
on my laptop with Poisson noise. for 30 images with Gaussian
noise, the run time is about 2 seconds. The FFTs only take about
1 second of this. So there is presently a huge performance hit. I think this can be greatly improved.

Important note: 30 summed images using Gaussian noise look
very similar to 30 summed images using Poisson. This is
because 30 images of Gaussian sampling simulate Poisson
sampling rather well. It’s very different though when talking
about a SINGLE frame:

ABOVE : GAUSSIAN noise (left) versus POISSON (right): for a simulated lunar image in a single 30 ms frame. The very low light levels are much better modeled in the Poisson frame on the right, where the flux level drops to a few photons per pixel — and very much better when the level drops well below a photon per pixel.

Gaussian (right) versus Poisson noise (left) — but for 30 coadded 30 ms images. The differences cannot now be seen by eye, since mathematically the samplings are now more or less the same thing.

I have now written my own subroutine to do Poisson noise, and it seems to reproduce very well the results of the code acquired from http://www.netlib.org/random/. It’s slightly slower though; thought I could improve on the run speed easily, but I was wrong!

Need to still check for small systematic differences between expected mean number of photons and the numbers actually returned by the Poisson routine, since we need to worry about this at the 0.1 percent level. More on that later.



MLORWP

Real World Problems Posted on Dec 14, 2011 11:32

We can go far by analysing various synthetic images – but real-world images are different and come with problems we have to solve and adapt to. I suggest we use this blog category to add things we have found. Now and then I will collect submitted items onto a Master List Of Real World Problems (MLORWP!).

This is the MLORWP I can think of right now:

1) Real images are not centred in the image frame – take this into account when using synthetic images for analysis of some sort – the synthetic images are all right in the middle of the frame.

2) Real images have slight variations in scale and rotation due to slippage etc in the hardware. While we can always measure what the problem is we need to make sure that e.g. synthetic images are generated for the same conditions (e.g. image rotation – the CCD is mor eor less free to rotate!).

3) Use of FFT methods to convolve images carry some consequences – one Chris put his finger on is the centering of objects by the very act of folding: We need to make sure that when we model real images the resulting synthetic image is offset by the right amount. Henriette’s Python project with Kalle Åström at Lund U could come in handy here.

4) The noise in real images is probably higher (never lower) than that given by Poisson’s distribution.



Wild Slew bug found?

Control Software Posted on Dec 13, 2011 15:48

We may have located a significant bug in the control software: A command was being sent during slewing, which is not allowed. It was sent due to some timeouts in the software.

Bug was found by Ingemar Lundström after some information from Howard Hedlund of Astro-Physics in Illinois.

Multiple meridian flips were tested after fixing the software – both from Engineering Mode and from scripts. No problems.

It is still an issue that Kill Switches should not be reached – we fear that some loss of calibration takes place then too, although the amount of de-calibration may not be total.



BBSO compared 4 ways

Post-Obs scattered-light rem. Posted on Dec 13, 2011 13:25

We remove scattered light from synthetic images and test the BBSO method in 4 ways.The synthetic images were generated with alfa=1.6. 1 synthetic image was used.

The BBSO method was first applied in the standard way – i.e. linear least squares fit to all pixels in 5 degree cones on the sky off the DS disc. Then we fitted to log-intensity, converted the correction into linear values and subtracted from the linear image. The two methods were applied to a patch on the DS disc 2/3rds and 4/5ths from disc center – i.e. close to the rim and not so close to the rim.The figure shows the percentage difference between the cleaned up image and the known value, as a function of the day of month. New Moon is near day 13. The absolute difference is shown but all corrected intensities were larger than the true intensities.Red symbols show the results for the standard method (i.e. linear regression), blue is for the log-method. Crosses show the results at 4/5ths from disc center and boxes show for 2/3rds from disc centre.We see that results with errors below 10% are available for only about 5 days. We see that the log-method is somewhat better than the linear method, and that best results are found close to the rim.We also see an asymmetry – as thelunar phase passed Half Moon the enalysis switched to the other side of the lunar disc. Apparently a 50% improvement in the derived intensity can be had by using the log-intensity method, but mainly on one side of the Moon. The reason is unclear. As images were generated ideally – i.e. without libration and without distance variations a fixed coordinate for the patches in which the corrected intensity was evaluated always accesses the same pixels in the synethtic image – except when we switch to the other side of the disc after New Moon.



JD2455905

Observing log Posted on Dec 12, 2011 11:49

Dec 10 2011 Lunar Eclipse.Observed with Chris. Fairly clear skies. Temperature was sub 30 degrees F so plenty of shutter stick. Of 1500 images about 2-300 were useable.Made animated GIF of the eclipse.Image position wuite stable, but not perfect – clearly the fix by Ingemar which now enables interpolation helped a lot, but table may still be to coarse – otherwise slight ‘jitter’ in position may be due to ssome mount issue.



S/N image

Error budget Posted on Dec 07, 2011 16:20

From night JD2455864 V-filter moon images I have selected images with similar count levels in the bright part of the Moon. This resulted in 23 images. These I have aligned and coadded (mean-half-median method) thus obtaining a coadded object image O, and a standard deviation image, delO. From these two images (and images with estimates of B, delB, F, delF) I constructed a S/N image calculated pixel by pixel. I found the DS to have S/N ~2-3 and the BS to have S/N ~30-40. The S/N image looks like this

While the DS S/N value might be realistic, the BS S/N value is much lower than expected. This is likely due to the fact that co-add mode necessarily is observed close to a rising or setting moon. On this particular night the Moon was setting, and the 23 selected images are obtained with a 30 minutes interval – in which time the Moon went from an altitude of 22 degrees to 17 degrees. The typical BS count level falls in this period from 27900 to 26400. A decrease of about 5.4%.
Next I tried scaling the images after alignment before coadding. I have scaled them using an 11×11 area in Mare Crisium as reference. This area had a mean in the unscaled coadded image of 10300, and I have scaled so that each image has the value 10500 in that area.
scale_factor = 10500.0/AVG(im[360:371,230:241])
This gave a DS S/N ~1.5 and BS S/N ~100+. The BS S/N seems pretty stable of the exact scaling factor, but the DS S/N increases dramatically (S/N ~12) if I use for instance 15000 instead of 10500. I simply don’t think we can count on the S/N values calculated per pixel basis after scaling.

Conclusions and thoughts: If we scale the moon images to counter the setting/rising moon we seem to be able to achieve a S/N on the bright side similar to the poisson noise. It is much more troublesome to estimate the S/N on the dark side. The S/N on the dark side is very sensitive to the method of scaling and the scaling factor. But perhaps a S/N of 2-3 (as determined without scaling using only images selected to have at least somewhat similar bright side count levels) is a decent estimate.



Working method for albedo measurement

Post-Obs scattered-light rem. Posted on Dec 05, 2011 01:17

My last post was on techniques to measure the ES using artificial images as input.
Three boxes are defined on the image which are mainly sensitive to alpha, ES and threshold. Box 1 (off the BS limb) turned out to be completely dependent on alpha alone — and one can get a very accurate alpha measurement from just measuring the photons in the box (see plot below). The scatter in alpha is only 0.01 in tests of the method based on semi-realistic synthetic images made by coadding 10 x 30ms images together and properly including Poisson noise. Error in the bias level or registration of the moon and overall normalisation of the image is NOT yet included. Having determined alpha, it takes a little care to then compute the amount of light due to BS scatter in the box on the ES limb, and subtract it. Doing this, one then gets the actual ES level in the box. This correlates very well with input albedo and can be empirically turned into an output albedo (see plot below), with a scatter of 1% — our target accuracy. So it looks to me as if 1% is achievable! Yay! However, there are quite a few systematic effects not yet included here, all of which will shift us hither and thither. So that’s yet to be done!


Basically I think this method is the same as Peter and Jacob are pursuing at the moment — simply trying to fit away the unwanted scattered light in the ES box — except that I do it “empirically” by building a calibration from a range of synthetic data with different alphas and albedos — whereas they are working from a single frame — which is what we will actually have to do in practice.

Anyway, all of the above gives considerable oomph to the idea that the glass is 90 percent full!



New analysis method

Post-Obs scattered-light rem. Posted on Dec 03, 2011 10:36

I’ve developed an kind of ’empirical’ method for measuring alpha and ES on a frame, I am working from 30 co-added 30 ms exposures of a fake moon, so the S/N is rather good.

Here’s how it works:

Three boxes are defined on the lunar image as shown below


Image shows an example synthetic lunar image (30 co-added 30ms exposures)
for some value of alpha and earth albedo (threshold is identically zero in
the tests done to date).

Box 1 : (far left) designed to be mainly sensitive to alpha. (11×21 pixels)

Box 2: (middle) mainly sensitive to ES level on DS. (26×21 pixels)

Box 3: mainly sensitive to threshold level — i.e. the bias (51×21 pixels) (not analysed here).

A set of synthetic lunar images is created for a range of values of albedo and alpha, chosen randomly in the range albedo = [0.0, 0.6] and alpha = [1.6,2.0].

For each alpha and albedo pair, the total photons are measured in boxes 1 and 2.
I then ’empirically’ calibrate relations between the photon counts in the boxes and alpha and albedo.


Above is the plot for the counts/pixel in Box 1 — it correlates extremely well with alpha and has no measurable dependence on albedo. The lower plot implies alpha can be measured with an accuracy of ~0.01 from the counts in in box 1. (Note well that alpha here is the power to which the PSF is raised — it is already a power law like r^-1.6, and it is then raised additionally to power alpha).


The counts/pixel in box 2, selected to be mainly albedo sensitive, are indeed very well correlated with albedo. There is a very small dependence on alpha (lower panel). Taking into account both dependencies results in recovering the albedo with an accuracy of about 0.03 — compared to a typical albedo of 0.3, for an accuracy of 10%. We require 1% accuracy! Not sure what’s driving the 10% figure at this stage. Is it merely Poisson?

We can improve things by using more of the frame — the boxes were selected pretty arbitrarily just to get the ball rolling. The modeling is also pretty ideal at this stage as the error in the threshold is not yet included, and one has to have a good synthetic moon image to get started — any error in the synthetic moon (i.e. before scattered light and Poisson sampling is included) will bias the method.

Update: method now much improved — fitting functions work much better using log of the counts in the boxes, rather than the counts in the boxes (as I should have realised — we are dealing with a power law in the scattering!). This leads to great improvements, as discussed in the blog entry above.



The Bias acts up

Bias and Flat fields Posted on Dec 02, 2011 16:30

We have seen variations in the bias level (and dark current) that we do not understand. We went back to all 6000 dark frames that we have accumulated since the days in Lund until now. We plot the median bias level against date, and we plot the median bias level against chip temperature.

The top panel shows that usually the bias level is near 397 or so, but that excursions have occurred – the plot even omits one point showing a frame with median bias 1028. Near day 270 (JD 2455745 = July 2nd 2011) a very strange event occurs. In the bottom panel we see that that event is unrelated to temperature issues.

The red line, is a fit to the temperature-dependent data – the slope is about 0.067 counts per degrees F, or 0.037 counts/degree C.

The present problems with the camera, showing bias levels near 410 does not seem to strange any more, but we are unshure if the dark current has previously been as strange as it is now.

We note that during the observation of asteroid YU55 we saw what we thought was a strong flat field pattern – but it could not have been that since the dust spots are absent.

Can a flat-field like pattern be generated by high dark current?



Simple Errorbudget Co-add

Error budget Posted on Dec 01, 2011 16:06

I have aligned and co-added the V-filter moon images from night JD2455864. 16 frames were rejected due to poor correlation when aligning (turned out they were overexposed on the bright limb). This left me with a total of 110 frames.
I have selected an 11×11 pixels area on both the bright and the dark side. The selected areas are shown in the figure below.

The errorbudget formula used is for co-add mode, so exposure time cancels out, and in this first simple version, scattered light is ignored.

For the earthshine I find S/N = 1.7, and for the moonlight I find S/N 128.2.

I have experimented with two different approaches to improve the S/N of the dark side. One way is to increase the number of mean counts, O, in the selected area. For night 864 V-filter O was in the range 398-405. Unfortunately the bright limb was overexposed in the images with higher values than this.
The other approach is to have more images from the same night in the same filter. I have investigated N from 1-1000.
Finding the right exposure time, where the bright limb is well-exposed near saturation, is very important. We can achieve a S/N higher than 3, if we have about 400 frames and a mean dark side count level of 405.
Hopefully we can achieve an even better S/N closer to the new moon. To be continued…



The Three Dark Sides

Post-Obs scattered-light rem. Posted on Nov 30, 2011 17:35

Using synthetic images where the DS intensity is known, we have generated ‘fake’ observed images by convolving with a PSF and adding noise (Chris’s syntheticmoon.f used; alpha=1.8). We then forward modelled these images using the EFM method (i.e. using the ‘observed’ image BS as the source and allowing that to scatter in model images until the model images matched the input image).

Subtracting the model images left us with the DS corrected (the BS goes away, per construction). We extracted DS intensities only (adding the BS is a separate chapter of complexities!). We extracted DS intensities from the corrected image, from the synthetic observed image, and from the known input ideal image. We plot these images as a fucntion of the day number during one month.

The black symbols show the DS intensity in a 15×15 patch at 2/3 of the radius for the synthetic observed images (i.e. the ones that suffer from realistic scattered light from the BS); the blue symbols show the known intensity from the ideal pre-scattering images; the red symbols are from the EFM-corrected images. The period of New Moon is excluded because our method fails there (there is not enough crescent to make a good model of the halo from). Near Full Moon is inaccessible since the DS is so small then and it becomes difficult to extract the DS intensity (nowhere to put the patch!).

Annotations in the upper panel show the choices of various settings of the analysis code.

Bottom panel shows the difference between true and corrected DS intensity expressed as percent of the true value. In the upper panel points inside the two sets of dotted lines have been selected for analysis in the lower panel – the mean absolute error is printed.

We should next apply the BBSO method to these images.

We see that this method also reaches the 1% error level but that the number of days during which this level is attainable is smaller than with the FME method.



5 Filters, One Night returns

Post-Obs scattered-light rem. Posted on Nov 24, 2011 11:26

For the night of 2455864 we have tested the effect of using a 5×5 patch on the sky vs an 11×11 patch. This is the patch in which the average is taken and through which average the model curve is forced to pass. We had noted that a 5×5 patch was small compared to some of the wiggles in the sky background so we tested the use of 11×11 patches.

We also test the effect of using 64-bit rather than 32-bit code – we do this with ‘no change of compilers or compiler flags’.

Finally we tested the effect of taking smaller steps in alfa. The figure summarizes these three experiments:
The difference in detected alfa was small when we went from 5×5 to 11×11 patches. The average change was near 0, the standard deviation apeared to be about 1 alfa step (i.e. 0.001), the max and min of the change were -0.6 and 0.4 but only 2% of the 384 images suffered a change in alfa larger than 0.01 in absolute value (i.e. outside the plotting range). We conclude that there is a tiny indication of a low-bias in determined alfa when using 5×5 patches, so we will use 11×11 from, now on.

For the 32-vs-64 bit analysis we found that for step size in alfa of 0.001 the change in going from 32 to 64 bit was most often 0. A small fraction of images had a +/- change of 0.001 in alfa – i.e. the step size.

The effect of smaller steps was negligible.

We next looked at changes in the determined DS intensity.

For each filter and for each of the three modes of the EFM code run on the CRAY we extracted the median DS intensity in a box 11×11 pixels large positioned 2/3 of the radius on the dark side. We also extracted the BS from an 11×11 box positioned at the center of gravity of the image in the observed image.

Run1 was the test for the effect of using a 5×5 vs 11×11 patch on the sky and the BS to force the model through.

Run2 was the test for effect of changing from 32 to 64 bit code.

Run3 was the test of the effect of using smaller step sizes in alfa.

We found that:

For all filters there is a 6 to 7% effect on the BS/DS ratio of going from a 5×5 to 11×11 patch on the sky when fitting the model to the observed image.

The effect of changing from 32 to 64 bit code was negligible, as above – the largest change in BS/DS ratio was 0.02% in the VE1 and VE2 filters; the others were at the 0.007-0.009% level.

The effect of smaller step sizes when searching for the optimal alfa was a little larger – in the range from 0.19 – 0.9 %; largest in the VE2 filter.

We conclude that careful choice of the way in which the model is fitted to the sky and the BS is warranted.



5 Filters, One Night

Post-Obs scattered-light rem. Posted on Nov 22, 2011 16:28

We have used the ’empirical forward model’ method to reduce 384 images from night 2455864.

Using the DMI CRAY we stepped through 1000 values of alfa from 1 to 2 for each image and required the best possible fit on the sky part of each image.

We plot the best value of alfa found for each filter againts the time of observation:


We see that the values for alfa are narrowly grouped for most filters, except for the VE1 filter which jumps between two values. Unexpectedly we see that the blue filters have high alfas (i.e. narrow halos) while the reddest filter (VE2) has the smallest value of alfa (i.e. the broadest halo). This is opposite of what we might expect from physics, and the code plotting these data must be reviewed.

The scatter of alfa for e.g. the IRCUT filter is 0.0036 which is 0.2 %. Since we had 98 IRCUT values the standard deviation of the mean alfa is about 0.00036 which is about 3 times larger than the required accuarcy estimated on this page below, to achieve a DS accuracy of 0.1%.

Visual inspection of the fits show that improvements are possible, so more experiments with parameter settings is called for. The above were results from fits of the absolute residuals. Next we try fitting the relative residuals, i.e.

(obs – model)/obs

instead of

(obs – model).

We have now tested that – and it was not a good idea: fits got worse.



No significant dark current

Bias and Flat fields Posted on Nov 20, 2011 14:05

I have plotted the mean dark value as function of exposure time from the dark frames obtained night JD2455883. The range of exposure times is 10-200 seconds and thus covers all higher exposure times we might be interested in. No dark current is observed, only the usual scatter due to the 20 minutes period. The period was clearly seen in a plot from the same data of dark count as function of time since first frame.

I think it is safe to say that we no longer have to make dark frames – only bias frames. This will save observing time, and therefore there should be time to ALWAYS obtain a bias frame before and after each science frame or flatfield. This will improve the scaling of the superbias.

The horizontal line in the plot is the mean value of the superbias. It can be seen that it is not in the middle of the scattered dark values. I have seen this before, as well as the opposite with dark values generally being higher than the mean of the supebias. However, in both cases values are within plus/minus 0.5 ADU and the scaling of the superbias means it is not a problem.



JD2455886

Observing log Posted on Nov 20, 2011 13:53

Clear, but 80% RH and T=5.6 C in dome.

Halo enormous – no sign of DS despite phase. Exposures of the BS only in the 1000’s – must update exposure time – heavy clouds??

Shutter clearly open during readout – dragging downwards.

Also sticking so that saturation in stripe downwards happens – i.e. shutter started to close, then stuck open for a long time. T now at 5.2 C.

T now at 4.9 C.

Experimenting with SETIMAGEREF and MOVEMOONTOREF because it seems the mount calibratio I did earlier tonight was not quite right. Noticing that when Moon is half out of the image frame downwards I get a reflecting coming in from the upper side of the frame – like was saw during the tau Tauri occultations in July – then it was left to right. So, something in the optical train is causing reflections onto the CCD when the source is not well centered.

More and more clouds now. T now 4.8 C.



Testing BBSO method

Post-Obs scattered-light rem. Posted on Nov 18, 2011 16:50

Henriette and I ran the BBSO linear extrapolation method for removing scattered light with two settings of a parameter that is arbitrarily set. We got output for many files with these two settings and can therefore compare the two results for a single image and measure the difference.

On the image below we see the relative differences between two pairs of two such processed images.

On the left-hand side of the upper image the white cones correspond to percentage differences in the 10-30% range. They gray or black areas are much lower.

The second image is doing better with most differences in the single-% range.

The images processed here are single exposures in CoAdd mode – so there is lots of noise on the DS and the sky. Performance of the BBSO method will probably improve with higher SNR.

We changed the processing parameter (cone width) from 8 degrees to 6 degrees. BBSO uses 5 degrees (in the JGR Qiu et al paper). We can now look at all images where we have these two processing results and make some summary statistics.

Below are two histograms of just that for 383 pairs of BBSO-method corrected images- the first shows the histogram of mean difference values (they are in %). The other is the histogram of the median difference value. Both measures of performance are centred over 0% – the mean difference is a broader distribution than the median – by about a factor of 10 (FWHM mean is 2% while FWHM median is 0.2%). The differences were calculated for all pixels (DS sky and DS itself, but excluding BS and BS sky) corrected by the BBSO method evaluated with the two settings as described above.

The result suggests that correcting the DS for scattered light using the BBSO method and extracting a subregion mean will give unbiased results with a few % scatter, while extracting the median of some corrected subregion will give the answer to a few tenths of a percent.



JD2455883 darks

Observing log Posted on Nov 18, 2011 16:02

I have obtained 100 dark frames with random exposure times in the interval 10-200 sec. This will allow me to determine with certainty if there is any substantial dark current.

The telescope performed as it should and all images were saved without trouble.



Photos from MLO

Showcase images and animations Posted on Nov 18, 2011 15:08

The Earthshine observatory on Mauna Loa. Housed in the previous Groundwinds dome (left), our telescope (right), and Torbjörn and Henriette who installed the telescope and the system with Dave Taylor.



Temperature near the shutters

Shutters Posted on Nov 18, 2011 11:23

A webcam on the PXI now points to a thermometer display showing the tmperature somewhere on the telescope and somewhere in the dome. When the heaters are turned on the telescope readout cycles between about 16 degrees C and more than 60 C.

We presume that the temperature is telling us something about an area near one of the two shutters. Since the shutter shows the sticking behaviour through these temperature cycles the sticking problem is either unrelated to the temperature of the shutter or the temperature we can see is not from an area close to the shutter inside the telescope.

We think that the sticking shutter is the one inside the telescope because the system that reads the shutter time also indicates failure whenever a picture is ‘bias like’, which would not be the case if the failing shutter was the front barrel one.



Wet

Observing log Posted on Nov 18, 2011 11:06

Extremely wet and cloudy these days! Wanted to observe Moon.



Upgraded software

Control Software Posted on Nov 17, 2011 02:16

Interpolation enabled. GOTOMOON centers better now. SETIMAGEREF tested.



Square aperture for CCD

Mechanical design Posted on Nov 15, 2011 17:49

It has become evident that the square aperture in front of the CCD chip can shift – or rather, that the CCD camera can shift relative to the mask. The mask is present to help the frame-transfer procedure by shading the part of the frame that holds the transferred image during readout. If the CCD was not half-shielded the ‘smearing effect’ would occur due to illumination during readout. The mask is a set of adjustable blades, apparently that are fixed wrt the telescope body. The CCD itself is held to the body by a thread and the correct position is indicated by two steel pins that should touch, visible from the outside. It has occurred that the CCD has ‘unwound’ and this caused ‘dark corners’ to appear in the images. The camera had indeed ‘unwound’ a bit and was rotated back into proper position.

Following that incident a ‘black bar’ has appeared in the side of the image – perhaps due to slippage of the camera wrt the blades, or the blades’ motion.

The issue is further discussed here.



Historic material on Earthshine

Links to sites and software Posted on Nov 15, 2011 17:33

Leonardo da Vinci’s notebooks: http://www.fromoldbooks.org/Richter-NotebooksOfLeonardo/section-14/item-902.html

A reference to Humboldt in 1774: http://www.britastro.org/journal/pdf/120-4forum.pdf also, Kepler and Galileo.



Diffuser test

Bias and Flat fields Posted on Nov 12, 2011 13:08

See this entry for a test of the Diffuser

http://iloapp.thejll.com/blog/earthshine?Home&post=83



JD2455877

Observing log Posted on Nov 12, 2011 10:19

Testing the Diffuser. it sits in the ND filter wheel – that is, in the collimated beam. Being situated there it removes any spatial information originating in the flat screen itself, the front objective and the first lens of the collimating system. Flat fields taken this way have a central brightening that looks spherical or parabolic in shape.

Perhaps, if that darkening could be understood and subtracted by some fitting procedure, we can use the diffuser to get good high spatial frequency information about the FF? It seems to remove the huge gradients that otherwise characterize the FFs we have been getting from sky, dome and lamp in the past.

The difference between darkest corner and centre is about 10%. What is the origin of the circular feature, and what does its position off-center tell us?

We fit a second-order surface to the image above and subtract it. Add the mean back in, and normalize. The result is here:

While extremely similar in appearance to the other ‘superflats’ we find that the central red patch is about 1% above the rest.



Asteroid

Showcase images and animations Posted on Nov 09, 2011 12:59


Here is the resulting DMI news on the net: http://www.dmi.dk/dmi/dmis_teleskop_fanger_asteroiden_yu55



Rainbow Angle

Links to sites and software Posted on Nov 06, 2011 14:53

Table of “Rainbow angle” as seen from MLO. The angle is SEM. Times are UTC. Sun is down.



Questions

Post-Obs scattered-light rem. Posted on Nov 06, 2011 09:48

With various systems to simulate realistic observations at hand – Chris’ noise simulator and Hans’ ideal image simulator – it becomes possible to pose some questions, such as:

1) Is it better to reduce every
blessed image and then average derived results, or should images be
coadded and then reduced?

2) Knowing that the bias has a 1-count
amplitude, 20-minute period is it then a sensible
strategy to scale a low-noise ‘superbias’ to the observed but noisy bias?

3) What is the relationship between precision in alfa and scatter in
DS/BS ratio, in the presence of realistic noise?

A partial, idealized, answer to 3) is hinted at by considering the change in DS and BS intensities as a function of change in alfa:

DS change: -15.9935 %
BS change: 0.0893807 %
alfa ch: 0.995023 % [Note that PSF was normalized]

I.e. If we change alfa by 1% we get a 16% change in a typical DS point and a 0.1% change in a typical BS point. This suggest that we need to know alfa to an accuracy of 0.6% (alfa was 1.7) or 0.006. This would be the typical step-size of a grid-search, for instance, and the tolerance on any downward-descent search. Perhaps it is easy to get such accuracy with methods that ‘fit the sky’ – such as both the BBSO and our own forward methods.

How does the above change in the presence of realistic noise?



Color separation

Links to sites and software Posted on Nov 03, 2011 08:43

You can use the convert command from ImageMagick to separate JPG images into the R G and B fields easily:

convert -separate -channel R Im.jpg R.jpg
convert -separate -channel G Im.jpg G.jpg
convert -separate -channel B Im.jpg B.jpg



Flatfield errors

Error budget Posted on Nov 02, 2011 18:11

I have compared four central 30×30 pixel areas in all the available sky master flatfields. A minimum of 3 masters were available in each filter. The selected regions a (blue), b (red), c (pink) and d (green) are shown in the figure. The flatfield used in the figure is the B master for night JD2455827.

For each filter and region, I have checked the difference between the largest and smallest mean value, expressed as a percentage of the smaller value. These (worst case) changes lie in the range 0.03-0.36%. I seems we can count on the error in a master flatfield to be very low!

The best master flatfields seem to be the B and V filter with only a single case of a change above 0.1%.

The worst region of the four is the green region. This is true for all filters. Perhaps these worst case changes in the worst region can be used as an estimate for the error in a master flatfield?



BBSO code

Post-Obs scattered-light rem. Posted on Nov 02, 2011 16:31

Now have a non-interactive version of the BBSO method for removing scattered light, written in IDL.

It takes about 10 seconds to ‘clean up’ the DS of one image.

The code is in SCIENCEPROJECTS/EARTHSHINE/ and runs by using a script:

./go_linearclean.scr filename.fits

a file is produced called

LINCLEANED_filename.fits

The method is non-intercative because the user no longer has to specify disc rim and disc center with a cursor. The estimation is done by edge-detection methods and an algorithm to fit a circle to three points in the plane, not on a line.



Refraction on JD2455729

Observing log Posted on Nov 01, 2011 10:51

On the July 26 2011 occasion of observing the star tau Tauri being occulted by the Moon we followed the Moon from just after Moonrise until Sun rose some hours later. During that time we tracked the star at sidereal rate so the Moon drifted through the frame.

Determining the radius and image-plane coordinates of the lunar disc we plot these:
We note that the radius increases slightly as the Moon rises – probably an effect of differential refraction decreasing as airmass approached 1. At most the radius estimate increased by about 1.5 pixels during the sequence. The standard deviation about the line is 1 pixel, telling us what the precision is of the method used to estimate lunar disc properties (it works by fitting a circle to points on the edge of the lunar disc).

We also see that the Moon did not follow a straight line across the image plane. The deviation is up to 5 or 6 pixels – about half an arc minute. This may be a refraction effect or a tracking effect or indeed an orbital effect – the mount we have does not track the Moon in declination.

In summary we see that the effect of differential refraction across the field is not more than half an arc minute at large air masses, consistent with physical estimates.



Photon noise in Canon camera

Error budget Posted on Nov 01, 2011 04:19

I have measured the pixel noise in the Canon EOS 35 mm camera. This is the same one as used to measure the halo of the moon at large distances (up to 15 degrees) from my Sydney backyard.

60 exposures were taken of a flat white surface, in groups of 10, with 5 exposures times from 1 second down to 1/200th of a second. (A 6th exposure time was discarded due to saturation).

Frames converted from CR2 format to fits format using cdraw and convert:

ls *.CR2 | awk ‘{print “dcraw -T -4 ” $1}’ | bash
ls *.tiff | awk ‘{print “convert ” $1, $1″.fits”}’ | sed -e s/.tiff.fits/.fits/ | bash

the tiff files this produces still have colour information, which I wasn’t expecting. The fits files are of course grey scale, but I suspect they are averaged over the three colours (RGB), so that the apparent photon counts are three times smaller than the actual photon counts.

The standard deviation and mean flux in 9 randomly chosen individual pixels was computed across the of 10 images obtained for each of 5 exposure times. The square root of the mean (times 3) is then plotted versus the standard deviation of the values, yielding 45 noise measurements (i.e. 9 pixels x 5 exposure levels of the pixel = 45). If the noise is anything like Poisson noise, these should lie on the 1:1 line.

This proves to be approximately the case as seen below.


It’s pretty noisy though! (i.e. the noise measurement is noisy).

Inspection of the fits files ds9 shows that there are clear correlations on various scales in both the X and y directions in the images — so the camera is surely introducing smoothing of some sort, so there is only so far one can push this analysis!

Conclusion : Poisson noise is a fair first order approximation for the behaviour of a commercial camera CCD chip.



Photos from the site

Showcase images and animations Posted on Oct 31, 2011 15:12

Some photos of the MLO site taken by Henriette:
http://www.dropbox.com/gallery/27975715/1/Earthshine?h=425dae



JD2455865

Observing log Posted on Oct 31, 2011 06:12

Observing Moon. Temperature in the 7-5 degrees C range (falling). Shutter sticking now and then. Weather very similar to previous days – clouds and rain into the late afternoon and then just at sunset the skies clear.

Dome ‘nudge’ working well and Dome Match also, although some work is needed on the table of pointings.

Halo around the Moon.



Sunglint software

Links to sites and software Posted on Oct 30, 2011 15:59

Here is the FORTRAN code that predicts the position of the Sun glint given the position of the Moon and the time.

PROGRAM SUNGLINT_MOON
c—————————————————————————
c code that calculates longitude and latitude of the sunglint
c as seen from the Moon
c—————————————————————————
c compile with: gfortran -O3 -fbounds-check sunglintFORTRAN_moon.f -o sung.exe
c or like this: f77 sunglintFORTRAN_moon.f -o sung.exe
c—————————————————————————
c IMPLICIT DOUBLE PRECISION (A-H,O-Z)
IMPLICIT logical (A-Z)
DOUBLE PRECISION GMT,SANG,DIFF,GLON,GLAT,GLAT1,AERR,GLON1,SLON1
DOUBLE PRECISION SLAT1,DEC1,DLAT,DLON,OLDGLAT1,DEC,RA,ASAT,ZSAT
DOUBLE PRECISION OLDGLON1,GLON2,U1,U2,U3,U4,V1,V2,V3,HANG1,HANG
DOUBLE PRECISION F0,G0,DUDT1,DUDT2,DUDT3,DUDT4,DUDP1,DUDP2,DUDP3
DOUBLE PRECISION DUDP4,HOURANG,ATND,DVDT1,DVDT2,DVDT3,XX1,XX2
DOUBLE PRECISION DVDP1,DVDP2,DVDP3,T1,T2,T3,T4,TEMP1,TEMP2,TEMP3
DOUBLE PRECISION AK,AH,TEMP4,TEMP5,TEMP6,DFDT,DFDP,DGDT,DGDP,JD
DOUBLE PRECISION ALATD,ALOND,ECLAT,ECLON,PI,SLAT,SLON,ALT,AZI
PARAMETER (PI=.3141592654D1)
CHARACTER DATE*15,TIME*8,INSTR*8
INTEGER HOUR,MIN,SEC
c READ IN DATE,TIME(GMT) AND SUB-LUNAR POINT (SLAT,SLON)
PRINT ‘(/5X,A,$)’,’ENTER DATE (DD-MM-YYYY):’
READ ‘(A)’,DATE
PRINT ‘(/5X,A,$)’,’ENTER TIME(GMT) (HH:MM:SS):’
READ ‘(A)’,TIME
PRINT ‘(/5X,A,$)’,’ENTER SUB-LUNAR POSITION (SLAT,SLON)’
READ *,SLAT,SLON
c CONVERT TIME (HH:MM:SS) TO DECIMAL HOURS
WRITE (INSTR,'(A)’) TIME
READ (INSTR,100) HOUR,MIN,SEC
100 FORMAT (3(I2,1X))
GMT=FLOAT(HOUR)+FLOAT(MIN)/.6D2+FLOAT(SEC)/.36D4
c COMPUTE POSITION OF SPECULAR POINT (GLAT,GLON)
CALL ECCS (DATE,GMT,ECLAT,ECLON,JD)
CALL EQCS (DATE,GMT,ECLAT,ECLON,DEC,RA)
CALL NEWRAPH (DATE,GMT,SLAT,SLON,GLAT,GLON,AERR)
CALL GECS(GLAT,GLON,SLAT,SLON,ASAT,ZSAT)
CALL HOCS(DATE,GMT,GLAT,GLON,DEC,RA,ALT,AZI)
PRINT 200,DATE,TIME
200 FORMAT (//,5X,’DATE: ‘,A12,10X,’TIME:-‘,A10,'(GMT)’/)
PRINT 300
300 FORMAT (5X,’POSITION OF SUN :’/5X,17(‘=’)/)
PRINT 400,ECLON
400 FORMAT (5X,’ECLIPTIC LONGITUDE:’,F7.2)
PRINT 500,DEC,RA
500 FORMAT(5X,’DECLINATION:’F9.2/5X,’RIGHT ASCENTION:’,F9.2)
PRINT 600,ALT,AZI
600 FORMAT(5X,’ELEVATION:’,F9.2/5X,’AZIMUTH:’,F9.2/)
PRINT 700
700 FORMAT(5X,’POSITION OF THE MOON:’/5X,27(‘=’)/)
PRINT 800,SLAT,SLON
800 FORMAT(5X,’SUB-LUNAR POINT:(SLAT=’,F6.2,’,SLON=’,F7.2,’)’/)
PRINT 900, ASAT,ZSAT
900 FORMAT(5X,’ELEVATION:’,F9.2/5X,’AZIMUTH:’,F9.2/)
PRINT 905
905 FORMAT(5X,’POSITION OF THE SPECULAR POINT :’/5X,32(‘=’)/)
IF (AERR .EQ. .0D0) THEN
PRINT 910,GLAT,GLON
910 FORMAT (5X,’LATITUDE:’,F9.2/5X,’LONGITUDE:’,F7.2)
ELSE
PRINT 915
915 FORMAT(5X,’GLINT DOES NOT LIE WITH IN THE VIEW AREA’)
END IF
END

DOUBLE PRECISION FUNCTION HOURANG(DATE,GMT,GLON,RA)
IMPLICIT LOGICAL (A-Z)
c IMPLICIT DOUBLE PRECISION (A-H,O-Z)
CHARACTER*15 DATE
DOUBLE PRECISION GMT,GLON,RA,GLON1,HANG
DOUBLE PRECISION LST,GST
GLON1=GLON
IF (GLON1 .GT. .18d3) GLON1=GLON1-.36D3
lst=glon1/.15d2+gst(date,gmt)
IF (LST .GT. .24D2) LST=LST-.24D2
IF (LST .LT. .0D0) LST=LST+.24D2
HANG=(LST-RA/.15D2)
IF (HANG .LT. .0D0) HANG=HANG+.24D2
HOURANG=HANG
RETURN
END

SUBROUTINE NEWRAPH(DATE,GMT,SLAT,SLON,GLAT,GLON,AERR)
IMPLICIT logical (A-Z)
c IMPLICIT DOUBLE PRECISION (A-H,O-Z)
INTEGER KK
DOUBLE PRECISION HSAT,ERAD,PI,RAD,RADI,CDIFF,DEC,RA,SLAT,GLAT
DOUBLE PRECISION GMT,SANG,DIFF,SLON,GLON,GLAT1,AERR,GLON1,SLON1
DOUBLE PRECISION SLAT1,DEC1,DLAT,DLON,OLDGLAT1,jd
DOUBLE PRECISION OLDGLON1,GLON2,U1,U2,U3,U4,V1,V2,V3,HANG1,HANG
DOUBLE PRECISION F0,G0,DUDT1,DUDT2,DUDT3,DUDT4,DUDP1,DUDP2,DUDP3
DOUBLE PRECISION DUDP4,HOURANG,ATND,DVDT1,DVDT2,DVDT3,XX1,XX2
DOUBLE PRECISION DVDP1,DVDP2,DVDP3,T1,T2,T3,T4,TEMP1,TEMP2,TEMP3
DOUBLE PRECISION AK,AH,TEMP4,TEMP5,TEMP6,DFDT,DFDP,DGDT,DGDP
DOUBLE PRECISION ALATD,ALOND,ECLAT,ECLON
CHARACTER*15 DATE
PARAMETER (HSAT=.384d9,ERAD=.6378D7)
c PARAMETER (HSAT=.35786D8,ERAD=.6378D7)
PARAMETER (PI=.3141592654D1)
RAD=PI/.18D3
RADI=.18D3/PI
CDIFF=.1D1/.36D4*RAD
CALL ECCS(DATE,GMT,ECLAT,ECLON,jd)
CALL EQCS(DATE,GMT,ECLAT,ECLON,DEC,RA)
GLAT=(DEC+SLAT)/.2D1
SANG=(.12D2-GMT)*.15D2
IF (SANG.LT. .0D0) SANG=SANG+.36D3
DIFF=dabs(SANG-SLON)
IF (DIFF .GT. .18D3) THEN
DIFF=.36D3-DIFF
GLON=DMAX1(SANG,SLON)+DIFF/.2D1
ELSE
GLON=DMIN1(SANG,SLON)+DIFF/.2D1
END IF
IF (GLON .GT. .36D3) GLON=GLON-.36D3
IF (DIFF .GT. .12D3) THEN
AERR=-.1D1
RETURN
END IF
GLAT1=GLAT*RAD
GLON1=GLON*RAD
SLAT1=SLAT*RAD
SLON1=SLON*RAD
DEC1=DEC*RAD
KK=0
150 KK=KK+1
DLAT=GLAT1-SLAT1
DLON=GLON1-SLON1
OLDGLAT1=GLAT1
OLDGLON1=GLON1
GLON2=GLON1*RADI
HANG=HOURANG ( DATE , GMT , GLON2 , RA)
HANG1=HANG* .15D2*RAD
U1= ( HSAT+ERAD) *DCOS ( DLAT) *DCOS( DLON) -ERAD
U2=DSIN(GLAT1)*DSIN(DEC1)+DCOS(DEC1)*DCOS(GLAT1)*DCOS(HANG1)
U3=DSIN( DLON)
U4=-DCOS(DEC1)*DCOS(GLAT1)*DSIN(HANG1)
V1=HSAT**.2D1+.2D1*ERAD* (ERAD+HSAT)*(.1D1-DCOS(DLAT)*DCOS(DLON))
V2=DCOS ( DLON ) *DSIN (DLAT)
V3=DSIN(DEC1)-DSIN(GLAT1)*U2
F0=DASIN(U1/SQRT(V1) )-DASIN(U2)
G0=ATND(U3 ,V2 ) -ATND(U4 ,V3)
DUDT1=- ( ERAD+HSAT) *DSIN ( DLAT ) *DCOS( DLON)
DUDT2=DCOS(GLAT1)*DSIN(DEC1)-DSIN(GLAT1)*DCOS(DEC1)*DCOS(HANG1)
DUDT3= .0D0
DUDT4=DCOS ( DEC1 ) *DSIN(GLAT1 ) *DSIN(HANG1)
DUDP1=- ( ERAD+HSAT) *DCOS ( DLAT ) *DSIN ( DLON)
DUDP2= .0D0
DUDP3=DCOS( DLON)
DUDP4= .0d0
DVDT1= .2D1*ERAD* (HSAT+ERAD) *DSIN ( DLAT) *DCOS (DLON)
DVDT2=DCOS( DLAT) *DCOS ( DLON)
XX1=-.2D1*DSIN(GLAT1 )*DCOS(GLAT1)*DSIN(DEC1 )
XX2=DCOS(GLAT1) *DSIn(GLAT1)*DCOS(DEC1)*DCOS(HANG1)
DVDT3=XX1+XX2
DVDP1=.2D1*ERAD* (HSAT+ERAD) *DCOS( DLAT) *DSIN (DLON )
DVDP2=-DSIN( DLAT) *DSIN ( DLON)
DVDP3=.0D0
T1=.1D1/(SQRT(.1D1-(U1/SQRT(V1))**.2D1))
T2=.1D1/(SQRT(.1D1-U2**.2D1))
T3=.1D1/(.1D1+(U3/V2)**.2D1)
T4=.1D1/(.1D1+(U4/V3)**.2D1)
TEMP1=(SQRT(V1)*DUDT1-U1*DVDT1/(SQRT(V1)*.2D1))/V1
TEMP2=(SQRT(V1)*DUDP1-U1*DVDP1/(SQRT(V1)*.2D1))/V1
TEMP3=(V2*DUDT3-U3*DVDT2)/V2**.2D1
TEMP4=(V2*DUDP3-U3*DVDP2)/V2**.2D1
TEMP5=(V3*DUDT4-U4*DVDT3)/V3**.2D1
TEMP6=(V3*DUDP4-U4*DVDP3)/V3**.2D1
DFDT=T1*TEMP1-T2*DUDT2
DFDP=T1*TEMP2-T2*DUDP2
DGDT=T3*TEMP3+T4*TEMP5*DVDT3
DGDP=T3*TEMP4+T4*TEMP6*DVDP3
AK=(F0*DGDT-G0*DFDT)/(DGDP*DFDT-DFDP*DGDT)
AH=-(AK*DFDP+F0)/DFDT
GLAT1=GLAT1+AH
GLON1=GLON1+AK
IF (GLON1.LT. .0D0) GLON1=GLON1+.2D1*PI
IF (GLON1.GT. .2D1*PI) GLON1=GLON1-.2D1*PI
IF (GLAT1.LT. -PI/.2D1) GLAT1=-PI-GLAT1
IF (GLAT1.GT. PI/.2D1) GLAT1=PI-GLAT1
ALATD=dabs(GLAT1-OLDGLAT1)
ALOND=dabs(GLON1-OLDGLON1)
IF (ALOND .GT.PI) ALOND=.2D1*PI-ALOND
IF (KK .GT.150) THEN
AERR=-.2D0
RETURN
END IF
IF (ALOND .GT.CDIFF .AND.ALATD .GT.CDIFF) GOTO 150
GLAT=GLAT1*RADI
GLON=GLON1*RADI
AERR=.0D0
RETURN
END

SUBROUTINE ECCS (DATE , GMT , ECLAT , ECLON, JD)
IMPLICIT LOGICAL (A-Z)
c IMPLICIT DOUBLE PRECISION (A-H,O-Z)
DOUBLE PRECISION GMT , ECLAT , ECLON, JD,ELE,ELP,ECC,PI
DOUBLE PRECISION EC,JDEP,RAD,AJDATE,DAYS,AN,M,AM
CHARACTER*15 DATE
PARAMETER (ELE=.279403303D3 ,ELP=.282768422D3)
PARAMETER (ECC=.167131D-1 ,PI=.3141592654D1)
RAD=PI/.18D3
JDEP=.24478915D7
JD=AJDATE(DATE,GMT)
DAYS=JD-JDEP
AN=(.36D3/.365242191D3)*DAYS
M=DINT(AN/.36D3)
AN=AN-(M*.36D3)
AM=AN+ELE-ELP
IF (AM .LT. .0D0) AM=AM+.36D3
EC=( .36D3 /PI ) *ECC*DSIN(Am*RAD)
ECLON=AN+EC+ELE
IF (ECLON .GT. .36d3) ECLON=ECLON-.36D3
IF (ECLON .LT. .0D0) ECLON=ECLON+.36D3
ECLAT=.0D0
RETURN
END

SUBROUTINE EQCS ( DATE, GMT , ECLAT , ECLON , DEC,RA)
IMPLICIT LOGICAL (A-Z)
c IMPLICIT DOUBLE PRECISION (A-H,O-Z)
CHARACTER*15 DATE
DOUBLE PRECISION JD,JDEP,PI,GMT , ECLAT , ECLON , DEC,RA
DOUBLE PRECISION RAD,RADI,T,EP,ECLON1,ECLAT1,TEMP1,TEMP2
DOUBLE PRECISION AJDATE,DEP,ANR,DNR,ATND
PARAMETER (PI=.3141592654D1)
RAD=PI/.18D3
RADI=.18D3/PI
JDEP=.2451545D7
JD=AJDATE(DATE,GMT)
T=(JD-JDEP)/.36525D5
DEP=.46815D2*T+.6D-3*T**.2D1-.181D-2*T**.3D1
EP=.23439292D2-DEP/.36D4
ECLON1=ECLON*RAD
ECLAT1=ECLAT*RAD
EP=EP*RAD
TEMP1=DSIN(ECLAT1)*DCOS(EP)
TEMP2=DCOS(ECLAT1)*DSIN(EP)*DSIN(ECLON1)
DEC=DASIN(TEMP1+TEMP2)*RADI
ANR=DSIN(ECLON1)*DCOS(EP)-DTAN(ECLAT1)*DSIN(EP)
DNR=DCOS(ECLON1)
RA=ATND(ANR,DNR)*RADI
RETURN
END

SUBROUTINE HOCS(DATE,GMT,GLAT,GLON,DEC,RA,ALT,AZI)
IMPLICIT LOGICAL (A-Z)
c IMPLICIT DOUBLE PRECISION (A-Z)
DOUBLE PRECISION GMT,GLAT,GLON,DEC,RA,ALT,AZI,U1,U2,V2
DOUBLE PRECISION PI,RAD,RADI,HANG,HANG1,GLAT1,DEC1
DOUBLE PRECISION HOURANG,ATND
PARAMETER (PI=.3141592654D1)
CHARACTER*15 DATE
RAD=PI/.18D3
RADI=.18D3/PI
HANG=HOURANG(DATE,GMT,GLON,RA)
HANG1=HANG*.15D2*RAD
DEC1=DEC*RAD
GLAT1=GLAT*RAD
U1=DSIN(DEC1)*DSIN(GLAT1)+DCOS(DEC1)*DCOS(GLAT1)*DCOS(HANG1)
ALT=DASIN(U1)*RADI
U2=-DCOS(DEC1)*DSIN(HANG1)*DCOS(GLAT1)
V2=DSIN(DEC1)-DSIN(GLAT1)*DSIN(ALT*RAD)
AZI=ATND(U2,V2)*RADI
RETURN
END

DOUBLE PRECISION FUNCTION GST(DATE,GMT)
IMPLICIT LOGICAL (A-Z)
c IMPLICIT DOUBLE PRECISION (A-Z)
CHARACTER*15 DATE
DOUBLE PRECISION JD,JDEP,T,T0,TT,GST1,GMT,AJDATE
JDEP=.2451545D7
JD=AJDATE(DATE,.0D0)
T=(JD-JDEP)/.36525D5
T0=.6697374558D1+.2400051336D4*T+.25862D-4*T**.2D1
TT=DINT(abs(T0)/.24D2)+.1D1
T0=T0+TT*.24D2
GST1=GMT*.1002737909D1+T0
TT=DINT(GST1/.24D2)
GST1=GST1-TT*.24D2
GST=GST1
RETURN
END

DOUBLE PRECISION FUNCTION ATND(Y,X)
IMPLICIT LOGICAL (A-Z)
c IMPLICIT DOUBLE PRECISION (A-H,O-Z)
DOUBLE PRECISION x,y,pi,dir
PARAMETER (PI=.3141592654D1)
IF (X .EQ. .0D0 .OR.Y .EQ. .0D0) THEN
IF (X .EQ. .0D0 .AND.Y .EQ. .0D0) DIR=.0D0
IF (X .EQ. .0D0 .AND.Y .GT. .0D0) DIR=PI/.2d1
IF (X .EQ. .0D0 .AND.Y .LT. .0D0) DIR=.15D1*PI
IF (Y .EQ. .0D0 .AND.X .GT. .0D0) DIR=.0D0
IF (Y .EQ. .0D0 .AND.X .LT. .0D0) DIR=PI
ELSE
DIR=ATAN(Y/X)
IF (Y .GT. .0D0 .AND.X .LT. .0D0 ) DIR=DIR+PI
IF (Y .LT. .0D0 .AND.X .LT. .0D0 ) DIR=DIR-PI
IF (DIR .LT. .0D0) DIR=DIR+.2D1*PI
END IF
ATND=DIR
RETURN
END

SUBROUTINE GECS(GLAT,GLON,SLAT,SLON,ASAT,ZSAT)
IMPLICIT LOGICAL (A-Z)
c IMPLICIT DOUBLE PRECISION (A-H,O-Z)
DOUBLE PRECISION HSAT,ERAD,PI,GLAT,GLON,SLAT,SLON,ASAT,ZSAT
DOUBLE PRECISION RAD,RADI,GLAT1,GLON1,SLAT1,SLON1,DLAT,DLON
DOUBLE PRECISION U1,V1,U2,V2,ATND
PARAMETER (HSAT=.384d9,ERAD=.6378D7)
c PARAMETER (HSAT=.35786D8,ERAD=.6378D7)
PARAMETER (PI=.3141592654D1)
RAD=PI/.18D3
RADI=.18D3/PI
GLAT1=GLAT*RAD
GLON1=GLON*RAD
SLAT1=SLAT*RAD
SLON1=SLON*RAD
DLAT=(GLAT1-SLAT1)
DLON=(GLON1-SLON1)
U1=(HSAT+ERAD)*DCOS(DLAT)*DCOS(DLON)-ERAD
V1=HSAT**.2D1+.2D1*ERAD*(ERAD+HSAT)*(.1D1-DCOS(DLAT)*DCOS(DLON))
U2=DSIN(DLON)
V2=DCOS(DLON)*DSIN(DLAT)
ASAT=DASIN(U1/SQRT(V1))*RADI
ZSAT=(ATND(U2,V2)+PI)*RADI
IF (ZSAT .GT. .36D3) ZSAT=ZSAT-.36D3
RETURN
END

DOUBLE PRECISION FUNCTION AJDATE(DATE,GMT)
IMPLICIT LOGICAL (A-Z)
c IMPLICIT DOUBLE PRECISION (A-H,O-Z)
DOUBLE PRECISION A,B,C,D,MONTH,JDBC,GMT,DAY,YEAR
PARAMETER (JDBC=.17209945D7)
CHARACTER*15 DATE,INSTR
WRITE(INSTR,100) DATE
100 FORMAT (A15)
READ (INSTR,200) DAY,MONTH,YEAR
200 FORMAT (2(F2.0,1X),F4.0)
DAY=DAY+GMT/.24D2
IF (MONTH .LT. .3D1) THEN
YEAR=YEAR-.1D1
MONTH=MONTH+.12D2
END IF
A=DINT(YEAR/.1D3)
B=.2D1-A+DINT(A/.4D1)
C=DINT(.36525D3*YEAR)
D=DINT(.306001D2*(MONTH+.1D1))
ajdate=b+c+d+day+jdbc
return
end



Noise model

Post-Obs scattered-light rem. Posted on Oct 30, 2011 11:38

I have made an attempt to figure out what the noise levels are at various fluxes in individual frames.

28 images of the nearly-full moon were used. They were very carefully aligned (using subpixel rebinning and the iraf task imshift). The shifts were determined with a fortran program which centered the moon using center of light — this was written because no iraf task I could find could easily align such images without reference stars.

Below I show in green circles the regions that were used to measure the standard deviation of pixels in a 10×10 box on the lunar face. These were chosen because they are relatively flat and have a range of luminosities. All the pixels in the scattered light off the lunar edge were also used, in a sector starting from the center between the position angles 40 and 50 degrees (i.e. off to the upper right at 45 degrees).

The standard deviation of the selected pixel positions over all the 28 exposures was computed, and a plot made of the sqrt of the mean counts in the pixel versus the standard deviation of the counts.

If the noise is Poissonian, then these are related 1:1. The plot below shows that this is only approximately the case (solid line). A closer model is that the noise is about half of what is expected (dotted line). Why the noise is sub-Poissonian is open to speculation, but must be due to some slight smoothing we don’t account for (possibly iraf introduces some smoothing when rebinning the data for the subpixel shifting).

The points at the lower left are those from the scattered light off the lunar edge. The other points are the pixels in the 5 small patches indicated by the green circles in the previous figure.

It’s difficult to fill in more points because it’s hard to find flat regions of the appropriate luminosity. Flat regions are needed because the technique is very sensitive to slight misalignments in the images (if one examines regions where the luminosity is changing rapidly — such as near the lunar edge, craters, strirations etc on the lunar face — the standard deviation goes haywire).

Assuming Poissoninan noise in test data generated from synthetic moons appears to be a conservative option, as it is pretty close to the correct value and possibly an overestimate, so currently I’d recommend that.

Update and error correction: The ADU conversion for this chip is approx. 3.8. The statistics above are were done in ADU, not photons. This of course accounts for the factor of two (=sqrt(3.8)) problem above. Thanks to Peter for spotting this error.

Using photons instead of electrons gives a pretty close fit t the 1:1 relationship. We could try this for a range of lunar exposure times, using the same flat regions on the moon and the halo off the edge, to fill in the gaps for a range of luminosities.



JD2455864

Observing log Posted on Oct 30, 2011 06:43

Getting the setting Moon in clear skies.

Plenty of ‘filter stick’ and it comes in sets – when we change filters the shutter ‘unsticks’ and we are good for one set of images, then we change filter and there is the chance that we get good data or bad data.

We also see a periodic shifting of the Moon inside the frame. Either the JPL table of Moon positions is slightly bad, or the interpolation is done.

Question: Is the table of Moon positions interpolated or do we go for ‘nearest neighbour’?


See the telescope and the Moon!



Comparison of flats within a single twilight session

Error budget Posted on Oct 28, 2011 18:37

I have compared two well-exposed flatfields in the B-filter from the dusk session night JD2455856 with a time difference of about 28.5 minutes.

Each flatfield was bias-subtracted with the proper scaled superbias, had a fitted surface subtracted, and was then normalized. A percent difference image was created as the difference compared to the earlier flatfield.
perc = ((late-early)/early)*100

The mean of this image is of course very close to zero (0.0014), but more interesting is the standard deviation (0.53).

The strongest part of the familiar diagonal pattern is still visible in the percent image (see left part of figure). I have used Image J to rotate the percent image and plot the horizontal profile of the yellow box (top right part of the figure) selected to be perpendicular to the diagonal darker areas. This profile is plotted in the bottom right part of the figure.
It can be seen that the diagonal structure is 0.2pp darker than the immediate neighborhood.

The diagonal structure does change within a relatively short time-frame and this will result in an uncertainty in the science frames when they are flatfield corrected. So far it doesn’t seem to be a large change, but more investigations are necessary. The more likely explanation for the change is temperature fluctuations. Perhaps it is possible to investigate if the diagonal structure changes with a period comparable to the change in bias level…



Fit the ES directly?

Post-Obs scattered-light rem. Posted on Oct 28, 2011 13:16

Peter sent me images of the ideal crescent moon for a range of Earth albedos — 0.0, 0.29, 0.30 and 0.31. I made a map of the scattered light to see how much brighter the ES (Earthshine) is than the scattered light from the crescent. Over most of the dark side, ES does indeed dominate over the scattered component — which augers well for measuring its ratio relative to the BS (bright side).


Top left: ideal moon outside the atmosphere with NO earthshine at all (Earth albedo = 0) i.e. we see the BS (brightside) only.
Top right: scattered light from the BS only.
Bottom left: scattered light of the same ideal moon, but with the ES for albedo=0.30 included.
Bottom right: the difference between the top right and bottom left images.

Interestingly, one can subtract the modeled BS to obtain the ES over the entire lunar disc. The isophotes of the remaining light are then nicely round around the full, ES-only illuminated disc, as seen in the lower right panel. This might be a way to check that the solutions are coming out the way they should — i.e. fit the BS and scattered light from it only, remove it — and the rest should be the ES. Errors in modeling the BS will show up as asymmetries in the ES that remains. Perhaps we could think about solving for the ES directly, by fitting a lunar model with a BS only, and recovering the ES on the lunar face directly?



Hohlraum lamp flat fields

Bias and Flat fields Posted on Oct 26, 2011 16:33

There is an ‘integrating sphere’ in the dome at MLO and we can now use it for flat field observations. Preliminary results suggest that the problems with gradients are as procounced as with other methods, and that the ‘interference pattern’ seen in VE2 with other methods are different when seen using the lamp.

The ND0.9 filter was also used and flat fields attempted – but maybe the shutter is prone to sticking during long exposures and the low temperatures we had today (4-5 degrees C).

The lamp is at Altitude 4 degrees and Azimuth 291 degrees. Some experimenst should be performed to find the best alignment.



Sunglint Map

Showcase images and animations Posted on Oct 26, 2011 15:35

The reflection of the Sun travels across the Earth as seen from the Moon. Since a majority of the earthshine originates near the sunglint it is of interest to map the position of the sunglint.

Below is a map of the footprints of the sunglint for all observations we have so far, from Hawaii. Since almost all the observations are morning observations the sunglint is East of Hawaii and has travelled across South America, Mexico and even North America (the last one may be an error and relate to observations of the Moon from Lund).

A few evening observations have been performed and they correspond to sunglint positions in the Indian Ocean, South-East Asia and Australia.

Close parallel tracks are separated by one day.

On the map below I show all the sunglint positions for the year 2011 – at half-hourly intervals. Times are selected for the Moon being up and the Sun being down. Blue points have Moon’s illuminated fraction less than 25% – i.e. acessible with the CoAdd technique.

The assymetry between the two ‘patches’ is not yet understood by me.



Color of Earthshine

Post-Obs scattered-light rem. Posted on Oct 25, 2011 20:32

After reduction of the observations the color of the earthshine can be considered. It will depend on the type of surface that predominantly reflected the sunshine but should be even across the DS at any one time – apart from lunar surface details, of course.

Any systematic gradient in the color of ES across the DS is a sign that the scattered light removal was imperfect – so a test can be set up.



HD2455859

Observing log Posted on Oct 24, 2011 17:04

Strange ‘dragging’ on at least the VE1 images – and yes, readout drag is up-down, not right-left.

Many frame dropouts. Temp. is between 4 and 5 degrees C.

B and V perform well at this very almost new moon, but VE1 and VE2 seem to need boosting relative to the bluer bands.

Wait, the VE2 is fine – it is the VE1 that is too low, as if it, in particular, wants to stick.

Noting that the external view of the VYSOS-20 position is the best for detecting the coming of Dawn.

VERY little scattered light from the BS tonight – even at very high airmass.



The Sunglint

Links to sites and software Posted on Oct 24, 2011 13:47

This IDL code will create a map showing the Earth and the point on the surface where the sunglint travelled, as seen from the Moon, during a specified time interval.

test_sunglint.pro

Look in idl_tools/



Living Planet site

Links to sites and software Posted on Oct 24, 2011 12:02

Helps visualize the Earth as seen from the Moon at user-selectable times: http://www.fourmilab.ch/cgi-bin/Earth

We need to learn how to use this site for finding the ‘sunglint’ coordinates on Earth. We also should understand if the site can show the actual cloud cover at a given time.



JD2455858

Observing log Posted on Oct 23, 2011 17:39

4-5 degrees C. Less thin cloud than previous nights. Shutter misses quite a few images. Towards dawn shutter working better. Sky noticeably brighter by 5:25 on all-sky camera, but not in CCD images.



JD2455857

Observing log Posted on Oct 22, 2011 17:01

Getting multi-band images of crescent Moon. Bands of thin clouds rolling past – vissible mainly as increased halo around Moon and in the eq. diff. images. T between 6 and 7 degrees C. Not very many failed frames.

Dawn started 5:45. by 5:55 sky-level was 445 – normally 397.

Time Level
6:00 505
6:02 530
6:04 930 – bias pattern starts to be seen
6:11 Earthshine side weaker than patterns in bias. Stopped.



JD2455856

Observing log Posted on Oct 22, 2011 10:37

Sky flats in the morning. Stopped when trailed stars were seen. VE1 flats ‘blotchy’ and strange.



Working Scripts

Control Software Posted on Oct 21, 2011 21:56

MOON.csv – uses the GOTOMOON and MOVEMOONTOREF script commands to take lots of images in all filters. Takes 11 images at a time through B,V,VE1,VE2, and IRCUT filters, then loops back. 100 times.

The present version of GOTOMOON does not put the Moon in the center of the image field. We understand why that is, now: Ingemar Lundström spotted that there is a table-lookup based on ‘nearest previous neighbor’ rather than an actual interpolation. As the table is spaced at 2-minute intervals the Moon will slide from image-edge to image-center at 2-minute intrvals, which is what we see. This will be fixed in a later version of GOTOMOON.

DOMEAZ placed in a script appears to work well – i.e. no crazy round-trips, and works well in the East at azimuths like 75 degrees.

DITHERMOON.csv – takes ‘dithered’ images of the Moon – i.e. Moon moves about in two nested Releaux triangles. All filters. Takes about an hour to go through all filters at all dithered positions.

LOOPING.csv and FFdusk.csv – a pair of scripts that are used to start and do flat-fields at DUSK (i.e. not dawn). LOOPING.csv is started before dusk and takes B-band images on the NAS – the user checks if the exposure level is such that the event has begun. When the level is right (i.e. not saturated and in the 50000s) then a Normal Shutdown is performed, System_Data_Table.csv is edited and FFdusk.csv is placed instead of LOOPING.csv. Then that is started in the MASTER CONTROLLER and that should now run through the dusk event for about 20-30 minutes extending the exposure times in such a way that well-exposed images are obtained all the way until full darkness. Tricky to do due to the ‘bump’ during dusk and also if there are any clouds, watch out.Well, the ‘bump’ is NOT visible here. Typical. But usually there is a ‘bump’ on the dusk part of the bottom curve for illumination (at 6PM). That bump can cause you to make a false start so that later the light is too high and all the images end up over-exposed. The nump lasts about 15-20 minutes. Good example on the dawn shoulder! Might be clouds low on the horizon or whatnot.

ALTAIR.csv, JUPITER.csv and all STARNAME.csv scripts – have the coordinates of the star in the script, so only start these scripts if that object is above the horizon! Loops through all filters with (somewhat) appropriate exposure times.



JD2455856

Observing log Posted on Oct 21, 2011 16:11

Lots of thin cirrus! 9 degrees C. Taking lunar images. Will follow the Moon up in case it clears before dawn.

Can confirm that GOTOMOON script command works – goes right to the Moon. [Added later:] but, weirdly, seems to drift slightly. Since the table is OK it must be the calibration that goes slightly off – polar axis alignment issue?

DOMEAZ was supposed to put the dome where the telescope is pointing – and it worked for a while, and then started sending the dome off to strange places. Perhaps some homing is needed on the dome before using the command?

Testing the combination

GOTOMOON
MOVEMOONTOREF

to see if it keeps the Moon centered. At least the MOVEMOONTOREF does not send the telescope anywhere crazy.


Nice screendump of what the Front Camera Saw earlier today. The Pleiades!



GPU- versus CPU-based modeling of scattered light

Post-Obs scattered-light rem. Posted on Oct 21, 2011 02:53

I have spent some time examining to what extent GPUs could help us speed up scattered light fitting of ideal moon images to the observed images, while visiting Swinburne in Melbourne.

Ben Barsdell (Swinburne) had some c++ code to convolve two fits files using an NVIDIA compatible GPU (GTX480) running CUDA. The FFT library is FFTW, the same as I have been using for CPU-based modeling of the scattered light.


Upper left: ideal lunar image (intensity is log scale).
Upper right: convolved with our best PSF using CPU based FFT.
Lower left : convolved with PSF using GPU.
Lower right: ratio of the two methods (seen in more detail below).

On a desktop my fortran code calling FFTW does the three FFTs needed —
ideal moon image, psf image, their multiplication in the Fourier
domain, and the inverse FFT for the final result — in about 1100
milliseconds (ms). (This excludes the time needed to insert the 512×512
images into 1536×1536 images to sufficiently reduce edge wrapping
effects, which brings the total runtime to about 3000 ms). So the FFTs
are taking about 300 ms each.

Using the GPU, we attained FFT
speeds of about 40 ms each, a speed of a factor of 10 or so. This is
typical of what Ben expects for such applications (he is doing a careful
study of the types of astronomical problems GPUs can be profitably
applied to, and where the bottlenecks typically lie.)

Ostensibly
this means we can speed up our light modeling code by about 10 times — possibly
quite a bit more because the overheads per modeled image can be reduced
quite a lot by careful programming, since we want to explore a large
parameter space, but don’t need to do the same overheads each time we
run.

VERY IMPORTANT: the CPU code did the FFTs in double complex precision, whereas the GPU was doing single complex precision.


We
compared the output images of both methods — for the case of scattered
light from an ideal moon with a power law fall off PSF with a slope of
about r^-2.8.

VERY IMPORTANT: there is significant structure left in the methods if we divide the CPU output by the GPU output, as shown int he image above.

The scatter in the mean about 1.0000 is 2.028E-4 and there is a frame minimum of 0.9936 and frame maximum of 1.003, so the deviation from unity in the ratio of the two methods is not worse than ~0.7% anywhere on the frame. There is certainly structure, and it looks like it might be too much for our purposes! (We would like this to be better than 0.1% at the very worst). We are checking if this has to do with the single precision used.

Unless this problem can be solved, it is not clear that the speed gain with the GPUs
is worth having!



JD2455854

Observing log Posted on Oct 20, 2011 18:24

9 Degrees C. Some thin cloud from the start – then more and more.

Lots of Moon observations in ‘dither’ mode, through all filters.

Day started with CCD flatlining. Reboot. Then all OK, Still have to set sub-degree offsets on pointing to put Moon in middle. Why? Will test the MOVETOREF command.

FW apparently OK – no noticeable ‘sticks’.



PSF of Vega obtained by Tom Stone

Exploring the PSF Posted on Oct 20, 2011 03:10

Peter sent Chris a file called “ROLO_765nm_Vega_psf.dat”

which contains the PSF of Vega measured by Tom Stone.

Here is the profile:

the horizontal scale is log of r in arsecs.

the inner part is something like a Gaussian, like King finds.

the halo outer part goes at r^-2.4, in the range 1.2 < log(r) < 2.0,
so steeper than King but not as steep as our steepest profiles
to date (these are r^-3).

The very outer parts are certainly affected by sky subtraction,
as the light will be well below sky at log(r)>2, one could fix this
probably by choosing the sky so that the halo continues at the
same gradient, but this is of course arbitrary.

It seems that for a wide range of telescopes and instruments,
the distant stellar light falls off as a power law — we see this
in the ES telescope, a 35 mm camera, King’s profile, and a
range of other instruments/telescopes summarised in a paper
by Bernstein (ApJ, 666, 663, 2007).

The slope of the power law can change even on the same
telescope/instrument! We see different powers on different
nights. This is important!



PSF Profile for Altair

Exploring the PSF Posted on Oct 20, 2011 03:02

Attached plot is the Altair profile from the latest run.

The data don’t go quite as deep as Menkab, but are highly
interesting all the same.

for the IR, V and VE2 filters, the slope is very close
to our canonical r^-3
or so which we get for the moon.
it was a very clear night, so that’s SUPER!

for the B and VE1 filters the halo is rather different — as
can be see in the plot!

we might have some sky subtraction problem with VE2, so let’s
not worry about that too much at this point.

the peculiar profile is definitely B — as before with Menkab —
there is a peculiar hump in the psf at log(r) ~ 0.8 to 1.0
(6 to 10 pixels). We saw the same in Menkab and wondered
about focus.

Could there be something funny going in with internal
reflections in the B filter, so that an extra halo appears around
stars in that filter but not in the others — or at least not
prominently?



Light far from Moon in Canon 35 mm camera

Exploring the PSF Posted on Oct 20, 2011 02:58

We obtained images of the full moon from Sydney with the Canon
35 mm camera, reaching out to about 15 degrees from the moon
— three times further out than the King profile reaches. We are
curious about how far out it goes as a power law.

The plot shows the profile done for a range of exposure
times, getting longer and longer, at the cost of overexposing the
moon and halo around it, but better sampling the faint halo
far away.

I have shown the raw data merging into the background sky in the
bottom panels. Subtracting this sky gives the halo only profile in
the middle panel for the range of exposures.

Top panel shows the overall profile by using only the valid (not
overexposed) parts of each curve — by simply accounting for the
different exposure times — nothing more has been done. They
sit on a pretty well defined locus all the way from the edge of the
moon out to the edge of the frame (~15 degrees out) and follow
roughly r^-1.8. It’s probably flatter than this closer in. This is of
course the scattered light, not the PSF per se, but very far from the
moon — like > a few degrees — the moon is essentially a point source
anyway (it’s only 30 pixels across on a chip 3900 pixels wide).

I learned something important! The profile is r^-1.8 when sky subtracted!
We can follow it well beyond the point that the halo is much fainter than
the sky. So King I think is talking about the sky subtracted profile as well.
I have been wondering when the King profile would actually join
the sky and become a DC component… of course it doesn’t, that’s
the point. The DC component, i.e. Rayleigh scattering (?) is caused
by the bright source as well, but is not considered part of the profile.
I guess. So if we were trying to model the scattering over very large angles
we’d have to consider both components. My guess is that for our
ES telescope the field of view is so small that the scattered light is
the halo only — the Rayleigh component is negligible. One starts
to see it in the 35mm camera data at about 1000 pixels (1 pix
is approx 1 minute in these data) so about 10 degrees to 15
degrees from the moon (lower panel shows a blue curve which
essentially joins the bias level at large r, whereas the green and
black curves are flattening out to a level which is bias + a fair
bit of sky). In these data, the sky is about 5*2.5 = 7.5 magnitudes
below the full moon surface brightness (seen in upper panel, with
brightness of the moon at log(r)=0 and the (subtracted) sky having
similar brightness as the halo at about log(r)=3 — beyond which one
traces the halo below the sky).



MOVEMOONTOREF

Control Software Posted on Oct 19, 2011 14:13

MOVEMOONTOREF is a command to place the center of gravity of the Moon’s illuminated disc in the center of the image. Has been tested with GOTOMOON which must preceed it.

Seems to work.



Weird saturation pattern

Andor camera field experiences Posted on Oct 19, 2011 14:09

This image of the Moon was obtained on JD 2455849.
Notice the strong saturation at the lower left corner – on the sky! Other parts of the Moon are safely exposed in the 10000-30000 counts range.

Here is a supposed ‘dark’ frame showing something similar:
Most of the dark frame is at the bias level of 397 – the corner shoots into 65000!

This was during our ‘everything looks like a bias frame we think something is getting stuck’-period.

Both frames above are available in the BADEXAMPLES/ directory as FITS files.



JD2455853

Observing log Posted on Oct 19, 2011 14:00

Various things failed and none of the planned observations could be obtained:

Telescope did maverick meridian flip that we cut short as it neared the floor.

The camera flatlined for none of the known reasons.

Then some part of the FW or rotary stage system got stuck. There is an issue with the large rotary stage acknowledged by several parties. I additionally think the FW can get stuck at times – witness the failed VE1 observations on JD 2455849.

PXI is at the moment unreachable over the internet following an iBoot power cycle that failed to re-boot the machine.



Dome Azimuth

Dome issues Posted on Oct 19, 2011 12:32

Upper plot shows (red labelled crosses) observed relationship between dome azimuth (value next to cross) and telescope azimuth/altitude. The contours are due to a bi-linear interpolation.
Lower plot shows the same data but interpolated with ‘kriging’.



The 10 ms limit

Shutters Posted on Oct 19, 2011 10:03

After speaking to Ahmad yesterday about the shutter performance I am not
sure we fully understand it. If I understood Ahmad correctly there will
be shutter dropouts now and then due to the temperature-related issue
that causes the shutter simply to not open. Then there is the issue that
sometimes the shutter opening time is measured close to zero without
actually being close to zero – an issue due to an occasional ‘spike’ in
the signal from the shutter which causes a timing circuit to start and
stop very quickly.

So these two modes can lead to images that have a measured exposure time
of close to 0 – about half actually are close to zero and the rest are
OK but the measured exposure time is no good.

Can the measured exposure time be trusted for other situations?
Apparently it depends on how short the requested exposure time is.

I took about 200 images of the moon from our database and extracted
requested and measured exposure times as well as the total counts in
each image. I then considered the standard deviation of the fluxes
(=counts/time) for both the measured and the requested exposure time. I did this for increasing exposure times by raising a lower limit on the
times considered.
The black line is for measured
exposure times and the red is for the flux based on the requested
exposure time. For exposure times below 10 ms the standard deviation of
the flux is smallest if you use the requested exposure time. For
exposure times longer than 10 ms the standard deviation is smallest if
you use the measured exposure time.

The result is the above graph. We only have data for exposure times up to 25ms – this can be extended later.

There is a clear change in behavior at 10 ms . this is also reflected in this

lab report by Ahmad and Rodrigo.

It would seem that only very shaky
results can be had for exposure times below 10 ms. That is, if you
really need the absolute flux in an image do not expose under 10 ms.
CoAdd mode will still work, of course, since the whole image is affected
in the same way by shutter variability and we do not need accurate inter-image fluxes for
this mode to work.

The issues here do have an impact on the ability to work with the
non-simultaneous modes, and some careful analysis of the measurement
error is needed – e.g. by using a constant-flux lamp in the dome or just the inside of
the dome or the clear sky during shorter intervals.

As the lamp is now accessible we can test this!



The Spur

Optical design Posted on Oct 18, 2011 22:21

Ahmad pointed out that ‘the spur’ seen in coadded images of stars, and interpreted as an effect of the shutter staying open while readout is performed, is due to something else – not readout; because the readout direction is at right angles to the spur.



Cusp Angle and the Offset

Control Software Posted on Oct 16, 2011 12:57

A few test cases for making sure we are talking about the same thing when we set up the cusp angles and KE offset.

The agreed convention is to measure angles positive East of North. Blade starting position (at least conceptually) is with the KE covering the East side of the lunar disc and pointing North-South. After the angle is set the blade is positioned in offset (positive only by convention – the KE has to be advanced from disc center enough to over the bright parts and a little more of the DS. Note that the blade is FIRST rotated so that the BS is covered.

JD2466871 November 5 2011: The Moon is bright on the WEST side (the side towards DEcreasing RA). The phase is over half, i.e. more than half the Moon is bright. Judging from the display in ‘skychart’ the Cusp Angle is -195 degrees, i.e. to cover the BS the blade is rotated by negative 195 degrees. A solution would also be to rotate it by +165 degrees. Since the BS is still vissible the blade is now advanced at right angle to its edge by about 3/4 lunar radii – i.e. by 0.19 degrees.

JD2455866 October 31 2011: The Moon is less than half and is bright in the West side.
Judging by the display on skychart the blade should be rotated -186 or +174 degrees. The blade should be advanced although the DS is more than half the disc, since we want to get away from the cusps, so advance by 1/4 lunar radius or 0.12 degrees.

Remember – the blade advancement is always positive – never negative. It pushes FORWARD from its starting position at disc center and covers MORE of the disc. As the example on JD2455866 shows the advancement is always at least 1/8 degree and can reach as much as almost 1/4 degree.



Dome Home and the Extra Circuits

Dome issues Posted on Oct 16, 2011 08:09

Doing the homes on the dome, to see if it needs re-doing if all you do is turn off the system and turn it back on. I.e. – will the dome do the ‘two circuits’ first time you set a dome AZ?

This is what happens:

1) You press the Find Home button on the Dome tab and the dome does a whole or almost a whole circuit and stops at Home which appears to be due south.

2) You ask it to go to some other AZ – the dome does a whole or almost a whole circuit and settles on home – i.e. Due South

3) You ask it again for an AZ and it goes there.

4) Stop on the EM. Turn power off the mount on iBoot (i.e. channel 5). Turn the power ON on the iBoot for the mount. Turn the EM on again.

5) Aak for a dome AZ – and the dome goes there.

So, it seems you get two Find Homes at the start and then the system works, across a power cycle.

Later: some results seem to indicate that the ‘reset’ is done by Turning labView Off – as in rebooting the system or using the ‘Exit’ button (i.e. so that no instances of labView are seen in the Task manager). Umm.



JD2455849

Observing log Posted on Oct 14, 2011 18:37

More ‘dithered’ observations of the Moon. VE1 filter images kept being ‘bias frames’. How does the shutter know that the VE1 filter is in? Or is something else going on?

Pointing with absolute coordinates is awkward – from night to night the mount calibration seems to go slightly off so that new offsets have to be tested in a time-consuming manner. If the polar axis alignment is not good, would it cause this problem?

CCD seemed to show new startling ‘saturation’ modes.

here



Shutter dissapointment – part II: The Rubbing It In

Shutters Posted on Oct 13, 2011 17:30

The Canon shutters we tested – see here and here, seems better than the Uniblitz shutter we have – at a fraction of the cost. For very few dollars we could have had a substantially more reliable shutter.



Another problem

Shutters Posted on Oct 13, 2011 17:19

We have come across another problem with the shutter: Not only is it variable when it shouldn’t be – i.e. we have complete frame dropouts and we have a spread in observed counts from sources that are constant – stars at short exposure time excluded here due to scintillation – but we also now have examples of FITS file headers that show the measured to exposure time to be 0.000000000000001s when it is patently not – for instance, the image is fine and shows a well-exposed Moon.

This limits our possibility to use the ‘not simultaneous’ modes of observing – i.e. BBSO and modified BBSO.



Calculated flat field

Bias and Flat fields Posted on Oct 13, 2011 16:02


Flat field calculated using Chae’s method. It is based on 115 lunar images ‘dithered’ randomly around a central point. The dust speks near the middle are recognized from conventional FFs. The two diagonal stripes are also recognized as part of this CCDs extraordinarily strong fixed-pattern. The gradient from blue to pink and white corresponds to 2%. The image is based on B-band images from the night JD2455847.

It turns out that ‘slopes’ in this method has to do with the normalization of the source intensity – the method depends on the source being constant, but the source is not constant if it is partially on the dge of the image frame, is it? We are working on this.



Altair and Menkab – the psfs

Exploring the PSF Posted on Oct 12, 2011 12:12

We now have good radial profiles of Altair and Menkab. Altair was observed in very good conditions, and its profile is close to the expectation of r^-3 for the diffraction limit — in three of the 5 bands. In B and VE1 funny things are happening which we don’t yet understand. The Menkab profiles have halos like r^-1.5 — much broader. The plots below compare both stars:

Altair:

Menkab:

We think that Menkab might have been taken on a not-quite-perfect night photometrically. We’re looking into this, so watch this space!



Altair shines again!

Exploring the PSF Posted on Oct 12, 2011 10:01

Notes on the photometry for ALTAIR on JD2455845 (about October 11, 2001)
————————————————————————

Altair was observed with the B,V,VE1,VE2, and IRCUT filters. Dark frames were obtained before and after. Images had the interpolated dark field subtracted (i.e. the average of the closet DF before and after was formed and subtracted).

The IDL routine BASPHOTE.pro was used with a 9/9/12 annulus aperture to find the counts in each stellar image. The exposure time was read from the FITS header and was the actually measure exp time, not the requested one.

Fluxes were calculated for each filter and the standard deviation of the fluxes expressed as a percentage of the mean flux.

Results:

Band SD
——————
B 8.5%
V 8.4%
VE1 9.9%
VE2 4.5%
IRCUT 7.6%

Exposure times were in the range from 0.17s to 1.1 s for the various bands.

Some frame-dropout was noted – about 5 frames in 100 failed to open at all.

The variability in the flux is consistent with previous considerations – i.e. there is scintillation. The increase in variability with blueness may be because there is more scintillation in the blue or because exposure times are shorter in the blue, or both.



PSF of Menkab in 5 bands

Exploring the PSF Posted on Oct 09, 2011 05:38

Peter took data on Menkab in 5 bands (October 7th, 2011) – B, V, VE1, VE2 and IRCUT. Surprisingly, the halo around the star follows r^-1.5 in all bands. Was the night a bit hazy? On a clear night we have been expecting r^-3.

The central PSF, within ~5 pixels, is very regular, as the stars are nicely round (Altair, with the best inner PSF to date, was a very boxy image — see posting on this below).

The VE2 image has a significantly broader transition region to the outer halo. Quite strange!

The 5 profiles are here :

The are compared to the profile we got for Altair in V band a few nights ago and our best guess profile in V from the modeling the scattered light from the moon.

The halo slope is about r^-1.5 in all 5 bands for Menkab. They also wiggle a bit, unlike when we get r^-3, which is straight as best we can tell. The “shutter bounce” effect is also much harder to see because, presumably, the flatter halo is washing it out. It’s visible but weak in the 2-d images.

This is all a bit surprising and very potentially very important. We may be much more dependent on the atmospheric conditions on each night that we thought.

It would be very good to do aperture photometry of the star as a function of time in each band for each obtained image, to check if the night really was “photometric”.



Periodic bias level

Error budget Posted on Oct 07, 2011 11:50


I have ckecked that the periodic behaviour of the bias-level is the same over the whole area of the CCD. I have chosen night JD2455745 for the investigation because all the dark-frames have the same exposure time (60sec).

I found the mean of five small areas of the CCD and plotted them together with the mean value obtained from the full area of the CCD as a function of time since first frame. The selected areas represent the four corners of the CCD (1 pixel away from the edge) and the center of the CCD. I therefore expected that each area would have its own mean level, but the same period and amplitude as the whole frame.

I have tried this for two different area sizes, 8×8 pixels and 16×16 pixels. The periodic behaviour can be seen in both cases, but it is quite muddy for the small area size. The above image shows the 16×16 case. The red stars are the mean values of the whole frame and each colour represent one of the five 16×16 areas. All the five areas follow the same period with roughly the same amplitude as the whole CCD.

From this I conclude that it is safe to assume the periodic behaviour is a shift in the bias-level of the whole CCD area. It is likely to have to do with the thermostatic control of the temperature of the CCD.



MEDIAN ADJUST

Control Software Posted on Oct 04, 2011 20:18

One of the scripts uses the MEDIAN ADJUST command – it adjusts the key word for exosure time so that a better guess is used, based on the previous exposure time and attained flux level. It seems to work in the DOMEFLAT_COMMANDED.csv script, except for B and VE2 filters, where the exposure times suggested is 180 seconds. This is the limit (to avoid wasting too much time) and the corresponding images are underexposed. This is basically because the CCD camera is RED-sensitive, not BLUE-sensitive, and because the paint on the inside of the dome at MLO is not very reflective at red wavelengths. The V, VE1 and IRCUT images are properly exposed to about 51000 counts as they should be.



JD2455838

Observing log Posted on Oct 04, 2011 19:51

Apparently managed to ‘unstick’ the previously stuck filter-wheel stage that caused blocking of the FOV by the edge of the aperture of the stage that holds the SKEs. This was done from the Engineering Mode by ‘finding the homes’. The Large Rotary Stage did not reset in such a way that the green light indicating success ever lit up. The Little Rotary Stage rotated, cleared the blockage, and its light lit up.



PSF in a Canon 35 mm camera

Exploring the PSF Posted on Oct 04, 2011 14:45

We decided to look at the scattered halo light around the moon in a conventional 35 mm Canon camera, out of sheer curiosity.

The field of view of the Canon of about 10 degrees across, with images of 3888×2592 pixels. There is a pincushion effect toward the very edges but otherwise the halo is well traced. 12 images starting from an exposure time of 1/2500 seconds and increasing exposure time by factors of two at each step were taken. The images were taken from Chris’ back garden in Sydney on a very clear night in September 2011.


Upper plot: halo around the saturated moon (seen at left) for the five longest exposures. The uniform separation of the profiles by 0.3 (in the log) is just as expected for factors of two increase in the exposure times, so the system appears to be linear. Lower plot: log-log plot of the halo flux versus distance shows a very closely followed power law with slope ~ -1: this we call a “Toto” profile (= r^-1). The earthshine camera instead shows a “Mitzi” profile, r^-3, which is as expected for a diffraction limited telescope. A 35 mm camera has a great deal more scattered light!



Papers we may need for reference

Relevant papers Posted on Oct 04, 2011 11:36

Andor camera test report, Darudi.

Darudi and Rodrigo report on Spectral Analysis of Earthshine telescope.

Astro-Physics v4.12 manual for Keypad:

SPIE paper by the team, on the telescope:

Dravins et al 1998:

Dravins et al 1997a (the scintillation figure is on p 186):

Dravins et al 1997b:

Bernstein 2007 (with table of PSF alfa’s):

Middlemass et al 1989 (interesting discussion of color-dependency in PSFs):

The Chae paper showing the alogorithm for calculating flat fields from dithered observations of extended sources.



Diffraction-limited optics

Optical design Posted on Oct 03, 2011 14:03

According to theory, the aberration for a lens in a circular aperture, like ours, is given by the Airy function which is proportional to (BESELJ(r,1)/r)^2.

For a single wavelength the function looks like this:

Theory also predicts that assymptotically the envelope of the Airy function should go as 1/r^3, which is also plotted on the graph above.

For mixed-wavelengths as in our case the curve becomes smooth(er) and we should expect to see our PSF as a 1/r^3 envelope. Steeper envelopes than this should not be observed in correctly analysed optical systems. This has bearing on the 1/r^2 ‘King profile’.



FW stuck

Mechanical design Posted on Oct 03, 2011 13:02

Stuck Filter Wheel hindering further observations:

Two dome flats showing a stuck FW. A script was used to get these images. In between the images we used the manual Engineering Mode to ‘do the homes’ (i.e. to force each wheel and stage to traverse its range and find its home position). Apparently the “Large Rotary Stage” is still stuck after doing this.



What the Skycam saw

Observing log Posted on Oct 03, 2011 08:29

A selection of Skycam shots to show what you can see. From top left : a hazy night, a very hazy night (ice crystals?), a very clear night, water on the sky cam and the laser star, the skycam on a wet afternoon and the crescent moon on a bright sky just before sunset.



JD2455836

Observing log Posted on Oct 02, 2011 08:56

Skyflats, Moon and Altair.

Skyflats started too late and are not well exposed – I started at 6:13, Sunset was close to 6:20 so should perhaps start closer to 6:00 next time.

Moon was run through various settings of SKEs and the KEDFs as well as the NDs – but images appear strange – as if the devices did not set right or the input data was wrong.

Altair was imaged through variable cloudiness allowing us to see both very clear PSFs and the misty PSFs.

The NAS was acting up – images were not being received on it. Had to reboot.



Altair maps the stellar psf!

Exploring the PSF Posted on Oct 02, 2011 08:23

We took a few hundred 0.4 second exposures of Altair, to measure the stellar PSF. Plot below shows Altair (green) versus our current best model (red) of the PSF based on fitting the moon. Note x axis is log(r/pixels).

The match is excellent, especially for the outer halo with its r^-3 powerlaw! Disagreement in the inner PSF (r<3 pixels, log(r)<0.5) is because Altair is very elliptical — so the radial PSF shown here has a lot of scatter. Note that this might be due to saturation/miscentering during the coadd – we’ll look at that.



Timings

Mechanical design Posted on Oct 01, 2011 08:06

We have learned this about timings:

Color filters – it takes from 15 to 45 seconds to change a color filter. Mainly it takes 20 s, with exceptions.

SKE – it takes a full minute to set the SKE.

An image takes 0.3 s to be downloaded and written to disk.

Dome takes about 2 minutes to open.

One circuit of dome takes about 2 minutes.



JD2455835

Observing log Posted on Oct 01, 2011 07:44

Rain on MLO. Moon up.

Tested setting the SKE from a script – this failed. The values for cusp offset and cusp angle were taken from the System_Dataxxx.csv master spreadsheet rather than the script itself (“protocol”).

We actually have no, documented, proof that the setting of the SKE or KEDFs works, right?



Dome and telescope pointing

Observing log Posted on Oct 01, 2011 07:21

Ben has created a map of where the dome should be for a range of telescope pointings. Map works well so far! We are adding data points to it as we go along, and also mapping out the edges of the dome slit at various positions:


A better resolution version is available here:

http://www.astro.utu.fi/~cflynn/dome_pointing_plot.ps



Comparison of brute force and FFT methods for computing scattered light

Post-Obs scattered-light rem. Posted on Oct 01, 2011 02:11

Tested two methods for scattering light from an ideal moon. The Ideal image is shown upper left (log scale) with a completely dark sky, narrow BS crescent and ES.

Bottom left is the moon after scattering the light using our current PSF and a brute force method — simply add all the pixels in the ideal image multiplied by the PSF. Bottom right is the same, but using an FFT method instead. The scattered light at the edges of the frame is too high to simply use the 512×512 frame in the FFT, instead the idea moon is placed in the center of a 1536×1536 frame, the FFT carried out, and then clipped back to 512×512. Upper right is the ratio of these two methods. There is a lot of structure at a very low level — the display limits on the ratio are set at 0.9993 to 1.0009, and the ratio is with 0.1% of unity everywhere on the frame. This should be sufficiently accurate for our purposes, but we should double check!

The histogram of pixel values in the ratio image is shown below:



JD2455834

Observing log Posted on Sep 30, 2011 13:57

Testing the Focus Search tab in the Eng. Mode. Works nicely – but how do you reset the scale on the graph?

Also tried to ‘open’ and ‘close’ the Front Iris (elsewhere labelled as 30mm and Open). This produced no effect what so ever on diffraction spikes or flux levels. Does the Iris work? In which position is it stuck?



Front Iris not affecting throughput?

Shutters Posted on Sep 30, 2011 13:05

Chris and Peter observed the bright star Markab on the night of September 30 2011. We noticed that changing the size of the Front Iris — from 30 mm setting to the 40 mm setting had no effect on the amount of light coming from the stars. In both positions we were getting peak counts in the star of around 56000, and total of 246000 in a 10×10 box centered on the star.

We were expecting the flux to increase in going to the 40 mm iris from the 30 mm iris by a factor of (40./30.)^2 = 1.8.

Are we misinterpreting what this iris does – or did it not move? We will ask Dave about this cosmic mystery!



Night with strong halos

Exploring the PSF Posted on Sep 29, 2011 13:46

Sep 29 — Peter and Chris observed Markab — noticed much broader halo around the stars than usual. Image below shows that this can even be seen in the sky camera.

Left: clear night from 2011-09-27, Jupiter in center of frame. Right: not photometric conditions 2011-09-29, Jupiter has a slight halo. Note that the bright “star” right above Jupiter is possibly an internal reflection (of Jupiter?) in the sky camera.


We will observe Jupiter through whatever this stuff so we can compare directly to previous determinations of the PSF.



Astigmatism

Exploring the PSF Posted on Sep 29, 2011 03:30

The plot shows astigmatism of the psf across the image plane. A short single exposure of the open cluster M7 was used, exposure time = 60 seconds. Ellipticity is shown by the length of the lines in the lower plot — and the lines are aligned with the position angle of the sources. The upper plot shows that ellipticity rises pretty sharply towards the CCD edges — from about 0.05 in the center to about 0.4 at the edge. n.b. ellipticity is defined as e = 1 – b/a, where a is the semimajor axis and b the semiminor axis. Ellipticity at the center of the field is about 0.05 – not 0.0. These results are for the ellipticity of the inner core of the PSF (it’s measured out to 3 pixels radius). Whether the power-law halo of the PSF is non-round on any scales is still under study.



Shutter timing stability from full moon observations

Shutters Posted on Sep 29, 2011 03:19

On night of Septembter 10, 2011, 28 images on the almost full moon were obtained at 0.015 second exposures. We compute the mean level of these images. They are dominated by the full moon, the sky flux is negligible, and the moon is well centered.

The mean count rate for these frames is 5546.7 counts / 0.015 sec and the standard deviation of the mean counts over all 28 frames is 8.4 counts – if this standard deviation is all due to shutter timing error for these frames, the timing error is ~1.4%. On these short timescales the shutter timing is at least this accurate.

Longer sequences with bias frames in between the lunar exposures would be good. None of the frames dropped out, but we know that this can happen occasionally, with as much as 20% of the frames dropping out.



Lessons learned.

Control Software Posted on Sep 28, 2011 20:42

Setting the SKE: can only be done from the System_Data_Table – not in a script (the command suggesting that you can do it from a script is not for this purposes).

How to avoid ‘Camera Flatlining’: See the entry under Observing log for JD2455831.



Flat fields

Andor camera field experiences Posted on Sep 28, 2011 20:41

We are examining flat fields – so far they seem to be very peculiar, and time-variable.



JD2455831

Observing log Posted on Sep 28, 2011 19:17

May have learned this important lesson:

The ‘flatlining’ of the CCD camera may be entirely due to the ‘running of two instances of LabView’. This occurs easily, it appears, if you close the VI for imaging while the Engineering Mode (EM) runs – or something like that: the danger is that an instance of Labview is started without you knowing it.
The danger is present when closing VIs. So, to guard against this, have the Task Manager running so that you can see whether you have more than one ‘instance of LabView’ running.

Also, you must have iBoot channel 5 ON when using any of the EM modes for controlling Filter Wheel, front Flap, Rear Flap, Side Flap, underside Flap or any other wonderful device on this telescope!



Summary of data analysis methods

Post-Obs scattered-light rem. Posted on Sep 25, 2011 09:14

For reducing data so that the terrestrial albedo can be determined we have forward and inverse methods.

Inverse methods:

1) The BBSO linear sky-fit method. Yields a cleaned-up DS so that DS/BS ratio can be found and used for the inversion. Probably works best near the edge of the DS as the scattered light is not a linear function of distance except far from the BS.

2) Andrew’s clean-up method based on fitting a model image convolved with a variable PSF until it can be subtracted perfectly from the sky-part of an observed image. As a model of the Moon an observation – minus the sky part and the DS – is used.

Forward method:

3) A synthetic model of the Moon is used within a convolution framework to produce an image that ‘looks just like the Moon and its halo’. Since a parameter in the method is the terrestrial SSA it is directly determined in this way, but does depend on the same assumptions about time, geometry, lunar reflectance and so on as does the inverting methods. Depending on how much of the lunar disc is included in the fit there may be a difference in how the knowledge of lunar albedo and lunar reflectance influences bias.



Two Jupiter PSFs compared

Exploring the PSF Posted on Sep 25, 2011 09:09

Chris made this plot from two observing sessions of Jupiter. The first involved approx. 100 imags while the second (green) involved approx 1700 images.

Comments from Chris: The plot shows the psf around Jupiter. About
200 x 0.1 second exposures were averaged (black dots) and about 1700 a
few weeks later (green dots). T

Black dots: the psf can be traced out to log(r) ~ 1.5, Green dots: the
psf is very similar to the first one and can be traced out to about
log(r) ~ 1.8.

Note well: the background level of the sky has been adjusted be a small
amount — by not more than the uncertainty in the sky itself — to allow
the points at large to fall off with the same power law out to 100
pixels.

The power law fall off of the PSF is ~ r^-3 in the outer regions.

Signs of the non-axisymmetric psf are seen as extra dots above the main
sequence of dots. This is mainly caused by shutter bounce, leaving a smear of photons to the right in the psf.

The clusters of points at particular radii and sticking up are the Galilean
satellites.

The black line shows our current (Sept 29, 2011) best estimate of the psf from forward scattering of ideal lunar images and fitting them to lunar observations. It’s very close to the Jupiter PSF at large radii. This is the reason we think the
psf is falling as a power law all the way to the edge of the frame.

The departure of the inner part of the PSF around Jupiter from the black line is caused by the finite size of Jupiter — about 6 pixels across.

The inner part of the PSF is still not measured anyway. We are working on this!



tau Tauri Occultation July 26 2011

Showcase images and animations Posted on Sep 24, 2011 17:05

http://vimeo.com/27064549

Moon observations during 2½ hours. Performed with Hans Gleisner.



shutter field experience

Shutters Posted on Sep 24, 2011 16:58

The shutter is a bit of a dissapointment! It fails 30-50% of the time, and there is considerable variability, and bias, in the exposure durations. Only for 1s exposures is the scatter less than 0.1% and the bias negligible. We have heaters on the shutter, but apparently not enough. Low temperatures (‘below 5C’) are supposedly not acceptable – yet we see failure at 9-10 C.

We also see ‘dragging’ of the image. [Image needed here!] This used to be due to the shutter staying open after the image started reading out. We see a variation on this: For the overexposed images of various stars blooming occurred in th evertical direction – well, ‘dragging’ of these spurs occurred in the image. This is hardly anything to do with photons arriving on the image during readout but is some issue related to electrons and their transfer from column to column – perhaps suggesting that the previous ‘dragging’ also is related to saturated or highly charge pixels.



JD2455827

Observing log Posted on Sep 24, 2011 16:55

Sept 24 2011: Observed Jupiter with Chris. Did approx. 2000 0.1s V-band exposures. Used a script that contained direct GOTO RA/DEC command, called JUPITER.csv. Co-added the images on ‘woof’. Chris later analyzed the resulting profile.



Poisson nature of detectors

Error budget Posted on Oct 24, 2009 19:29

Understanding the nature of the statistics of imaging devices.

We characterize a detector from a Canon 350D camera.



Optical Density

Error budget Posted on Sep 14, 2009 18:59

Finding the Optical Density of a neutral density filter.

Tests were performed to simulate the practical problem of determining the optical density of a neutral density filter using Moon observations.

Experiment 1: A real image of the Moon was divided by 100.0 over half the illuminated disc, resulting in an image that resembled what an image of the Moon seen through a focal-place ND filter would look like. The image was one that Tom Stone of the USGS donated, from the ROLO project. A synthetic image of the Moon for the time of observation was then generated using the DMI lunar simulator. Two types of images were generated – one using the Lambertian BDRF and one using Hapke’s BRDF. The synthetic image was then rotated and scaled so that it aligned with the real image of the Moon. This procedure included the application of a multiplicative factor over that part of the image which coincided with the ‘filtered’ part of the observed image. The factor was one of in total 6 parameters that were determined, using least squares techniques by minimising the residual between the ‘observed’ image and the synthetic image. Results: The values found for the ‘filter factor’ in the minimisation were 99.57 and 100.332 for the Lambertian image and the Hapke image, respectively. Since the actual factor was 100 we see that we are able to determine the filter OD to within 0.6 and 0.3 %. A second experiment employed a factor of 1000 and we determined 999.57 and 1005.9 – i.e. 0.04% and 0.6%. Notes on the procedures used: The minimisation method requires a good starting guess for all parameters – i.e. image shifts, image scaling, image rotation, image intensity and the factor for the OD of the simulated filter. This starting guess was obtained by first performing the minimisations to align and intensity-scale the images and then fixing these and determining the OD factor and finally letting all parameters be determined from the previous best guesses. To align the images only the image half that was not ‘under the filter’ was used. The residuals’ RMSE is very sensitive to small shifts in the image placements, due to the amount of detail in the images of the lunar disc – small shifts soon place dark areas over bright ones. It is speculated that smoothing of both images before the above procedure is applied could solve this problem. This is analogous to suggesting that the Moon is not observed in focus or at least not in perfect focus. The above was not performed with realistic Poisson noise on the observed filtered half, nor anywhere in the synthetic image. Scattered light from the lunar disc is not modelled in the synthetic image, but could be. Use of the residual RMSE as target of minimisation may be a bad idea – the numbers are small on the bright side of the filter and low on the dark side: Any contribution to the RMSE from the dark side should be weighted higher. Perhaps RMSE of the residual relative to the observed image should be minimised.

Experiment 2: Realistic Poisson noise was included in the ‘observed’ image on the dark filtered side. At first a filter OD of 10 was used, and minimisation recovered 10.04492 and 10.045737 – i.e. 0.4% and 0.5% errors. For larger filter factors initial results were dismaying: a large bias was apparent in the rsults – must consider again.



Cloud Monitor

Mechanical design Posted on Mar 22, 2009 08:33

A Peltier-element cloud-sensor is installed.



Uniblitz shutters

Shutters Posted on May 05, 2008 19:11

Here is the link to the Uniblitz home page – we are thinking of using the LS6 shutter: http://www.uniblitz.com/products/shutters_detail.asp?s=3&ss=6



Linearity of CCD/CMOS detectors

Error budget Posted on Mar 22, 2008 07:31

A discussion of detector linearity: http://turtle.as.arizona.edu/im/mdr16/linearity.html

Here are results of testing the Canon ESO350D linearity (section 1.3):

Nice article about Canon DSLR dark currenta nd gain issues: http://www.stark-labs.com/craig/resources/Articles-&-Reviews/CanonLinearity.pdf



Scattered light analysis

Optical design Posted on Mar 18, 2008 10:34

Here is Mette’s analysis of the BBSO telescope vs. the Lund design, in terms of Ghost intensities and scattered light from front lens surface.



Canon parts catalogue

Shutters Posted on Mar 18, 2008 10:28

This is the Canon EOS 350D parts catalog. The shutter unit is part CG2-1603

Added in 2013: I posted a query online about how to drive this shutter, but … [link]



Error budget version 1

Error budget Posted on Mar 17, 2008 17:14

Here is a first shot at starting to estimate the error budget for eartshhine observations, and the resulting terrestrial albedo. See Qiu et al paper (below) for the theoretical background for the relationship between observed earthshine/moonshine ratio and albedo.

Version 1 of notes:

Qiu et al paper: OK, there is a problem, send me an email instead.



Canon 350D shutter accuracy

Shutters Posted on Mar 15, 2008 20:10

Here are the results of some experiments I made with a Canon 350D to see how accurate its shutter works.