Earthshine blog

NASA recently released a set of images showing the Moon crossing the disc of Earth, as seen from the DSCOVR satellite orbiting Earth in the L1 Lagrangian point. From that point the satellite can keep constant watch on Earth, with the Earth disc always fully lit. NASA image, from the DSCOVR satellite.

This image is interesting because it brings together so many different pieces of information, and possibilities. With the Moon an unchanging object (surface cratering might change on timescales of 100s of millions of years) in the same frame as Earth it becomes possible to very accurately calibrate photometry of the Earth.

Why do we need good photometry of the Earth? Well, weather satellites already give us a flood of images of Earth, but the images – especially from older satellites – are mainly suited for weather forecasting – “Here is a cloud mass, and it is moving this way, so we think it will rain or be cloudy at such and such a place a little later”. This sort of information is of enormous value to society on a day-to-day planning basis, but for climate research the data have to be absolutely calibrated – in other words, it is no good if the pretty pictures of Earth are taken with different instruments that have different sensitivities, or with instruments that slowly alter their properties. The resulting data would not be inter-comparable.

Climate research needs to have data for Earth that tells the story of what happens over time – this is the difference between ‘climate’ and ‘weather’: Weather is tomorrows’ events, but climate is the mean of what things were like 20, 50 or 1000 years ago. If changes in climate can be detected, then we can start thinking about what causes the change. A lot is known already, but every bit of new, reliable information can be put to good use.

Looking at the DSCOVR images we note the cloud masses on Earth – if you have seen the animated film showing the Moon passing across the disc you will realize that the Earth is turning as we watch – so new cloud masses are being brought into view while others pass beyond the limb of the Earth causing variability in the average brightness of Earth. Link to the animation is here.

Using image-analysis software it is possible to pick out the parts of the image that are Earth and the parts that are Moon, and look at the statistics of the numbers. The images are ‘RGB’ images and each RGB-image contains one image for the red light, one for the green light and one for the blue light. Because the Moon is in the image there is the huge benefit that even if the camera settings or properties varied slightly in time then the ratio between Moon and Earth brightnesses in the three colour bands would be independent of these errors or slow changes.

So, with these wonderful properties of the NASA image, how useful is it, in reality?
The amount of clouds on Earth relate directly to the ‘radiative balance’ that drives climate. More clouds mean less light get into the climate system to warm it up. Therefore, variations in amount of clouds will alter the temperature. We all know it is cooler when a cloud draws in front of the Sun, but over larger scales and longer times, how much will mean temperature change for a given change in cloud cover?

Estimates indicate that a 1% change in global mean cloud cover will change global mean temperature by 1 degree C – roughly (could be half that, could be twice that), so if we could be sure to measure mean cloud cover to an accuracy of 1/10th of a percent we could expect to have data that gave us important information in the clouds-vs-temperature question.

In the NASA image the variations in the clouds seen, due to rotation, amount to some 3% in the short segment of images provided. The images are taken about 15 minutes apart from the DSCOVR satellite so in that short film (containing 12 images with a full Moon disc in view) we cover almost a quarter turn of the Earth. Aha, so we now realize that the amount of clouds on Earth varies by several percent from hemisphere to hemisphere and probably day to day. Next day the amount of variability is some other number and some cloud masses have dissolved while new ones have been formed – so on a daily basis cloud variability on Earth is high. Uncertainty is therefore high, as to the actual number of clouds present – weather forecasting may care where the clouds are and how many there are, but climate investigations need more precise (less scattered) data.

If we average many images of Earth the variability would average out and the uncertainty go down. How many pictures of Earth, like the ones here, would you need to have for the average cloudiness to be of use in the clouds-vs-temperature question?
The answer turns out to be that with about 1000 independent images the average would have an uncertainty of 0.1%.

How do we get ‘independent’ images? The phrase refers to ‘statistical independence’ and has to do with how long you must wait for a fresh cloud-scape to be present over the visible disc of the Earth. Weather systems can last several days so you can perhaps get an independent image of Earth every 2 or 3 days.

It therefore looks like data gathered over a decade could produce data for cloudiness that could be used for investigations into climate change mechanisms via the cloud-mechanism. This answer holds equally well for ordinary satellite data except these data have the added problem of instrument drift.

Of course, the Moon only crosses the Earth’s disc as seen from the DSCOVR satellite every so often – you certainly do not get a sequence of images like the above every day – or Month. So is it useless to use these data for the climate-change purpose then?

Well, all satellites have a basic problem – they do not last long in deep space due to radiation damage from cosmic ray particles and outbursts from the Sun, so they get replaced every several years or so. This will probably also have to be done with the DSCOVR satellite, if there is funding for it. The point I make here is that the presence of the Full Moon in the image frame every so often allows a re-calibration of the DSCOVR satellite instruments that give unprecedented accuracy. Geostationary satellites are between the Moon and Earth and do not capture the Moon in-frame very often, and when they do it is as a small object peeking out behind the Earth in a corner of the picture. These chance-occurrences are, however, used nowadays to calibrate satellite instruments, by EUMETSAT and others. Classically, satellites were calibrated on bright white surfaces on Earth, such as salt deserts – this gives about 1% precision and accuracy, while the lunar methods are able to reach a few tenths of a percent accuracy.

Another comment that can be made based on the NASA image is that the Moon appears dark against the Earth. In principle this is well understood because the lunar soil is dark while Earth is full of bright clouds, icecaps and a scattering atmosphere. Desert are also very bright, but forests are dark, as is the cloud-free ocean. With the images at hand we can readily measure how bright the Moon is relative to the Earth in each of the R, G and B colour-bands. The ratio should give the ratio of albedos (reflectivities) of Moon and Earth, because the geometry is the same – Sun is behind the DSCOVR satellite and both Earth and Moon are ‘Full’ at the time the images were taken. I did this and find a ratio about 2 or 3 times higher than the standard ratio expected. The Moons’ albedo is in the range 0.05 to 0.10 while Earth is near 0.3 so the ratio should be 1/6 to 1/3, roughly, but actually comes out as 0.8.

Why is this? I think there may be two reasons – first of all the back, usually hidden, side of the Moon is brighter than the front because there are more mountains and they are bright. The front has lunar Mare and they are darker than the mountains. Second, we are seeing the Moon under Full Moon conditions here and albedo, or reflectivity, depends on the angle of viewing, just as any surface you look along. Near vertical incidence the lunar soil appears brighter than at other phases. This is known as the ‘opposition surge’ and explains why the Full Moon, as seen also from Earth, appears brighter than at other times – not just because a larger fraction of the Moon is illuminated then but because a lit region simply reflects more of the incident light when the incidence and viewing angle is 0 degrees – straight down and straight back out again.

Why does albedo depend on the viewing angle? Any rough surface has shadows behind boulders and mountains and only when the angle of incidence is the same as the angle of reflection will the shadows be hidden from the observer – looking at a lit and rough surface from another angle and the shadows can be seen, leading to a lowered average reflectivity. The lunar also has small beads of glass and crystalline material which contribute to the enhanced reflectivity during Full Moon. Is there no ‘opposition surge’ for Earth? After all the geometry for Earth and Moon is the same here so a surge for the Moon might also cause brightening of Earth? I don’t know yet – but note the pretty ‘sunglint’ seen in the centre of Earth during the animation.

The images used here are not ‘science grade’ but have been optimized for viewing online so transformation of colours have been applied in order to yield a pleasing and realistic image – perhaps the scaling has caused an offset in the brightness ratios, explaining the large Moon/Earth albedo ratio. The raw science data will be released by NASA later, and consists of 10 very narrow band-pass filters.

We might pause to wonder why all this effort is worthwhile? The answer is simple and has to do with reproducibility of observations – we need to have many estimates of all properties of the climate system in order to understand it fully, and winnow out the erroneous data – a point made brilliantly by Ruth Mottram in her excellent blog sternaparadisaea.net.

As they say on Mythbusters “If it’s not reproducible, it ain’t Science!”

A very clever alternative to watching Earth from Space in order to determine its reflectivity, is to stay on Earth, or near it …, and watch Earth from there – by using Earthshine observations! And the rest of this blog is about our attempts to do just that.

This is an image of the Moon occulting Saturn. Saturn is evidently brighter than Earthshine – however, this is a phase near Full Moon so the Earth is near New, and thus faint. Still, eh?
From APOD. Image by Carlos Di Nallo.

For our ‘splendid night’ JD2455945 we can have two estimates of the albedo of Earth:

One estimate is provided by the ‘profile fitting’ where albedo is directly determined as a model parameter of the Sun-Earth-Moon system, and another estimate is given by the B-V determination we have from the cancelling halos.

We get:

From profile fitting albedoB/albedoV: 1.217 +/- 0.003
From B-V: albedoB/albedoV: 1.202 +/- 0.005

The error on profile fitting results is propagated from the standard deviation of several determinations of albedo in B and V and thus is ‘internal error’.
The error on the second estimate is taken from the paper and is ‘internal’ also – the dependency on error of the solar B-V value drops out.

The two estimates are similar but, given errors, formally different.

The fitting result is based on 6 fits in total – three in the B image and three in the V image.

The amount of clouds on Earth changes from day to day, and as clouds reflect sunlight (with B-V=0.64) more clouds cause B-V of the earthlight to drift towards the solar value, while less clouds shows the Earth bluer. How much do we expect the terrestrial B-V colour to change day to day on the basis of changing cloud masses?

Inspection of GOES West full-disc satellite images of Earth shows that the local noon (i.e. disc is fully illuminated) brightness variations due to changing clouds amount to about 3.2% of the mean brightness. As the Earth below is dark (Pacific Ocean) almost all of the brightness is due to clouds so we are not too far off by saying that the amount of clouds varies by 3.2% around its mean value on a daily basis. With Earth on average being almost 67% cloudy we can re-use the Stam models of expected terrestrial spectra to see what the expected changes in B-V is due to cloud variations.

We find that B-V will vary, with a standard deviation near 0.005 around the observed value of B-V=0.44.

This is smaller than the total observational error we have. We have shown that the theoretical Poison-noise limited uncertainty would be 0.005 in B-V, but we cannot observe that well. Yet.

So – we should not expect to see values for B-V very different from 0.44, which helps explains why we get the same value as Franklin’s mean value using just a single (but precise) observation.

We may be able to see 2 and 3 sigma deviations in the cloud cover, however.

The purpose of our telescope was never to observe daily cloud variations – in the long term we hope to be able to qualify that we can set limits to climate-change induced changes in albedo.

Also, the above is just an investigation into whether we can use COLOUR changes to quantify albedo changes – we still have to quantify how well our direct-photometry measurements can see albedo changes. Paper II.

In an interesting paper by Antilogus, et al subtle effects of electron interactions inside the CCD material itself are discussed, which could be a tool for understanding and perhaps quantifying CCD nonlinearity.

We have tested our own Andor BU987 CCD camera for these effects and see them.

Plots of row- or column-neighbour means vs correlations. Top row: first panel: full set of values of means vs column-correlations
– we note the strong non-linearities that set in at about 50.000, and
some outliers.

Top row: second panel: for just the range of mean values below the onset
of non-linearity a robust regression is performed (red line).

Bottom row: same as top row but now for row-correlations.

We note the general correspondence to results in the ‘brighter-fatter’
paper: rows are more strongly correlated with their neighbours than are
columns – the ratio of slopes is about 2.0.

Can this be used quantitatively to correct for non-linearity in our CCD? The camera linearity was already tested for linearity while in Lund. See Figure 8 in the report.

In this entry we started exploring the Stam 2008 set of model Earth spectra. That entry was based on the ‘land only’ set of Stam models. Ocean-only models are also provided, so we can mix these with the land models to get more realistic Earth spectra. In Stam ocean models the albedo is assumed to be 0 apart from a specular contribution, so all colours are dominated by the Rayleigh scattering and a bit of sunlight reflected at the glint point. In land-only models the vegetation red edge has a large contribution, so we expect some changes when a land-ocean mix is introduced. We assume that Earth is 26 % land surface in what follows.

As before we calculate the spectrum of a range of cloudyness percentages. We find that:

B-V for Stam cloud free forest model is: 0.19
B-V for Stam cloudy forest model is : 0.54

B-V for Stam cloud-free ocean model is : -0.23
B-V for Stam cloudy ocean model is : 0.53

Note how similar the 100% cloud cases are for land-only and ocean-only, as expected. There is a small effect of the surface below the clouds (which is one of the features of the Stam set of models – others have ignored this effect).

Note also how blue a cloud-free ocean only Earth would be – this is, I expect, the B-V of our own blue skies on a cloud-free day, as we have discussed elsewhere. I think we should try again to measure blue-sky B-V with our DSLR calibrations.

As before we estimate slopes of the B-V vs cloud % relationship.

Change in % cloud cover per mmag change in observed B-V: 0.31 %pts/mmag
Change in % cloud cover for total observed error in B-V: +/- 6.3 %pts
Change in % cloud cover for internal observed error in B-V: +/- 1.6 %pts

The total B-V errors we have are +/- 0.02, mainly due to uncertainties in the Solar B-V as pointed out by Chris. If we leave all measurements relative to the fixed (but somewhat unknown solar B-V) we have internal errors of just +/- 0.005 mags and that implies we could determine Earth’s cloud cover to within something like +/- 1.6%-points on the basis of observed colour alone. Not bad!

With a technique for observing B-V on all nights we could thus complement direct albedo-determinations (found with edge-fitting for instance). The colour-method has drawbacks and benefits relative to the direct albedo determination in that a difference is used instead of an absolute fit.

This is a point towards why we need to understand how the halos are formed and just why they cancelled on that night!

It is also fuel for fixing the SKE because with an SKE more images could be obtained where the remnant halos in B and V almost cancelled on the DS.

With the CCD working again, we have some hopes of understanding the SKE problems, as well as performing some very careful lab observations with halo due to optics only.

Stam (A&A 482, pp.989-1007, 2008) has provided spectra of model Earths. From these we can construct B-V colours and compare to our observations.

Using the F0 and F1 model coefficients from Stam (cloud free and 100% cloudy, respectively) we combine these linearly as a function of cloud-cover fraction, and apply Johnson B and V filter transmission curves, and calculate B-V.

We check the method’s correctness on the Wehrli Solar spectrum (we get B-V=0.64) and for Vega (we get B-V=-0.014). Both are very close to the accepted values, so we trust the method for other spectra.

We generate a set of models going from cloud free to 100% cloudy, and get:

Is it just my eyes or is this a really lousy rendition? I wish the blog software allowed for better pictures! Anyway, what we (almost) see is that the observed B-V for earthlight corresponds to about 40% clouds on Earth, with our observational uncertainty of +/- 0.02 mags corresponding to quite a wide range of cloudiness: +/- 8%.

Colours are thus not the way to determine cloudiness on Earth!

Conversely – the variations in cloudiness on Earth is considerable from day to day (at some phase angles +/-20%) so we should be able to use colour variations as an indicator of cloud variations, along with the direct albedo measurements.

The Stam model is based on assuming a Lambertian forest surface; Earth seen at phase angle 90 degrees, and polarization ignored.

Our (considerable) errors on observed colour is due to the conversion from earthshine colours to earthlight colours and the problems of taking the effect of the lunar surface into account. Our observational error on earthshine colour is 0.005 mags. This would allow us to determine Earth’s cloud cover to about 2%-points from B-V colours alone.

A telescope in space (or on the Moon) looking directly at Earth could have colour errors that were even smaller.

The model has to account for orientation of Earth and so on, of course. This is not the case in the Stam model.

We use synthetic models of the Moon, illuminated by a model Earth, to generate fits to our observations. The properties of the theoretical models determine not only the fit itself but also the quality of the fit. Two important choices are made in generating the theoretical models – the lunar
surface albedo map has to be chosen, and the lunar reflectance model has
to be chosen.

We can choose surface albedo either directly from
the Clementine map, or as this map scaled visually to better match the
1970s albedo map by Wildey.

We can choose the reflectance models
in several ways – current models being tested are ‘the new Hapke 63’ and
the ‘Hapke-X’ reflectance model.

We test how well these choices work by considering the RMSE of the ‘edge-profile fits’
generated between the theoretical Moon models and the observed ones.
Combing the two choices above in all possible wasy we get these results:
There are four histograms here but pairwise hide behind the other. The major effect is that due to the choice of albedo map (blue vs green above) – the scaled Clementine map has 25% lower RMSE than does the un-scaled map.

The effect of reflectance model is very minor and can barely be seen

The RMSE is calculated from the profile of 20-row averaged ‘cuts’ at the dark side edge, starting 50 pixels from the edge and ending 50 pixels onto the disc. Fitting was performed vith the MPFITFUN routine in IDL, using a 4-parameter fitting model where intensity offset, horisontal pixel shift, terrestrial albedo and PSF-parameter ‘alfa’ were varied to obtain the optimal fit. RMSE was then calculated from the residuals between observed edge-profile and model same.

The Clementine map was scaled (see http://earthshine.thejll.com/#post253 ) against the Wildey map in the sense that the lower-resolution grid from the Wildey map was interpolated by the Clementine map and the scatter-plot generated. An obvious offset and slope difference existed and from the robust linear regression of one onto the other a scaling relationship for the Clementine map pixel values was generated and applied to all pixels in the Clementine map.

So, we may be able to help constrain the sort of work current space-missions do! We should not avoid to point this aspect of the work out, in the ‘big article’.

Puzzling, at first, is the almost complete absence of an effect due to the choice of reflectance model. I can see effect clearly when total flux vs phase angle is plotted, so we simply do not have a large sensitivity to reflectance at the edge of the disc. Fair enough, only a local set of pixels are involved and the flux of the model bay be quite wrong while the fit at the edge is till very nice. So we should constrain our models by fitting many different aspects of our data – both the reflectance model against observed total flux as function of phase, and the albedo map by edge-fitting.

The subject of Clementine vs Wildey has been discssed in this blog before. Here:

http://iloapp.thejll.com/blog/earthshine?Home&post=304

and

http://earthshine.thejll.com/#post253

Hasinoff et al have (link) shown how images of the DS edge can be used to invert for the illuminating light field (i.e. earthshine) and receive low-resolution maps of terrestrial albedo distribution.The pdf file is here:

Some notes on this:

1) The edge pixels that can be observed from Earth are differently illuminated because the source of light is extended and at a slight angle with the observer’s position.

2) Sally Langford described a method to find the edge of the lunar disk by using the Laplacian operator on the image and in essence do edge-detection. This method will be influenced by the fact that the edge is not uniformly lit. This applies, in principle, to our edge-fitting method too – except we have so low resolution that only 1-2 pixels at the edge show the effect. We do fit these – perhaps we should not.

3) We should be able to test the method Hasinoff et al describe from purely archival images found on the net – or we could try to get our own images (with the eyepiece projection method it should be possible to get high-magnification images of the DS edge). Or with the help of some willing amateur astronomer.
Or by applying for ‘service time observations’ at some large telescope.

In the last plot in this entry http://iloapp.thejll.com/blog/earthshine?Home&post=375 we found the rise in fitted albedo unreasonable. Let us review what we have:

1) The albedos are derived as a scaling-parameter inside the Earth model we use. This model assumes the Earth is a Lambertian sphere with uniform albedo.
2) We use Hapke 63 to describe the reflectance of the Moon, and Clementine albedos for the lunar albedo.
3) We fit the DS edge – adjusting parameters in the synthetic lunar-image model so that a good fit is found between observed DS-edge profile and model DS-edge.
4) We see a 30-40% rise in albedo as Moon approaches Full (and Earth New).
5) We see a much weaker dependence of time-of-day in albedo. From considerations of the Opposition Surge (which may be inadequately represented in Hapke 63) we expected a drop in albedo towards end of day, and a rise at the start of day. We do see weak signs of the drop at the end of the day, but at the start it is ambiguous.

5) makes me think the problem is not the Opposition Surge in Hapke 63. 4) makes me think it is the total flux of the synthetic models that depend incorrectly on lunar phase. 1) makes me think that perhaps the Earth is not very Lambertian and that the discrepancy becomes much worse, particularly towards the crescent New Earth (Full Moon).

If scattered light is incorrectly included in the fit at the edge we would find a larger fitted albedo as lunar phase approached Full Moon, as we do – but there is NO WAY the fitting method is making a 30-40% bias-like error – it would be blatantly obvious when the fits were inspected.

What’s going on? I would like to compare the observed run of total fluxes against the model-based expectation.

Tests:

First we fit the edges of realistic but synthetic lunar images (made with Chris ‘syntheticmoon.f’ code):

Here models with albedo=0.30 have been edge-fitted – just like observations. We see that the edge-fitting method is completely able to retrieve terrestrial albedo.

So, there is a problem of some sort with the total flux of synthetic models. Let us look directly at the total flux from the synthetic images:

Here are shown (black) lunar model image magnitudes (adjusted to just about the right V magnitude at Full Moon) against lunar phase [FM=0], and expressions fro the same from Allen (red: 1955 edition, orange: 1975 edition), and from the JPL ephemeris system (green). JPL is based on Allen 1975 but adjusts fro actual distance to the Moon. Only ou rmodels allow for libration – hence the East-West assymetry in the black symbols.

It seems our synthetic images have a smaller range in total flux against phase than do the Allen and JPL expressions. By 1.5-2 magnitudes for some phase ranges, relative to Full Moon flux – or Full Moon is less bright by 1.5-2 magnitudes than are the Allen expressions, relative to intermediary phases. Our Moon images, relatively speaking, lack flux as we approach Full Moon.

In normalized-image edge fitting this will cause the model expression that is fitted to have been scaled by too small a number, making the step-to-sky at the DS edge too large, so albedo should compensate during the fitting by adjusting down. We see the opposite …

Could it be that our near Full Moon albedos are all right? But they are so large – 0.4 and more at small phases: but what do we know? Perhaps we are making a relative error in estimating what the ‘answer should be’?

In this entry: http://iloapp.thejll.com/blog/earthshine?Home&post=372 we looked at the evolution of earthshine phase angle as a function of time of day. This is an important subject for us because the lunar surface reflectance function has an ‘opposition peak’ which results in the infamous Opposition Surge near Full Moon.

Our observations have sampled the peak from just under 1 degree to about 1.5 degrees. From the plot in the above post it is clear that at JD start (noon in Greenwich, midnight at MLO) we sample the peak in the direction towards the peak (large to small phase angles). At the end of the JD we are sampling away from the peak (small to large angles).

The two situations are represented by Westerly and Easterly observing directions – Moon setting and rising, respectively.

There is a consequence of this ‘daily sweep’ of the range near the OS: as our reflectance model – Hapke 63 – does NOT contain the right OS we will see a rise or fall in the fitted albedo.

On the basis of the above argument I am making a PREDICTION: when we look at the fitted albedos we should see them INCREASE at the start of the day (because phase angles are dropping and to compensate the lacking reflectance the albedo has to go UP), and at the end of the day the fitted albedo will DROP.

Next, is a plot of the V-band albedos and the time of day:

We see albedos plotted against (JD mod 1). For each day one colour has been chosen, but some colours had to be reused. Each day has sequences of 3 points – this is because three fits of the model image to the observed image is performed.

To the left we see no slope, or perhaps a little rise towards the end now and then. To the right we see a general slope downwards. As predicted.

If the reflectance was correctly representing the Opposition Surge there should be no change in albedo as the day passed. We may be able to model the OS so that we can get the slopes to go away thereby determining the OS observationally. Troublesome will be the absence of a clear slope to the left, though. Hmm.

The large difference in albedo from day to day is mainly due to a phase-dependency in the determined albedos. The variations in terrestrial albedo are more subtle. We need to first understand why the fitted albedos depend relatively strongly on lunar phase. It could be due to problems related to model-fitting (although I doubt it since we essentially fit the step-size at the edge of the DS disk and this cannot be corrupted by scattered light). It could also be due to an incorrect reflectance function representation in the model used. This would not be the same dependence as discussed above related to near-opposition angles. The phase dependence I speak of is shown in the second row of panels below (the first being a repeat of the above plot):
Above (second row of panels) each ‘strip’ is a nights observations. V-band albedos only shown. In this representation New Moon is towards the sides and Full Moon is at 0 degrees.

We use Hapke’s 1963 formulation for the reflectance function in making the synthetic models that are at the heart of converting observed earthshine intensities to terrestrial albedos. H63 has a part, at small phase angles, that correspond to the reflectances for the DS – on the BS, phase angles are large as the source is the Sun and the observer sits on Earth – but for the DS the source is the Earth. Seen from the Moon, the Earth is about 2 degrees in diameter – just what is the effective source-Moon-observer (i.e. phase) angle? For our roughly 500 observations we calculate the effective DS phase angle from geometry and a simple model of the sun-lit part of the Earth based on NCEP cloud cover, and geometry. The intensity-weighted distance between MLO and each sun-lit pixel is summed so that the effective photo-centre distance, in degrees as seen from the Moon, is arrived at. This is the plot:

Photo-centre phase angle for DS observations plotted against time of day. No correction for actual Moon-Earth distance have been performed; a standar distance of 384000 km has been used. Each linear sequence of points corresponds to an observing night, showing the evolution of phase angle as Earth rotated.

We see our observations distributed in a characteristic way: most observations early in the JD have decreasing phase angles; most observations at the end of the JD have increasing phase angles. The median phase angle is something like 1.8 degrees, with some as low as 0.5 degree and some as large as 1.6 degrees.

Since MLO is on Hawaii, opposite Greenwich in longitude, the JD change occurs for some observing nights in the middle of a set of observations – so the above plot should really be plotted from -0.5 to 0.5 instead of from 0 to 1.

We see that we are sampling H63 at reflectances near the ‘opposition peak’ and that the range of phase angles we have may sample enough that the errors H63 has in representing this part of the true reflectance function could be important in our work on reducing observed intensities to terrestrial albedos. We ought to have available several reflectance formalism so that we can see the importance during data-reductions. Hans has provided such additional formalism in the synthetic code, but is is complex to use (for me) so we must make an effort to get used to the new code, for the ‘large paper’.

Since we are calculating absolute calibrated B and V magnitudes (on the ‘lucky night’ 2455945) for the DS and the BS we can convert these to surface brightnesses, for comparison with e.g. Pallé et al published work.

The formula for surface brightness is

mu = mag + 2.5*alog10(w*w*N)

where mag is the magnitude determined from a patch containing N pixels, with each pixel covering wxw arc seconds. In our case
w=6.67 arseconds/pixel and
N is 101 and 113 for the 6×6 selenographic degree patches we use (+/-3 deg). With the magnitudes for B and V, BS and DS from the paper – but using N=1, since we report average magnitudes per pixel, we get:

DS:
mu_B = 14.29 m/asec²; SD=0.06
mu_V = 13.54 m/asec²; SD=0.06

BS:
mu_B = 6.21 m/asec²; SD=0.05
mu_V = 5.31 m/asec²; SD=0.06 (all SDs are internal error estimates based on pixel bootstrapping with replacement).

BS is about 8.23 magnitudes brighter than the DS – there are published numbers for these quantities (e.g. Franklin). His Table 1 has differences of about 10 mags between DS and BS. He may be talking about magnitudes per area – not magnitudes per pixel, like we are.

Pallé et al in this paper
http://adsabs.harvard.edu/abs/2007AJ….134.1145M
give plots showing mags/asec² and for the phase we have (about -140 on their plots) they have a BS-DS difference of 8.4ish mags – so we are within 0.2 mags which seems possible, given the scatter they show in Fig 1.

Not sure I like or understand why Franklin is 1.5-2 mags different in the DIFFERENCE – would that come about when you differ by mags/pixel and mags/area?

In this entry we introduced a Monte-Carlo based model for the spectrum of earthlight. We are able to use the model to estimate B-V for outgoing light, given input for cloud albedo, surface albedo and so on – and cloud fraction.

From the NCEP reanalysis project we can take the global total cloud fraction product (‘tcdc’) and calculate the global mean value for different times (we pick year 2011 here) and use the values for the B-V model.

I have done this. In 2011 TCDC varied between 50 and 55 percentage points. delta(B-V) [i.e. the differences between the solar spectrum B-V and the earthlight B-V, in the model] varied from 0.090 to 0.075 at the same time. That is, earthlight became more solar coloured for larger cloud cover – good – and implies that Earth is bluer when there are fewer clouds.

The change in B-V is 0.015 mags – can we measure that at all?

In our little paper we have relevant results. Uncertainties on B and V are at the 0.005 to 0.009 mags level, so that differences are at the 0.012 level (worst case) for a B-V value – on the ‘lucky night’.

But on the Full Moon night of 2455814 we have errors of just 0.001 and 0.002 on B-V.

We need to understand why the error can be so different. We promised 0.1% accuracies at the start of this project – and seem to be able to get it with 0.001 mags error – but why not on both nights?

Obviously, the error on the DS will be larger since there are much fewer photons – but this does not help explain why the errors are similar on the ‘lucky night’.

One point is that on 2455814 we measure B-V from one image generated by alignment of the B and V images in question. On 2455945 we measure B and V in seperate images and take the difference of those means. There are ‘cancellation’ issues at play here – surface structure will add to the variance of single -band images while some of the structures cancel (particularly if the images are well centred) in difference images.

Added later: yes, there is an effect (obviously, duuuh) – if we calculate the DS-BS difference from areas in the B-V image, instead of in the B and V images seperately, and perform bootstrap sampling on the pixels involved, we get the mean over the resamplings and its standard deviation to be

-0.155 +/- 0.005

whereas the difference between <B> and <V>, and its error calculated from error propagation, is

-0.154 +/- 0.014.

That is, we have a third the uncertainty. This will be used in the paper.

The uncertainty is still a bit high, though. (5 times what we promised, or to be fair about 5/sqrt(2)=3.5 times. since the above is a difference and not a directly measured quantity – which was the thing about which we made promises!).

It could be that noise on the low-flux DS is dominating here – this remains to be seen. And we still need to understand why it is still higher than for the Full Moon night – but things are making a bit more sense now.

I have looked through all suitable images close to full moon (i.e. at high Sun-Earth-Moon angle) and only came up with a few relevant nights, all within 10 degrees of full moon. We were keen to make a colour map across the face for figuring out what colour range is there, and what colours are reflected under BS sunlight, rather than DS Earthlight.

JD2455814: is the best full moon data available — it’s shown on the left. We ran it through our colour pipeline and get a B-V ~ 0.9. The B and V images turn out to be slightly different sizes, which made the colour map hard to produce. Later it has turned out that this may be due to distortions introduced in the shift to align the images at all. Work is proceeding on this.

JD2455905: data taken when we were following an eclipse — Dec 09, 2011, total lunar eclipse. No non-eclipsed images, so nothing useful on the Brightside colour night (we should check the Moon is red though).

JD2456082 is also eclipsed — partially. No useful colour data as we have only eclipsed images. June 3, 2012

JD2455847 looked like a promising night — full moon at an airmass of 1.54. However, when I make a colour map (right panel above), it has a B-V of ~0.4 across the face. Can only assume that the exposure times are wrong in the headers (as has happened occasionally). One corner of the moon is slightlty eclipsed for
some odd reason in one filter only (V) — opposite Tycho. Perhaps we hit
the dome in this filter? These data should clearly not be trusted further.

We have noted interesting behaviour of our data at the lunar limb, link here. It seems that our data show a convergence of intensity ratios, between certain symmetric points on and near the limb, towards unity. We saw the same in model data.

The model we use is based on the Hapke 1963 BRDF (i.e. reflectance) model and looks like this:

BRDF ~B(phase)*S(phase)* 1/(1+cos(e)/cos(i))

where e and i are the angles of emission and incidence. For a given instant the phase is the same everywhere (almost) so the only angular dependency that remains is the last term above. On the limb we can have various values of i but only one value of e – namely pi/2 – recall that i is the angle between local normal vector and the Sun, while e is the angle between local normal vector and the direction towards the observer.

cos(pi/2) is 0, of course, so the last term above is unity everywhere on the lunar limb. The ratio of the reflectance in two points on the limb therefore reduces to unity. The ratio of two limb intensities should therefore be equal to the ratio of the albedos at those two points.

So, there is no clue here as to why we observe, and the model gives, intensity ratios near unity along the limb. Mystery remains!

Our observation of unit intensity ratio certainly is consistent with Minnaerts reference to Schoenberg’s statement – namely that the ‘intensity along much of the bright limb is a constant value’. Intensity ratios between limb points would give unity.

In 1941 Minnaert published a paper about lunar photometry – in particular the reflectance. There is an idea in that work which we can use to view our data, and perhaps learn something.

The intensity we observe in any pixel of the lunar disk is:

Intensity ~ albedo * reflectance(i,e,alfa)

Where ‘albedo’ is a measure of the darkness of the material in the pixel and ‘i,e,alfa’ are angle of incidence, angle of emission and lunar phase, respectively.

Consider now two points A and B on the lunar disk placed such that iA=iB and eA=eB. Furthermore we have that the phase, alfa, is almost constant all over the image (alfa is the angle between a point on the Moon and the observer and the Sun – and apart from the finite size of the Moon compared to the distances to Sun and Earth this angle is the same for all points on the image of the Moon). In these two special points we thus have

reflectance(iA,eA,alfa) = reflectance (iB,eB,alfa) so that the ratio of observed intensities from A and B are simply:

I_A/I_B = albedo_A/albedo_B

By finding many pairs like A and B we could investigate how ratios of albedos across the Moon vary – or compare the ratio from observations to our map of albedo – at the present “Clementine scaled to Wildey”.

We now do just this!

From the synthetic image software code that Hans wrote we have, for every pixel on the model image, its angle of incidence and emission. We can therefore select an arbitrary point A on the BS and find its symmetric point ‘B’ also on the BS, extract the observed intensity ratio A/B as well as the same ratio for the ideal model image and tabulate these against solar zenith angle (i=SZA) and earth zenith angle (e=EZA). We have done this for 1000 randomly distributed points and show the results here:

Top panel shows observed (black) and model (red) A/B ratios plotted against EZA. Small EZA (in radians) correspond to points near the middle of the terminator on the disk while large values correspond to points closer to the limb, near the sky. Second panel shows the same plot but now using solar zenith angle (also in radians) as the x-axis – small SZA correspond to points near the sub-solar point, in this image near the limb, while large SZA correspond to points near the terminator where the Sun is low in the sky as seen from the Moon. Bottom left plot is the ideal ratio against observed ratio. Green line is the robust regression line and blue line is the diagonal. Bottom right is a small image of the Moon with points A and B plotted as white dots.

We emmidiately notice the tendency for observed ratios to be larger than ideal ratios – at least for the median to large values of the ratio: the observed ratios lie above the diagonal line in bottom left panel.

The observed ratio has a larger span not just because of ‘noise’ – careful inspection of ‘slices’ across e.g. dark Mare [not shown here] reveals that observed A/B ratios reach more widely separated extremes there than does the ideal ratio slice across the same slice.

This seems to imply that the albedo map we use is too ‘flat’ – it should be scaled so as to give more extreme darks and lights by perhaps 25%, judging from the plots.

We also have to wonder why albedo ratio is a function of EZA and SZA – nearer the terminator or middle of the disc ratios are simply higher than near the limb. As intensities are smaller there while brighter near the limb we wonder what is going on – remember that the ratio is reflectance-independent by construction! Note also that both observations and models behave in this way. It cannot be ‘nonlinearities in the camera’ since the model behaves in the same way, nor ‘scattered light in the observations’ since the scattering-free model behaves in the same way. The model is also assuming a spherical surface – no hills, boulders or crevices for shadows to hide in.

Here is a repeat of the above for a different lunar phase:

Similar features are seen here – towards small SZAs the ratios go to 1. As before the general picture is that variations observed from dark to light are larger than in the model. This reminds us that we are using a fixed albedo map in the synthetic model – we can expect different dark/light ratios in different wavelength bands. The above scalingmay sereve to achieve this?

We shoudl repeat for many examples from each band and each lunar phase to see which features are general.

There is a comment by Minnaert in, “Planets and Satellites” ed. Kuiper and MIddlerhurst, Vol III. p. 222 to the effect that the “bright limb shows the same brightness over 3/4 of its length and that this was noted by Schoenberg (1925) and this was only rarely checked” – wonder if that last remark is still true? We can certainly confirm that the albedo-ratios tend towards 1 near the limb, but we do not understand yet if these two observations are linked. Time to read Schoenberg!

Some of the notes discussed below, in the next several postings, are collected in this doc:

There are some considerations of how how much albedo change we expect during global warming – and the detectability of such changes are discussed.

No.

But we – and BBSO – calculate albedo by comparing the earthshine measured on the Moon to the intensity of earthshine predicted by a terrestrial model based on a uniform Lambert sphere.

We test how well this works by taking a series of GERB satellite data for several weeks across a year and extract the total flux from the whole-disk images. The MSG satellite bearing the GERB instrument floats over lon,lat=0,0 so always sees the same part of Earth. Johanne has extracted images for every fifteen minutes for several weeks in a year. We plot that (top panel, below).

We use the eshine synthetic code Hans wrote to generate Lambertian images for a full cycle of sunrise over earth. As the code is based on what the Earth looks like from the Moon we pretend that one month is like one day and thereby can extract the phase law for the Lambertian uniform-albedo Earth in order to compare it to the GERB data. We also plot that (second panel, below). [Note that the difference between view from satellite and view from Moon may be important: Sun-Earth-Viewpoint angles should be the same, and if the Sun and Moon are not at similar latitudes as Sun and Satellite we could be generating artefacts in what follows: we should see if we can use the synthetic code for satellite viewpoints. For now we ought to find dates when the Moon was at latitude 0 (like the satellite) and the Sun also same latitudes – tricky to do.]

Lastly we divide Gerb fluxes by Lambertian fluxes, correct for the fact that Geostationary orbit and the Moon are at different distances and multiply that corrected ratio by the uniform albedo used in the Lambertian models. We plot that (bottom panel, below).

We see that the albedo does not come out constant. This is not surprising since the Earth has real clouds that drift around – but that is only what gives the thickness of the thick line of points in the last plot above. The ‘wiggles’ are due to the inadequacy of the Lambertian model. Near New Earth (Full Moon: never observed) the derived albedo rises. Near Full Earth (New Moon: attractive to observe due to strong eshine, but difficult due to Moon close to Sun) the albedo is flat. At intermediary values (20% and 80% of the cycle) the Lambertian albedo is relatively high so that the derived albedo is lowered.

How can we use these insights to understand what Johanne shows in plots of how derived albedo evolves during the nights?

Our aspiration is that the above can give insight into

1) the ‘phase dependency’ we see in derived albedos when we plot all data corresponding to all phases during the morning branch – i.e. Moon setting over Western Pacific/Australia, and

2) the nightly tendency to have falling albedo through the night, for that same branch.

As for 1 the reader should look in the May 3 presentation at slide 22; as for 2 the reader shoul dlook at slide 23.

We have to figure out whther the above plots explain any of these sightings. Could the almost quadratic phase dependency seen when all data are plotted be due to the ‘dip’ near 20 and 80% of the cycle? Could the ‘nightly slopes’ be due to the same?

[edited versions:] We have discussed many ways of extracting albedo from our data. We first considered the ‘ratio over ratio’ methods – they consisted of extracting counts from patches on areas of the scattered-light cleaned images in the DS and the BS – the ratio of DS/BS observed to DS/BS model is proportional to the albedo of the Earth. The other method, used more recently, is based on ‘profile fitting’ near the DS sky edge.

We have now arrived at comparing the 5-color albedos derived from these two methods:

Right: Albedos for positive and negative lunar phases (Full Moon is at 0 degrees) from the DS/BS method where the “BBSO log” method has been used to remove scattered light. Left: Same but for the profile fitting method. Note different
vertical scales on axis. The same nights were considered for both
plots, but not all are present in both, due to outlier removal, etc.

We see, for both methods, a rise in the derived albedo as phases nearer Full Moon are considered. This is possibly due to effects of scattered light from the BS which has been incompletely handled by the respective methods. The values found with the two methods are quite similar – apart from increased scatter and less colour-separation in the DS/BS method. The data for the positive and negative branch of the phase diagram are not similar, in either case.

From tests shown elsewhere in this blog we do not expect the halo to interfere with the DS for large absolute lunar phases – i.e. near new Moon. The above diagrams shows lowest values for phases near 110 degrees. There is a slight increase in values larger than this – what can that be due to, if the halo is interfering less and less? Well, we must again remember that the above results are model-dependent and the Model may be adding its own fingerprint. For instance, it may be that the synthetic images we use have a phase-dependent error in their representation of lunar and/or terrestrial albedos. Note that the same synthetic models are used in the DS/BS method as in the profile-fitting method.
This question can be addressed by studies of the effect on the ‘bend’ seen above of different BDRF models.

In posts below we have discussed how to best investigate colour differences. Here we saw that sky brightness and exposure time problems can be detected.

Using selected good images in B and V we found the pairs that also were close in time, and generated B-V images. We noted (discussed here) that B-V on the BS is not always near the published value of 0.92, even in images selected for not having (obvious) exposure time problems. We wonder if the value 0.92 is more of a classical photometer value? That the colour of the whole BS on average is 0.92? Perhaps – but we also wonder if the reflectivity of the Moon has a phase dependence so that the B-V colour, even if a BS average, is lunar phase dependent?

Here, we choose to bring the B-V value of the BS in our selected images to 0.92 in order to study which values we get for B-V on the DS. This in an attempt to see if we can discuss what the B-V color of the DS, and therefore of Earth is.

For a range in lunar phases equivalent to illuminated fraction from 35% to 50% we plot the B-V values of a slice across the lunar disc, through the centre and 40 rows wide. We average over the rows:

We have aligned the images by the deep cut, which corresponds to the BS/DS border – the terminator. On the right of this we have the BS and on the left the DS.

We see that the BS is level. We have offset the image values so that the BSs are near 0.92 (by eye). We see that the DS has a slope. We see some level differences in these slopes but the slopes themselves are fairly similar. For one of the profiles the B-V reaches as low as 0.5ish, but there is still a slope.

On the basis of that I think we ought to say that “B-V for earthshine is at or below 0.5 in absolute terms”. Better may be to say that “the DS B-V is 0.42 below the BS B-V level.”

The B-V os sunlight is 0.656 ± 0.005 (Chris has measured a similar value of 0.64). If we also know that the sunlight on the Moon appears to us to have B-V of 0.92 then we can infer that one reflection from the Moon reddens the sunlight by (0.92-0.656)=0.264. The Sun also shines on Earth (lucky us!) and that light has its colour altered by the Earth – when it strikes the Moon and comes back to us it has B-V of 0.5 or less. Knowing what one reflection off the Moon does to the B-V color we infer that B-V of Earthshine is 0.25 or less. Franklin performed UBV photometry [with a ‘diaphragm’ of diameter near 1 arc minute – it means he observed areas on the DS, preventing the BS light to enter the photometer, but not BS-scattered light from optical elements before the photometer] on the Earthshine and found that B-V ‘for Earth’ was 0.17 below B-V for the Sun – this implies that B-V for Earth is 0.47. We have a value (from our lowest value) slightly above that.

Some notes:
1) The above slope is rather straight. The halo itself is there because the halo from B and V have not cancelled. This must be telling us something about the scattered light halos? Probably that it is linear in a lin-log plot – which is what we use using the ‘BBSOlog’ method. What would Ve1-VE2 images look like? We cannot tell since the halo is not (primarily) due to atmospheric scattering and Mette knows no rule for how scattering in lenses depends on wavelength. We have to try, before we know if Vegetation edge data can be extracted in this way.
2) We also see that we have no images without a B-V gradient across the DS. If we had had an SKE we might hope to achieve ‘clean’ B-V images. But we don’t. So we can’t.
3) All is not lost – when we fit images profiles directly we compensate for the halo and can therefore extract actual albedo data that does not depend on the presence of the halo.

To check consistency we now look at V minus V images, where the two V images are chosen to be closer in time than 30 minutes but not taken at the same time. We correct for extinction; we convert the raw images (bias subtracted, of course) into instrumental magnitude images by calculating the flux from the nominal exposure times and taking -2.5*log10, and then align them and subtract. We shoul dget images of 0s since the fluxes, once corrected for extinction, should show the same flux – at least on the BS where the Sun is shining. We do this and get a royal mess:

In each frame the insert shows a color-contour plot of the V-V image. The graph shows the usual slice across the middle of the image, averaging over 40 rows.

We see in upper left panel a clear offset at the terminator – i.e. the DS has different level but the BS are similar. Upper right shows a failry decent pair of images – the terminator is giving some problems and the DS as well as the BS differences are offset from 0 by a small-ish amount which coul dbe caused by an error in exposure time of some 10% or so (not unlikely). Lower left shows that while the two BSs are at teh same level then the DSs differ violently. The lower right shows a really nice example of two images agreeing.

What is going on? We hand-inspect the above images and see that in the case of the lower left image the sky level is much higher in one of the two images used, although they are observed less than 30 minutes apart. The counts inside mare Crisium are 3 and 11 in the two cases – i.e. a factor of about 4 or a magnitude difference of 1.5 – well outside the plot frame. So, from this we learn that we have a method to detect stable sky conditions! It also tells us that using images from different filters should be done with great care – even small differences in sky conditions will cause the DS to shoot off!

As it is, these images are merely bias subtracted – there is no individual correction for a ‘pedestal’ due to sky conditions. Luckily we can still use these images for albedo work in that we expressly fit a pedestal term!

What else can we learn? Well, any error in actual exposure time will influence the DS and the BS with the same factor – hence the magnitude differences plotted above will become offsets for both DS and BS – hence, upper right is consistent with ‘wrong exposure time’ in one of the images. Having assigned one filter and received another will have the same effect as an error in exposure time – if the DS and BS are altered by the same factor. I wonder if ‘wrong filter’ could camouflage as ‘wrong exposure time’? Since DS and BS have different colors I doubt it – but we should investigate this.

So, three things learned:

1) Wrong exposure time will lift or depress DS and BS by same amount.
2) More sky brightness in one image than in the other – affects DS only.
3) Image pairs like the ones used for lower right panel, above, are probably both OK.

I have located all ‘good images’ in B and V. That is, all B and V images, made from stacks of 100 images, that are on Chris’ list of ‘good images’. I have furthermore identified all pairs of B and V images that are taken less than 1 hour apart. For all these pairs I calculate a B-V image, by using Chris’ calibration of flux against known standard stars. I allow for an extinction correction based on kB=0.15 and kV=0.10. I get airmasses from the Julian date of the image, and IDL software. I plot a colour contour plot as well as a ‘slice’ across each image. The slice is made up of an average of 40 rows centred around the row that goes through the centre of the B-V image. A total of about 33 image pairs have been plotted in this way. Here is a (large) pdf file containing the plots.

There are several strange things to see. Generally we get the profiles shown here. That is – the BS is somewhat flat, while the DS slopes towards the DS sky. Oddities include images where the DS is as flat as the BS – just at another level. Is that ‘extremely good nights’? I think not. In some, the DS is much higher than the BS – that could be images in which the exposure time is incorrect due to shutter problems. Or filter-wheel problems! In many the BS is flat, but not near the 0.92 value we expect based on publications. For small differences with this ‘canonical value’ I think we could be talking about exposure time uncertainties. If we are at B-V=1.0 instead of at 0.92 we could have an 8-10% error in the exposure time. The times are short and it does not seem unlikely we have that big a problem. Sigh.

I will suggest that we ‘correct the BS value’ to 0.92 by simple shifting, and then consider how our DS B-V values look. With some ‘deselection’ of some obviously bad images we can perhaps arrive at a base set of good B-V images.

I think the result will be that few show any leveling off in the ‘slope of B.V wrt distance to BS’.

We have few images at small lunar phase (i.e. small illuminated fraction). Chris has analyzed one good pair of B and V images at small phase, but it is not in my pipeline of good images – the automatically detected centre coordinates and radius are way off (because the sicle is so small that automatic methods do not work). Our new student Johanne is inspecting these images by hand and will adjust radius and centre coordinates and then we should have a few more images to play with at small phase. These may show us if we can get far enough away from the BS that the DS slope in B-V levels off and shows the true earthshine B-V color.

[Note added later: we have updated this posting using extinction corrections for B and V. kB=0.15 and kV=0.1 were used, and airmass=1.67. JD was near 2456016.82]

In preparing for the EGU 2013 conference, where we will have a poster on color results from the earthshine project, I want to show a ‘B-V’ image of the Moon – since we can, and may be the first to publish such a thing.

Using Chris’ calibration of our filters against standard stars in M41 and NGC6633 I can reduce observed images in any filter to ‘magnitude images’ in the same filters. By taking the difference between the B and V image I thus arrive at a ‘B-V image’. Here is a slice across the middle of that image:

Second image added later: We now correct fore extinction:
Here, the BS is to the right of the ‘cut’ in the disk, and the DS is to the left. It appears that the BS is higher than the published value for ‘moonshine’ of 0.92 [see e.g. Allen, “Astrophysical Quantities”, 4th ed, Table 12.14]. Added later: Even after correcting for extinction the B-V is still higher than Allen’s value – we now get B-V near 0.95-1.0. Perhaps now, not so much larger than that value?

I am not quite sure about the DS. There is probably a slope because the halo in B and V are not identical and thus do not cancel. But as the BS is approached the color of the halo does approach the BS value, since that is where the light comes from. So I understand that part.

At the extreme left of the DS there is least influence from halo. The value is a bit up and down, but appears to be lower than the BS by up to 0.3. If B-V is a smaller number in one place compared to another it means that B is smaller than V in the first place, relative to the second – i.e. on the DS B is relatively smaller there than on the BS – since these are magnitudes it means we have shown that earthshine is blue – the Earth is indeed ‘a blue marble’!

Here is the B-V image itself, with cleverly chosen colours to reinforce our message: Sky is masked out, BS is to the right, DS to the left.

I’ll try to redo this for a lunar phase with less halo.

We have already shown a strange behaviour of the VE2 filter – or what appears to be a problem related to the use of the VE2 filter. The post is here.

Now we are able to report something even stranger (OK, just as strange then). We appear to have a relationship between the ‘VE2 pedestal’ and the airmass of the observation:

In the upper frame we see the old VE2 pedestal vs exposure time plot. In the lower frame we see that the pedestal is also a function of the airmass! If the pedestal was proportional to airmass we could, perhaps, understand the phenomenon as something showing a flux proportional to the amount of atmosphere (I am thinking of some sort of airglow here) – but NO, the pedestal height is smaller the larger the airmass is!

Is this some electronic problem related to how the telescope is oriented? Then why the inverse proportionality on exposure time too? Why not the other filters?

What is going on here??

In this entry: http://iloapp.thejll.com/blog/earthshine?Home&post=304 we investigated the effects of using a lunar albedo map based on scaling the Clementine map to the older WIldey map vs. scaling the Clementine map so that lunar mare and highlands matched what is published in the literature. [Note: the point being that the ‘Clementine map’ we have available is just a picture file – jpeg! – so that pixel values have to be scaled to albedo values somehow.]

We were doing this fitting in the ‘new way’ which is to fit ‘profiles’ starting on the sky on the DS side and extending onto the DS itself, modelling the contribution from the scattered BS light. In doing this we saw that improved fits could be obtained if the lunar map albedo was ‘stretched’ so that dark and bright areas better matched our observations. We did this stretching by eye and were able to improve the formal fits.

We have now compared the fits on 535 images done using the ‘Clementine scaled to Wildey’ map and the ‘Clementine stretched by eye’ maps.

We have found that the RMSE (that is, the square root of the sum of the squared residuals [i.e. observed profile minus best fitted model profile]) is improved using the scale by eye’ map – in two ways: a subset of images were poorly fit using the other map; they are now much better fit, and the mean RMSE of the finally selected images is lower.

We selected ‘good fits’ on the basis of alfa (the PSF-width parameter) having to be in a narrow range and that the relative uncertainty on the fitted albedo should be below a certain limit.

Mean log10(RMSE) is now near -1.1 in units of counts/pixel. We fit a profile that is 150 pixel columns long – 50 pixels on the sky and 100 on the DS.

Mean relative fit-uncertainty on the albedo is near 1.5% when using ‘counting statistics errors’ on the observation.

We note that the VE2 images more frequently have a larger ‘pedestal’ or sky offset after bias removal than the other filters. While most filters have an offset of near 0.2 counts (+/-0.5 or thereabouts) the VE2 offset is more often near 4 or 5 (+/- 1-2 counts). What is the cause of this? The bias frames surrounding the VE2 exposures have been spot-checked and seem OK – bias is near 400. A few, otherwise perfectly all right, VE2 images have a sky offset of 10-40 counts! Observations from the same night in other filters show nothing like this – some sort of nIR ‘fog’? Does the sky emit nIR light?

Does this have a conseqeunce for the ‘tunnel selection’ of VE2 images, done by Chris? [A data-selection method designed to take into account shutter and filter-wheel problems by requiring that image total fluxes follow a known phase-curve.]

While the ‘new fitting method using stretched lunar albedo maps’ formally works best of all methods we have seen so far, it is very empirical. Later we may be able to use the method to produce a ‘best fitting lunar albedo map’ that improves on the original Clementine map. We may then also be able to compare to what the LRO people (and LLAMAS) are finding.

After the scattered light has been removed we still have the task of converting the observations to a terrestrial albedo. This is done, by us and the BBSO, with the use of a model for how the Moon reflects light. This involves assuming a lunar albedo.

The BBSO has a set of lunar-eclipse observations that they use to find the DS/BS patch albedo ratio – we use the Clementine map. That map is available to us only as an image file. We scaled that image so that the lunar mare and highland albedos correspond to what the literature states.

An alternative would be to scale it to match what lunar albedo maps show. The only digital lunar albedo map we have is the 1970s Wildey map, which Tom Stone gave us. In this blog: http://iloapp.thejll.com/blog/earthshine?Home&post=253
we performed a regression-based scaling of all the pixel values in the Clementine map to corresponding (or interpolated) pixel values in the Wildey map, using the two maps’ coordinate systems.

We therefore have the possibility of fitting our observations to the lunar map of Clementine scaled to Wildey (let us call that the “Wildey map” from now on) or the Clementine map (“Clementine map” – but note that both are Clementine-based!) scaled to the highlands and mares. This choice has been investigated.

We find that the contrast in the Wildey map is lower than in the Clementine map, which affects the quality of fit.

Same lunar-edge profile (white line) fitted with two different models (red line) – one uses the Clementine lunar albedo map, scaled to match lunar highlands and mare; the other is the Clementine map scaled, using every pixel, to fit the Wildey lunar albedo map. Notice the smaller contrast between highs and lows in the upper image (Clementine scaled to Wildey) compared to the lower (Clementine scaled to highlands and mare).

There is evidently a difference in the quality of the fits – neither being particularly good, failing to match the observed highs and lows. The conseqeunces for the albedo fitted is at the 3% level.

The mean difference in the terrestrial albedo determined using the two maps:
16 albedo values determined using both Clementine-scaled-to-Wildey (vertical axis, in top plot) and Clementine alone (h. axis). We see that the two albedos are near the diagonal – with only a few outliers. The histogram shows the absolute difference between the two albedo determinations – it is about 1% of the albedo value.

As long as the large spread in albedo values (from 0.23 to 0.42) is real and not due to some data-treatment bias, we have an effect from the choice of lunar albedo map that is at the level of the errors from pixel statistics (shown elsewhere in this blog).

We would like the known biases to be less than the effects due to scatter, so perhaps some work on the lunar albedo is in order? We can look at the mean values of the two maps – there was no requirement that mean albedo be conserved, and we can look at various ways of stretching the contrast, while maintaining mean value – this is a nice piece of work for a student project.

Performing a hand-adjustment of the contrast scaling, while maintaining the original Clementine map mean, we can refit the relevant profile. We get this:

Smaller RMSE, small but important change in ftted albedo.

The best data will be those that have almost no interference from the halo at all. Those exposures, if they can be identified, will have the least scattered light on the DS and will therefore be the easiest to clean up. Here we try to identify those exposures. We simulate the effect of the halo on ideal images and measure the amount of flux added by the halo – before any attempts to remove it. We loop over relevant values of lunar phase and ‘alfa’, the PSF-width parameter. We calculate the change in flux for a region near the DS rim and express the change in percent. The contour plot shows contours at 1,2,4 etc percent change. Overplotted are red data points showing locations of real exposures.

We see that the least polluted images are found at large phase (near New Moon) and for narrow PSFs (large alfa values). This is not surprising! We see that we have several exposures taken under conditions that allow just 1,2,4 etc percent change in DS intensity.

If these images are the least polluted by BS-scattered light, we should have the most success when applying our scattered-light removal techniques to these. If they have a 10% effect due to scattered light, and we can remove 99% of that we have reduced the error induced by the BS halo to the 0.1% level.

We select 6 nights and extract albedos in 4 bands:

Get better pdf file here:

Here are albedos determined for 6 nights. We see 4 filter bands (VE2 is still being processed). Each color represents one night. There are 3 determinations of albedo from each image – hence points come in columns of three.

We see:

1) Albedos can differ in level from night to night – e.g. blue and red in panel 1.

2) Albedos can also be almost constant during a night – e.g. purple and green points in panel 1 vs red or blue.

3) Some nights have large scatter – e.g. green vs blue

4) B albedos are higher than V and others

5) VE1 and IRCUT albedos are very similar – the filters are also very similar, so this is good for reproducibility.

6) While some albedos evolve smoothly on a given night IRCUT shows a ‘dip’ – e.g. blue crosses in B vs blue points in IRCUT

Questions of interest:

a) Why are some nights noisy? E.g. the green points. Information on airmass, alfa and bias frame statistics, and RON, are available.

b) Can the nightly evolution (i.e. slope) of the points be related to anything happening on Earth?

c) Can the change in slope in IRCUT be related to anything special?

We have now reduced all the good data, and have arrived at the intrinsic properties of each image. We extract now the ratio of the DS-patch to the total flux and divide this by the similar ratio extracted from the synthetic model. This ratio of ratios is the same as the ‘Lambertian sphere terrestrial albedo’ and is the quantity published by e.g. Goode et al.

We select data for airmasses less than 2 and for statistical error less than 1.5%. This error is that due to mainly pixel-selection inside the ‘patches’ on the lunar surface (one near Grimaldi, one inside Crisium). Monte Carlo bootstrap sampling of the pixels inside a lon-lat box were resampled, with replacement, and the standard deviation of the consecutive means calculated. Errors due to image alignment, improper scattered-light removal, synthetic model problems and so on, are not included.

Here is the plot of extracted albedo against lunar phase for the VE1 filter (the filter with least scatter: IRCUT is almost identical to this plot): [scroll down for discussion]

The colours designate the method used for scattered light removal: red is the EFM method, orange or yellow is the ‘linear BBSO method’, and green are the raw data. Crosses and diamonds indicate the sign of the lunar phase – crosses are for positive phase. Error bars (error due to counting noise, not image alignment etc) have been plotted over the symbols, but are all smaller than the symbols themselves.

What do we see? There is scatter and there is dependence on lunar phase.

First, let us discuss the lunar phase dependency: For small phases we are essentially getting closer to Full Moon and the scattered light is more and more of a problem. The Raw data are doubled by phase 80 degrees, showing the importance of removing the scattered light. BBSO-linear is better than no removal all while EFM is better than BBSO-linear. There is also another lunar phase dependency – notice how the albedo rises for large lunar phases. Since the halo is smallest here this is not an effect of scattered light! The albedo is a composite quantity, consisting of quantities measured in observed and modelled images. The scattered light influences mainly the model, and mainly at small lunar phases. The reflectance model soley influences the synthetic lunar image, of course – I think we are seeing the influence of the (inadequate) reflectance model at large lunar phases – the albedo rises artificially due to this.

The there is the scatter. We see ‘daily progressions’ for some connected lines of data points. For instance, by 90 and 95 degrees phase. These points represent our candidates for geophysics! I think the ‘progression’ along these lines are due to different parts of Earth )oceans, clouds, continents) rotating into view for these observing sequences.

Below is the same plot, for the B filter. Apart from more scatter we see that the EFM seems able to remove most effects of scattered light, compared to the BBSO-linear method. A stronger ‘upturn’ is possibly seen at large phase.

Following on from post we now inspect satellite images one week apart in order to understand the natural variability found in satellite images of the same area.

Upper panel: average image pixel value for sequences of MTSAT images, 1 hour apart, for almost one day in March 2013 and a week later (red curve). Image pixel values is in arbitrary units but is proportional to pixel brightness. Lower panel: difference between upper panel black and read curves, expressed as a percentage of the mean of ther ed and black curves.

The difference between the two curves is on the order of 10% and varies from 8% to 18% during one day.

This tells us two things: Albedo (or something proportional to albedo) can vary by roughly 10% over a week. Albedo can vary during one day by almost as much.

This is useful information to have when we interpret the earthshine data.

We keep in mind that the smooth variations in the black and red curves in panel one are due to the day/night cycle – not intrinsic albedo variations: but the difference between the curves and the variability in the difference tells us about albedo variations.

We can reduce our observations to a number that is equal to the ‘Lambertian albedo of Earth’ at the moment of observation. That is, for a sphere behaving like a Lambertian sphere we can find the single-scattering albedo that gives the same earthshine intensity as we observe. This takes phases and all relevant time-dependent distances into account.

In order to understand if the data we get are realistic we wish to compare to satellite images of the Earth. From work published by others [the Bender paper] we are told that the terrestrial albedo varies by many percent from pentad [avg, over 5 days] to pentad.

Many of the best observations we have have ‘sunglint coordinates’ over South-East Asia. One geostationary satellite hanging over that spot is the Japaneese ‘MTSAT’. We may be able to get data from it, but have only ready access to a Meteosat that hangs over the Indian Ocean.

From the Indian Ocean Meteosat we have extracted a series of images for a given day, in order to start to understand what sort of variability we shall expect on time-scales that are shorter than a pentad.

We have access to half-hourly images from the satellite and have 12 hours of data – 25 images. We take the average of each image and plot it:

The black line is the observed mean intensity of the whole-disc image, and the red curve is a 6th order fitted polynomial. The difference between the black and the red curve has a standard deviation of 0.6% of the mean of the black curve.

The large-scale behavior of the black curve is due to the phase – we see half a day pass as seen from the satellite so Earth changes phase from new to full to new again. Variations over and above that would be due to changes in earth’s reflectivity. There is a pronounced sunglint in the Arabian Sea near Noon.

Some pixels in the image are saturated.

What do we learn from this?
By removing the fitted curve we learn how much variability there is as a day passes. Some of this would be due to the curve not actually being a 6th order polynomial – so the variability we get from the residuals are an upper limit. Fitting a 7th order polynomial lowers the S.D. to 0.4% of the mean.

It seems that the presence of a sunglint for some of the frames (the sunglint moves on to land – Horn of Africa – in the local afternoon and does not ‘glint’ in the sea anymore) does not generate a ‘spike’ of any kind in the average brightness.

We should remember:
That average brightness is not the same as albedo since the brightness depends on albedo times a reflectance function. But most of the reflectance behavior is removed via the polynomial fit, we think.
That cloud-patterns on this scale hardly vary much.
We do not yet fully understand the processing of the satellite images – are they normalized somehow? Were they all taken with the same exposure time?

So?
On this day at least variability in the albedo was much less than 1%.
We should inspect several days of images to see if the average level differs much from day to day.
We should try to get information from MTSAT, and at higher cadence – we can observe the earthshine with minute spacing.
Try to get more technical information about the satellite images.

The Laplacian method has been discussed here

http://iloapp.thejll.com/blog/earthshine?Home&post=283 .

We now extend the method to include an identical analysis step applied to synthetic images generated for the moment of observations. We then take the ratio of the results from observations and the results from the models. This will eliminate the effects due to geometry and reflectance (as long as the reflectance model used for the synthetic images is correct), leaving only the effects of changes in earthshine intensity. So, this is another ‘ratio of ratios’ result. The ratios involved are, in summary:

(Laplacian signature at DS/ Laplacian signature at BS; in observations) divided by
(Laplacian signature at DS/ Laplacian signature at BS; in models).

We use only the ‘good images’ identified by Chris in this entry: http://iloapp.thejll.com/blog/earthshine?Home&post=277

We start by inspecting the dependency of (Laplacian signature at DS/ Laplacian signature at BS; in observations) on phase [attention: this is not yet the promised ratio of ratios – just a ratio! :)], in each of the available filters:

We see the morning and evening branches folded to the same side, for comparison. We see an offset between the branches in each filter, and we see the dependency on lunar phase. We next look at the same ratio but from models. Since we do not have colour-information in our models we redundantly now show plots for each filter as if the models were color-dependent – they are not: but the points available in each filter are different, of course. Recall that a model is generated for each observation:

We see a very similar pattern – dependency on phase (but steeper this time). We also notice that the branches are closer together than in the observations.

What does this mean? The branches are separated, in observations, due to different distributions of light and dark areas on the eastern and western halves of the lunar disc facing Earth. Our Laplacian method samples pixels right at the edge of the lunar disc and has evidently met areas of different albedo. The separation in observations is not reproduced in the models – this can imply that the model albedos are incorrectly distributed (i.e. mares and craters etc in the wrong places) this is somewhat unlikely as we use one of the most detailed lunar albedo maps, from the Clementine mission. However, our model does not sample colour – the map used, and thus the ratio between ‘light’ and ‘dark’ – is taken from the 750nm Clementine image [I think; must check!]. This map was stretched to match the older Wildey albedo map [see here: http://iloapp.thejll.com/blog/earthshine?Home&post=253 ] which was made in such a way that the ‘filter’ the WIldey map corresponds to is a combination [see: http://iloapp.thejll.com/blog/earthshine?Home&post=286 ] of the Johnson B and V filters. We see the most well-reproduced branch spacing in the V band observations, compared to the models. This implies we have some colour-information about the two halves of the Moon – or a tool for how to scale the lunar albedo map when different colours are to be considered.

We next inspect the ratio of the observations and the models – the ratio of ratios:

For each filter is shown the Albedo derived – it is the ‘ratio of ratios’ spoken of above and is identical to the one used in BBSO literature – it is the albedo relative to a Lambertian Earth-sized sphere. The model albedo used was 0.31 so the ‘actual’ albedo derived is the above times 0.31.

We see that there is a phase-dependence in this – particularly in one branch, with the observations being relatively brighter than the models at phases nearer Full Moon, compared to phases nearer new Moon.

Since we have seen the EFM method produce less phase-dependent albedos [see here: http://iloapp.thejll.com/blog/earthshine?Home&post=252 ] we think the Laplacian method needs further development and investigation before it can be used.

It is worth listing why the Laplacian method might be useful:

1) It does not require careful alignment of model and observation. The signal is extracted from a robustly defined location in each image.
2) It is a ‘common-mode-rejecting’ method and is not dependent on image resolution.
3) It is relatively fast.

On this blog we have previously investigated the laplacian method of estimating earthshine intensity. [See here: http://iloapp.thejll.com/blog/earthshine?Home&post=273
and here http://iloapp.thejll.com/blog/earthshine?Home&post=272 .]

One of the limitations we realized then was that image resolution influenced the results from the Laplacian edge signature – the more the edges of the image are smeared the lower the Laplacian signature estimate will be.

We now revisit the method after realizing that the whole image is affected in the same way by resolution issues – the intensity estimate derived from the DS and BS edges will both depend on image resolution in the same way. The DS estimate is proportional to earthshine intensity, while the BS estimate is proportional to ‘moonshine’ intensity (plus earthshine – a vanishing contribution). The ratio of the two estimates will be independent of resolution – i.e. the ratio has the ‘common mode rejecting’ property.

We convolved ideal images of the Moon with the alfa=1.8 PSF and estimated both the ‘step size’ at the edges (on DS as well as BS) and the Laplacian signature. We plot them below:

The top panel shows the step size estimate against the Laplacian signature estimate on the DS only. We see an offset, with Laplacian estimate being smaller than the step size estimate. This is caused by the resolution issue – the edge is fuzzy and the Laplacian is degraded. The step size is estimated robustly from the original ideal image (i.e. before the convolution is performed) and is thus not dependent on image resolution.
The bottom panel shows the ratio of DS and BS estimates – x-axis is the step size estimator and the y-axis is the Laplacian estimator. We now see that the dependence on image resolution is gone from the Laplacian estimate. The correlation between the two estimates is good, at R=0.92, but not perfect. Noise was not added to the images. Estimates of Laplacian were performed on the line through the disc centre. The step size estimator is based on the average of 9 lines through the disc center.

[Added later:] Since we are working with synthetic models we know the actual earthshine intensity in each image, so we can compare what the actual intensity is to what is found with the ‘step’ and the ‘Laplacian signature’ methods:

In the first panel we see what we saw above – that the step and Laplacian methods are rather consistent. In the panel below we compare to the actual earthshine intensity.

Since the Laplacian signature method (and the step size method) literally express the DS/BS intensities ratio (which is only proportional to the earthshine intensity) we have different values along the x- and y-axes. The geometric factors having to do with distances and Earth’s radius are not compensated for.

The main result is that the relationship between actual earthshine intensity and the Laplacian signature method (and by extension, the step size method) are not quite proportional – there is a slight curve. In interpreting the above we should keep in mind that the Laplacian estimate of DS illumination from the edge derivatives is in a different role than the estimate of BS illumination – the former is not very geometry dependent while the latter is: That is, earthshine is due to light from a source we are sitting on doing our observations from, while the BS illumination is more angle-dependent in that the reflectance properties of thr Moon come into play to a larger degree – at the moment I am not sure whether there is an angle-dependence ‘along the BS edge of the disc’ that can cause a problem in interpretation. Will need to look at this.

Another thought: is it possible to make a calibration relation between actual earthshine intensity and the Laplacian DS/BS signature estimate?

What B-V colour should we expect for the earthshine?

We will here estimate it by using the change in colour of Sunlight that has struck the Moon once, and the colour of Earth as estimated from spacecraft.

The Sun’s B-V is +0.650 [Allen, 1973] [Holmberg et al, MNRAS, 367, 449, 2006]

The Full Moon’s B-V is +0.85 [Lane&Irvine, AJ 1976 78, p. 267]
[vdBergh has +0.876 for Mare Serenitatis;
Allen 4.ed. table 12.16 has ‘Moon’ B-V 0.92;
Gallouet (1963) has +0.94;
Wildey & Pohn (1964), AJ vol 69, p.619 have a range of values near +0.86 to +0.87 (their work seems good and a milestone).]

The Earth’s B-V is 0.2 [Allen 3. ed, but appears based on a 1961 work – so pre-spaceage?]

The Moon’s DS B-V is 0.64 on average given data in [Franklin (1967), JGR 72, p 2963]

If Sunlight is reddened by one reflection off the Moon by 0.85-0.65=+.2 mags, then we expect earthshine, bounced once off the Moon to redden by the same amount.

If the Earth has B-V=0.2 as seen from space then seen after one reflection it ought to be redder by +0.2 or appear to us observing it from Earth at B-V=0.4. This is not what Franklin measured.

Basically, we do not yet know Earth’s B-V colour! I am making inquiries, and we shall see.

Note that Danjon did lots of colour observations of earthshine – but in the Rougier system. Wildey [JGR vol 69, p.4661+] refers to a transformation from Colour Index (“C.I.”) in the Rougier system to B-V in the Johnson system – but without giving numerical details. The method is based on transformations using the Full Moon and the Sun colours.

The transformation should be made specific and the data from Danjon placed online. Another student project!

The earthshine comes from the various parts of the Earth that are turned towards the Moon, and the Sun – all the clouds and oceans and deserts and ice-caps that are illuminated and visible from the Moon contribute to the Earthshine. Which parts contribute most?

We take a representative image of Earth, as seen from space, and investigate where the flux mainly originates.

Splitting the above JPG image into the R, G and B channels we can analyses where e.g. 10, 50 and 90 % of the light comes from. That is – we seek the pixels that contribute these fractions of the total flux, and identify them in images. Note that R,G and B refers to other wavelength intervals than the B, V VE1 VE2 and IRCUT bands we have – our B band is bluer than the ‘B’ used in JPG images.

In the three frames we see 3 rows (B, G and R, from the top) – on the left in each panel is the original R,G or B band image, while to the right are the pixels contributing to the 10, 50 and 90 percentiles of the total flux in the image. The order of the panels is: top left 90%, left bottom: 50% and top right is 10%.

In the 90% images at top left we note that the B image (top row) looks different from the R and G images – the light in the B band comes from atmospheric scattering – Rayleigh scattering, and aerosol scattering – as well as the ocean and the clouds; other bands have more of their flux coming from clouds.

Variations in the blue may therefore tell us more about the atmospheric state than do the other, redder, bands. The Rayleigh scattering is due to molecular scattering – as long as the composition of the atmosphere is the same this ought to be constant in time; but some of the blue scattering is also due to aerosols and thus we may have a tool to investigate variations in the aerosol load. The longer-wavelength bands will tell us more about the continents. All bands are quite dominated by clouds – a small cloud can reflect as much light as a larger un-varying continental area.

The above is repeated here on another image of the Earth – more realistic as it is half-Earth. Image from Apollo 8.

And here is the B,G,R images and the 90% percentiles:

Top to Bottom: B,G,R, Left: R,B or G-band image – right: 90th percentile image.

We again see that the light contributing to the blue image (top) is more diffusely distributed than in e.g. the red (bottom) case where most of the light comes from variable features like clouds. This implies that we should expect larger variability in our albedo data for the red images than the blue image.