This is another report in a series of spot checks of Tom's sample images. If you don't like analysis and numbers, don't read this. Richmond, please add this to my other reports on the Web page. I'd appreciate comments so I can clarify some points made. Some "attaboys" would be appreciated. - Herb
The question was recently asked: "if the VCO clock varies, how much does that impact the results?" Seemed to me one way to gauge this was to look at the dark pixels (the covered pixels at the far end of each row) and see if they vary. Within some amount of standard deviation, they should be steady. But if the pixel clock varies, the latency time in the column will vary, and the dark current is proportional - provided temperature is constant. (If not, a drift could be temperature drift. Tom claims a .1 degree temperature stability: perhaps someone can provide an ADU per delta T value?)
I was looking at this stuff anyway, computing statistics...so I did an analysis of Tom's "sample" dark frames (dX500244.fts, where X is 0,1,2). I'll give away the ending: I found a consistent drift in mean dark current per column over short runs relative to the entire run.
Method: for each file, I peeled off columns 790, 791, 792, 793. Columns were counted from 0, and values converted such that 32768 --> 0. Even if we debate the "darkness" of the dark pixels, these are undenyibly dark in this file.
For each column, I computed the mean (sum/rows), "mid" (max/2 -min/2), standard deviation of the mean. I ALSO computed a mean for groups of 50 rows, and computed (mean of all - mean of 50). I reported this difference for the first 50 rows, the next 50 rows, etc. Any drift would show as a change in this difference. I printed all the differences in order, then eyeballed them for any pattern.
Results: All three files shows a consistant drift in the mean of 50 pixels (relative to the mean of all pixels in the column) from minus 1.5 standard deviations, through zero, to plus about .75 standard deviations. Again, this is a drift in the difference between the mean of 50 rows and the mean of all 890 or so rows. A side note: one standard deviation is about 20 ADU's, or .3 percent of the mean dark current. (Anyone want to cross check this as reasonable dark current noise?)
Example: I'll report one file and 4 columns as an example. (The values were hand-recorded so they may look odd):
file d0500244.fts
-----------------
row 790 791 792 793
mean 6737.5 6647.88 6622.14 6627.31
"mid" 6656.0 6665.0 6640.0 6652.0
st. dev 21.67 21.47 20.72 20.84
1st diff -35.82 -35.28 -33.82 -35.69
last diff +16.34 +15.00 +14.66 +15.91
avr diff +2.2 +2.38 +2.4 +2.4
The "avr diff" is the average of all the differences. The "diff" values
reported are actually the largest of the first few diffs, and the largest
of the last few diffs. "Few" is four or less. Simple arithmetic yields
the equivalent diffs in standard deviations.
For file "d1500244.fts", the mean is about 5475 or so; st. dev 17.7; and the difference goes from -29 to +14. For file "d2500244.fts", the mean is 5990; st. dev. 21; and the difference goes from -33 to +16. Again, the same values of drift for each file, when normalized to standard deviations from the mean.
Comments: If this represents a VCO drift, it will impact timings and exposures. If this is a temperature drift, it only impacts the dark current and can be adjusted for in the dark current subtraction. So it's important to "nail down" what this drift is from. If this analysis holds up, other camera holders should see how their cameras perform under this "test".
Arne reminds me (20 Mar 97) that the "drift" I report may be dark current drift AND instrumental drift (e.g. read bias); and these drifts may or may not correleate with temperature. There is talk about adding "overscan" to the camera software to read past the last column pixels to sample the CCD read amplifier bias and subsequent electronics.
Raw Data: Here's a typical output for column 790, for file "d0500244.fts". Note that I also include a "histogram" of the values in 20 bins from min to max, with a "*" to indicate the mean: I wanted to see the distribution, and it looks "fairly" smooth but the mean is not quite in the center of the distribution (I think).
Dist2 ver 0.9 Mar 17 1997 Input=d050i790., Output=dout. <---[file is columns 790-799 from .fts file.] 896 data points read mean of 896 values is 6637.500000 min is 6588, max is 6724, "mid" is 6656.000000 the standard deviation is 21.665270 the distribution over 20 less than or equal bins between min and max and of width 6 is: 1 0 4 13 37 104 132 128 130 *73 62 42 39 29 25 23 14 9 16 10 running differences between average and local average, computed every 50 rows as avr - localavr is: -35.82 -24.42 -13.98 -7.26 -2.98 3.14 6.34 8.78 12.82 9.66 10.50 11.10 13.78 13.42 13.82 16.34 average difference is 2.20
Note: I have the code for these computations, wrote it myself in very plan, command-line driven C. It's easily enough done by any *real* numeric tool, but my code is yours if you wish. An independent check would be appreciated.
Chris suggested that my finding of "drift" in Tom Droege's dark pixels (columns 790 to 794, counting from 0) was likely a temperature drift, which he had previously noted. He suggested I look at other people's images. This was a surprising tough assignment: only Tom is posting raw data for the most part. I found an old image of Glenn Gombert's, the image with an "asteroid" streak in the far end of some rows; and an image of Michael Gutzwiller's. I could not find any of Michael Richmond's raw images. Perhaps camera owners could be encourage to keep at least a FEW raw images "around" for the curious.
Files from Gombert and Gutzwiller did not show the "drift" that Tom's camera image did. The shifts in local average pixel value (run of 50) vs the overall average (run of 890 or so) were smaller and varied inconsistently, well below the standard deviation of the overall average. I also found that Glenn's image, originally posted as "an asteroid trail", actually extended into the dark pixels and I had to "edit" an affected row out (it's the only Gombert image I could find) to remove its impact. This effect should be noted.
I think my method of monitoring the dark pixel columns - comparing runs of 50 pixels to the entire run, normalized to the standard deviation of the whole run - is a good measure of "stability". And it apparently also offers some degree of automated "outlier" monitoring.
Droege file "d0500244.fts":
row 790 791 792 793 mean 6737.5 6647.88 6622.14 6627.31 "mid" 6656.0 6665.0 6640.0 6652.0 st. dev 21.67 21.47 20.72 20.84 1st diff -35.82 -35.28 -33.82 -35.69 last diff +16.34 +15.00 +14.66 +15.91 avr diff +2.2 +2.38 +2.4 +2.4The "avr diff" is the average of all the differences. The "diff" values reported are actually the largest of the first few diffs, and the largest of the last few diffs. "Few" is four or less. Simple arithmetic yields the equivalent diffs in standard deviations.
Gutswiller file "32t0433.814" and Gombert's file "30T0478.642":
Reporting columns 790, and reporting the most negative and positive differences in absence of a clear pattern:
site gombert gutzwiller mean 8094.85 9567.12 "mid" 8092 9569 st. dev 11.90 26.73 max neg diff -2.79 -4.08 max pos diff 3.99 3.84 avr diff .01 -0.04These differences are only a few ADU's, well below the standard deviation.
Here's a typical output for column 790, for Tom Droege's file "d0500244.fts". Note that I also include a "histogram" of the values in 20 bins from min to max, with a "*" to indicate the mean: I wanted to see the distribution, and it looks "fairly" smooth but the mean is not quite in the center of the distribution (I think).
Dist2 ver 0.9 Mar 17 1997 Input=d050i790., Output=dout. [Tom Droege's "d0500244.fts", col 790] 896 data points read mean of 896 values is 6637.500000 min is 6588, max is 6724, "mid" is 6656.000000 the standard deviation is 21.665270 the distribution over 20 less than or equal bins between min and max and of width 6 is: 1 0 4 13 37 104 132 128 130 *73 62 42 39 29 25 23 14 9 16 10 running differences between average and local average, computed every 50 rows as avr - localavr is: -35.82 -24.42 -13.98 -7.26 -2.98 3.14 6.34 8.78 12.82 9.66 10.50 11.10 13.78 13.42 13.82 16.34 average difference is 2.20
Here's a similar output for Gutswiller file "32t0433.814", column 790: note the "difference of local averages" goes from minus to plus in no clear pattern, and is well below the standard deviation:
Dist2 ver 0.9 Mar 17 1997 Input=32t790., Output=32tout. [Gutswiller file "32t0433.814", col 790] 897 data points read mean of 897 values is 9567.123746 min is 9462, max is 9676, "mid" is 9569.000000 the standard deviation is 26.733985 the distribution over 20 less than or equal bins between min and max and of width 10 is: 1 2 0 1 0 11 26 43 79 98 128 *139 118 99 72 40 22 12 4 1 running differences between average and local average, computed every 50 rows as avr - localavr is: -4.08 -0.68 -3.58 -1.04 -3.64 0.18 1.90 2.32 2.36 0.42 -2.88 2.00 3.84 3.16 1.12 -2.10 average difference is -0.04
Finally, here is the output for Gombert's file "30T0478.642", an image which includes a so-called "asteroid streak". This produces a spike artifact in the (not so?) dark columns, which I edited out of the data set. The "difference of local averages" is also below the standard deviation and shows no obvious pattern of drift.
Dist2 ver 0.9 Mar 17 1997 Input=30t79x., Output=30tout. [Gombert's file "30T0478.642", col 790] 896 data points read mean of 896 values is 8094.853795 min is 8058, max is 8126, "mid" is 8092.000000 the standard deviation is 11.896871 the distribution over 20 less than or equal bins between min and max and of width 3 is: 2 0 2 3 10 9 28 41 50 60 83 55 104 *70 87 53 91 48 37 27 running differences between average and local average, computed every 50 rows as avr - localavr is: -0.83 -2.71 -2.05 -1.27 2.13 0.27 3.63 0.13 1.55 -2.65 0.51 -2.79 -2.49 -1.21 3.85 3.99 average difference is 0.01
For Gombert's file, there was a bright streak which apparently extended into the so-called "dark pixels". If these are truely dark pixels, this must be an instrumental effect, maybe the previously mentioned "saturation crossing over from one camera to another" problem (see the TASS Tech Note on Mark III hardware problems which I hope references this). The edited row, from cols 790 to 799, and its successor row are:
10216 9464 8952 8626 37007 40301 33066 54255 11198 0 8096 8064 8078 8046 37011 40305 33064 54255 11199 0Note that columns 790 through 794 are all impacted.