Thu, Nov 30: 9:10 AM: Kim continues - Special SNAP classes appear in menu for creating new class SNAP Par Verb is a Verb which is also a Noun in other words, it's a verb which may contain control parameters which can be saved for later use - Example: Alex creates a SNAP Noun the new Noun has template with examples of all the sorts of parameters and methods that can be placed within a Noun SNOID = unique ID of an object within the SNAPSim database explains how special comments /** * special JavaDoc comments */ will automatically be placed into generated documentation Kim suggests that it makes code more robust if one protects variables as much as possible from modification by code outside its class - Example: Kim creates a simple Runfile, using SNAP Runfile menu item Again, the SNAP Runfile template provides examples of all the sorts of things which may be included in the Runfile One must explicitly request items (Nouns) to be saved, using a construction such as cds.save(spectrum); One can then later retrieve information saved in a Dataset via a construction such as cds.getByIname(AbtractSpectralFunction.class, "pick me"); Using "cds.save()" creates database files on the local computer's hard drive. One can transfer these files to other people, who can move them to his dataset directory, then read them in using cds.get... - the Calibration group will be especially interested in the class AbstractSpectralFunction important features: valueAt -- gives value at a given wavelength integrate -- performs integration over a range of wavelengths all current interpolation in AbstractSpectralFunction is done via linear methods. It is not difficult to modify the class so that interpolation is done via different methods - that is something that the Calibration team might want to do ... - the Calibration group may also be interested in PSF classes, in snap.snoop.physics.util.sb2d - there is the MINUIT numerical computation package in snap.snoop.physics.util.fit 10:50 AM: Stoughton talks about pixel-level simulations - shapelets are used throughout pixel-level simulations based on Hermite polynomials - SNAPSim does not have a realistic universe just SNe no galaxies, stars no realistic RA, Dec distributions there is the Dark Energy Survey (DES) catalog, which is a realistic simulation which we might copy or adopt or adapt - there are methods to deal with standard CCD effects dark current, readnoise, bleed trails cosmic rays charge transfer inefficiency persistence non-linearity one can easily extend the framework to deal with additional detector effects - gov.* and snap.* gov.* packages are generic pixel detector routines gov.fnal.eag.des.* packages are from Dark Energy Survey includes giant simulation of big region on the sky snap.* can generate catalogs of objects, observing strategy, mission details, etc. gov.fnal.eag.util: cosmology gov.fnal.eag.sim: actual pixelated images gov.fnal.eag.sim.extractor: simple SExtractor extracts positions, fluxes, etc. from images gov.fnal.eag.sim.catalog: describes objects see astro-ph 0506393 for example of galaxy descriptions gov.fnal.eag.sim.pixel: generates the pixel images - performance for DES finished 3340 pointings for DES each 62 CCDs, each 2048x4096 each pointing takes about 2 hours on a 3 GHz machine, 500 MB memory - charge transfer inefficiency (CTI) simple, quick implementation slow, accurate implementation - includes a simple version of drizzle co-addition to combine several short exposures writes temp files to temp space on disk eventually, temp files will be written with full provenance - snap.snoop.physics.pixel SNAP's version of the gov.fnal.eag.* stuff - Stoughton now shows some demos see snap.snoop.physics.pixels.examples ShapeletGalleryDemo CosmicRayDemo ShowMeVega - we run the ShowMeOneGalaxyManyTimes demo images are created in a temporary directory outside of the SNAPSim directory structure: on my machine (MacOS X) /tmp/ImageRepository.michaelrichmond/ images are 64-bit floating point integer FITS files (also option to create PNG images?) - JUnit tests -- a very simple example shows simple class, holds an integer; can be set, get, incremented convention: make "test" subdir for each class, put JUnit in "test" select "Create JUnit Test Case" new option in "Run" menu: Run JUnit Test use methods such as "assertEqual" or "assertNotNull" can include tolerances in the "assertEqual" test - JUnit test results Stoughton shows the results of automatic nightly running of the JUnit tests results put on a web site for easy browsing one can quickly find the code which failed its tests Q: what about the image analysis code? A: doesn't really exist yet first attempts at using Sextractor Q: how stringent are the constraints we must place on PSF in order to measure SNe accurately enough A: (a discussion follows) issues of dithering Q: how many SNe will have spectra taken by SNAP? A: all 2000 of them 2000 SNe assumes 3 years of SN survey get lots of "extra" images of the survey area when the telescope is pointed to take spectra of SNe Kim: one could rotate the telescope during these exposures others: why? please don't Q: What is "SNAP Lite?" A: Under some circumstances, one could modify the telescope to be 1.8 meters in diameter instead of 2.0 meters 01:53 PM: Gary Kushner describes the upper level of simulation - we update code snap.jars.release snap.snoop.physics.simulation - history first, designed low-level tools of simulation carefully but top-level stuff was left up to users to write themselves lots of people tried doing similar things independently disaster Kushner has been working on a high-level script for all to share users can configure it to fit their needs no need to modify code to configure the script - can configure from a text file or from the command-line uses a batch-ish language or scripting language such as Perl, Python - top level is divided into domains Runfiles have sample template runfiles as examples BuildLightCurveTemplates, e.g. utility code extracts bits of results from runfile output Example: createStandardSNAPSimulation create source data create observed light curves fit light curves do cosmology - one key: don't modify parameters inside the Java code itself instead, can use a text file or command line to set parameters parameters are saved with datasets for future reference - .bat files contain the simple commands which run domains can edit with ordinary text editor - createSourceData can create a set of parameters see snap.snoop.physics.simulations.samples file "user.properties" ### ### Configuration for SNAP Default Parameters ### SNAPDefaults.TargetSeed = 5000 SNAPDefaults.SNSeed = 6000 SNAPDefaults.DustSeed = 7000 SNAPDefaults.MaxTargets = 10 # # ### ### Configuration for GarySim Default Parameters ### GarySim.MaxTargets = 15 # # ### ### Configuration for Aperture ### Aperture.ExposureTime = 310 Aperture.MaxTargets = 20 - simulationParameterFactory.java has list of all the current parameters createSNAPParameters() method will generate an instance of the parameters note that the "Calibration" parameter will be relevant to us editing the "user.properties" file will make a change apply all the time - one can change parameters on the command-line, too "simRunner" will run a Java program, but allows you to modify parameters within the program -- just for the one execution example: to run the "FitLightCurves" stage with only 20 targets, SimRunner -r snap.snoop.physics.simulation.domainBased.FitLightCurves -D maxTargets=20 - state of the code (Nov 30, 2006) some features need to be added much needs to be documented "late in the development stage" - example: try to run one or two stages of top-level simulation in users .snoop/ directory, user.properties is simple file which is top precedence snoop.properties is next in priority Java code is lowest in priority - "PlotSData" is a verb which creates graphical displays see the "verbs" item in Package Manager for list of verbs in each package - the logfile output will print out values of lots and lots of parameters at the start of a run these values are stored with the output of a runfile (?) values do not _yet_ have units attached; is on ToDo list - one can use the DatasetBrowser or GUIBrowser to examine output of Runfiles - standard method of operation create a dataset run one stage of the "pipeline", using dataset as Antecedent thus, all the info generated so far is available and will be copied into the output of the current stage then save the dataset then run the next stage, incorporating dataset as Antecedent again and so on by default, the framework will grab the most recently produced dataset with a given name _and_ was closed without an error one can also specify a dataset by datasetID, if desired the datasetID values are guaranteed to be unique, but are not sequential - "inames" are supposed to be human-readable version of identifiers examples: targets, UserParms, SNAP Filter 7 one can use the "view" option in DatasetBrowser to look for nouns by their iName the "Saved By CDS" column in DatasetBrowser tells you if a noun will be saved automatically - the "DB Compact" option from "Run" menu item tries to save space deletes all datasets which were closed with an error deletes all but the most recent version of a dataset with same name - how to share datasets go to the directory in which datasets are written e.g. /home/users/simdata identify all the files belonging to a dataset put all those files together into a single archive via tar or zip or other utility copy the archive file to another user's computer and simdata directory other user un-archives the files other user can then read the datasets - simulations can be run through a scripting language the framework understands a simple list of commands optionally with parameters for each command if want more fancy stuff, use scripting language such as Perl - we run as example BuildQuicklyLightCurveTemplates (takes several minutes) - we try a batch file use Run menu pick "Batch Domain Based" can now use the dialog window to add Arguments - see snap.snoop.framework3 examples and documentation for the top-level framework written from CS point of view, not astronomer point of view - to run a batch file from the command line see snap.snoop.framework3.tools use the Python interface use symlinks, not "cp", to copy code outside of Eclipse directories go to .../workspace/snap.snoop.framework3/tools/ then type for example (it takes a long, long argument name) ./runsim.py snap.snoop.physics.simulation.runfiles.create.CreateStandardSNAPSimulation this will set up a boatload of environment variables for Java, classes, etc. this _should_ start running the Java executable (but doesn't actually do so at the moment) "runjava.py" script builds up enviroment vars for any Java executable "runsim.py" is designed specifically for framework Java programs - using batch files set up enviroment so that .../workspace/snap.snoop.framework3/tools/runbatch.py is in PATH then go to directory containing a batch file, for example, RunCompleteSimulation.bat then type from the command line runbatch.py RunCompleteSimulation.bat 4:15 PM Deustua: list of tasks for Calibration team within framework - Deustua has a presentation listing possible tasks - July 2006 Calibration Review -- what needs to be done must resolve controversy over photometric requirements study and determine impact on less-than-optimal performance of the spacecraft on the science - simulations will address some of the most pressing issues feasibility and impact of "color calibration" across visible-IR impact of detector characteristics on calibration (rumors of changing focal plane to be HgCdTe only, no CCDs) spectrophotometric accuracy of IFU effect of filter errors on photometric accuracy angle of incidence structural defects (bubbles, surface errors) production uniformity effect of flatfield variations effect of thermal variations effect of focal planet behavior on PSF and astrometry optimal cadence for calibration maintenance end-to-end simulation - discussion need some (lots?) of input stellar spectra need a mechanism for creating filters in a flexible fashion a "filter factory" the "nonlinearity" correction currently does not deal with non-linearity in the rate of incoming photons, as has been seen in some IR detectors filters: how large is the difference between the shadows cast from a piece of dust ? a. from starlight coming through optics b. from light from Ring of Fire good question ... may require outside code study - Deustua: a big deadline for accomplishing something is around June-July 2007 expect a progress report around Feb-Mar 2007 we need to be able to present a concrete operational plan for calibration time and tasks in the mission - see Deustua's powerpoint presentation http://stupendous.rit.edu/richmond/temp/deustua_11_16.ppt