SHARC DATA REDUCTION COOKBOOK Draft 2, Revised 7/22/98, D.C.Lis Send questions or comments to dcl@submm.caltech.edu Note: SHARC data reduction programs are no longer supported on the summit VMS machines. Use the programs installed on the Sparcs. (0) Tcl/Tk Wrapper Instead of running camera from a terminal window, as described below, you can now use a Tcl/Tk wrapper to input all the relevent parameters and run the program in the background. Simply type camtk on the Unix prompt. For simplicity, only the basic data reduction parameters can be set using the wrapper. Default values are used for the remaining parameters. If you think that having some additional options available would be useful, please send me email and I will modify the program (1) General concepts Submillimeter continuum observations at the CSO are made in beam switching mode, in which the output represents the difference in signal between two positions on the sky. With SHARC, the beam is switched using the chopping secondary. To achieve the best sky subtraction, the two beams must be at the same airmass, which implies that the chopper throw is in azimuth. The standard observing mode with SHARC is on-the-fly (OTF) mapping. In this mode, the telescope is scanned at a fixed rate in azimuth, which is the direction of the chopper throw, while the array is aligned vertically. The resulting map is a differential image of the source which has to be restored and regridded onto a regular grid in the equatorial coordinate system. This is done using a program called `camera', which uses NOD-2 subroutines for image restoration. There are two ways you can make a two dimensional map out of the SHARC OTF data. The samples taken by the different pixels of the array in a given azimuth scan will form a map in Az-El coordinates. Alternatively, the samples taken with a single array pixel over a range of scans in an OTF map will also make a two dimensional map in Az-El coordinates. Camera will let you look at you data in both ways. The first mode, called scan mode, is used primarily for pointing, focusing and deriving gains for different pixels in the array. The second mode is typically used in the final data reduction. (2) Pointing To check the pointing of the telescope and measure the chopper throw on the sky go to a strong point source, like a planet or a compact HII region: UIP> PLANET URANUS Make a single OTF scan across the source: UIP> OTF 200 5 5 5 5 5/ALT Open the data file in `camera' and select pointing mode (p). The program will plot the dual beam image and will leave you in cursor mode. If the data are noisy, hit `s' to smooth the data in the direction of the scan (the data are usually highly oversampled in this direction by default). Place the cursor on the peak of the positive beam and hit the spacebar. The location of the cursor (Az-El offsets) will be written in the terminal window. Place the cursor on the negative beam and hit the spacebar. The difference between `azoff' values for the two cursor positions is the chopper throw. Hit `e' to exit from cursor mode. Restore the image using the value of the chopper throw that you have just measured. Set the second parameter (L-to-R amplitude) to -1 in the negative beam is on the left, otherwise set it to 1. The restored image will appear in the graphics window and you will be placed again in the cursor mode. Place the cursor on the peak in the restored image and hit `p'. New values of the fixed pointing offsets (fazo and fzao) will be given in the terminal window. Apply these values in the UIP : UIP> FAZO xxx UIP> FZAO xxx and repeat the procedure several times until you reach convergence. (3) Focusing Default values of the focus and pointing offsets are loaded automatically whenever the SHARC command is issued the first time (i.e. when switching from heterodyne mode). You can change the focus of the telescope by issuing the following UIP commands: UIP> FOC/OFF=xxx UIP> FOC For each focus offset, make a scan across a strong source and reduce the data in pointing mode as described above. Write down the peak intensity in the restored image for each focus offset. Choose the focus offset that gives the highest intensity. Note the shape of the beam as well. It should be round. It will be elongated when out of focus. (4) Phases Move the telescope to a strong continuum source such as Jupiter or Saturn. measure the chopper throw and pointing corrections as described in (2). Apply the pointing corrections. Put one of the beams on source: UIP> AZO xxx/CHOP where xxx is one half of the chopper throw that you have measured. Execute the following UIP command: UIP> PHASE_12 This assumes that pixel number 12 is the reference pixel. Other com files exist for other reference pixels. If the source is weak, you should increase CHOPS_PER_INT to get better signal-to-noise ratio. You will need to re-phase any time Labview is restarted or the chopper throw or frequency are changed. (5) Gains After you have pointed and set the phases you can measure the relative gains of different pixels in the array. First make sure that the chopper throw is set properly UIP> SEC xxx yyy where xxx is the chopper throw you have measured and yyy is the chopping frequency. Make several `sideways' scans across the source. In this mode, every pixel of the array is scanned across the source in zenith angle. UIP> FF_SHARC This will make two scans with the source placed in the positive and negative beam, respectively. You should see only one beam on the strip chart display. Reduce the data using the `gain' mode of camera. A file `gains_*.dat' will be created for each `sideways' scan you reduce. In the end you will have N `gains_*.dat' files, where N is the number of `sideways' scans you have taken. Exit `camera'. Run `graphic'. From within `graphis' execute the following command: GRAPHIC> @ "/home/hapuna/dcl/bin/gains.graphic" 1 N where n is again the number of sideways scans you have taken. The file `gains_ave.dat' will be produced in the current directory. Rename it, using today's date in the name (e.g. gains_1sep96.dat). You will need this file in the final data reduction. (6) Reducing Maps Make a map of the source using the OTF command, e.g. UIP> OTF 270 240 5 5 5 4/ALT When choosing the length of the scan, make sure there is some baseline at both ends. Otherwise you won't be able to restore the data properly. Always add the chopper throw to the length of the scan you have determined based on the extent of your source (i.e. you have to set the scan length to 270' to get a 240' size of the restored map with a 30' chopper throw). Beware that the noise in the restored image increases roughly as the square root of the scan length divided by the chopper throw. This is something to keep in mind when yon make long scans with a small chopper throw. When the map is finished, run `camera' and reduce the data in `map' mode. You will have an option to de-spike and de-stripe the data. These functions are described below. Most of the time you want to choose automatic processing mode. You will be asked a number of questions during reduction of the map taken with the first pixel of the array. Parameters given will be applied to processing of subsequent maps. You can safely look at the first few scans while a large map is still running. If any of the camera pixels are noisier than the remaining pixels you can blank them and exclude them from the data reduction. This can be done using `camera.dat' file which has to be placed in the directory from which you run camera (see below). At the end you will be prompted again for the next filename. If you reply `radec', the program will add all the maps, re-grid them onto a regular grid in the equatorial coordinate system and display the final image on the screen. This is a rough version of your final map which is written in FITS format to the file `filename_ext.fits', where `filename.ext' is the name of the data file you are working on. The reduced data is written to the file `camera.radec'. When you exit `camera' properly using the `quit', command this file is renamed to `filename_ext.radec'. You should thus quit `camera' after reducing each data file. (7) De-spiking A simple de-spiking algorithm is implemented in CAMERA. If you choose to de-spike your data, the program will calculate the mean and the rms for the samples taken in a given scan with a given pixel of the array. It will the search for data points which satisfy the condition (data-mean)/rms > ncut where ncut is a user specified cutoff and will replace these points with the average of four neighboring points in the same scan. If you have a compact source in the map, you can specify an avoidance region. Only points outside of the avoidance region will be included in calculations of the mean and the rms. Note that this procedure will not work if there is a strong source in the map. (8) De-striping In windy conditions you often see a lot of correlated noise in the data. You can try to remove this correlated noise using a simple de-spiking algorithm built into CAMERA. For each integration, the program will calculate an average signal for all pixels of the array and it will subtract the average from all the pixels. If you have a compact source in the map, you can specify an avoidance region. This algorithm will not work for extended sources. (9) CAMERA.DAT There are several bad pixels in the array. The information about which pixels are bad is stored in the header and the data taken with this pixels will not be processed. In addition, on occasion some pixels (in particular pixels 17 and 18) may become much noisier than the remaining pixels. If you want to exclude any additional pixels from from the data reduction, copy the file /home/hapuna/dcl/bin/camera.dat to your current directory. Edit this file and change the second column from 1 to -1 for the pixels you want to exclude. (9) Adding Maps You will use a program called `regrid' to produce a final map in the equatorial coordinate system. An example input file for `regrid' can be found in `/home/hapuna/dcl/bin/regrid.inp'. Here is the list of parameters you will have to specify on successive lines in the command file: (a) Ra and Dec for the center of the final map. (b) Limits of the final map in arcsec (+rao, -rao, -deco, +deco). (c) Pixel size of the final map in arcsec (Ra and Dec). Typically 5 arcsec is a good number. Pixels don't have to be square. (d) Minimum number of data points required per pixel of the final map. (e) Cutoff limit for weights of maps to be included. You should run the command file for the first time with this parameter set to zero. Weights of the maps will be printed at the end along with the average, min and max values. If all the weights are within a factor of a few, you probably want to leave this parameter at zero. If one or more maps have weights significantly lower than the remaining maps, you may want to exclude them from the final average. To do it, set the cutoff above the weight of the maps you want to exclude and run the command file again. (f) Data file name. This is the `.radec' file created by `camera'. (g) Range of maps to be read from the data file. (h) Pointing corrections in Ra and Dec in arcsec. Normally these are set to zero. You can correct pointing errors in `camera' before transforming the data to the equatorial coordinate system. (i) Scaling factor from mV to Jy (i.e. Jy/mV). (j) The following lines include the gains for different pixels of the array. You can include here the file `gains_ave.dat' (see above) and delete the lines corresponding to pixels which were blanked. The bad pixels will have a gain of -1 in `gains_ave.dat', but you also have to delete lines corresponding to any additional pixels you might have excluded in CAMERA.DAT. If you want to average several maps, repeat (f) - (j) for all the maps you want to include. Otherwise type quit on the following line. Leave the following three lines as they are in the example command file. The final result of the command file is a FITS image `regrid.fits'. You will have to rename after running the command file, so that it doesn not get overwritten.