README File for CALDB 4.8.0 Upgrade Only Instructions 06 Sept 2018 DE Graessle, Chandra CALDB Manager Includes brief Intro, Upgrade download and installation instructions, and pointers for further assistance. To learn more about the Chandra CALDB, and to find this information on the WEB, you are referred to http://cxc.harvard.edu/caldb/index.html. For Release Notes to the Chandra CALDB4, and specifically CalDB 4.8.0, see http://cxc.harvard.edu/caldb/downloads/releasenotes.html CalDB 4.8.0 requires CIAO 4.5 (or later version) because of an upgrade to the caldb4 software that was included since the release of CIAO 4.5 in December of 2012. The caldb4 upgrade in question provides for the proper selection of time-displaced CalDB files with overlapping validity date ranges, which does not exist in earlier CIAO versions. CalDB 4.8.0 is optimized for CIAO 4.8 (or later version) which includes an upgrade to the acis_process_events tool for handling CC-mode observations, specifically to perform the CTI-correction properly for them. As a result, the EDSER SUBPIX algorithm is now applied in a-p-e to CC-modes, where it had not been before. These instructions apply only to those wanting to upgrade their CalDB version 4.7.3 or later version to version 4.8.0 quickly, installing only the updated calibration files and index updates. You must have installed CalDB 4.7.3 or later version previously, or this upgrade installation will not work correctly. Be sure to use the CIAO command "check_ciao_caldb" to verify your CalDB version before proceeding with these instructions. Condensed Release Notes for CalDB 4.8.0 ------------------------------------------------------------------------------------------ I. INTRODUCTION CalDB 4.8.0 is an upgrade to the Chandra CalDB, which includes the following items: * ACIS T_GAIN Epoch 73+74 updates (Feb 01 - Jul 31 2018) * ACIS early N0005 T_GAIN header corrections (47 files) For the CIAO 4.10 / CalDB 4.8.0 release notes see How CalDB 4.8.0 Affects Your Analysis at http://cxc.harvard.edu/ciao/releasenotes/ciao_4.10_release.html#HowCALDB4.8.0AffectsYourAnalysis ------------------------------------------------------------------------------------------ II. SUMMARY OF CHANGES A. ACIS T_GAIN Epochs 73+74 (Feb-Jul 2018) Location: $CALDB/data/chandra/acis/t_gain/ Filenames: acisD2017-11-01t_gainN0005.fits acisD2017-11-01t_gainN0006.fits acisD2018-05-02t_gainN0002.fits acisD2018-05-02t_gain_biN0002.fits The ACIS Calibration team has released the semiannual time-dependent gain (T_GAIN) updates from analysis of ECS data for the period Feb 01 - Jul 31 2018 (epochs 73+74), and hence the above files have been generated as a result, for immediate release. The secular time variations in the corrections calculated from these new data are similar in magnitude and direction to previous upgrades, wherein maximal changes are a few percent or less of the deltaE/E value. In particular the variations at the ACIS-I aimpoint are significant, whereas those at the ACIS-S aimpoint are very small for this period and the previous epochal period (epochs 71+72, Aug 01 2017 - Jan 31 2018) which is now finalized. NOTE: Users with observations taken between (DATE-OBS=) 2017-11-01T12:00:00 and 2018-05-02T12:00:00 are advised to reprocess with their data with finalized versions of the applicable T_GAIN files. Users with observations taken since 2018-05-02T12:00:00 will have access to an updated provisional T_GAIN file (one of the version N0002 filenames above); They will not however have a finalized file until after Epoch 76 (Nov 01 2018 - Jan 31 2019) is completed. For a presentation of the specific values, see the Technical Details Section for these data in the CalDB 4.8.0 release notes. For more information on the ACIS time-dependent gain in general, see the ACIS T_GAIN why page. PIPELINES/TOOLS AFFECTED: ACIS Level 1 pipeline tool acis_process_events. CIAO script chandra_repro uses acis_process_events to reprocess ACIS observation data. THREADS AFFECTED: Reprocessing Data to Create a New Level=2 Event File B. ACIS T_GAIN N0005 Early Epochs Header Modifications Location: $CALDB/data/chandra/acis/t_gain/ Filenames: acisD2000-01-29t_gainN0005.fits acisD2000-05-01t_gainN0005.fits acisD2000-08-01t_gainN0005.fits acisD2000-11-01t_gainN0005.fits acisD2001-02-01t_gainN0005.fits acisD2001-05-01t_gainN0005.fits acisD2001-08-01t_gainN0005.fits acisD2001-11-01t_gainN0005.fits acisD2002-02-01t_gainN0005.fits acisD2002-05-01t_gainN0005.fits acisD2002-08-01t_gainN0005.fits acisD2002-11-01t_gainN0005.fits acisD2003-02-01t_gainN0005.fits acisD2003-05-01t_gainN0005.fits acisD2003-08-01t_gainN0005.fits acisD2003-10-19t_gainN0005.fits acisD2003-11-18t_gainN0005.fits acisD2004-01-01t_gainN0005.fits acisD2004-02-01t_gainN0005.fits acisD2004-05-01t_gainN0005.fits acisD2004-08-01t_gainN0005.fits acisD2004-11-01t_gainN0005.fits acisD2005-02-01t_gainN0005_revA.fits acisD2005-05-01t_gainN0005_revA.fits acisD2005-08-02t_gainN0005.fits acisD2005-11-01t_gainN0005.fits acisD2006-02-01t_gainN0005.fits acisD2006-05-01t_gainN0005.fits acisD2006-08-01t_gainN0005.fits acisD2006-11-01t_gainN0005.fits acisD2007-02-01t_gainN0005.fits acisD2007-05-01t_gainN0005.fits acisD2007-08-01t_gainN0005.fits acisD2007-11-01t_gainN0005.fits acisD2008-02-01t_gainN0005.fits acisD2008-05-01t_gainN0005.fits acisD2008-08-01t_gainN0005.fits acisD2008-11-01t_gainN0005.fits acisD2009-02-01t_gainN0005.fits acisD2009-05-01t_gainN0005.fits acisD2009-08-01t_gainN0005.fits acisD2010-02-01t_gainN0005.fits acisD2010-05-01t_gainN0005_revA.fits acisD2010-08-01t_gainN0005.fits acisD2010-11-01t_gainN0005.fits acisD2011-02-01t_gainN0005.fits acisD2011-05-01t_gainN0005.fits The ACIS Calibration Team has begun analysis peculiar to their own purposes of External Calibration Source (ECS) data taken with warmer than usual FP_TEMP values; part of their procedure is to reprocess these ERs removing the CTI correction, but having the "apply_tgain" parameter set to "yes", so that a T_GAIN file should still be selected and applied. Ordinarily these ERs are not in GRADED data mode, so that the N0006 (full CTI correction) T_GAIN files are selected for these. With the CTI correcction OFF, CTI_APP = NNNNNNNNNN, and the corresponding N0005 file would be selected. For some of the warmer FP_TEMPs, however, it has been found that the default FP_TEMP boundary condition range was set to (148.2-158.2)K, which for some 24 ERs, a-p-e returns "NO FILES FOUND" for the T_GAIN, and turns off the apply_tgain parameter, leaving the data uncorrected. There were however four additional cases where ERs were taken with normal FP_TEMPs, but still no T_GAIN could be found. It was determined that these cases were at periods crossing the epochal boundaries of the T_GAIN files. The effective date ranges for these cases had been fixed for the usual N0006 (CTI-corrected) T_GAINs in CalDB 4.5.0 (June 2012). See also the CalDB caveats page 38 ACIS OBS_IDs with "Problem Reading TGAINFILE However the corresponding N0005 files had not been corrected for this situation. Hence, turning off the CTI correction causes the T_GAIN CalDB search to fail for them. The solution is to correct the Calibration Validity End Dates CVED* (and Times) CVET* in those N0005 file headers to be the same as those for the corresponding N0006 files. It has been decided for the sake of consistency that all of the -120C N0005 T_GAIN files from 29 Jan 2000 through 2011 May 01 should have their FP_TEMP ranges reset to (148.2 - 188.2)K, the standard setting used for all current T_GAINs for the OBS_MODE=SECONDARY case, which is the setting for the ACIS ECS ERs. Such a change will not affect the reprocessing of ordinary observations. It will, however, allow the procedure used by the ACIS Calibration team to be performed on any observation (ER or OR), with a non-CTI T_GAIN always being available, regardless of the FP_TEMP, the DATE-OBS, or the DATE-END of the observation. ------------------------------------------------------------------------------------------ III. Instructions for installing the upgrade from CalDB 4.7.3 or later version to 4.8.0. The following are manual installation instructions. You may also download and install CalDB 4.8.0 using the "ciao-install" script at http://cxc.harvard.edu/ciao/download/, or follow the instruction at http://cxc.cfa.harvard.edu/caldb/downloads/index.html. Before you begin, you must have write privileges in your local CalDB area. If you don't have your system administrator do this upgrade! A. As system admin, verify your current CalDB version. With CIAO installed and loaded, enter the command "check_ciao_caldb" If you do not see that the CalDB version is 4.7.3 or later version, you need to use the MAIN CalDB download procedure from Table 1 of the CIAO CalDB downloads page. If the version shown is 4.7.3 or later, proceed with the following: B. Downloading Instructions: Set your working directory to $CALDB, and verify that you are there. Please download the "upgrade" tar ball and either or both of the other tar balls you may need (as indicated below) into the $CALDB area. The following FTP links will allow you to access the files from the Chandra Data Archive. You may also use your web browser to locate the CalDB Download instructions, and proceed from there: http://cxc.harvard.edu/caldb/downloads/index.html CALDB UPGRADE FILE: ftp://cda.harvard.edu/pub/arcftp/caldb/upgrade/caldb_4.8.0_upgrade.tar.gz The ACIS BKGRND files have been updated to include (1) eleven N0002 files in Group G for 2012-01-01 through the present. Added as of the release of CalDB 4.7.6 on 18 Aug 2017. ftp://cda.harvard.edu/pub/arcftp/caldb/acis_bkgrnd_4.7.6.tar.gz Three new HRC-I BKGRND files, effective for calendar years 2015-2017 respectively, have been added to the CalDB as of version 4.7. (14 Dec 2017). As has been the case since their original addition to CalDB, they are packaged separately in the following tar file: ftp://cda.harvard.edu/pub/arcftp/caldb/hrc_bkgrnd_4.7.7.tar.gz TABLE 1: Files, sizes, and md5sum values: FILENAME Ship Size Unpacking space md5sum (bytes) caldb_4.8.0_upgrade.tar.gz 1,496,162,584 3.9 GB cc80e98ec23969ab3953aa6a972e5a8e acis_bkgrnd_4.7.6.tar.gz 1,387,251,364 2.9 GB 192d4871813c9d5d3e857726195d1261 hrc_bkgrnd_4.7.7.tar.gz 2,323,747,030 3.2 GB 3c9b4e5d6966929793d726b12ae4a4b2 C. Unpacking instructions: We prefer that you use gtar with the appropriate options to unzip and untar the files in a single step, to save disk space and to apply the safest unpacking procedures: gtar xvUfz caldb_4.8.0_upgrade.tar.gz gtar xvfz acis_bkgrnd_4.7.6.tar.gz gtar xvfz hrc_bkgrnd_4.7.7.tar.gz LINUX users: You may omit the "U" uption in the first gtar command. D. Upgrade verification checks See the CalDB downloads web page for steps to verify your upgrade installation, at http://cxc.harvard.edu/caldb/downloads/index.html#dl_config. - Dale Graessle Chandra CalDB Manager