README File for CALDB 4.9.2.1 Upgrade Only Instructions 14 July 2020 DE Graessle, Chandra CALDB Manager ======================================================================== This file was updated to include a "rev_A" version of the CalDB 4.9.2.1 upgrade tar file for downloading. The "rev_A" version was generated to remove a second copy of the P2_RESP N0008 ACIS FITS file in the tar ball, which caused problems on Mac systems, in particular. ======================================================================== Includes brief Intro, Upgrade download and installation instructions, and pointers for further assistance. To learn more about the Chandra CALDB, and to find this information on the WEB, you are referred to http://cxc.harvard.edu/caldb/index.html. For Release Notes to the Chandra CALDB4, and specifically CalDB 4.9.2.1, see http://cxc.harvard.edu/caldb/downloads/releasenotes.html These instructions apply only to those wanting to upgrade their CalDB version 4.9.0 or later version to version 4.9.2.1 quickly, installing only the updated calibration files and index updates. You must have installed CalDB 4.9.0 or later version previously, or this upgrade installation will not work correctly. CalDB 4.9.2.1 is compatible with CIAO 4.12 or later versions only. It is NOT to be used with CIAO 4.11 or any ealier versions of CIAO. Be sure to use the CIAO command "check_ciao_caldb" to verify your CalDB version before proceeding with these instructions. Condensed Release Notes for CalDB 4.9.2.1 ------------------------------------------------------------------------------------------ I. Introduction CalDB 4.9.2.1 is an upgrade to the Chandra CalDB for public release, which includes CalDB 4.9.2. * ACIS FULL T_GAIN LIBRARY UPGRADE Epochs 1-80 * ACIS FP_TEMP Boundary Condition Updates for certain BADPIX, CTI, DET_GAIN, FEF_PHA, OSIP, P2_RESP, QEU, and the most recent T_GAIN files. * HRC-S T_GMAP upgrades for the full mission time line. * HRC-S QE Version N0015 file set * HRC-I QE Version N0012 file set * ACIS BKGRND FILES Group G Blank-sky/Group F Stowed with header (FP_TEMP) mods only For the CIAO 4.12 / CalDB 4.9.2.1 release notes see How CalDB 4.9.2.1 Affects Your Analysis, at http://cxc.harvard.edu/ciao/releasenotes/ciao_4.12_release.html ------------------------------------------------------------------------------------------ II. SUMMARY OF CHANGES A. A FULL UPGRADE to the ACIS T_GAIN LIBRARY Location: $CALDB/data/chandra/acis/t_gain/ Filenames: acisDYYYY-MM-DDt_gainN0008.fits acisDYYYY-MM-DDt_gainN0008B.fits acisD2019-11-01t_gain_bN0002_revA.fits acisD2019-11-01t_gainN0002_revA.fits where "YYYY-MM-DD" in the N0008 and N0008B filenames above are the Calibration Validity Start Dates (CVSD*) values in the Block 2 FITS HDUs of each file. There are 73 pairs of these files extending from 2000-01-29 (the beginning of normal -120C ACIS operations) until 2019-05-02 (ACIS T_GAIN combined epochs 77 and 78). With the release of ACIS T_GAIN Epoch 75-76 (CalDB 4.8.3, May 2019) the ACIS Calibration Team introduced and new algorithm designed optimally for averaging of data with diminished statistics. The results were a marked improvement in T_GAIN correction factor derivation. With the advent of the improved ACIS DET_GAIN N0008 file (CalDB 4.9.1, April 2020), ACIS Cal began to review the earlier ACIS T_GAIN epochs with the new gains applied to the data. The resulting new T_GAIN correction data showed a marked improvement over the previously-released T_GAIN files. See Item A of the Technical Details Section below for details. The ACIS Cal Team has decided to upgrade T_GAIN files for all -120C epochs (specifically epochs 1-74). It was also found that recompiling the T_GAIN data with (CHIPX x CHIPY) of (32 x 128) could reduce the size of the files significantly without compromising T_GAIN correction detailing. Hence all 148 active files are being upgraded (all of epochs 1-80) in this release. PIPELINES/TOOLS AFFECTED: ACIS Level 1 pipeline tool acis_process_events. CIAO script chandra_repro uses acis_process_events to reprocess ACIS observation data. THREADS AFFECTED: Reprocessing Data to Create a New Level=2 Event File B. ACIS FP_TEMP Boundary Condition Updates (Effective 01 July 2020) Location: $CALDB/data/chandra/acis/badpix/ Filenames: acisD2000-11-28badpixN0004.fits acisD2000-11-28badpixN0004B.fits Location: $CALDB/data/chandra/acis/cti/ Filenames: acisD2015-01-01ctiN0009.fits acisD2002-08-01ctiN0009B.fits Location: $CALDB/data/chandra/acis/det_gain/ Filenames: acisD2000-01-29gain_ctiN0005.fits acisD2000-01-29gain_ctiN0008.fits acisD2000-01-29gainN0005.fits Location: $CALDB/data/chandra/acis/fef_pha/ Filenames: acisD2000-01-29fef_pha_ctiN0004.fits acisD2000-01-29fef_phaN0005.fits Location: $CALDB/data/chandra/acis/osip/ Filenames: acisD2000-01-29osip_ctiN0005.fits acisD2000-01-29osip_ctiN0006.fits acisD2000-01-29osipN0006.fits Location: $CALDB/data/chandra/acis/p2_resp/ Filenames: acisD2000-01-29p2_respN0005.fits acisD2000-01-29p2_respN0008.fits Location: $CALDB/data/chandra/acis/qeu/ Filenames: acisD2010-02-01qeuN0005B.fits acisD2010-02-01qeuN0007.fits Location: $CALDB/data/chandra/acis/t_gain/ Filenames: acisD2019-11-01t_gain_biN0002_revA.fits acisD2019-11-01t_gainN0002_revA.fits As the Chandra spacecraft ages, it becomes increasingly difficult to maintain the ACIS focal plane temperature (FP_TEMP in the software and FITS data sets) firmly at -120C for the entire observing period of each orbit. This is particularly true during periods approaching and exiting the Earth's radiation zone. The CXC Mission Planning team has been considering allowing science observations to be scheduled during these warmer periods, specifically allowing ACIS-S/HETG and certain restricted imaging observations with reduced statistics to be taken with FP_TEMP up to -109C. The warmer temperatures were approved in advance of the 06 July 2020 observing load. The files listed above in the ACIS branch of CalDB have a boundary condition on FP_TEMP that is likely too restrictive for planned -109C observations. The boundary condition is currently set as 'FP_TEMP(151.16 - 164.16)K' (-121.99C - -108.99C). With this release, the boundary condition has been changed to 'FP_TEMP(148.15-168.16)K', or (-125.0 - -104.99)C, effective 2020-07-01T00:00:00 for all of the above CalDB files, except for the two provisional T_GAIN files, wherein the CBD20001 keyword has simply been changed without changing the effective date. (Since these are provisional files, they will be upstaged in the near future regardless, and so there was no need to add any further time-dependence to their listings.) For all of the above sixteen files excluding the T_GAINs, an extra CalDB keyword set applicable to science observations has been added, including CCNM* (codename or product), CDES* (alibration description), CVSD* (cal validity start date), CVST* (cal validity start time), and boundary conditions including the new FP_TEMP expanded range as required for proper file selection for dates after CVSD*=2020-07-01T00:00:00. Here "*" indicates the version number of the added keyword set, sufficient to distinguish it from pre-existing ones, and hence must be '0002' or higher version. Details are given in the Technical Details seciton below. These added keyword sets produce a new line for each relevant FITS file/Extension in the ACIS branch CalDB index file, whence it may be selected in the software for new, including warmer, observations. C. HRC-S Time-dependent Gain Map (T_GMAP) Version 3 Location: $CALDB/data/chandra/hrc/t_gmap/ Filenames: hrcsD1999-07-22t_gmapN0003.fits hrcsD2012-03-29t_gmapN0003.fits The rapid loss of HRC-S gain since the year 2015 has precipitated problems with filtering background events from LETG/HRC-S observation spectra, which is the basic purpose of setting the gain calibration on HRC-S. The HRC-S/LETG calibration team, specifically Brad Wargelin, has engaged in a 2-year project to develop a new time-dependent gain map method, to be applied in two separate T_GMAPs, one for before the 2012-03-29 HV change, and one for after that time. The new T_GMAPs require a different branch of software to apply them in hrc_process_events, which was released in CIAO 4.12 (and with DS 10.8.2), but which will only become active with the installation of this CalDB release. See the HRC-S T_GMAP why page for more information. PIPELINES/TOOLS AFFECTED: HRC Level 1 Processing Tool (SDP): hrc_process_events CIAO reprocessing script: chandra_repro, which uses hrc_process_events. THREADS AFFECTED: Reprocessing Data to Create a New Level=2 Event File D. HRC-S QE Version 15 Location: $CALDB/data/chandra/hrc/qe/ Filenames: hrcsD1999-07-22qeN0015.fits hrcsD2012-03-29qeN0015.fits hrcsD2013-01-01qeN0015.fits hrcsD2014-01-01qeN0015.fits hrcsD2015-01-01qeN0015.fits hrcsD2016-01-01qeN0015.fits hrcsD2017-01-01qeN0015.fits hrcsD2018-01-01qeN0015.fits hrcsD2019-01-01qeN0015.fits hrcsD2020-01-01qeN0015.fits hrcsD2021-01-01qeN0015.fits hrcsD2022-01-01qeN0015.fits Regular calibration observations have shown that the HRC-S QE is declining at long wavelengths (>60Å, or below 200 eV) for some time, decaying linearly since the 2012-03-29 high-voltage adjustment onboard. This spectral variation in the QE loss will, with this CalDB release, be accounted for using a series of time-displaced QE files in CalDB, which are annually adjusted each year beginning in 2013. See the HRC-S QE/QEU why page for summary information on the current status of the HRC-S QE and QEU calibrations. See the technical details section below for more specific information on this new calibration and its derivation. PIPELINES/TOOLS AFFECTED: SDP HRC-S Level 2 python script: tg_mkresponse.py generates +/- first order grating ARFs for LETG/HRC-S OBS_IDs. CIAO contributed tool: mktgresp will generate first and higher order as specified in a Type II PHA file, or according to a user specification of the "orders" parameter. THREADS AFFECTED: CiAO thread using the HRC-S QE: LETG/HRC-S Grating ARFs E. HRC-I QE Version 12 Location: $CALDB/data/chandra/hrc/qe/ Filenames: hrciD1999-07-29qeN0012.fits hrciD2000-07-23qeN0012.fits hrciD2001-04-19qeN0012.fits hrciD2001-10-16qeN0012.fits hrciD2002-04-14qeN0012.fits hrciD2002-10-25qeN0012.fits hrciD2003-04-25qeN0012.fits hrciD2003-10-06qeN0012.fits hrciD2004-04-05qeN0012.fits hrciD2004-11-05qeN0012.fits hrciD2005-05-15qeN0012.fits hrciD2005-11-01qeN0012.fits hrciD2006-04-25qeN0012.fits hrciD2006-11-21qeN0012.fits hrciD2007-09-16qeN0012.fits hrciD2008-09-11qeN0012.fits hrciD2009-09-15qeN0012.fits hrciD2010-09-16qeN0012.fits hrciD2011-09-29qeN0012.fits hrciD2012-09-29qeN0012.fits hrciD2013-10-05qeN0012.fits hrciD2014-10-03qeN0012.fits hrciD2015-08-09qeN0012.fits hrciD2016-06-15qeN0012.fits hrciD2017-02-20qeN0012.fits hrciD2017-08-20qeN0012.fits hrciD2018-02-16qeN0012.fits hrciD2018-08-25qeN0012.fits hrciD2019-02-27qeN0012.fits hrciD2019-08-24qeN0012.fits hrciD2020-02-20qeN0012.fits The HRC-I Calibration team has updated the QE files for HRC-I following on from the 2018 version (v11) which introduced a time-dependence into the HRC-I QE. In this version, the team extends the time dependence to include the latest epoch of observation. During recent calibration observations, a steady decline has been seen in the observed count rates and estimated fluxes of the soft source white dwarf system HZ43 and the hard source SNR G21.5-0.9. This upgrade maintains consistency with the HRC-S instrumental QE, particularly at low energies (below 626 eV), where the shape of the QE(E) is maintained in common. PIPELINES/TOOLS AFFECTED: CIAO contributed script: fluximage THREADS AFFECTED: CIAO thread: HRC-I Exposure Map and Exposure-corrected Image ------------------------------------------------------------------------------------------ III. Instructions for installing the upgrade from CalDB 4.9.0 or later version to 4.9.2.1. The following are manual installation instructions. You may also download and install CalDB 4.9.2.1 using the "ciao-install" script at http://cxc.harvard.edu/ciao/download/, or follow the instruction at http://cxc.cfa.harvard.edu/caldb/downloads/index.html. Before you begin, you must have write privileges in your local CalDB area. If you don't have your system administrator do this upgrade! A. As system admin, verify your current CalDB version. With CIAO installed and loaded, enter the command "check_ciao_caldb" If you do not see that the CalDB version is 4.9.0 or later version, you need to use the MAIN CalDB download procedure from Table 1 of the CIAO CalDB downloads page. If the version shown is 4.9.0 or later, proceed with the following: B. Downloading Instructions: Set your working directory to $CALDB, and verify that you are there. Please download the "upgrade" tar ball and either or both of the other tar balls you may need (as indicated below) into the $CALDB area. The following FTP links will allow you to access the files from the Chandra Data Archive. You may also use your web browser to locate the CalDB Download instructions, and proceed from there: http://cxc.harvard.edu/caldb/downloads/index.html CALDB UPGRADE FILE: ftp://cda.harvard.edu/pub/arcftp/caldb/upgrade/caldb_4.9.2.1_upgrade.tar.gz The ACIS BKGRND files have been updated to include (1) eleven N0002 files in Group G for 2012-01-01 through the present. Some header updates are being released as of the with CalDB 4.9.2.1 on 09 Jul 2020. ftp://cda.harvard.edu/pub/arcftp/caldb/acis_bkgrnd_4.9.2.1.tar.gz Three new HRC-I BKGRND files, effective for calendar years 2015-2017 respectively, have been added to the CalDB as of version 4.7.7 (14 Dec 2017). As has been the case since their original addition to CalDB, they are packaged separately in the following tar file: ftp://cda.harvard.edu/pub/arcftp/caldb/hrc_bkgrnd_4.7.7.tar.gz TABLE 1: Files, sizes, and md5sum values: FILENAME Ship Size Unpacking space md5sum (bytes) caldb_4.9.2.1_upgrade_revA.tar.gz 1,251,075,333 3.2 GB c1f046b19b89303b0e2d553756a66ce1 acis_bkgrnd_4.9.2.1.tar.gz 1,387,254,665 2.9 GB ea9c6dec632d6efae242b426513cc3f4 hrc_bkgrnd_4.7.7.tar.gz 2,323,747,030 3.2 GB 3c9b4e5d6966929793d726b12ae4a4b2 C. Unpacking instructions: We prefer that you use gtar with the appropriate options to unzip and untar the files in a single step, to save disk space and to apply the safest unpacking procedures: gtar xvUfz caldb_4.9.2.1_upgrade_revA.tar.gz gtar xvfz acis_bkgrnd_4.7.6.tar.gz gtar xvfz hrc_bkgrnd_4.7.7.tar.gz LINUX users: You may omit the "U" uption in the first gtar command. D. Upgrade verification checks See the CalDB downloads web page for steps to verify your upgrade installation, at http://cxc.harvard.edu/caldb/downloads/index.html#dl_config. - Dale Graessle Chandra CalDB Manager