README File for CALDB 4.9.2.1 Full Download Instructions 08 July 2020 DE Graessle, Chandra CALDB Manager Includes brief Intro, Download instructions, Unpacking instructions, Pointers for further assistance. To learn more about the Chandra CALDB, and to find this information on the WEB, you are referred to http://cxc.harvard.edu/caldb/index.html. For Release Notes to the Chandra CALDB4, and specifically CalDB 4.9.2.1, see http://cxc.harvard.edu/caldb/downloads/releasenotes.html This README refers to the FULL CalDB 4.9.2.1 installation, which should only be necessary here forward with a new installation of CIAO 4.12. CalDB 4.9.2.1 is an upgrade to CalDB 4.9.0. For those who have already downloaded and installed CIAO 4.12 with CalDB 4.9.0, only the upgrade file is actually necessary to upgrade your CalDB. CalDB 4.9.2.1 is compatible with CIAO 4.12 and later versions ONLY! Please do not attempt to upgrade your CalDB for CIAO 4.11 or older versions with CalDB version 4.9.2.1. None of the ACIS upgrades in CalDB 4.9.0 will be applied with the older software because of required software changes to do so. Furthermore, the HRC-S chip edge offset values will not be applied in CIAO 4.11 with CalDB 4.9.2.1, and users will not be notified by the software if this occurs. Condensed Release Notes for CalDB 4.9.2.1 ------------------------------------------------------------------------------------------ I. Introduction CalDB 4.9.2.1 is an upgrade to the Chandra CalDB for public release, which includes CalDB 4.9.2. * ACIS FULL T_GAIN LIBRARY UPGRADE Epochs 1-80 * ACIS FP_TEMP Boundary Condition Updates for certain BADPIX, CTI, DET_GAIN, FEF_PHA, OSIP, P2_RESP, QEU, and the most recent T_GAIN files. * HRC-S T_GMAP upgrades for the full mission time line. * HRC-S QE Version N0015 file set * HRC-I QE Version N0012 file set * ACIS BKGRND FILES Group G Blank-sky/Group F Stowed with header (FP_TEMP) mods only For the CIAO 4.12 / CalDB 4.9.2.1 release notes see How CalDB 4.9.2.1 Affects Your Analysis, at http://cxc.harvard.edu/ciao/releasenotes/ciao_4.12_release.html ------------------------------------------------------------------------------------------ II. SUMMARY OF CHANGES A. A FULL UPGRADE to the ACIS T_GAIN LIBRARY Location: $CALDB/data/chandra/acis/t_gain/ Filenames: acisDYYYY-MM-DDt_gainN0008.fits acisDYYYY-MM-DDt_gainN0008B.fits acisD2019-11-01t_gain_bN0002_revA.fits acisD2019-11-01t_gainN0002_revA.fits where "YYYY-MM-DD" in the N0008 and N0008B filenames above are the Calibration Validity Start Dates (CVSD*) values in the Block 2 FITS HDUs of each file. There are 73 pairs of these files extending from 2000-01-29 (the beginning of normal -120C ACIS operations) until 2019-05-02 (ACIS T_GAIN combined epochs 77 and 78). With the release of ACIS T_GAIN Epoch 75-76 (CalDB 4.8.3, May 2019) the ACIS Calibration Team introduced and new algorithm designed optimally for averaging of data with diminished statistics. The results were a marked improvement in T_GAIN correction factor derivation. With the advent of the improved ACIS DET_GAIN N0008 file (CalDB 4.9.1, April 2020), ACIS Cal began to review the earlier ACIS T_GAIN epochs with the new gains applied to the data. The resulting new T_GAIN correction data showed a marked improvement over the previously-released T_GAIN files. See Item A of the Technical Details Section below for details. The ACIS Cal Team has decided to upgrade T_GAIN files for all -120C epochs (specifically epochs 1-74). It was also found that recompiling the T_GAIN data with (CHIPX x CHIPY) of (32 x 128) could reduce the size of the files significantly without compromising T_GAIN correction detailing. Hence all 148 active files are being upgraded (all of epochs 1-80) in this release. PIPELINES/TOOLS AFFECTED: ACIS Level 1 pipeline tool acis_process_events. CIAO script chandra_repro uses acis_process_events to reprocess ACIS observation data. THREADS AFFECTED: Reprocessing Data to Create a New Level=2 Event File B. ACIS FP_TEMP Boundary Condition Updates (Effective 01 July 2020) Location: $CALDB/data/chandra/acis/badpix/ Filenames: acisD2000-11-28badpixN0004.fits acisD2000-11-28badpixN0004B.fits Location: $CALDB/data/chandra/acis/cti/ Filenames: acisD2015-01-01ctiN0009.fits acisD2002-08-01ctiN0009B.fits Location: $CALDB/data/chandra/acis/det_gain/ Filenames: acisD2000-01-29gain_ctiN0005.fits acisD2000-01-29gain_ctiN0008.fits acisD2000-01-29gainN0005.fits Location: $CALDB/data/chandra/acis/fef_pha/ Filenames: acisD2000-01-29fef_pha_ctiN0004.fits acisD2000-01-29fef_phaN0005.fits Location: $CALDB/data/chandra/acis/osip/ Filenames: acisD2000-01-29osip_ctiN0005.fits acisD2000-01-29osip_ctiN0006.fits acisD2000-01-29osipN0006.fits Location: $CALDB/data/chandra/acis/p2_resp/ Filenames: acisD2000-01-29p2_respN0005.fits acisD2000-01-29p2_respN0008.fits Location: $CALDB/data/chandra/acis/qeu/ Filenames: acisD2010-02-01qeuN0005B.fits acisD2010-02-01qeuN0007.fits Location: $CALDB/data/chandra/acis/t_gain/ Filenames: acisD2019-11-01t_gain_biN0002_revA.fits acisD2019-11-01t_gainN0002_revA.fits As the Chandra spacecraft ages, it becomes increasingly difficult to maintain the ACIS focal plane temperature (FP_TEMP in the software and FITS data sets) firmly at -120C for the entire observing period of each orbit. This is particularly true during periods approaching and exiting the Earth's radiation zone. The CXC Mission Planning team has been considering allowing science observations to be scheduled during these warmer periods, specifically allowing ACIS-S/HETG and certain restricted imaging observations with reduced statistics to be taken with FP_TEMP up to -109C. The warmer temperatures were approved in advance of the 06 July 2020 observing load. The files listed above in the ACIS branch of CalDB have a boundary condition on FP_TEMP that is likely too restrictive for planned -109C observations. The boundary condition is currently set as 'FP_TEMP(151.16 - 164.16)K' (-121.99C - -108.99C). With this release, the boundary condition has been changed to 'FP_TEMP(148.15-168.16)K', or (-125.0 - -104.99)C, effective 2020-07-01T00:00:00 for all of the above CalDB files, except for the two provisional T_GAIN files, wherein the CBD20001 keyword has simply been changed without changing the effective date. (Since these are provisional files, they will be upstaged in the near future regardless, and so there was no need to add any further time-dependence to their listings.) For all of the above sixteen files excluding the T_GAINs, an extra CalDB keyword set applicable to science observations has been added, including CCNM* (codename or product), CDES* (alibration description), CVSD* (cal validity start date), CVST* (cal validity start time), and boundary conditions including the new FP_TEMP expanded range as required for proper file selection for dates after CVSD*=2020-07-01T00:00:00. Here "*" indicates the version number of the added keyword set, sufficient to distinguish it from pre-existing ones, and hence must be '0002' or higher version. Details are given in the Technical Details seciton below. These added keyword sets produce a new line for each relevant FITS file/Extension in the ACIS branch CalDB index file, whence it may be selected in the software for new, including warmer, observations. C. HRC-S Time-dependent Gain Map (T_GMAP) Version 3 Location: $CALDB/data/chandra/hrc/t_gmap/ Filenames: hrcsD1999-07-22t_gmapN0003.fits hrcsD2012-03-29t_gmapN0003.fits The rapid loss of HRC-S gain since the year 2015 has precipitated problems with filtering background events from LETG/HRC-S observation spectra, which is the basic purpose of setting the gain calibration on HRC-S. The HRC-S/LETG calibration team, specifically Brad Wargelin, has engaged in a 2-year project to develop a new time-dependent gain map method, to be applied in two separate T_GMAPs, one for before the 2012-03-29 HV change, and one for after that time. The new T_GMAPs require a different branch of software to apply them in hrc_process_events, which was released in CIAO 4.12 (and with DS 10.8.2), but which will only become active with the installation of this CalDB release. See the HRC-S T_GMAP why page for more information. PIPELINES/TOOLS AFFECTED: HRC Level 1 Processing Tool (SDP): hrc_process_events CIAO reprocessing script: chandra_repro, which uses hrc_process_events. THREADS AFFECTED: Reprocessing Data to Create a New Level=2 Event File D. HRC-S QE Version 15 Location: $CALDB/data/chandra/hrc/qe/ Filenames: hrcsD1999-07-22qeN0015.fits hrcsD2012-03-29qeN0015.fits hrcsD2013-01-01qeN0015.fits hrcsD2014-01-01qeN0015.fits hrcsD2015-01-01qeN0015.fits hrcsD2016-01-01qeN0015.fits hrcsD2017-01-01qeN0015.fits hrcsD2018-01-01qeN0015.fits hrcsD2019-01-01qeN0015.fits hrcsD2020-01-01qeN0015.fits hrcsD2021-01-01qeN0015.fits hrcsD2022-01-01qeN0015.fits Regular calibration observations have shown that the HRC-S QE is declining at long wavelengths (>60Å, or below 200 eV) for some time, decaying linearly since the 2012-03-29 high-voltage adjustment onboard. This spectral variation in the QE loss will, with this CalDB release, be accounted for using a series of time-displaced QE files in CalDB, which are annually adjusted each year beginning in 2013. See the HRC-S QE/QEU why page for summary information on the current status of the HRC-S QE and QEU calibrations. See the technical details section below for more specific information on this new calibration and its derivation. PIPELINES/TOOLS AFFECTED: SDP HRC-S Level 2 python script: tg_mkresponse.py generates +/- first order grating ARFs for LETG/HRC-S OBS_IDs. CIAO contributed tool: mktgresp will generate first and higher order as specified in a Type II PHA file, or according to a user specification of the "orders" parameter. THREADS AFFECTED: CiAO thread using the HRC-S QE: LETG/HRC-S Grating ARFs E. HRC-I QE Version 12 Location: $CALDB/data/chandra/hrc/qe/ Filenames: hrciD1999-07-29qeN0012.fits hrciD2000-07-23qeN0012.fits hrciD2001-04-19qeN0012.fits hrciD2001-10-16qeN0012.fits hrciD2002-04-14qeN0012.fits hrciD2002-10-25qeN0012.fits hrciD2003-04-25qeN0012.fits hrciD2003-10-06qeN0012.fits hrciD2004-04-05qeN0012.fits hrciD2004-11-05qeN0012.fits hrciD2005-05-15qeN0012.fits hrciD2005-11-01qeN0012.fits hrciD2006-04-25qeN0012.fits hrciD2006-11-21qeN0012.fits hrciD2007-09-16qeN0012.fits hrciD2008-09-11qeN0012.fits hrciD2009-09-15qeN0012.fits hrciD2010-09-16qeN0012.fits hrciD2011-09-29qeN0012.fits hrciD2012-09-29qeN0012.fits hrciD2013-10-05qeN0012.fits hrciD2014-10-03qeN0012.fits hrciD2015-08-09qeN0012.fits hrciD2016-06-15qeN0012.fits hrciD2017-02-20qeN0012.fits hrciD2017-08-20qeN0012.fits hrciD2018-02-16qeN0012.fits hrciD2018-08-25qeN0012.fits hrciD2019-02-27qeN0012.fits hrciD2019-08-24qeN0012.fits hrciD2020-02-20qeN0012.fits The HRC-I Calibration team has updated the QE files for HRC-I following on from the 2018 version (v11) which introduced a time-dependence into the HRC-I QE. In this version, the team extends the time dependence to include the latest epoch of observation. During recent calibration observations, a steady decline has been seen in the observed count rates and estimated fluxes of the soft source white dwarf system HZ43 and the hard source SNR G21.5-0.9. This upgrade maintains consistency with the HRC-S instrumental QE, particularly at low energies (below 626 eV), where the shape of the QE(E) is maintained in common. PIPELINES/TOOLS AFFECTED: CIAO contributed script: fluximage THREADS AFFECTED: CIAO thread: HRC-I Exposure Map and Exposure-corrected Image ------------------------------------------------------------------------------- III. Instructions for MANUALLY Downloading and Unpacking CALDB 4.9.2.1 NOTE: The installation of CalDB 4.9.2.1 may be done automatically by using the new ciao-install routine, as a matter of course, when installing or upgrading CIAO. See the instruction at "Downloading CIAO ", http://cxc.harvard.edu/ciao/download/ Instructions for Downloading (Manually): Download all files into the same directory, where you intend to install your Chandra CALDB 4.9.2.1. You will need at least 20 GBytes free to download and unpack the basic structure. The ACIS & HRC background files are now packaged separately, so you don't have to install it to get the basic package. The background files add another 6.1 GB to the space you need. See the following ftp links to acquire the CALDB download tar files for version 4.9.2.1. MAIN CALDB DIRECTORY AND FILES: 1. ftp://cda.harvard.edu/pub/arcftp/caldb/caldb_4.9.2.1_main.tar.gz The ACIS and HRC-I background files (blank-field files) are now packaged separately, so if you want to use them you will need files 2 and 3. You need these files if you are going to run reproject_events to generate background files for your ACIS or HRC-I observations. 2. The ACIS BKGRND files tar ball was most recently upgraded with CalDB 4.7.6, 18 Aug 2017 to include version N0002 of the Group G event lists (effective 2012-01-01 through the present.) Some header updates were included in CalDB 4.9.2.1 in July 2020. ftp://cda.harvard.edu/pub/arcftp/caldb/acis_bkgrnd_4.9.2.1.tar.gz 3. The HRC-I BKGRND files tar ball has been newly upgraded in CalDB 4.7.7 (14 Dec 2017), to include the new background files effective for 2015-2017 calendar years, respectively. ftp://cda.harvard.edu/pub/arcftp/caldb/hrc_bkgrnd_4.7.7.tar.gz TABLE 1: Files and sizes: FILE FILENAME Ship Size(byte) Unpacking space md5sum 1 caldb_4.9.2.1_main.tar.gz 1,708,287,688 5.0 GB 80c776b5169d299226e20e9d130681b7 2 acis_bkgrnd_4.9.2.1.tar.gz 1,387,254,665 2.9 GB ea9c6dec632d6efae242b426513cc3f4 3 hrc_bkgrnd_4.7.7.tar.gz 2,323,747,030 3.2 GB 3c9b4e5d6966929793d726b12ae4a4b2 Instructions for unpacking: 1. Locate a place for your CALDB. Designate an empty directory to be assigned to the value of $CALDB. Unpack file 1 above from that location. 2. It is not necessary to edit the ./software/tools/caldbinit.* files. When you install or load CIAO, the location of the CalDB gets set up to the appropriate directory. 3. Unpack the ACIS "background" files (File 2 in the table above) if you expect to use them. The files take up a large amount of space. 4. Unpack the HRC "background" files (File 3 in the table above) if you expect to use them. The files take up a large amount of space. 5. Delete the tar files. The latest versions will be maintained online at CXC at all times. -- Chandra CALDB Manager