README File for CALDB 4.8.4.1 Upgrade Only Instructions 11 Sep 2019 DE Graessle, Chandra CALDB Manager Includes brief Intro, Upgrade download and installation instructions, and pointers for further assistance. To learn more about the Chandra CALDB, and to find this information on the WEB, you are referred to http://cxc.harvard.edu/caldb/index.html. For Release Notes to the Chandra CALDB4, and specifically CalDB 4.8.4.1, see http://cxc.harvard.edu/caldb/downloads/releasenotes.html These instructions apply only to those wanting to upgrade their CalDB version 4.8.2 or later version to version 4.8.4.1 quickly, installing only the updated calibration files and index updates. You must have installed CalDB 4.8.2 or later version previously, or this upgrade installation will not work correctly. Be sure to use the CIAO command "check_ciao_caldb" to verify your CalDB version before proceeding with these instructions. Condensed Release Notes for CalDB 4.8.4.1 ------------------------------------------------------------------------------------------ I. Introduction CalDB 4.8.4.1 is an upgrade to the Chandra CalDB, which includes the following items: * ACIS T_GAIN Epochs 75+76 and 77+78 emergency corrections For the CIAO 4.11 / CalDB 4.8.4.1 release notes see How CalDB 4.8.4.1 Affects Your Analysis, at http://cxc.harvard.edu/ciao/releasenotes/ciao_4.11_release.html#HowCALDB4.8.4.1AffectsYourAnalysis ------------------------------------------------------------------------------------------ II. SUMMARY OF CHANGES A. Emergency Corrections for ACIS T_GAIN Epochs 75+76 and 77+78 Location: $CALDB/data/chandra/acis/t_gain/ Filenames: acisD2018-05-02t_gainN0005_revA.fits acisD2018-05-02t_gainN0006_revA.fits acisD2018-11-01t_gainN0005_revA.fits acisD2018-11-01t_gainN0006_revA.fits acisD2019-05-02t_gain_biN0002_revA.fits acisD2019-05-02t_gainN0002_revA.fits The ACIS calibration team has reported an undetected error in the calibration products for the combined T_GAIN epochs 75+76, and 77+78, covering the bi-epochal periods of Aug 2018 - Jan 2019, and Feb - July 2019. The error occurred due to a single line of code in the recently modified file generator of the calibration interface product files used by the CalDB Manager to generate the ACIS T_GAIN files employed in the CalDB. Only ACIS chips 0, 1, 2, 3 and 6 have been affected. This error produced alternating columns of zero t_gain correction values in the DELTAPHA column of the CIPs, which were then transfered to the CalDB T_GAIN files in their DELTPHA2 and DELTPHA1 columns. The errant files, affected columns, Standard Data Processing (SDP) installation dates, and corresponding corrected files are listed in Table 1 below. TABLE 1: List of Affected T_GAIN files by name, with details, and their corresponding corrected filenames in CalDB 4.8.4.1. Errant T_GAIN File FITS Columns Affected Errant File Errant File Corrected T_GAIN File by the Error CalDB Version SDP Installation Date acisD2018-05-02t_gainN0005.fits DELTPHA2 4.8.3 2019-05-24T01:06:00 acisD2018-05-02t_gainN0005_revA.fits acisD2018-05-02t_gainN0006.fits DELTPHA2 4.8.3 2019-05-24T01:06:00 acisD2018-05-02t_gainN0006_revA.fits acisD2018-11-01t_gainN0002.fits DELTPHA2 4.8.3 2019-05-24T01:06:00 acisD2018-11-01t_gainN0005_revA.fits acisD2018-11-01t_gain_biN0002.fits DELTPHA2 4.8.3 2019-05-24T01:06:00 acisD2018-11-01t_gainN0006_revA.fits acisD2018-11-01t_gainN0005.fits DELTPHA1, DELTPHA2 4.8.4 2019-08-28T19:11:12 acisD2018-11-01t_gainN0005_revA.fits acisD2018-11-01t_gainN0006.fits DELTPHA1, DELTPHA2 4.8.4 2019-08-28T19:11:12 acisD2018-11-01t_gainN0006_revA.fits acisD2019-05-02t_gain_biN0002.fits DELTPHA1, DELTPHA2 4.8.4 2019-08-28T19:11:12 acisD2019-05-02t_gain_biN0002_revA.fits acisD2019-05-02t_gainN0002.fits DELTPHA1, DELTPHA2 4.8.4 2019-08-28T19:11:12 acisD2019-05-02t_gainN0002_revA.fits These errant CalDB files were then released to the public, the first four files on 23 May 2019 with CalDB 4.8.3, and the latter four files on 09 Sep 2019 with CalDB 4.8.4. Hence, any reprocessing by users (e.g. with chandra_repro) with either CalDB would have affected any ACIS observations employing ACIS-0,1,2,3,4, or 6 with a DATE-OBS of 2018-05-02T12:00:00 or any later date. The error was detected by 05 September, and corrected in processing on 2019-09-06T17:31:43 (UTC). Any data sets processed in SDP after that date are unaffected by the error. (See the DATE keyword in observation data file headers.) Details describing the exact nature of the error are given in the technical details section below. DS Operations have determined that 302 OBS_IDs have been affected in processing by the errors in T_GAIN, including 127 ACIS-I observations, and 175 ACIS-S observations that used ACIS-6 (S2). Among these are seven ACIS/Grating observations which are on the public archive. All of these OBS_IDs will be reprocessed for the archive using the correctioned T_GAIN files released in CalDB 4.8.4.1. Users and PIs will be notified in a timely fashion of these errors and their appropriate reparations to the data. PIPELINES/TOOLS AFFECTED: ACIS Level 1 pipeline tool acis_process_events. CIAO script chandra_repro uses acis_process_events to reprocess ACIS observation data. ------------------------------------------------------------------------------------------ III. Instructions for installing the upgrade from CalDB 4.8.2 or later version to 4.8.4.1. The following are manual installation instructions. You may also download and install CalDB 4.8.4.1 using the "ciao-install" script at http://cxc.harvard.edu/ciao/download/, or follow the instruction at http://cxc.cfa.harvard.edu/caldb/downloads/index.html. Before you begin, you must have write privileges in your local CalDB area. If you don't have your system administrator do this upgrade! A. As system admin, verify your current CalDB version. With CIAO installed and loaded, enter the command "check_ciao_caldb" If you do not see that the CalDB version is 4.8.2 or later version, you need to use the MAIN CalDB download procedure from Table 1 of the CIAO CalDB downloads page. If the version shown is 4.8.2 or later, proceed with the following: B. Downloading Instructions: Set your working directory to $CALDB, and verify that you are there. Please download the "upgrade" tar ball and either or both of the other tar balls you may need (as indicated below) into the $CALDB area. The following FTP links will allow you to access the files from the Chandra Data Archive. You may also use your web browser to locate the CalDB Download instructions, and proceed from there: http://cxc.harvard.edu/caldb/downloads/index.html CALDB UPGRADE FILE: ftp://cda.harvard.edu/pub/arcftp/caldb/upgrade/caldb_4.8.4.1_upgrade.tar.gz The ACIS BKGRND files have been updated to include (1) eleven N0002 files in Group G for 2012-01-01 through the present. Added as of the release of CalDB 4.7.6 on 18 Aug 2017. ftp://cda.harvard.edu/pub/arcftp/caldb/acis_bkgrnd_4.7.6.tar.gz Three new HRC-I BKGRND files, effective for calendar years 2015-2017 respectively, have been added to the CalDB as of version 4.7.7 (14 Dec 2017). As has been the case since their original addition to CalDB, they are packaged separately in the following tar file: ftp://cda.harvard.edu/pub/arcftp/caldb/hrc_bkgrnd_4.7.7.tar.gz TABLE 1: Files, sizes, and md5sum values: FILENAME Ship Size Unpacking space md5sum (bytes) caldb_4.8.4.1_upgrade.tar.gz 251,298,832 1.7 GB 29d72cbd78dbffca237b68145d5d80c1 acis_bkgrnd_4.7.6.tar.gz 1,387,251,364 2.9 GB 192d4871813c9d5d3e857726195d1261 hrc_bkgrnd_4.7.7.tar.gz 2,323,747,030 3.2 GB 3c9b4e5d6966929793d726b12ae4a4b2 C. Unpacking instructions: We prefer that you use gtar with the appropriate options to unzip and untar the files in a single step, to save disk space and to apply the safest unpacking procedures: gtar xvUfz caldb_4.8.4.1_upgrade.tar.gz gtar xvfz acis_bkgrnd_4.7.6.tar.gz gtar xvfz hrc_bkgrnd_4.7.7.tar.gz LINUX users: You may omit the "U" uption in the first gtar command. D. Upgrade verification checks See the CalDB downloads web page for steps to verify your upgrade installation, at http://cxc.harvard.edu/caldb/downloads/index.html#dl_config. - Dale Graessle Chandra CalDB Manager