Subject: New light curve software

Dear Silvia (and others):

This is an update of the status of the 'fits2lc' light curve 
program. The basis of fits2lc is the source illumination function
developed for midisky. The parameters of this source illumination
function still needs tuning (volunteers are invited!) but for my
own part i will now concentrate on extending fits2lc to handle 
'restricted imaging'. At the moment the program only works on 'full 
imaging' data. fits2lc requires the same environment as midisky, 
with a reference directory with auxiliary files (source catalogs, 
instrument characteristics, pointing data, etc).

I attach light curves from orbit 53 for NGC 6441 and Sgr A*. The
first source is relatively bright and frequently detected in
midisky, the second source has never been seen by JEM-X. Each run
of fits2lc will generate lightcurves in 4 bands, nominally: 
3-5 keV, 5-10 keV, 10-15 keV and 15-35 keV.

niels

*****

fits2lc" will produce a lightcurve for a single X-ray source from 
a single jmx2_raw.fits file (a single science window).

It use a particular naming convention for the fits files (rather
than their location in a directory structure) to obtain information
about the orbit, pointing ID, and sub_pointing ID. The naming
convention is that a file from JEM-X unit J, orbit XXX, science 
window (pointing ID) YYY and sub pointing ID ZZZ is named:
                        rJ_XXX_YYY_ZZZ.fits
The name is parsed from the .fits-end, so only the value of J, XXX,
YYY and ZZZ matters. The number of characters in each field should 
not matter, but I use leading zeroes to make nice looking lists of 
the files. 

The source coordinates and the bin size of the lightcurve (in seconds)
are specified in a control file like this:

The control file:         GRS_1915+105.dat
contains the text:        288.7983  +10.9456  100.0
(the numbers must be separated by blanks)

and this will tell fits2lc to generate a light curve with 100 s
binning for a source with the (J2000) coordinates (288.7983,+10.9456).
The name of the control does not matter, but it must have the 
extension '.dat'.

fits2lc is invoked like this:

./fits2lc   r2_057_035_0.fits   GRS_1915+105
(note: no '.dat' extension given on the name of the control file!)

It will then look for the GRS_1915+105.dat file and read the control
data. It will also look for a file called 'GRS_1915+105.chk'. If
this file exist the light curve data generated by fits2list will be
appended to the light curve files generated by previous invocations
of fits2lc. If the '.chk' file does not exist, new light curve
files will be initiated (any old ones will be erased) and a '.chk'
file will be generated. Thus, the batch file:

#! /bin/tcsh -f
rm -f *.chk
./fits2list r2_057_007_0.fits GRS_1915+105
./fits2list r2_057_008_0.fits GRS_1915+105
./fits2list r2_057_009_0.fits GRS_1915+105
./fits2list r2_057_010_0.fits GRS_1915+105
./fits2list r2_057_011_0.fits GRS_1915+105
./fits2list r2_057_012_0.fits GRS_1915+105

will first erase any existing '.chk' files and then generate a 
combined light curve from orbit 57 scw 7 to 12.

If the source in question is outside the JEM-X field of view for one 
or more of the specified data sets the program will just skip over the
data sets. (The program needs access to the pointing.dat file - the
location of this and other support files are specified in the usual 
way by the "sweethome" textstring embedded in the source code).

At present the output comes in a number of 'lc-.gnu files and is
tailored for being used as input for gnuplot. The output files
must be renamed or saved to another directory before a new run of
fits2lc is initiated.

I have made a special 'search_lc' version of the original 'search' 
program which listed science windows relevant for the study of a 
particular source. The new 'search_lc' will automatically generate 
all the batch files you need for running fits2lc and plot the results
for a given source. It is really nice and quick to work with.

*****

You will find the source code for fits2lc and search_lc in:

/r6/jemx/nlwork/c_fits2lc 
and
/r6/jemx/nlwork/c_search_lc

Note that all the programs now uses two "sweethome" text strings,
sweethome points to the directory cintaining the 'reference' directory
and sweethome2 points to the directory where 'pointing.dat' is placed.
It could be the same directory, but now, here at DSRI, it is not.  

search_lc will probably need some editing to adapt the batch files
to your local environment, but the footwork is done in the existing 
source code, so the adaptation should be easy.

The source code for search_lc is in a single file (search_lc.c),
but the source code for fits2lc is divided into 6 files:
fits2lc.c detmap.c getRAdec.c m_shapes.c m_strukdef.c and fits2lc.h

****

The reference directory can be found in:

/r6/jemx/nlwork/reference (unix)
and
/nfs/lin6/users/nl/nlwork/reference (linux)

(Silvia: you probably have everything you need, except possibly
the updated JEM-X source catalog: cat_combine_02.txt)

********************************

Qualities and weaknesses of fits2lc:

Qualities:

1) It works directly from the source coordinates - if you know where
   your source ought to be there is no need to run midisky first to 
   find the source and generate illumination patterns. And it allows
   you to make light curves for sources which are too weak for midisky 
   to localize.

2) It is reasonably fast, less than 30 seconds per science window.

3) It compensates for the limited accuracy of my source illumination 
   models by calculating always for 5 source positions simultaneously, 
   one position being the specified source position and the four other
   positions are displaced from the main target by 12 arcminutes 
   North, East, South and West. My assumption is that these positions
   are empty of source flux, and any flux assigned to these positions
   represent an error signal also present for the main target. I
   therefore subtract the mean signal from the four test positions
   from the signal found for the main target. This may be a bit
   ad-hoc, but it actually reduces the scatter due to the dithering
   significantly. It works, however, only for weak sources, where the
   determination of the background level is the main source of error.
   For strong sources the dominant error is the uncertainty in the
   determination of the strong signal itself, and there the better
   reliability of the background level is of less importance.

Weaknesses:

1) The source illumination model still does not give a satisfactory
   description of the vignetting effects. For strong sources like 
   GRS 1915+105 this is stilthe dominant error.

2) The program can only handle full imaging data. Restricted imaging 
   data are simply discarded. This is my top priority problem to solve.

3) The gnu-output is of limited use for pulsar studies. There we need
   to develop some 'event-flag' method, I think Peter has such a system
   at ISDC. 

4) Not fully compatible with ISDC - therefore no corrections for 
   variations in the detector gain with time or temperature.

**************************************************************