SDAST Chat meeting # 94: Tuesday 7th September 2004 Participants: Peter Kretschmar, Sami Maisala, Carol Anne Oxborrow, Jerome Chenevez, Niels Joergen Westergaard Subjects discussed: Instrument status: Switch off of JEM-X2 DFEE: for details see last 2--3 summaries of monday meetings Defective detector area determination: CAO to circulate clear description of proposal since it involves alot of overhead, and PK,NJW and SL to evaluate its usefulness Plans for OSA 5.0: for detailed list see document in the SDAST forum The CORX/CORY not-quite-a-problem: all under control: CAO's already corrected j_cor_position though it's not officially redelivered yet. NJW's updated IMOD, esp. CORX/CORY-MOD and introduced new table JMXi-PRED-MOD of raw pixels on a regular grid for non-skewed shadowgrams Adding areas of high background to either -DETE-MOD (determined offline) or updated dynamically in the newly proposed table by Sami. This proposal also to be discussed by email JC has been talking with Nicholas Produit at ISDC: NP needs a b/w figure of the JEM-X mask, and do the variance maps take into account vignetting? Next chat meeting: Wednesday 15th September 11.00 Action Items: AI040907_1 CAO 19 Sep. 2004 Update -DETE-MOD with new values of status OPEN mask contained in jemx.h version 7.23 AI040907_2 CAO 19 Sep 2004 Write and circulate email detailing proposal OPEN for j_cor_position to output a detector map like DETE-MOD for use in flux estimation AI040907_3 PK, NJW, SL October 2004 Read CAO's proposal and decide OPEN whether it would be used to such an extent as to be worth implementing (big overhead here) AI040907_4 SL, SMN 19 Sep 2004 Send CAO list of OSA 5.0 plans for forum OPEN AI040907_5 NJW 19 Sep 2004 Send NP file of JEM-X mask pattern OPEN AI040907_6 NJW 19 Sep 2004 Determine whether variance maps take into OPEN account vignetting, and let JC and NP know. **** Logging Started : Tue Sep 07 13:51:22 MEST 2004 *** smaisala (~smaisala@gemini.astro.helsinki.fi) has joined channel #jemxadr Hi there! END > Hi Sami! How's the weather in Helsinki, it's perfect hereEND Same here :) END > It's a little disappointing for the children that the weather's so nice after > the summer holidays. They're going to move the start of school back to > September instead of August next year, to take advantage of the good September > weather....or maybe the year after that! END *** peter (~pkretsch@crab.unige.ch) has joined channel #jemxadr Hi there, Jerome should be with us soon, he was discussing with Nicolas. END Hello everyone END > Hi there Peter and NJW! I hope you're both enjoying the good weather. I haven't > heard from Silvia or Stefan, so we'll wait for them a little. I have to confess > that I haven't done any JEM-X work for a couple of weeks, apart from answer > some emails, so I'm a little disorientated. END We're used to that :-) :-) JC is fighting with the technology Carol Anne: is 'disorientated' really good English ? END > Disorientated is good English; disoriented is good American END Alright, just sounded like a translation from Danish END *** jerome (~chenevez@isdcsf2.unige.ch) has joined channel #jemxadr Stefan is probably painting his garage roof today. END Hello Scandinavia! I am sorry to ba late... New system new ways to do things. Hello Jérôme, had a good trip ? END What were you speaking about...? END Y. Tak NJW. END > Hello Jerome! Since Silvia and Stefan still haven't turned up, let's assume > they're not coming and get started. > First on the agenda: We still have Sami starting with an S! END > Instrument status (JEM-X2 DFEE off etc.). NJW can you just give everyone a > quick run down on the JEM-X2 switch off and its (hoped for) effects? END I think they are covered in the minutes of the JEMX Monday Meetings. > Just a quick summary for those who might not read the monday summaries? END We hope to see a braking of the ageing process but cannet verify that yet END Don't we all read those ???? > Okay, for more details see the summaries of the last 2-3 Monday meetings, you > all know where they are presumably. END > Any comments? END We do read them! Thanks Jérôme END > Okay, next point on the agenda: Deffective area determination (what to do?) > I've been thinking of a way I could output a sort of update (for hotspots and > any other dynamically determined problem areas) version of DETE-MOD that > people could use further downstream to determine what area of the detector is > actually used in the analysis, but I don't really see how this can be done > without making such a data structure part of the SCW group. END I also think that since the exact definition depends on analysis parameters it has to be done (with common functions!) in each tool. END > My output of a map of defective detector regions would not depend on analysis > parameters, but every tool must access the map and use the information > individually, that's true. END > It's just struck me: DETE-MOD has to be updated for the new mask values in > jemx.h. I'll put an AI on myself to do this. END What is 'new mask values' ? END We could add such a map if we want. But it's quite some overhead. > What I want to know Peter, is can we add a DETE-MOD-type data structure to Sorry, now I know END > the SCW group. The question is: is it worth the hassle? Will other folk further > down the pipeline actually use this information? Stefan? NJW? END > Sami? Anyone? END In principle yes. That information is required for getting the correct flux derivation. END > So shall I go ahead and write SCREWs on the SCW group, the templates, > j_cor_position to fill the new data structure etc.? What do you think Peter? END My question is: will we actually use it, or rather build our own map based on the event status and the parameters for our tools? END Currently, I think we would include ev ents from a hotspot region that were not marked as "hotspot" Do we want that (then a map is misleading) or rather execlude everrything from there? END > The point iwth the map is that it would contained OR'ed values of the status > mask values, so by looking at the map data you could tell which problem > arises with which pixel, or which combination of problems (bad anodes running > through calibration areas etc.). Then find out which area have one or more of > the mask values you're excluding, and then determine the area that's unaffected > by these problems, as the live fraction of the detector. This way you don't > count pixels with two or more problems as dead two or more times. eND I do like the idea - it would also allow users to see where problem areas are identified by the ISSW I'm just a tad worried about investing effort an then not using it as much. > Me too!!! That's why I'd like some end-of-the-pipeline input. END NJW, will you use such a map? Can we define functions to read it in and manipulate the copy in memory (e.g. radiusLimit cut)? > yes, the radiusLimit problem also pretty much requires the use of a map, so I can see a good use of the map when deriving flux values with vignetting > that you can `see' how much of your problem areas are simply discarded by > using radiusLimit. END If noone can think of a reason not to have a map then let's go ahead for OSA5. END correction. I'm sorry but I didn't follow the earlier email exchange on deffective detector parts very closely, but I think that we need the proposal in writing e.g. from Carol Anne. I have to think it through. END > Okay, another AI on CAO to write a description of the proposal and circulate it. > An AI on NWJ and SL to read and comment on CAO's proposal. END > Any more comments? END Y No more comments END n N > next on the agenda: Individual plans for OSA 5.0 N Who starts ? END > We'll do it in order of height: Peter, Sami, Jerome, njw, cao END > Peter you go first. END Ah, if I always had precedence ... My plans are: + update scripts for reprocessed data (just happening) + better error handling in scripts + Include detector map, if we do this, in j_bin_ tools + Validation of binned background models (currently they are not good) END > You next Sami END My plans: + New bkg models + getBKg function, which might be useful for Stefan + possible bugs END > Jerome? END I should have added: common software functions! END If I am still around until the delivery to OSA 5, I will, as agreed in the last SDAST meeting at DSRI, modify the mosaic_weight program > Any mosaicking for OSA 5.0 Jerome? END to have compatible I/O with ISDC system. I guess the component will have to be renamed something like: > Perhaps we'll come back to JC later: NJW your plans please END + use pixel redistribution to get rid of skew shadowgrams + source finding in fine_resol images + better flux with proper normalization j_ima_mosaic. END Those are the most important pieces END > Sorry, Jerome you're plans came in just as I typed! END > My plans for OSA 5.0: > + Fitting anti-glitches in calibration spectra better (better initial guesses) > + Pixel shift correction in j_cor_position > + Possibly implrement output of a detector problem map from j_cor_position > + Handle mixed CPU mode data in j_dead_time_calc > + Use processing time multipliers for slower CPU modes (instead of tabulating > each mode) in j_dead_time_calc > END > Still have plans for releasing officially my offline trending tools and ICAL > tools........but you know how it is...END > Any comments? END N n n > Next: The CORX/CORY not-quite-a-problem. Could you tell us the status of this > now NJW? END The new IMOD will contain updated (that is: translated) CORX/Y tables as well as PLLX/Y that match the statement: x_cor = CORX(RAWX,RAWY) etc. Another asset is the redistribution table JMXi-PRED-MOD that is a large table how to map to 'raw' pixels on a regular grid. The upcoming JEMX library of shared functions will contain a function to read and interpret this table. What is missing is a test of what effect this change has on the source positions. I hope to get to that rather soon. END > So everything's under control! Any comments? END Let me get this straight: if we work with this, we first filter events, based on RAW positions and possibly Carol Anne's map. Then we redistribute. Well, I expect that you'll help me in testing this END No, Peter, there are two parallel routes: Either you get the corrected position from CORX/Y or you use the redistribution table with RAWX/Y as input to get the regularized pixel (or regular shadowgram). END > Any more comments? END Another thought: could we define regions of usually high background and STATUS flag these too? (And include that in the map)? END > I think j_cor_position would have a hard time determining what's background and > what's not, but Sami could always take the map in the SCW group and update it That is a more clever solution that just a radius limit. I think we could. It is > wtiht the background information. END No, not per ScW, as part of the IMOD. Sami updating it sounds also interesting. END a good idea. To CAO: we define these regions of high background off-line (Sami does (?)) and from that updates the map END > Yes, if it's part of IMOD, DETE-MOD, then anything's possible if it's determined > off line first. END > I was thinking more of dynamically determining problems, like the hotspots. END Niels L. is using this idea in his midisky program END Doing it in the bkg step would be more logical, but then we should always run that one. END > I have to go at 15.30, so don't be surprised if I suddenly disappear - send me > an email about any AI's you've completed (if you accidently happen to have Let's wind down - I have sent you a mail with our current OSA plans. END > done that sort of thing!) END > Let's discuss the bkg additions to a detector faults map by email. > Has anyone by any chance completed any AIs? END I closed AI_9_5 but it seems still open on: http://www.dsri.dk/~oxborrow/sdast/AIList.html END No, but I have completed something that could have been an AI: I have produced two or three functions to become part of our shared library. END > I've just closed that one Peter. Shared functions we'll discuss in the chat > next week, okay? END OK > When shall we chat next week: is Wednesday 11.00 okay? END OK Fine with me END OK ok A question to NJW I am just transmitting from Nicolas Produit: He'd like a (big) W&B map of the JEM-X mask, possibly in a ps or jpg file. I mean I have seen such a picture on your wall NJ. Could you send Nicolas something he could use? Thanks. As we were discussing with Nicolas, it raised the question: do the variance maps in NJW images take account of the vignetting? If yes, why does mosaic_weight then have to reuse this info from the vignetting file. ? END I have to check, Jerome, in order to be quite sure. I'll let you know. END Ok. Thanks. END > Any more comments? ANy more completed AIs? END I thought we adapted this with me hacking your code. END I'd like to tell you that it is quite amazing to see how Peter manage to type as fast as he uses, though he only have one normal hand! END > It's not the size ofthe hands that count! END But perhaps the number END > I'm afraid I have to break up this merry banter. Chat with you all next God bye Good bye > Wednesday! Have a good weekend, and enjouy the good weather! END bye See you in Copenhagen on friday END