SDAST Chat Meeting #88: Wednesday 17th March 2004 Participants: Peter Kretschmar, Stefan Larsson, Silvia Martinez-Nunez, Jerome Chenevez, Niels Joergen Westergaard, Carol Anne Oxborrow Subjects discussed: Mosaiking: JC gave a quick status report. Was agreed by all that the main problem with mosaicking was that we have two separate tools: RW's that is ISDC compatible and JC's that is not. These run on two different input data sets: NJW's OSA results and NL's midisky images respectively. Peter stressed that general users MUST be able to make good images just the way the instrument teams do privately. JEM-X imaging is falling into disrepute. Something has to be done. Rings around strong sources are caused by rings in input imaging not by mosaicking. JC to look into having and ISDC-compatible file input function. A techical note on mosaicking has been circulated by NJW, with an aim to unifying our efforts. AI List: a slow painful trawl through masses of stuff that hasn't been done yet. Somethings moved to `General Works', a couple of AIs closed and a couple cancelled, most only re-dated. Shared Software: all this is now General Work. Everyone has a duty to send their shared software ideas to CAO for inclusion on the Shared Software Exchange Next SDAST meeting will be 3rd and 4th May, 13/4.00-18/19.00 on first day and 9.00- 15/16.00 on second day, at DSRI AOB: - JC has looked at GCDE from rev 171 and found the following sources (all well known):GX3+1, GX5-1, GX 354-0, GX 9+1, 1811-171, 1746-370, and GRS 1724-308, GX1+4. It was agreed that regular processing and posting of the GCDE and GPS data and sources should begin again, but it doesn't seem like anyone's got time for this - CAO reported that all the amplfiers on both units are working fine despite some spurious alerts from the ECAL software. **** Logging Started : Wed Mar 17 10:59:13 MET 2004 > Hello, gentlemen! It's a good turnout we've got already. Let's see if the Buenos dias muchachos, Silvia will join us in a few seconds. > Frozen North is still frozen......END *** silvia (~silvia@uhuru.dsri.dk) has joined channel #jemxadr > How's the weather in Geneva? It's positively spring-like here. Hi Silvia! END Helle Geneva! END hi everyone ! sorry I am late. Sami will not join us , he is on the way to the airport ! END I have some problems with my computer today - so if my messages are interrupted or funnily spelled, that's the reason. END Hum: HELLO! The weather is wonderful in Geneva, real spring time ! END *** larsson (~larsson@tenma.dsri.dk) has joined channel #jemxadr Hi everyone. END to CAO: thanks a lot for your wonderful email on gain calibration END Was Sami at ISDC? END > Hi Stefan! Does anyone know if Sami's joining us today? I haven't heard > from him since Munich. END No, he (Sami) is in Helsinki and he is leaving for holidays !END > Well, if you're all sitting comfortably, we'll begin: > Mosaicking, that's your speciality JC, so how about you start us off? END Well, we are improving our home S/W at the moment following suggestions from NJW but what would you like to know more specifically? END > How did it go with agreeing on I/O data structures? > What improvements are you looking at? > What happened to the rings round the weak sources? Are you able to produce the same, or at least compatbile data structures? END > Generally, everything you know, that we don't. END But how does he know what we don't know :-) END It did well but the work has to be done now and i am waiting to see what actually comes out before I can judge... NJW and I are writting a technical note on that. > I don't know anyting that's happened in the last month. END > Technical notes are good. END Neither do I END When can we expect the TN? END Actually I am looking at input parameters file format, following the PIL one. There can alos be improvements in the geometrical part of the S/W following SB suggestion to use a ready library from NOAO I've just sent the current version of the TN to you END that makes this part in a better way that we have done (I assume). But so far, the improvement is not dramatic. As regards the rings, I guess you mean around the STRONG sources. Thank you for theme, they are doing well... Ok, as far as we understand, the problems may come from the input images and not from the mosaicking processing. END Regarding input parameters - PIL is really easy to use and can be included in a program completely independently from the rest of the ISDC system. I've done that and could advise you how. END > So there's a good hope of removing them? END I do not know. Ask the processors of the images... END In summary, the work to do now for me is to made the changes necessary to take account of the I/O common format (mainly) we agreed about. Another question - what is happening here that could not be done in Roland's tool? Or should we replace the generic image_mosaic with a CTS compliant copy of this one in the future? I can mention here that the mosaic_weight program (in ist current version) is available on DSRI pub. FTP END What I definitely do not want - because it would increase the load on me for user support - is a proliferation of tools not following the established standards. END So in brief I see two options for the mid term: 1. Make mosaic_weight fully ISDC compliant and deliver it (as j_ima_mosaic ?) 2. Have Roland include the added knowledge in his tool. Whcih option are you aiming at? END > Thanks for the technical report NJW! ENd I guess the 2. one, unless the alternative can expresse another way: that is, Roland's S/W is going to be the official one at ISDC. We (at DSRI) are improving our home tool for internal purposes because there is always something new coming up and that can may be not be implemented at ISDC at the same rythm as new idea (good as well as bad) are fusing up here. So, when we think we have done something that may be intersesting for the rest of the community, then we will transmit the idea to roland, so he can implement it in his S/W. Point taken but then you should make a strong effort to have as much as possible identical input data for your tool and image_mosaic, as otherwise the transfer of algorithms is delay END Wouldn't it besier at some point to take imagmosaic as starting point and have an extended copy? END besier = be easier - this machine is swallowing input sometimes today. END The problem is that Roland's Soft and mine have always been quite different and >I guess they continue to diverge... END Is'nt that precisly why peter asked? END Well, but in my view working on a completely diffrently structured code does not help at all. END > The problem as I understand it, is that JC's and RW's programs work on two > completely differenct data sets. RW's uses NJW's images from OSA, while > JC's uses NL's images from midi-sky, or whatever. The latter images change > somewhat rapidly as per NL's whim, and this is why JC has to keep updating > his program. It's unlikely that JC's program will mosaic NJW's images, because > this is basically a private tool for the PI and his own images. END Thank you CAO to write that for me. END BUT THAT'S THE PROBLEM > We all know that's the problem Peter. It's the problem for all of ISDC . > Why do you think so few of the PIs turn up at ISDC Consortium Meetings It should, no it must mosaic also NJW's images since these are the only ones normal users get. > after the launch? They've got their data, they've got their own programs. END People are turning away from JEM-X analysis and having private toopls with improved capabilities is not helping. Have you ever noticed how many people speak well of RXTE. That is because after a certain time they could do good science themselves with generally available and supprted tools. Similar for BeppoSAX which took longer, also because at the beginning it was harder. OK - I'm stepping down from my soapbox, let's talk about practical matters: With good software design there is no real reason why you should not be able to read all possible JEM-X image formats. That's exactly what the proposed changes are aimed at (in th TN) END Yes, is is exactly what I am working on at the moment! > This is true Peter, except that NL has a tendancy to change things somewhat Roland's software can handle Niels' curretn format - actually better than Niels Jørgen's(!) - so there is no fundamental reason making this impossible. sorry NJW... END > faster than most people can keep up. ENd I know that, but if you spend your effort outside the system seen by users tehy will hardly notice. People do not care too much about the results you get, they want to see what results _they_ can get. END Since image_mosaic is probably not the easiest tool to digest either it may be the most practical solution to keep a separate development, but PLEASE, PLEASE try to stay within the smae framework AT FUNCTIONAL LEVEL as ... possible. I'm not sure you realize how bad the view of JEM-X imaging is becoming ... END If we do not have a workable _user_ solution, distributed via the normal channels, even more people will just give up and that would be a real shame! END I think we are going too far into politics now... Could not we concentrate on mosaicking? I mean, you are talking about imaging, not mosaicking here. END > Actually, the root cause of the mosaicking divergence, is actually the This is not politics, this is planning how to spend our efforts within the short time remaining before the next software release. > imaging divergence. Perhaps we should discuss what to do to improve the > ISDC JEM-X imaging, and hope that mosaicking can follow along with improvement > there. Just as a quick aside: why DO we have two sets of imaging routines. > I know NL's is slower, but that shouldn't be a worry for the ISSW, since the > imaging is allrun off line as OSA. END First: iamging is also run in QLA & SA. Second: I'm very much in favor of having also some halfway standards-compliant tool by Niels and am working as much as I can in that direction. END Having more than one option can actually be nice for the user - for comparison. END > Why don't we have a good refined_recon, now that so much algorithm related But back to mosaicking: when would you expect to have those changes done Jérôme? END Ok, the I guess we can say that when niels soft become ISDC (half) compliant, so will mosaic_weight also be. > work has been done here? NJW can you tell us what the hold up is here? END I guess in one month time or so. END And whatever happens we must have improved JEM-X mosaics for OSA4 which is SOON. END What is that? sorry I meant WHEN? END If I remember correctly we wanted s/w delivered for end of April to begin testing. END Ok, that should be find for me. END But remember. My pgm can only be as ISDC compliant as the input images are. So it will depend on Niels images format. END And I would be very glad to see a brief comparison of the internals (not each line, the structure and concepts) of image_mosaic vs. mosaic_weight. ENB Me too! :-) END And your pprogram could well be 100% compliant except for one function reading Niels' images. END Acknowledged. END All I'm asking for is that for any design decision you take, you see if you can represent this within the common ISDC framework and then work in that direction. END > Okay, let's wrap this up shall we? How about an AI on Jerome, to make an OK, shall we move to other topics - I might be cut off at 12:00 (reboor of sunrays) END > ISDC-image read in function? What are you still missing JC? END Nothing for the moment, thanks! END > Generally, perhaps we should continue this discussion between NL, NJW > and JC here at DSRI. END That is what we are doing. END > So let's move on: AI list. What have we all done? I've set up the requisite > webpages: I hope you've all given them a peruse. I need input from YOU, that > means ALL OF YOU, to fill out these pages with anomalies in your results > and such like. > So let's go down the list of AIs. Is this first one dead or what NJW? > AI26.9 NJW 1 Feb 2004 Look into making shadowgrams not based on OPEN weighted events, to handle grey filter I have looked at it and I know how, but I haven't done it yet END > Okay, new due date? Make it realistic please. END April 20 END > Thanks. The next one also needs a new due date. It'll be done when I > next deliver j_dead_time_calc: > AI26.11 CAO 1 Feb 2004 Check that deadtime logic works correctly OPEN for grey filtering and buffer flushing > Due date till be 20 April too. Any comments? END N AI26.12 is done END AI030828_1: This question is analyzed here e.g. with Carl. I'm not sure > Good stuff NJW! So let's move onto > AI030828_1 NJW 15 Feb 2003 Look into sources moving parallel to OPEN COSY / COSZ axes giving problems with vignetting, esp. GRS 1915 when there'll be a result END > Let's call it `ongoing' and set due date at end of March. Okay? END OK > next, for JC, how's this coming along. I think it's got rather lost: > AI27.5 JC 30 Jan 2004 Send NJW Xe-energy resolution results for SVR OPEN Yes, as I said last monday at our DSRI metting, I cannot find a period long enough to make this job. I try but i do not think it will have an end... END So should this be moved en-block to another date? END But i connot promise anything to any date. END > If it can't be done, with the calibration data that's just become available, > perhaps we should cancel this one, and try again with the next Crab > calibration. Any comments? END It will not be long enough anyway. that is the source of the problem. END Isn't there other data that can be used ?? END > So we just cancel? END > When we next get a very long empty field observation, we will ressurect > this one. Sorry, ressurect. END To NJW: Not after the last summer. END To CAO: OK END > Next one is for the two DSRI guys: > AI27.7 NJW,JC 1 Apr 2004 Make spectral response matrix for REST events OPEN and non-imaging data modes Not done but there is still hope END What can I do? END > I think this should just be for NJW since he's our spectral response guru. END Let's discuss that later END > What about the due date? Will this get done? END > Okay, next AI for Peter and Silvia: > AI27.8 PK,SMN 1 Mar 2004 Adapt scripts to handle the existence of OPEN multi-mode data Make it April 20 as well END That one is in the works, make it March 25. END > Peter? Silvia? Are you with us? END Yes ... > What about this AI? END Peter already said, make it March 25 END > Okay, next one, Peter and NJW: > AI27.9 PK,NJW 1 Mar 2004 Make spectal response data structure templates OPEN with data mode keyword tag I've sent an email to NJW and was waiting for a response :-) [could not resist] END > That's `ongoing', what about a new due date? END April 20 END > I think the gist of this one has been done, not by NL, but at least by the There is some magic in April 20, right :-) END > technical note just sent by NJW: > AI27.10 NL 6 Feb 2004 Update mosaicking discussion to a formal OPEN document that can be put on the DSRI document page > Yes, we'd noticed this special date too. END > Any comments, about me closing NL's AI? END I think you can. END N I mean no comments, you can close it END > Okay, I hope Stefan's close at hand.....here's a couple for you: > AI27.16 SL 1 Mar 2004 Verify that XSPEC c-statistic approach works OPEN AI27.18 SL 15 Jan 2004 Run rev. 53 data on GRS1915 (PIDs 65, 66) ONGOING and compare results with NL's results Set a new date. 15 April END > For both? END I only see one... > There are actually three Stefan: AI27.16, AI27.18, AI27.22, now they're all > due 15th April. END > Next one for PK and NJW: > AI040121_1 PK,NJW 27 Feb 2004 Make first draft of jemx.shared1.c library OPEN with headers of suggested shared functions and circulate to the rest of SDAST Good point - we started somewhere but did not really continue. Nies Jørgen - could you give me some feedback on my original propositions? If this happened soon, I could turn in some functions also by Mar 25. END > Okay, new due date, and remember, now that we have a Shared Functions Exchange > you are duty bound to send me a copy of all these headers so that I can > put them on the net. END > Next is related and concerns us all: > AI040121_2 ALL 4 Feb 2004 Provide PK and NL with suggestions for OPEN shared functions that could benefit our own software, and presumably, everyone elses Yes, Peter (sorry, a disturbance) I'll do that END > Is this closed, or are we still expecting to produce shared function > suggestions? Remember, all suggestions should be sent to me too for public > documentation purposes. I've seen precious few suggestions myself. END Not much has happened indeed. JC, you could for example propose a read image function ... END Basically it is still open, but perhaps we should remove it from the AI list and call for suggestions now and then ? END > Okay, this AI is ongoing and now falls under general work. END OK OK > Next PK and SL: > AI_MMM_1 PK, SL 19 Mar 2004 Agree on a spectrum output function and OPEN develop prototype > Can I mark this `Ongoing' or has it been forgotten? END both :-) END Ongoing - let's try to settle it this week still, Stefan and have the due date Friday - OK? > Is a more realistic due date needed? END Lets try Friday then. END > So due date remains the same. Shall we take bets on actual completion by > Friday? > NJW here's a Friday AI for you: > AI_MMM_2 NJW 19 Mar 2004 Code prototype for a common image output OPEN function > Will you need more time? END Give me a week more, it exists but needs to become more general END > So that's `ongoing' with new due date. END Y > Wake up all you astronomers out there, this one's specially for you!!! > AI_MMM_3 SL,PK,JC,NJW 19 Mar 2004 Decide on a list of required common modelling OPEN functions, possibly implementing different algorithms > Again, ANY kind of brain activity related to shared software functions must > always be reported to me for our S/W Exchange. END Yes, that one has not seen much progress despite my attempts on occasion. Any ideas out there? Can we have something before next week Friday? END We don't need a complete set but some ideas would be very helpful END OK, lets see if we can do something by next Friday. END > Okay, new due date for that one. I think this following one will become > general work, now that I've initiated the webpage: > AI_MMM_6 ALL 19 Mar 2004 Send CAO email detailing what they want and can OPEN contribute by way of shared functions, including their own private tools > Does everyone agree? END OK OK ok > It does however, mean that I'll bug you at regular intervals with emails > begging for common function ideas and prototypes. END > SL and I both have similar AIs. Mine will be done today, what about you SL? > AI040303_1 CAO 12 Mar 2004 Write update to Know Issues file at ISDC OPEN detailing conditions where gain smoothing fails AI040303_2 SL 12 Mar 2004 Write update to Known Issues files at ISDC OPEN detailing the spectral jump problem for off axis sources. > Where exactly is the Known Issues file Peter, and how do we get permission to > update it? END You don't update it directly - you send your input to Isabelle. I should have done it but (teaching) it will not be untill Friday. END Sorry to leave you now, other obligations, AI040303_4 will be done in time END The Known Issues are accesible from the _public_ downlowad page: http://isdc.unige.ch/index.cgi?Soft+download bye *** njw has left channel #jemxadr : (njw) > Thanks, consider it done. END > The last two AI's aren't due till the end of the month, so let's leave them. > Any comments in general about AIs? END OK. N n N N > Okay, next point on the agenda, very quickly: > 3) Next SDAST meeting, May 3rd and 4th > Are you all aware that the dates have been changed because I'd only have > limited participation the week before? END Y Y Y Y. But I'd like to know at what time we start. END > We start after lunch on the first day (Monday) and work late (18.00 or possibly > later), then start at 9.00 on the second day, and finish about 15.00 or 16.00 > to let people get to their planes. Is that alright with everyone? END Fine Good. END ok > Okay, just checking that you all know. > Next agenda point is about shared software, and all I wanted to say is that > the shared software exchange is rather empty at the moment - rather like a > Soviet-era shop, and I need you to send me ideas and offers of great things > you can share with others. Any comments? END We'll try ... END I'll send what I have next week - in whatever state it is then. END > Thanks Peter: I have a feeling you have a lot of material for this page! END > Has anyone got any AOB? END N N I have 1, short, which does not need long discussion. Has anyone looked at the recent GCDE scans? No No - too busy with all sorts of stuff. But I could analyze that quite quickly, I presume. END no Last year we used to send each other analysis results... I have just gone through the whole rev. 171 which covered the very central parts of the GC and Yes and we should restart that kind of business. END I only found a few sources with OSA3,though I used a quite low dectsig level (15). The known sources found are: GX3+1, GX5-1, GX 354-0, GX 9+1, 1811-171, 1746-370, and GRS 1724-308. GX1+4 END > Good work Jerome: it's nice to see that someone's keeping up with the data! END Thanks Jerome for the information. I am sorry but I do not have time for GCDE analysis now. END I am still loking at the eventual new sources... END Thanks! In theory it would have been Silvia's turn as Inst. Rep. here - but she must work on her thesis. END I am the only one to be hungry...? END > My only piece of AOB is to report that the occassional alerts coming from No > ECAL are totatlly spurious, due presumabely to that strange bug that only > turns up every few months. Nothing to worry about - the amplifiers on both > units are still working perfectly and have not significantly changed their > behaviour in over a year. END > If that's all the AOB, let's head off for lunch! END Bon appetit! END Guten Appetit! END buen provecho ! ENd Smaklig maaltid > Velbekomme! END *** Signoff: larsson () 'See' you soon. END *** Signoff: peter (ircII/tkirc) *** silvia has left channel #jemxadr : (silvia)