SDAST Chat Meeting #96: Tuesday October 19th 2004 Participants: Peter Kretschmar; Sami Maisala; Niels Joergen Westergaard; Jerome Chenevez; Carol Anne Oxborrow Subjects discussed: Status of CAO's webpages: Forum, Software exchange, anomalies tracker, chat meetings, Monday meetings. Shared software exchange page to be transferred to NJW Anomalies tracker to be discontinued Chat meeting transcripts and Monday meetings summaries to be continued Peter's Integration webpage status: is it used? Yes, but Peter can't maintain it much longer An ISDC JEM-X webpage for operators and duty scientists, maintained by DSRI is needed, but who will be responsible for it? NJW to present this idea at next Monday meeting JMXi-DETE-STA proposal meets with approval: CAO to write the necessary SCREWs New spe_pick, j_ima_shadowgram and j_ima_basic_recon still to be delivered for OSA 4.2 For OSA5.0: j_ima_mosaic will be added and j_ima_src_find improved; j_cor_position to implement DETE-STA; Is background handling useful? Sami to present some imaging and source finding results in Stockholm. Sami has created some background accessing functions that should be evaluated by NJW and SL. Beta versions available at: http://www.astro.helsinki.fi/~smaisala/SDAST/software/j_get_bkg-1.0.0.tar.gz ADD to be updated in November for OSA 4.2 ISDC rotation: Silvia in Oct.; Niels in November; Stefan in December Action Items: AI041019_1 NJW 25 Oct 2004 Present need for ISDC JEM-X status page at OPEN next Monday Meeting at DSRI and see if anyone is interested in creating and maintaining such a page. AI041019_2 CAO 25 Oct 2004 Write SCREW on j_cor_position to implement OPEN the new JMXi-DETE-STA proposal for OSA 5.0 AI041019_3 SM 18 Nov 2004 Make images and source lists with and without OPEN background subtraction, and draw some conclusions about circumstances in which using BKG helps and when it doesn't AI041019_4 NJW, SL 18 Nov 2004 Test Sami's latest BKG access functions and OPEN give him constructive feedback about how they work **** Logging Started : Tue Oct 19 11:01:08 MEST 2004 > hello guys! any signs of life from the Frozen North? I don't think Silvia will Godmorgen Peter, Carol Anne og Jérôme ! > be joining us - she's still struggling with her thesis presumably. END She definitely is, she sent me some pages yesterday evening. END *** smaisala (~smaisala@gemini.astro.helsinki.fi) has joined channel #jemxadr > We'll wait 5 minutes for Stefan. Hi Sami, it's good that you could join us! END Hi all. Good to 'see' you all. END > Okay, no sign of SL, so let's start. I know the agenda looks quite long, but > several items are quite short. First I'd like to say something in general > about the webpages I maintain: they're very time-consuming, and I doubt whether > anyone outside of DSRI actually reads them. They are: > SDAST Forum > Monday meetings summaries > Shared software exchange > Analysis Anomalies Tracker > I propose to shut down the last two: I've never heard the slightest indication > that anyone ever looks at these. Please take a look at them RIGHT NOW and > tell me if they're of any use to you the URLs are: > http://www.dsri.dk/~oxborrow/sdast/shared_sw_exchange.html > http://www.dsri.dk/~oxborrow/sdast/analysis_anomalies.html I find it useful to have the S/W exchange page. The second seems less used. Is it really that much work to keep that page? Besides spending hours finding colour shades of maximal intimidation power :-) :-) :-) ? END > Jerome and I were just chatting about the shared software functions, and it's I'd also like to keep the shared_sw_exchange at least END > clear that it has always been your initiative Peter, and will die a very rapid > death as soon as you leave ISDC, so I think that one's for the axe, definitely. > END > If NJW and SL think they'll do any work in this area, and they find the page > useful, I'll transfer the files to njw's care. END I'm not so sure: Arvind is definitely encouraging me to keep involved in some of my current activities and I'd much rather chuck some other tasks than this one. My feeling is quite simply that we are lost if we do not pursue ... this approach. END Moving this page over to NJW is an option. But he should be able to move some of his duties (IC files especially) elsewhere ... END Alright , Carol Anne, if you teach me how to file the emails. > That's fine: I'll show NJW how to do it. END Good to hear that you'll still be around Peter. And yes - it would be good > shall we just scrap the anomalies tracker and assume that problems arising to give the IC responsibility to somebody else. END > in the results are the responsibility of a particular programmer, and everything > will just get straightened out if we all do our SPRs and SCREWs? END I think the anomalies tracker was finally much less used, as it also is somewhat redundant with the SPR/SCREW system, so maybe we drop that one. END > Good. consider it done. What about the other pages: does anyone read the > Monday meeting summaries and the chat meeting transcripts EVER? END I've come back to those on occasion - you need these things rarely, but when you need them, they can save you. END I have used the chat transcript on an occasion where I missed the chat END Yes. I would say as Peter. You are glad to know wher to find an info, even if it is only once! END > Okay, we'll keep the transcripts and summaries. END > Any more comments on this subject? END Nope No N NNN > Next point. Peter, what do you want to say about an ISDC JEM-X page? END I have this "Integration Status" page ahich also noone seems to be looking at. On the other hand, our shift team would like to have a page with summary information about JEM-X but also > I take a look occassionally to see if there are any outstanding SPRs/SCREWs > I've missed. END about the software status, current problems etc. I look that page also quite often. END Question: is this "Integration Status" page up to date? END > But I can also do this with the ISDC SPR/SCREW search. I think your page What they want is a page that helps them understand what they see. It has been on my to-do list for a long while to make such a page, but first of all, I have some doubts of ever actually doing it > was very useful while we were in the delivery/transfer phase - but maybe it's > redundant in the maintenance/update phase. END and second, I will probably not be able to maintain it further. So, should we have a small project of setting up a more usueful page (I would still help) and then have that maintained from DSRI? > With a regular influx of small SPRs/SCREWs maintenance of such a page is quite > time-consuming if you realy want to keep it up to date. Again the ISDC I'd rather drop the SPR/SCREW business but have descriptive text about our status. END > SPR/SCREW system is probably better to search in. Perhaps we should all be > more mindful of keeping the Known Issues file maintained by Isabelle constantly > updated, and drop both our anomalies tracker and Peter's page since they both > repeat this basic information. END OK, that's one part. But our shift team (think of new students as duty scientists) could also use a nice summary page of what JEM-X is, how is it CURRENTLY operating (dead anodes, gain evolution, ...) and what we know about scientific results (beware of new sources if you have a bright source close by) ... END All this in plain English, ideally with some images and things to look out for. END > It sounds nice Peter, but I can't be a part of it - Planck is neglected Let present this idea at our next monday meeting to find 'volunteers' - the > enough as it is. END information is around, so the job is to make it accessible in a straightforward manner END * njw Sorry 'Let present' = 'Let me present' END Exactly. There is no real urgency either, but my fear is that if we do not set it up in the next months it will just go down the drain. END > With the losses in personnel in the next few months: JC probably leaving; And it should be kept updated, e.g. with things like "01.06.2005 Reactivation of JMX2 switch off of JMX1". > Peter leaving; Sami and Stefan uncertain about their funding; me trying to do > my 40% for Planck, I think that quite a lot of things are going to go down > the drain. END All the more reasons to still do the larger, one-time job before :-) END > Okay, any more comments? NJW will present this proposal on Monday and we'll > see if there're any takers. END Yeap. OK > Next: the proposal about j_cor_position outputing a detector status map > JMXi-DETE-STA, seems to have met some approval. I still haven't heard an > answer to my basic question:will NJW and SL actually use the information > contained in DETE-STA? END Yes, it is our intention to use this information. I have not yet had time to think more deeply about how to implement it in the flux extraction from images but I still think it is the way to go END And Stefan also plans to do this - we discussed this a bit during the Analysis Workshop. END > I will write the SCREW for this change and alert the data structure people > (Katja?) that we'll be adding an data structure to our science window group > tables. Any more comments? END N N n n One: please mention clearly that this is for OSA 5 and not before. END > Good, now who's got outstanding deliveries for OSA 4.2? Confess! END > right you are Peter: It'd be a miracle if I could find time before the end of I'm working on an improved spe_pick. The scripts should be OK. END > the year! END > Sami? NJW? Are you all up to date with deliveries? END Just fixing one SPR. I will deliver it today. END > No more for OSA 4.2? Any comments? END No > Now what about plans for OSA 5.0? END A new j_ima_mosaic is still planned. I have not really made a big job there yet, but it has always be my plan to consecrate november month on that. END Sorry, I was elsewhere for a moment > So this will be part of the observation pipeline? END OSA4.2: j_ima_shadowgram and j_ima_basic_recon will be delivered soon Yes. at the same place as the current mosaic S/W by Roland. END OSA5: j_ima_src_find will be improved END No big plans from my side, except maybe getting directly involved in j_src_spectra (and _lc), assuming the script work is small or goes to Silvia in the new year (to be discussed in Stockholm) END > Sami? Any plans for background handling? END Not any improvements before you are actually using them and give me hints what should be improved. END Sami, have you done any test of your bkg that could tell you if it has to be improved...? END My understanding of the BKG situation is that nobody has had or taken the > I would like to put a REALLY big AI on Sami in time for the Stockholm meeting: > I think we should see images (dozens of examples at least) with and without time to see how much the use of the background improves on the image quality > background corrections; source lists (again MANY examples) with and without and source finding. Sami has done something and the answer is maybe, I have > background correction (can we pick out weak undetected sources with BKG); done a little and the answer is maybe END > spectra, with and without; lightcurves with and without. In other words: > Sami needs to sell his background handling to us: is it a black box or a > black hole??? This is many years work here and I think it's about time we > found out if it works. An inordinate amount of meeting time has been devoted > to this subject without any results thus far. END I know it is not so easy to use bkg data structures so I have made two shared functions to access bkg models. function jGetBkgmodels returns all data structures from given SWG and jGetBkg returns background pixel in given detector pixel. The background correction option has been included in the imaging step, so there I think part of the problem is that there is a mismatch of expectations sometimes. is a possibility for everyone, including Carol Anne, to test out the effect (even though, as Sami has found out, there are cases where the background correction is overdone ) END How about just to use (or skip) or not use (or skip) the BKG level in the OSA pipeline? > No I think it's for the developer to prove that is software does what it's I agree with CAO that the Sami has to ell us his product. END Sami needs feedback before progressing, while others have waited for progress before changing their code. END > supposed to do. Again, if this doesn't get integrated into the general running ->sell So how should Sami sell it? For the moment he can run imaging cases with and without, is that enough? > of the pipeline before Sami gets moved onto another project, I think it will > be lost in the future: as from the New Year we are possibly loooking at a And for me there is no doubt we will one day hit the point where we must use this option to progress further. Unfortunately we have been kept busy with other issues. It would be a good first start. END > critical manpower shortage. END So the baseline for me is: j_src_*** must include the option in OSA5 so we can progress there, but also Niels' code. END > yes, exactly. Let's have him start by running images with and without, and > showing the results in Stockholm. END I hope the new shared functions make life to easier access the bkg so we could add the bkg correction to the j_src_*** Stefan has it on his todo list END Has anyone heard anything from Stefans side lately? END > Are we all agreed that we would like to see images with and without background > handling, and also source lists from these images, at Stockholm? END Yeah! We met him at the analysis workshop and before that he was on holiday END > When will SL be adding BKG to his components, I thought this was done > years ago! END CAO: You already have seen those images! END No, it was never done - one of the problems of having a >10000 line monster as code. END > Can you do more? Can you do something that tells us if this is an improvement? > Can you tell us if we NEED the BKG step, or if we should just drop it as soon > as manpower gets short? END I can make new set of images and results to the next meeting. END So I think we may need a clear conclusion about the bkg improvement on images. END > That's great Sami!END > Okay, I've put an AI on Sami to tell us how much the background handling > improves the imaging and source finding. I think we're going to have to > decide who's going to take over Sami and Stefan's work if/when they move on > to something else. Ditto Jerome's contributions (mosaicking, GPS/GCDE analysis) > END > Any more comments? END N N n N > Next ADD update: Peter would like an update to do with 4.2. I think we can > manage a quickie update in November, what do the rest of you think? END I have some important changes so I must give my contribution by November END > That sounds good, NJW. Anyone else with important changes to offer? END > Any comments at all? END Shall we not move on? END > Finally, AI list: any completed AIs that are still on the list and should be > closed? END N N N N > Oh well, better luck next time. END > Any other business? END Nothing I can think of. END Y. Who is going to ISDC next? On my plan it says: Silvia for October... Then niles in Nov. and Stefan in december. END niles-> Niels Silvia was here but worked on her thesis. END I see. What do you sy Peter? Is there any need of presence anyway? END sy -> say > Perhaps you should get in touch with Silvia and remind Niels and Stefan. > Any other business? NJW? Sami? END With Peter Leaving ISDC there is so much more reason for us to be there as much as we can. In fact Niels has promised to Tierry to make an effort to not only be at ISDC but also respond to questions with high priority. END Yes. But that will be on another schedule for 2005, which I guess will plenty be discussed I agree. END about in Stockolm END > Yes, I agree with need a strong presence at ISDC once Peter's strong presence > is gone. END The beta version of shared functions are available at http://www.astro.helsinki.fi/~smaisala/SDAST/software/j_get_bkg-1.0.0.tar.gz. If any of you have time, please send me a comments how they work. END > Thanks Sami! I'll put that URL on the forum. If you can send me any with/without > images to put on the forum that would be great! END I'll try ... END > Can you all be thinking of agenda items for Stockholm please? If there's no > more business, I think we should all go for our lunch. Goodbye everyone. END Bon appetit! Good bye! Could you repeat dates and times for Stockholm, thanks END Good idea - Guten Appentit! 18-19 Nov. 2005 Mener du 2005 ???? Sorry: 2004!!! I am hungry :) Bye bye *** njw has left channel #jemxadr : (njw) *** jerome has left channel #jemxadr : (Quit) Goodbye, 'see' you next week. *** Signoff: peter (ircII/tkirc)