SDAST CHAT: Review of ADD 8.0 (draft version) Wednesday 19th November 2003 Participants: Peter Kretschmar, Niels Jørgen Westergaard, Silvia Martinez-Nunez, Jerome Chenevez, Carol Anne Oxborrow, Stefan Larsson, Sami Maisala Action Items: None **** Logging Started : Wed Nov 19 12:58:12 MET 2003 I've noticed that many of your components have version 3.8.0 and delivery > Hello Peter and NJW: perhaps I should give prizes for those who turn > up first to the meeting? END date 15/10/2003. Do you have a big chunk you ship off simultaneously ? END *** silvia (~silvia@uhuru.dsri.dk) has joined channel #jemxadr To Carol Anne: Indeed you should, at least today ! END To NJW: Yes indeed - some smaller issues often get fixed in bunches with 1-2 days of intense work all culminating in parallel. To Carol Anne: I definitely agree - very good idea! END > I wish I could say the same Peter (solving problems in bunches); all my > components are so different that each one has its own special pathology. > Hi Silvia! Hope you've getting everything ready for your visit next week - *** Sami (~smaisala@gemini.astro.helsinki.fi) has joined channel #jemxadr > we're very demanding guests you know. END > Hi Sami! Jerome should be hear soon, he only stepped out to the kitchen. END Hi all ! Everything is ready for your visit except me - I caught the fluEND > hear==here END Hello everybody.END > Do they have Spanish flu in Spain, or is it called English `flu there? END To Silvia: sorry to hear that - I hope you'll go home and treat yourself right after the chat. END *** larsson (~larsson@tenma.dsri.dk) has joined channel #jemxadr > Hi Stefan! We're all here except Jerome, who's just coming. Silvia has Hello Silvia, Sami and Stefan END > permission to rest a bit as we chat. END *** jerome (~jerome@uhuru.dsri.dk) has joined channel #jemxadr Is a quite silly spanish flu, only attacks to the stomach , but in a very unpleasant way ! END > We're all here so the party can begin. Same kind a Paul has (had?) ? END I am afraid yes, NJW END > Are we agreed with Peter's procedures and my suggestion to go chapter > by chapter? END Yes. Hello everyone! END Y Hi Jérôme END > Hello Jerome! END Shall we begin? END > If you're all sitting comfortably, then I'll begin. > First I'd like to say that I hope you've all done all the work mentioned > in the blizzard of emails I sent you in the last couple of weeks - don't > have time to go through every component, and I don't always know what > inputs/output/parameters/functions they use. Sure ... we would never dare not to do our homework :-) END > That's not quite true Peter, I'm still clearing up rampant AUXL-ORBI-NRTs > in the document. > Second point: please give me your corrected paper copies of the document > in spain so that I can merge the minor corrections when I get home. > NJW and JC are excused hauling the ADD to spain first. > So chapter one: any big points? END N No No No A rephrase of a sentence and some additions to the acronym list. I'll email. END > Peter, this morning you mentioned that V1 is now known as Technical > processing. Have V2 and V3 been given names too, and if so what? END > No need to email, just give me your corrected copy NJW. END From what I see here, V2+V3 is called Scientific Processing for anything connected to SA/QLA/OSA. I think in many cases we could just drop the V2/V3 association without harm. > Shall we continue with the terms V1, V2, V3 or are these obsolete? END > Okay, will do. END > So chapter 2: SRD. I've only one comment. SR 18.8.3 is the only one which > isn't definitely discarded or definitely satisfied. It reads: > J-IMAGE-GENERATION, Consistency with other instruments: If possible, > the image data structure should be consistent with those used by IBIS and > SPI. > How do we stand here? Is consitent the same as identical, or just similar? > END Similar is enough. And that is definitely fulfilled. END > As for 18.20.3 (mosaicing), will we have a JEM-X specific mosaicing tool > or is it assumed that we'll always use the ISDC generic tool? END My preference is the latter - simply to save a lot of effort. As you have seen, Roland has made a lot of progress. This would probably work with the normal ISSW images, if we had variance images in there as well. END > So for the time being we assume that our OBS tools (basically mosaicing) > are taken care of by ISDC? END *** Signoff: larsson () *** larsson (~larsson@tenma.dsri.dk) has joined channel #jemxadr > Any more comments on SRD chapter? END (Just switched computer for a more relaxed working position) END N Should we add j_bkg_scal_calc also in 18.7.x ? END sorry 18.17.x END Probably yes - you figure out where it matches best. END > Yes Sami, I think you're right, I've listed it for 18.7.1, 18.7.2 and > 18.7.3 in the component description. END > Any more comments on requirements? END N N n N N > On to chapter 3: comments please END I have two changes in 3.1 that need general agreement (I think): p. 15, 4th bullet: -> To make shadowgrams with corrected event position and energy. and energy. Same page: Remove last bullet (not a job for science analysis software). Section 3.6 "The background problem" 2nd paragraph, needs a rewrite END PS sorry for repeating END I agree with the changes. END They'll be handed over to Carol Anne, then. END > For that last bullet, how about "Monitor corrections to see whether > tables need updating" I'm thinking of j_ical here. END Yes, that's fine END OK What about dead anodes monitoring? END > Sami, can you do the rewrite on the background problem, I think that's > your contribution. END Isn't that included in "perform a running check on the instr.'s performance" ? OK, I see. END Yes. I can do that. Please send me a comments of the messy parts. END I will END Figure 3.3 (p. 20) has still old bkg data structures. END > Peter, figures 3.1, 3.2 need to show that JMXi-GAIN-CAL-IDX goes into > j_correction and that HK goes into j_dead_time. Have you still got the > pages I faxed to you a couple of weeks ago? Also figure 3.3 doesn't > seem to be updated at all. END Changes to figure 3.1/3.2 - OK. For figure 3.3 I do not have the .fig file - as I told you in a mail. Here the question is also if this one shouldn't be broken up into different figures: (1) A figure showing the full functionality of OSA (Fig 3.1+additions) and one showing all the other interactive parts. Hmm, there should have been a (2) somewhere ... For the latter, I would need some clear descriprion of your tools - can I assume that the current OSA text is OK? END' > Yes I was going to suggest a figure that shows the pipelines and the > observational stuff. Also, there was a suggestion some time ago to > show how the ISDC levels branch after a certain point. If you could make > a new diagram to replace 3.3 that contained all these elements it would > be good. END > Or even two figures END > Next: I think you need to write somewhat more fully on background > handling in the imaging, binning and source extraction stages, Peter, NJW and Stefan > We really need to know how you employ sami's output and whether there > are occassions when you don't use it at all. END I completely agree with Carol Anne !END Accepted. END Alright, I'll write something for j_ima_shadowgram END I'm not sure, I thought I said everything I had to say - can you explain what is missing in my case? END > If there aren't any more details that's fine - things haven't become > more complicated in the last year? END > My next question is whether we should include jemx.h as an appendix to No - as I 'just' bin Sami's data. END > the ADD or not. `Just' > binning is fine if that's all you do. END W.r.t jemx.h - it would make sense as an appendix. To PK: sorry Peter ! you are right, your section about bkg handling is clear END > Okay, next point: I'm updating figures 3.5, 3.6 and 3.7 with all PK's > suggestions and any others I receive between now and next thursday. > Also j_data_preparation is being purged. I've changed your j-data-preparation > description Peter, to make it a `package' and include the Rev. file pipeline > components too. j_sci-format is being purged throughout too, and in this > regard I think Stefan needs to write a piece for the detailed design > chapter describing the different data structures he produces (PHA1 PHA2) > and explain how these are needed for generic tools. END > Any comments? END > Finally, have we changed our ideas about dream tools? Have we become > more realistic, or are we dreaming of even more wondrous tools in > the future? END Well, the current description is very generic. I'd drop the last bullet which doesn't describe anything. Should we mention a true multi-pointing analysis here? What about a j_ima_src_find++ working on mosaics? END > On the contrary it describes far too much! I'll drop it - we don't want > to invite PIs to give us lots of work. Yes j_ima_src_find for mosaics > sounds good, but won't this be covered by some generic ISDC tool? END What about the ISDC special tools like pick_spe, pick_lc...? END I'm also more inclined to put here only a very limited list of components To CA: There is no generic ISDC tool to search for sources as the knowledge of what peak could be releveant seems rather specific. Maybe we'll have one someday but not yet. END * njw more or less ready. Better to document facts than dreams. END To Jerome: spe_/lc_pick and there's nothing special about them - they're described in our general documentation. Do you want to include them in the ADD? Could be done for the OSA part. END On source finding: It's true - the PSF depends a lot on the input images END > No I'm not looking to include more of the ISDC components now - only > those that are deeply inbedded in our pipeline. I'll add the source > finding tool as a dream item. END I mean, maybe they could be mentioned there if they fulfill at least a part of this multi-pointing analysis we are talking about. END I am sorry but my stomach is "shouting" me. I am going home. I hope all of you will have a nice trip to Spain. See you on Monday. END > If there are no more comments on chapter 3, let's go on with chapter 4 *** silvia has left channel #jemxadr : (silvia) > Detail Design. > Take care Silvia, see you on Monday. END > First I'd like Stefan to write a bit on PHA1/PHA2 output data formats and > extend his `Usage with XSPEC' to `Usage with XSPEC and XRONOS' - this > is really to cover the bald patch where j_sci_format once was END > All comments about chapter 4 welcome. END > I'd also like to include two figures showing the current ADC/PHA and > PI/KeV mappings if no one minds. It's something I've often found useful END I can add something more there CA although some of the j_sci_format is now in the isdc tools. I will try to explain a bit more regarding the use of XSPEC and XRONOS with our data products. END A good idea. END > That would be great, thank you, Stefan END Figure 4.1 is now obsolete, I think. Arrays of that type assume raw event positions. The 7.05/7 mm pixel size has been given up. END The section 4.7 needs a rewrite when the new convention has been implemented. END > I'll scrap the figure, but can you then pad out your section on > detector positions? END OK END > Don't you mean section 4.5 is obsolete, that's the detector positions Stop! > one. If section 4.7 is obsolete JC needs to know and needs to do a > rewrite. END I am not sure I agree about that figure. I do use that in my software so maybe it is good to have it Sorry Stefan, perhaps we need to discuss this in Valencia END described in the ADD even if there is no direct refernce. > But if the 7 pixels to every 7.05mm of collimator grid is obsolete, so > is your software. END I was ready to ask: Should the part 4.7 Flux Units not be modified regarding the recent discussion we have had about normalisation? Now, what is the status? END > I think it should. END OK that part I dont use. so in that sence I agree. To Jérôme: yes END Yes indeed > I'll remove the figure. I also have responsiblity for updating the > detector posistions section, 4.5 (I'll talk to you aobut this NJW) and > JC updates 4.7. NJW do you need to update the piece on spectral data > handling? END What piece ? END * njw You mean section 4.6 ? END I will go through 4.6 to make the update with regards to > Yes, 4.6, that's yours for some dark historical reason. END xspec and xronos use etc. END Thanks, Stefan END Section 4.6.2 seems to be in order END > Section 4.6 is in NJW's directory at add/njw/SpectralData.tex END > That's all I've got on chapter 4, now on to component descriptions, > any one want to get the ball rolling with general remarks? END Stefan, I'll mail a copy to you END (Fine NJ) END > Alright, I'll start: Should we agree on a common way to give dates ? END We could. > Many many components lacking real resource estimates: code size, CPU time, memory required for the data segment etc.etc. We were asked to do this almost 2 years ago. Is it still important ? END > Resources: None or Resources: No unusual is not very helpful. END > It was important then, why isn't it important now? END Machines grow bigger END That is the question. END Also, the software is running. The idea was probably to see if any problems were expected.? END And Peter says? END Yes, indeed. If we had, e.g., decided that we do background handling with the 'gigacube' approach, that would have been important information. " Sorry, just messed up my input. > But we could apply Stefan's logic to the entire ADD: if ISDC's got the Should we replace "No unusual" by another wording indicating only insignificant resource usage? > software running now, why the documentation, why not just look at the > code, the results, the unit tests etc. END The ADD is supposed to help understanding the code. I don't consider that piece of information important to understand the code END It should be a useful document for any newcomer. > Is the consensus to scrap the resources field, or simply fill it > with something generic instead of giving real numbers? ENd The latter END I'd leave the existing real data, where we have it. Otherwise the latter. Then there is an option to flag some unusually big memory demands END The latter unless there are significant requirements. END So we all agree - where does "unusually big" begin? 256 MB, 1 GB? END With our current ISSW I think we have nothing "unusually big" END > Okay, I think `No unusual requirements' will fit most cases. Presumably > it's the big instruments that have big demands. END Should we insist on a person name when just an institution is given as responsible for a component ? END > Yes I think a person should be responsible for each, and Mr Newcomer should > know who to contact with questions. ENd I agree END ok ok > Everyone should put their names on their comonents. ENd > My next general point is that there are several components with no > design remarks. After several years of evolution for each component this > strikes me as a little strange. Please add remarks to all your components: > algorithms, structure, input anomalies, output quirks etc. etc. Why > you did it the way you did. Don't forget to mention all the times ISDC > forced and inelegant solution on us! (Just kidding, sort of, Peter) END I have not even ignored that :-) END > Please remember this is also the DDD so we need D (details) END Anything more here? > Any comments? ENd > Shall we move on to the Data Dictionary? ENd The main point is that really some details should be added instead of remaining unaccessible in the more or less porous grey matter between the ears ,... END Yes, let's move on. END Good idea! > I have no particular remarks about the data dictionary, what about the > rest of you? END Neither do I. It is actually a useful resource. END N No N N > Okay, the bibliography has been updated and a couple of documents added > since last week. Any comments? ENd I suggested to add ISDC JEMX Analysis User Manual. Is it OK? END No, except that you've done a good job, Carol Anne, considering that > Yes, that's been overlooked, it's a good suggestion END we've only spent 2 hours and we're approaching the end END no comment > Finally, the index. I've added a mass of new indexing commands throughout > the document so that the index is a complete ISSW description in itself. > Mr (or Ms) Newcomer should be able to look up in the index any data > structure, input parameter, component and see how it's used by all the That index is really helpful. END > other components/parameters/data structures. There were some bugs in the > indexing, but with JC's sharp eyes I've ironed them out and the final > version should be even more comprehensive (and even bigger!) END Should we have an index for the index...? Just kidding :-) END > I hope you are just kidding! Any more (more constructive) comments? END No Would it be worthwhile to put a small paragraph with advice for this famous n newcomer on how to read this document such as the comprehensive index and having the JEMX user manual in the other hand? END > I can add such a comment at the beginning of the `Scope of the document' > section in the first chapter. END Good idea - something like a Quick Guide to this Document. > Well that's it for the document. Can I remind you all (except NJW and JC) > to bring your corrected copies to Valencia. I won't be doing any more on Peter, I have a list of comments for you. Is it sufficient to bring it > the ADD this week because I'll be preparing for the meeting, so all to Valencia ? (Easier than writing an email) END > updates can safely be left after the meeting. END To NJW: yes that should do. END > One last thing: is the Valencia agenda to everyone's liking? END Yes. I will just go to bed before 3 o'clock I guess. END Agenda looks OK END Y The agenda is OK y > So there's only one more thing to say: See you all in Sunny Spain! END Have a nice trip. Bye END *** njw has left channel #jemxadr : (njw) See you! END > Bye See you there. END > ! *** Signoff: Sami ([x]chat) *** jerome has left channel #jemxadr : (BYE)