evlatst3


.	STATUS	ISSUE	COMMENT/RESOLUTION	SUBMITTED	PRIORITY	VERIFIED
.
.		KEY: R=Resolved; NR=Not Reproducible, NA=No action
.
.
.		Functionality (general)
.		Should fill the scratch columns at fill time (rather than when doing a setjy, etc)	Requires NAUG agreement.	EF - 19Dec	Medium
.		importuvfits - antenna ids are assigned oddly (like original-1).		EF - 19Dec	High
.		can import* add to an existing MS	currently only available at the tool kit level; can add parameters to the task to enable this.	MR - 20Dec
.	NR	importuvfits crashes casapy	We've had three people on this and can't seem to reproduce it (after looking at the data and the logs).	FO - 19Dec
.		importarchive is slow (360 vs 34)		WB - 18Dec
.	R	Need an imhead task to get an image summary	Added to daily	CB - 19Dec
.	R	browsetable fails	Should be more robust (handled better in distribution)	EF - 17Dec, MR - 20Dec, JU - 21Dec
.		Logging
.	R	logging-casapy.log stopped getting output	Should be fixed; also added a 'refresh' button if this should happen.	FO - 18Dec
.		logging-want date optionally more accurately		FO - 18Dec
.	R	logger: difficult to copy and paste multiple lines	Fixed (only for RHE currently)	WB - 18Dec
.	R	logger is impractical for log messages longer than screen;Starting around line 1746 of: /home/parallax2/CASA/EVLA2006/AC629-X/casapy.log	Fixed	WB - 18Dec
.	R	First message was not readable; text was shifted up by half a line; message priority was 'ORMAL'. After restarting this went away.	Fixed	DW - 19Dec
.	R	feature request: click on origin and see parameters used in the task.	Added a task history - use listobs to see	WB - 18Dec


.		listobs formatting improvements: could derive name of VLA configuration, int time in seconds (could be hours if over some limit), explicitly print units of time (LST, etc), fieldnames run into RA, column entries not aligned, would like text layout of array, would like a similar summary of the gain table, some indication of whether a table is calibrated or not)	ms.summary issue	MR - 20Dec	High
.		more sense to change time from least to most significant (currently day month date time year)	ms.summary issue	MR - 18Dec	High
.		Plotting
.	NR	plotcal crash - glibc detected - double free	Smells like a table lock problem; investigating-opened with another tool to obtain a lock and then plot - still okay; not reproducible by user/developers.	CB - 18Dec
.	R	plotxy error: --------> plotxy(vis='test_ok.ms', xaxis='time', yaxis='amp', fieldid=0) DATASTRING: [TIME, AMPLITUDE(CORRECTED_DATA[1:4,1])] Reading data... Time spent reading from disk : 3.9 sec. <type 'exceptions.TypeError'>: only length-1 arrays can be converted to Python scalars. Tue Dec 19 22:37:54 2006 SEVERE : [ msplot::plot() ] Exception Reported: TPPlotter: From Python.	Investigated. .matplotlib/.ipython directory issue.	DW - 19Dec
.		plotxy - spwid=-1; -1 default		CB - 18Dec
.		plotxy -set antennaid=27; plots whole data; bad selection gives all data		CB - 18Dec, GvM -18Dec, EF - 19Dec	High
.	R	plotxy time labels are mushed; plotxy: data set spanning several days; automatic scaling/tick marks are not good; x-labels interfere.	Improved	CB - 19Dec, GvM - 19Dec
.		uvdist units should be in klambda		CB - 19Dec, MR - 20Dec	Medium
.		plotxy - correlation should not be case sensitive		CB - 18Dec	Medium
.		plotxy - would like to plot residuals (corrected - data, etc)		NK - 20Dec	?
.		plotxy - RL LR can't be selected individually		CB - 18Dec	High
.		plotxy - Tkinter errors		GvM - 18Dec	?
.	R	plot problems	mv .matplotlib/.ipython; in Cookbook (testing notes)	FO - 18Dec CB - 18Dec
.	NA	plotxy: plotspeed is slow (across the network); antenna plot took 14 seconds.	I believe this is an nfs issue. It's fairly sprightly when using local disks. Will re-open as needed.	GvM - 19Dec
.		plotxy - show polarizations in different color		CB - 18Dec	Medium
.		plotcal - needs more documentation		CB - 18Dec	High
.		plotcal: don't plot 0amp/0phase flagged values		EF - 18Dec	High
.		plotcal: need label of ant index and name		EF - 18Dec	High
.		plotcal: time labels are mushed together; axes labels improved (when zooming)		EF - 18Dec, CB - 19Dec	High


.		plotcal: antennaid should be a list not single value		EF - 18Dec	High
.		plotcal: how run with a useful timerange. Plot of BP solution insists on displaying a timerange much longer than the length of the calibrater observations. Either plotcal should detect the length of the calibration observations and set its timerange to the length or it should be possible to enter a timerange for the plots.		FO - 19Dec	Medium
.	R	plotcal: what is caldescid	It is a collection of correlations that were observed in the same way (with the same spectral window setting).	EF - 18Dec
.		plotcal: need ability to select source		GvM - 19Dec	High
.		plotcal: plotrange for abscissa and ordinate		EF - 18Dec	High
.		plotcal: parameter for time/frequency range to process		EF - 18Dec
.	R	plotcal: would like an option to connect the dots	plotsymbol='r-'	FO - 19Dec
.		plotcal: does not allow individual source/field_id, time range, or polarization selection		GvM - 19Dec	High
.		plotcal: overplot spw or pol with different colors		EF - 18Dec	Medium
.		plotcal: subtraction of cal values (R-L) or two spw		EF - 18Dec	High
.	R	plotcal: simpler iteration controls	unified with plotxy controls	EF - 18Dec
.		plotcal: matplotlib boxes should have popup help		EF - 18Dec	Medium
.		plotcal: zoom plots in x (not enough resolution)		EF - 18Dec	Medium
.		exportuvfits: accurate but poor performance		EF - 18Dec	Medium
.		would like on-the-fly correction in plotxy to check results before going on		CB - 19Dec	Medium
.		flagxy,plotxy - channel plots first channel is 1 not 0.		WB - 20Dec	High
.		want to plot at a given position angle (u=0, v=0- degs); have a uvpa parameter; difmap and AIPS both have this		MR - 20Dec	Medium
.		plotxy: yaxis should have same options as x axis.		MR - 20Dec	Medium
.		missing help for timerange		MR - 20Dec	High
.		flagging


.		flagxy - s (show option) should allow a write if they look good.		GvM - 18Dec	High
.		flagxy - want to be able to flag on a channel 0 dataset and then transfer the flags.		GvM -18Dec, CB - long	High
.	R	flagxy - diskwrite=True only works when region is set.	Updated in-line help to be more clear about the use of this parameter.	GvM -18Dec
.	R?	flagxy - persistent boxes, other oddities.	Seems to be peculiar to vmware handling of X	GvM -18Dec
.	R	Need TVFLG	Need to use the Flagging Options panel and set Flagged data to background to really see data.	FO - 18Dec
.	R	flagdata - can't select 'RL' or 'LR'	This is a task level restriction (I didn't include it in the list of options and so the Option checking is preventing you from proceeding). This is fixed in the daily.	CB - 19Dec
.		flagxy - when setting start=5; nchan=5; the edge channels appear on plot border with no margin.	So don't do that. Seriously, this is a matplotlib autoscaling issue and we likely need to adjust it. Having a plotrange parameter (as suggested above) might also alleviate this.	WB - 20Dec	Medium
.	R	flagxy: Couldn't flag data	Complicated mix up in the start channel; provided work around. Real fix in progress	DW - 20Dec
.		flagxy: error when acquiring lock on ms		DW - 20Dec
.		flagxy: don't understand flagging - if I flag one pol will it flag both. Need option to do both - look at one and flag one, look at one and flag two.	Currently, the flags can't be extended in scope.		High
.		viewer
.	R	Viewer: non-flagged values are grey	The workaround for the grey-out problem is: in the Options panel, select 'Show Flagged Regsionn…'->Masked to Background'. (Flags will be black as in TVFLG).	WB - 18Dec, DW - 19Dec
.	NR	Viewer: hung when changing viewer params	Need some help to try to reproduce this.	WB - 18Dec
.		viewer: update button in "load data' window is disabled.		WB - 20Dec
.		Calibration
.	R	B, BPOLY problems	Fixed and patched into the stable (AOC)	GM - 18Dec, EF - 18Dec
.	R	uvrange parameter is not working; this frustrated efforts for those with resolved calibrators. specifying a uvrange other than [0] caused problems.	Fixed in daily; patched into stable (AOC)	CB - 19Dec, GvM - 19Dec, FO - 19Dec
.	R	Can't apply 'T' solution in Bandpass	This was a restriction in the interface that should have been relaxed in the new version of calibrater. Fixed.	FO - 19Dec
.	R	fluxscale: irrelevant warning	Fixed in daily; patched into stable (AOC)	WB - 18Dec
.		Need to set the signal-to-noise ratio using in gaincal.		CB - 19Dec	Medium


.		plotcal neglects 2nd poln for B and BPOLY		GM - 19Dec, EF - 19Dec	Medium
.		plotcal multiplot ignored for BPOLY		GM - 19Dec	Low
.	R	calibration models - make available by default;	use /home/AIPS/TEXT/STARS for X (CB); note this is more complicated that originally thought. We have the high frequency images in the appropriate format/scaling. However, the images coming directly from AIPS are generally convolved subimages requiring scaling; AIPS doesn't in fact use these images but the associated clean components that come with them which are then scaled on-the-fly so the CC fluxes sum to the current SETJY value for the calibrator.	CB - 18Dec
.		D-term calibration produced NaNs		WB - 18Dec
.	R	Two calibrators with different uv ranges so need to run gaincal twice. Not clear on logistics - if I specify a caltable that already exists will it automatically append ? If I set append=False, will it complain or delete the old table?	Need to set append=True or will overwrite.	GvM - 19Dec
.		Look for Nans when trying to solve		WB - 18Dec	High
.		setjy : Compute the model visibility for a specified source flux density If this is meant to sound like the AIPS task of the same name, it doesn't sound like it does the same thing at all. SETJY in AIPS simply calculates the flux calibrator's flux density based on standard formulae, setting the fundamental flux density scale. This value is entered in the header but nothing else is done. help setjy confused me further. It seems to say it acts like AIPS, but the main help again claims to "compute the model visibility", which sounds very odd to me -- either one is computing the expect visibilities for the entire data set for a trivial model, or one is computing a single (!) visibility. Both options seem silly.		MR - 18Dec
.	R	Used a T solution to apply before bandpass.I got into trouble with this calibration because I first tried to correct for the phases before I calculated the bandpass table. I mistakenly? tried to use T solutions for this correction. I gave nonsense when I calculated the bandpass shape. However, when I used G solutions everything was fine. I suppose the issue might be how the T solutions are described. In principle, one could calculate a constant instrumental offset as well as the mean tropospheric correction but it appears that is not what is done. Thus the T solutions must only work when the phase offset between the two polarizations has been removed. This should be mentioned in the initial discussion of T solutions.	Frazer will give us any additional info as needed but okay so far.	FO - 19Dec
.		gaincal - shouldn't 'P' always be applied here (and in correct)?	NAUG agreement required here. Note: P calculation isn't trivial and so will cause performance hit and may not be worthwhile (non-polarization observations).	WB - 20Dec
.	R	Todays data set was AG650 -- 8 hour track, A array, 1665 MHz 15 channel continuum of Supernova Remnant CTB80 at around 33 degrees dec. I didn't do sophisticated imaging of the source but still got great results with little effort. This data set might be good for pointing self-cal as it has about 150 mJy of compact power near the half power point. B and BPOLY both seem to work, G worked, but not GSPLINE (noted below)	Fixed GSPLINE parameters (underlying interface had been tweaked).	WB - 20Dec
.		If asked to solve on a dataset that is completely flagged it correctly says 0 solutions found but does not clean up well (doesn't remove table lock - throws exception).		WB - 20Dec	High
.		msselect='' gives a warning message. Reduce the severity of the comment or remove (this is a common case).		WB - 20Dec	Medium
.		smooth gives low amplitude points that are not understood		FO - 21Dec	Medium
.		Imaging
.		clean -scales not in param list but wanted for multiscale	multiscale will be a separate task. coming soon.	WB - 20Dec	Low
.		Cookbook
.		CB Organization: Need listobs after load data		FO - 18Dec, CB - 18Dec
.		CB content: Browsetable needs more info		FO - 18Dec
.		CB content: Need info on when a process is done		FO - 18Dec
.		CB: browsetable: how to expand tables		FO - 18Dec


.	CB: browsetable: how to get other (sub)tables	FO - 18Dec
.	CB figures: browsetable figs are broken up (Mozilla)	FO - 18Dec
.	CB doc: How to enter a boolean/type checking	FO - 18Dec
.	CB organization: include link to flagautocorr with warning	CB - 18Dec
.	CB/browser - open new browser when select link	CB - 18Dec
.	Add example where you restore a taskname.last file; all examples thus far use intentionally saved files.	FO - 19Dec
.	documentation on split should be improved. Not clear what splittingn corrected data will do if data wasn't corrected.	DW - 19Dec
.	cookbook typos	WB - 18Dec
.	Would like to get rid of menus on left of cookbook; color coding of examples (light brown) is too hard to see; use uniform italic/bold for emphasis (not _).	MR - 18Dec
.	Version control: have it at the top (and up to date) and for each section (date/time). Have a summary of the last 10 changes.	MR - 18Dec
.	Printing: It would be nice to have a printable version with page numbers and a table of contents that refers to those page numbers. Printing chops off the right hand side of the page.	MR - 18Dec
.	Too much duplication of help files. Need more overviews - better user-oriented description of calibration; put MS description in some context	MR - 20Dec
.	Formatting: identify what the user types with a different font; different font for shell access.	MR - 20Dec
.	Use boxes to set off certain information - summaries, details; cookbook is too flat; examples: python oddities (ranges, 0-based indexing), ms_select examples, antennas discussion, CASA oddities (scratch columns, global params, required calibration order, zero-based indexing - mark those that will be removed), pylab commands.	MR - 20Dec
.	Need sections/chapter numbers for easier cross-referencing.	MR - 20Dec
.	mixture of pylab, python and task can be confusing - define all terms (like method).	MR - 20Dec
.	List of typos/bugs from report	MR - 20Dec
.	Need to point out that .last file is saved whenever a task is run (put in saveinputs section)	MR - 20Dec
.	Need section on tasks and tools	MR - 20Dec
.	Need to mention the use of 'go'	MR - 20Dec


.		autoflag output is unclear (rows versus pixels)		MR - 20Dec
.		Don't clearly elaborate logging (ipython.log versus casapy.log)		MR - 20Dec
.	R	History and Searching: how do you clear a line once you've searched and discovered you don't want to execute.	It's readline and has all the same commands. <CNTRL>-k will clear the line.	MR - 20Dec
.		Want overview of CASA first: everything in tables, calib separate from data, flags done how, then do details of MS, also words on units		MR - 20Dec
.		Distribution
.	R	Need daily version in CV	I have installed daily casapy rpms on merger & shadowfax in cville. These were built today using the active development branch that is used to build the daily installation in socorro. The "stable" rpm is what you get when you just run "casapy" from the command line upon opening an xterm. To use the "daily" rpms, do the following: bash-3.00$ . /usr/lib/casapy/20.0.696svn-001/casainit.sh CASApy version 20.0.696svn release 001 bash-3.00$ casapy CASA Version 2.0 Rev 711 Built on: Tue 2006/12/19 02:15:06 UTC [...] there is also a casainit.csh. This is the first iteration at generating daily rpms, and it is currently done by hand. As a result, it will only be "near-daily", but over the next couple of weeks I will work to automate the process. After it is automated, the "daily" rpms will automatically be installed on all RHEL machines in cville. With each generation of a new "daily" the version release number (currently "001") will be incremented. This will permit having multiple versions of "daily" installed when necessary. The remainder of the version number will stay the same, e.g.: stable release => 20.0.696-001 first daily => 20.0.696svn-001 second daily => 20.0.696svn-002 etc. The version number reported within casapy or from casapyinfo will reflect the version in the repository (currently 20.0.711). Each new "daily" will replace the former. There is one important caveat with using "daily" rpms. While they are built just like a release would be, they receive no testing and will likely even fail to compile on some days	CB - 18Dec
.
.		User Interface
.		First I want to note that with the task system I find I seldom need to refer to cookbook or reference documentation. I think this is a really good sign! So far I've found the granularity of the tasks appropriate and find the tools still easy enough to access (but with more documentation needed). I think it should be a goal to prevent the number of tasks from spiraling out of control -- perhaps no more than 12 or so tasks within a task group is a reasonable cutoff.
.		Data selection is too heterogeneous. Want to avoid using indices for most things and use names instead. This needs to be unified throughout calibration, plotting, imaging. Detailed comment: I like that in fluxscale you specify sources by their names. Why do all other tasks have fieldid's? i.e. gaincal. I would much rather enter calsources for gaincal than try to be sure that I've written a msselect properly. In general, I don't think msselect belongs in tasks.	In progress	All - 19Dec	High
.	R	If an error occurs in the task, the rest of the execution should be suspended. currently it goes on and generates more errors which become increasingly obscure.		EF - 20Dec, SM - 15Oct
.	R	Set the task default when do an execfile 'taskname.last'	implemented	GvM - 19Dec
.	R	Importarchive: doesn't import from the archive	Is importvla better? If so will change	FO - 18Dec
.	R	Importarchive: why does it need bandcode	You can restrict the filling process to a single band if there are multiple within the specified archive file(s)	FO - 18Dec
.		listobs - need ability to print directly to file		FO - 18Dec, CB - 18Dec, MR -20Dec, JU - 21Dec	High
.		listobs - verbose=false should tell date of obs		CB - 18Dec
.	R	listhistory - doesn't include import info	added to import* tasks	CB - 18Dec
.		change datacolumn selections to: raw, corrected and model (use of data is confusing).		CB - 19Dec	High
.		correct uses msselect rather than specifying sources.		CB - 19Dec	High
.		importarchive - option to not include ac data		CB - 18Dec	Medium


.	R	gaincal solint=float	typecast int to float for solint (all tasks)	CB - 18Dec, WB - 20Dec
.		fluxscale requires caltable but correct requires gaintable - shouldn't these be the same		CB - 19Dec
.		help and the PAGER		FO - 18Dec, MR - 18Dec
.		defaults for some are "False"		FO - 18Dec
.	R	Imaging: tried to make stokes IQUV (clean) only see I	error in clean task. fixed.	WB - 18Dec
.		go function fails		WB - 18Dec
.	NA	protect task values	It's Python baby; not aware of a way to do this (have looked quite a bit).	WB - 18Dec
.		formatting for some parameter values		WB - 18Dec
.	R	allow: inp clean rather than inp 'clean'		WB - 18Dec
.		browsetable is 1-based		WB - 18Dec
.	R	alphabetical listing is non unix	Sorry - it was easier to add to the other way!	WB - 18Dec
.		help - would like explain function		MR - 20Dec	High
.		importarchive: case sensitivity of parameters like bandcode is annoying		WB - 20Dec	Medium
.	R	inp - if colon(:) separating parameters from comments were replaced with # then inp would be valid python and could be cut and pasted more easily.	Agreed and in fact was the first go; can't recall why it was changed.	WB - 20Dec
.		Need to plot out antenna names not indices - very confusing when choosing a reference antenna	This is what is done now. Compare the listobs output (using verbose=True) and the plotants output; you will see that they are the names. Ah, not the problem, the problem is that you then have to specify the id, not the name. In progress.	FO - 20Dec
.		tablein in accum should probably have some other name to avoid getting it filled by other tasks through as a global parameter		FO - 20Dec
.	R	input correct gives wrong one line info		GvM - 20Dec
.	R	task invert allows specification of clean type parameters		GvM - 20Dec
.	R	default stokes in invert causes an error		GvM - 20Dec
.	R	remove multiscale from clean algorithm		WB - 21Dec


.	R	seg fault when re-ran clean and have existing output files already existing.		GvM - 21Dec
.		contsub: would like to add a column rather than replacing the corrected data.		GvM - 21Dec	Medium
.		importarchive has projectname parameter that isn't used (commente out)		JU - 21Dec	High
.		importarchive default bandname gives error.		JU - 21Dec	High
.		Need a global lastexit.par file to save the stat of casapy with the option to execute this file on restart of casapy (a *rc file setting); this could also tie into the restore command, e.g., restore(mode=last) versus restore(mode=default).		SM - 22Dec, CB - 22Dec	Medium
.		Have fluxscale (optionally) set the model data column to the derived value.		FO - 22Dec	Medium
.		Must be trivial to switch between different versions of the software (helps evaluate whether the user is doing something silly or if there is a problem with the new version); ideally this should enable switching versions at the task level without leaving the system.		MR - 22Dec, MC - 22Dec	Medium
.		Discussion/Verification
.	R	You can get the casa/aips images from directory /home/thuban2/efomalon/casa/cassini J0334_AIPS.FITS and FORNAX_AIPS.FITS is from the data calibrated in AIPS J0334_CASA.FITS and FORNAX_CASA.FITS is from the data calibrated in CASA Some information: 1. The data were read from the VLA archive directly into casa and aips 2. Some minor editing and the usual fluxscale were done 3. The source J0334-401 is the calibrator and gaincal was made for each 2-min scan; both in casa and aips. 4. The calibration was applied to J0334-401 and its target sources FORNAX-A. 5. The CASA calibrated data were then read into AIPS in order to use the same imaging/cleaning algorithm for both data sets. The images are 2048x2048, Clark clean with 2000 iterations, no cleaning boxes. 6. The two FORNAX-A images are virtually identical. The faint ripples are caused by very extended structure. The peak brighness is 85 mJy and the rms noise is 0.12 mJy, a dynamic range of 700. 7. The two J0334 images are essentially self-calibrated images of this source since it was its own calibrator. The casa image is better quality than the aips image. The peak = 1.02 Jy with an rms noise of 0.17 mJy, a dynamic range of 6000. Comments: The FORNAX-A images are limited in quality by the phase uncertainty in the calibration. Hence, both images are nearly identical. The J0334 images, made from self-calibrated data, should be higher quality and are. I am a bit surprised that the casa image is significantly better than the aips image. I couldn't redo the calibrations (because of the polarization calibration bug), so I am not certain that the editing and calibrations are identical---but I think they are. So my tentative conclusion is the the gain and phase calibrations done in casa are more accurate than those done in aips, probably at the level of a percent or a few degrees at most. The FORNAX-A images are more dominate by phase noise between the calibration scans, so are not sensitivity to small calibration differences on the calibrator observations.	Very pleased that they're so close.
.	R	CASA/AIPS weights: I've looked at the data weights for an archive data set that was read into aips and casa. It is a 1.4 GHz, low declination observation, so the variation of system temperature with elevation is very clear. The time dependence of the weights via their shapes is very similar to a level of 1 or 2 percent for casa and aips. However, the casa weights are consistently 1% for the aips filled data set. The casa weights near source transit are about 530 / Jy2. The aips weights are about 50,000 / Jy2 Now, the rms noise of the calibrated data (for this data set) is about 0.04 Jy. Hence, (1 / 0.04Jy)2 = 625 / Jy2. The rms noise of the raw data is about 0.004, arbitrary units and (1 / 0.004) = 62,500. Is this how the weights are normalized? Thus, the casa and aips weights are similar in their general dependence, but are normalized by about a factor 100 differently. This scaling difference should be reconciled since combining aips-reduced and casa-reduced data would be a problem with the relative weighting. I forgot that AIPS determines the revised weights on the fly in a multi-source data set, using the amplitude in the CL table. After splatting so that the AIPS weights become the true final weights, the difference between the CASA and AIPS weights are less than 1% and follow the same diurnal (elevation) signal.	I gather that now the final, calibrated weights agree to better than 1% -- is that right? If so, that's very reassuring. Note: Should check all Ifs (MR - 18Dec)	EF - 18Dec
.
.