$Header: /cvsroot/nco/nco/doc/TODO,v 1.233 2003/08/21 23:32:00 zender Exp $ 

******************************************************************************
NCO Wish list:
Numbers were assigned in chronological order
Organization:
Autoconf build items: Items specifically related to building and installation
ncap-specific items: These items do not affect rest of NCO
NCO-wide items: 15 high priority items are shown first, then another ~100 items
******************************************************************************

Autoconf-build items:
06. Track VERSION in ./configure, use doc/VERSION for now
11. Use autoconf standard for VERSION, HOSTNAME, USER
14. Replace #ifdef tokens in NCO code with autoconf-style tokens, e.g., #ifdef HDF5 -> #ifdef HAVE_HDF5, etc. Other tokens to consider are WIN32, _OPENMP, I18N, _LIBINTL_H, USE_FORTRAN_ARITHMETIC (deprecated). Of course must modify bld/Makefile to pass these style tokens as well so that config.h is not _required_, and that token use is consistent.
19. Implement autotools processing of doc/nco.texi to generate standard documentation: nco.info, nco.dvi, nco.ps, nco.pdf
20. DODS linkage appears broken on Linux
22. Many compiler flags in configure.in should be compiler specific--not just OS specific, e.g., case $host in *-cray-unicos) case $CC in CC) CFLAGS= ....

ncap-specific items:
04. Fix memory leak in ncap_var_cnf_dmn() or delete routine completely after replacing it with ncap_var_stretch()
07. Combine binary operations into binary_op_var_var and binary_op_var_att types to reduce number of functions
08. Cure final few bothersome compilation warnings, mainly in bison.simple.
Note that fixing these via hardcoded prototypes should allow for pure_parser
to be turned off, since this apparently changes the prototypes.
ncap.tab.c (warning with Linux when compiling parser):
/usr/lib/bison.simple:432: warning: implicit declaration of function `yylex'
This must be something like
extern int yylex(YYSTYPE *,prs_arg); /* Prototype for lexer */
but where?
ncap.tab.c (IRIX warnings caused by using IRIX cc rather than GNU gcc, so 
function prototypes are dumbed-down in bison.simple, and size should be
cast to (unsigned char)):
cc-1177 cc: WARNING File = /usr/freeware/share/bison.simple, Line = 334
  The indicated argument is incompatible with the corresponding formal parameter.
        __yy_memcpy ((char *)yyss, (char *)yyss1, size * sizeof (*yyssp));
cc-1177 cc: WARNING File = /usr/freeware/share/bison.simple, Line = 336
  The indicated argument is incompatible with the corresponding formal parameter.
        __yy_memcpy ((char *)yyvs, (char *)yyvs1, size * sizeof (*yyvsp));
09. Add outer loop over files so script may apply to each file
12. Ensure lexer does not perform un-necessary work during initial_scan (e.g., LHS casting...)
13. Test that LHS casting works regardless of value of LHS template. If true, remove value initialization block
14. If tally is superfluous in ncap then do not ever allocate it
15. Link to GSL functions, especially gsl_sf_* (special functions)
16. Get ncap build working with pure_parser turned off so lex/yacc builds work
17. Start in console mode if no script and no output file specified, e.g., nco> 
19. Make all binary operators handle ordering of operands consistently, e.g., divide(1,2)=1/2 not 2/1
20. Print out contents of each non-comment statement when dbg_lvl==1
21. Add two-argument functions like fmodf to ncap_sym_init() (using va_args?)
25. Turn arbitrarily fixed size array tokens NCAP_ATT_LST_NBR_MAX, NCAP_SPT_NBR_MAX into dynamically allocatable arrays
26. Make sym_tbl_nbr dynamic by implementing sym_add(), sym_rm() functions or linked list
27. Write and implement ncap_var_var_pow(), currently only ncap_var_att_pow() and ncap_att_att_pow() work
28. Make provision for pack() and unpack() to copy all attributes except scale_factor and add_offset from disk into new variable on disk?
29. Rewrite nco_var_pck() to use var_scv() arithmetic operations rather than more complex var_var() operations
30. Allow -S to specify multiple input scripts to be sequentially parsed
31. Put globals fl_spt_crr,ncl_dpt_crr,ln_nbr_crr in prs_arg or remove fl_spt from it
32. Need filename expander and $NCO_DATA_PATH handling so #include's may be generic
33. Document new math/rounding functions (acosh, floor...)
34. Make parse errors always print current line to terminal rather than silently failing
36. cygwin builds of ncap.l and netcdf fail because of dependence on syslimits.h. non-kludge fix may require updated gcc and mingw packages
37. Implement float function definitions in nco_rth_flt as inline to improve speed for MACOSX builds (inline is OK in C99 but will it break C89?)
38. Finish dimension-reducing operations (min/max/avg)
41. Make ncap function prototypes consistent with rest of NCO
42. Add switch allowing derived variable to take on all attributes of specified variable
43. Alphabetize function list in ncap
44. Fix four (incorrect?) warnings with Linux/GCC build caused by passing pointer plus integer:
../src/nco/ncks.c:392: warning: implicit declaration of function `nco_msa_wrp_splt'
../src/nco/ncks.c:396: warning: implicit declaration of function `nco_msa_clc_cnt'
../src/nco/ncks.c:436: warning: implicit declaration of function `nco_cpy_var_val_mlt_lmt'
../src/nco/ncks.c:476: warning: implicit declaration of function `nco_msa_prn_var_val'

Highest priority NCO-wide items that You can help with:
59. Add "averaged over dimension x from y to z" attribute to ncwa-processed variables
97. Add x_op=average... attributes to averagers for ncar_csm
124. Use `install' program and mkdir -p in bld/Makefile method
128. Ensure target 'dir' is made before *.d dependency files 
152. Change make or nco_dst.pl to add docs to /usr/doc in RPM
164. Spiff up NCO homepage web page (Logo?)
165. Create Makefile dependency to do 'make dir' if lib, obj, bin directories do not exist
178. Make _FillValue behave same as missing_value
191. Create short man page for ncap which points user to info pages or online manual
212. Ensure structures are always passed by reference (default is pass by value)

Remaining items of lesser priority:
28. Use var.nc_id as file ID from now on and rewrite old routines to do so
32. Implement packing with scale_factor and add_offset in all pertinent operators
50. Allow user-specified scalar weights to ncwa
51. Make getopt() a shared subroutine?
53. Add -t wall clock timer option? (then again, users can always use, e.g., timex())
55. nco_var_cnf_dmn() should be able to expand (x,64,128) array into (nx,64,128) array (maybe)
58. Add -o fl_out switch to supercede positional argument (GNU style)
60. Make something that can mask a field without having to average over a dimension
66. Make history attribute get appended, rather than overwritten, when in append mode
69. ncks option for file/field summary: var name list,var min,max,avg,mss_val,# mss_val,
70. ncxx with no options should do usg_prn() without error msg (like ncwa does)
75. Add warning to ncwa triggered by averaging over lat without weighting by gw with NCAR_CSM_FORMAT
79. Allow ncrename .$var_nm syntax to work on all operators (so variables not present cause warning, not error)
82. Make libnco namespace-safe by prefixing all library subroutines with nco_
86. Extend stride/wraparound capability to all operators possible
92. ncrcat -O -v base_time,time_offset arm.cdf arm.cdf foo.nc;ncks -H -C -h foo.nc | m gives bad time_offset on LINUX not SUNMP (problem with LINUX adding record coordinate to open file?)
99. Allow ncatted to dump file in ncatted format
103. Implement trapezoidal rule option in ncra, ncea
104. Investigate/Document reliability of -A operator when appending from large rank variables to small rank files. Add error message when variables are same size but one has a degenerate dimension.
In general, adding large rank variable to small rank file does work, e.g.,
ncks -C -O -v one_dmn_rec_var in.nc foo.nc
ncks -C -A -v three_dmn_rec_var in.nc foo.nc
ncdump foo.nc
105. Add easy-to-read tabular display option for file contents for ncks
106. Add switch to keep degenerate dimensions with ncwa, remove them from ncra
109. Change copyright printing so order is NCO vrs, netCDF vrs, URL, Copyright
111. Investigate whether using ncwa to average files where variables of type 
NC_CHAR have a record dimension causes core dumps or other problems.
i.e., LSM variable timecom caused subtle ncra problems for a while.
Problem caused when weight or mask has dimension not in dimension extraction list
ncwa -O -C -D 5 -v fl_nm -w gw ~/nco/data/in.nc ~/foo.nc
fails because fl_nm has dimension char_dmn_lng but gw has dimension lat
call to nco_var_fll() for gw requires that lat be in dmn_lst
113. Rethink normalization switch options in ncwa
114. Fix nco_var_cnf_dmn() so that returned weight always has same size tally array as template variable
115. When sum of denominator in ncwa (i.e., sum of weight) is zero, SIGFPE results 
Users should not be doing weighted averages if denominator sums to zero, but...
Desired fix here is a graceful exit instead of core dump
121. Problem reported when mss_val = 0.0 in averagers. C and Fortran 
versions of nco_var_add_real() etc. do not handle missing_value attributes in 
exactly the same way. C versions test both operands agains mss_val,
Fortran tests only op1. Since running averages are initialized to 0.0 before
entry, problems arise in C version when missing_value = 0.0. Only fix seems
to be checking whether tally == 0 before checking whether op2 == mss_val
123. Add HTTP protocol to fl_get() (in addition to DODS)
127. Make mss: signal retrieval from NCAR mass store
134. Problem on CRAY with ncra test #2, averaging float whose
missing value is double:
ncra -O -C -v rec_var_flt_mss_val_dbl in.nc foo.nc;ncks -H foo.nc
ncra -O -C -d time,0,1 -v rec_var_flt_mss_val_dbl in.nc foo.nc ; ncks -H foo.nc
Not too worried about this since it is Cray specific, 
Maybe 1.0e36d does not convert to 1.0e36f on Crays?
143. Add -W or -q switch to turn off squelch most warning messages in ncra, ncrcat (especially warnings about non-monotonicity)
144. Add option(s) to select all N-D variables where N=0,1,2...
145. Talk to Brian and implement latest netCDF and CF conventions, e.g., time_op
146. Allow proxies for coordinate variables so that, e.g., hyperslabs
may be specified for "date" even though "time" is the coordinate. 
Proxies must be one dimensional and monotonic, of course.
147. Use builtin mkstemp() so file is built in fast directory rather than, say, NFS-mounted directory like /fs/cgd  
148. Put in more useful error diagnostics for Schweitzer's data holes
These only appear in ncra/ncrcat because ncks treats same hyperslab as a wrap!
Must define a consistent convention here
cd ~/nco/data
ncks -O -C -d lon,10.0,80.0 -v time_lon in.nc foo.nc;ncks -H foo.nc
ncrcat -O -C -d lon,10.0,80.0 -v time_lon in.nc foo.nc;ncks -H foo.nc
149. Add ncks hyperslab tests to nco_tst.sh to catch inconsistencies like #148
150. Fully automate RPM production, e.g., 'make rpm' should build rpms and upload to redhat.com
151. Figure out how to dynamically modify web pages so files with
changing version numbers can be directly linked
154. Add mirror capability to web pages
158. Add policy for hyperslabbing single level values to User's Guide. 
Multi-file operators with record coordinate
treat single point hyperslabs differently than single file operators 
or than any operators with single point cuts on non-record coordinates.
See explanatory notes in nco_lmt_evl().
159. Add capability to handle superfluous intermediate files to ncra,
ncrcat when lmt_typ = dim_idx.
This would allow these multifile operators to handle hyperslabs where,
for example, every other file was not needed. This capability is already
implemented for lmt_typ = crd_val. This is an issue only when stride
is non-unity.
160. Add documentation of ncra, ncrcat superfluous file capabilities to nco.texi
162. Add GiNaC (Computer Algebra System) support to ncap www.ginac.de (maybe?)
166. Document 'make test' so people know it exists and use it
167. Improve/redesign test suite so it is more comprehensive and easier to add to
169. Merge TODO into sourceforge Task manager and bug manager as appropriate
173. Add switch to ncra which allows user to remove degenerate time dimension from result
174. Setup ncra,ncea,ncwa to do standard deviations sdn with one call
176. Replace Bison, Flex components of ncap with smaller C++ parser?
177. Make RPM production allowable by non-root, since building RPMs as sudo root screws up permissions in obj/bin directories of user
179. add netCDF to required packages list in RPM build
182. Add -L switch to print license terms
187. Are OpenMP critical regions sufficient for netCDF or must all other threads stop completely?
188. Make binary RPMS build with DODS option
189. Verify ncrcat multithreading works, produced "not a netCDF file error" once
190. Make typ_pck et al. use nc_type enum or something sane for default
192. ncdiff will sometimes subtract files of incommensurate sizes without complaint
ncks -O -v one_dmn_rec_var -d time,0,1 in.nc foo1.nc
ncks -O -v one_dmn_rec_var -d time,0,2 in.nc foo2.nc
ncdiff -O foo1.nc foo2.nc foo3.nc
This may make sense, e.g., for 1-D record variables of different lengths, but not for multidimensional variables
Former case should print warning, latter case should exit with error
193. Convert dimension sizes from type "long int" to type "size_t"?
195. Itanium builds!
196. ncwa errors in nco_tst.sh tests 15--29 when built with HDF4 libraries
No ncwa problems when built with NETCDF2_ONLY with Unidata libraries
197. Construct HDF format in.hdf from in.cdl for testing HDF-enabled NCO:
ncks -O in.nc in.hdf 
Command executes but in.hdf is full of screwy values
198. HDF-enabled ncgen does not parse in.cdl
199. Internationalize (i18n) NCO with gettext()
202. Make nco_tst.sh somehow summarize its own errors, e.g., "Total errors reported = 12" so that regression testing is easier. Perhaps nco_tst.sh should be rewritten in Perl or Python to accomplish this.
203. Perform benchmarking tests to quantify arithmetic performance improvements, I/O bottlenecks, etc.
206. Add sample NCO script, such as dst.sh, to doc directory
209. Add rules to automagically make MANIFEST and update Sourceforge download area
214. Disable threads > 1 on SGI temporarily because thread #0 is doing all the work, why?
215. nco_var_cnf_dmn() does not abort when variables do not conform because dimension list of one is subset of other but ORDER of dimensions differs, e.g., a(lat,lev,lon) !~ b(lon,lev). Attempting this will return incorrect answers!
222. Warn when ncea/ncra/ncrcat/ncecat operate on scaled variables ("Results will be meaningless if scale_factor and add_offset are not identical in each file")
224. Copy var_nm from command line so program, not system, owns memory. Then modify nco_var_free() to release var_nm structure member, and nco_var_dpl() to copy it. Same for dimension names and nco_dmn_dpl()
225. Pass aed by reference in nco_aed_prc()
226. Is xrf in nco_var_cnf_dmn() is really necessary? If not, remove it and make wgt arg const var_sct * const
228. g++ linking requiring #ifndef GNUC++ around fabsf,fmodf defs in nco_var_scv.c
230. Keep track of type of missing_value field in variable structure for reasons described in nco_cnf_var_typ()/nco_cnv_mss_val_typ()
232. Warn when arithmetic operators encounter packed variables?
233. Makefile.old rule for make tst appears broken for long FTP rule
237. Break down nco.texi into bite-size chunks, e.g., one file per section
238. Replace y=(typ *)nco_malloc(nbr*sizeof(typ)) with y=(typ *)nco_malloc(nbr*sizeof(*y))?
239. Create one global compatibility file (nco_xpf.[ch]?) activated by HAVE_ or NEED_ actions so do not have multiple platform dependent #includes as is case now with HAVE_STRCASECMP, HAVE_STRDUP requiring nco_sng_utl.h
241. Add run-time switch which requests that output be missing_value where ANY of the input fields are missing_value. 
244. Get builds working on testdrive.hp.com
245. Eliminate all instances of passing naked constants
246. Deprecate USE_FORTRAN_ARITHMETIC from Makefile
250. ncks -s should refuse to print strings formatted with %s unless they contain NUL-terminator. This will avoid segfaults like
ncks -C -H -s "%s" -v fl_nm ~/nco/data/in.nc
251. Does ncks -B correctly write packed variables in binary format?
255. Desktop icon for NCO
256. Turn udunits into .deb
257. Put hybrid_sigma_pressure etc. definitions into in.cdl
258. Develop syntax/procedure for specifying lists of hyperslabs for single pass, low memory consumption MSA jobs like Martin Schultz's in Discussion forum on 20030506
259. Variadic function extension to nco_malloc() to support printing debugging messages?
260. Warn (but work) when coordinate variable has more than two dimensions
261. Switch from toggling defaults to --no-option format for all Booleans. e.g., --abc should ensure alphabetization is True, and --no-abc should ensure it is false, rather than toggling.
264. nco_tst.sh failure:
ncap: WARNING not searching for /in.nc on remote filesystem, using local file n.nc instead
nco_err_exit(): ERROR nco_create
Permission denied
265. ncbo: Promote before operations using nco_typ_cnv_rth() in ncbo
266. ncbo: Finish ncbo.1 manpage
268. ncbo: use ncap var_cnf_dmn() technique so fl_1 fl_2 broadcast interchangeably
269. SGI autotools ncap build fails because HAVE_GETOPT_H gets passed even though getopt_long struct tokens not defined
270. Automate uploading of tagged .deb builds to ftp://ftp.debian.org
************************************************************************
End NCO Wish list
************************************************************************