ChangeLog for mvmf, in narrative order (oldest at top, newest at end) In general each entry is: yyyymmdd:who:where multi-line indented narrative text "who" is who made the change, and is one of: mem Mark E. Mallett "where" is the gross classification of where the change(s) were made, and is one or more keywords (comma-separated if more than one), from one of: mfl The MFL language misc Library code from the 'misc' directory mvmcas The mail client assessment server program mvmda The mvmda Mail Delivery Agent program Note: mvmda used to be called 'mvmf'; some older entries may be confusing as a result. Some of them have been fixed up a little. mvmf Generic; the package in general, also used if the edit involves many parts mvmtr The mvmtr Mail Transport Agent Receiver program cusp Something in the cusp area tool One of the other tools or applications rel A note about a "release" and/or version bumps Some applications have version numbers (these are noted in the "rel" lines). Version number is major.minor.fix : - major is major release level (0 is development, 1 will be first official number) - minor is minor release level, incremented upon each new version containing a new feature. <100 is alpha, 100+ is beta or better. - fix starts at 0 for each major.minor, and is incremented for each version that changes only existing features (bug fixes or improvements). Some new features may sneak in under a 'fix' increment. Little changes (like cosmetic stuff) are not necessarily noted here. Edits prior to the first one were not noted, sorry. Note that there is some schizophrenia about the terms mvmf vs mvmda in the notes, as the code was originally just an MDA called 'mvmf' and later evolved to a package called 'mvmf' that includes an MDA called 'mvmda' --BEGIN-- 20030829:mem:mfl in Mfl/mfl-pp.c, pp_define wasn't nulterm'ing the macro body, which produced random results (generally OK in freshly allocated memory but not OK if memory had churned some). Fixed. 20030829:mem:mvmda add version.h and -v, initialize to version 0.100.001 . Numbering scheme is major.minor.fix: - major is major release level (1 will be first official number) - minor is 3-digit minor release level, incremented upon each new version containing a new feature. - fix starts at 001 for each major.minor, and is incremented for each version that changes only existing features (bug fixes or improvements). 20030902:mem:mfl Turn preprocessor inside out. Formerly, the top-level input loop was responsible for looking for "#" at beginning of line. This had the unfortunate effect that preprocessor statements were not recognized inside compound statement blocks (and sieve blocks). Change to recognize preprocessor introducer inside the mfl_token routine (low level). 20030902:mem:mvmda Change version format already ... don't pad digit strings with zeroes (so version is 0.100.1). 20030920:mem:mfl Add a hack in the "header" statement. Specifying a header name of "-body-" will test against the cached part of the body. No translations, no mime selection, just pretend the cached body string is a header to test. This is a hackish response to the "swen.a" virus where we want to test for "September 2003, Cumulative Patch". The style of this hack was chosen so that when it is removed, the only harm will be that the effect stops working (the syntax will not cause an error). And the hack *will* be removed when we have a real "body" test. 20031001:mem:rel mvmda:0.100.2 20031001:mem:mvmda When invoked with -q, mvmf translates "hard error" internal code into exit code that tells qmail to remove the message from the queue so it won't be tried again. Minor fix to make sure "hard error" is translated to exit code 0 when -q is not used and when QMAIL support is compiled in (i.e. it's assumed qmail is running). Version bump because of necessity of installing this copy. 20031008:mem:rel mvmda:0.101.0 20031008:mem:mfl Add :regex string match 20031008:mem:mvmda Add -p for pass-through mode 20031102:mem:rel mvmda:0.102.0 20031102:mem:mfl Initial implementation of "string" type. Supports variable instantiation, assignment, coercion, and addition (append). Other functions to come as needed. Also some fixes required to prevent strings inside of aggregates are not yet done. 20031110:mem:mfl Add mfl-func module; code for C-like functions. Parsing of function calls is now implemented. Remove parse node type PN_FUNC and replace with distinct PN_FUNCREF and PN_FUNCDEF. 20031111:mem:mfl Some more string handling support stuff. 20031111:mem:mfl Rework the sieve functions which take string lists, and let a C-like expression appear wherever a string literal can. Because of sieve's use of spaces as value separators here and there, had to make a restriction that a string expression can only appear where a space does not separate terms. Mostly that means that string expressions can be used inside of a string list. So, this works: exists [(string)"X-Check-" + x] but not this: exists (string)"X-Check-" + x 20031219:mem:mfl Check for a null reference in valptr_resolve, which could happen for an uninitialized string pointer. 20031221:mem:mfl parse_compound was not complaining if EOF seen before closing brace; make it do so. 20031221:mem:mfl Above fix revealed that parse_compound was still returning the pointer to the parse tree even if the parse tree had been freed because of error. Change return epilogue slightly to make sure this doesn't happen. Make some adjustments in a few other functions to use the same style. 20031223:mem:misc On receipt of a message with naked C/R characters in the header, mvmf was not terminating each header line as of the CR. This really shouldn't happen, but if it does, it creates a mess. So-- update omm.c to use getbtrec() instead of getbline() when reading header lines, and specify that naked CRs can terminate lines. 20031223:mem:rel mvmda:0.102.1 20031223:mem:misc Need more aggressive fixing of CRs: rework omm_write_body to transform naked CRs and CRLFs into newlines. Also rework the code that quoted "From " sequences. 20031223:mem:misc Some changes in omm: make omm_read_header validate header lines and stop reading the header when it sees a non-header line (previously it stopped when it saw a blank line, but that was wrong). Also do some other misc code changes to do more consistent header validation. 20031224:mem:rel mvmda:0.102.2 20031224:mem:misc Bug in the above omm_write_body fix: if there is no message body this called in_charback() after getting EOF (stream was empty of course). Fix to check for EOF; will also fix in_charback to detect both EOF and null stream. Install needed, so version bump again. 20031226:mem:rel mvmda:0.102.3 20031226:mem:mfl Some function renaming for clarity: parse_type -> parse_type_usage, parse_type_type -> parse_type_usage_tree, parse_type_decl -> parse_decl_stmt. Begin better documentation on the type parsing. 20031226:mem:mfl Remove the STfunc symbol table and store function names in the STglobals table along with other variables. Make built-ins work this way. Add generic pointer type PO_PRIVATE for any private information associated with a VALUE (e.g. a function control block). 20031226:mem:mfl A bunch of stuff for initial support of mfl functions. Fork the parsing of initial values a little better; treat function body as an "initial value" for a function name. mfl functions working; argument definitions parsed but ignored. Function prototype (function declaration without a body) gives warning. 20031226:mem:mfl Add "return" statement (and SCRETURN status code) 20031227:mem:mfl Rename PN_TYPE to PN_DECL -- it has always meant a declaration and not a type, so make it be more meaningful. Rename PN_TYPREF to PN_TYPEREF. It bugged me. Add conditional MFL_ALLOW_REDECLARATIONS to control whether redeclaration will replace a variable or produce an error. 20031228:mem:mfl Fix a major bug in mfl_token - the digit string for TK_INTEGER was not terminated until *after* attempting to parse a suffix (like 'K'), but values were taken from that string prior to this. So even in a simple case like 1234/10 the '10' might be seen as '1034' or 10 plus remnants of some older buffered string. That this bug lasted until now is frightening. Unfortunately this will require a release as soon as practical. The whole digit parsing subsection should probably be reworked anyway. Much of this mfl_token code is ancient and fuzzy. 20031228:mem:mfl mfl functions now accept function arguments. Bugs undoubtedly lurk: needs testing. mfl functions now coerce return value to the declared type. 20031226:mem:rel mvmda:0.103.0 . Minor version bump because of mfl functions. and return statement. 20031228:mem:mfl Add sieve statement 'C' to execute a C-like block. 20031228:mem:mfl Stuff for string operators.. Remove patmat_* routines from Misc/omm and migrate them to the MM library as strmatch_* routines. Add $_match() built-in-function to access the last matched pattern or subpattern. Add op_seval member to OPINFO-- the rtc (routine-to-call) for an operator for strings. Add logic to eval_rel to access this new op_seval member. Add support for these string relations: '==', '!=', '<', '<=', '>', '>=', '=.', '=?', 20031230:mem:mfl "envelope" didn't check for "envelope" capability. Make it do so. 20031231:mem:mvmf Some misc stuff from a compile under FreeBSD 20040101:mem:rel A raw release to the outside. mvmda 0.104.0 20040102:mem:mfl val_coerce: add coercion from string to numeric types, e.g. (float)((string)"4.5") 20040102:mem:mfl Factor some code dealing with floats. 20040102:mem:mfl Fix a bug in default printing of a (double) value. (value was ok, printing it wasn't.) 20040103:mem:mfl minor fix to "if" statement parsing: use mfl_sigtoken() instead of mfl_token() when looking for opening paren in test portion. Otherwise '(' had to come on same line as "if" which is just wrong. 20040104:mem:mfl Add parsing for initializers on variable definitions. Initializers can now be used with scalers, structs, and arrays (but not unions). Started with code partially optimized per data type but made much of it fairly generic using eval/assign functions that, while clearer, is less efficient than the per-type stuff. Probably not a big deal. 20040105:mem:rel Repackage, flaws and all, before the next set of changes. mvmda 0.105.0 20040108:mem:mfl Bug fix: missing RHS of infix operator (e.g., "int x; x+;" or "struct { int a; } x; x.; " would coredump. Now report error about non-postfix operator. This bug is probably a result of a pre-changelog change allowing expression parsing to return a null tree without error. 20040108:mem:mfl Bug fix: "typedef { int x; }" would coredump. Similar to above: some prior change made it ok to return a null type usage. Minor change to the flow in parse_typedef to do better validation. 20040108:mem:mfl Null type usages like "int ,,,;" didn't cause a syntax error (nor did it cause an execution problem): the leading commas were simply skipped. I'm all for allowing whitespace, but this is clearly an error: make it complain. 20040120:mem:rel Required to catch a bugfix in mml-str.c mvmda 0.105.1 20040127:mem:misc Change some 'long' types to 'time_t' in msgstore.c 20040127:mem:misc Fix a bad call to strncmp in omm.c . Add some missing includes of to a couple of .c sources (to catch prototypes for strcmp et al). 20040203:mem:rel mvmda 0.105.2 20040203:mem:mfl The 20030902 preprocessor edit didn't go far enough. A function ps_skipfalse() which looked for the end of a conditional would never get the chance to find it, because the tokenizer (mfl-token()) is now responsible for invoking preprocessor directives. i.e. while ps_skipfalse() was busy looking for (say) "#endif" the tokenizer would execute the #endif before ps_skipfalse ever got the chance to see it. Added some mfl-pp.c logic to better deal with the fact that preprocessor directives are coroutines with the mfl-token() stuff. It could probably be cleaned up more.. 20040303:mem:rel Note: significant enough to warrant build + version bump. (two internally, because all prior versions were erroneously released as their *next* version and not their current one. e.g. 0.105.1 was erroneously released as 0.105.2) mvmda 0.105.3 20040303:mem:mfl Allow "C" escape to be used as a test statement e.g.: sieve { if C { xxx } } ... Previously "C" was marked as "ACTION" only. This is considered a fix, not an enhancement. 20040303:mem:mvmf Relink to catch a change in bstr_dup() which now preserves the terminating byte. 20040303:mem:tool Some recent change made the mvmflogsum utility not compile. Needed to include . 20040304:mem:mvmda Change the default location of the system-wide includes directory to be /usr/local/lib/mvmf/include (i.e., add "/include"). 20040305:mem:rel mvmda 0.105.4 20040305:mem:mfl Bugfix: in eval_plusstr (string addition), if the first term wasn't a string, it recursed with the terms reversed. This meant the strings were concatenated the wrong way (second then first). Further, there was absolutely no need for this test since the first term was stringized via a general routine anyway. Removed the test. 20040305:mem:mfl Bugfix: three(!) in stringval_eval(), which produces a temporary reference to a nul-terminated string. First, if it was evaluating an expression, it was assuming the result is a string. call val_coerce to make sure. (This only is a problem when people use scripts in unexpected ways: but isn't that often the case with bugs?). Second, must make sure to call bstr_nulterm on the result, to guarantee the string is nul-terminated. Third, must set sv_litF to FALSE in this case, as it's not a literal, it's a value struct. 20040305:mem:mfl Bugfix: in val_finish(), for the complex storage cases, the comment said "release the heap reference" but the code said heap_access. Change to heap_release(). 20040305:mem:mfl Bugfix: in exec_decl_value_func(), once the function has been registered successfully, remove the function arguments block from the xinfo pointer in the parse tree node, so that it doesn't get deleted by tree freeing. (Just like we do with the function body!). 20040305:mem:rel mvmda 0.105.5 20040305:mem:mfl Bugfix: string definition with initializer didn't work. Needed to properly construct heap reference for the lvalue used in the assignment. Fixed in exec_decl_value(). (May need later adjustment when we do more work on strings to allow a string to be a member of an aggregate.) 20040305:mem:mfl Change: remove TC_DOUBLE typecode from mfl.h and from lots of case statements. It was never used since "double" is a TC_FLOAT with a size of 8. 20040305:mem:mvmda New: add some ending stats in the log, including elapsed time, user and system processor time, and memory used. 20040305:mem:rel mvmda 0.105.6 20040321:mem:mfl Change: many changes in 'string' types (initial implementation was meant to be a steppingstone). Too many particulars to list. 'string' type is now implemented using REFSTRs which do not require NUL-terminated strings, can contain references to substrings, etc. Also make many changes to use ESTRs to generalize the text handling code. Some code forking to clarify things and isolate particular functions. New functions for various temporary results. Stuff like that. 20040321:mem:mfl Add initial version of 'substr' built-in function. 20040321:mem:mfl Bugfix: clean up some accesses and releases of values and objects, add better detection of access/release problems. 20040321:mem:mfl Bugfix: various places (mostly in SIEVE code) where evaluation return value (e.g. rvalP) was not always set, particularly in error cases. Noted mainly because of access/release cleanups. 20040321:mem:mfl Remove VF_HEAP flag. New pointer object types make this less relevant; easier to dynamically determine the object type. 20040321:mem:mfl Add FSV storage/object type. 20040321:mem:mfl Change various internal SIEVE support routines calling sequences to avoid passing rvalP return value around where unnecessary. 20040324:mem:mfl Remove the HEAP struct which was based on BSTR, replace with DVS (dynamic value/variable storage). "HEAP" was a bad name choice; also the original choice to base auto storage on BSTRs was partly because of the idea that that would support the initial "string" types implementation. (A more elaborate string type was always planned, but I needed to get some string functions up before getting there.) 20040327:mem:mfl bugfix: at some point some function parsing code was moved into the function module (mfl-func) but "mfl-parse.h" was not included. One of the parsing routines (parse_type_usage()) was being called incorrectly and not caught because the function prototype wasn't being used. 20040401:mem:mfl bugfix: in the SIEVE module, various places that follow an LSEQ parse tree node list were wrong. The list was always moving left even in the case where the node wasn't an LSEQ (non-LSEQ is a parser optimization for the last node, to avoid creating an extra node to follow). This has only come up in circumstances where an expression's root has both a left and right node-- oddly, this has meant that more complex expressions would not be bit by this bug. 20040401:mem:rel mvmda 0.106.0 20040406:mem:rel Relink with new mml to catch an improvement to the match_wild stuff. mvmda 0.106.1 20040414:mem:mvmda For bounces, add a note about who the intended recipient was, if known (with a possibly mangled address because of qmail's rewriting). 20040414:mem:rel mvmda 0.106.2 20040415:mem:rel Relink to pick up a bugfix in mml-str . mvmda 0.106.3 20040611:mem:mfl Bugfix: in val_string_val, source string can be "stolen" if stealF is TRUE and if the reference count is 1. Comments said that, however code didn't check the refC. When a string is improperly stolen (because of this bug) it can be altered improperly. Note: this string stealing stuff was part of the first-pass implementation of string types, and doesn't really make sense in the refstr implementation. Probably best to rip it out. 20040615:mem:misc Bugfix: in msg_write, failure to open a new message did not output an error message. (The errno was carefully preserved around the potential error message writing, and then the error cause was reported, but the actual error message was not written.) 20040616:mem:mfl Add ^ variants of string comparison and string match operators (but not regex). e.g. s1 == "foo" for case sensitive compare; s1 ==^ "foo" for case insensitive; s1 =? "*foo" or s1 =?^ "*foo" . 20040616:mem:mfl Bugfix: logical operators && and || were implemented as generic infix operators: evaluate both terms, then apply operator. This means that short-circuit evaluation was not being done. 20040616:mem:mfl Bugfix: the operator table for the "&=" referenced a logical AND function rather than bitwise and. Discovered when logical AND function was removed in short-circuit evaluation fix (previous entry). Change to use the proper bitwise AND function. 20040616:mem:rel mvmda 0.106.4 20040803:mem:mfl Bugfix: parse_sizeof() -- when assembling a PN_TYPEREF node, one branch of the parse node has a pt_info element whose od_P pointer points to a TYPE. The corresponding od_del (function to destrory the od_P info) is supposed to be NULL here to inhibit freeing the TYPE info. The default handler is dealloc()-- result is inadvertant freeing of TYPE node when sizeof() used. Found with malloc debugging on. 20040803:mem:misc Bugfix: msg_bounce() -- The subject for a bounced message is supposed to be "Auto: original-subject" (original-subject being the original subject). Somehow in the conversion to using REFSTRs for header storage, the old BSTR offset was used to skip over the header name, so we got "Auto: Subject: original-subject" instead. Fix to use the offset from the REFSTR. 20040804:mem:mvmda Bugfix: in mvmf.c, mvmf_check() and mvmf_normal(), the mfl script file was opened, the command stream was processed via mml-in routines, and the mfl script file was closed. The problem is that the low-level mml-in routines will fclose the file when EOF is reached. This means we closed the same handle twice. Fix by removing the outer fclose(), but this probably indicates that better stream unwinding is needed. Bug found only on linux systems; aborts reported by Peter Haag led to finding this. 20040804:mem:rel mvmda 0.106.5 20040806:mem:mvmf Replace the $(.OODATE) construct in mfl/Makefile and misc/Makefile with $(?) which seems to be more universally supported. Per Peter Haag. 20040806:mem:rel mvmda 0.106.6 2004????:mem:misc sieve "redirect" is supposed to preserve the envelope "from" if it can, rather than using the current user. This *might* not happen if the Return-Path info isn't present or differs from the envelope From in the environment; change msg_forward to take a flag that says whether to more explicitly preserve the envelope from, and if that flag is set, to use any ascertained sender as an explicit from address when invoking the mailer. If the flag is not set, just let the mailer do its thing (which will probably take its cues from Return-Path, depending on the mailer). 2004????:mem:rel mvmda 0.106.7 20041003:mem:misc Tear apart all the code dealing with reading mail messages, rewrite with MIME in mind. This is as planned: the old stuff was just there to get by until we could work on this. Mail messages now use "msgpart" data structures which represent individual message parts, potentially recursively. Also add the notion of message containers. The old "omm" (open mail message) stuff has been almost completely stripped away and many old omm_xxx functions have been converted to msgpart_xxx functions. "omm" now just provides a few entry points to deal with the active message; everything else goes through msgpart data and functions. Some of this is a step backwards in efficiency as there's a lot of byte-by-byte I/O now. Leave for future optimization. 20041003:mem:misc The new message container stuff now has support for writing message data to temporary storage if the entire message can't be cached and if the container file doesn't support seeking. Prior to this, non-cached parts of messages could only be accessed multiple times (e.g. for multiple actions on a message) if the input file was seekable. 20041009:mem:mvmda Add "-b" command line option to specify the total amount of memory to use for body caching. Use this instead of the hardwired number used previously. -b is bounded by a minimum and maximum specified at compile time- the maximum limit is used by default. 20041015:mem:misc Bugfix: msgpart_hdr_next() (and its predecessor) was defined such that it optionally matched an initial substring of a header. However all places where callers were using this option assumed that it was matching a full header name completely. e.g. if passed "Subject" the function would match "Subject-x". Redefine this function so that it selects header names completely, not partially-- if we ever really need a function with that other behaviour, we'll implement it as a different name. 20041015:mem:misc Bugfix: also in msgpart_hdr_next(), the function is supposed to set the REFSTR fetch pointer to point to the first non-blank character in the header body (after the header name and colon). It was not doing this if given NULL as the header string to match (i.e., match anything); also, it was not doing this correctly if the initial matched substring was not the entire header name (moot now, as that behavior was changed, see previous entry). 20041015:mem:misc Also: msgpart.h had a template for a function msgpart_hdr_prev() analogous to msgpart_hdr_next() but that function was never defined. Add generic function msgpart_hdr_step() to step forward or back, and reimplement msgpart_hdr_next() (and add msgpart_hdr_prev()) simply to call that function. 20041018:mem:mvmf Add $(LDDEBUG) to the Makefiles (specifically to make it easier to add -pg profiling via setting environment variable). 20041009:mem:rel mvmda 0.107.0 20041021:mem:mfl Bugfix: in do_exec_svbis_header (the sieve 'header' test) there is a hack to treat a header name of "-body-" as the cached part of the body. If no body is cached, the test is supposed to return FALSE. Was actually returning SCDONE (indicating match); change to return SCOK (no match). This only came up because the msgpart structures no longer necessarily have a body initialized, whereas the old omm stuff always initialized the body cache for the entire mail message. 20041021:mem:rel mvmda 0.107.1 20041022:mem:mvmda Add -S command line option to start scripts in sieve mode. I suppose this qualifies as a new feature. 20041022:mem:rel mvmda 0.108.0 20041128:mem:mfl Bugfix: the 20040616 fix to && and || needed fixing again, argh. That new code failed to get the rvalue of the first term (so it sometimes worked, but only when the first term didn't have an lvalue preserved). 20041128:mem:mfl,misc Changed the omm and sieve stuff to have a notion of the "current message part." Header and body tests will be done against the currently selected part. Initial selection is the top-level message part. 20041128:mem:mfl Add various $msgpart_ built-in-functions. Not sure these will stay around; adding them to get some script level access to the new(ish) mime (messagepart) code and data structures. Added selection functions: $msgpart_go_child(), $msgpart_go_next(), $msgpart_go_parent(), $msgpart_go_top() which select the appropriate part as the "current message part." At some point I'd probably like to have functions (such as something like 'msgpart_next()') that return a structure pointer for access by the mfl code, but we aren't at that point yet. 20041128:mem:mfl Similarly, add mfl built-in functions: $msgpart_cur_delete(), $msgpart_cur_undelete() to mark/unmark current messagepart for deletion. 20041128:mem:mfl Rename mfl/mfl-data.c to mfl/mfl-misc.c . 20041128:mem:rel mvmda 0.108.1 20041205:mem:mfl Misc code forking, changes, and new routines to support header access mfl functions. 20041205:mem:mfl Add mfl built-in functions: $msgpart_hdr_first(), $msgpart_hdr_last(), $msgpart_hdr_next(), $msgpart_hdr_prev() This is an initial stab; may want to change the stepping functions to take a comparison arg, e.g. instead of: hp = $msgpart_hdr_next( hp ); have hp = $msgpart_hdr_next( hp, "Cc:" ); 20041205:mem:mfl val_print: for a null pointer (PO_UNDEF or value of zero), don't attempt to print the pointed-to value, but print instead. 20041205:mem:mfl val_coerce, val_coerce_ptr, val_get_int: Changes to allow integer 0 to be coerced to a pointer type, resulting in PO_UNDEF type and a pointer value of 0; allow pointer to be coerced into an int, for pointer comparisons. (This may need further attention for platforms where a large int type can't hold a pointer value; however the only goal at this time is to allow comparison to zero.) 20041205:mem:mfl Add "break" and "continue" C statements. 20041212:mem:mfl Some code factoring in mfl-func and addition of some generic helper routines. 20041212:mem:mfl Create new module mfl-bif-str and move string- related built-in-functions out of mfl-func and into mfl-bif-str (reworked slightly to use some of the newer mfl-func functions). Rename 'substr()' to '$str_sub()'; Rename 'sx()' to '$str_debug()'; Rename '$_match()' to '$str_match()' (however, keep '$_match()' as a synonym for at least one alpha release in order to upgrade my scripts easily). Add '$str_byte()' 20041212:mem:mfl Remove built-in-function '$_envelope()' from the mfl-func module. Reincarnate it in the mfl-bif-msgpart module as '$msg_envelope()' with different semantics. Change from: BOOL $_envelope( destvar, thing ) to string $msg_envelope( thing ). Invalid or missing "thing" now return "" rather than FALSE. 20041212:mem:mfl Create new module mfl-bif-misc for as-yet- unclassified built-in-functions. Move '$_getenv()' and '$_parse()' from mfl-func into mfl-bif-misc. No built-in-functions now remain in mfl-func, which was a goal (all are now in mfl-bif-xxx modules). '$_getenv()' may have a short life in this module (and with that name)-- it is likely to go into an "environment" or similar module along with things like getting the date/time, username, homedir, etc. 20041212:mem:mfl Add preprocessor directives: #ifdef_func #ifndef_func #ifdef_var #ifndef_var #include_noerr 20041212:mem:mvmda Add constants SYSTEMRC and USERRC to mvmf.h; one to define a system-wide rc mfl file to interpret on startup, the other for a per-user rc file. Probably only one or the other should be used, as you can always use one of them to invoke others via #include_noerr. Also define USERRCENV as the name of an environment variable that will override user-level rc filename at runtime (i.e., user may always specify a rc filename by setting that environment variable.) 20041212:mem:mfl Add mfl_func_hook() to call a function if it is defined. The idea is to allow the user to define hook functions that may be called at certain events. 20041212:mem:misc Add stuff in the OMM struct to store "responsible sender ip"s -- the IP address of system(s) that have sent the email and that we can therefore test against DNSBLs. Add omm function omm_rip_add(), and add mfl built-in-function $msg_rip_add() to add an IP to this list of responsible IPs. 20041212:mem:misc Fork dnsbl_check() into dnsbl_check_ip() (to check a specific IP address) and dnsbl_check_omm() (to check the responsible IP addresses listed in an OMM). This breaks some of the result-caching that we had in there, which assumed there would be just a single IP address tested during the execution of the program, so that has been commented out for now until we can fix it (not really clear that we need to). 20041212:mem:mfl Change the sieve dnsbl statement and the :dnsbl command to use dnsbl_check_omm(). Will likely add an option to the sieve dnsbl statement to use specific IP(s) after working out a syntax. 20041212:mem:tool Also change the stools/dnsblc utility to use dnsbl_check_ip(). 20041212:mem:misc,mvmda Remove mf-hdrmisc module in mvmf/ and the hdr_prescan() function contained therein. hdr_prescan() was some some hardcoded C code to look for X-Peer-Info header lines in order to find the IP address that will be used for the "dnsbl" stuff. Remove calls to hdr_prescan() at some high-level places where the message is opened. Replace all this with a $hook_msg_read_after() hook call inside the omm module where the message is opened and read. Also, in mvmf.c, add a hardcoded definition of $hook_msg_read_after() mfl function that is only defined if it isn't found (e.g. in one of the rc files). All of this makes it possible to install site-local or user-local mfl code to parse IP addresses out of message headers rather than having this logic hardcoded into mvmf. 20041212:mem:mfl Change type_is_string() function to return TRUE *only* for string type. Prior to this it would return TRUE for a string type as well as a pointer to string. This is a remnant of earlier string stuff where we were trying to be a little more liberal to make up for a less complete implementation of strings. Right now this is conditionalized in case it breaks something. I doubt it will, and if it does I'll probably fix the breakage rather than revert to the old behaviour, but we'll see. 20041213:mem:rel mvmda 0.108.2 20041218:mem:mfl Add ":ip" option to dnsbl sieve statement. So: dnsbl :ip "127.0.0.2" "spamhaus" "xbl" or dnsbl :ip [(string)"127"+".2"] "spamhaus" "xbl" This form does not require that a mail message be open. 20041219:mem:mfl Add mfl-bif-control module, for built-in-functions relating to mvmf controls. The immediate purpose is to be able to give some admin control over who can use an upcoming pipe facility (i.e. "file" a message into a pipe) since one of the goals of mvmf was to give delivery capabilities to people without necessarily giving them access to the underlying system. So rather than just have a toggle that controls the ability to use a pipe, create this system of built-in functions that allow either the user or the admin to set control values. Control values have a class (e.g. type of value: integer etc). Each class has its own limit structure. Integer values have low and high bounds for the user and low and high bounds for the admin. The "admin" is simply MFL code that runs while admin mode is in effect. Admin mode is in effect for the system-wide mvmfrc file. Admin mode itself is an integer control under this system, so that an MFL script can revoke it via a control setting. At any rate, it's likely more work and thought remains for this. Add integer control value "admin". Add integer control value "pipe_allow". Also add mvmf module mvmf-controls. Add built-in-functions: $admin_int_set $admin_int_setlim $control_int_set $control_int_setlim All this to control a TRUE/FALSE value. 20041219:mem:mfl In mfl-sieve, in the "fileinto" command: if command starts with "|" it's considered to be a pipe request (to write the message to a shell command). Add function msg_pipe() in the msgstore module. This is crudely implemented with popen() at the moment: Will probably replace with a pipe() implementation at some point along with a long-planned plug-in capability (i.e. where the admin can set up some commonly-used plugins that can be used as pipes without having to allow general pipe access to everyone). Maybe the next weekend effort. 20041219:mem:rel mvmda 0.109.0 20041228:mem:misc Bugfix: the hdr_prescan() function that was removed at 20041212 used to set a flag From_F if a "From " line was seen. Lack of this flag setting is causing an extra "From " line to be written to mbox style storage. Fix. Detecting the "From " line when the message is being filed is more correct anyway: the header could conceivably have changed between the prescan and the write. 20041228:mem:rel mvmda 0.109.1 20041230:mem:rel It's been a year since the first limited offering; now incorporate the "new bsd" license. This constitutes a "new feature." mvmda 0.110.0 20050131:mem:mvmtr Begin work on mvmtr, the MTA receiver sandbox project. 20050131:mem:misc Create qmail-queue-if module. 20050203:mem:mvmf Do some more work on bounce processing. The whole bounce stuff is thorny: some situations do need bounces, yet bounces can be abusive (thus the whole bounce suppression logic that's been here since day one). I'd like to move some bounce suppression to MFL hooks, so I've made a weak start here. Doubtless this needs improvement. At any rate: Rename omm_errnote() to omm_delivery_error() and change its semantics slightly. omm_errnote was only used for delivery errors anyway, this makes it more special. Formerly, there was wrapup logic to generate bounces on some conditions (and to inhibit them according to various good practices). Now, whenever any delivery error happens, an MFL hook $hook_delivery_error() is called to decide what to do. This hook can request a message rejection, and further, it can stipulate that rejection ONLY happen at SMTP time. (An LMTP engine is planned RSN (real soon now), not to mention a compatible SMTP agent being sandboxed, so this is a small step in that direction). Now at wrapup time, the reject requests are potentially honored (again, subject to bounce-inhibition logic). Add $msg_reject() built-in-function. The idea is that $hook_delivery_error() may be able to decide which errors are important, and what particulars are important. For example, an over-quota mailbox might cause a bounce if it's newly over quota, or if it's sent from some particular address, but if it's been over quota for a month and not checked at all recently, over-quota conditions ought to be ignored. (I still need to provide MFL access to those parameters- but that's the general kind of idea.) OTOH I think that in the future, SMTP-time invocations will be predominant anyway. If the user doesn't supply a $hook_delivery_error function, a default one is provided. At this writing, the default one merely inhibits all non-SMTP-time rejection. 20050203:mem:misc In omm_delivery_error, log some stuff about the error. 20050203:mem:mvmda Bugfix: in mvmf_init, this test: if ( ( pwdP != NULL ) || ( pwdP->pw_uid == myuid ) ) should be if ( ( pwdP == NULL ) || ( pwdP->pw_uid != myuid ) ) 20050203:mem:mvmda Add string control type to mvmf-controls (see 20041219 edit). String controls have two properties: a string value, and a "may only be set in admin mode" flag. (I can imagine a string validation function propery as well but will cross that when needed, if ever). Add "$admin_string_set" MFL built-in-function. Add "$control_string_set" MFL built-in-function. Add string control "log_private_name" which is the name of the private(personal) logfile. 20050203:mem:misc Flesh out the log_priv code so that whenever a log_priv logging is done, and if the string control "log_private_name" is set, this name will be translated and used as the private log file. (i.e., log_priv() now finally does something.) If the source string starts with "~/" the homedir is substituted. There are no other special encodings as yet. Now, for example, user can put this in their .mvmfrc file: $control_string_set( "log_private_name", "~/mvmf.log" ); and log_priv lines will be written to that file. 20050203:mem:rel Push out to focus on some more complex things. mvmda 0.111.0 20050205:mem:tool Make mvmflogsum utility work with the "plog" private log format. (Basically just a matter of enabling it; it was already mostly there.) 20050208:mem:mvmf Move version.sh out of mvmf/ into misc/ and make it a bit more general so it can be used by multiple utilities. Likewise change the naming convention for the version.h file (now prog-version.h for some program 'prog'). 20050210:mem:mvmf Add ChangeLog preamble about versions. 20050210:mem:misc Rework the controls stuff slightly to make it available to multiple applications: - rename mvmf-controls to just "controls"; - move the module to the 'misc' area; - remove the controls data from the module and require that such data be published by each application instead. - notice that the "admin" control name was missing from the mvmf controls table; add it. Some control names are still required by some utility functions; i.e. all applications will need to define some certain minimum set of controls in order to use those functions. 20050210:mem:mvmda There was a global variable "OmmfP" that was only ever used in the ":rdmail" interactive command. Use a local variable instead. Also in :rdmail, if omm_openfp failed the file was not closed. Mostly harmless, but add fclose in that case. 20050211:mem:misc Minor edits to reflect changes in mml's mml-iopf stuff that adds some generality in iopf input. 20050212:mem:misc Since it works, move xMTP stuff out of mvmtr and into misc/xmtp where it was intended, so that it can be used by other applications (e.g. for the planned LMTP interface to mvmf). 20050213:mem:misc Change the msgpart container (MSGCON) so that the input can either be from a FILE (as before) or from an MML_IN stream. This to accomodate reading msgpart from an xMTP input stream (e.g. the DATA part of an SMTP/LMTP connection). Tempting to generalize it to require an MML_IN stream, but that would lose the ability to seek into the input FILE if the input is indeed seekable. MML_IN container input is always non-seekable and requires a temp file if the message can't be entirely buffered within the defined limits. 20050220:mem:mvmda Bugfix: if compile-time cpp constant QMAIL isn't defined, '-S' command-line option wasn't available. There are different getopt strings with and without QMAIL, and -S had been omitted from one of them. Also, usage() shouldn't mention '-q' if QMAIL isn't defined, so fix that. Noted by Bron Gondwana. 20050220:mem:mfl Bugfix: if in -S mode, eval_tree didn't know what to do with a PN_SVBIS parse node for a naked sieve statement. Make it call exec_sieve(). (Alternatively, I might want to treat any '-S' mode script as if it were a big compound statement, i.e., pretend there are curly braces around the entire script. I think this is better, but time may suggest otherwise). Noted by Bron Gondwana. 20050221:mem:mvmf Add djb's "cdb" lib to the distribution, as part of the mvmtr effort. Update LICENSE file and add a README.cdb . This is part of the mvmtr effort and will likely only stay if mvmtr is kept. 20050301:mem:mvmf Fix a bug in 20040616 ChangeLog entry... had incorrectly given an example using regex comparisons instead of wild comparisons. 20050305:mem:tool Create stools/cdbgen utility (for mvmtr stuff) 20050305:mem:rel Put a checkpoint so that I can make some more potentially disruptive changes. The next version will probably have some renames (so that 'mvmf' no longer means a specific program, but means the suite of tools instead). mvmda 0.112.0 20050310:mem:misc Bugfix: in msgpart_hdrtoken, variable cC keeps track of how many characters are in the source token being gathered, not in the result token. However the result token was being NUL-terminated with tokP[cC] = NUL; Change the code somewhat, remove this bug. 20050310:mem:misc Bugfix: misc/version.sh had an error in the 'gen' operation. 20050315:mem:mvmda Rename the 'mvmf' mail delivery agent program to 'mvmda' 20050315:mem:mvmf Many structural changes. This code tree was originally constructed to support a single application: the mail delivery agent (although the MFL part began as a separate thing). Many support routines "knew" that they were part of the delivery agent. I've started disentangling things to make them more separate; this involves changes to some include files, to various manifest constants so that names are more special purpose, removal of global state, etc. This is likely to be a job that is going on for quite a while. 20050410:mem:mvmf Reuben Farrelly sent me a compile log from a Fedora Core 4 system; use this to clean up a bunch of issues mainly relating to missing system-level include files. 20050410:mem:mvmf Similar to above: compile with extra warnings enabled and fix some more minor things, especially "unsigned char" vs "char" "conflicts" and various places where functions were used but not declared. 20050415:mem:mvmf Rework the ChangeLog to as part of the disentanglement noted at 20050315. Entries are now less version-specific and are tagged with a gross category of where the changes apply. 20050415:mem:mvmf Incorporate mvmtr into the kit. Nobody should want it at this stage, but it's easier for me just to leave it in. Fold mvmtr's ChangeLog into this one (just one entry so far..) 20050415:mem:mvmf Change the mvmf website to reflect the new package structure. 20050415:mem:rel Initial release w/ name changes (mvmda). mvmda 0.113.0 mvmtr 0.1.0 20050416:mem:mfl Bugfix: parse_bis_break and parse_bis_continue were returning an uninitialized variable "status" . Evidently the platforms I've used are zeroing the stack memory, and it's lucky (or perhaps not) that SCOK gets defined as 0. Remove the variable, just return SCOK 20050417:mem:mfl Bugfixes: The prior fix is an inspiration to check for other dependencies on SCOK being zero. Did an inspection, found: - in parse_decl_stmt, "status = 0" - in exec_sieve, "if status == 0" - in val_pmember, "if status == 0" In the same spirit, added another SCxxx in the enum definition, so that SCOK is not zero (could have used an explicit assignment to start the enums at other than zero, but I'm happy to have an SCNOTHING too. 20050417:mem:mfl Bugfixes: more fallout from SCxxx checking: val_isint() is defined to return BOOL, but actually returns STSCODE and all usages interpret the return value as STSCODE. Change to really return BOOL, and change the places where used to interpret the return as a BOOL. 20050417:mem:mfl A user discovered an artificial limit in the size of a literal quoted string (in particular, in a "reject" command, but it could have been anywhere). The reason: a literal string is just a token. The tokenizer stuff in mfl-token was at one time written to used fixed-size token buffers. It was changed at some point to use BSTR buffers, but they were allocated with the same maximum size as the prior fixed-sized buffers. This is dumb, because the main point of BSTRs is to support strings that can automatically grow. Change the allocation of token BSTRs to remove the old limit, and set up the token BSTRs with a much higher maximum. One downside here is that there is a pool of reused token structures. Each structure in the pool will never be removed, even if its BSTRs grow huge. Might want to think about discarding a token structure when it's returned to the free pool if its BSTRs sizes are above some threshold. 20050417:mem:rel Version bumps because of mfl fixes. mvmda 0.113.1 mvmtr 0.1.1 20050418:mem:misc Bugfix in version.sh -- a missing closing single quote. Odd that it didn't cause an error on systems here. 20050418:mem:mfl Bugfix: in mfl-pp, if a preprocessor directive is not recognized, the code attempted to skip the rest of the line by doing in_char() until '\n' seen. This would not recognize any comments on the line, so e.g. in: #foo /* multi-line comment here */ AA the comment stuff on the second line would generate an error too. Fix to use token input to skip to end of line. Note this causes 'AA' to be ignored, but that's probably correct. [Noticed during code inspection looking for bad SCxxx comparisons, per 20050417 entry] 20050418:mem:mfl Bugfix: similarly, and found the same way, if an #else transitions to a true state, anything on the line with the #else was not being handled. So: #ifdef NOTDEF .. some false stuff .. #else foo int i; #endif would cause an error interpreting "foo int i;". Fix to call ps-endline() when the #else transitions to true. 20050419:mem:mfl Bugfix: func_treearg_estr() was performing an expression evaluation, taking a pointer to the string result, and then releasing the evaluated value. Change the calling interface to return both the string and the evaluator value; make it the responsibility of the caller to release the value when done with the string. As with several prior bugs, found via compile on an FC4 platform. 20050426:mem:rel Version bumps because of mfl fixes, wanted to include more new stuff but got distracted. mvmda 0.113.2 mvmtr 0.1.2 20050430:mem:mfl Bugfix: bif_msg_envelope was setting the result string to "" if the evaluated result was non-NULL; the test should have been against NULL. The result was to always return an empty string. Bug apparently introduced in the 20041212 edit when the envelope function was reworked. 20050430:mem:mfl Relational comparisons between enums was never implemented. e.g.: enum {NONE, FWD, REV} dir = FWD; if ( dir == FWD ) ... would give 'Invalid arguments to "==" ' Add the enum case to val_common_type(); While there, rework val_common_type slightly: don't pre-init the result to NULL and replace it when a valid result is made, instead set the result to NULL only where it applies. 20050430:mem:mfl Philosophical change inspired by prior edit. Before this, enum value names were being installed as int types, and val_coerce() was allowing coercion between enums and various other types. So you could say: enum {B,W} bw; enum {F,R} dir; dir = B; bw = 3; with impunity. Add a compile-time constant "MFL_STRICT_ENUMS" that, when in effect, changes all of that. (The compile-time constant will likely be removed at some point.) enum value names are installed with their enum type, and no implicit coercion is allowed between an enum and any other type. Explicit typecast is still allowed. Some val_coerce() logic is changed to differentiate between forced and non-forced coercion. There's still some enum funkiness remaining, e.g. you can still do dir = B + B; as it would require more fundamental evaluator changes to fix that. Implicit conversion to string is still allowed, and enhanced by this edit as well: string s; dir = B; s = dir; value of 's' is now "B". (Can you tell I needed to use enums in a script?) 20050430:mem:mfl Bugfix: in typecast parsing, some status returns were not being checked. MFL code like this: i = (enum)3; would cause a warning but not propagate error up, resulting in attempt to execute a bad parse tree. Fork typecast parsing into its own routine, and do better at it. 20050526:mem:mvmtr Initial stab at mvmtr man page. What the hey. 20050628:mem:mvmf First pass at autoconfig (autoconfiscation). Improvement undoubtedly needed, but it's a start. 20050628:mem:rel Version bumps for release. mvmda 0.113.3 mvmtr 0.1.3 20050707:mem:mvmf Make misc/errwarn.c take its program name from a string control "Ctrl_log_prog_name" and add corresponding control named "log_program_name" to applications that use it. 20050731:mem:mfl Rework the way Sieve tagged options are parsed and handled. The prior code had a two stage check of option names that was unnecessary; it used static variables to store option values; it initialized all possible option information on every sieve statement whether relevant or not. 20050820:mem:misc Change function name "msg_bounceable" to "msg_autoreplyable" to better reflect its use; add a few more checks and tweak it a bit. 20050821:mem:misc Break out message origination stuff from being inline where it was needed (forward, bounce) and create msg_send_begin() and msg_send_end() 20050821:mem:mvmf Add string control "Ctrl_datadir" and corresponding control name "data_directory"; all applications must support this (if only to resolve linker references). This is the path to the directory where data files are stored (a la a ".rc" directory) 20050821:mem:mfl First pass at "vacation" Sieve extension. Still somewhat rudimentary, and requires ":from" tagged argument. If ":handle" is not given, a handle of "default" is used- not sure about the recommended behaviour of synthesizing a handle out of the various arguments. 20050821:mem:rel Version bumps mvmda 0.114.0 (new feature: vacation) mvmtr 0.1.4 (no news here) 20050822:mem:misc configure broke with "--disable-vacation"; evidently the first use of some macros establish some configure state, and when they are inside a false conditional (in this case looking for "dbm" libraries) that state never gets established. Rearrange some common header checking to be first (I had wanted it to be after some of the more non-standard elaborate checking so as not to waste users' time if that failed). Probably this is documented somewhere but I'd forgotten it already. There was also a missing VACATION conditional around the definition of the "vacation" verb in mfl-sieve.c . Also update the 'mvmdalogsum' utility to know about vacation log entries. 20050822:mem:rel Version bumps because of mfl-sieve conditional fix. mvmda 0.114.1 mvmtr 0.1.5 20050823:mem:mfl mfl lexical input: add '#' to begin a to-end-of-line comment. '#' was already used as the preprocessor introducer character at the beginning of a line (e.g. "#include"). Because of this conflict I've decided to phase in a different preprocessor introducer, namely '@' . Installers may now choose to keep either the old '#' introducer or the new one '@' . In order to make '#' somewhat compatible with the #-style comments, there's also a compile-time constant (and configuration switch --disable-preprocessor-invalid-warnings) to inhibit warnings about bad preprocessor directives, thus making a bad #-style preprocessor directive look like a comment. But the recommended mode will be to only use '@' for preprocessor. 20050823:mem:mfl bugfix: a missing argument to #include was not detected, resulting in bad memory access. (An assignment that was supposed to be inside of an 'if' block was past the block..) 20050823:mem:mfl Also in the preprocessor: change various places that call 'warning()' to use 'mfl_warning()' instead, the difference being that mfl_warning will report the line number. Dunno why this wasn't right. 20050824:mem:mfl Add Sieve's "text:" lexical input (not just in sieve mode but everywhere). 20050824:mem:mfl Change the capability name for mvmf's "dnsbl" sieve extension to be the vendor-specific name "vnd.mvmf.dnsbl" (a long-needed correction). Add a configuration switch to keep the old name as an alias. Also add code to support alias Sieve capability names. 20050824:mem:mfl Change the way tokens are pushed back (after a token lookahead). Formerly the raw input text was pushed back for re-intepretation. The new method pushes back the token so that the input text doesn't have to be reparsed. This is considered experimental and must be enabled via a configure switch. 20050824:mem:mfl Bugfix: the sieve "reject" action was checking for a duplicate "reject" action, but it was registering a "redirect" action (so the test for a duplicate would never succeed). Obviously a cut&paste error. 20050824:mem:mvmda,mfl Add "-n" option to mvmda, and corresponding changes to the mfl code, to report actions that would have been taken rather than actually take them. The intent is to help with script debugging. Suggested by Jon Salz. This implementation may yet be imperfect. 20050824:mem:rel Version bumps mvmda 0.115.0 mvmtr 0.1.6 20050920:mem:misc Edits to reflect changes in mml's string matching functions calling interface(s), as well as a bugfix there. 20050928:mem:mfl Add mfl-bif-db module for DB built-in-functions. Initial set contains $cdb_open, $cdb_get_string, $cdb_close, and $cdbu_get_string. 20050930:mem:rel Version bumps mvmda 0.116.0 mvmtr 0.1.7 20051001:mem:misc Update msg_autoreplyable() to add "listserv" and "majordomo" to the addresses that can't be autoreplied to (per sieve vacation draft). 20051002:mem:mfl Bugfix to mfl-bif-db already. The _get_string functions were supposed to return a string pointer, but I had them returning a string. This was hidden by default type promotion (e.g. when assigning the result to a string pointer variable). 20051002:mem:mfl Philosophical change resulting from prior edit. Earlier in mfl's life, string and string pointer types were made to operate very similarly, so a script writer could use strings and string pointers reasonably interchangeably. As part of this, an assignment to a string pointer would attempt to create a string out of the source value if possible. So for example assigning the int value '32' to a string pointer would just do a default type conversion, creating a string "32", using a free-standing pointer for assignment to the string pointer. This causes problems, for example trying to NULLify a string pointer by assigning a 0 to it (as in string *sp; sp = 0; ) would actually assign a pointer to a string "0"; Change this so that only source string values (and pointers to byte arrays) would do this string conversion. Explicit typecasts will still work where the explicit conversion used to, e.g.; string *sp; sp = (string)0; 20051002:mem:misc Create msgpart_write_references() function to output appropriate reference headers for replies; forked from code in vacation (and msg_bounce), and improve via creation of "References" header (formerly only "In-Reply-To" was created). 20051002:mem:misc Add some function(s) to construct a date string in rfc2822 format for use in message parts. Change msg_bounce() and sieve vacation code to use these for a Date header, rather than constructing it on the fly (and in fact msg_bounce wasn't adding a Date header, relying on the mail system to supply one). 20051002:mem:rel Version bumps, to get the db bugfix out. mvmda 0.117.0 mvmtr 0.1.8 20051025:mem:mvmf Some standard stuff in the configure script was wrongly inside an "enable sieve" conditional. (Not that anybody would have disabled sieve..) 20051129:mem:misc The variables holding envelope to/from strings are only set under some circumstances and inside some conditionally compiled code. However, the variables should be nul-terminated no matter what. Move the nulterm calls out to a point past compiled or run-time conditional execution. 20051201:mem:mfl Per request of davec, add mfl built-in functions $str_lower() and $str_upper() that copy & case-convert their string argument. I'd rather figure out some better string manipulation schemes-- these are hack functions to work around for that lack. Note that they assume ASCII input; these will have to be revisited for UTF-8 operation. 20051213:mem:misc Add misc/mvmfmisc and misc/authmisc modules. 20051213:mem:mvmtr Add SMTP AUTH with LOGIN and PLAIN methods, and other random work. 20051214:mem:mfl Add $msg_rip_n() and $msg_rip_nth(); add $ip_rev(). 20051214:mem:mfl Add mfl-bif-dns module with a few simple DNS functions. Pretty minimal, but they were things that I needed for a specific use. Still need to add $dnsbl -type functions to use dnsbl.conf definitions (needlessly hardwiring domain names in scripts is asking for trouble)-- will do that soonish. 20051214:mem:mfl Bugfix: add a counter to the free token list, and ensure that the list always has some number of free tokens on it. Tokens on this list are used in round-robin fashion, with mfl_tokreserve() used to temporarily take a token out of the list if it has a long-life need. The reserve/release depth can get higher than I had anticipated for some complex parsing (in conjunction with the newish FASTER_TOKBACK conditional code); this change helps to ensure that the round-robin use remains viable. 20051214:mem:misc Bugfix: somewhere along the line I had broken msgpart_hdrtoken, one result of which is that multipart messages weren't be recognized as such (and the MIME structure wasn't being assimilated). Fixed. 20051214:mem:mfl Bugfix: when FASTER_TOKBACK is enabled, mfl_token() has to recurse after it processes a preprocessor directive, rather than loop for next char; this is because the new style of token pushback may require grabbing the pushed back token at outer level, whereas it doesn't with the old style of pushing back the character stream. This problem is extremely unlikely, but it did crop up. 20051214:mem:rel Version bumps: mvmda 0.118.0 mvmtr 0.2.0 20051218:mem:mfl Bugfix: built-in-functions $cdb_get_string() and $cdbu_get_string() were attempting to set the returned string from a nul-terminated string, but should have been from a counted string. 20051218:mem:mfl New: can now add to a string pointer. A string pointer is a reference to a part of another string; adding to this pointer moves the reference window up or down. Fork some code to handle this. 20051218:mem:mfl There were two different code paths to handle addition to a pointer and subtraction from a pointer. These are really the same thing and should be handled with the same code. The only real point was to handle underflow on subtraction; who knows what I was thinking-- that needs to be done on addition too when the number being added is negative. Remove the subtraction code and make it call the addition code with the second term negated. Add underflow check to pointer addition operation. 20051219:mem:mfl Bugfix: release of pointer values wasn't always being handled correctly, resulting in some falsely persistent ephemeral objects. The implementation of string pointer addition helped show this. Didn't cause any execution errors, but with lots of string manipulation there could be lots of refstr chains created. 20051219:mem:mfl Bugfix(2): bug in built-in-function $str_debug (ironic, eh): in some cases a value could be released without being initialized. Which lead to finding that in func_treearg_string(), one of the error returns returned 'STATUS' instead of 'status' -- somewhere in the system include path, 'STATUS' was defined to be the same value as our SCERR. Ugh! 20051219:mem:mfl Added string functions: $msg_address() $str_bx() $str_bx_set() $str_count_control() $str_count_digit() $str_count_highbit() $str_count_lower() $str_count_upper() $str_find_token_delimited() $str_has_control() $str_has_digit() $str_has_highbit() $str_has_lower() $str_has_upper() $str_is_control() $str_is_digit() $str_is_highbit() $str_is_lower() $str_is_upper() $str_length() 20051221:mem:mfl Bugfix: "for" and "while" statements are supposed to return the value that bubbles up from the last execution of their loop, and care is taken to maintain that value properly. If no loop was executed, the value was never initialized. This could cause unpleasant side-effects at a higher level when the uninitialized value was eventually released. A very surprising old lurking bug. 20051221:mem:mfl Thinking about that last bug: the C-like statements are defined with a flag that controls whether they can be used as a term in an expression. Some of these are required to be on (e.g. 'sizeof') and some are required to be off. Others could go either way. I added a configuration-time option --enable-c-statement-terms to allow this to be set when the program is built. I suppose it could also be a run-time option. 20051221:mem:mfl Relatedly-- in C-like mode, add the ability for a compound statement ( {...} ) to be used as an expression term. Each statement inside the braces still has to have a trailing semicolon, so e.g.: x = 3 + { if (flag) 10; } + 4; Not sure anyone really needs to do this, but what the hey. 20051221:mem:mfl Bugfix: I don't know when this broke, but postincrement and postdecrement operations were returning the new value, not the old. e.g.; int x, y; x = 0; y = x++; y is supposed to be zero. It was 1. 20051223:mem:mfl Bugfix: reference to function name without calling function would coredump. e.g.: int f(){3;}; f; In val_ref(), add code to give "unimplemented" warning for TC_FUNC type. Some code had been conditionaled out... but put the case back in. 20051223:mem:mfl Bugfix: the way the preprocessor and tokenizer are intertwined caused problems when skipping false conditions. The inside of a false condition was skipped by looping getting tokens until the preprocessor state became true. That's a pretty bad idea, for example in: @ifdef OUT x = "blah; @endif the @endif is still inside the quoted string that is being tokenized. I never noticed this because my usage was always in commenting out already-good code. Add mfl_ppftoken() as a glue between the preprocessor's false body skipping and the requirements of the tokenizer. This should have the side-effect of being a lot faster. 20051223:mem:mfl Add functions $control_int_get() and $control_string_get() to complement the control-setting functions. 20051223:mem:mfl Bugfix: in pp_invoke, two bugs. First, if any argument token has a raw string, that should be used instead of the parsed string. Second, the arglen was being bumped by the bs_bL BSTR element, which is the buffer size; bs_bC, the number of bytes actually stored, should be used. 20051223:mem:rel Version bumps: mvmda 0.119.0 mvmtr 0.3.0 20051229:mem:misc Need to #include in msgpart.c 20051231:mem:files Update example mfl files to use '@' preprocessor introducer instead of '#' 20051231:mem:mfl Slightly better initializion of internal $HDRi$ struct to calculate size needed for pointer rather than assume it. 20060101:mem:mvmf Update copyrights for 2006 20060101:mem:cusp Add 'cusp' directory and configuration 20060101:mem:cusp Write clamdif utility for use as cusp 20060103:mem:mfl,cusp Add mfl-bif-cusp module, with an initial set of $cusp functions: $cusp_close() $cusp_define() $cusp_open() $cusp_read_line() $cusp_write_end() $cusp_write_message() $cusp_write_string() $cuspu_message() 20060103:mem:rel Version bumps: mvmda 0.120.0 mvmtr 0.4.0 20060104:mem:mvmf Configuration option --enable-faster-tokback is now the default. The option and old code should be removed.. 20060104:mem:mfl Bugfix: in mfl-parse module, parsing of "break" and "continue" statements was not requiring the trailing semicolon. Any supplied semicolon was being seen as another statement. This would mess up, for example, an if..else statement where the 'if' didn't use a compound body (no braces). 20060104:mem:mfl Add $msgpart_cur_body() function. 20060113:mem:mfl Rename $_parse() to $mfl_parse() 20060113:mem:mfl Add $mfl_exec_string() and $mfl_exec_string_sieve() 20060113:mem:tool Make 'cdbgen' recognize continuation lines in the input file. Involves a one-letter change to the source program (change fgetbtrec() call to fgetbtrecc()). 20060114:mem:mvmf In every application that doesn't use pthreads (currently everything except mvmcas) add a call to mml_fthread to keep mml code from dragging in the entire pthreads suite. The call itself doesn't do anything; the reference causes the linker to load the mml module that has some pthread stubs. 20060117:mem:mfl Change the preprocessor to have two separate path search lists for the different include types (system-wide and user-local). Also improve the @include parsing somewhat. 20060117:mem:mfl Add built-in-functions: $mfl_incdir_add() $env_cwd() $env_cwd_set() $env_homedir() $env_variable() 20060117:mem:mfl Rename built-in-function(s): $_getenv() --> $env_variable_assign() 20060117:mem:rel Version bumps: mvmda 0.121.0 mvmtr 0.5.0 20060118:mem:mfl,mvmf Remove the --enable-faster-tokback option; faster-tokback is now the standard. Remove FASTER_TOKBACK conditional code, remove old token pushback code altogether. 20060118:mem:mvmf Fix cut&paste date error in ChangeLog entry last :rel 20060118:mem:mfl Add code to define some preprocessor macros in MFL. Initial names are the $MEX_xxx exitcode names and the $QMEX_xxx qmail exitcode names if QMAIL is defined. 20060119:mem:mvmda New hook: $hook_exitcode(code) is called when mvmda exits, to return a desired exit code based on the passed internal code (which should be one of the $MEX_xxx values). 20060119:mem:mvmda Remove hardwired qmail-specific exit code translation code if QmailF (qmail mode, -q) is on. Replace with the instantiation of a hardwired $hook_exitcode() MFL function if QmailF is set and if the hook hasn't already been defined. This will preserve backward compatibility for those who don't define $hook_exitcode and who use qmail. 20060119:mem:mvmf Some embedded MFL code sequences (executed out of the C code) was using "#" for MFL preprocessor character, which would only work if configuration option --enable-preprocessor-via-hashmark was used. Have the configure script generate a constant for the optimum character to use, and use that character when needed in any mfl code executed from the C code. 20060121:mem:mfl Bugfix: in parsing an mfl function declaration, the function prototype is attached to the type reference. Parsing of a complex type (rather, something on the left-side of the type, like a pointer), involves recursively calling the type usage parser, and in this case the prototype info was not passed up to the caller correctly. So in: string *f(string x){ $str_sub(x,1,1); }; 'x' would not be recognized as a formal argument. This, however, would work: typedef string *sptr; sptr f(string x){ $str_sub(x,1,1); }; since there is no recursion in parsing the typedef reference when the function is declared. Fix by having the recursive caller bubble up the returned/attached function prototype when it calls itself in this situation. This handling of function prototype parsing is probably not ideal, but so far it's at least as ideal as any other... 20060122:mem:mvmf Change the way program "controls" are implemented. Formerly, each control had its own variable, and all controls were listed in a control name table defined by each application. Controls could be looked up by name or accessed directly via the individual variable. This allowed quick/easy access to the control values, but had the unfortunate side-effect of requiring each application to maintain a bunch of control variables just to satisfy linker references (because of generic functions that accessed some control variables directly). Now, controls are registered and accessed by name via the controls module; there are no tables/variables outside the scope of the controls module. Controls lookup may be a little slower, but code maintenance is easier. 20060122:mem:mfl,mvmda Add a string control "email_address_primary" that specifies the user's primary email address. In the sieve vacation statement, use this control's value as the default if no :from address is given. This address will also be used in the vacation address validation (i.e. it qualifies as a known address that must appear in one of the recipient header lines for vacation to execute). 20060126:mem:mvmf Rework the locking for storing new messages. Prior was hardcoded dotlocking and fcntl locks for mbox (dotlock under conditional) and no locking for maildir. Add stuff in configure to check for various locking types. Make locking in misc/msgstore.c consult controls for each mailbox type: "lock_mbox" etc, value is a comma-separated lock type, each done in the order given. This allows some control of locking by the installer and the user. Anticipate extending the syntax of this value string at some point (e.g. for alternate lock file, # of retries and so forth). 20060127:mem:rel Version bumps: mvmda 0.122.0 mvmtr 0.6.0 20060206:mem:mfl Bug in executing 'sizeof' built-in. sizeof worked when given a type as argument, but when given an expression it evaluated the expression and returned the result: int i = -1; sizeof(i); // would give -1 Fix to evaluate and return the size of any expression given. NB: because of the way the interpreter works, it's necessary to evaluate the expression to figure out its size. Thus, sizeof expressions can have side effects: int i = 0; sizeof(i = 3); sets i to 3. (OTOH I dunno if any script will ever need sizeof.) 20060215:mem:mvmf Minor changes to accomodate the removal of IP4 type from mml. 20060220:mem:mvmf Fix some incorrect uses of ARGLIST macro; would have cause problems if defined for non-ANSI mode. Non-ANSI mode definitions probably wouldn't work here and there for other reasons anyway; foolish consistencies are at play here. 20060220:mem:misc Some functions in errwarn module were doing some very strange manipulations of stdarg/vararg handles in order to pass a varargs list from one function to another. This was a "what was I thinking of?" moment- after all, that's what varargs supports directly. Thought of while doing something similar (and straightforward) in mml. 20060225:mem:mvmf Replace uses of atol() to mml's atod() . Mainly for consistency, but this will enable recognition of leading sign and skipping of leading spaces on systems whose atol() doesn't do this. If there are any. 20060227:mem:mvmda There was conditional code based on a macro named SYSTEMRC to enable the use of system-wide mfl rc file. Problem was, the real macro name as set in mvmda.h is MVMDA_SYSRC . Make it right. Also the MFL 'include' directive generated inside this conditionaled code wasn't right: fix to use dynamically calculated preprocessor char (see 20051231 edit), and fix to use "" rather than <> construct. 20060227:mem:mfl Add built-in-function $str_supe() 20060228:mem:mfl Add built-in-functions $str_union(), $str_find(), $str_findi(), $str_find_byte(), $str_findi_byte(). 20060308:mem:mvmda Initialize SIGPIPE to ignore the signal; the default is to terminate the process, which is rather rude. 20060316:mem:misc Bugfix: msgpart_hdrtoken() is supposed to, among other things, return an RFC2045-compliant token. It wasn't doing this very well. Add a table of character attributes and use this table when validating token characters. 20060316:mem:misc Improve hdr_name_skip function(s) to use the character table mentioned in previous entry. 20060317:mem:misc While looking at msgpart.. move code that checks for MIME boundary into its own function, and use this function in the places where the checks are done. 20060317:mem:mvmf Some minor updates to accomodate mml's new policy that for refstr rs_bX, -1 now means uninitialized. Of course, that policy is to accomodate the next entry: 20060317:mem:mfl Change $str_findxxx functions so that reverse searches are from rs_bX to the beginning of the string, and so that they leave rs_bX behind the found string (now successive calls will do the reasonable thing, just like for forward finds) Also add rs_bX output to $str_debug(). 20060317:mem:rel The tokenizing bug (20060316) warrants a push. Version bumps: mvmda 0.123.0, mvmtr 0.7.0, clamdif 0.1.1 20060321:mem:mfl Bugfix: In mfl_func_exec, which invokes a user-defined MFL function, the values in the argument frame are initialized with the actual arguments. However, the values in the frame were never being released when the function returned. While this would not cause any execution errors, it could result in some falsely persistent values (e.g. if the passed values are strings). 20060327:mem:mvmf Bugfix: a few edits ago had added some calls to mml's ip-to-bytearray (and the opposite) functions without remembering that they use network byte order in the IP address. This broke most of the DNS stuff including dnsbl. 20060329:mem:rel Forced by DNS bug. mvmda 0.123.1, mvmtr 0.8.0 20060330:mem:mvmf Minor changes in some controls (cntrl_xxx()) function interfaces. 20060404:mem:misc Bug/Funcfix: prior to this, any 'epilog' in a multipart MIME message (the epilog is anything following the terminating MIME boundary) was not being accepted and therefore not stored. This functionality was just never written (and there were comments in the code to that effect). This edit addresses that: the epilog is now accepted and preserved. This also makes it a lot more likely that any messages that are MIME-broken will at least be entirely accepted. 20060405:mem:misc Add another data structure layer to mail message handling. Formerly there was a message container object (representing e.g. the folder the message is in, or the stream it is received on), messageparts linked to represent the MIME tree, and an OMM (open mail message) for the message that is being worked on. Add an MMSG to contain info about a particular message, linking it to the top messagepart. This provides a place to keep context about a message. 20060405:mem:misc Be smarter about quoting of "From " lines in mail message body. "From " quoting is the insertion of ">" before some "From " lines; this is needed in an mbox folder in some cases. Prior to this edit, "From " quoting was *always* being done, even if sending to a pipe or simply sending a message. Now, "From " quoting is only done when writing to an mbox. (Could potentially add one or more controls on this behaviour.) 20060405:mem:misc Similar to above, be smarter about terminating the message with a blank line. When writing to an mbox, one must guarantee that a message has a blank line to separate it from the message that will come next. Now, only add this blank line when writing to mbox folder. 20060405:mem:mvmda Bugfix: there was a hack to force qmail-mode if the default storage type was specified as maildir. Stop that, it was wrong. The qmail-mode flag has almost stopped being relevant anyway: its only purpose is to instantiate a default "$hook_exitcode" MFL hook function if one hasn't been supplied in a start/rc file. This mode will probably be removed in the future, and the "-q" option either be made into a no-op or removed. 20060406:mem:misc Improvements in handling the maximum bodycache parameter. Much easier (and possible) with the MMSG structure to keep this sort of context. 20060409:mem:mvmda Remove the old :dnsbl command, which was really only supposed to be a stopgap while the MFL stuff was being developed. Other :xxx commands are probably ripe for removal as well. 20060410:mem:mvmtr Much unrecorded development work up to now (no real need since it was slowly progressing to a usable state). It's now reasonably ready for alpha use; big changes past this point will be logged. 20060410:mem:rel mvmda 0.124.0, mvmtr 0.9.0 20060412:mem:mvmtr Bugfix: improper initialization and later checking of ommP in DATA command in an abnormal case. 20060414:mem:misc,mfl Bugfix: sieve "size" test had been broken. Make it work again. It is now relevant to the currently selected message part (which by default is the entire message). 20060414:mem:mfl Add new functions: $msgpart_epilog() $msgpart_cur_size_body() $msgpart_cur_size_hdr() 20060415:mem:mvmf Some modules maintained free lists of allocated/ returned memory, for rapid re-allocation of the same kind of thing over and over. Replace some of this with use of mml's erualloc and derualloc (allocate/reuse). Some private lists have been left alone, but these have been removed: misc/xmtp's XMTPDSS list; and some stuff in mvmcas (which doesn't get detailed change logs yet). 20060417:mem:mfl Add new functions: $str_quote_regex() $str_quote_wild() to return a new string with special characters quoted for use in regex or wild(matches) matching. These may need future embellishment to add more specials. 20060420:mem:mfl In mfl-sieve, in the "header" test, if :count is used, make it return the actual header count as its value if the test returns true and the count is non-zero. This is kind of a whim, but it didn't really cost anything, so now one can say: x = sieve { header :count "ne" "Received" "0" }; and x will be the number of Received lines instead of just 1 as before. 20060420:mem:mfl Bugfix: the 20060404 edit had introduced a couple of bugs when reading large attachments: - when the input is seekable, the wrong test was being done to see if the entire input should be skipped (it was not using the fancy new flag that was introduced); - when the body or epilog string could not fit entirely in memory, the beginning of the non-cached area on disk wasn't always being set correctly; and in another separate instance, the end of the non-cached area wasn't being set correctly if there was a MIME boundary string involved. These required some special circumstances to be triggered, but not really special enough. 20060420:mem:rel mvmda 0.124.1, mvmtr 0.9.1 needed because of stupid MIME bug above. 20060425:mem:mvmtr Change control name "smtp_server_rcpt_badmax" to "smtp_server_badrcpt_max" .. 20060425:mem:mvmtr Add control "smtp_server_badrcpt_delay". 20060426:mem:cusp,tool Initial stabs at various man pages that I had neglected to do earlier: clamdif, cdbgen, dnsblc, mvmdalogsum 20060503:mem:misc Add stuff to treat the body of message/rfc822 message part as an implied child. IMAP does this; RFC2046 sort of hints at it, maybe; at any rate it's a reasonable leap of logic that I hadn't made until now. This will allow easier MFL script handling of message/rfc822 parts (e.g. ARF reports). It's not clear whether to apply this to other message/* types; certainly there are others where the body contains RFC822-style headers that could be treated as a child. But they don't all. 20060503:mem:misc Bugfix, sort of: msgpart_write_body() was being too smart. It wants to guarantee that there is at least one blank line following the header, but was trying to intertwine this logic with writing any body/prolog string. If the prolog string itself ended with (or, in a typical case, was entirely) one or more blank lines, then one of those blank lines was being suppressed. This could end up removing a blank line if there are extra blank lines between the header and the first multipart part. Probably not a huge deal, but it's nice if the output is the same as the input (when the input is valid). 20060503:mem:misc Bugfix: in msgpart_read_headers() a test to check for an mbox-style "From " line had an error, such that this sort of From_ line would have been accepted anywhere in the header instead of just the first line. 20060503:mime:misc Make sure the mime type/subtype are defaulted if missing. Among other things, this allows really bad input, like text with no headers (e.g. an MFL script, if you make a typo and feed one to mvmda, as I did) to be processed better. A more likely case is a blob of junk fed to mvmtr's DATA phase. 20060503:mime:misc When writing a message part, if there are no header fields at all where one is required (e.g. writing the outermost part, or a message/rfc822 part), supply one. The supplied one is dumb; there is probably room for improvement. 20060504:mem:mfl Bugfix: exec_mfl_str(), which parses and executes an MFL string at runtime, would sometimes see a pushed-back token from an outer parsing context. This mainly happens when a hook function is called as a side-effect of some other script action. Fix (for now anyway) by protecting the pushed-back token list around the exec_mfl_str() call. 20060505:mem:misc In msgstore module, the msg_end functions were calling msg_abort on some errors; this really needs to be left to the caller so that the caller can generalize the abort handling no matter where the error happened (e.g., prior to msg_end or during it) -- also so the errno doesn't get obscured. Looks like this was something that I had "improved" recently. (This is unlikely to have affected anyone; it would only apply to errors on close.) 20060505:mem:misc Bugfix: The 'mc_prereadC' element of the message container struct (MSGCON), which is the length of any line that should be re-used (i.e. when a line has been peeked at), was erroneously declared as a BOOL, aka a char, which overflows after 127. How this hasn't been noticed 'til now? It is mostly used when peeking at the first MIME child header line or a MIME boundary line; those lines are almost always reasonably short. New handling of implied children (for message/rfc822) exposed it to longer lines. 20060506:mem:mfl An attempt to use a pointer or a string pointer as a string would give a warning and an error result (prompting the application to exit). e.g.: string s, *sp; s = sp; While this is bad MFL code, it shouldn't exit. (Whether certain runtime errors should cause exit is probably debatable; maybe should be subject to some control parameter.) 20060506:mem:misc Fix xmtp's dot-unstuffing in the DATA dialog (long on the todo list); it was removing a period at the beginning of the line only if followed by another dot; supposed to remove no matter what follows. 20060506:mem:mvmtr Change the handling of the recipient list. Each unique recipient address is remembered and all RCPT-TO mentions of that address refer to the single copy. This was the original intent; the interim implementation was a quicker implementation (although it turned out not really to have been any simpler) to get the application up and running. 20060506:mem:mvmf Update various Makefile.am file that haven't already been done to load the calculated pthreads library before mml. This helps to satisfy real pthreads references for any application that uses pthreads, and use mml's fake pthreads stubs for those that don't. 20060507:mem:misc As more controls are being used, change the storage from a linearly-searched sorted table to an AVL tree of control names. Not a huge deal, mainly the product of tinkering. Also add general functions to define controls out of a table, and to walk the controls tree, e.g. so an application may print its control values. Also, remove CTL_BOOL type, as it has never been used. 20060507:mem:misc Bugfix: control_int_get was returning an int and accepting an int as a default, even though int controls are long. cntrl_define_int was similarly in error. (Other functions have it right...) 20060507:mem:mvmda Change control initialization from inline code to table-driven, to go with the previous edit. Add '-V' flag to print version and controls in an MFL-compatible format. 20060507:mem:mvmtr Ditto for mvmtr. 20060507:mem:mvmtr Add "smtp_server_strict_sessions" control: when enabled (default), adheres to RFCs regarding completing a session even when one would prefer not to (i.e. keep doing the dialog until client QUITs or disconnects). It's fairly common these days not to do this; this can be set by the admin (and it can be adjusted by hooks if desired to alter the behaviour at any time). Also add "smtp_server_ss_helo" to augment the strict sessions control for the HELO command; even if strict sessions are otherwise wanted, sometimes one wants to drop after a bad helo. 20060507:mem:mvmtr Add greylisting support, which is mostly triggered via an mvmcas message. 20060508:mem:mvmtr Add smtp_server_msgsize_max control, and implement the size limit. (also in misc/xmtp module) 20060510:mem:rel mvmda 0.125.0, mvmtr.0.10.0 20060518:mem:misc Cleanup in misc/xmtp.c xmtp_getmsg result handling. 20060521:mem:mfl Bugfix: @include (et al) was not checking the preprocessor falseness level, and would include a file even when in a false state. Bug probably introduced as part of an edit around 20051223; amazing that this went this long without being fixed. 20060522:mem:mvmtr In mvmcas_check, attempt to detect UDP message that isn't from mvmcas, but is a late reply from a former DNS request (possibly from a different program that exec'd us). Currently just check for a reply from port 53, but could probably also check explicitly for a reply from mvmcas port. 20060526:mem:mfl In mfl_misc module, add tables to define mfl- specific controls. Initial controls are "mfl_trace_functions" and "mfl_trace_bif". In mfl_init(), add the initialization of controls via these tables. 20060526:mem:mfl Add Mfl_traceF to globally enable any MFL tracing that is further individually enabled/disabled via controls. Add built-in-function $mfl_trace() to set/unset this flag (it might be neater to do this via a control, but I don't want to poll a MFL control all the time internally, and I can't set a global flag from a control since controls don't (yet) have side effects.) Add tracing of MFL function calls enabled via the "mfl_trace_functions" control, and add tracing of MFL built-in-functions enabled via the "mfl_trace_bif" control. Tracing goes into the private logfile. 20060526:mem:mvmda Add call to log_begin() in mvmda_interactive, i.e. enable logging when run in interactive mode. Why not.. 20060603:mem:mfl Add "C" statement for C mode, and "sieve" statement for sieve mode. It's nice to be able to get into a particular mode without having to know what mode you were in, particularly for support files that ought to be @include-able in any context or mode. (One from the to-do list, ta-da.) 20060609:mem:mvmf Edits and updates to acommodate interface changes in mml's input stream stuff- the point of which is to enable better error handling for streamed input, not to mention a better way of finalizing streams (via a "finish" handler). 20060619:mem:mvmtr Don't add Received (et al) headers until we know we are going to accept the message. Among other things, this lets the $hook_data() see the raw input message before we tinker with it. 20060629:mem:mfl Bugfix: vacation [Bug report from Alastair Johnson]. In exec_sieve_get_bisargs, which collects tagged options for a sieve statement, a variable 'i' was being reused for collecting :addresses values, but 'i' was already in use by an outer loop. ick. Remove variable 'i' altogether, and use better names for the variables used. (probably a good use for a loop-local variable.) 20060629:mem:rel mvmda 0.126.0, mvmtr.0.11.0 20060630:mem:misc The 20060406 edit (adding MMSG) broke the msg_bounce() function, by adding the correct code in a slightly wrong place. Core dumps could result when using 'reject'. 20060630:mem:rel mvmda 0.127.0, mvmtr.0.12.0 Required for this bug fixed today. 20060718:mem:misc Bugfix: in cntrl_define_str, if no initial value was supplied, value pointer was left uninitialized. Likely to affect almost everyone not using a memory-clearing memory allocator. [Bug report from Alastair Johnson.] 20060718:mem:rel mvmda 0.127.1, mvmtr.0.12.1 Required for the important bugfix above. 20060719:mem:mfl Bugfix: in mfl-sieve-vacation, parsing of the "vacation" statement was not requiring the terminating semicolon. This would cause syntax errors in sieve-only mode, where naked semicolons are not being ignored (this needs fixing, too-- naked semicolons are not necessarily legit but I like to allow them). [Bug report from Alastair Johnson.] 20060810:mem:mfl Make the 20060504 fix slightly less hackish by adding (and using) functions to preserve the tokback state. This still exposes more detail to callers than I'd like. 20060810:mem:mfl Get rid of the AITYPE typedef in mfl.h, which was a remnant of some really old code and was only used in pointer subtraction (eval_minus()). Update eval_minus() not to use it. 20060810:mem:mfl Get rid of constant VDATSIZ in mfl.h, which was no longer used anywhere. 20060814:mem:mvmf The following edits tagged 20060814 are related, done over a period of about 10 days; an initial set of edits relating to startup processing was done; a new approach was inspired, previous edits yanked out, and these put in their place. The overall goal is for mvmda to cede a lot of its control flow to the system-wide rc file. These changes related to mvmda startup processing were based on some mailing list discussions with Nathan Angelacos and Alastair Johnson. 20060814:mem:mvmda Fork init code into multiple functions; move some of it into startup init, others into later init. 20060814:mem:mvmda Do not require any script name to be given on the command line. Do not complain if more than one script name is given on the command line. 20060814:mem:mfl,mvmda Add mfl_program_args() to supply argc/argv/argn as MFL variables $ArgC/$ArgV/$ArgN. Call from mvmda. 20060814:mem:mvmda Add '-e' and '-E' command line options to supply two different lists of strings to MFL arrays. '-e' strings are placed in $Opts_e[] array, with string count in $Opts_eC and array length in $Opts_eL. Similarly, '-E' goes into $Opts_E[], $Opts_EC, and $Opts_EL. 20060814:mem:mfl Rename support functions mfl_exec_string() and mfl_exec_string_sieve() functions to mfl_run_string() and mfl_run_string_sieve(). Ditto the corresponding built-in function names. I'd tried to keep "exec" to mean "execute already-compiled code" and these are not meant to have that meaning. This may cause some scripts to have to change, but better to nip this in the bud. 20060814:mem:mfl Add support routines mfl_run_fname() and mfl_run_fp(). 20060814:mem:mfl Add built-in-function $mfl_run_fname(). 20060814:mem:mfl Add modes string argument to MFL built-in-function $mfl_run_string(), a la $mfl_run_fname(). Remove built-in-function $mfl_run_string_sieve() as superfluous. 20060814:mem:mfl Add built-in-function $mfl_protect_admin() 20060814:mem:misc Since I'm about to have another directory path (in addition to the include paths), add a function path_dir_add() to potentially add a directory to a linked list of directory names, doing de-duplication. 20060814:mem:mfl Add hook directory path stuff. Data + functions mfl_func_hook_dir_add() and mfl_func_hook_find(). 20060814:mem:mfl Modify mfl_func_hook() to call mfl_func_hook_find(). 20060814:mem:mfl Add built-in function $mfl_hookdir_add(). Requires admin capability. 20060814:mem:mvmda Remove hardcoded hooks from the source code. All desired hooks now need to be provided externally. 20060814:mem:mfl Add pp_init() to initialize the preprocessor module; call it from mfl_init(). 20060814:mem:mfl Modify preprocessor paths to use linked lists for directory paths (rather than reallocated arrays of pointers); use path_dir_add() per above. 20060814:mem:mfl Add built-in function $msg_open_fname() 20060814:mem:mvmda Remove opening of incoming message; defer this to the system-wide rc script. 20060814:mem:mvmda Remove -b command-line argument, which specified body cache size. 20060814:mem:mvmda Remove the ":rdmail" command for interactive use; user can use $msg_open_fname(). 20060814:mem:mvmda Publish the operating mode as a value for MFL string variable $OPMODE, one of "CHECK", "CONTROLS", "INTERACTIVE", and "NORMAL". 20060814:mem:mvmda Remove a lot of the hardwired control flow, especially for "check" and "normal" modes; it's now deferred to the system-wide rc file. 20060814:mem:mvmda Publish MVMF_LIBDIR and MVMF_SYSDIR environment variables. 20060814:mem:mvmda Remove the initialization of preprocessor search path(s); defer to the system-wide rc file. 20060814:mem:mvmda Change default location of system-wide rc file to be in $MVMF_LIBDIR/mvmda/ rather than in $MVMF_LIBDIR/ . This is more proper, plus with new system-wide rc functionality this will make migration easier. 20060814:mem:mvmda Remove support for user-level RC file. Admin can do this in system-wide RC file if so. 20060814:mem:mvmda Add configuration --enable-mvmda-builtin-sysrc ; if enabled, if there is no system-wide rc file available, supply a hard-coded script (inside the mvmda code). Otherwise, fail with an error. 20060818:mem:mvmda Add mvmda-sysrc.5 man page. Also do other documentation updates to go along with the above suite of control-flow changes etc. 20060819:mem:mfl Function $mfl_incdir_add() was not interpreting its path argument correctly; it was using the string through its NUL terminator. Since it could easily be passed a substring, it needs to pay attention to the string length. Also change the interface to pp_incdir() to use an ESTR rather than a char *. 20060819:mem:mfl Previous edit prompted an audit of other built-in functions to see if they are all properly handling string arguments, and some of them did need fixing. This also led to changes to some of the internal function interfaces (e.g. the controls interface) to work with ESTR encapsulated strings rather than nul-terminated strings. 20060820:mem:mfl Add control "mfl_mode_sieve" -- whether to start parsing input streams as sieve (vs C-like). Use this value to set the mode in mfl_run_xxx() if it hasn't been explicitly set. 20060820:mem:mvmda,mvmtr Remove global SieveF, there is no use for it. '-S' option in mvmda command line now sets the "mfl_mode_sieve" control. Remove '-S' flag from mvmtr command line. 20060820:mem:mvmda Add mvmda-mfl-bif module for MFL built-in functions for mvmda only. Add built-in function: $mvmda_run_fname() Doubtful that anyone but me will want this; it's to provide $mfl_run_fname() -like service on a script file that might have very old style mvmda commands in it. Anyone who started with an mvmda that had MFL support in it would not be using those old commands in script files. 20060821:mem:mvmda Make -V print the location of the system-wide rc file, and whether or not built-in rc support is enabled. 20060821:mem:rel mvmda 0.128.0, mvmtr.0.13.0 20061022:mem:mvmtr In smtpd, "NOOP" is supposed to allow (and ignore) text after the command. We were giving a syntax error if there was text. Repair. 20061109:mem:mfl Do the same for sieve "address" test as we did for the "header" test in the 20060420 edit. i.e., if a :count test returns true, return the actual count. x = sieve { address :count "gt" ["to", "cc" ] "0" } and 'x' will have the address count. 20061111:mem:misc Considerable work on message store semantics: a first pass at generalizing the message store back-end, and simultaneously at adding namespaces. Formerly, message storing operated on a single set of folder information parameters, including folder type, layout type, and folder root path. The msgstore module contained hardwired code to know about mbox and maildir types. Now, the storage types are in their own modules. Each specific type makes reference to a general type in which the specific type belongs (e.g. "mbox" within "file" types), however all manipulation is via the specific type handlers (which in turn may call upon the general type handlers). All folder name references now go though namespace(s). When a folder is written, its name is used to find a namespace which then specifies how the folder is to be handled. Namespaces are defined via MFL functions. A user (or administrator) must define at least one namespace in order to write to a folder. 20061111:mem:mvmf Tweak various things to adjust to the previous edit; some code, plus example startup scripts and the like. 20061111:mem:misc While at it, update the maildir filename format to conform to newer (post-2003) guidelines. 20061111:mem:mvmda The -l (layout), -r (root), and -s (storage type) options are now all obsolete, as their parameters are now associated with storage handlers and namespaces. These mvmda options now do nothing; in the future they will be removed. 20061111:mem:mvmda,mvmtr Remove the "lock_mbox" and "lock_maildir" controls; with namespaces these are no longer relevant. 20061111:mem:mvmda If not specified, the standard mailbox (folder) name now defaults to just "INBOX" . The administrator or user should define a namespace to cover this name. 20061111:mem:mvmf Add the -b option to mvmda to force the use of a built-in script instead of the system-wide mfl script. This option must be enabled by new option --mvmda-builtin-sysrc-forceable at configuration time. 20061111:mem:mvmda In -V mode, and if there's a built-in sysrc, print the script to stdout if another -V is given (i.e., mvmda -VV). 20061113:mem:mvmda Add the -d flag; extend the user-related initialization to take this into account, including becoming the specified user if running as root, and verifying that the user that mvmda is running as is the same as the one specified. Also publish the USER environment variable and make sure it correctly represents the user running. 20061113:mem:misc When writing a "From " line to mbox, a return address must be used. Up until this edit, the address used was the envelope sender supplied by the MTA. This was only available if running in qmail mode, and then only if directly invoked from the qmail delivery process; in other cases when this address wasn't available we used "mailer-daemon". We also had available the "ascertained envelope sender address" which was calculated by omm_learn() (i.e. from the Return-Path header field). So now if the envelope sender address isn't available, use the ascertained envelope sender if that's present. 20061113:mem:rel mvmda 0.129.0, mvmtr.0.14.0 20061115:mem:mfl All sieve language tokens are supposed to be case insensitive. Somehow the command tokens have always only been accepted in lowercase (although all other token comparison in sieve mode, such as :options, was case-insensitive). Fix. 20061118:mem:mfl Bugfix, one from the FLAWS file. pre-increment and pre-decrement on a string value took a copy of a released value, which caused havoc. e.g.: string s = "foo"; s++; // worked, yielded "foo1" (s = s + 1) ++s; // caused a problem. 20061206:mem:mvmtr Code (in get_local_ip_addrs()) that tried to find all local IP addresses was too fancy, and tried to use if_nameindex() if available. Unfortunately that doesn't really help, and in fact hinders, so rip that part out and fix the other code path to be a little better. That other code path seems to handle known environments just fine anyway. Also fix up the #includes a bit in that module for better portability. 20061208:mem:mfl In Sieve capability names (in "require"), case is supposed to matter. We were doing case- sensitive compare of capability names. Not a big deal, but fixed. 20061208:mem:misc In xmtp.c, clean up some cases in parsing of mailbox names: - parsing of localpart when localpart was a quoted string was broken; - parsing of "atom" (in localpart) needs to allow for backslash quoting of any non-control US-ASCII char; - change both atom and quoted-string parsing to use the Mchartbl[] character attributes table from msgpart.c, add some attribute bits to help. 20061211:mem:mvmtr Change the interface to the smtpd dialog hooks so that the hooks can return more and better info. 20061211:mem:misc Bug in controls.c : the key comparison callback routine for the control name AVL tree would return a match if the key was a substring of the comparison node. It hadn't taken into account that one string was a counted string, and the other was NUL-terminated. 20061212:mem:mfl $env_variable_assign() is supposed to return 0 if variable doesn't exist, 1 otherwise. It was returning 0 if variable existed and was empty. Fix. 20061214:mem:mvmtr Add various built-in helo checking to supplement the $hook_helo() capability. I think it's worth having some built-in checks for common cases. New string control smtp_server_helo_check can contain a list of space-separated built-in checks. The initial set of checks includes "auth", "dns", "dnsclient", "dynamic", and "syntax". New integer control smtp_server_helo_check_defer controls whether the HELO check should be done at HELO time, or deferred until MAIL FROM time. Dynamic check also adds new hook $hook_dynamic_ip . 20061218:mem:mvmda A little more re setting up user environment on program start. If root and there's a non-zero process group id, add that group to the list of groups when setting up the group list. This part of supporting setgid installation. 20061218:mem:misc Add sanitize_process function to help set up the process environment (e.g. for fork/exec). Among other things, make sure the group list is sane, and that the running user matches the defined user at startup. Add calls to this function in places where the program forks/execs. 20061218:mem:misc Add pexec_std() to fork/exec a child program and set up pipes, sanitize the environment, etc. Mostly this steals code from the mfl $cusp_open stuff. 20061218:mem:mfl Make the mfl $cusp_open stuff use pexec_std(). 20061227:mem:mvmtr Change the deferral mode of helo and connect checks so that they are deferred to RCPT TO time, rather than the initial stab's deferral to MAIL FROM time. RCPT TO makes more sense as it enables more per-user things (and hopefully will hook into individual filters at some point). 20061227:mem:mvmtr Add support for "PASS" (free pass from some checks) from mvmcas. Also support "pass" in addrmap. 20070101:mem:mvmf Update copyrights for 2007. 20070103:mem:misc,mfl Add 'tailhook' element of namespace, specifying an MFL hook to use to edit the tail portion of the foldername after namespace prefix stripping. Add mfl built-in-function $msns_tailhook() to set the tail hook. This is a hack that would be better addressed by lamda expressions in MFL, but since we don't have that, we do this. If lambda is added to mfl, this may be removed. 20070108:mem:mvmf Rearrange the "files" subdirectory. Add some things such as sendmail mailer definitions, other mvmda scripts, mvmtr scripts. 20070108:mem:rel mvmda 0.130.0, mvmtr.0.15.0 20070325:mem:mvmtr Misc stuff to support connection-status feedback to (and from) mvmcas. 20070327:mem:mvmtr Change the connect_check functions and control names to be client_check instead, since that's more proper. client_check *might* be done at connect time, but it also might be deferred. 20070327:mem:mvmtr Add connect_check function that really does a connect-time check. Add $hook_connect() MFL hook processing. 20070327:mem:mvtr When doing connect-time rejection, if the client gets an SMTP response use an 554 code instead of 421. No sense deferring, the client should get the immediate reject. 20070328:mem:mvmtr check_helo() and check_client() each have a loop that looks up each check string in a table; the loop should break when the string is found, otherwise it wastes time looking at the rest of the table after processing the string. No major harm there, but fix. 20070328:mem:mvmtr generic_ip_check() should never return MDV_ACCEPT, as that is unconditional accept. This could inhibit, for example, some built-in HELO checks. Fix the one place where it did to return MDV_DUNNO. 20070328:mem:mvmtr Update check_helo() to set a meaningful SMTP reply string when HELO name matches our IP. 20070329:mem:mvmtr In DATA command, if no valid recipients, we were returning a 503 code, but 554 is more proper. 20070330:mem:mvmtr Close session if too many bad commands, null commands, commands with syntax errors; also add control "smtp_server_badcmd_max" that defines the limit. 20070330:mem:mvmtr For grins, add a bit more to the HELP command (including logging). Also log the HELP command; I think it's a curious enough thing to log. 20070330:mem:mvmtr In DATA, if there is any pending input after the message has been collected, don't accept, and terminate the session. This is to ward off senders with buggy dot-stuffing and buggy pipelining. This would include, for example, hijacked web proxies that are used to initiate SMTP sessions. 20070404:mem:mvmtr Modify the client_check and helo_check logic so that some of the inline checks are forked off into checks that can be referenced by name in the appropriate control string (like the other checks). For client_check, this is "pass", and for helo_check, these are "hook", "me", and "pass". This allows a little greater administrative control over the order of checks and, indeed, whether they are done. 20070404:mem:mvmtr Add new controls "smtp_server_mail_checks" and "smtp_server_mail_check_defer" to control the checking that is done on the MAIL FROM argument. These are analagous to the client_check and helo_check controls. Implement the checks, with initial set "auth", "dns", "hook", "pass", and "qmail". 20070404:mem:mvmtr Previous edit lets us defer the MAIL FROM check to RCPT TO time, so add that check there. (Some things have been deferred that weren't originally planned on, some further rework is inevitable, including giving more info to the MFL hook(s).) 20070404:mem:mvmtr Initialize the controls "smtp_server_client_check", "smtp_server_helo_check", and "smtp_server_mail_check" with some default values, in case admin doesn't supply them. 20070404:mem:mfl Bugfix: portability error in mfl-eval.c in using the character class tests islower() et al. First, it wasn't #including ctype.h as it should. Second, it was assuming that the return value from these tests would fit into an 8-bit value. On some systems (like the linux system where the bug was observed), a successful return from islower, for example, was 512. This would keep some of the built-in-functions like $str_has_lower() from working. Easy fix, dumb mistake. 20070405:mem:mvmtr RFC2821 says naked "postmaster" address must always be accepted. We still allow the MFL hook to look at the address and make a decision (if the admin wants to violate this rule, they can do it), but if the hook doesn't make a decision, make sure we accept "postmaster". Other policy decision points can still override it (again, by the express decision of the admin). 20070405:mem:mfl Bugfix: sieve tagged options, e.g. ":copy", are parsed as a sequence of colon character plus a symbol. Trouble was observed when the option name was the same as a formal MFL function argument. Consider an MFL function to do a specific DNSBL lookup: int scheck( string ip ) { return( sieve { dnsbl :ip [ip] "spamcop" "*" }); }; The tokenizer saw the ":ip" as ':' followed by a formal function argument "ip". The tokenizer was really the wrong place to be doing this argument lookup; remove the TK_ARG token type, and do the lookup in the expression parser (in the function parsetree_mktknode()) where it more properly belongs. Kind of an obscure one... 20070405:mem:mvmf Rename the "docs" directory to "notes". People expect documentation in "doc" or "docs" -- I have been keeping reference material and random notes in there. 20070405:mem:mvmf Misc document/reference/sample file updates. 20070405:mem:rel mvmda 0.131.0, mvmtr.0.16.0 20070406:mem:mfl Try to make sure that character string literals (LITSTR, quoted strings) can contain embedded NUL characters. This means using counted strings in a few places where NUL-terminated strings were assumed. Some work probably remains to be done here, as there are various idiosyncracies with the equivalence of literal strings and character arrays, and the fact that such things can be included in aggregates. There are also many places where strings (etc) are printed using simple %s printf formats, for warnings and errors and so forth. 20070406:mem:mvmtr Add mvmtr-mfl-bif module for MFL built-in functions for mvmtr. Add built-in function: $mvmtr_log() to add a standard-format message to the SMTP log. 20070414:mem:mvmf For some reason I have been putting almost all include files (.h files) into the include directory. I can't really explain why, it's not a habit of mine, although I do usually like to keep commonly-needed .h files in a shared area so that each .C file doesn't have to know about the filesystem organization. At any rate, I've started to reverse this by moving all of mvmtr's private .h files in with mvmtr. 20070425:mem:mfl In sieve "deleteheader" if :last is given and there is no :index, I had defaulted :index to 1. The spec says :index is required with :last, so make it be that way. 20070425:mem:mvmtr The next bunch of mvmtr changes began pretty much as soon as the last release, but are noted here as of a later date. 20070608:mem:mvmtr I was unhappy with the implementation of built-in checks almost from the moment of the last release: that check names were only unique within the enclosing scope (e.g. 'helo' and 'connect' checks both had a "generic" check); that there was an associated "defer" with each check list; that the internal checks didn't play as nicely with MFL hooks as I'd like. So there are some of change entries reflecting this. There were also a bunch of un-noted internal changes to support some of these things. 20070608:mem:mvmtr Introduce unique checks that are referenced by name ("named checks"). Each check has a particular meaning and can be invoked at any time that checks are done. 20070608:mem:mvmtr The list of checks at this time is: client-generic, client-hook, helo-dns, helo-dnsclient, helo-generic, helo-hook, helo-me, helo-syntax, mail-dns, mail-hook, mail-qmail, rcpt-addrmap, rcpt-hook, rcpt-qmail. (The qmail checks are only available if QMAIL support is compiled in.) 20070608:mem:mvmtr Introduce check sequences via "annotated checklist" functions. An annotation checklist specifies a list of named checks to be done in order, and some annotations that affect the way each check or each check list context is interpreted. Among other things, each individual check is expected to set, upon rejection, a score value that can optionally be used by the annotated checklist process. The annotated checklist functionality provides a nice bridge between hardwired built-in flow and elaborate extensibility via MFL hooks. 20070608:mem:mvmtr Change the various "_check" controls so that they provide an annotated checklist to be processed. Remove the "_defer" controls as they are no longer relevant. 20070608:mem:mvmtr The list of "check" controls is now: "smtp_server_connect_check", "smtp_server_helo_check", "smtp_server_mail_check", and "smtp_server_rcpt_check" . Each can provide an annotated checklist to be called at connect time, HELO time, MAIL FROM time, and RCPT TO time. 20070608:mem:mvmtr Remove the "addrmap" control since the addrmap check is no longer automatic. Other checks were also removed or renamed, but this is noted specially. 20070608:mem:mvmtr Most MFL hooks are no longer called automatically, but are now called out of an annotated checklist instead. This means the admin must specifically invoke the MFL hooks wherever they are wanted; or to look at it another way, the "hooks" are now into annotated checklists instead of into MFL hooks directly. The one remaining automatic MFL hook is $hook_data . The annotated checklist mechanism also provides for specifying alternate hook names; it's very handy to use different specialized hooks. 20070608:mem:mvmtr The "$hook_connect" MFL hook is no longer called at connect time. The "client-hook" named check can be used in any checklist; this calls an MFL client hook function (by default named "$hook_client"). 20070608:mem:mvmtr Add the "RISET" internal functionality. RISET is a set of SMTP return and logging information that relates to a command being processed. Using RISET, each smtp-time check (i.e. named checks) can build up a list of things to be logged and things that might be returned to the SMTP client. RISET entries for a particular check can be backed out if the check is to be undone in some way; RISET entries for a particular command can be ignored if the context is inappropriate; RISET log entry code suffixes can be modified once added (e.g. to indicate a soft failure instead of hard failure); or ultimately they can be logged and returned to the SMTP client in a standard way. 20070608:mem:mvmtr Add a "smtpd_reply" function that will return a reply to the SMTP client in a standard way, as modified by controls. (Note that this function is only to be used in cases where it makes sense.) In particular, the log RISET entries can be used to build up a list of keywords, and they can also be used to build up a line-item response to the client. This is just a stab at an interface by which to convey extended reply information to SMTP clients. 20070608:mem:mvmtr Add string control "smtp_server_reply_ext_hard" - if this control has a value, it is used to control and extend the information returned by the smtpd_reply function for any 5xx return code (hard failure). Similarly, add "smtp_server_reply_ext_soft" control for 4xx (soft fail) return codes. 20070608:mem:mvmtr Add MFL built-in function "$smtpd_score1_set" to allow the setting of the suggested score value for the check being processed. 20070608:mem:mvmtr Add MFL built-in function "$smtpd_riset_add" to add a RISET entry associated with the check being processed. 20070620:mem:mvmtr Scale back some of the 20061211 change that allowed smtpd dialog hooks to return lots of info bundled into a string. With the reworking of the way hooks and checks operate, all but the acceptance code is now irrelevant anyway. It may be that I'll want to allow embedding of other relevant stuff into the return string, i.e. to make providing this info cleaner than using various function calls to give that info; will cross that bridge when or if needed. 20070813:mem:misc Add xmtp_flush() function. 20070813:mem:mvmf Change xmtp_reply() and xmtp_vreply() to remove the nowF flag from the calling sequence; this was a flag that could force the output to be flushed. I don't think this belongs in this interface, it requires too much specific context about whether the output needs to be flushed. Require application to do a separate xmtp_flush(). 20070824:mem:mvmtr Amend the 20070608 change so that each SMTP command has its own control specifying part of the format of a generalized smtp reply (used by smtpd_reply()). The control names are now smtp_server_reply_xxx_ttt where "xxx" is replaced by the lowercased command name, and "ttt" is replaced by "hard" or "soft" depending on the rejection class (5 or 4). Not all commands have such controls; currently limited to "MAIL" and "RCPT" and "DATA". I may want to make the reply format completely specifyable at some point but who knows. 20070825:mem:mvmtr Rework the startup code somewhat; factor the session startup into a more manageable section. Make some of the blocking decisions a bit more orderly, e.g. greylisting pays attention to the session policy controls, other blocking decisions are similarly better sanitized. 20070825:mem:mvmtr For annotated namelist (checklists), add an annotation '#' meaning skip the entry, allowing one to easily comment out an entry in a namelist. 20070831:mem:mvmtr Also add flags 'I' and 'i' checking if the client is allowed to relay. 20070905:mem:mvmtr Edit at 20070327 looks wrong; greeting should be 421 if closing immediately, 220 otherwise. 20070905:mem:mvmf Change various mdns_query() calls + cname following code to use mdns_query_canon() which follows cnames. 20070909:mem:mvmtr mvmtr needed to call msgstore_init() if one wants to file a copy of an incoming message (which I guess I hadn't tried for a while). 20070910:mem:misc msstif_maildir_msg_new() (new-message handler for maildirs) wasn't calling lock_parse() before setting locks. Thus locking info was uninitialized for maildirs. A baffling oversight. Also add a sanity check in lock_lockset() and unlock_lockset(). 20070913:mem:mvmtr Always set the "admin" control to 1. Prior to this it was set to 1 only when reading the startup script, but let's leave it at 1 for the benefit of hooks running during the SMTP session. 20070913:mem:mvmtr Remove the control "email_address_primary" . I had it in there for the day when mvmtr might call user-level mfl scripts, but we'll think about that if that ever happens. 20070913:mem:mvmtr Make mvmtr log the size of the message body on successful delivery. Currently the body size exclusive of the headers, may change that later to include the headers. 20070914:mem:rel mvmda.0.132.0, mvmtr 0.17.0 Since most changes have been to mvmtr there hasn't been a release in a while. This release is mainly due to the 20070910 edit, which affects mvmda, but I've also been wanting to get out the newer mvmtr with its significant changes.