doc updates

--HG--
branch : pmacs2

BUGS

@@ -1,7 +1,7 @@
 2007/09/14:
 known deficiencies:
-  1. a single action (global search and replace) may produce N actions that need
-     to be individually "undone". This can be annoying.
+  1. a single action (global search and replace) may produce N actions that
+     need to be individually "undone". This can be annoying.
   2. pasting large blocks of text via X, etc can be far too slow.
   3. opening files larger than 5-10k can be very slow
   4. the minibuffer can't get larger than 1 line when it wraps

IDEAS

@@ -1,3 +1,8 @@
+2007/10/20:
+
+Tags for matching should be allowed to be multi-character, and should be
+a list of data/type pairs (rather than several lists).
+
 2007/08/11:
 
 Instead of storing method state in the method objects (for mode callback stuff),

MODES

@@ -50,15 +50,18 @@ defined by self.colors, a dictionary mapping token-names to a tuple consisting
 of at least a foreground color and a background color.
 
 Explaining how to write a Grammar is outside the scope of this document; see
-lex2.py and mode/*.py for examples. Some important points to note:
+lex3.py, mode2.py and mode/*.py for examples. Some important points to note:
 
     * Regexes are applied to only one line of the document at a time.
-    * All regexes must match at least one character.
+    * All regexes must match at least one character (the newline counts).
     * All tokens must consist of at least one character.
     * A rule that matches must generate one or more tokens.
     * Any input not matched by a rule will end up in a "null" token.
     * Tokens can't "look" for other tokens (but they can use 0-width assertions
       to test for data on the current line).
+    * Regions of text which begin and end with recognizable tokens can be
+      lexed using a different sub-grammar using RegionRule, etc. This nesting
+      can be arbitrarily deep.
 
 5. Indentation level detection