/*
========================================================================================================================

                                              Copyright  2014 Allen Baker

------------------------------------------------------------------------------------------------------------------------
File:        IncludeFileProcessor
Originator:  Allen Baker (2014.03.29 00:47 -06:00 Saturday)
LayoutRev:   4
------------------------------------------------------------------------------------------------------------------------
$RCSfile$
$Revision$
$Date$
======================================================================================================================== */
package cosmicabyss.com.app;

import java.io.*;
import java.util.*;


import cosmicabyss.com.lib.*;



/*
======================================================================================================================== *//**
Description<PRE>
   This program scans a text file and echos it to an output file while recursively replacing any line of the following
   form:
      #Include: [filename]
         where [filename] is the optional name of a file
   with the content of the file with the name: Filename

   The original line that begins with the #Include: keyword (not case sensitive) is omitted from the output file. This
   means that if you set up an HTML file as a template with variables as #Include: file names, you can substitute
   a file name whose contents will be included inline. Alternatively, you can substitute an empty string for the file name
   and nothing will happen except that the #Include: line will be left out of the output file.

   The output file this program creates will be named exactly as the input file except it will have the input file's
   extension duplicated. If the input file has no extension, the output file name will be the input file name with the
   extension .txt added.
      If the input file name is: /mypath/tuna.html the output file name will be /mypath/tuna.html.html

      If the input file name is: /mypath/tuna      the output file name will be /mypath/tuna.txt
</PRE>

Usage<PRE>
   java cosmicabyss.com.app.IncludeFileProcessor [-h -q -r -s] [-l [logFile]] {-x fileSpec} {-ifile fileName} {-xfile fileName} fileSpecs+
      Arguments can be placed in any order on the command line.

   -h means HELP.
      It causes this usage message to be displayed, and terminates the program.

   -q means QUIET.
      It stops the program from sending log messages to stdout. The program sends log messages to stdout by default;
      this option is the only way to stop it from doing that.

   -r means RECURSE.
      It causes the program to recurse down through the trees of subdirectories that are rooted at the directories
      specified in the file specifiers that are on the command line.

   -s means SUBDIRECTORIES.
      It causes the program to match subdirectory names that match the file specifiers on the command line.

   -l means LOGFILE.
      It takes an optional file name parameter, and causes the program to send log messages to the named file. If the
      file name is not present, the program generates a default file name. If the environment variable LOGDIRECTORY is
      defined, the program places the file in that directory. If not, the program places the file in the current
      directory. The program continues to send log messages to stdout too, unless the -q option is used.

   -x means EXCLUDE.
      It requires a fileSpec, and causes the program to not process any files that match the fileSpec. This control may
      be put on the command line as many times as needed to specify all the files that should be excluded from
      processing.

      Enclosing the fileSpec in quotes is REQUIRED for any fileSpec that contains embedded spaces or wildcard
      characters.

   -xfile means EXCLUDE FILE.
      It takes a required file name parameter, and causes the program to read each non-empty line in the named file and
      treat it as the filespec argument to a -x control. This control may be put on the command line as many times as
      needed to specify all the files that contain filespecs for files that should be excluded from processing.

   -ifile means INCLUDE FILE.
      It takes a required file name parameter, and causes the program to read each non-empty line in the named file and
      treat it as a fileSpec that identifies files to include in processing. This control may be put on the command line
      as many times as needed to specify all the files that contain filespecs for files that should be included in the
      processing.

   fileSpecs identifies the files to process.
      It is a required parameter and is one or more file specifiers that identify the files that the program will
      process.

      As many fileSpecs may be put on the command line as needed to specify all the files that should be included in
      processing.

      Enclosing the fileSpec in quotes is required for any file specifier that contains embedded spaces.

      Enclosing the fileSpec in quotes prevents the command line interpreter from trying to replace wildcard characters
      itself instead of letting IncludeFileProcessor do it.

      The quotes surrounding the fileSpec are not needed if it contains no wildcard characters or spaces.

</PRE>

<P>
<DL>
   <DT>
      <B>Example usage:</B>
      <DD>
         <PRE>
            *java cosmicabyss.com.app.IncludeFileProcessor -r -x"*.bat" "..\*.*"
         </PRE>
      </DD>
   </DT>
   <DT>
      <B>View Source:</B>
      <DD>
         <A href="IncludeFileProcessor.java.html">
            IncludeFileProcessor.java
         </A>
      </DD>
   </DT>
   <DT>
      <B>Author:</B>
      <DD>
         <A href="mailto:sourcecode.v01@cosmicabyss.com">
            Allen Baker
         </A>
      </DD>
   </DT>
</DL>
*//*
======================================================================================================================== */
public class IncludeFileProcessor
   {



   /*
   =====================================================================================================================
   @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@[  Constructors  ]@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
   ===================================================================================================================== */



   /*
   =====================================================================================================================
   <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<[  Public Instance Constructors  ]>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
   ===================================================================================================================== */



   /*:                                          
   ===============================================================================================================
   ===============================================================================================================
   =============================================================================================================== *//**
   This method creates a IncludeFileProcessor class object and runs the application to completion.

   <P><B>Implementation: </B><A HREF="IncludeFileProcessor.java.html#000">View source</A>

   @param
      pArgs contains the command line arguments with which the class was invoked as an application.
   *//*
   --------------------------------------------------------------------------------------------------------------- */
   public <Type> IncludeFileProcessor(Type[] pArgs) throws Exception
      {
      boolean  successCode = true;

      successCode = initializeInstanceConstantsAndVariables(pArgs);
      if (successCode)
         {
         successCode = processAllFiles();
         }
      cOut.println
         (
         CLASS_NAME.string() +
         (successCode? " completed successfully." : " completed with problems.")
         );
      cOut.println();
      cOut.stopLog();
      }



   /*
   =====================================================================================================================
   <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<[  Protected Instance Constructors  ]>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
   ===================================================================================================================== */



   /*
   =====================================================================================================================
   <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<[  Private Instance Constructors  ]>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
   ===================================================================================================================== */



   /*
   =====================================================================================================================
   @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@[  Initialization  ]@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
   ===================================================================================================================== */



   /*
   =====================================================================================================================
   <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<[  Private Class Initialization  ]>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
   ===================================================================================================================== */



   /*:                                          
   ===============================================================================================================
   ===============================================================================================================
   =============================================================================================================== *//**
   This method is the class static initialization block which is run the first time the JVM loads this class code
   and before anything else in this class is accessed

   <P><B>Implementation: </B><A HREF="IncludeFileProcessor.java.html#001">View source</A>

   *//*
   --------------------------------------------------------------------------------------------------------------- */
   static
      {
      /*
      ================================================================================================
      if required, make sure all the class constants and variables are initialized the first time the
      JVM loads this class code and before anything else in this class is accessed
      ------------------------------------------------------------------------------------------------ */
      initializeClassConstantsAndVariables();
      }



   /*
   =====================================================================================================================
   @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@[  Methods  ]@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
   ===================================================================================================================== */



   /*:                                          
   ===============================================================================================================
   ===============================================================================================================
   =============================================================================================================== *//**
   This method runs the IncludeFileProcessor class as a standalone application. It's the method that's called when the class
   is invoked from the command line by using the java application launcher - "java".

   Usage<PRE>
   java cosmicabyss.com.app.IncludeFileProcessor [-h -q -r -s] [-l [logFile]] {-x fileSpec} {-ifile fileName} {-xfile fileName} fileSpecs+
      Arguments can be placed in any order on the command line.

   -h means HELP.
      It causes this usage message to be displayed, and terminates the program.

   -q means QUIET.
      It stops the program from sending log messages to stdout. The program sends log messages to stdout by default;
      this option is the only way to stop it from doing that.

   -r means RECURSE.
      It causes the program to recurse down through the trees of subdirectories that are rooted at the directories
      specified in the file specifiers that are on the command line.

   -s means SUBDIRECTORIES.
      It causes the program to match subdirectory names that match the file specifiers on the command line.

   -l means LOGFILE.
      It takes an optional file name parameter, and causes the program to send log messages to the named file. If the
      file name is not present, the program generates a default file name. If the environment variable LOGDIRECTORY is
      defined, the program places the file in that directory. If not, the program places the file in the current
      directory. The program continues to send log messages to stdout too, unless the -q option is used.

   -x means EXCLUDE.
      It requires a fileSpec, and causes the program to not process any files that match the fileSpec. This control may
      be put on the command line as many times as needed to specify all the files that should be excluded from
      processing.

      Enclosing the fileSpec in quotes is REQUIRED for any fileSpec that contains embedded spaces or wildcard
      characters.

   -xfile means EXCLUDE FILE.
      It takes a required file name parameter, and causes the program to read each non-empty line in the named file and
      treat it as the filespec argument to a -x control. This control may be put on the command line as many times as
      needed to specify all the files that contain filespecs for files that should be excluded from processing.

   -ifile means INCLUDE FILE.
      It takes a required file name parameter, and causes the program to read each non-empty line in the named file and
      treat it as a fileSpec that identifies files to include in processing. This control may be put on the command line
      as many times as needed to specify all the files that contain filespecs for files that should be included in the
      processing.

   fileSpecs identifies the files to process.
      It is a required parameter and is one or more file specifiers that identify the files that the program will
      process.

      As many fileSpecs may be put on the command line as needed to specify all the files that should be included in
      processing.

      Enclosing the fileSpec in quotes is required for any file specifier that contains embedded spaces.

      Enclosing the fileSpec in quotes prevents the command line interpreter from trying to replace wildcard characters
      itself instead of letting IncludeFileProcessor do it.

      The quotes surrounding the fileSpec are not needed if it contains no wildcard characters or spaces.

   </PRE>

   <P><B>Implementation: </B><A HREF="IncludeFileProcessor.java.html#002">View source</A>

   @param
      pArgs contains the command line arguments with which the class was invoked as an application.
   *//*
   --------------------------------------------------------------------------------------------------------------- */
   public static void main(String[] pArgs) throws Exception
      {
      IncludeFileProcessor  thisIncludeFileProcessor = new IncludeFileProcessor(pArgs);
      }



   /*
   =====================================================================================================================
   <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<[  Public Class Methods  ]>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
   ===================================================================================================================== */



   /*
   =====================================================================================================================
   <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<[  Public Instance Methods  ]>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
   ===================================================================================================================== */



   /*:                                          
   ===============================================================================================================
   ===============================================================================================================
   =============================================================================================================== *//**
   This method sets the ConsoleStream for this object.

   <P><B>Implementation: </B><A HREF="IncludeFileProcessor.java.html#003">View source</A>

   @return
      a reference to this object

   @param
      pConsole is a ConsoleStream through which this objects sends its output.
   *//*
   --------------------------------------------------------------------------------------------------------------- */
   public IncludeFileProcessor setOut(ConsoleStream pConsole)
      {
      cOut = pConsole;
      return this;
      }



   /*:                                          
   ===============================================================================================================
   ===============================================================================================================
   =============================================================================================================== *//**
   This boilerplate method is used for testing an instantiated object of this class and may include any code the
   developer chooses.

   <P><B>Implementation: </B><A HREF="IncludeFileProcessor.java.html#004">View source</A>

   @return
      a reference to this object
   *//*
   --------------------------------------------------------------------------------------------------------------- */
   public IncludeFileProcessor test() throws Exception
      {
      cOut.titledPrintf
         (
         "\"HELLO WORLD!\"",
         "%s  %s  %s",
         "I'm an object of the", CLASS_NAME, "class, and I approved this message."
         );
      return this;
      }



   /*
   =====================================================================================================================
   <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<[  Protected Class Methods  ]>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
   ===================================================================================================================== */



   /*
   =====================================================================================================================
   <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<[  Protected Instance Methods  ]>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
   ===================================================================================================================== */



   /*
   =====================================================================================================================
   <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<[  Private Class Methods  ]>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
   ===================================================================================================================== */



   /*:                                          
   ===============================================================================================================
   ===============================================================================================================
   =============================================================================================================== *//**
   This method is called from the class static initialization block to make sure all the class constants and
   variables are initialized the first time the JVM loads this class code and before anything else in this class
   is accessed

   <P><B>Implementation: </B><A HREF="IncludeFileProcessor.java.html#005">View source</A>

   *//*
   --------------------------------------------------------------------------------------------------------------- */
   private static void initializeClassConstantsAndVariables()
      {
      }



   /*
   =====================================================================================================================
   <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<[  Private Instance Methods  ]>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
   ===================================================================================================================== */



   /*:                                          
   ===============================================================================================================
   ===============================================================================================================
   =============================================================================================================== *//**
   This method initializes all the instance constants and variables

   <P><B>Implementation: </B><A HREF="IncludeFileProcessor.java.html#006">View source</A>

   *//*
   --------------------------------------------------------------------------------------------------------------- */
   private <Type> boolean initializeInstanceConstantsAndVariables(Type[] pArgs) throws Exception
      {
      boolean  successCode = true;

      successCode = processCommandLine(pArgs);
      if (successCode)
         {
         }
      return successCode;
      }



   /*:                                          
   ===============================================================================================================
   ===============================================================================================================
   =============================================================================================================== *//**
   This method performs the core processing functions of this application.

   <P><B>Implementation: </B><A HREF="IncludeFileProcessor.java.html#007">View source</A>

   @return
      a success code of true if all goes well, false otherwise.
   *//*
   --------------------------------------------------------------------------------------------------------------- */
   private boolean processAllFiles() throws Exception
      {
      boolean  successCode = true;
      String   fmtStr;
      /*
      ================================================================================================
      tell 'em what we're gonna do
      ------------------------------------------------------------------------------------------------ */
      cOut.println("Plan","Will process " + iFileNameList.size() + " files.");
      if      (iFileNameList.size() > 99999) fmtStr = "%06d";
      else if (iFileNameList.size() >  9999) fmtStr = "%05d";
      else if (iFileNameList.size() >   999) fmtStr = "%04d";
      else if (iFileNameList.size() >    99) fmtStr = "%03d";
      else if (iFileNameList.size() >     9) fmtStr = "%02d";
      else                                   fmtStr = "%d";
      /*
      ================================================================================================
      do it for each file in the FileNameList
      ------------------------------------------------------------------------------------------------ */
      int  filesProcessed = 0;
      for (XString  fileName : iFileNameList)
         {
         XFile       file             = new XFile(fileName.trim());
         XString     canonicalPath    = new XString(file.getCanonicalPath());
         XString     extension        = new XFile(canonicalPath).getExtension();
         XFile       outFile          = extension.isEmpty()? new XFile(canonicalPath.concat(".txt")) : new XFile(canonicalPath.concat(extension));
         XString     outCanonicalPath = new XString(outFile.getCanonicalPath());
         TextWriter  outWriter        = new TextWriter(outCanonicalPath);
         /*
         ================================================================================================
         tell 'em which file is being processed next
         ------------------------------------------------------------------------------------------------ */
         filesProcessed++;
         cOut.printf("[" + fmtStr + "] Processing:   %s\n",filesProcessed,canonicalPath);
         cOut.printf("[" + fmtStr + "]    Output to: %s\n",filesProcessed,outCanonicalPath);
         /*
         ================================================================================================
         process the file
         ------------------------------------------------------------------------------------------------ */
         successCode &= processThisFile(canonicalPath,outWriter);
         outWriter.close();
         }
      /*
      ================================================================================================
      tell 'em what we did
      ------------------------------------------------------------------------------------------------ */
      cOut.println("Result","Processed " + filesProcessed + " of an expected " + iFileNameList.size() + " files.");
      return successCode;
      }



   /*:                                          
   ===============================================================================================================
   ===============================================================================================================
   =============================================================================================================== *//**
   This method performs the core processing functions of this application on one file.

   <P><B>Implementation: </B><A HREF="IncludeFileProcessor.java.html#008">View source</A>

   @return
      a success code of true if all goes well, false otherwise.

   @param
      pFileName is the name of the file to process.
   *//*
   --------------------------------------------------------------------------------------------------------------- */
   private boolean processThisFile(XString pFileName, TextWriter pOutWriter) throws Exception
      {
      boolean   successCode = true;
      TextFile  inFile      = new TextFile(pFileName);
      /*
      ================================================================================================
      this method makes no sense on directory files; it must be a normal file
      ------------------------------------------------------------------------------------------------ */
      if (inFile.isDirectory()) return false;
      /*
      ================================================================================================
      this is our input file
      ------------------------------------------------------------------------------------------------ */
      TextReaderIterator  inIter = inFile.iterator();
      /*
      ================================================================================================
      read each line and process it separately.
      ------------------------------------------------------------------------------------------------ */
      while (inIter.hasNext())
         {
         XString  line        = inIter.next();
         XString  trimmedLine = line.trim();
         /*
         ================================================================================================
         if the line IS NOT an "include" line, we just echo it to the output
         ------------------------------------------------------------------------------------------------ */
         if (! trimmedLine.startsWithIgnoreCase("#Include: "))
            {
            pOutWriter.println(line);
            }
         /*
         ================================================================================================
         else if the line IS an "include" line, ...
         ------------------------------------------------------------------------------------------------ */
         else
            {
            /*
            ================================================================================================
            trim away the "#Include: " keyword string
            ------------------------------------------------------------------------------------------------ */
            trimmedLine   = trimmedLine.trimLeftIgnoreCase("#Include: ").trim();
            /*
            ================================================================================================
            if what is left contains a string treat is as the name of a file to include inline and send its
            contents to the output.  Note, this part of the code will recurses to do the actual inclusion.
            Furthermore, the original line with the "#Include: " keyword is thrown away - not echoed to the
            output.
            ------------------------------------------------------------------------------------------------ */
            if (! trimmedLine.isEmpty())
               {
               XFile    file;
               XString  canonicalPath;

               file          = new XFile(trimmedLine);
               canonicalPath = new XString(file.getCanonicalPath());
               cOut.printf("Importing: %s\n",canonicalPath);
               successCode = processThisFile(canonicalPath, pOutWriter);
               /*
               ================================================================================================
               if something goes wrong, this code will exit the loop and cause the unwinding of any recursion
               that happened.
               ------------------------------------------------------------------------------------------------ */
               if (! successCode) break;
               }
            /*
            ================================================================================================
            else if the line had no file name on it, just throw the line away
            ------------------------------------------------------------------------------------------------ */
            }
         }
      /*
      ================================================================================================
      tell 'em how it went
      ------------------------------------------------------------------------------------------------ */
      return successCode;
      }



   /*:                                          
   ===============================================================================================================
   ===============================================================================================================
   =============================================================================================================== *//**
   This method displays the command line arguments that this program was invoked with.

   <P><B>Implementation: </B><A HREF="IncludeFileProcessor.java.html#009">View source</A>

   @param
      pArgs contains the command line arguments with which the class was invoked as an application.
   *//*
   --------------------------------------------------------------------------------------------------------------- */
   private <Type> void showArgs(Type[] pArgs) throws Exception
      {
      XString  str = new XString("");
      for (int i=0; i<pArgs.length; i++)
         {
         str = str.concat(UString.toString(pArgs[i]) + "   ");
         }
      cOut.println("Command Line Arguments",str);
      }



   /*:                                          
   ===============================================================================================================
   ===============================================================================================================
   =============================================================================================================== *//**
   This method processes the command line arguments and based on what it finds, sets the instance variables.

   <P><B>Implementation: </B><A HREF="IncludeFileProcessor.java.html#010">View source</A>

   @return
      a success code of true if all goes well, false otherwise.

   @param
      pArgs contains the command line arguments with which the class was invoked as an application.
   *//*
   --------------------------------------------------------------------------------------------------------------- */
   private <Type> boolean processCommandLine(Type[] pArgs) throws Exception
      {
      boolean  successCode = true;
      /*
      ================================================================================================
      Greetings !
      ------------------------------------------------------------------------------------------------ */
      System.out.println();
      System.out.println();
      System.out.println();
      System.out.println();
      cOut.banner(CLASS_NAME);
      System.out.println();
      /*
      ================================================================================================
      tell the user what command line arges this program started with
      ------------------------------------------------------------------------------------------------ */
      showArgs(pArgs);
      /*
      ================================================================================================
      this cannot be done statically because it throws an exception.
      ------------------------------------------------------------------------------------------------ */
      iGlobalProperties = new GlobalProperties();
      /*
      ================================================================================================
      get ready to process the command line arguments
      ------------------------------------------------------------------------------------------------ */
      CommandLine  cmdLn = new CommandLine(CLASS_NAME,pArgs);
      /*
      ================================================================================================
      these are the valid command line controls
      ------------------------------------------------------------------------------------------------ */
      cmdLn.addControlsFromString(":hqrsl::x:");
      cmdLn.addControl("-xfile","required");
      cmdLn.addControl("-ifile","required");
      /*
      ================================================================================================
      list the valid controls
      ------------------------------------------------------------------------------------------------ */
      cmdLn.listControls();
      cOut.println();
      /*
      ================================================================================================
      interpret the arguments on the command line
      ------------------------------------------------------------------------------------------------ */
      HashSet<XString>          controlsWithoutArgs = new HashSet<XString>();
      HashMap<XString,XString>  controlsWithArgs    = new HashMap<XString,XString>();
      ArrayList<XString>        excludedFileSpecs   = new ArrayList<XString>();
      ArrayList<XString>        includedFileSpecs   = new ArrayList<XString>();
      boolean                   result              = cmdLn.commandLineArgs
         (
         iGlobalProperties  ,
         controlsWithoutArgs,
         controlsWithArgs   ,
         excludedFileSpecs  ,
         includedFileSpecs
         );
      /*
      ================================================================================================
      if the command line was messed up, abort the program
      ------------------------------------------------------------------------------------------------ */
      if (includedFileSpecs.isEmpty())
         {
         cOut.println("COMMAND LINE ERROR: MANDATORY control NOT on command line:  fileSpecs");
         }
      if ((result == false) || (includedFileSpecs.isEmpty()))
         {
         usage();
         cOut.titledSeverityPrintf
            (
            Const.HALT,
            CLASS_NAME,
            "Command line error."
            );
         cOut.stopLog();
         System.exit(-1);
         }
      /*
      ================================================================================================
      arguments for the FileNameList with their default values.  Some of these arguments may be
      modified by commandline controls.
      ------------------------------------------------------------------------------------------------ */
      boolean  recurse             = false;
      boolean  fullyQualifiedNames = true;
      boolean  matchSubdirs        = false;
      boolean  sort                = true;
      boolean  collapse            = false;
      /*
      ================================================================================================
      process commandline controls without arguments
      ------------------------------------------------------------------------------------------------ */
      for (XString  ctl : controlsWithoutArgs)
         {
         cOut.println(ctl.string() + " control is present.");
         if (ctl.equals("-h"))
            {
            usage();
            cOut.stopLog();
            System.exit(-1);
            }
         recurse      |= ctl.equals("-r");
         matchSubdirs |= ctl.equals("-s");
         }
      if (controlsWithoutArgs.size() > 0) cOut.println();
      /*
      ================================================================================================
      process commandline controls with arguments
      ------------------------------------------------------------------------------------------------ */
      Set<XString>  set = controlsWithArgs.keySet();
      for (XString  ctl : set)
         {
         XString argStr = controlsWithArgs.get(ctl);
         cOut.println(ctl + " control is present with the argument: " + argStr);
         }
      if (controlsWithArgs.size() > 0) cOut.println();
      /*
      ================================================================================================
      get a list of all the files names that will be processed
      ------------------------------------------------------------------------------------------------ */
      for (XString  fileSpec : includedFileSpecs)
         {
         cOut.println("Files matching this fileSpec will be INcluded: " + fileSpec);
         }
      if (includedFileSpecs.size() > 0) cOut.println();
      for (XString  fileSpec : excludedFileSpecs)
         {
         cOut.println("Files matching this fileSpec will be EXcluded: " + fileSpec);
         }
      if (excludedFileSpecs.size() > 0) cOut.println();
      iFileNameList = new FileNameList
         (
         includedFileSpecs,excludedFileSpecs,recurse,fullyQualifiedNames,matchSubdirs,sort,collapse
         );
      /*
      ================================================================================================
      ------------------------------------------------------------------------------------------------ */
      return successCode;
      }



   /*:                                          
   ===============================================================================================================
   ===============================================================================================================
   =============================================================================================================== *//**
   This method writes a usage message to the log

   <P><B>Implementation: </B><A HREF="IncludeFileProcessor.java.html#011">View source</A>

   *//*
   --------------------------------------------------------------------------------------------------------------- */
   private void usage()
      {
      cOut.println("Program Description",DESCRIPTION);
      cOut.println("Usage",USAGE);
      }



   /*
   =====================================================================================================================
   @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@[  Nested Types  ]@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
   ===================================================================================================================== */



   /*
   =====================================================================================================================
   <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<[  Public Class Nested Types  ]>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
   ===================================================================================================================== */



   /*
   =====================================================================================================================
   <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<[  Public Instance Nested Types  ]>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
   ===================================================================================================================== */



   /*
   =====================================================================================================================
   <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<[  Protected Class Nested Types  ]>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
   ===================================================================================================================== */



   /*
   =====================================================================================================================
   <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<[  Protected Instance Nested Types  ]>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
   ===================================================================================================================== */



   /*
   =====================================================================================================================
   <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<[  Private Class Nested Types  ]>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
   ===================================================================================================================== */



   /*
   =====================================================================================================================
   <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<[  Private Instance Nested Types  ]>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
   ===================================================================================================================== */



   /*
   =====================================================================================================================
   @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@[  Constants  ]@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
   ===================================================================================================================== */



   /*
   =====================================================================================================================
   <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<[  Public Class Constants  ]>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
   ===================================================================================================================== */



   /*
   =====================================================================================================================
   <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<[  Protected Class Constants  ]>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
   ===================================================================================================================== */



   /*
   =====================================================================================================================
   <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<[  Private Class Constants  ]>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
   ===================================================================================================================== */



   /*
   ================================================================================================
   CLASS_NAME  : the name of this class
   DESCRIPTION : the man page description of this application
   USAGE       : the man page usage information for this application
   ------------------------------------------------------------------------------------------------ */
   private static final XString  CLASS_NAME  = new XString(IncludeFileProcessor.class.getName());
   private static final XString  DESCRIPTION = new XString
      (
      // <Description>  Preprocessor generated String definition, Don't mess with it.
      "This program scans a text file and echos it to an output file while recursively replacing any line of the following  \n" +
      "form:                                                                                                                \n" +
      "   #Include: [filename]                                                                                              \n" +
      "      where [filename] is the optional name of a file                                                                \n" +
      "with the content of the file with the name: Filename                                                                 \n" +
      "                                                                                                                     \n" +
      "The original line that begins with the #Include: keyword (not case sensitive) is omitted from the output file. This  \n" +
      "means that if you set up an HTML file as a template with variables as #Include: file names, you can substitute       \n" +
      "a file name whose contents will be included inline. Alternatively, you can substitute an empty string for the file name\n" +
      "and nothing will happen except that the #Include: line will be left out of the output file.                          \n" +
      "                                                                                                                     \n" +
      "The output file this program creates will be named exactly as the input file except it will have the input file's    \n" +
      "extension duplicated. If the input file has no extension, the output file name will be the input file name with the  \n" +
      "extension .txt added.                                                                                                \n" +
      "   If the input file name is: /mypath/tuna.html the output file name will be /mypath/tuna.html.html                  \n" +
      "                                                                                                                     \n" +
      "   If the input file name is: /mypath/tuna      the output file name will be /mypath/tuna.txt                        \n" +
      ""// </Description>
      );
   private static final XString  USAGE       = new XString
      (
      // <Usage>        Preprocessor generated String definition, Don't mess with it.
      "java cosmicabyss.com.app.IncludeFileProcessor [-h -q -r -s] [-l [logFile]] {-x fileSpec} {-ifile fileName} {-xfile fileName} fileSpecs+\n" +
      "   Arguments can be placed in any order on the command line.                                                         \n" +
      "                                                                                                                     \n" +
      "-h means HELP.                                                                                                       \n" +
      "   It causes this usage message to be displayed, and terminates the program.                                         \n" +
      "                                                                                                                     \n" +
      "-q means QUIET.                                                                                                      \n" +
      "   It stops the program from sending log messages to stdout. The program sends log messages to stdout by default;    \n" +
      "   this option is the only way to stop it from doing that.                                                           \n" +
      "                                                                                                                     \n" +
      "-r means RECURSE.                                                                                                    \n" +
      "   It causes the program to recurse down through the trees of subdirectories that are rooted at the directories      \n" +
      "   specified in the file specifiers that are on the command line.                                                    \n" +
      "                                                                                                                     \n" +
      "-s means SUBDIRECTORIES.                                                                                             \n" +
      "   It causes the program to match subdirectory names that match the file specifiers on the command line.             \n" +
      "                                                                                                                     \n" +
      "-l means LOGFILE.                                                                                                    \n" +
      "   It takes an optional file name parameter, and causes the program to send log messages to the named file. If the   \n" +
      "   file name is not present, the program generates a default file name. If the environment variable LOGDIRECTORY is  \n" +
      "   defined, the program places the file in that directory. If not, the program places the file in the current        \n" +
      "   directory. The program continues to send log messages to stdout too, unless the -q option is used.                \n" +
      "                                                                                                                     \n" +
      "-x means EXCLUDE.                                                                                                    \n" +
      "   It requires a fileSpec, and causes the program to not process any files that match the fileSpec. This control may \n" +
      "   be put on the command line as many times as needed to specify all the files that should be excluded from          \n" +
      "   processing.                                                                                                       \n" +
      "                                                                                                                     \n" +
      "   Enclosing the fileSpec in quotes is REQUIRED for any fileSpec that contains embedded spaces or wildcard           \n" +
      "   characters.                                                                                                       \n" +
      "                                                                                                                     \n" +
      "-xfile means EXCLUDE FILE.                                                                                           \n" +
      "   It takes a required file name parameter, and causes the program to read each non-empty line in the named file and \n" +
      "   treat it as the filespec argument to a -x control. This control may be put on the command line as many times as   \n" +
      "   needed to specify all the files that contain filespecs for files that should be excluded from processing.         \n" +
      "                                                                                                                     \n" +
      "-ifile means INCLUDE FILE.                                                                                           \n" +
      "   It takes a required file name parameter, and causes the program to read each non-empty line in the named file and \n" +
      "   treat it as a fileSpec that identifies files to include in processing. This control may be put on the command line\n" +
      "   as many times as needed to specify all the files that contain filespecs for files that should be included in the  \n" +
      "   processing.                                                                                                       \n" +
      "                                                                                                                     \n" +
      "fileSpecs identifies the files to process.                                                                           \n" +
      "   It is a required parameter and is one or more file specifiers that identify the files that the program will       \n" +
      "   process.                                                                                                          \n" +
      "                                                                                                                     \n" +
      "   As many fileSpecs may be put on the command line as needed to specify all the files that should be included in    \n" +
      "   processing.                                                                                                       \n" +
      "                                                                                                                     \n" +
      "   Enclosing the fileSpec in quotes is required for any file specifier that contains embedded spaces.                \n" +
      "                                                                                                                     \n" +
      "   Enclosing the fileSpec in quotes prevents the command line interpreter from trying to replace wildcard characters \n" +
      "   itself instead of letting IncludeFileProcessor do it.                                                             \n" +
      "                                                                                                                     \n" +
      "   The quotes surrounding the fileSpec are not needed if it contains no wildcard characters or spaces.               \n" +
      "                                                                                                                     \n" +
      ""// </Usage>
      );




   /*
   =====================================================================================================================
   @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@[  Variables  ]@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
   ===================================================================================================================== */



   /*
   =====================================================================================================================
   <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<[  Public Class Variables  ]>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
   ===================================================================================================================== */



   /*
   =====================================================================================================================
   <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<[  Public Instance Variables  ]>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
   ===================================================================================================================== */



   /*
   =====================================================================================================================
   <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<[  Protected Class Variables  ]>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
   ===================================================================================================================== */



   /*
   =====================================================================================================================
   <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<[  Protected Instance Variables  ]>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
   ===================================================================================================================== */



   /*
   =====================================================================================================================
   <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<[  Private Class Variables  ]>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
   ===================================================================================================================== */



   /*
   ================================================================================================
   cOut : formatted and time-stamped console output.
   ------------------------------------------------------------------------------------------------ */
   private static ConsoleStream  cOut = ConsoleStream.getSingleton();



   /*
   =====================================================================================================================
   <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<[  Private Instance Variables  ]>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
   ===================================================================================================================== */



   /*
   ================================================================================================
   iFileNameList     : the list of all file names that match the filespecs given on the command
                       line.
   iGlobalProperties : a copy of all the environment variables.
   ------------------------------------------------------------------------------------------------ */
   private FileNameList      iFileNameList     = null;
   private GlobalProperties  iGlobalProperties = null;



   }  // class IncludeFileProcessor



   /*
   =====================================================================================================================
   @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@[  Notes  ]@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
   ===================================================================================================================== */



   /*:                                          
   ===============================================================================================================
   ===============================================================================================================
   =============================================================================================================== *//**
   This

   <P><B>Implementation: </B><A HREF="IncludeFileProcessor.java.html#012">View source</A>

   @return
      This function returns

   @param
      pX is the
   @param
      pY is the
   --------------------------------------------------------------------------------------------------------------- */
      /*
      ================================================================================================
      ------------------------------------------------------------------------------------------------ */



   /*
   =====================================================================================================================
   <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<[  Section Level 1 Banner  ]>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
   ===================================================================================================================== */



   /*
   =====================================================================================================================
   ::::::::::::::::::::::::::::::::::::::::::::[  Section Level 2 Banner  ]:::::::::::::::::::::::::::::::::::::::::::::
   ===================================================================================================================== */



   /*
   =====================================================================================================================
   --------------------------------------------[  Section Level 3 Banner  ]---------------------------------------------
   ===================================================================================================================== */



   /*
   =====================================================================================================================
                                               [  Section Level 4 Banner  ]
   ===================================================================================================================== */