Update Automation

1The requirement for update automation depends on the nature of the website, i.e. in terms of the frequency of change and the potential for structural re-organisation. There is also the issue as to whether the website allows interactive updates to be initiated by its users and the number of qualified people available to maintain the website. In terms of this specific website, the frequency of change is relatively slow, as updates are typically developed as an offline document, which is then converted to a set of webpages. The actual website itself does not allow interactive updates to be initiated by the user, but does support a forum, which is run as a third-party server-side application; so is not the primary focus of this discussion. However, over the years, the page structure of the website has changed considerably and it is this aspect that is most dependent on the automation developed, so far, and critical given a support staff of one!

So what is the nature of automation?

Basically, the automation consists of a series of offline programs, written in visual basic, that allow new content to be defined and inserted into the navigational tree structure. There is also a program that facilitates the conversions of the content from a document format, i.e. WORD, into a basic HTML output file. The following description does not address the details of the visual basic programs, just the general principles.


By and large, the file <itree.txt> has to be updated manually using a suitable text editor, although the underlying visual basic programs keeps this overhead to a minimum and then produces all other secondary files required to maintain and update the website menu and file structures. The following table defines the structure of itree.txt in which any line without a semi-colon is treated as a comment line, while semi-colons enclose fields that the program interprets and actions.


; ;0;1;Home;#;1;Site Home Page;Site,Home,Page;
;    ;1;2;Status;#;1;Site Status Report;Site,Status,Report;
; ;0;5;Worldviews;#;1;Worldviews;Worldviews;

Field Purpose
1 Allow line to be indented with spaces to reflect the hierarchical <depth>
2 Specifies the <depth> of the line item within the tree and
is also used to calculate the section number
3 Defines a unique file ID number, which precedes the descriptive label.
New updates set this field to '+' so as to allocate the next free number.
This number then appears in the next version of itree.txt and becomes
the unique numeric reference for this file in subsequent updates.
4 Provides the label used in the tree menu and is appended to the ID
number to create the HTML file name, e.g. 1.label.html
5 Optional field that allows a specific filename to be allocated, if required.
Otherwise the default value # conforms to the description above.
6 An index to the relative path from web root. Default=1
7 Full webpage description used by MakeMap
8 Metadata used by MakeMeta


The <otree.txt> file is automatically generated by MakeMenu and acts as both a cross check of the allocations within the navigational tree and as an input into secondary programs, e.g. MakeMap.


0;1; ;0;1;Home; ;Y;N;X;2;P;1;R; ;~
1.Home.html;1;Site Home Page;Site,Home,Page;

Fields Purpose
1-2 <item count> and <section number> derived from the input <depth>
3 null field to assist readability
4-6 <depth>, <input order> and <label>
7 null field to assist readability
8-13 <state>, <parent>, <next>, <previous>, <1st-child>, <type>
14 null field to assist readability
15-18 <filename>, <path index>, <description>, <metadata>


This program is the main program that helps support the Mysearch website tree menu system. Basically, the programs reads the input file <itree.txt> and processes it into the output file <otree.txt> plus creates the javascript file that defines an array, which is used by a runtime javascript that supports the tree-based menu navigation. As such, itree.txt is essentially a master file that defines the menu structure of the entire website. New pages can be added to the site by simply adding a new line with the default file number (+), which the program then assigns the next number in order. Within the structure of the itree.txt file, the name of the page within the tree is specified, which is then combined with the assigned number to create the HTML filename. There is also an extended description of the page, which is subsequently used by MakeMap plus the keywords used by MakeMeta.

Note: An existing page can be moved to a new position within the hierarchy of the website by simply changing its position within the itree.txt file. The only change to this line, which might be required, is when the new position is at a different <depth> in the tree structure.


This programs also reads the input file itree.txt, identical to MakeMenu, and processes it into the output file otree.txt plus the javascript file that defines an array that is used by the runtime javascript programs associated with a specific website sitemap, e.g. 1 or 2. The format of the sitemap is similar to the tree menu on each page, but has an extended description and displays a decoded depth index field, e.g. 1.2.1


This program converts the HTML source generate by Microsoft WORD to a stripped down version that removes all verbose class definitions that are embedded in the original Microsoft HTML file. However, all basic tags are preserved such that the WYSIWYG formatting used when creating the document under WORD are preserved. The output HTML generated by MakeHTML is then simply cut and pasted into the actual HTML page file maintained under Expression Web, which includes the links to all required CSS class definitions. There is a certain amount of manual editing required to link the generate HTML to the required CSS classes, although Expression Web tends to minimise this overhead and this amounts to no more work than that which would be necessary, if using a HTML editor for all content development.


This program takes the otree.txt file from MakeMenu as its input. The otree.txt file defines all the .html files required by the website which are then created conforming to a template.html file. In part, the Mysearch website is maintained using Microsoft Expression Web, which supports the concept of a 'Dynamic Web Template (DWT)' that acts as a master template for all webpages. As such, Makefiles creates all the required files defined in otree.txt.


This program updates the meta data in the html files, as defined by otree.txt. As per MakeFiles, otree.txt can be sourced from MakeMenu or MakeMap and edited to define some updated subset of the website. When adding just a few new webpages, there is little overhead in manually updating the metadata definition in the <head> section of the HTML file. However, this process is quite tedious when the updates correspond to dozens of new pages.


This program makes the sitemap.xml file for the Mysearch website. This program uses otree.txt from MakeMenu as its input to define all the .html files to include into the sitemap.xml file. Whether this file really helps a website like Mysearch attain any tangible benefits from any search engine is questionable, but was again developed primarily as a educational exercise.


This program updates the itree.txt input file for meta data. Basically, the page description label is chopped into keywords and then added to the itree.txt as a new metadata field. It is now a somewhat redundant program as the metadata is now usually added to itree.txt, when the new pages are required and inserted.