jWebTools: open-source project
jWebTools is a software toolkit for creating websites from simple to complex as requirements demand. jWebTools is written in Tcl and like Tcl jWebTools is a "platform" to build on, making all kinds of websites possible. jWebTools is perfectly suitable for producing fully static sites, but is also capable of generating pages on the fly. Content can be added dynamically, for example via AJAX/fetch APIs or standard server response. jWebTools includes its own backend server specifically for handling dynamic content.
This site, Thinairarts.com is an example since it was developed and is maintained using jWebTools. The job of managing busy websites for thriving businesses or enthusiastic communities of interest is a good problem to have! But keeping pace with demands for revision means frequently regenerating single pages up to entire sites.
Many tools are available to automate website maintenance. jWebTools includes a make-like utility pmake which can be set up to render site pages. Writing "rules" for a Pmakefile differs from typical makefiles though it's generally simpler. Running pmake is an easy way to manage generating site updates for deployment.
Tcl's tremendous adaptability and rock-steady reliability make it an ideal language for implementing this project. Website generation is naturally an off-line, time-tolerant task. Nonetheless, jWebTools operations aren't sluggish. Generating this website's 75 pages conpletes in just over two seconds, a decent performance for this type of application.
Download jWebTools!
You can access the latest version of jWebTools using our jWebTools fossil repository
Or if you prefer, go directly to the repo download page: jWebTools Downloads
You can find the jWebTools documentation here: jWebTools documentation
One option is installing fossil and cloning the repo, or just get the archive in tar.gz or zip format—it really couldn't be simpler!
The jWebTools Tools
jWebTools is a versatile construction kit originally developed for building websites. However it's also extremely useful in many other kinds of Tcl projects.
First let's look at how jWebTools works for websites.
Installing jWebTools
Installation is pretty straightforward. Once the archive is downloaded and decompressed, you'll have a single directory. Copy the directory to the usual location for extensions. Generally that's will be something like: /path/to/Tcl/lib/jwebtools.
Note that jWebTools has a few dependencies. First it uses new features of Tcl8.7 or 9.0. jWebTools uses the sqlite3 and tcllib extensions. Currently installing these on 9.0 is possible but both sqlite3 and tcllib need modifications to run on 9.0 so using 8.7 is therefore recommended.
Once installed, use jWebTools by putting the usual declaration in Tcl source files: package require jwebtools.
What's in the jWebTools Toolkit?
The jWebTools extension directory contains 20 files that begin with "oo...". These are TclOO classes that define the working elements of HTML (and other important functionality). There are also 11 non-TclOO utility files containing namespaces with procedures useful in a variety of contexts.
Html elements
-
Html
Html (defined in ooHtml.tcl) provides the <html> tag. The Html constructor takes arguments "-head" and "-body" with values of TclOO objects that render the expanded head and body elements of the web page.
Normally, Html isn't used directly but rather serves as a superclass for the derived class which actually defines the web page to be created.
Like all element classes in jWebTools, Html has a "write" method. In Html, calling "write" outputs everything in <html> to the file specified in the constructor's -filename argument which by default is the name of the class derived from Html.
-
Head
The Head class (ooHead.tcl) defines objects that write <head> elements for a web page. Head contains classes for the usual <head> elements such as links, meta, preload, title, script, style, description and charset. These elements have unique constructor signatures reflecting their uses.
-
Body
The Body class and body tag classes (ooBody.tcl) are derived from the Element abstract class. While a large number of body element classes are available it's also easy to define additional tag classes by subclassing Element directly, or just adding the new tag to the list of tags in ooElement.tcl.
Elements already defined currently include: a area b body button code div em figure figcaption footer h1 h2 h3 h4 h5 h6 header li map ol p pre span u ul img input.
Composite elements
These are convenient prewritten but highly configurable compound elements. This list provides a general description of the element's capabilities and usage. See documentation site for more details.
-
Menu
The Menu class (in ooMenu.tcl) defines a menu system for website pages.
Menus consist of a main menu block, submenu and submenu blocks, button text and menu buttons with associated actions.
As it's designed the menu is instantiates a configuration supplied as as nested Tcl objects. For example:
set m [Menu: -es [subst { [MenuItem: -href /author.html -label "The Author"] [Submenu: -level 0 -arrow left -title Galleries -menuitems [subst { [MItem: -href /galleries/monotypes.html -label "Monotypes"] [MItem: -href /galleries/photography.html -label "Photography"] [MItem: -href /galleries/screenprings.html -label "Screenprints"] }]] }]]Note that "MItem:" is the same as "MenuItem:".
-
WebImg
WebImg encloses an image in a <figure> element. This allows WebImg to have great flexibility re: placement, size and specification of other attributes via a large number of configuration options. It makes precise control of image rendering relatively easy and convenient.
Furthermore WebImg is fully "componentized" so that it generates corresponding CSS which is never confounded with any other element on a page no matter how many WebImgs are used. Thus WebImgs are ideal for building galleries of all sorts. This example shows typical instantiation of WebImg objects:
set camB [WebImg: -src /img/cameliaB-465x333.jpg ∖ -srcset {/img/cameliaB-233x167.jpg 233w, /img/cameliaB-465x333.jpg 465w, /img/cameliaB-698x500.jpg 698w} ∖ -sizes {(max-width: 625px) 50vw,(max-width: 1000px) 30vw, 30vw} ∖ -hRes 698 -vRes 500 ∖ -caption {{J Altfas} {Camelia B} screenprint 2020} ∖ -alt {SIS, abstract faces}] $camB css -margin 0 -height $ht $camB writecss $fdcssIn this example the object (camB) calls its "css" method which generates CSS according to input values, then writes CSS to the file previously opened for writing. The unique ID inserted into CSS declarations are also used automatically in corresponding html elements.
-
Slider
Slider displays several images in sequence, with a fade-in-out transition. It's highly configurable beginning with instantiation of the class taking a list of WebImg objects to be displayed. Size and placement of the Slider and vertical size of images can be given as arguments as well.
Setup begins with creating the WebImg objects and their generated CSS. A list of these objects is needed for Slider's invocation along with other Slider options.
Like all other jWebTools elements Slider is easy to use. A typical invocation is like this:
my mkSliderImgs 18.15em $fdcss Slider create sldr -imgls $SliderImgs -width 30 -height 19.5 puts $fdcss [sldr sliderCss] set Slider [sldr mkSlider]A set of WebImgs is made in mkSliderImgs, each WebImg set to 18.15em height and CSS written into the open file with descriptor $fdcss. Slider creates the object "sldr" which calls Slider methods writing out its CSS to the page CSS file and producing a finished Slider ready to be placed in a parent html element.
Sliders don't automatically transition among images. However this behavior is easily accomplished using a simple javascript "driver" (included in the repo).
A predefined page
-
Contact (html page)
A prepared page designed to accept visitors' messages (and page-specific comments). Implemention of contact.html permits dyamic feedback to the visitor, and also sending notification via email to a site administrator re: visitor's submitted message. Messages are stored in a specific sqlite3 database.
Contact.html is also used for visitor comments to specific pages. This works by using the referring page to differentiate origins of comments. The comment's info is stored in a sqlite3 database. Comment submissions also trigger notification to the designated administrator. By default comments are moderated and must be approved. The
Custom versions of the Contact class can be made by subclassing Contact. Most likely all that needs to be done is provide methods mkHead and mkBody in the derived class, and including whatever changes or improvements deemed appropriate for one's use case. (The derived class needs to be given a name other than Contact, but keeping "contact.html" as the generated output is OK.)
Utilities (class-based)
-
Log
The Log class implements logging for jWebTools programs. While simple to use it's important since several classes take a Log object as a required argument. A program could have more than one log and multiple logs could write to the same log file or separate files.
Log takes one argument, name of the logfile to write to. If logfile is omitted or empty string, logging goes to stdout, otherwise to the specified file.
There's one method, "writelog", which writes formatted output to the log file. Output lines start with date-time followed by whatever is in the arguments. A "special" argument is "-nnl" which suppresses end-of-line newlines. Other arguments are joined and output as a string.
-
Component
Component is class implementing unique IDs for CSS and html markup such that similar entities are kept distinct on a given page. Examples were mentioned above in the sections covering the WebImg and Slider classes.
Component works by assigning pseudorandom identifiers to named entities, and using the unique (within a given page) ID in both CSS declarations and attribute lists for corresponding html elements. This idea is also applied to javascript code such that a page could contain more than one instance of an item using the same js code but needing a separate script (or particular parts, e.g., storage) to work cleanly.
The constructor takes as arguments a name list of entities (e.g., CSS class names) to be "componentized". The ClassDict keeps the "database" within Component and thus can retrieve the ID for a name. Calling method "selector" with the css class name returns "class.name[s12345]" for use in CSS files. Method "attr" or "html" returns 'class ... s12345=""'. Method "ident" works for js, returning id_s12345.
As a TclOO class, Component encapsulates the ClassDict therefore enables multiple Components to exist in the same page without interfering with one another. That circumstance may not arise frequently but it's useful when it does.
-
Comment System
jWebTools provides a basic "comment system". It means pages can activate a comment dialog for visitors' use. This is a simple type of comment interface. There's no support for threaded comments, reply buttons, etc. It's similar to commenting on many blog sites.
Activating comments requires classes CommDiv and CommScript. CommScript is normally instantiated at the end of the html body. CommDiv is placed at the end of the main div of the page. It serves as the place where comments are placed.
Commenting is a dynamic feature using javascript AJAX calls to get comment info from the server. As a dynamic service it requires the jWebTools backend server to be running. This is readily accomplished using nginx. Other web servers could possibly be useful as well, but configuration of these options isn't directly supported by jWebTools.
Routing is handled on the server through the CommentRecv class which subclasses the abstract class Dynpage. CommentRecv is by default one of the dynamic pages configured by the backend server class JWTserv, described below. Fortunately, using comments is really quite a bit easier than this brief overview might suggest. Detailed instructions are in the process of being written.
-
Approve
Approve is a simple class which interaces with CommentDb to provide a way for administrators to approve (or not) comments left by visitors. By default, comments are subject to moderation. Adminstrators can log on to the server (via SSH) and use the approve utility to list pending comments and approve those waiting. This is pretty basic but sufficient for the simple jWebTools system. Of course it can be developed further as need to meet website requirements.
-
JWTserv
The jWebTools server class, JWTserv is elaborate in construction, but even so it's quite easy to use. All that's necessary is setting configuration options via the Config namespace. (Config is described below.) The server is started by calling "server", and it's up and running.
While JWTserv can be used as a standalone server during website development, it's intended as a backend server for dynamic pages via reverse proxy in association with nginx. Configuration for nginx follows basic conventions. An example is included in the repository.
-
Smtp
Smtp is another complex class with numerous configuration options. It's purpose in jWebTools is primarily for sending email notification to a site administrator when messages or comments are made by visitors. However Smtp can also be used as a standalone mailer which might be particularly beneficial to some users.
Configuring Smtp requires estabishing a file with default options. This is well described in ooSmtp.tcl. Config contains options for pointing at the correct file.
-
Pmakebase, PMakefile
jWebTools includes the class Pmakebase which defines methods intended to create make-like utilities. An example is Pmakefile in the repository which should be modified to suit individual website projects. Of course there are many other utilities that could be used for managing jWebTools-based development, but Pmakebase and derived classes are probably more closely aligned with jWebTools functionality.
One modular approach divides dependencies by type of file. CSS dependency is usually limited to CSS files whereas web pages are more likely to rely on multiple sources, such as class files, utilities, images, and other data source files.
Pmakebase provides methods Build, used to process a single class/object, and BuildX which handles a group of files or a directory in batch mode. Both methods have numerous options that refine or control the build process. Dependencies can be specified on a general and/or per file basis. CSS dependencies are specified separately since CSS has it's own routines for combining and compressing to optimize for downloading to web clients.
Generating page files via pmake/pmake.ps1 is reasonably expeditious, ~17ms/page on a Windows laptop. CSS generation is much slower owing to complexity of combining and minifying CSS files. Consequently generating CSS takes ~597ms/base_CSS_file (combined, minified). Fortunately most of the time only a small subset of all CSS files needs to be reprocessed during site updates.
Utilities (namespace-based)
-
cmdargs
The cmdargs namespace enables command-line arguments for procs and methods. This interface is similar to named arguments of Tcl commands.
The primary procedures in cmdargs are chkArgs and chkArgsCi. The difference is the latter is the case-insensitive version. (There's also chkArgsv and chkArgsvCi. The "v" variants set a variable, _CA_allvar, with all var names added by the chkArgs* procedures.) An example of setting up for use with a Tcl proc:
proc doSomething {args} { cmdargs::chkArgsCi $args -dstr {color "orange" size "large" timing ""} puts $color puts $size puts $timing ... }So now doSomething can be called as follows:
doSomething -color "red" -size small -timing good --> red small goodSo what's happened is the parameters given in -dstr become local variables in the proc or method. The value of the variables is the same as given in the proc call behind the associated named argument. Note that the case of the named arg doesn't affect the name of the variable rendered, that is, though called with -SIZE the variable is still "size". It just makes calling the proc more convenient.
chkArgs(Ci) takes up to 4 lists with different data types. -dnum takes number arguments, -dstr is for strings, -dbool takes boolean args (as 0,1). -denum takes enumerated lists such as {green blue purple} with the 1st item being the default.
Boolean args work in a special way. If given -dbool {hasbrain 0} the default value is 0 if -hasbrain isn't used in the args to the proc. However calling doSomething -hasbrain means variable "hasbrain" will have value 1 in the proc. Of course to be explicit -hasbrain could be followd by 1 or 0, but usually that's not necessary. Leaving it out renders hasbrain == 0, using by itself makes hasbrain == 1.
-
Config
Config is a very important namespace as many clases depend on its information. Basically the namespace consists of many variables and procs. However variables are by default set to very generic values and need to be overridden by settings in custom files and included where necessary.
Variables to override depend on which operating system and environment is used. The major distinction is between Windows and Unix systems. In Config no assumptions are made regarding file locations. Every assignment can and probably will need to be overridden. This local configuration file is referred to here as "localconfig.tcl" but can be named anything the user prefers though should end in .tcl for clarity.
A customization file could look like this:
# localconfig.tcl set Config::contactRcvTo jsmith@myisp.net set Config::contactRcvFrom jjones@anotherisp.com set Config::PlatformRoot C:/Users/USER/usr/local/prog/www set Config::MsgFilePath C:/Users/USER/usr/local/prog/messages/messages.txt ...Once localconfig.tcl is complete, starting the jWebTools server is simple:
package require jwebtools source /path/to/localconfig.tcl serverSimilarly localconfig.tcl needs to be sourced in a Pmakefile and may be needed elsewhere.
-
Resource
The Resource namespace contains procs used in JWTserv and Dynpage. Chances are jWebTools users won't have a need to use Resource directly unless modifying the server or using Dynpage (or subclass) in some novel way.
-
Util
Like Resource, the Util namespace defines procs used internally. However one that jWebTools users conceivably could employ is Util::fileType which takes a file path and returns a list of two items, the file's mime type and a boolean value (true if file is binary).
-
Additional utilities
The jWebTools extension directory contains a number of other files used internally. While it's possible users could find something of interest therein, the procs in the files are probably not readily usable for external application. By all means users are welcome to inspect, use and improve the code.
Subdirectories (css/, js/, shell/)
-
js/, css/
These directories contain js and css files needed for full functionality of several components of jWebTools. These files ordinarily should be copied to the working webroot for your site under directories with the same names (js,css). (That's a common convention and unlikely to be a problem.)
-
shell/
The shell directory has several files that can be used (with some revisions) locally to activate command-line functions. These include pmake.ps1/pmake.tcl, hsmtp.ps1/hsmtp.tcl ( standalone emailer), expandCss.ps1/.tcl, expand-minifyCss.ps1/.tcl, approve.tcl. Because of the wide variability of file system layout among different systems and users, specific guidance can't be given regarding modifications to make.However examining the files it should be clear enough to users how to modify paths and names of executables in order to get these working.
jWebTools for websites
There are a couple of ways to create sites using jWebTools. One is subclassing the Html class and instantiating objects of html element classes. Another way is using the convenient "Tag" namespace to generate everything needed without having to get directly involved with TclOO. Both ways will be illustrated below.
Using jWebTools classes
Except for Html, all html elements are represented by a subclass of the abstract class <Element>. To create a page, begin by deriving from Html:
oo::class create Page {
superclass Html
constructor {} {
next -head [my mkHead] -body [my mkBody] -outfile page.html
}
method mkHead {} {
Head new -title "My Page" -Es [subst {
[Description new "What my page is about"]
[Link new ...]
}]
}
method mkBody {} {
Body new -Es [subst {
[H2 new -x {Headline...}]
[Div new -> {id importantstuff} [subst {
...
}]]
}]
}
}
There are a few things to note in this example. In several places the cmdargs interface is used, this is a convenient way to specify input, much like command arguments in Tcl where the argument name is preceded by a dash, and after that the argument value. This will be described in more depth in a later section.
Also note the general layout of element calls. For example, the "tag" name, Body is followed by new, then -Es. Here -Es is an abbreviation for -elements followed by a list of elements that exist within the body elememt. In this case there's an h2, then a div with its own list of elements to be rendered within it. It should be obvious this follows the normal flow of html markup.
An advantage of this way of creating pages is the clean separation of logic for sections of page code. That makes what each is doing more readily understandable and contributes to easier site maintenance.
Using the Tag namespace
jWebTools provides the Tag namespace as an alternative way to use objects for page creation. Consider this example:
namespace eval Page {
namespace import ::Tag::*
variable html
proc construct {} {
variable html
set html [Html: -head [mkHead] -body [mkBody] -outfile page.html]
}
proc write {} {
variable html
$html write
}
proc mkHead {} {
Head: -title "My Page" -Es [subst {
[Description: "What my page is about"]
[Link: ...]
}]
}
proc mkBody {} {
Body: -Es [subst {
[H2: -x {Headline...}]
[Div: -> {id importantstuff} [subst {
...
}]]
}]
}
}
This example looks a lot like the previous one. Only here we're using an ordinary namespace and procs to implement the page. Notice that the prior "Head new" is now just "Head:". Every html tag class is present in the ::Tag namespace as class name + : (colon). Thus "Head:" is the same as "Head new". IOW "Head:" is a proc that returns an object that can be used directly. Just remember to put "namespace import ::Tag::*" at the top of the namespace where tags will be used.
Of course Tag::* elements can also be use in pages written as classes. Syntax is the same, Tag class followed by a colon:
...
method mkHead {} {
Head: -title "My Page" -Es [subst {
...
}]
...
}
The only issue with this is where to put the "namespace import ::Tag::*" line. It doesn't work to put it at the top of the class definition. However it usually works if put in the constructor, or in the first method.
constructor {} {
namespace import ::Tag::*
next -head [my mkHead] -body [my mkBody] -outfile page.html
}
...or...
method mkHead {} {
namespace import ::Tag::*
Head: -title "My Page" -Es [subst {
}]
}
jWebTools: for websites, and more...
This overview of jWebTools's many capabilities can only hit the highlights. But we hope the descriptions of the toolkit's basic functions are at least clear enough to pique your interest in what it really can do! Give it a try, let us know your impressions, your feedback is valuable. A prime goal is making constant improvement to jWebTools. Your input helps us get there and that is definitely appreciated!