Introduction

Allegro Webactions is a Lisp-based framework for creating web sites. It is distributed with Franz Lisp Allegro Lisp and the open-source Portable Aserve library. For detailed information, see:

The key features that I find useful about Webactions are:

Tip: Edit Webactions CLP pages with an editor that knows about HTML, if you have one, rather than your Lisp code editor.

Unfortunately, Webactions is pretty limited compared to a framework like JSTL. I've defined some extensions in clp-exts.lisp.

Installation

If you aren't using Allegro Lisp, use QuickLisp to get the portable open-source version of AllegroServe.

The file cs325.lisp loads both AllegroServe, Webactions, and the extensions described below automatically.

Download vote-demo.zip and extract the files into your CS325 code directory. This is a simple example of a Webactions site, using the extensions described below.

Load the following files into Lisp:

Loading webapp.lisp should load two other files, vote.lisp and date-time.lisp. These are normal Lisp files, defining some simple functions for tallying votes and printing dates and times. They are not web-related in any way.

Start the web server by executing (net.aserve:start :port 8000).

Go to http://localhost:8000/vote-demo/. If you see a login page, things are working. Enter any name you want and your password. (Shhhh! Don't tell anyone but your password is your name reversed.) Try voting for some food, signing in as someone else, voting for another food, etc. See how the drop-down menu appears after the first vote and changes from that point on.

Defining a webapp

One of the nice things about this framework is how easy it is to declare the structure of the website, like this example from the Webaction introduction:

(webaction-project "simpleproject"
        :destination "site/"
        :index "home"
        :map
          '(("home"   "pageone.clp")
            ("second" "pagetwo.clp")))

This defines an internal project application name (simpleproject), a directory where files will be stored(site/), and names for the internal CLP pages, which are HTML files that may contain Lisp elements.

In the modern Model-View-Controller (MVC) approach to applications, CLP and HTML pages are the views and controllers. The model is your Lisp code. Any real webapp will have forms that, when submitted, trigger a call to Lisp functions to update the model, before sending a CLP view to the user. This can be specified very directly in a project definition, like this example from the Webactions reference:

(webaction-project "sample" 
          :project-prefix "/mysamp/" 
          :destination "/usr/proj/sample/" 
          :index "main"
          :map '(("main" "main.clp")
                ("signin" action-sign-in)
                ("choice"   "choice.clp")
                ("vote"      action-vote)
                ("thanks"  "thanks.clp")))

The project-prefix says that all URL's for this webapp will start with /mysamp/. The map list says the the URL /mysamp/signin will cause the function action-sign-in to be called. That URL will appear in the action attribute of an HTML form, presumably on the main.clp page.

Things are not quite as simple in the web world as the Webactions examples suggest. Modern webapps usually handle simple page retrieval (GET calls) differently than form submissions (POST calls) for a better user experience. While we won't be doing true REST-ful web applications, we will move in that direction.

More typical is the vote-demo webapp. Here is the project definition for that system:

(webaction-project "vote-demo"
      :project-prefix "/vote-demo/"
      :destination *vote-home*
      :index "show/login"
      :map
        '(;; return a resource
          ("show/login" "login.clp")
          ("show/choice" action-check-sign-in "choice.clp")
          ("show/results" action-check-sign-in "results.clp")
        
          ;; respond to a post
          ("do/signin" action-sign-in "choice.clp" (:redirect t))
          ("do/vote" action-check-sign-in action-vote "results.clp" (:redirect t))
        ))

There are three resources: the login page, the "pick a food" page, and the results page. Each of these has a URL that goes to a CLP page, with an intermediate check to make sure the user has signed in.

There are two controller actions: signing in and making a choice. These are handled by the functions action-sign-in and action-vote that update the model. Then the user is redirected to a resource. Redirection means the resource's URL is sent to the browser and the browser automatically sends a new request for that URL. While this may seem like an extra step, it dramatically improves the user experience. See here for why.

Two useful things to know about returning URLs from functions:

The specific URL and function name patterns used above are handy for keeping things straight, but not required.

Defining a webapp with webgen:make

To keep things simple for this course, I've made a super-simple webapp generator to set up a webapp project and directory for you in the style of vote-demo. Download and extract the files in webgen.zip into your CS325 code folder. Load webgen.lisp into your Lisp. Call: 

(webgen:make appname)

Use a simple name with no spaces or punctuation.

webgen:make will create the following directory and files in your CS325 code folder:

appname/
      home.clp
      login.clp
      webapp.lisp

webgen:make does this by copying the files in the directory webgen-templates and replacing all occurrences of the string webapp with appname. So if you want to change the default setup, just modify those files.

To test the generated webapp, load appname/webapp.lisp into Lisp. If you haven't started the server already, call

(net.aserve:start :port 8000)

Now try the link http://localhost:8000/appname, replacing appname with your webapp's name. Make sure the login page appears and that the welcome page appears if you enter a name and password. The password is the name in reverse.

If the links work, you can start editing these files to make a real application.

The webapp.lisp template uses several handy techniques that are worth pointing out.

(defpackage #:appname
      (:use #:common-lisp #:net.aserve #:net.html.generator))

    (in-package :webapp)

The above defines a package appname, so that you can load different webapps without any name conflicts. You may need to modify the defpackage to use additional packages. For example, any serious application will need one or more other packages of code that are independent of the web application.

(defparameter *appname-home*
      (let ((load-name (or *compile-file-truename* *load-truename*)))
        (namestring
        (make-pathname :host (pathname-host load-name)
                        :device (pathname-device load-name)
                        :directory (pathname-directory load-name)))))

The above defines a private variable *appname-home* to hold the directory that currently contains webapp.lisp.

(Webaction-project 
    "appname"
    :project-prefix "/appname/"
    :destination *appname-home*
    :index "home"
    :map
    '(;; respond to a GET
      ("login" "login.clp")
      ("home" check-login "home.clp")
      
      ;; respond to a POST
      ("do-login" do-login "home" (:redirect t))
      ))

The above project definition does several important things:

The definition of check-login in turn is defined to store the user's requested URL in a session variable before calling do-login. This means do-login can send the user to that URL after a successful login, instead of sending the user to the default home page.

Defining new CLP tags

One key part of Webactions are Common Lisp Pages (CLP). These have the extension .clp and are analogous to Java's .jsp pages and .Net's .asp pages. Like the JavaServer Pages Standard Tag Library (JSTL), CLP uses HTML-like tags to call code, rather than inserting programming code into the HTML, as is done in JSP and PHP web pages.

Webactions comes with very few tags, but defining new ones is pretty easy. For example, suppose we wanted a tag to insert today's date into an HTML file, like this:

<p>Today is <clp_date/> -- rise and shine!</p>
Note the closing slash. CLP tags have to follow XHTML syntax.

This isn't built in to Webactions, so we'll do it ourselves. (Note: many of the examples below come from the vote-demo files.)

First, define regular Lisp code to do most of the work. In this case, define a function to get the current time and format it into a human-readable string. For information on the functions I used to do this, see the Hyperspec chapters on the time functions and format. 

(defun get-date-string (&optional (utime (get-universal-time)))
      (multiple-value-bind
            (second minute hour date month year)
          (decode-universal-time utime)
        (format nil "~a ~2,'0d, ~d" (get-month-string month) date year)))

    (defun get-month-string (month)
      (nth (1- month)
          '("Jan" "Feb" "Mar" "Apr" "May" "Jun" "Jul" "Aug" "Sep" "Oct" "Nov" "Dec")))

This is a function we can test in the Lisp listener:

> (get-date-string)
    "Aug 11, 2011"

Now we define the CLP tag to insert this string into the HTML stream, using the AllegroServe HTML printing macro.

(def-clp-function clp_date (req resp args body)
      (html (:princ-safe (get-date-string))))

The parameter list (req resp args body) is required but in this simple case, we don't need to use any of the parameters. Now when we put <clp_date/> into a CLP page, it will be replaced with the current date before being sent to the user.

A CLP function name needs to have form module_function-name. The underscore is used by Webactions to identify CLP tags in the HTML.

For more examples, see the Webactions documentation.

Some additional tags

Webactions out of the box provides limited access to Lisp. I've defined some additional tags to remedy this. All of them let you evaluate a string containing a CLP Lisp expression.

CLP Lisp expressions

A CLP Lisp expression (CLE) is a string containing Lisp code. The string is read, using normal Lisp rules. Any symbols of the form $name are replaced by the corresponding query, request, or session scope value of the CLP variable name, without the dollar sign and in lower case. For example, the CLE "(vote:get-favorite $name)" on a page with the URL test.clp?name=John would have the value of (vote:get-favorite "John").

[EXPERIMENTAL] Since many CLE's are primarily retrieving data from objects, there's a special syntax to support some common ways to extract information. If a CLE has the form "($var key ...)" and the value of var is value, then the value of the CLE will depend on the type of value. If it is:

In the last case, the package of the key will be the same as the package of the class or structure.

Keep the Lisp expressions short and simple! They should mostly just call functions defined in your Lisp application, that you can unit test easily.

clp_eval

<clp_eval [var="var-name"] value="lisp-code" [package="pkg-name"]/>

This tag evaluates the given CLP Lisp expression. If a variable is specified, the result is stored in it, and nothing is written to the HTML, otherwise the result is written directly to the output HTML. If a package is given, the code is read using that package, otherwise it is read using the COMMON-LISP package.

Here are some examples.

<p>This site is <clp_eval value="(machine-instance)"/>.</p>

This inserts into the HTML the string returned by the Common Lisp function machine-instance. Note the closing slash.

<p>This date is <clp_eval value="(time:get-date-string)"/>.</p>

This inserts into the HTML the date string returned by a function get-date-string defined in the package time. This is an alternative to defining a tag like clp_date. Defining a tag would be clearer if this was commonly needed.

<clp_eval var="site" value="(machine-instance)"/>

This sets the CLP request variable site to the string returned the Common Lisp function machine-instance. Nothing is printed to the HTML stream.

<clp_eval var="voters" value="(vote:get-voters)"/>

This sets the request variable voters to the result returned by (get-voters) in the package vote. Nothing is printed to the HTML stream.

Look in the vote-demo CLP files for more examples of clp_eval.

Caution: If you insert a CLP tag inside an attribute value, you must use single quotes on the attribute value, so that the Webactions parser doesn't get confused with the double quotes the CLP tag requires. For example, this code won't work: 
    <input type="text" name="user"
      value="<clp_eval value="$user"/>">
You have to write this instead: 
    <input type="text" name="user"
      value='<clp_eval value="$user"/>'>
Notice that <clp_eval value="$var-name"/> is equivalent to <clp_value name="var-name"/>.

clp_foreach

One of the most common things to do in dynamic HTML pages is to display a list of data, such as

Webactions doesn't provide a tag for this so I defined clp_foreach. The syntax is:

<clp_foreach var="var-name" [status="status-var"]
        items="lisp-code" [package="pkg-name"]>
      body
    </clp_foreach>

lisp-code is read and evaluated as with clp_eval. The result should be a list. clp_foreach will set the request variable var-name to each element of the list and then insert a copy of body in the HTML. body may contain more HTML and/or CLP forms. The effect of this is to replicate the body for each element of this list.

If status="status-var" is given, then on each iteration status-var will be bound to a dotted pair whose CAR is the loop count (one-based) and whose CDR is the length of the list. This can be used to print an item's position in a list. It can also be used in conjunction with clp_when to do something different with the first or last element of a list.

Here's an example of using clp_foreach to build a simple set of options for an HTML select menu. This is appears in vote-demo/choices.clp.

<select>
      <clp_foreach var="food" items="(vote:get-choices)">
        <option value="<clp_value name="food"/>">
          <clp_value name="food"/>
        </option>
      </clp_foreach>
    </select>

clp_when

Sometimes you want some HTML to appear only if some data is present or some condition is true. Webactions has a plethora of IF tags but they're very specialized and only work with numbers. Therefore, I've defined a general clp_when tag. The syntax is:

<clp_when test="lisp-code" [package="pkg-name"]>
      body
    </clp_when>

lisp-code is read and evaluated as with clp_eval. The body will be inserted into the HTML if and only if the the result is not NIL. body may contain more HTML and/or CLP forms.

Here's an example of using clp_when in conjunction with a clp_foreach status variable to insert horizontal rules after every item in a list except the last.

<clp_foreach var="voter" status="status" items="(vote:get-voters)">
      <clp_value name="voter"/>
      <clp_when test="(< (car $status) (cdr $status))"><hr></clp_when>
    </clp_foreach>

clp_choose

If you want to select one of several alternatives, nest clp_when elements inside a clp_choose like this:

<p>
      <clp_choose>
        <clp_when test="(null $favorite)">
          Nothing selected yet.
        </clp_when>
        <clp_otherwise>
          You picked <clp_value name="favorite" />.
        </clp_otherwise>
      </clp_choose>
    </p>

The first clp_when that is true will be the only HTML generated. Use the optional clp_otherwise for a final default case.

CLP form elements

The CLP extensions include several tags for HTML forms to make it easy to populate a form with existing data. This is a common addition to web frameworks to make it easy for users to update information.

clp_input

The clp_input tag works just like the standard HTML input tag, except that the value attribute can be a CLP Lisp expression. This can be used to populate a text or hidden field with prior data, e.g.,

<clp_input type="text" name="user" value="$user"/>

Note that clp_input needs to be closed with />.

clp_menu and clp_option

One of the more annoying bits of code for populating HTML forms is setting the currently selected item of a menu. To make this a lot less hassle, use clp_menu instead of select and clp_option instead of option.

A clp_option will be marked as selected if its value equals the value of the clp_menu. For example,

<clp_menu name="favorite" value="($user :favorite)">
      <clp_option value="apple">Apple</clp_option>
      <clp_option value="banana">Banana</clp_option>
      <clp_option value="orange">Orange</clp_option>    
    </clp_menu>

Debugging webapps

In most systems, it's a little tricky to debug web applications because they run as separate processes. Error messages and trace output gets sent to a console stream, not to your Lisp Listener window.

To see the console window,

Web variable scope

Web server applications in any framework need some way to share information between functions as they are called in processing a request. Terminologies vary but most, including Webactions, support three common variable scopes. A variable here means a key in some table of key-value pairs.

All of the above maintain data separately for each user. If you want some global data shared among all users, you store that in global application variables, rather than web variables. Do this as little as possible! Remember that many people may be accessing shared data at the same time, so concurrency problems can arise if you try changing global data on the fly.