[main]Notes on TeXmacs

Adding a dialog to build websites

In this article we document the construction of a user interface to the website building facilities of TeXmacs.

Generating websites

TeXmacs can be used to build websites, in fact this blog, the main TeXmacs website and a couple of other websites (e.g. one here and another one here) are currently written and maintained exclusively within TeXmacs.

To avoid the need of exporting manually every single page to HTML, some support has been added to build websites in the form of Scheme procedures, to be found in the module (doc tmweb) (which can be found at TeXmacs/progs/doc/tmweb.scm). The two relevant exported procedures are tmweb-update-dir and tmweb-convert-dir.

For example the user can invoke TeXmacs for the command line with something like

texmacs -x "(tmweb-convert-dir \"sourcedir\" \"target-dir\")" -q

which executes (option -x) the Scheme command

(tmweb-convert-dir "sourcedir" "target-dir")

and then exits (option -q). Here source-dir and target-dir are two filesystem paths to the source and target directories for the conversion. The procedure tmweb-convert-dir will take every .tm file in source-dir, convert it to its HTML counterpart and save it in target-dir, all the other file types are just copied (see the sources for more details). tmweb-update-dir will convert only files which are newer than their counterpart in target-dir, if it exists. This is suitable for incremental updating a website which has been already generated at least once with tmweb-convert-dir.

TeXmacs (here we refer to version 1.99.15 but this functionality existed already for some time) disposes of menu items to initiate both operations (creation and update). They can be found in Tools→Web→Create web site... and Tools→Web→Update web site.... The dots (...) indicate that both items will execute some operation only after an interaction with the user (this is standard for all menu items, see e.g. File→Save as...). In particular in this case these items calls the scheme procedures tmweb-interactive-build or tmweb-interactive-update. The first, for example, looks like

(tm-define (tmweb-interactive-build)
  (:interactive #t)
  (user-url "Source directory" "directory" 
    (lambda (src)  (user-url "Destination directory" "directory"
      (lambda (dest) (tmweb-convert-directory src dest #f #f))))))

The declaration (:interactive #t) indicates to TeXmacs that this procedur will prompt the user for some information. In particular, the dots in the menu item are added automatically by TeXmacs to reflect this declaration. The meaning of the actual code is quite intuitive: the procedure user-url prompt the user for some information, the UI is supposed to show somehow the string "Source directory" to give an hint to the user about the content of the information required. The argument "directory" indicates that valid results should be an URL pointing to a directory in the filesystem. Finally the third argument to user-url is a Scheme closure accepting one argument. When this procedure is evaluated, it initiate a UI dialog with the user which can select a directory and then press “Ok” to confirm the selection. At this point the closure is called with an argument given by a string representing the URL just chosen. In the case above this result appears in the variable src. Subsequently the closure evaluate an additional user-url invocation, which asks for a destination directory an provide a second closure which looks like

(lambda (dest) (tmweb-convert-directory src dest #f #f))

This closure receives the destination directory in the variable dest and has also access to the variable src because this last variable is visible (thanks to the lexical scoping of Scheme) to all the expressions inside the first lambda. As a result the procedure tmweb-convert-directory will be invocated with the user provided values for src and dest and two additional boolean arguments (the first indicates whether we need to update or not and we will ignore the second here).

Albeit tmweb-interactive-build is easy to understand and write, it is not very convenient, since it does not remember the user's choices from previous interactions and oblige the user to repetitive actions, for example when developing some new feature of the site which requires frequent updating.

A new interface

The goal of this article is to use TeXmacs Scheme widgets to develop a new dialog which allow a nicer interaction with the user.

In particular we want to remember previous choices and give a more comprehensive perspective of the parameters in a single window, instead of proposint to the users a sequence of questions without any trace of the current state of the interaction. The following picture shows the design we aim for for this dialog:

Figure 1. The new dialog widget

This dialog will have to be initiated by a procedure (with no parameters) which we will call website-interactive-builder. Later on we can associate this function to some menu item to give easy access to the user to it. We have to manage three values: the source directory, the target directory and a boolean value to indicate creation or update. The function must read these values from a store (persistent through executions of the program), set up the dialog with these values, run the interaction, retrive the values after it and proceed with the suitable actions according to the new values. In particular, if the user didn't canceled the operation, we need to store away the new values for future invocations. This persistency of the information can be obtained by leveraging TeXmacs preference system, in particular we have two procedures get-preference and set-preference at our disposal. Their use is very simple: we can associate strings to given labels (also strings), the associations will persist across executions. If a label do not have any associated values so far get-preference will return the string "default". A possible form of website-interactive-builder is then the following:

(tm-define (website-interactive-builder)
    (let ((src (get-preference "website:src-dir"))
          (dest (get-preference "website:dest-dir"))
          (update? (get-boolean-preference "website:update-flag")))
      
    (dialogue-window (website-widget src dest update?)
       (lambda (flag src dest update?) 
           (if flag (begin 
              (set-preference "website:src-dir" src)
              (set-preference "website:dest-dir" dest)
              (set-boolean-preference "website:update-flag" update?)
              (if update? 
                    (tmweb-update-dir src dest)
                    (tmweb-convert-dir src dest)))))
        "Website tool")))

We choose labels "website:src-dir", "website:dest-dir", "website:update-flag" for the three parameters. The let initialises local variable with the previous values of the parameters and then we invoke (dialogue-window widget func). This procedure takes two arguments widget and func: the first is a widget constructed via tm-widget and the second is a closure which will be passed to the widget. The actual widget is constructed in the invocation to (website-widget src dest update?) which returns the executable code which will eventually display the dialog in Fig. 1.

This is achieved thanks to a custom description language accessed via the tm-widget macro.

Describing widgets in Scheme

The macro tm-widget allows to use Scheme forms to construct a wide variety of UI elements from a set of basic primitive elements. For detailed information please refert to Help→Scheme extensions→Customizing and exending the user interface. Our goal here is to provide an example of how the technique described in the help pages apply to our present problem.

Let us give right away a possibile definition for (website-widget src dest update?):


(tm-widget ((website-widget src-dir dest-dir update?)  cmd)
 (padded
  (vertical
   (hlist >>> (text "Website creation tool") >>>)
   ===
   (refreshable "website-tool-dialog-source"  (hlist 
     (text "Source dir      :") // //
       (hlist
         (input (when answer (set! src-dir answer))
                "file" (list src-dir) "40em")
         // //
         (explicit-buttons 
            ("choose" 
              (cpp-choose-file 
                 (lambda (u) 
                    (set! src-dir (url->string u))
                    (refresh-now "website-tool-dialog-source")) 
                 "Choose source dir" "directory" "" 
                 (string->url src-dir)))))))
    ===
    (refreshable "website-tool-dialog-dest"  
      (hlist 
        (text "Destination dir:") // //
        (hlist
          (input (when answer (set! dest-dir answer))
                 "file" (list dest-dir) "40em")
          // //
          (explicit-buttons 
            ("choose" 
               (cpp-choose-file 
                  (lambda (u) 
                     (set! dest-dir (url->string u))
                     (refresh-now "website-tool-dialog-dest")) 
               "Choose dest dir" "directory" "" 
               (string->url dest-dir)))))))
    ===
    (hlist
      (text "Update:") //
      (toggle (begin (set! update? answer) 
                     (refresh-now "website-tool-dialog-buttons"))
              update?))
    ===
    (refreshable "website-tool-dialog-buttons"  
       (bottom-buttons >>>
          ("Cancel" (cmd #f src-dir dest-dir update?)) // //
          (if update? 
              ("Update" (cmd #t src-dir dest-dir update?))) 
          (if (not update?) 
              ("Create" (cmd #t src-dir dest-dir update?))))))))

Note that all this should not be understood as standard Scheme code: the evaluation rules of Scheme are superseded by the macro tm-widget which will receive in input the description above and will use it to compile this description of a widget to a sequence of invocation of Scheme procedures. In this way we can provide a high-level description of widgets without need to code them in Scheme with all the gory details. tm-widget will take care of this for us.

Let us describe the meaning of some of markup tags used above

(padded …) describes a widget which contains other UI elements and pad them with some space around;
(vertical …) compose its arguments in a vertical list of UI elements;
(hlist …) do the same but horizontally;
>>>, ===, // are primitive elements (i.e. their definition is complete without referring to other content) which indicate various kind of spacing among the sorrounding elements;
(text "a string") define a UI element which show the given string;
(refreshable "label" w) indicates that the widget w must be refreshed when the procedure (refresh-now "label") is evaluated. It allow widgets to dynamically change in response to some action happening elsewhere in the code.
(input code "file" (list choice1 choice2 …) "40em") represents an text input UI element which accepts file names and which will have a horizontal size of 40em. The code form will be evaluated every time the text field change, within that form the symbol answer refers to the value of the text field, if answer is #f then the evaluation is not a result of an interaction with the user, so one has to check for this (It is not clear to me the reasons of this design, but it happens to be so, a deeper analysis of tm-widget inner design would be needed to get a grasp on this).
(explicit-buttons ("label1" form1) ("label2" form2) …) describe a row of buttons with various labels. When they are pushed the respective forms are evaluated. For example, in the code above, the first button calls the function cpp-choose-file with parameters as follows:
```
(cpp-choose-file 
    (lambda (u) (set! src-dir (url->string (url->string u))                    (refresh-now "website-tool-dialog-source")) 
    "Choose source dir" "directory" "" 
    (string->url src-dir))))
```
This function (implemented in C++) opens the GUI file chooser dialog and initiate the modal interaction with the user to ask her a path of kind "directory". The default directory is given by (string->url src-dir). When the user terminates the dialog pressing the “Open” button, then the chosen URL is passed to the closure in the variable u, we store this value in the src-dir variable (which is visible in all the syntactic context of (website-widget src-dir dest-dir update?), i.e. in the whole definition of the widget, and then we invoke (refresh-now "website-tool-dialog-source") to update the corresponding refreshable widget, i.e. update the text field with the appropriate value.
(toogle form value) describes a UI checkbox which takes its value from the value and which when switched (i.e. toggle its state) evaluates the form form with the symbol answer evaluating to the current state of the checkbox.
(bottom-buttons …) describe the buttons which should be located in the bottom right corner of the dialog. Typically these are the “Cancel” and “Ok” buttons. In our case we want to show a different label whether we are going to “Create” the site or only “Update” it.

Note the use of the parameter cmd which will be passed to (website-widget src-dir dest-dir update?) by dialogue-window. That is, (dialogue-window a b) expects as its argument a a closure with one argument (i.e. (lambda (cmd) …)) and b another closure (with arbitrary many arguments) and upon execution, it call the closure a with b as argument. Therefore in the widget we have that cmd points to the closure b. In our case calling (cmd #t src-dir dest-dir update?) inside the widget will result in the evaluation of the closure

(lambda (flag src dest update?) 
           (if flag (begin 
              (set-preference "website:src-dir" src)
              (set-preference "website:dest-dir" dest)
              (set-boolean-preference "website:update-flag" update?)
              (if update? 
                    (tmweb-update-dir src dest)
                    (tmweb-convert-dir src dest)))))

with flag set to #t and with the other three argument set to the appropriate values. We can then finalize the interaction by storing away the values into the program's preferences and either call tmweb-update-dir or tmweb-convert-dir.