]

# The Httpd Module and Toadhttpd

Sean Woods

Test & Evaluation Solutions, LLC

In this presentation we will cover:

- Why we need a reference web server
- The Httpd Module in Tcllib
- The Toadhttpd web server project

On an interesting note, this presentation is running from an instance of Httpd

Folks at home can tune into:

http://www.etoyoc.com/yoda/papers/tcl2018/Httpd_Slides.html

and replay this presentation on their own.
It's actually a cute bit of Javascript I stole picked up from Cyan Ogilvie.

| Error serving /yoda/papers/tcl2018.Httpd_slides.tml: The server encountered an internal server error:
    Must crash here
     while executing
    "error "Must Crash here""
        invoked from within
    "subst $tmltxt"
        (class "::httpd::content.file" method "content" line 42)
        invoked from within
    "my content"

This is just joke, go onto the next slide

# Why we need a reference web server

The user interface of choice in this day an age is no longer Tk.

# Why we need a reference web server

The user interface of choice in this day an age is no longer Tk:

- Tcl needs to explore new ways of generating human machine interfaces.
- Newer platforms are stressing browser based interactions.
- Especially on mobile platforms.

But hypnotoad, don't we already have a host of web servers for Tcl already?

# Why we need a REFERENCE web server

The Tcl community has already developed a number of web servers.

The problem is:

- The serious implementations are too massive, complex, and/or needy to be embedded in a desktop application.
- The embeddable implementations are basically toys.
- The implementations which support server side scripting, each go off an implement server side scripting in a completely different way.
- Testing. Testing. Testing.

# The Httpd Module

- Started off life as version 4 of Tclhttpd.
- Introduced Coroutines and TclOO to the core of the httpd server
- Added bootstrap and jquery integration
- Added a highly adaptable dispatch engine
- Server was implemented as an OO object
- Could be loaded in a package inside of normal Tcl software

Ended up being nothing like Tclhttpd.

# Simple Dustmote Example:

    package require httpd

    ::httpd::server create HTTPD port 8015 doc_root ~/htdocs
    HTTPD start

QED

There are three players: The Server, the Coroutine, and the Reply Object

# The players

The process starts when a socket event invokes a call to the server's connect method.

# The players

The connect method creates a coroutine which takes control of the socket, and uses the private Connect method of the server to bootstrap the rest of the reply process.

# The players

The coroutine reads the request line and the MIME headers, and passes that information to the Server's dispatch method

# The players

The dispatch method returns a dict

# The players

The coroutine then creates an instance of httpd::reply

# The players

The coroutine passes the dict and the socket over to the Reply's dispatch method.
The reply object mixes in behaviors according to the instructions in the dict, and uses those mixed in behaviors to generate the reply.

# The players

After the reply is returned, the socket is closed, the object is destroyed, and the coroutine terminates. Leaving only the server, to wait for the next socket open.

# Simple Serverside Content Example

The Httpd module only provides a fallback hander for static server content from doc_root.

To generate server side content directly from a url requires modifying the Server object. The Dispatch_Local method is empty by default, and intended to be replaced by the application. If that method returns a non-empty dict, the server ends the dispatch process, and returns that reply to the coroutine, who will in turn pass that to the httpd::reply instance.

# Simple Serverside Content Example

    # Hijack the dispatcher method on the server
    oo::objdefine HTTPD {
      method Dispatch_Local data {
        # Only match up requests for demo
        if {[dict get $data REQUEST_URI] eq "demo"} {
          set reply $data
          dict set reply mixin content ::mydemo
          return $reply
        }
      }
    }

# Simple Serverside Content Example

The default behavior for an httpd::reply is to invoke a method named content. It is intended that the content method be mixed in from some other class. The content method populates an internal value named reply_body. For easy of use, the httpd::reply class includes a puts method which will append to that buffer, and mimic the heady days of CGI where we used to spew data to stdout.

# Simple Serverside Content Example

    clay::define ::mydemo {
      method content {} {
        my puts [my html_header {Hello World!}]
        my puts {<h1>Demo page</h1>}
        my puts "Generated at [clock format [clock seconds]]"
        my puts [my html_footer]
        # NOTE METHOD DOES NOT RETURN A VALUE
      }
    }

# Simpler Serverside Content Example

Httpd has a plugin to automate the process of adding simple direct Urls:

    HTTPD plugin dispatch ::httpd::plugin.dict_dispatch

    HTTPD uri direct * demo {} {
      my puts [my html_header {Hello World!}]
      my puts {<h1>Demo page</h1>}
      my puts "Generated at [clock format [clock seconds]]"
      my puts [my html_footer]
      # NOTE METHOD DOES NOT RETURN A VALUE
    }

# Simple Serverside Content Example

GET/POST form information is available as a dict from the FormData method.

    HTTPD uri direct * time {} {
      my puts [my html_header {Hello World!}]
      set formdat [my FormData]
      my puts {<h1>Demo page</h1>}
      if {[dict exists $formdat format]} {
        set format [dict get $formdat]
      } else {
        set format "%Y-%m-%d %H:%M:%S"
      }
      my puts "Generated at [clock format [clock seconds] -format $format]"
      my puts {<form action=/[my request get REQUEST_URI]>}
      my puts "<input name=format value=\"$format\">"
      my puts {<input type=submit></form>}
      my puts [my html_footer]
    }

# Simple Serverside Content Example

The raw mime data is also available for complex request handling. In fact, the entire dict that was built by the dispatch process is stored in the objects clay data structure.

    HTTPD uri direct * rawmime {} {
      my puts [my html_header {Hello World!}]
      set mimetxt [my clay get mimetxt]
      my puts {<h1>Raw Mime</h1>}
      my puts "<pre>$mimetxt</pre>"
      my puts [my html_footer]
    }

# Simple Serverside Content Example

The headers for the reply can be controlled via the reply ensemble. CGI encoded data from the request is available through the request method.

    HTTPD uri direct * texttime {} {
      my reply set Content-Type text/plain
      my puts "This URI was [my request get REQUEST_URI]"
      my puts "Generated at [clock format [clock seconds]]"
    }

# .tml files

The httpd module's file server mechanism also understands Tclhttpd's old .tml file templates.

    > cat ~/htdocs/demotml.tml

    [my html_header {Hello World}]

    <h1>Demo page</h1>
    Generated at [clock format [clock seconds]]

    [my html_footer]

Note: the tml files are run inside of httpd::reply objects, so your templates can (but do not have to) invoke the methods of that object.

# Rest API Example

    HTTPD uri direct * restapi {} {
      # Have the reply do its own dispatching
      set URI [split [my request get REQUEST_URI] /]
      lassign [split $URI] base method uuid
      set class [find_class $method]
      if {$class eq {}} {
        # Errors will turned into the appropriate HTTP reply
        tailcall my error 404 {Not Found} {}
      }
      # Mix in a new behavior
      my clay mixinmap rest $class
      my reply set Content-Type application/json
      # Pass off control to a method brought in by the mixin
      tailcall my RestContent
    }

# Toadhttpd

The httpd module is very basic. It is intended to be a foundation to build on, not the entire building.

To run a real web facing server requires a lot of facilities that are beyond the scope of what an embedded web server would provide. At least a web server that is small enough for inclusion in tcllib.

A parallel project, Toadhttpd extends the httpd module in tcllib into fully fledged web server.

# Toadhttpd

Fossil Repo: http://fossil.etoyoc.com/fossil/toadhttpd

- Hardened against botnet attacks
- Plug-in architecture
- Bootstrap and jQuery Integration
- Sqlite Based:
  - Dispatch Engine
  - Cacheing Mechanism
  - Logging
  - IP Blocking
- SCGI Proxy Support
- Built-in Fossil Mirroring Support

# Q&A

And if you want to play with httpd, (and you have a recent Tcllib installed)

just:

    package require httpd