2-restms.txt

This document defines RestMS, a RESTful Messaging Service.  RestMS provides web applications with enterprise-level messaging via an asynchronous RESTful interface that works over plain HTTP/HTTPS.  RestMS extends RestTL and is itself extended through optional //profiles//.

* Name: www.restms.org/spec:2/RestMS
* Version: draft/4
* Editor: Pieter Hintjens <ph@imatix.com>
* Contributors: Steve Vinoski <vinoski@ieee.org>, Brad Clements <bkc@murkworks.com>

++ License

Copyright (c) 2009 by the Editor and Contributors.

This Specification is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version.

This Specification is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, see <http://www.gnu.org/licenses>.

++ Change Process

This document is governed by the Digital Standard Organization's [http://www.digistan.org/spec:1/COSS Consensus-Oriented Specification System].

++ Goals and structure of this document

This document defines RestMS, a RESTful Messaging Service.  RestMS provides web applications with enterprise-level messaging via an asynchronous RESTful interface that works over plain HTTP/HTTPS.  RestMS extends RestTL and is itself extended through //Profile// specifications.

We describe:

* How the RESTful Transport Layer (RestTL) is used as the basis for RestMS
* Each of the resource types that RestMS implements
* How RestMS is extended by profiles
* The set of response codes that a client should be able to handle.

++ Language

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED",  "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119[((bibcite rfc2119))].

These terms are introduced by, and have specific meaning, in this specification:

* Domain - a set of resources managed by one RestMS server or virtual host.
* Feed - a destination for messages published by applications.
* Pipe - a source of messages delivered to applications.
* Join - a relationship between a pipe and a feed.
* Message - a package of data consisting of an envelope and zero or more contents.
* Content - an opaque binary payload, with a MIME type and encoding.
* Profile - a set of feed, join, and pipe types that work together to solve a given functional domain.

++ Using the Transport Layer

[[[spec:1 | RestTL] specifies standard rules for representing resources, and standard mechanisms for working with them in a RESTful fashion over a plain HTTP client-server network.

Implementers should apply RestTL's rules to all resources and methods that RestMS defines, unless RestMS explicitly states otherwise.  For example there are certain resources (such as contents) and methods (such as POST on a feed) that do not follow the RestTL strictures.  These are clearly noted.  All others follow RestTL.

++ RestMS resource types

+++ Domains

Domains follow these rules:

* A domain is a public collection of profiles, feeds, and pipes.
* The server MAY implement multiple domains and MAY allow routing of messages between domains.
* Domains MAY be used to partition resources for authenticated access control.
* Domains are configured resources: clients do not create or destroy domains.
* Domains act as namespaces for pipes and feeds.
* The client and server must agree in advance on the domains that exist.
* The server SHOULD implement a default public domain named "default".

The URI for a domain is:

[[code]]
http://{server-name}[:{port}]/restms/domain/{domain-name}
[[/code]]

The URI for the default domain is:

[[code]]
http://{server-name}[:{port}]/restms/domain/default
[[/code]]

RestMS allows these methods on a domain URI:

* GET - retrieves the domain representation.
* POST - creates a new feed or pipe within the domain.

A domain document specifies the domain properties, and has references to all profiles and public feeds that the domain contains.  The XML format of this document is:

[[code]]
<?xml version="1.0"?>
<restms xmlns="http://www.restms.org/schema/restms">
  <domain title="{description}">
    [ <profile
        name="{name}"
        title="{description}"
        href="{URI}" /> ] ...
    [ <feed
        name="{name}"
        title="{description}"
        type="{type}"
        license="{license}"
        href="{URI}" /> ] ...
  </domain>
</restms>
[[/code]]

The domain document does not necessarily list all feeds: clients may create these as private and thus make them inaccessible through discovery.

+++ Profiles

Profiles follow these rules:

* A profile is a named set of semantics that the server MAY implement.
* The client SHOULD check that the domain contains the profiles it needs.

The URI for a profile points to a profile specification, which is a human-readable web page.  That URI can be fetched using the standard GET method.

+++ Feeds

Feeds follow these rules:

* A feed is a //write-only// ordered stream of messages received from one or more writers.
* The order of messages in a feed is stable per writer.
* Feeds deliver messages into pipes, according to the joins defined on the feed.
* Clients can create dynamic feeds for their own use.
* To create a new feed the client POSTs a feed document to the parent domain URI.
* The server MAY implement a set of configured public feeds.

RestMS allows these methods on a feed URI:

* GET - retrieves the feed.
* PUT - updates the feed.  The feed name and type cannot be modified.
* DELETE - deletes the feed.
* POST - sends a message to the feed or stage a content on the feed.

The XML specification for a feed this format:

[[code]]
<?xml version="1.0"?>
<restms xmlns="http://www.restms.org/schema/restms">
  <feed
      name="{feed name}"                    mandatory feed name
    [ type="{feed type}" ]                  optional type
    [ title="{short title}" ]               optional title
    [ license="{license name}" ]            optional license name
    />
</restms>
[[/code]]

The feed type defines the storage and routing semantics of the feed.  The allowed feed types depend on the profiles that the server implements.   If the client attempts to create a feed with an unknown type, the server responds with "400 Bad Request".

+++ Pipes

Pipes follow these rules:

* A pipe is a //read-only// ordered stream of messages meant for a single reader.
* The order of messages in a pipe is stable only for a single feed.
* Pipes receive messages from pipes, according to the joins defined between the pipe and the feed.
* Clients MUST create pipes for their own use: all pipes are private and dynamic.
* To create a new pipe the client POSTs a pipe document to the parent domain's URI.
* The server MAY do garbage collection on unused, or overflowing pipes.

RestMS allows these methods on a pipe URI:

* GET - retrieves the pipe representation.
* DELETE - deletes the pipe.
* POST - creates a new join for the pipe.

The XML specification for a new pipe has this format:

[[code]]
<?xml version="1.0"?>
<restms xmlns="http://www.restms.org/schema/restms">
  <pipe
    [ type="{pipe type}" ]                  optional type
    [ title="{short title}" ]               optional title
    />
</restms>
[[/code]]

The XML description of an existing pipe has this format:

[[code]]
<?xml version="1.0"?>
<restms xmlns="http://www.restms.org/schema/restms">
  <pipe
      name="{pipe name}"                    server-generated hash
      type="{pipe type}"                    actual pipe type
    [ title="{short title}" ]               title, if specified
    >
    [ <join href="{join URI}"
        type="{join type}"
        address="{address pattern}"
        feed="{feed URI}" /> ] ...
    [ <message
        href="{message URI}"
        address="{address literal}"
        message_id = "{identifier}" /> ] ...
    [ <message href="{message URI}" async="1" /> ]
  </pipe>
</restms>
[[/code]]

The pipe type defines the routing semantics of the pipe.  The allowed pipe types depend on the profiles that the server implements.   If the client attempts to create a pipe with an unknown type, the server responds with "400 Bad Request".

Pipes are not shareable: if more than one client accesses the same pipe, the results will be unpredictable and none of the clients will receive the full set of messages, nor any cleanly defined subset.  The proper way to distribute messages between clients is to use a service or rotator feed, and a set of ondemand pipes, one per client.

Pipes on service or rotator feeds should be cached: if a client creates a new pipe each time it starts, messages can be left unprocessed (and thus lost) in unused pipes.  Clients should therefore record the URI of the pipe they are using, and attempt to use this pipe when re-starting.  If the pipe no longer exists, they should create a new pipe and record that URI.

+++ Joins

Joins follow these rules:

* Joins specify criteria by which feeds route messages into pipes.
* Joins are always dynamic and always private.
* Clients MAY create joins at runtime, after creating pipes.
* To create a new join the client POSTs a join specification to the parent pipe URI.
* If either the feed or the pipe for a join is deleted, the join is also deleted.

RestMS allows these methods on a join URI:

* GET - retrieves the join representation.
* DELETE - deletes the join.

The XML specification or description for a join has this format:

[[code]]
<?xml version="1.0"?>
<restms xmlns="http://www.restms.org/schema/restms">
  <join
  [ type="{join type}" ]                    optional type
    address="{address pattern}"
    feed="{feed URI}" >                     the feed to pull from
    [ <header name="{header name}" value="{header value}" /> ] ...
  </join>
</restms>
[[/code]]

The join type defines the selection semantics of the join.  The allowed join types depend on the profiles that the server implements.   If the client attempts to create a join with an unknown type, the server responds with "400 Bad Request".

+++ Messages and contents

Messages and contents are tightly coupled resources that follow these rules:

* Messages are envelopes that can optionally carry contents.
* Contents are blobs of data with a MIME type set by the sender.
* Contents MAY be embedded in the message resource OR carried as separate resources.

Message/content publication and delivery are distinct processes with no overlap.  The use of message and content resources is the only area in RestMS that deviates from the standard resource access model defined by RestTL.  Specifically, in the RestTL model a client creates and then works with a resource within the same container resource.  However, message and content resources are created in one container (feed), and retrieved from a different container (pipe).

++++ Publication

Publication works thus: the client stages zero or more contents, and then posts a message that refers to those contents.

To stage a content the client does an HTTP POST to the feed URI, providing a typed opaque binary content.  That is, the body of the POST request provides the content body, and the Content-Type: header specifies the MIME type of the content.  The server replies with (if there were no other errors) "201 Created" with the URI for the staged content in the Location: header.

[[code]]
Client                                     Server
  |                                           |
  |  1.) POST to feed URI                     |
  |      Content-Type: {MIME-type}            |
  |      POST body is content                 |
  |------------------------------------------>|
  |                                           |
  |  2.) 201 Created                          |
  |      Location: Resource URI               |
  |<------------------------------------------|
  |                                           |
[[/code]]

Note that although the message is nominally the container for contents, the client does not POST a content to a message.  There is, in fact, no message resource at publication time, only at delivery time.

The Content-Type: on the content post request MUST not be a structured document type ("application/restms+xml", "application/restms+xml", or "text/xml").  If the Content-Type: is a structured document type, the server will interpret the POST as providing a message, not a content.

The response MUST have no Content-Type: header, and if it has a Content-Length: header, the value of that header must be 0.

On staged contents, //only// the Content-Type: header is preserved through to delivery.  To send content properties, the sending client MUST define these in the message resource.

Once the client has staged the zero or more necessary contents, it publishes the message by POSTing a message specification to the feed URI.  The XML format for this specification is:

[[code]]
<?xml version="1.0"?>
<restms xmlns="http://www.restms.org/schema/restms">
  <message
    [ address="{address literal}" ]
    [ message_id = "{identifier}" ]
    [ reply_to="{address literal}" ]
    [ <header name="{header name}" value="{header value}" /> ] ...
    [ <content href="{content URI}" ... />
    | <content
        type="{MIME type}"
        encoding="{encoding}">{content value}</content> ]
  </message> ...
</restms>
[[/code]]

The specification can contain multiple message elements: clients can stage multiple contents and then publish them in a single operation.

A message specification consists of a set of properties (attributes of the message element), a set of header elements, a set of content references, and a set of embedded contents.  These work as follows:

* The address is used in routing, and has feed-specific semantics.
* The headers may used in routing, and to carry arbitrary information.
* The message_id is an application-assigned identifier string.
* The reply_to tells an eventual recipient where to send a reply message.
* The content hrefs refer to previously staged contents.
* The content elements, if they have an element value, hold embedded contents.
* Implementations may add other properties with implementation-defined meaning.

Clients may publish any mix of staged and embedded contents.  Implementations MAY set arbitrary limits on this.  Embedded contents are encoded either as "plain" or as "base64".

The server processes each message element by routing the message+content, using the feed semantics, and the defined joins, into zero or more pipes.  It then automatically deletes the staged contents.

If a client attempts to publish the same staged content more than once the server MUST reply with "404 Not Found", and not route the message.  If a client attempts to publish a message referring to contents that were staged to a different feed, the server MUST reply with "403 Forbidden", and not route the message.  The server SHOULD verify the content URIs of all messages in the request before routing any message in the request.

It the server successfully processes the request, it replies with "200 OK".  Depending on the profile specification of the feed, it MAY return a message resource.  It MAY NOT return a Location: header.

RestMS allows these methods on a staged content:

* GET - retrieve the content.
* DELETE - deletes the content.

++++ Delivery

Delivery works thus: the client retrieves a message, then retrieves zero or more contents, and then deletes the message.

To retrieve a message the client must first have retrieved the pipe to know the message URI.  The message will often be an asynclet as defined in RestTL.  The client retrieves a message using the GET method.

The XML description of a message returned by the server is identical to the message specification posted to a feed.  The message description contains zero or more content elements; the receiver MAY retrieve the contents which have a href URI, using separate GET methods.  The Content-Type: of the retrieved content will be the same as the Content-Type: of the published content.

During routing, the server MAY reformat staged contents as encoded, and vice-versa.  The 'type' attribute of the embedded content corresponds one-for-one to the Content-Type: header of a staged content.  Staged contents are not encoded: if a server reformats a base64-encoded content to a staged content it MUST decode the embedded content according to IETF RFC 4648 [((bibcite base64))].  Similarly, if a server reformats a staged content to an encoded content it MUST encode the content as base64 unless the content contains only plain text characters.

After it has processed a message the client MUST send a DELETE request on the message URI.  When the server deletes a message it also deletes any contents that message contains.

RestMS allows these methods on a message that is held in a pipe:

* GET - retrieves the message representation.
* DELETE - deletes the message //and all older messages// from the pipe.

RestMS allows these methods on a content that is part of a delivered message:

* GET - retrieve the content.
* DELETE - deletes the content.

++ RestMS Profiles

This specification does not specify the semantics of feed, join, and pipe types.  These semantics are covered in separate //profile// specifications.  Profiles are the main area of extension and experimentation in RestMS.  Profiles follow these rules:

* The profile name is the specification name (number/Name).
* The server MUST list all the profiles it supports, in the domain document.
* The server MUST NOT implement two profiles that have overlapping feed or pipe types.
* The client SHOULD check that the profiles it plans to use are implemented.
* All profiles are optional: a RestMS server MAY implement zero profiles.

A profile defines the semantics of a set of feed, join, and pipe types. As part of that, the profile specification MAY extend the properties of feeds, joins, or pipes.  A profile SHOULD work within the constraints defined by RestTL as far as possible.

Join types are inherently scoped by feed types.  That is, two profiles can use the same join types: the semantics of the join type is dependent on the type of feed to which it is attached.

Profiles may depend on other profiles.  The profile specification MUST list the profiles it directly depends on.  When a server implements a particular profile it MUST implement all profiles that profile depends on, recursively until all dependencies are resolved.

Clients cannot enable, disable, or switch between profiles.  Servers MAY offer such functionality as part of their configuration.

++ Response status codes

In RestMS these server reply codes have specific significance:

* 200 OK - the request completed normally.
* 201 Created - a POST request succeeded.
* 204 No Content - an empty PUT request completed, with no effect.
* 304 Not Modified - a conditional GET request completed, with no effect.
* 400 Bad Request - a resource document was invalid.
* 403 Forbidden - a requested method was not allowed on a resource.
* 404 Not Found - the resource did not exist.
* 412 Precondition Failed - a conditional PUT or DELETE was not carried out.

Additionally, clients should be able to handle all HTTP errors, including:

* 401 Unauthorized - authentication is required.
* 413 Too Large - the request was too large.
* 500 Internal Error - the server suffered an internal error.
* 501 Not Implemented - the requested functionality is not implemented.
* 503 Overloaded - the server was overloaded.

[[bibliography]]
: rfc2119 : "Key words for use in RFCs to Indicate Requirement Levels" - [http://tools.ietf.org/html/rfc2119 ietf.org]
: rest : Roy Fielding, "Representational State Transfer (REST)" [http://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.htm www.ics.uci.edu]
: base64 : "IETF RFC 4648" - [http://tools.ietf.org/html/rfc4648]
[[/bibliography]]