Shattered Crystalline Matrix

This is the fourth post in a series of articles about Mantissa.

In the previous article I described how an offering can provide powerups to be included in a product, which will then be installed on a user store; in this installment, I will discuss what form these powerups can actually take, and how they allow you to expose your application’s functionality to the user.

One of the most commonly-implemented powerup interfaces in Mantissa is INavigableElement. Mantissa has a somewhat generalized idea of “navigation”, whereby a nested menu structure can be defined through INavigableElement powerups, and then displayed by different implementations for different protocols; for example, the web view has a visual dropdown menu system, whereas the SSH server presents a textual menu system. A typical INavigableElement powerup implementation will look something like this:

from zope.interface import implements
from axiom.item import Item
from axiom.attributes import integer
from xmantissa.ixmantissa import INavigableElement
from xmantissa.webnav import Tab

class PonyCreator(Item):
    """
    Powerup for creating and managing ponies.
    """
    implements(INavigableElement)
    powerupInterfaces = [INavigableElement]

    ponyQuota = integer(allowNone=False, default=10)

    def getTabs(self):
        return [Tab('ZOMG PONIES!', self.storeID, 1.0)]

INavigableElement only has one method, getTabs, which returns a list of “tabs” or menu items to be presented in the nav. The primary components of a tab are a title (which is how the item is displayed in the UI), the storeID of an item in the same store which the tab points to, and a float between 0.0 and 1.0 indicating the sort priority of the tab (higher values sort sooner). In this case, we have the tab pointing directly at the PonyCreator item itself; in order for this to work, we’ll need some extra code to allow PonyCreator to be exposed via the web.

In order for an item in a user’s store to be privately accessible via the web by that user, it needs to be adaptable to the (somewhat poorly-named) INavigableFragment interface. This is almost always done by defining an adapter from the item type to INavigableFragment:

from twisted.python.components import registerAdapter
from nevow.athena import LiveElement
from xmantissa.webtheme import ThemedDocumentFactory
from xmantissa.ixmantissa import INavigableFragment, ITemplateNameResolver

class PonyCreatorView(LiveElement):
    """
    Web view for Pony management.
    """
    implements(INavigableFragment)

    title = u'Pony management'
    docFactory = ThemedDocumentFactory('pony-creator', 'resolver')

    def __init__(self, ponyCreator):
        super(PonyCreatorView, self).__init__()
        self.ponyCreator = ponyCreator
        self.resolver = ITemplateNameResolver(self.ponyCreator.store.parent)

registerAdapter(PonyCreatorView, PonyCreator, INavigableFragment)

Our element will be wrapped in the Mantissa shell when it is rendered, so we cannot control the page title directly from the template, but the title attribute provides a way for our element to specify the page title. ThemedDocumentFactory is used to retrieve the template through the theme system; the arguments are the name of the template (‘pony-creator’) and the name of the attribute holding the ITemplateNameResolver implementation used to retrieve the template. This attribute is set in __init__ using a slightly awkward method; the template resolver should really be passed in by Mantissa somehow, but currently there is no mechanism for doing this, so instead we retrieve the default resolver ourselves from the site store.

This is all that is needed for hooking some code up to the web view; any further UI behaviour would be implemented in HTML / JavaScript in PonyCreatorView, usually by invoking additional methods defined on PonyCreator.

Next up: Sharing, or “How do I publish public / shared content?”

This is the third post in a series of articles about Mantissa.

The primary code entry point in Mantissa is the offering. Your application can provide an IOffering plugin in xmantissa.plugins, which hands some chunks of your code over to be used by Mantissa. There is no real reason to implement IOffering yourself; instead, simply instantiate xmantissa.offering.Offering in your plugin module. For example, you might have an xmantissa/plugins/ponies.py module containing the following code:

from twisted.python.filepath import FilePath

from xmantissa.offering import Offering

import ponies
from ponies.ponies import PonyCreator
from ponies.rainbows import RainbowEditor
from ponies.theme import PoniesTheme

plugin = Offering(
    name=u'Ponies',
    description=u"""
    A web-based system for generating ponies.
    """,
    siteRequirements=[],
    appPowerups=[],
    installablePowerups=[(u'ponies', u'Pony creation and editing', PonyCreator),
                         (u'rainbows', u'Rainbow editing', RainbowEditor)],
    loginInterfaces=[],
    themes=[PoniesTheme('base', 0)],
    version=ponies.version,
    staticContentPath=FilePath(ponies.__file__).sibling('static'))

There are a number of elements here, some of which we will only return to later, but I’ll run through all of them here briefly.

• name: A short identifier used as the name of this offering.
• description: A long description explaining what this offering is.
• siteRequirements: Interfaces that this offering requires from the site store.
• appPowerups: Powerups to be installed on the app store.
• installablePowerups: A list of 3-tuples, (name, description, item class). Each of these tuples describes a powerup that can potentially be installed on a user store.
• loginInterfaces: This allows your offering to provide additional ways of logging into Mantissa. For example, a webmail offering might provide interfaces that are used when logging in via SMTP or POP3.
• themes: The themes system provides a way to provide and override the templates used by your widgets.
• staticContentPath: A path pointing at the root of your static content. The file hierarchy at this location will be served up by Mantissa at /static/YourOfferingName.
• version: A Version object specifying the version number of your offering.

For more information, take a look at xmantissa.ixmantissa.IOffering, which documents each of these attributes in more detail. The main highlights here, though, are appPowerups, for the powerups you want installed on the app store, and installablePowerups, for powerups you want to be available for installation on the user store.

To set up a Mantissa site, the first step is to run axiomatic mantissa; this will prompt you for an admin password as part of installing the base Mantissa components. To start the site up, you run axiomatic start; usually you would pass the -n flag during development, which prevents the server from daemonizing, and directs log output to the console instead of to a log file. If you log in as the admin user (admin@localhost by default) via the web interface, you can install an Offering through the admin UI; alternatively, you can install offerings from the command line by running axiomatic offering install SomeOfferingName.

When your offering is installed, the powerups in appPowerups will be installed onto the app store for your offering. The installablePowerups you provided will not be installed anywhere immediately, instead they will be exposed in the admin UI for products. A product is an administratively-defined collection of powerups; the admin can select any of the installablePowerups from any of the installed offerings when creating a product. Products are used in conjunction with signup methods, which are also locally configured by the admin. A signup has a product associated with it; when a user signs up using this method, all of the powerups that comprise the product will be installed on their user store.

Whew! This may all seem like more indirection than is actually necessary, but if you view Mantissa as a layer that multiplexes between users and applications, it may make a little more sense.

Next up: Powerup interfaces, or “How does the user actually interact with my code?”

This is the second post in a series of articles about Mantissa.

As you may have already surmised, Mantissa is designed to host more than one application in the same deployment. Due to the way Mantissa builds on Axiom, it is important to first understand the data model, before starting to understand how the components of your application plug into Mantissa.

A Mantissa deployment, or “node”, consists of a top-level Axiom store called the site store, and various substores contained therein. The substores fall into two categories; user stores, and application stores. The site store primarily contains IService powerups for the various services running in the node (think HTTP server, SMTP server, etc.), and the substore items for the user / app stores. It also contains the IRealm implementation for authentication (using twisted.cred, naturally), along with account / login method information for each user account, as well as other configuration data such as API keys (think PayPal API, or similar).

The user stores are where your actual application data will be stored in a typical Mantissa application. Each user has their own Axiom user store (SQLite database), where data for that user is stored. The account / login method items for that particular user are also duplicated in the user store, to allow user stores to be transferred between site stores. Items in different user stores should not interact with each other, except via the interstore messaging API. This forms the core of Mantissa’s scaling model; while each individual user’s data must all be located on the same node, different users may have their data stored on different nodes, allowing you to spread your users across as many nodes as you need to in order to achieve the desired performance.

Application stores are sort of a special kind of user store; internally, they’re treated as a user store with a username corresponding to the name of the Offering (offerings will be discussed later), and a domain of None. Each application installed on a Mantissa node will have an application store; exactly what to put in there isn’t entirely clear, the best I can come up with is “application-global data”.

Mantissa has a system called “sharing”, whereby access to items in one user’s store can be granted to other users, or even anonymous public access granted. In the case of “websharing” (which works over HTTP), the items are accessed using a URL that specifies which user’s data is being accessed, as well as the identifier of the particular item. For example, the URL might look like http://myapp.com/mithrandi/timeline or http://mithrandi.myapp.com/timeline (“mithrandi” is the username, “timeline” is the share ID).

For a user-centric application (like Google Docs or Dropbox), this works pretty well; if the data in your application is primarily of global scope, then this may not be a good fit at all. There are some ways around this: firstly, not every “user” needs to be an actual live user. Thus, you can divide your data across “artificial” user stores, and share it to the users that need to have access to it, retaining Mantissa’s scalability in the process. For “global” data access or operations, you can send a message to each user store to perform a certain query or operation, and then collate the results; this model of “send the computation to the data” is similar in some ways to MapReduce.

However, if your application truly does not fit into the model, then one possibility is to store the application data in a completely separate service, using Mantissa only for configuration and authorization. The fact that Mantissa stores certain data in Axiom does not prevent you from making queries to an external PostgreSQL database, or Cassandra database, or web service, or whatever else you can come up with; it just means that you derive no specific benefit from Mantissa’s database functionality.

Another serious caveat is that while the conceptual model and APIs for Mantissa scalability are mostly in place, the actual implementation of scaling is not there at all. There is currently no networked implementation of the interstore messaging API, there are no tools for managing a cluster of Mantissa nodes, distributing or migrating user stores, backup/replication of user stores, and so on. The foundation for building these tools and components exists, but at the moment, you’ll need to build them yourself (or pay someone to do it for you) if you need them. Thus, if you are looking for “Facebook size” scalability straight out of the box, Mantissa is probably not for you. On the other hand, depending on your needs, building the tools you need as you grow your application may not pose a significant challenge, so if you’re not afraid of writing a little code, don’t be scared off right away by this. You could also avoid relying on Mantissa for scalability, by storing your application data in another datastore that already meets your scalability needs

Next up: Offerings, or “How do I expose my application to Mantissa?”.

This is the first post in a series of articles about Mantissa; a Python application server built on Twisted, Nevow, and Axiom.

“Application server” is a somewhat fuzzy term that seems to mean different things to different people; but in essence, Mantissa serves as a layer that joins your actual application code to the various ways (different protocols, for example) that your application may be accessed. Or, to put it another way, when writing a Mantissa application, the deployment target is Mantissa, not “a web server”, even if your application is only going to be accessed from the web. That’s not to say that Mantissa completely abstracts away the details of the protocol, making it impossible to control how your application is presented; rather, it provides common ground to mediate that interaction. It provides mechanisms for the common portions of functionality, such as access control, data storage, authentication and authorization, and also makes provision for defining standard interfaces through which your application is exposed to the various protocols you may wish to make use of.

This series is not going to be of the “build your own wiki in 60 seconds” kind. While there are some quirks and rough patches in Mantissa that may make it not the best choice for hacking together an application really quickly, it is probably still something that is doable; but unfortunately, I’m not the right person to be writing that sort of article. What I hope to do, instead, is provide a more in-depth introduction that allows you to understand and embrace (or reject!) the design philosophy of Mantissa for longer-term projects, while noting incomplete areas and other potential pitfalls along the way that may need to be avoided.

This is also not going to be an Axiom, Nevow, Python, or Twisted tutorial; while I will touch on aspects of those components as they relate to Mantissa, you will need to look elsewhere for further detail.

That’s it for now, except to thank Jeremy in passing for nagging me about this until I got around to doing it (or starting it, at least).