Is Mandela dead yet? Find out now!

UPDATE: Site is dead, so link removed.

Jonathan just ran into this on his new HTC Desire (updated to 2.2 “Froyo”); I haven’t confirmed it on any other handset, but I imagine the problem is common to many HTC Sense + Android handsets. Anyhow, I figured I’d put this information out there to help anyone that runs into this issue. If anyone has a contact at HTC, please pass this along, so that they can look into a fix/workaround.

When “Facebook for HTC Sense” tries to connect to Facebook to retrieve contacts or profiles, the client sends Expect: 100-Continue in the HTTP header. Facebook’s HTTP servers do support 100 Continue, but some proxy servers (such as squid) do not. If you have configured your phone to use a proxy server, or are behind a transparent proxy (which many ISPs do inflict on their users), and the proxy server does not support this, it will (correctly) respond with 417 Expectation failed response. At this point, HTC’s client does not attempt to retry the request without requesting 100-Continue, but instead fails with an error message. This results in an empty Facebook directory / contacts list, and an error when you attempt to refresh it.

If the proxy server is on your own network, simply bypassing it should solve the issue for you. If this is your ISP’s proxy server, then you’ll either need to work around it somehow, or contact HTC and/or your ISP for assistance.

I’m a little bored, and the whole internet (well, at least the portion of it composed of Desire owners) seems to be going crazy waiting around for the recently-announced rolling out of the Android 2.2 update for the HTC Desire, so I thought I’d poke around a little in the OTA update process.

UPDATE (2010/08/03): The update seems to be rolling out to most people, but some people still aren’t getting it, even though they have unlocked/unbranded phones. From what I’ve been able to discover, HTC appears to be employing an IMEI “blacklist” of some kind; checkins with specific IMEI values fail to return the 2.2 update, but anything else (including completely invalid IMEI values) returns the update. I’m still not sure whether this is just their way of staggering the update, or if there is some other reason for withholding the update from the affected devices.

UPDATE (2010/08/05): HTC now seems to either have a bad content server, or be in the process of pulling the update from their servers, including the update file itself on liveimage.htc.com. If it is deliberate, I have no idea why they’re doing it, and I can’t currently find any information that sheds any light.

The process seems to be relatively straightforward: when you hit “Check now”, the phone sends an HTTP POST to http://andchin.htc.com/android/checkin (not https, thus all in the clear, making it easy for me to sniff :P), with a Content-Type of “org/x-json” (wat), and a JSON-encoded payload. The payload looks as follows (I’ve pretty-printed it and stripped personal identifiers, but it’s still good enough to get a response):

[javascript]

{

"locale": "en_GB",

"checkin": {

"build": {

"product": "bravo",

"serialno": "HTXXXXXXXXXX",

"changelist": "174215",

"build_type": "user",

"carrier": "htc_wwe",

"radio": "32.36.00.28U_4.06.00.02_2",

"firmware_version": "1.21.405.2 CL174215 release-keys",

"bootloader": "0.80.0000",

"id": "htc_wwe/htc_bravo/bravo/bravo:2.1-update1/ERE27/174215:user/release-keys",

"revision": "129"

}

},

"model_number": "HTC Desire",

"logging_id": 0,

"imei": "",

"id": 0,

"digest": "8fccfd93dcbe837d072e5f1494da2"

}

[/javascript]

Note the “carrier” value there; this is “htc_wwe” for phones that are not carrier-locked / otherwise carrier-customized. If you had a Sprint phone, for example, this would be “sprint”. I believe this value is ultimately determined by the the “androidboot.carrier” kernel parameter passed to the kernel by the boot loader.

I’m not sure what “digest” is of; it is 29 hex digits long, which doesn’t make any sense for a cryptographic digest. It doesn’t seem to be a personal identifier of any kind, though; if you modify it, you get some “upload crash” response back, which contains (among other things) the correct digest, so I can only guess that it corresponds to the product information in some way. I’ll be interested to see if it changes once I have the new update installed.

The response that comes back looks like this:

[javascript]

{"time_msec":"1280538364433","stats_ok":true,"intent":[{"action":"android.server.checkin.FOTA_CANCEL"}]}

[/javascript]

The “intent” part will probably be obvious to Android developers (it’s specifying what the UI should do next), and time_msec is a UNIX timestamp in milliseconds, corresponding to the time the response was sent.

If an update *is* available, the response looks like this instead:

[javascript]

{

"stats_ok": true,

"intent": [

{

"action": "android.server.checkin.FOTA_UPDATE",

"data_uri": "http://liveimage.htc.com/OTA_Bravo_HTC_WWE_1.21.405.2-1.15.405.4_release_P4ohim1mtp6zggcyon.zip",

"extra": [

{

"name": "promptMessage",

"value": "System Upgrade"

},

{

"name": "promptFeature",

"value": "Update:HTC application improvement"

},

{

"name": "promptVersion",

"value": "System upgrade1.21.405.2 (27.07 MB)We would recommend using a free Wi-Fi hotspot or an unlimited data plan to apply this update. If not, standard data connection charges may apply. Any questions? Contact us via http://www.htc.com/www/CA_Hotline.aspx"

},

{

"name": "promptSize",

"value": "27.07 MB"

},

{

"name": "promptMinutes",

"value": "30,..."

},

{

"name": "timeoutSeconds",

"value": "120"

},

{

"name": "force_update",

"value": "false"

}

]

}

],

"time_msec": "1280513282008"

}

[/javascript]

Now, it seems as if they are planning to do a rolling release of the 2.2 update, somehow. I assume this means that they will only be responding with the updated version to some requests, but I don’t know exactly what mechanism they’ll be using to do that. Oh well…

Encumbered forever by desire and ambition,

there’s a hunger still unsatisfied;

our weary eyes still stray to the horizon,

though down this road we’ve been so many times.

— Pink Floyd, High Hopes

Introduction

As some of you already know, I recently became the proud new owner of an HTC Desire. You can find the detailed specs elsewhere, but the phone is pretty much a Nexus One with an optical trackpad, real buttons, a little extra flash memory, and the HTC Sense software. This post isn’t exactly going to be an in-depth review, but I thought I’d give a brief rundown of my impressions, and also the apps I’m using on the phone at the moment.

Overall, I’m pretty happy with the phone. I can get 1-2 days of use between charges (depending on usage), which is good enough for my purposes, and that does include push email (quite a lot of email, in the case of my personal account) and syncing all sorts of stuff on a pretty regular schedule. The phone UI is generally way more responsive and snappy than my previous phones (Nokia E71, E65), and the touch screen works reasonably well. Also, I can actually get apps for a lot of the services I use now, whereas I couldn’t before (not many people are making Series 60 apps).

Input

The lack of a real keyboard turned out to be not as bad as I expected. Tap-typing with a single finger is not as fast as using both thumbs was on the E71, and holding the phone so that I can use both thumbs is somehow awkward enough that I never did it. Selecting predicted words is a lot easier with the touch screen, though (no need to scroll through the list). However, I am now using the Swype Beta, an alternative input method where you “swype” your finger through the letters of the word you want, without lifting it, along with a lot of other nice tweaks to the on-screen keyboard. It works pretty nicely; it doesn’t always select the right word, but you can finish your sentence (or paragraph) and then go back and select a different word where it made mistakes. If a word isn’t in the dictionary, or isn’t really a word at all, you can just tap it out; regular words will be added to the dictionary after you press space, but you can also add “non-words” like “http://” or “.co.za” for convenience. I’m fast enough on the Swype keyboard now that I can use it for taking notes during a conversation in real-time, which is great. Unfortunately, the beta is closed at the moment; if you didn’t have a chance to grab the app while it was open, you’ll have to wait around until they decide to let people have a crack at it again, or they open it up to everyone.

The multi-touch screen is great. A lot of apps (like Maps, or the browser) support the “pinch” and “spread” (what’s the real name for the opposite of pinch?) to zoom in and out, which easily lends itself to an “arbitrary precision” navigation style: zoom out to cover a lot of ground, then zoom back in to precisely select a button or other page element. The “momentum-based” scrolling mechanism (again, what the heck is this actually called?) is also used by most apps, and is similarly awesome. It sort of behaves like a globe: flick the screen to start scrolling; over time, it slowly slows down, but you can touch the screen to “catch” it once you get to where you want to be, or flick it again to keep going.

Multi-tasking

The Android approach to multi-tasking is awesome, and I kinda wish my desktop operating systems worked the same way. Under the hood, it’s basically just standard Linux processes being scheduled by the kernel, but with a bit of a twist. In terms of the UI, there’s no explicit process management. Some apps have an “exit” option, and you can burrow into the Applications list to “Force Stop” any app, but there isn’t even any built-in way to get a list of currently-running apps (although third-party apps for this purpose are available), simply because you don’t really need it. When the OS runs out of memory (think “OOM killer” if you’re familiar with Linux), it kills off the oldest apps to free up memory. In addition, provision is made for apps to save and restore their state. As a result, if you switch back and forth between apps, they’ll probably still be running, so switching will be immediate. If you open up an app that you haven’t used for a while, it may not be running anymore, so it’ll take a little longer (but usually not that much longer) for the app to start up again and restore the state. Most apps don’t do anything if they’re not in the foreground, so there’s no need to worry about wasting your battery running apps you’re not using. There are also “services”, which are for long-running background tasks that shouldn’t be killed off to reclaim memory; synchronization, for example.

Unfortunately, some people seem to be confused by all of this; as a result, there are a plethora of “advanced task killer” apps and suchlike on the Android market. These are all just about pointless; all you are likely to accomplish by running around killing random processes is breaking certain functionality like background synchronization that you turned on. If you want to stop your contacts syncing, for example, just go turn that off in the settings, instead of using an app that kills the process every time it starts up. Also, if an app really is draining your battery in the background, it’ll show up in the battery info screen, and you can then uninstall it or otherwise take steps to solve the problem. The only exception to the latter is built-in functionality that gets aggregated into the “Android Platform” entry; it may not be obvious what you need to disable or reconsider if “Android Platform” shows up as using too much juice.

Intents

Think of intents like file type associations on steroids. Instead of just “open this file which is of some type”, you also have “open this URL” and “dial this phone number” and “view these map coordinates” and “search for this text” and “display the home screen” and “the battery is running low” and just about anything else you can think of, including brand new intents created by third-party developers.

Since any application can register itself to handle an intent, this means that third-party applications can do anything from hooking in as a phone dialler (eg. a SIP client), to replacing the home screen; just about any aspect of the “default” OS software can be replaced. This can also be done selectively; for example, if you follow a link to a Google Maps URL (even from another web page in the browser), you’ll be given the option to open it in the “Maps” app as well as opening it in the browser.

The net result is that third-party software can replace and enhance the OS in a practically unlimited fashion; and different third-party apps can interoperate and integrate without a great deal of special effort being exerted on the part of the application developers.

GPS / Location

The Desire has A-GPS functionality. My E71 also supposedly had this functionality, but it never seemed to work; getting a GPS lock could take up to 5 minutes if I started the Ovi Maps app up in a different location to where I last closed it. On my Desire, under optimal conditions (outdoors, clear view), I can generally get a GPS lock in seconds, no matter how far I’ve moved since the last time I got a lock. I can even get a GPS lock in under 30 seconds standing indoors in the middle of my house! I guess this is probably a combination of a better A-GPS implementation and a better GPS receiver, but I’m definitely much happier with the results.

Battery lifetime

A few quick notes I wanted to make on battery lifetime. Firstly, the battery lifetime seems to go up after the first several charge-discharge cycles; I guess this probably applies to all batteries in this class, but I thought it was worth mentioning. Secondly, active wifi use generally uses less battery than an active cell data (GPRS/EDGE/3G/HSDPA) connection; but an idle wifi connection will use more battery than an idle cell data connection. You can adjust the wifi “sleep” policy to take advantage of this by disabling the wifi connection after a certain timeout, although you may wish to avoid the extra cell data charges. Also, if you are in an area with poor cell reception, the cell radio will use more battery as the transmit strength is increased in order to successfully transmit; this applies even if you are not making use of cell data.

My applications

You can see a list of all of the applications I have installed (thanks to AppBrain), but not all of them are worth mentioning, so I’ll run through the interesting ones here, with some commentary. Unfortunately paid apps are not currently available from the Android Market in South Africa, which some apps are either not available at all, while others are only available in free+ads form.

• Aldiko Book Reader

  A pretty decent eBook reader for ePub format books. It includes support for purchasing books from some sites (they have a mechanism whereby any online store can integrate their catalogue); otherwise you can download and import them from anywhere you can find them, or copy them onto the phone from your PC.

• Android Agenda Widget

  This is a ridiculously customizable home screen widget for viewing your calendar and tasks. It has support for “styles”, but you can also individually customize just about every aspect of the widget. It also provides widgets in almost every every possible size from 1x1 up to 4x4. It supports the various calendars (Google, HTC, etc.) as well as Astrid and some other tasks app that I can’t remember offhand.

• AppBrain App Market

  The name is a little misleading: this is not a separate market, but just a way to sync and share your installed apps list, as well as scheduling installs / uninstalls via the web interface (which will be performed the next time you sync using the phone app).

• Astrid Task/Todo List

  A nifty tasks list manager. You can schedule multiple reminders for tasks, tag them, track expected time / time spent, and sync with Remember The Milk.

• ASTRO File Manager

  Android does not come with any kind of file manager, although you won’t need one very often since apps tend to avoid explicit file management. Still, there is the occasional need to rename or move a file around; it can also be very useful to transfer files over the network, especially with the SMB module.

• Barcode Scanner

  Scan a variety of barcodes, including QR codes and UPC / EAN barcodes. Many sites provide download / market links for apps in the form of QR codes; scanning the code from another display is obviously easier than retyping the URL.

• Dropbox

  Nothing much to this app; it just lets you view / download files you have stored on Dropbox.

• Evernote

  Create and access your Evernote notes, including geotagging support.

• Foursquare

  Check in on Foursquare, get alerts when your friends check in nearby, view nearby venues, and search for venues.

• FoxyRing: Smart Ringtone

  I’m not actually using this app currently, but you may find it useful regardless. The app allows you to adjust your ringtone settings based on ambient noise levels, time of day, and location, as well as overriding them for selected contacts.

• Glympse

  Unlike Google Latitude, which shares your location to all of your contacts the whole time you have it active, Glympse provides more limited and granular sharing. When you create a “Glympse”, you specify a lifetime (eg. 30 minutes) after which the Glympse expires, and optionally start and destination locations; a unique code is then generated. You can then share the URL by text message or email, or you can post it on Facebook. The recipients can then view your location in realtime, until the Glympse expires, either when the specified time has elapsed, or by you manually expiring it at any time of your choosing.

• Google Goggles

  This app is described as a “visual search” application. You take a picture of anything, and the application attempts to recognise it. This might mean scanning a barcode / QR code, scanning and OCRing text (which you can then translate or search for), detecting a well-known landmark, a work of art, and more.

• GPS Status

  A simple app that allows you to visualize the GPS satellites in range, how many satellites you are locked onto, and see other information from the GPS receiver.

• Kindle for Android

  Amazon’s Kindle application. Great for reading Kindle books, but if you can get them in ePub, rather use Aldiko (see above), as the Kindle app is not nearly as featureful.

• Nimbuzz

  This is an IM / voice chat app. It has support for Skype, which is useful, as Fring no longer has Skype support, and the official Skype Android app is currently only available to Verizon subscribers through a special deal. It also supports a variety of IM networks, including Facebook, but unfortunately generic Jabber support is no longer available.

• Real Signal

  A simple app that gives you a configurable cell signal strength meter, as well as allowing you to see the actual numerical signal strength.

• Remember The Milk

  Only usable by RTM Pro subscribers (although you get a 14-day trial of Pro if you aren’t a subscriber). This lets you sync your RTM tasks to your phone, create tasks on your phone, and includes a widget for displaying tasks. Not strictly necessariy in conjuction with Astrid (see above), but I find it useful for managing additional task views (eg. my shopping list).

• ScrobbleDroid

  Just what it sounds like: scrobble your music activity to last.fm. Compatible with various music players.

• Shazam

  Want to know what song is currently playing? Capture an audio sample via the phone’s microphone with this app, and then search their online database to identify the song.

• Shopper

  This app overlaps with Google Goggles somewhat (see above). You can scan media (DVDs and CDs) or books by barcode or cover image, and then search for them online with Google Shopper; useful for doing some comparative shopping when you’re in a physical store.

• SpeedView

  A simple speedometer app that uses the GPS to track your speed.

• TripIt

  Sync your TripIt travel plans to your phone; allows for offline access, so you can get your important travel details even if you can’t get an internet connection from where you are.

• Wifi Analyzer

  Visualize the wifi networks in range, including the channel and signal strength. Useful for selecting an unused channel for your wifi network, or picking an internet access point.

This is the third post in a series of articles about Mantissa.

The primary code entry point in Mantissa is the offering. Your application can provide an IOffering plugin in xmantissa.plugins, which hands some chunks of your code over to be used by Mantissa. There is no real reason to implement IOffering yourself; instead, simply instantiate xmantissa.offering.Offering in your plugin module. For example, you might have an xmantissa/plugins/ponies.py module containing the following code:

ponies.py (Source)

from twisted.python.filepath import FilePath

from xmantissa.offering import Offering

import ponies
from ponies.ponies import PonyCreator
from ponies.rainbows import RainbowEditor
from ponies.theme import PoniesTheme

plugin = Offering(
    name=u'Ponies',
    description=u"""
    A web-based system for generating ponies.
    """,
    siteRequirements=[],
    appPowerups=[],
    installablePowerups=[(u'ponies', u'Pony creation and editing', PonyCreator),
                         (u'rainbows', u'Rainbow editing', RainbowEditor)],
    loginInterfaces=[],
    themes=[PoniesTheme('base', 0)],
    version=ponies.version,
    staticContentPath=FilePath(ponies.__file__).sibling('static'))

There are a number of elements here, some of which we will only return to later, but I'll run through all of them here briefly.

• name: A short identifier used as the name of this offering.
• description: A long description explaining what this offering is.
• siteRequirements: Interfaces that this offering requires from the site store.
• appPowerups: Powerups to be installed on the app store.</li>
• installablePowerups: A list of 3-tuples, (name, description, item class). Each of these tuples describes a powerup that can potentially be installed on a user store.
• loginInterfaces: This allows your offering to provide additional ways of logging into Mantissa. For example, a webmail offering might provide interfaces that are used when logging in via SMTP or POP3.
• themes: The themes system provides a way to provide and override the templates used by your widgets.
• staticContentPath: A path pointing at the root of your static content. The file hierarchy at this location will be served up by Mantissa at /static/YourOfferingName.
• version: A Version object specifying the version number of your offering.

For more information, take a look at xmantissa.ixmantissa.IOffering, which documents each of these attributes in more detail. The main highlights here, though, are appPowerups, for the powerups you want installed on the app store, and installablePowerups, for powerups you want to be available for installation on the user store.

To set up a Mantissa site, the first step is to run axiomatic mantissa; this will prompt you for an admin password as part of installing the base Mantissa components. To start the site up, you run axiomatic start; usually you would pass the --nodaemon flag during development, which prevents the server from daemonizing, and directs log output to the console instead of to a log file. If you log in as the admin user (admin@localhost by default) via the web interface, you can install an Offering through the admin UI; alternatively, you can install offerings from the command line by running axiomatic offering install SomeOfferingName.

When your offering is installed, the powerups in appPowerups will be installed onto the app store for your offering. The installablePowerups you provided will not be installed anywhere immediately, instead they will be exposed in the admin UI for products. A product is an administratively-defined collection of powerups; the admin can select any of the installablePowerups from any of the installed offerings when creating a product. Products are used in conjunction with signup methods, which are also locally configured by the admin. A signup has a product associated with it; when a user signs up using this method, all of the powerups that comprise the product will be installed on their user store.

Whew! This may all seem like more indirection than is actually necessary, but if you view Mantissa as a layer that multiplexes between users and applications, it may make a little more sense.

Next up: Powerup interfaces, or "How does the user actually interact with my code?"

People new to working with event-driven programming in general, and event loops in particular, often seem to have a hard time figuring out how to run their code when it needs to be run. For example, the event loop in Twisted is called the reactor (or perhaps more accurately, the event loop is part of the reactor). The reactor can be started and stopped, but the catch is that the current implementation only allows you to start one reactor once per process. Once you’ve stopped it, it can’t be started again. Now, this is really just an unfortunate consequence of two historical design artifacts; the reactor is a global singleton, so you can only have one, and it doesn’t manage the process-global state well enough to be able to start again once it has stopped. However, while it would be nice for those two issues to be fixed, the reality is that the usual answer to “how do I stop and start the reactor more than once?” is “you don’t need to, and you don’t want to.”

The idea of an event loop is, roughly, “don’t call us, we’ll call you”. When an event occurs, the event loop takes care of dispatching that event to the relevant code in order to handle the event. While there may be some layers of library / framework between your application code and the low-level event handler, ultimately it’ll usually be some part of your application code that gets invoked in response to the event. This means that instead of writing your program in the style of “do A, then do B, then do C”, you write it in a way that looks more like “when X happens, do A; when Y happens, do B; when Z happens, do C”.

So, how do you kick things off? If you fire up an event loop without doing anything else, generally no events will ever occur / be received, and so your program will just run forever doing absolutely nothing. Thus, in the startup phase of your program before you start the event loop, you will need to perform some operations that will cause events to be received later on. For example, an HTTP server would probably want to open a socket and start listening on port 80; alternatively, you might have some time-based event, along the lines of “run this code every 30 seconds”. The startup phase is not the only time that you can set your event sources up; you can also do this in any of your application code that handles other events that occur. For example, your web server might open a connection to a database server when it receives a request for a certain URL.

That’s really all there is to it; figure out which event sources you need to set up intially before you run the event loop, and figure out what other events will occur during the lifetime of your application that would require setting up additional event sources, no matter what those event sources might be (making connections, starting timers, completion of a prior operation, etc). While this post may be slightly Twisted-specific, the concepts are pretty general: any decent event loop will necessarily allow both cases (initial setup, and additional operations while the event loop is running). The key is to think of your application as code run in response to events, not just a linear series of operations.

This is the second post in a series of articles about Mantissa.

As you may have already surmised, Mantissa is designed to host more than one application in the same deployment. Due to the way Mantissa builds on Axiom, it is important to first understand the data model, before starting to understand how the components of your application plug into Mantissa.

A Mantissa deployment, or “node”, consists of a top-level Axiom store called the site store, and various substores contained therein. The substores fall into two categories; user stores, and application stores. The site store primarily contains IService powerups for the various services running in the node (think HTTP server, SMTP server, etc.), and the substore items for the user / app stores. It also contains the IRealm implementation for authentication (using twisted.cred, naturally), along with account / login method information for each user account, as well as other configuration data such as API keys (think PayPal API, or similar).

The user stores are where your actual application data will be stored in a typical Mantissa application. Each user has their own Axiom user store (SQLite database), where data for that user is stored. The account / login method items for that particular user are also duplicated in the user store, to allow user stores to be transferred between site stores. Items in different user stores should not interact with each other, except via the interstore messaging API. This forms the core of Mantissa’s scaling model; while each individual user’s data must all be located on the same node, different users may have their data stored on different nodes, allowing you to spread your users across as many nodes as you need to in order to achieve the desired performance.

Application stores are sort of a special kind of user store; internally, they’re treated as a user store with a username corresponding to the name of the Offering (offerings will be discussed later), and a domain of None. Each application installed on a Mantissa node will have an application store; exactly what to put in there isn’t entirely clear, the best I can come up with is “application-global data”.

Mantissa has a system called “sharing”, whereby access to items in one user’s store can be granted to other users, or even anonymous public access granted. In the case of “websharing” (which works over HTTP), the items are accessed using a URL that specifies which user’s data is being accessed, as well as the identifier of the particular item. For example, the URL might look like http://myapp.com/mithrandi/timeline or http://mithrandi.myapp.com/timeline (“mithrandi” is the username, “timeline” is the share ID).

For a user-centric application (like Google Docs or Dropbox), this works pretty well; if the data in your application is primarily of global scope, then this may not be a good fit at all. There are some ways around this: firstly, not every “user” needs to be an actual live user. Thus, you can divide your data across “artificial” user stores, and share it to the users that need to have access to it, retaining Mantissa’s scalability in the process. For “global” data access or operations, you can send a message to each user store to perform a certain query or operation, and then collate the results; this model of “send the computation to the data” is similar in some ways to MapReduce.

However, if your application truly does not fit into the model, then one possibility is to store the application data in a completely separate service, using Mantissa only for configuration and authorization. The fact that Mantissa stores certain data in Axiom does not prevent you from making queries to an external PostgreSQL database, or Cassandra database, or web service, or whatever else you can come up with; it just means that you derive no specific benefit from Mantissa’s database functionality.

Another serious caveat is that while the conceptual model and APIs for Mantissa scalability are mostly in place, the actual implementation of scaling is not there at all. There is currently no networked implementation of the interstore messaging API, there are no tools for managing a cluster of Mantissa nodes, distributing or migrating user stores, backup/replication of user stores, and so on. The foundation for building these tools and components exists, but at the moment, you’ll need to build them yourself (or pay someone to do it for you) if you need them. Thus, if you are looking for “Facebook size” scalability straight out of the box, Mantissa is probably not for you. On the other hand, depending on your needs, building the tools you need as you grow your application may not pose a significant challenge, so if you’re not afraid of writing a little code, don’t be scared off right away by this. You could also avoid relying on Mantissa for scalability, by storing your application data in another datastore that already meets your scalability needs

Next up: Offerings, or “How do I expose my application to Mantissa?”.

This is the first post in a series of articles about Mantissa; a Python application server built on Twisted, Nevow, and Axiom.

“Application server” is a somewhat fuzzy term that seems to mean different things to different people; but in essence, Mantissa serves as a layer that joins your actual application code to the various ways (different protocols, for example) that your application may be accessed. Or, to put it another way, when writing a Mantissa application, the deployment target is Mantissa, not “a web server”, even if your application is only going to be accessed from the web. That’s not to say that Mantissa completely abstracts away the details of the protocol, making it impossible to control how your application is presented; rather, it provides common ground to mediate that interaction. It provides mechanisms for the common portions of functionality, such as access control, data storage, authentication and authorization, and also makes provision for defining standard interfaces through which your application is exposed to the various protocols you may wish to make use of.

This series is not going to be of the “build your own wiki in 60 seconds” kind. While there are some quirks and rough patches in Mantissa that may make it not the best choice for hacking together an application really quickly, it is probably still something that is doable; but unfortunately, I’m not the right person to be writing that sort of article. What I hope to do, instead, is provide a more in-depth introduction that allows you to understand and embrace (or reject!) the design philosophy of Mantissa for longer-term projects, while noting incomplete areas and other potential pitfalls along the way that may need to be avoided.

This is also not going to be an Axiom, Nevow, Python, or Twisted tutorial; while I will touch on aspects of those components as they relate to Mantissa, you will need to look elsewhere for further detail.

That’s it for now, except to thank Jeremy in passing for nagging me about this until I got around to doing it (or starting it, at least).

There are already any number of blog posts, wiki pages, and so on spread all over the internet that cover this topic. However, all of them seem to be based on outdated information, making the instructions and configuration involved more convoluted than necessary. So, without further ado, here's the easy way to set 6to4 up on Debian.

First, you will need to calculate your 6to4 IPv6 address prefix, which is based on the IPv4 address of the host you are using as your 6to4 router. For example, if your router's public address is 10.10.10.1 (this is NOT actually a public address):

$ ipv6calc --action conv6to4 10.10.10.1</code>
No input type specified, try autodetection...found type: ipv4addr
No output type specified, try autodetection...found type: ipv6addr
2002:a0a:a01::

Add 1 to the end of this to obtain your router's address, 2002:a0a:a01::1 in this case. Next, you will need to add an entry for the tunnel to /etc/network/interfaces:

auto tun6to4
iface tun6to4 inet6 v4tunnel
    address 2002:a0a:a01::1
    netmask 16
    gateway ::192.88.99.1
    endpoint any
    local 10.10.10.1
    ttl 255

Replace 2002:a0a:a01::1 with your IPv6 router address, and replace 10.10.10.1 with your public IPv4 address; 192.88.99.1 is the anycast address for the nearest 6to4 gateway, so leave that alone unless you know what you're doing. You can now bring the tunnel up with ifup tun6to4, and you should have IPv6 connectivity.

UPDATE: Derek points out this handy site in the comments that will calculate some of the above for you automatically.

Your 6to4 prefix is a /48, allowing you to allocate 2 ^ 16 (65536) /64 subnets below this. In the usual case of a small home / business network, you won't need more than one of these, so just pick one to use for your network. For example, if we pick DEAD, the network prefix would be 2002:a0a:a01:dead::/64. You can manually assign addresses to the hosts on your network, but it will probably be easier to do EUI64-based autoconfiguration; this allows each host to automatically select an address based on their MAC address when they receive a router advertisement. In order to send router advertisements, you will need to install radvd, and then put something like the following in /etc/radvd.conf:

interface eth0
{
    AdvSendAdvert on;
    prefix 0:0:0:DEAD::/64
    {
        AdvOnLink on;
        AdvAutonomous on;
        Base6to4Interface ppp0;
    };
};

Replace eth0 with the name of your network interface; this is the interface on which router advertisements will be broadcast. You could hardcode your 6to4 prefix, but it's more convenient to use the Base6to4Interface option to have radvd calculate it for you; replace ppp0 with the interface for your public internet connection, and the prefix will be altered accordingly. If your public internet connection is not on a separate interface, then just remove this option, and replace the prefix address with the full address as shown earlier.

Your hosts should now have performed EUI64-based autoconfiguration and configured a public IPv6 address for themselves, unless you have disabled this for some reason. If you need to manually calculate the auto-configured address for a particular host, you can do so given its MAC address:

$ ipv6calc --action prefixmac2ipv6 --in prefix+mac --out ipv6 \
    2002:a0a:a01:dead::/64 11:22:33:44:55:66
2002:a0a:a01:dead:1322:33ff:fe44:5566/64

Replace the prefix and MAC address with your own, of course. If IPv6 privacy extensions are enabled, this address will be assigned to the network interface, but an additional temporary anonymous address will be assigned based on a randomly-generated identifier. The temporary address will be used for outgoing connections, thus avoiding exposing your MAC address to every host you connect to; the permanent address can still be used for incoming connections, allowing you to use this address in DNS entries and so on. Privacy extensions will typically be disabled by default on Linux-based hosts, and enabled on Windows hosts.

There is usually no real reason to disable privacy extensions; however, there is another feature enabled by default on Windows hosts that you may wish to disable. This feature randomizes the identifier used for the permanent address, separately from the temporary addresses assigned by privacy extensions. The randomly generated identifier should be persisted, so the address will not change, but it will bear no relation to the MAC address, thus preventing you from being able to calculate it. If you wish to disable this feature, run the following command with Administrator privileges:

netsh interface ipv6 set global randomizeidentifiers=disabled

If you also wish to disable privacy extensions, you can use the following command, but note that this is not necessary if you just want persistent EUI64-based addresses:

netsh interface ipv6 set privacy disabled