if(dojo.storage.manager.isInitialized() == false){
  dojo.event.connect(dojo.storage.manager,
                     "loaded", Moxie,
                     Moxie.initialize);
}else{
  dojo.event.connect(dojo, "loaded",
                     Moxie,
                     Moxie.initialize);
}

var results = dojo.storage.get(key);

try{
  dojo.storage.put(key, value,
                   saveHandler);
}catch(exp){
  alert(exp);
}

var saveHandler = function(status, keyName){
  if(status == dojo.storage.PENDING){
     // ...  
  }else if(status == dojo.storage.FAILED){
     // ...
  }else if(status == dojo.storage.SUCCESS){
     // ...
  }
}

try{
  dojo.storage.put(key, value,
                   saveHandler);
}catch(exp){
  alert(exp);
}

See the automatically generated API reference

Requires

• dojo.lang
• dojo.flash
• dojo.event

Provides

dojo.storage

A local durable cache

dojo.storage.manager

Manager class that autodetects available storage and instantiates best one; can be used for lower level control

Contributors

Brad Neuberg, bkn3@columbia.edu - Module maintainer and creator
Alex Russel, alex@dojotoolkit.org - Created early version of Dojo Storage
JB Boisseau, jb.boisseau@eutech-ssii.com - Created WHAT WG storage provider for Dojo Storage

FAQ

Help! Dojo.Storage Isn't Working!

Some possibilities:

• You must have the following SWF files in the same directory as your dojo.js file: flash6_gateway.swf, storage_dialog.swf, Storage_version6.swf, Storage_version8.swf
• Make sure your web server is returning the correct MIME type for SWF files: application/x-shockwave-flash swf
• You are on an unsupported browser. The following are supported: IE 6+, Firefox, Safari.
• You have a really old version of Flash, or no Flash, and don't have permission on your computer to upgrade software (perhaps you aren't the administrator account and don't have permission to install Flash?)
• You're being served from an https domain, where there is a known bug right now that needs to be fixed (I can't fix it because I don't have an SSL/HTTPS environment setup; any volunteers?)
• Your event listener for the Dojo Storage Manager is incorrect. See below.

It's very important that your code is right for listening to when storage is ready to be used. Your event listener must be at the top level and executed while the page is loading. Dojo.Storage writes thing out into the page during page load that won't work afterwards (i.e. it uses document.writes). Study the source code for the Moxie example storage application carefully before asking questions; compare your code and see what is different.

How Do I Write a Storage Provider?

See the FlashStorageProvider for an example. Basic steps:

• Find the file that corresponds to your render environment; for example, if your provider will run in the standard web browser environment, then you will want to put your storage provider into src/storage/browser.js. If a file does not exist for your render environment yet, such as a Konfabulator storage provider, then you would create a new file corresponding to your environment, such as src/storage/konfabulator.js or src/storage/xpcom.js. All the providers for a given render environment must be in the same source file. If you have added a new file for a new environment, make sure to add the existence of this environment to src/storage/__package__.js so it is loaded for your new environment.

• Subclass dojo.storage:
  

  dojo.inherits(dojo.storage.browser.FlashStorageProvider, dojo.storage);

• Implement the dojo.storage methods, such as put() and get(). You MUST also implement the isAvailable() method in a solid way, since this method will be called by the Storage Manager to see if your provider is available in the given environment.
• Tell the Storage Manager about our existence at the bottom of your source file; remember to order these registrations in the order you want storage providers to be chosen. For example, if you have three storage providers, such as a What WG provider, a Flash provider, and a proprietary Internet Explorer storage provider, and you want these to be preferred in this order, you would have the following at the bottom of your source file:

  // register the existence of our storage providers
  // choose What WG provider first
  dojo.storage.manager.register("dojo.storage.browser.WhatWGStorageProvider",
  				new dojo.storage.browser.WhatWGStorageProvider());
  // if not available, use Flash
  dojo.storage.manager.register("dojo.storage.browser.FlashStorageProvider",
  				new dojo.storage.browser.FlashStorageProvider());
  // if still not available, prefer IE method
  dojo.storage.manager.register("dojo.storage.browser.InternetExplorerStorageProvider",
  				new dojo.storage.browser.InternetExplorerStorageProvider());

• Initialize the Storage Manager:
  

  // now that we are loaded and registered tell the storage manager to initialize
  // itself
  dojo.storage.manager.initialize();

How is Dojo Storage Related to AMASS?

AMASS, or the Ajax Massive Storage system, is an older, proof-of-concept version of Dojo.Storage written by the same author as Dojo Storage, Brad Neuberg. Dojo.Storage replaces AMASS. It is more full featured, better tested, and is supported across more browsers than AMASS. AMASS will no longer be supported.

I Heard Dojo Storage Can Be Used For Offline Access. How?

For offline, Julien Couvreur discovered the necessary HTTP headers. The core part of it is to use HTTP caching; your site must have the following kinds of HTTP headers on:

• Etag
• Last-Modified
• Expires
• Cache-Control

Etags and Last-Modified are on by default on Apache 2.0; the rest have to be turned on in httpd.conf with mod_expires:

LoadModule expires_module modules/mod_expires.so

<Directory "c:/dev/dojo/">
  ExpiresActive On
  ExpiresDefault "access plus 1 month"
</Directory>

The page is now in the browser cache after the first access. In IE and Firefox, the user has to go to File > Work Offline to work offline, and then can simply just navigate to the app's URL. In Safari, this is not necessary, and you can just go to the URL. I like to provide a link that can be dragged to the toolbar.

The offline and dojo.storage work together, because whether you are offline or online you can access the same persistent storage, saving data while offline then syncing when online. Expect a dojo.offline and dojo.sync in the future that will provide abstractions for common operations like this. I'm looking for financial sponsors on this if you are interested so that I can finish and open source it (email me).

The Dojo.Storage presentation goes into offline access as well:

http://codinginparadise.org/weblog/2006/05/new-slides-from-ajax-experience-talk.html

You should note that offline support is alpha, since there are issues with cache clearing.

Who created and supports Dojo Storage?

Dojo Storage was created by Brad Neuberg and is supported by him. Please refer all questions to the Dojo mailing lists before emailing Brad directly, as the email load is getting large. Brad might charge to answer your question, since he does Dojo Storage for free and makes money through open source consulting.

How Can I Help With Dojo Storage?

There are two big ways you can help Dojo Storage. The first is by creating new storage providers; a nice one would be for Internet Explorer's proprietary storage mechanism.

The other big way to help is with squashing bugs, especially with the https bug that affects Dojo Storage, since I don't have a testing environment for it. See a list of all Dojo Storage bugs.

Email Brad Neuberg if you are available to help with either of these.

Why Did You Use Flash?

Why Flash? Flash now has a greater installed base than Internet Explorer; Flash 6+ has a 97.1% penetration across the installed base of PCs, while IE 5, 6, and 7 have 64.7%. Flash is probably one of the most installed pieces of software on the planet.

The Flash Storage Provider uses Flash as a hidden runtime to extend the browser, because of its ubiquity and cross-platform/cross-browser qualities. When browsers get native persistence support, your dojo.storage applications will continue to run since you write them against the generic dojo.storage APIs.

What Is The Security of Dojo Storage?

The Flash Storage Provider has exactly the same security characteristics as cookie based storage. Data is siloed by domain, so other domains can't access this data. The data is stored on the local disk, usually in the user's home directory.

When working with Dojo Storage and it's Flash Storage Provider you should take exactly the same care you would take with cookies -- i.e., don't leave yourself open to Cross Site Scripting Attacks (XSS).

What Is The Maximum Amount of Storage I Can Do?

Using the Flash Storage Provider, this breaks down into three questions:

1. How much total data can I store? If the user has given you permission, there should be no limit. I have successfully stored up to 10 megabytes, and just stopped there.
2. How much data can I send over in one put()? For Internet Explorer on Windows and Firefox on all platforms, I can send over about 700K or 1 megabyte in a single put() and the performance is fine; however, for Safari on Mac, you can really only send over about 300 to 500K in one put() with acceptable performance. There will be a difference between saving a string object, which is much faster, than serializing a large JavaScript tree, which might get very slow if you are working with a set of JavaScript objects that are hundreds of K.
3. What are the total number of records I can have? What if I have hundreds of thousands of keys? I have never tested this, so am not sure where it will fall over.

For the WHAT WG provider, there is actually currently no upper-limit on the amount of storage a site can store! The WHAT WG spec recommends 5 megabytes, but Firefox implements no upper-limit. This is obviously a bug and will be fixed by Firefox.

What About A Query Layer For Dojo Storage?

In 2005 sent out a call for someone to layer over the very cool Trim Path Java Script SQL layer over AMASS (an earlier version of Dojo Storage), and someone took up the call and created a prototype linking them together. You should check it out; its very cool. No one has done this with Dojo Storage, but it shouldn't be hard. This might be very useful for offline Ajax apps.

What About Syncing?

This is a hard problem that is difficult to find general solutions for. I am looking for financial sponsorship to create and open source this. Email me if you are interested.

I Don't Care About Dojo Storage or Dojo Flash - How Can I Turn Flash Off In My App?

If you are using a 'kitchen sink' build of Dojo, you will get Dojo Storage and Dojo Flash included by default. This will cause the Dojo Flash material to be written into the page. If you don't want this to happen, you can disable all of Dojo Flash with the following djConfig variable:

var djConfig = { disableFlashStorage: true; }

How Does The Underlying Dojo Flash System Work?

I'll briefly describe dojo.flash here, which is the layer that seperates out JS and Flash communication and which is used by the Flash Storage Provider. It might be useful for folks who are encontering any issues with the storage system due to Flash.

Cross-browser, fast, reliable JS+Flash communication is really hard and ugly, so I encapsulated this portion out into it's own layer. The great thing is you don't have to know any of this externally; dojo.flash and dojo.storage work together to figure out your Flash capabilities, and use the appropriate mechanism internally. Zero hassle.

Dojo.flash provides several major services:

• dojo.flash.Info - Is Flash available + what version of Flash?
• dojo.flash.Embed - Embeds Flash into page for Flash+JS communication
• dojo.flash.Communicator - Provides uniform, fast, reliable, JS + Flash communication
• dojo.flash.Install - Uniform installation and upgrading of Flash

dojo.flash.Communicator was the real doozy to create; it was a pain in the butt, to put it lightly. This area of the system provides a method abstraction between Flash + JS. For example, JavaScript can call sayHello(), which is some Flash method, while Flash can execute DojoExternalInterface.call("dojo.storage.save", resultsHandler) to run some JS method.

The heart is something called DojoExternalInterface, which is a backport of Flash 8's ExternalInterface to Flash 6. I didn't want to create this backport, but the complexity of handling all the internal tricks I was doing to make this stuff work required that I wrap this magic in some kind of maintainable API. DojoExternalInterface makes it possible to register Flash methods that can be called from JavaScript:

DojoExternalInterface.initialize();
DojoExternalInterface.addCallback("put",
                                 this, put);
DojoExternalInterface.addCallback("get",
                                 this, get);
DojoExternalInterface.addCallback("remove",
                                 this,
                                 remove);
DojoExternalInterface.loaded();

There are three ways to do Flash+JS communication:

1. Using LiveConnect/ActiveX + fscommands - Flash 6
   Pro: Extremely fast, can send very large data, mature
   Con: Only works on IE and Firefox
2. Using ExternalInterface - Flash 8
   Pro: Easy to use, Works on Safari
   Con: Unbelievably slow, performance degrades O(n^2), serious serialization bugs
3. getURL/LocalConnection/New Flash object for each call - Flash 7
   Pro: Very cross platform
   Cons: Destroys history, serious data size limitations and performance issues

Only methods 1 and 2 are acceptable for the Flash Storage Providers needs. It turns out that we use method 1 for IE and Firefox, and method 2, ExternalInterface, for Safari. I found workarounds to fix ExternalInterface's serious bugs, so that it works on Safari and is fast and serializes correctly, but these workarounds only work on Safari.

The Flash 8 communication support for Safari took 3 months to figure out; pain in the behind. I won't go into detail on the method and workarounds, but here is a bit of info on the workarounds:

• We chunk method call data into many different small calls through ExternalInterface,
  which makes performance linear rather than O(n^2), which is what the standard ExternalInterface's performance is.
• I used a debugger to find hidden JS serialization methods used by the Flash plugin, which internally uses XML serialization and made all sorts of mistakes in terms of serializing characters - for example, it doesn't escape characters and also uses eval() so its slow as dirt. I found a way to bypass this internal serialization, do it all manually, and make the method call myself using an undocumented JS function.
• I now double encode and decode all XML characters on both sides:
  & --> && This is very important for persisting XML, and I did lots of testing around this to catch every single problem character (including nulls, for example)

With these workarounds, performance and reliability on Safari are great. Unfortunately, these workarounds only work on Safari, so we can't use ExternalInterface cross browser.

Check out this testing page, which provides a lower level interface to dojo.storage; there are quick links on the left to fully save an entire book into the storage system, in this case Faust by Goethe at 250k courtesy of Project Gutenberg; there is also a quick link to save an example RSS XML feed into the storage, in this case an atom feed from my weblog. Try this on Safari; in the past it took minutes and froze the machine to save the book, now it takes seconds.

I won't go into the Flash 6 communication necessary for IE and Firefox support; there were lots of fun different things that had to be figured out, like how to center the Flash dialog even if you are across many different kinds of HTML doctypes, browsers, and platforms, or isolating timing issues like the fact that the fscommand infrustructure on some versions of IE comes up after your Flash applet is already running.

Can I Force a Certain Storage Provider to Be Used? Can I Disable Certain Storage Providers?

Several djConfig variables exist to help you control what storage provider is used. The first is called forceStorageProvider, and will cause the Dojo Storage system to automatically use the given provider, even if the platform doesn't support it or a better one is available:

djConfig = {forceStorageProvider: "dojo.storage.browser.WhatWGStorageProvider"};

Two other djConfig flags exist: disableWhatWGStorage, which can be true or false and which will disable WHAT WG storage and fall back to using a different one; and disableFlashStorage, which can be true or false and which will disable using the Flash Storage Provider. The last flag is useful if you want to make sure that Flash is never used, for example if you don't want to have Flash touch your codebase.

Note that the forceStorageProvider and disableWhatWGStorage/disableFlashStorage flags are mutually exclusive; you should choose to use EITHER forceStorageProvider or disableWhatWGStorage/disableFlashStorage. Things won't work correctly if you try to mix the 'force' and 'disable' flags.

I'm Having Wierdness with the FileStorageProvider on Internet Explorer 7!

On Internet Explorer 7, XMLHTTPRequest does not work on the local file system; this breaks Dojo's package loading, which uses XHR. To use the FileStorageProvider on IE 7 with your code, make sure to use a release build of dojo.js (i.e. the one that packs everything into a single dojo.js file), with preferably the 'storage', 'moxie', or 'kitchen_sink' profiles to make sure everything got packed into there and won't be loaded dynamically at runtime by XHR.

When loading from file:// URLs, the user is prompted for permission to run secure code; what's up?

On Firefox, the first time you load a page that uses Dojo Storage (and therefore the FileStorageProvider), you will be prompted on whether to give this page permission to access the local file system. If the user approves, you will not be asked again. On IE 6 and 7, you will also be prompted, but will be prompted each time the page is loaded. After you have given permission the page will work normally.

Can I Use Dojo Storage With Other Non-Dojo, JavaScript Libraries?

Dojo Storage can be used with other JavaScript libraries, such as TIBCO or Prototype. This FAQ question will provide pointers to sites where others have integrated Dojo Storage with their own libraries.
Luke Birdeau has integrated Dojo Storage with the TIBCO GI library. See details on the TIBCO forum.

About

Author:	Brad Neuberg
Version:	0.5
Copyright:	Dojo Foundation, 2006
Date:	2006/11/18