No doubt, the massed choirs of Windows 95 users are
singing from the rooftops, because Office 95 under Windows 95 allows long
filenames. They think this is going to be of so much more use to them; 8+3 was a
dreadful limitation, even 15 years ago.
But what narrow thinking! Going from an 8+3-character filename to a
256-character version of the same thing is progress only in as much as it's a
move from disastrous to meagre on the usefulness scale.
Let's look at it another way. If you were to take all the documents on
your desktop
computer-or, better still, your corporate LAN-and put them into one directory,
would 256-character filenames really be enough? No, of course not! You'd still
want the filtering, sorting, and collating facilities offered by subdirectories.
But as soon as you put documents into subdirectories, you're scattering your
data into the unknown far reaches of your hard disks, never to be seen again in
a hurry, because the retrieval tools are so weak and rigid.
Indeed, to look at it from still another perspective, think about a database
design. You would definitely want a unique key-or a unique combination of
keys-on a database entry. Allowing multiple entries to have the same master key
is a recipe for disaster. Suggesting that a database manager scatter data
randomly across multiple tables so you can sort and query them more easily would
undoubtedly end in a major argument. The database manager would argue that all
you need is to add another field or two to each record to uniquely define it.
Now spin that mindset into a disk-filing system, and you can see how even a
256-character filename strategy is pathetically weak. You need unique document
identifiers and multiple "database-alike identity fields" to
categorize your data. Hence, even 256-character filenames are a largely useless
solution for the late '90s, especially since gigabytes of disk space storing
hundreds of thousands of files cost mere hundreds of dollars today.
Not surprisingly, Microsoft has a strategy in place to sort this mess out. It's
called Cairo. For what seems like an eternity, Microsoft has been proclaiming, "We
are all on the road to Cairo." Indeed, this future, still unseen release of
Windows NT has been coming for so long that it has gained almost mythical
status. Private discussions with Microsoft have clearly indicated that Cairo
hasn't vanished, that work has been steadily progressing over the last couple of
years, and that 1996 is going to be the year of the grand roll-out of Cairo-at
least initially-in beta form.
So, I thought it would be useful to explore the current state of play to see
where this mythical Cairo technology is coming from. Hopefully, by the end of
this journey from Office to Cairo, you'll agree that large components of Cairo
have been sitting on your desktop in full view for nearly two years.
Of course, there are many facets to a large project like Cairo. Although I can't
speculate now about how all of them will work, one clear thread is worth
pursuing: how the object-storage system works today-how it might well be
developed for Cairo and how it will help solve our data storage and retrieval
problems.
Office and Storage
The questions must have crossed your mind: When you embed an Excel worksheet in
a Word document and then save the document, how does Word "know" how
to store the Excel file? Where is the Excel file stored? And if it's stored in
the Word file, is it converted to a Word table?
Since you can reload the Word file and then double-click on the Excel file to
bring up an in-place editing instance of Excel with all the editing and
formatting capabilities of Excel supported, then the Excel worksheet must be
stored natively within the Word document. These functions simply wouldn't be
possible if Word converted the Excel table to a Word table.
If you embed a different kind of document, such as a PowerPoint slide, the same
logic applies: The native PowerPoint presentation is stored within the Word
file. Indeed, Word has no native concept of a presentation so it's hard to
conceive of how such a Word-centric conversion could work.
Normally, this is the stuff of private data formats. It would immediately
suggest that Microsoft had written some private Office data format that could
store any data file from any Office application. That would make complete sense,
except for one thing: You can embed any data file from any Object Linking and
Embedding (OLE) 2-supporting application in a Word file. The implication is
clear. This "embed and store" mechanism can't be relying on private
formats, because you can embed objects from applications that have no connection
with Microsoft. Thus, adding a Visio drawing is no more difficult than bringing
in an Excel worksheet.
There must be a standardized, open-storage, container-and-content model at work
here, something that any vendor can tap into. There is! It's called Structured
Storage, and it's been around since the release of OLE 2.0.
Structured Storage is fascinating for many reasons, not the least of which is
that so few people have actually looked carefully at how it works. Fortunately,
Microsoft has tools that let you look inside a Structured Storage file to see
what's going on. The best tool to use is DFVIEW.EXE, which is a Structured
Storage DocFile viewer supplied with Microsoft Developers Network (MSDN) Level
2. In addition, there's a good book on OLE 2 by Kraig Brockschmidt entitled
Inside OLE, Second Edition from Microsoft Press. The chapter "Structured
Storage and Compound Files" is detailed and informative, although as a
whole the book is a deeply technical reference that warrants several readings.
To illustrate how Structured Storage works, let's look at a basic Word 7 file. I
created a new Word document, put two words into it, and saved it to C:\tempword1.doc. I then loaded this file into DFView (see Screen 1).
You can see that it shows a hierarchical layout with the top level being the
storage file itself. This is the file on the disk. At the next level, there are
four streams of data: The first is called Comp-Obj for Compound Object; the
second is called WordDocument and contains the actual Word document data. The
third and fourth are a SummaryInformation stream and a
DocumentSummaryInformation stream. These last two contain summary information
tags like Author, Date Created, Size, Last Printed, and so on. Each of these
four streams is a separate data stream in the Structured Storage file.
Having closed the DFView display of the file and reopened it in Word 7, I
inserted a simple two-cell, two-row Excel 7 worksheet into the Word file, using
the Excel button provided on the Word button bars. Then, I resaved word1.doc and
reopened it in DFView (see Screen 2).
Things are definitely becoming interesting. There are still the same four Word
object streams in place, but there is also a new stream, called ObjectPool. If
you open the ObjectPool stream, you'll see another storage object with an
obscure numbered label. If you then open this object, you'll see what looks like
another complete document stream, just like the one at the root of the tree.
There are the same CompObj, SummaryInformation, and DocumentSummaryInformation
streams. But the WordDocument stream has been replaced by a Book stream. This
contains the Excel Book worksheet information-the Excel worksheet object that
was inserted into the Word document. In addition, there are a couple of extra
streams here to render the object when its container is running. In other words,
when you are editing the Word document, you want to see a visible representation
of the contents of the Excel object, even though Excel isn't running at that
point.
Let's take this one step further and put a WordArt object inside the Excel
object, which is inside the Word object. Phew, my head is spinning. How about
yours?
Screen 3 shows what's happening-I have in-place activated the Excel object, and
then inserted the WordArt 2 object into the Excel object. You can't currently
in-place edit more than one level deep-OLE 2 doesn't support multiple nesting of
all the negotiation that goes on between the various applications. So, although
Word will negotiate with Excel to bring up the Excel "look and feel"
in the Word framework, it can't negotiate with WordArt to go two levels deep.
Let's look at this triple-layer sandwich using DFView to see the result of this
multiple embedding (see Screen 4). Not surprisingly, there is a WordArt object stored inside the Excel object, which is stored inside the Word object.
What Does It Mean?
OLE 2 containers can contain objects of completely foreign formats, and as far
as the container is concerned, the contents of the embedded object are opaque.
Word doesn't know-or need to know-how Excel stores its internal data. Nor does
Excel need to know how WordArt stores its data. They simply need to know how to
contain them, and this is defined in the Structured Storage standard.
Although I only inserted one object into another, there's no limit on the number
of objects you can insert in parallel. For example, I could have put two Excel
worksheets, each containing different objects, into the Word document. The
resulting tree structure is easy to imagine. Just think of Structured Storage as
an object filing system. The similarities between what DFView has revealed and
what's in a standard directory/subdirectory/file structure are plain.
Into a Binder
Office 95 includes a new application called Binder. It allows you to take Office
documents and put them into a composite file together. Binder embodies the
concept of putting the contents of a folder (or a directory) into a single
object. You can then switch among the various objects in a binder, and Binder
does all the in-place editing activation necessary using OLE 2.
How does it work? Binder uses Structured Storage for its file store. For
example, I created a new Binder object and dragged and dropped the composite
Word-document object into the binder. Then I created a new Excel object in the
binder (see Screen 5). Look carefully at this structure. It has two main Binder
objects, called 1 and 2. The first is the Word document with the Excel and
WordArt embedded objects. The second is the Excel sheet I dropped into the
binder. Finally, look at the standard Binder type at the top level, together
with its SummaryInformation and DocumentSummaryInformation streams.
Are you starting to see a pattern? The reality is: With Structured Storage,
Microsoft has been running file-sized full-blown object filing systems. Indeed,
a Structured Storage file could be regarded as a full-blown storage engine in
itself-a sort of prototypical Cairo Object Filing System! This is why, when some
corporate IT managers call Microsoft's ability to create an Object Filing System
into question, I point out that there are already hundreds of millions of
documents out there that use Structured Storage and they work just fine, thank
you. Microsoft has been using this storage engine in Office ever since version
4.x.
Binder is important, because it's the first application from Microsoft that
works "one layer out" from the main applications. Binder is a
container of Word, Excel, PowerPoint, etc. objects. As such, it is effectively
an active container surface into which you can pour data objects to work on
them. Imagine the Windows 95 shell allowing direct Binder-type operations, and
you can see how a Cairo active desktop might well be a container system itself.
Near Future
Now, let's make a leap sideways to a forthcoming piece of software from
Microsoft-Exchange Server (see "Groupware: The PC Team Sport" on page
33). As you may know, Exchange Server replaces Microsoft Mail and brings, at
last, a full Messaging Applications Programming Interface (MAPI) to the desktop
in a proper client/server email system.
However, Exchange Server has more important tricks up its sleeve. It has Public
Folders, which automatically replicate data among Exchange Server engines around
your enterprise. If you wish, you can use Public Folders to store email. But
it's far more potent to drop Structured Storage files in them. Just fire up File
Manager (or Explorer, if you're running the Windows 95 shell), and drag and drop
a pile of Word, Excel, and PowerPoint documents into a Public Folder. Exchange
Server swallows up the files and stores them within its active storage engine
(see Screen 6).
Now the magic can really start. Look carefully at the columns in the Folder
view-no great surprise. There are columns detailing From, Subject, Received, and
Size. However, after a little fiddling with the Folder columns settings, I've
made dramatic changes to the information shown (see Screen 7). Exchange Server
is "surfacing" the properties applied to each object-in this case,
Manager, Client, Value, and Authorised. Manager is a standard property held in
the Summary tab of the file Properties window in each Office application. The
other three are custom properties that I added to each object (see Screen 8).
Since Exchange Server is storing these documents in an active repository, they
can be replicated and distributed around the organization. I can attach full
security rights to the Folder and its contents. But most important, I have
completely removed the importance of the filename from the object. No longer do
I need to refer to an object only by its filename.
Although Windows 95 and NT have full 256-character filenames for files, that's
far from enough. Length is not the issue; the granularity and the number of
attributes that can be attached to an object is the real key. For example, if I
want to mark all the files belonging to Project X, I don't want to have to put
Project X into each filename. It would be far better to have a Project Name
attribute attached to each object. And Structured Storage, Office, and Exchange
Server allow me to do this. These tags are stored in the SummaryInformation and
DocumentSummaryInformation streams. Each Structured Storage Office-compatible
application has these streams, even for subobjects in a compound embedded
object.
Power Exchange Operation
For anyone interested in a really powerful document management system, Screen 9
should raise the hair on the back of your neck. Exchange Server has, at my
request, sorted and collated the objects in this Public Folder by the Manager
property and presented the results in a tree diagram. You can collate up to four
levels deep, so there's no problem displaying a tree of customers, where each
customer's information is sorted by "Invoice Paid? Yes/No" and then by
date, or something like that.
If you attended the December 1993 Microsoft Windows "Chicago"
Professional Developers Conference in Anaheim, California, you'll probably
remember the awe-inspiring demonstration of Cairo given on the first day. The
sorting and collating I've described was demonstrated as a Cairo feature.
On to Cairo
At this point, I must make it clear that I'm dropping out of the realm of the
known and into the realm of informed speculation. However, what I'm describing
is logical and consistent with the above.
Exchange Server uses a modified Jet database engine for its storage. It's known
that Cairo has a native Object Filing System (OFS) based on the NT File System
(NTFS) disk format. NTFS is a very flexible storage engine and has many features
similar to those of a transactional database engine. For our purposes, the most
important feature is that each file object can have an unlimited number of
properties attached to it. (I don't want to go into too much detail on NTFS.
Suffice it to say, Helen Custer's Inside the Windows NT File System from
Microsoft Press is required reading.)
Reading Custer's book and matching it up to the Exchange Server and Structured
Storage information, coupled with discussions with senior Microsofties, leads me
to believe that NTFS will be extended under Cairo. It will appear to be
backwardly compatible with the current NTFS, but it will have a whole new set of
active processes running on top of it. In other words, the Jet database-style
engine of Exchange Server will be replaced by the Cairo OFS, based on NTFS.
NTFS-based OFS has everything you need: It can hierarchically store objects; it
can index on properties; it can store Custom properties; and it is robust and a
proper filing system. For example, when you save a Word file, it talks to the
local OLE storage engine running on your machine. This engine creates the
Structured Storage disk file, and Word pours its contents into the file. The OLE
libraries manage all the file handling.
When you store documents in Exchange Server, Word talks to the OLE storage
engine which talks to Exchange. Word basically pours its contents into the
Exchange Server engine, again via the OLE libraries. No major changes were
necessary to allow Word to do that. In the Cairo future, the OLE libraries will
talk to the Cairo OFS store, and Word will pour its data into that instead.
What more will Cairo uncover? Exchange Server only sees the outer layer of the
onion. It doesn't look inside the Structured Storage object. It pulls out the
SummaryInformation, but embedded objects within the document are still hidden.
Cairo will actually break a Structured Storage document into its component
parts. This will allow it to display all the components in the document and to
show all the standard and custom Summary tags for each object too. Remember,
each object in a compound document has its own set of tags.
This revealed structure will enable multi-user document creation and editing.
Since Cairo, by virtue of its NTFS host, will be able to do database-style
record locking on each individual component of the Structured Storage document,
it will be possible, for example, for several users to work on different parts
of a compound report, enabling much finer document indexing and retrieval
processes. After all, a list of "hot words" on each object is just
another set of custom property tags that can be individually indexed, sorted,
and collated in the Cairo OFS storage engine.
The Path Is Clear
By focusing on how Structured Storage works and the benefits it brings, you can
see the direction that Microsoft is taking and how Exchange Server is a big step
along that road. Ignore all the email parts of Exchange Server, and look
closely-it's really Cairo 0.75.
It's also important to realize that Cairo won't be just a server-side solution.
To operate at its best, it will need client-side applications that understand
both OLE and Structured Storage. It's not surprising that Microsoft's own Office
product has been doing precisely this for the past two years.
I'm told that there will be a clean and simple upgrade path from Exchange Server
to Cairo itself, when Cairo is ready to ship. Indeed, Exchange Server 2 will
probably be fully Cairo-hosted. This is a sweet victory for Cairo, because all
the indications are that Exchange Server 1 was never really supposed to happen.
According to all the time scales and published plans, we should be running on
full Cairo by now.
The delays in Cairo-and in getting NT 3.5 out the door-mean that Microsoft had
no choice but to put together an interim solution. Exchange Server 1 is that
solution. And although it may be an interim solution, it offers some extremely
interesting technologies for the future that you can put to work today. The
exodus from the filename starts now.