[odf-discuss] Miguel on OXML

[marbux]

On 2/1/07, Alex Hudson <alex at stratagia.co.uk> wrote:
> Pamela Jones wrote:
> > http://www.robweir.com/blog/2007/01/more-matter-with-less-art.html
> >
> > He answers Miguel ably.
>
> That's slightly in the eye of the beholder; I think he misses many of
> Miguel's points (particularly, what Miguel said on formulas has been
> completely misunderstood).
>

Miguel should have alerted his readers that he was touching on an area
that has been highly controversial and better distinguished his point
from all the disinformation Microsoft has been pushing on the formulas
issue. In the same article that echoed a fair number of highly
debatable Microsoft talking points, I think it's understandable that
folks would misunderstand a point that he didn't develop. Brevity is
not invariably the right approach.

> As I understand what the Foundation are doing with their plugin, this is
> basically the approach they've taken: they translate to some level of
> ODF, but then everything else is essentially Microsoft-specific tags
> added in. More or less, OXML on top of ODF.

Less. They go directly between ODF and the in-memory binary
representation of the document, using Word's facility for native
support of file formats. The tags they are using are the optional tags
from the ODF spec's section 1.5 for handling of foreign elements. They
aren't Microsoft-specific tags, although my understanding is that
adding compatibility with the Microsoft binary formats was a strong
consideration in the section's addition to the spec. But the
procedures in that section are generic, not vendor-specific. E.g.,
Corel is undoubtedly using them to add ODF support to WordPerfect.
What the Foundation's plugin does has absolutely nothing to do with
OXML.

>
> Whatever you think of that specific approach (it's very much how
> Microsoft output HTML, for example, with their mso- additions in CSS),
> it does make OXML less necessary. I would also love to see ODF take the
> better parts of OXML too (as much stick as it gets, the spreadsheet part
> of OXML is by expert opinion _very_ well designed - Microsoft know Excel
> users extremely well).

That depends on your criteria. If viewed as a specification for a
mature application's implementation of a particular flavor of XML plus
binary blobs, it has some merit (although there are still major
warts).

But if viewed as a software standard, it sucks mightily because it is
only a specification for compatibility with a particular vendor's
product, the replication of the Lotus 1,2,3 bug in implementation of
the Gregorian calendar that was deliberately replicated in Excel being
a rather good example of just how poorly EOOXML was designed as a
standard.  There was no effort to enable full implementation by other
vendors. Indeed, there is a mountain of evidence (6,039 pages) that
blocking full implementation by other vendors was a primary design
goal. We are not talking about an open standard here; we are talking
about yet another vendor lock-in strategy from Microsoft disguised in
a standard's clothing.

>
> I personally think we may have missed a trick with the contradictions
> stuff: I suspect we're sending ISO the wrong message. I do think,
> though, all the materials that have been put together will be _very_
> useful in the next stage.
>

Agreed. I will be pleasantly surprised if we win the contradictions
phase. Microsoft will have a much more difficult time avoiding
discussing Ecma 376's warts in the final vote on adoption as an
international standard.

> One problem that Microsoft have, that Miguel correctly identified, is
> that they've shipped Office 2007. That means they have _very little
> room_ in which to resolve ballots.
>

If by "they" you mean Microsoft, I agree with your second sentence.
But if you mean the folks who are supposed to be looking at Ecma 376's
technical merits to decide whether to assert contradictions, there is
plenty of room. I regard Miguel's relevant statement as important
recognition that a single vendor's application tail has improperly
wagged the standards dog, producing a proposed standard that would if
adopted grant a single vendor a monopoly on migrating Microsoft Office
legacy documents to its own vendor lock-in flavor of XML plus binary
blobs.

Miguel deserves stern rebuke for his failure to to call to his
readers' attention that vendor-specific file format specifications are
ineligible for the status of open international standards reveal. He
points to an enormous deficiency of Ecma 376 without alerting his
readers to a very serious failing of Ecma 376, painting it as a reason
for the standards national bodies not to do their jobs. He's a smart
and knowledgeable guy who has the trust of a lot of folks in the open
source community. His article exploits that trust rather than honoring
it.

> I don't know if we've understood the Microsoft position well enough, but
> I wonder if their backs are somewhat against the wall here.

More than somewhat. It is widely accepted that the Microsoft Windows
and Office monopolies are highly interrelated and that the Windows
monopoly fails if the Office monopoly fails. The Office monopoly fails
if ODF becomes the dominant file format standard for office
productivity software. As the existing ISO international standard, ODF
usage is legally compelled for governments and the trickle-down from
government adoption will displace Microsoft's control over the private
sector as well. If Microsoft can not achieve ISO international
standard status for Ecma 376, Microsoft is out of the business of
supplying Office to governments unless it fully supports ODF. And if
it does, the Microsoft Office monopoly is broken.

Microsoft clearly has its back to the wall and Ecma 376 is a
last-ditch effort to maintain its monopoly businesses and to extend
those monopolies into the emerging but rapidly-growing business
processes software market where applications act as routers of
information rather than end points. Reasonable minds can differ, but
in my opinion Microsoft is trying with Ecma 376 to migrate its major
vendor lock-in point from Microsoft Office to its new Sharepoint and
Exchange integrated server hubs. Those free retrofits coming in a few
days for earlier Office versions to read and write MS Office Open XML
(plus binary blobs) mark the first time in my memory that Microsoft
has enabled both backward and forward compatibility of its Office
applications. Up to now, Office has offered only backward
compatibility. This is a major change in Microsoft's Office Upgrade
Treadmill marketing plan that has produced untold billions of dollars
in monopoly profits for Redmond.   Any reasoned analysis of what
Microsoft is up to with Ecma 376 needs to take that change into
account.

These are
> the logical outcomes:
>
>    1. OXML passed, as-is, with few or minor changes (ODF only required
>       "editorial" changes - and even that was a stretch - I'm not sure
>       OXML will escape so easily)
>    2. OXML passed, but with "significant" changes
>    3. OXML not passed
>
> Look at it from Microsoft's point of view. 3. is obviously not a great
> outcome for them; but is 2. preferable? If OXML was passed, but in a
> format that was basically incompatible with Office 2007, that would be a
> *huge* problem. I think the choices open to Microsoft are more or less 1
> or 3. That helps us immensely.
>

Agreed, but there is an implicit 4th option to add to the list from
Microsoft's standpoint. It's the one Microsoft is trying to avoid.

4. Abandon Ecma 376 and fully implement ODF.

This is only my opinion, but is based on a lot of study of the
situation. Microsoft's crystal ball failed initially. Its managers
either ignored its lawyer's advice or its lawyers were asleep at the
wheel. It's pretty plain that, while Microsoft did oppose ODF's
adoption as an ISO standard, the company did not devote major
resources to lobby down the ISO standardization of ODF.

To me, that is rather compelling evidence that Microsoft's managers
were unaware of the full significance of ODF's adoption as an
international standard. Two separate treaties now legally compel ODF's
adoption by governments as part of their national standards. Because
of government's huge influence as crossroads for communications, the
trickle-down effect of national file format standards into the private
sector are enormous.

>From the circumstances, it is pretty plain to me that Microsoft wasn't
aware of the impact of the treaties until we began publicizing it and
Massachusetts and several E.U. national governments rejected
Microsoft's own XML formats in their national government software
procurement processes. Up that point, it's plain that Microsoft had
proceeded on the assumption that standardization was not a significant
relevant factor.

Ecma 376 is Microsoft's response. The message Microsoft sends with it
goes something like this:

"OK, we will play your silly little standards game. But no way, no how
are we going to let go of our control over our existing customer base.
We locked those customers in fair and square and nobody else gets to
sell software to them. So if you want to migrate all those documents
stored in our secret binary formats to XML, the only way we will allow
it is to our own flavor of XML plus binary blobs. Just ignore the fact
that the new formats are vendor lock-in formats. You have no choice
but to give us our own personal international standard so we satisfy
the legal requirements. So rubber stamp Ecma 376 or forget about
migrating to XML"

That is why the OpenDocument Foundation's work is of such critical
importance. It says in effect to the world's governments, "you don't
have to give in to Microsoft's extortion. You can migrate to ODF
without Microsoft's cooperation. You can reject Ecma 376 as an
international standard"

There has been a lot of confusion on this list about the Foundation's
da Vinci plug-in design goals. While full interoperability remains a
goal for later versions that depend on changes to the ODF
specification being made in version 1.2, full interoperability between
Microsoft Office and conformant apps designed to the ODF 1.0
specification is not a feasible goal as I understand it.  What is
feasible is full fidelity in migrating Microsoft Office binaries to
ODF 1.0, migration without data loss. Foreign elements that can not be
mapped are preserved. That means, for example, that if a migrated file
comes up with incorrect formatting or some such problem, the file can
be reopened in Microsoft Office for repair work or viewing because the
unmappable elements are preserved in ODF, wrapped in
<foreign></foreign> tags.

The ability to preserve the data is of critical importance to
governments. They operate under legal restrictions requiring that all
information be preserved. Lossy file conversions to ODF are not an
option for them. The Foundation's work is the only work I am aware of
that attempts to fulfill this market requirement for ODF. To my
knowledge, no one else is working on full fidelity migrations from the
Office binaries to ODF.

If that market requirement is fulfilled, then governments can lawfully
migrate their files to ODF and build their future documents in ODF
from the git-go. The records will be preserved. They can keep a few
copies of Microsoft Office for the times when they need to view or
print documents in their original formats.

If that market requirement is not fulfilled, then there is an enormous
 legal barrier to governments migrating to ODF.

I am told by the Foundation's developers that, once we have
applications that are conformant with ODF v. 1.2, full
interoperability is an achievable goal. It will be feasible to map
every Office feature to ODF. Until that time, ODF files generated in
Microsoft Word using the da Vinci plug-in will not be fully compatible
with, e.g., OpenOffice.org and KOffice.

The 1.0 version of the Foundation's plug-in was optimized for full
interoperability of ODF files generated by MS Office with OOo. Such
files are not fully compatible with Microsoft Office's feature set. At
the request of the Commonwealth of Massachusetts, the da Vinci version
is optimized for compatibility with Microsoft's binary file formats.
My understand is that when OOo becomes ODF v. 1.2-conformant, full
interoperability between OOo and MS Office can be implemented.

But until then, I'm told that only full fidelity (no data loss) can be
achieved, not full interoperability.  I think it would aid discussion
on this list if we could clearly delineate in our discussions of the
Foundation's work between "full fidelity" and "full interoperability."
The latter necessary implies "full fidelity" as well, but in the
present context, "full fidelity" does not necessarily imply "full
interoperability," e.g., "full fidelity" will often be a subset of
"full interoperability." We need to be clear whether we are discussing
one or the other concepts if we wish to achieve clear communication.

All of the above is based on information derived from sources other
than from testing the Foundation's products myself, mostly from a
lengthy and continuing dialogue with the Foundation's developers and
some considerable research into methods and concepts they discussed.
This is my own characterization of what I was told and learned. I am
not speaking for the Foundation, I have not asked them to review this
post before I published, I am not a software developer, and there may
be errors. You've been warned. :-)

My 2 cents,

Marbux