[odf-discuss] Private deal to approve OOXML? More evidence surfaces

[marbux]

KEY: So I don't have to keep writing out ISO/IEC:29500 here, I'll use
the following convention. ISO/IEC:29500 is OOXML. Microsoft's eventual
implementation of it is MOOXML.

[more]

On Sat, Apr 12, 2008 at 4:55 AM, Ian Lynch <ian.lynch at zmsl.com> wrote:
>
>  > Having achieved international standardization for OOXML, Microsoft now
>  > has a legal argument that its software cannot be legally disqualified
>  > as a national technical regulation or a government procurement
>  > specification on legal grounds that its formats are not an
>  > international standard.
>
>  But how can that possibly stand up if the application doesn't support
>  the standard? It's like putting forward MS Works for a tender that
>  mandates compatibility with MS Office. MS Works does not support MS
>  Office formats sufficiently to be of practical use.

Microsoft has said they expect to have OOXML-conformant support out by
the end of the year. Same problem as with ODF here: Conformance is so
loosely defined that what is conformant OOXML is largely meaningless.

Just because it's MS
>  doesn't mean anything. The only governments I can see falling for this
>  are governments that would have ignored the ISO issue in any case. Let's
>  face it, few governments have completely displaced .doc with odf yet and
>  it would take a long time to do so whatever happened with OOXML.

Agreed. Rip out and replace is an expensive and troublesome way of
migrating to a new format. The advantage Microsoft has at the
enterprise level in this area is that they make it easy with
Office/MOOXML whereas the migration to ODF is lossy because you cannot
map completely between unextended ODF and the MS binary formats. .
Particularly for governments with lots of data in legacy Office binary
data silos, this is a significant factor.

Those
>  that really understand the issue will say ok we'll consider your OOXML
>  compliant products alongside odf products. Then any objective comparison
>  is going to eliminate OOXML if there is no practical means of producing
>  and editing such files.
>
I question that conclusion. It's an apples and oranges comparison that
is necessarily subjective. E.g., there also is no practical means of
producing and editing featureful ISO/IEC:26300 files unless you define
ISO/IEC:26300 as the standard plus later revisions and app-specific
extensions (the OOo code base).  Microsoft has a factually strong
argument that it isn't an issue of file formats but rather of the
choice in applications. The argument has an element of truth precisely
because ODF implementations are not interoperable and there's this
incredible stack of special purpose apps built as extensions atop MS
Office.  It's also yet another reason why migrating from MS Office to
OOo and clones is so difficult. The OOo code base does not have
anything even approaching the variety of special purpose extensions
available for MS Office. E.g., in the law office market, we have only
two options if you want the competitive advantage of automation, MS
Office and WordPerfect Office.

Take a look at the Finnish Ministry of Justice migration. They were
able to migrate somewhat over 3/4 of their seats from proprietary
Notes/Domino and MS Office to OOo but had to keep 1,500 or so MS
Office seats for compatibility with other branches of government and
profession-specific apps. And they've got that hard rock of
non-interoperability between MS Office and OOo remaining.

>
>  > As to government procurement specifications, Microsoft gains an
>  > argument that governments cannot legally specify only ODF as a
>  > procurement specification.
>
>  They can if it's the only one that can practically do the job. Even if
>  they specify both, they can reasonably put in the tender. "At least one
>  application provided must fully comply with ISO 26300 or ISO (OOXML)."
>
>
>
>  >  I think it is a weak argument legally,
>  > particularly since both treaties are intended to eliminate duplicative
>  > but incompatible standards/specifications rather than to create
>  > interoperability barriers, but it's a colorable argument.
>
>  I see this as a great opportunity for odf. "In head to head choice
>  between OOXML compliant products and odf compliant products, government
>  chose odf 100%" Since there is no practical implementation of OOXML who
>  would ever be able to procure a product based on it?
>
There will be a "practical implementations" of OOXML out probably by
the end of the year. To boot, Novell, Sun, and Corel are working on
OOXML support in StarOffice/Novell OOo/WordPerfect Office.

>  Great marketing opportunity for odf.
>

Only if you are willing to push the myth of ODF interoperability,
IMHO. Or at least that's the tack taken by the big ODF vendors. I see
legal, moral, and ethical barriers to joining in that effort.

>
>  > With that background, we are down to the issues you raise. The reason
>  > Microsoft's argument is colorable is because like OOOXML, ODF is not
>  > designed for interoperability. As with OOXML, there are no full
>  > featured implementing editors available that write to the
>  > international standard without application-specific extensions. E.g.,
>  > if you want to save a file to ISO/IEC:26300-2006 in OOo or Lotus
>  > Symphony, you will find no option to do so in the file save dialogs.
>  > The heavyweight ODF editors are all writing to  ODF 1.2 plus
>  > app-specific extensions without any user ability to write to
>  > unextended ISO/IEC:26300.
>  >
>  > If you want ODF apps that interoperate without data lossiness,
>  > everyone has to use the same app and version. This is why the real ODF
>  > standard is the OOo code base, not what it says in the ODF "standard."
>  > The only way ODF achieves a somewhat valid claim of "multi-vendor"
>  > interoperability is through the various clones of the OOo code base
>  > such as Star Office, Novell OOo, and Lotus Symphony. Coupled with Sun
>  > Microsystem's iron-clad grip on commit rights to the OOo code base and
>  > OOo's market share, what full ODF support and interoperability means
>  > is really defined by a single vendor, Sun Microsystems.
>
>  Well to me that is certainly the lesser of the evils and probably
>  unavoidable at present.

The issue I'll ask you to focus on is whether this is unavoidable in
ODF v. 1.2, or more particularly, whether you will personally do
anything to change that situation? It isn't just the ODF Foundation
now saying that ODF is not designed for interoperability. E.g., Rob
said a few posts back that if you do vendor extensions, you can't do
interoperability and that there are big issues with ambiguity in the
ODF spec. I praise Rob for his candor. But he also says he sees ODF as
a spec for vendors and users like governments to specify their
requirements, in effect as some sort of blank slate rather than as a
spec for a uniform and set of file formats for interoperable apps.

The choice I see is whether folks like you will simply acquiesce to
ODF remaining a specification not designed for interoperability or
whether it will become a standard designed for that purpose.  The only
reason the ODF TC has got away all these years with not implementing
the TC charter purpose of developing a standard for the processing and
interchange of documents among implementations is lack of resistance
fueled by the pervasive myth of ODF interoperability.

 I know from many years as an environmental activist/organizer that
the TC cannot withstand the pressure folks on this list could bring to
bear. You all were willing to become politically active to fight
OOXML; are you willing to apply some of that activism to improving the
ODF spec?

At least I can download OOo for free and use it
>  on my Linux box. Anyone with the resources can fork the project if they
>  really want to have their own versions a s Novel, and IBM have done. And
>  if I want to procure a fully ISO standard editor for my country I can
>  without having to pay a license fee.
>
Sure. but if you follow that line of reasoning, then this organization
really should change its name to something like the OpenOffice.org
Fellowship, unless you are willing to become active in fixing
OpenDocument so it is not just a OOo file format rather than a true
standard. Isn't it fundamentally misleading to say it's about the file
formats rather than one code base that implements it?

>
>  > The fact that ODF is not a standard that enables or requires
>  > interoperability in order to be conformant is what gives Microsoft's
>  > procurement argument whatever teeth.it has. From a technical
>  > standpoint, ODF can claim no higher ground than OOXML when it comes to
>  > interoperability save for the IPR legal barriers to interoperability.
>
>  And the fact there are some actual applications that implement it and
>  have been tested widely to show they comply with the standard - even if
>  these are all based on the OOo code base, it's still an advantage in
>  practical terms to a standard that has not one single application that
>  fully implements the specification. If the latest MSO fully implemented
>  OOXML there would at least be an argument. In these things it's like a
>  three legged stool. Practical technical implementation, ISO status, IPR
>  barriers are all relevant. Take one leg away and the stool falls over.
>

Do you realize that you are in effect attempting to rationalize having
a standard that is not a true standard? I.e., it is only because ODF
violates the law by granting conformance to non-standard
implementations that you can defensibly say that there are ODF
implementations that "have been tested widely to show they comply with
the standard."  In fact, there are no full featured ODF editors that
have been tested widely to show that they comply with the standard
because there are very nearly zero conformance requirements in ODF to
comply with. There is no reference point for evaluating compliance.

What has actually been shown is that, surprise, surprise, different
implementations of the OOo code base are interoperable and that
KOffice is not interoperable with the OOo code base. See e.g., the
test suite maintained on the Fellowship developers site. Compare it
with the Compound Document Formats WICD test suite, which compares
application support for  vendor-neutral reference points.
<http://www.w3.org/2004/CDF/TestSuite/WICD_CDR_WP1/>.

E.g., JTC 1 SC 34 specifically sought a change in JTC 1 Directives to
accommodate ODF's permissiveness approximately one month before the
latest version of the Directives was adopted.

JTC 1 SC 34 sought language from JTC 1 SWG Directives that would have
authorized that aspect of ODF permissiveness,  approximately one month
before the current edition of JTC 1 Directives was published. The SC
34 delegate reported the requested change to the Directives and SWG
Directives' response:

"Where standards, such as DIS [sic] 26300, only require conformance to
parts of the standard, there should be clearly documented boundaries,
at a high level in the document structure, to which conformance can be
claimed.

"Response: They felt that there was no way to accomodate [sic] this proposal."

K. J. Simonsen, ISO/IEC JTC 1/SC 34 N0835, Delegate report from JTC 1
SWG directives meeting - March 7-9, 2007 (March 22, 2007)
<http://www.jtc1sc34.org/repository/0835.htm>. Accordingly, the
requested change was not made in the version of JTC 1 Directives
currently in force.

Undeterred, OASIS responded with a corresponding amendment to its own policy:

"A specification that is approved by the TC at the Public Review
Draft, Committee Specification or OASIS Standard level must include a
separate section, listing a set of numbered conformance clauses, to
which any implementation of the specification must adhere in order to
claim conformance to the specification (or any optional portion
thereof)."

<http://www.oasis-open.org/committees/process-2008-02-05.php#specQuality>

But the pertinent question is whether there will be optional portions
of the ODF spec, i,.e., whether ODF v. 1.2 will comply with this JTC 1
Directives requirement:

"Standards designed to facilitate interoperability need to specify
clearly and unambiguously the conformity requirements that are
essential to achieve the interoperability. Complexity and the number
of options should be kept to a minimum and the implementability of the
standards should be demonstrable. Verification of conformity to those
standards should then give a high degree of confidence in the
interoperability of IT systems using those standards. However, the
confidence in interoperability given by conformity to one or more
standards is not always sufficient and there may be need to use an
interoperability assessment methodology in demonstrating
interoperability between two or more IT systems in practice.

...

"... In technical areas where there is a conformity assessment
methodology and an interoperability assessment methodology, the
relationship between them must be specified."

I.e., will ODF 1.2 include conformity requirements that are essential
to achieve the interoperability? Will complexity and the number of
options be minimized? Will the interoperable implementability of ODF
v. 1.2 between two or more *different* IT systems be demonstrable or
will we just have a demonstration of interoperability between
different implementations of the OOo code base?  Will there be
procedures for verifying the conformance to the interoperability
conformance requirements? Will the relationship between the
interoperability assessment methodology and confomity assessment
methodology be specified?

Or put more simply, will the myth of ODF interoperability come true in
ODF v. 1.2? If the South African government report that ODF v. 1.2
will be out for public comment  in May is accurate, the answer is a
resounding "no." It can't be done in that little time from where the
draft spec now stands.

But the big ODF vendors that dominate the TC have had since 2002 to
fix those problems. They have not lifted a finger to do so and have
actually made the situation far worse in ODF v. 1.2, e.g., the
Metadata SC's work. Interoperability work has been put off time and
again and folks like me who tried to push the issue, as I did with the
Metadata SC's work got lectures on manners.rather than
interoperability.


>  > So with international standard status for OOXML, Microsoft now has a
>  > fairly powerful argument that interoperability of ODF implementations
>  > is not a factor that can validly be claimed as an ODF advantage. I.e.,
>  > for an ODF vendor to argue in a procurement tender process that OOXML
>  > does not enable interoperability gets answered by a "pot calling the
>  > kettle black" response.
>
>  So just say yeah but there is no practical application supporting the
>  OOXML standard so we eliminate it on those grounds. A smart salesman
>  chooses arguments they will win, not old arguments that might not now be
>  relevant to closing the sale.
>
There will be implementations of OOXML that conform to the "standard,"
just as there are implementations of ODF that conform to the
"standard." The problem with both is that there is no standard in the
"standard." They are both signed blank checks for developers.


>  > Neither standard has valid conformity assessment procedures, largely
>  > because both "standards" are pretty much a blank check for
>  > implementing developers to do whatever they want. There is no concrete
>  > reference point in either standard that implementing developers could
>  > claim conformance to. Likewise, procuring entities have no ability to
>  > confirm that implementations of either "standard" conform to the
>  > "standard." There is no standard product specified in either
>  > "standard."
>
>  In that case why did MS make such a complex standard? It would have been
>  far easier to get the same result with a minimal spec. if all they want
>  is the symbolism of having an ISO number with no intention of a
>  practical implementation, why not just make it as simple as needed to
>  get it through? In fact that would have made it more likely to get
>  through.
>
If it had been a simple standard, then others could have implemented
it. Likewise, if ODF were not so ambiguous and permissive, others
could implement it. The problem is that neither are true standards.
They are merely legal disguises for a battle between the Microsoft
Office code base and the OOo code base.

>
>  > So in the final analysis, it's one non-standard standard pitted
>  > against another non-standard standard. Both have advantages and
>  > disadvantages. But neither offer the advantage of interoperability or
>  > leveling of the competitive playing field.
>
>  I disagree.  odf might not be perfect but it does level the playing
>  field. It is a hell of a lot better than a specification that will never
>  be implemented or one that has secret code structures only known to one
>  vested interest.
>
I'll strenuously disagree here. ODF does not level the competitive
playing field. It in effect  says you must clone the OOo code base to
achieve interoperability. And the complexity of the code base masks
the ambiguities in the ODF spec itself. E.g., tell me what the full
functionality is in sufficient detail to implement it for these
OOo-specific extensions to ODF identified in the OOo code base at
lines 169-211. <http://lxr.go-oo.org/source/sw/sw/source/ui/uno/SwXDocumentSettings.cxx>.

The KDE developers have been pushing since 2005 to get those
OOo-specific extensions into the ODF specification and fully
specified. No joy yet.

Or put another way, please explain to me how to code an app to achieve
interoperability in light of this ODF "conformance requirement:"
"Documents that conform to the OpenDocument specification *may*
contain elements and attributes not specified within the OpenDocument
schema."


>  > Both are vendor lock-in
>  > "standards."
>
>  Hardly the same. I can't see how odf actually locks anyone into Sun in
>  the same way as say .doc locks people into MS. IBM and Novell already
>  have OOo versions that are interoperable with the core community code
>  and Star Office so it's not locked into a single vendor. If OOXML was
>  implemented in a MS product and made available to others it's arguably a
>  step forward from .doc. I think there are degrees of bad here and odf is
>  a lot less bad than .doc so lumping all vendor interests as "Lock-in" is
>  far too simplistic.

I disagree, at least from a legal and technical perspective.
Compliance with the law and interoperability (i.e., substitutability
of goods) tends to be an all or nothing show. I.e., a little bit legal
is a lot like a little bit pregnant. And mapping one sequence of bits
to another either maps or it doesn't. To be sure, we can rank
violations of the law according to whether they are, e.g., 1st, 2nd,
or 3rd degree felonies or 1st, 2nd, or 3rd degree misdemeanors, etc.
But it does not remove the status of illegality.

I think where you go awry in your analysis is that you draw no large
distinction between standards and apps. What is permissible outside
the context of standards is quite different from what is permissible
within that context. Standards must require substitutability of goods.
Apps, however, may or may not implement standards. You blur the
distinction between standards and apps and in effect argue that the
world is better off with ODF when what you really mean is that the
world is better off with OOo. But if all we have is set of file
formats for OOo and clones to interoperate, which is the case, then
ODF is ineligible as a standard.


>
>  > Neither should have been adopted as a standard. Both
>  > violate well-established antitrust law.
>
>  Without getting bogged down in all the detail, from a user point of view
>  I'd say odf is many times more desirable than .doc and if ISO helps
>  proliferate odf and kill .doc and similar secret proprietary formats I
>  don't really care about theoretical violations that would have to be
>  tested in court to be sure.
>

Let's refine that a bit to say that from the point of view of some
users ODF is more desirable than proprietary formats. You make the
error of equating all users' needs to your own. I.e., if I am to
justify my hourly rates as a lawyer and compete with law offices that
are well-automated using MS Word or WordPerfect extensions, OOo is not
an option. It can't even generate the most common legal format, a
legal style line-numbered pleading format, without extremely
time-consuming kludges. Integrate with WestLaw or Lexis-Nexis,
generate a table of authorities in 3 minutes instead of 2-3 hours?
forgeddaboutit.

The productivity gains from using MS Office or WordPerfect Office are
simply too great. I can't compete if I use OOo. In fact, I was only
able to switch to Linux after my retirement because of these kinds of
problems. If you must routinely generate complex documents, OOo sucks
big time. [Long rant about how long I wasted trying to format
footnotes the way they are expected to look in legal documents
omitted. Let's just say that you have to edit settings in 3 separate
styles that want to override each other. I finally gave up after about
*70 hours.*.]

I see OOo from an entirely different perspective than you do. I see
OOo as a barrier to the development and implementation of truly
interoperable standards and applications because of OOo's enormous
market and mind share in the FOSS community that flows almost entirely
from the myth of ODF interoperability. Hey, it's FOSS and ODF is open
and interoperable, right? I need to use this to strike a blow for
FOSS, open standards, and interoperability. Not.

>  > Together, the only choice offered is a choice between evils. ODF may
>  > win that choice with some governments, Microsoft with others. But ODF
>  > no longer has the wholly undeserved status of being the only
>  > international standard in this market sector. I think a far better
>  > solution would have been to remove ODF's international standard status
>  > until such time as it becomes truly eligible for the legal status of
>  > an international standard or better yet that it never have been
>  > approved as an international standard before it was eligible. This
>  > "every vendor gets its own standard" madness really needs to come to a
>  > halt.
>
>  Except that the way the world is requires practical progress routes to
>  improvement. The question is usually not about perfection but is this
>  step an improvement on the previous. I believe odf is a significant step
>  forward when compared to .doc. odf is adoptable by any company and any
>  company can fork the OOo code base so it's a far leveller playing field
>  than with MS Office. To compare the Sun situation with the MS situation
>  in terms of vendor lock-in stretches the imagination.
>

What you are in effect rejecting is the fact that there are minimum
standards for standards, e.g., exit criteria such as those I quoted
above from JTC 1 Directives. One might still question how the battle
over OOXML might have played out if ODF advocates were not left in the
ethical position of the pot calling the kettle black, i.e., if ODF
were a real standard. If it were, then ODF advocates would have been
positioned to assert violations of JTC 1 Directives. But they couldn't
because they posed the issue as a battle between competing
non-standards. Hey, if Sun and IBM can have their own private
incompatible file format standard, why can't Microsoft. The whole
defensive strategy played right into Microsoft's hands because ODF
offered no lawful alternative.


>
>  > And the FOSS community really needs to come to grips with the fact
>  > that "open" and "interoperable" are not synonyms. Microsoft is not the
>  > only company out there that plays fast and loose with the technology
>  > or the law.
>
>  Companies are in business to make money. It's in their interest to
>  eliminate competition and they will do whatever it takes to do that even
>  illegally in some cases if they think they will get away with it. There
>  is no doubt that Sun sees it as in its interest to have office
>  productivity tools that are in wide use and generally acceptable to
>  governments that run on their machines. In order to get to that position
>  given MS's stranglehold on Office productivity tools they had to do
>  something pretty radical. OOo was that disruptive choice. Sun benefits
>  from community support, the community benefits from a free productivity
>  suite with a published file specification that is far easier for third
>  parties to use for writing reliable filters.

For writing one-way interop filters.

Maybe not full
>  interoperability, but a massive improvement for end users and tax payers
>  all over the world. So for me I'll settle for better not perfect. Yes
>  Sun is a winner out of this but then so is the community.
>

I think not. A choice between which vendor's code base you will be
locked into is a battle Microsoft already won. I think you place far
too much merely aesthetic value on "open" and "free." If we are to
build an interconnected Information Society, full interoperability has
to be at the very foundation of t he infrastructure. By compromising
for "not so evil,"  you simply create new interoperability, migration,
and adoption barriers that a truly interoperable standard must
overcome.

The question I pose to you is whether you care whether that true
standard will be ODF or something else? If you are unwilling to exert
pressure on the TC to make the myth of ODF interoperability come true,
all you are doing is locking your customers into another set of broken
file formats and making it more difficult for them to switch when a
real standard comes along.

>  Who knows at some point someone might develop software that can
>  automatically in real time work out any format and translate it into any
>  other format. In the current circumstances, we are where we are. odf is
>  the best hope for increased freedom in the use of office productivity
>  tools and can also catalyse further freedoms in other domains. We can
>  argue the details of degrees of interoperability but in the end what
>  matters is increasing freedom and lowering costs and odf does that.

You're blurring the distinction between OOo and ODF again and you're
also overlooking the fact that a single vendor controls the commit
rights to the OOo code base, a single vendor that is in a very deep
embrace with Microsoft through the 2004 Sun-Microsoft deal, pays
royalties on StarOffice to Microsoft, didn't bother to get patent
protection for OOo from Microsoft's 45 patents that read on OOo, and
is busy implementing OOXML support. Meanwhile, Microsoft rattles its
patent sabers against OOo.
<http://cnnmoney.printthis.clickability.com/pt/cpt?action=cpt&title=Microsoft+claims+software+like+Linux+violates+its+patents+-+May+28%2C+2007&expire=-1&urlID=22311448&fb=Y&url=http%3A%2F%2Fmoney.cnn.com%2Fmagazines%2Ffortune%2Ffortune_archive%2F2007%2F05%2F28%2F100033867%2Findex.htm&partnerID=2200>.
Moreover, the ODF TC is now under the control of a vendor, IBM, that
distributes its OOo codebase clone only in binary formats. If you do
not see in that set of facts rugs that can be jerked out from under
FOSS at any time by any of the three major players for different
reasons, our viewpoints differ irreconcilably.

I am no enemy of continuing progress. In fact, I would love to see ODF
become a true standard. But unless ODF comes up to the minimum
requirements for a standard, it has no future worthy of preservation.
It only creates an obstacle for the real standard when it finally
arrives by getting people locked into another non-standard.  It only
adds yet another layer of complexity in eventually interconnecting a
ubiquitously networked world, another interoperability barrier.

The only real question I see is whether you and other folks on the
list will exert some of the same energy you brought to the OOXML fray
into bringing ODF up to snuff. The black hat/white hat/ODF is
interoperable approach to fighting OOXML didn't work. Will you help
make the ODF interoperability myth come true? Or are you just riding
on Sun and IBM's coattails and helping them drag the rest of the world
onto a new set of formats that neither enable nor require
interoperability?

Best regards,

Paul