[odf-discuss] Private deal to approve OOXML? More evidence surfaces

[Ian Lynch]

On Sat, 2008-04-12 at 21:35 -0700, marbux wrote:
> KEY: So I don't have to keep writing out ISO/IEC:29500 here, I'll use
> the following convention. ISO/IEC:29500 is OOXML. Microsoft's eventual
> implementation of it is MOOXML.

ok :-)

> Microsoft has said they expect to have OOXML-conformant support out by
> the end of the year.

Well that is different.

>  Same problem as with ODF here: Conformance is so
> loosely defined that what is conformant OOXML is largely meaningless.

I'd say you get 100% conformant support for odf with OpenOffice and the
main derivatives and these applications are not controlled by Sun or IBM
or Novell in the same way as MS controls MSO. Thereis and always will be
the option to fork the code base. So you have multi-vendor support of
the applications with the opportunities for other vendors to use the
openly documented format to interoperate with their products to any
degree necessary or indeed to fork open office if that is what it takes
to make their own confomant product. None of that is true of MOOXML

> Just because it's MS
> >  doesn't mean anything. The only governments I can see falling for this
> >  are governments that would have ignored the ISO issue in any case. Let's
> >  face it, few governments have completely displaced .doc with odf yet and
> >  it would take a long time to do so whatever happened with OOXML.
> 
> Agreed. Rip out and replace is an expensive and troublesome way of
> migrating to a new format. The advantage Microsoft has at the
> enterprise level in this area is that they make it easy with
> Office/MOOXML whereas the migration to ODF is lossy because you cannot
> map completely between unextended ODF and the MS binary formats. .
> Particularly for governments with lots of data in legacy Office binary
> data silos, this is a significant factor.

So there is a trade off between lock-in to MS and much more freedom and
choice of supplier by going to odf. That really was the case before the
ISO stuff. If indeed MS does produce a fully MOOXML version of MSO then
they will have seen off the ISO advantage of odf assuming the ISO
process has not damaged them in the eyes of say the EU to the extent
they require their own admin to use odf on say risk of lock-in grounds
rather than ISO standard grounds.

> Those
> >  that really understand the issue will say ok we'll consider your OOXML
> >  compliant products alongside odf products. Then any objective comparison
> >  is going to eliminate OOXML if there is no practical means of producing
> >  and editing such files.
> >
> I question that conclusion. It's an apples and oranges comparison that
> is necessarily subjective. E.g., there also is no practical means of
> producing and editing featureful ISO/IEC:26300 files unless you define
> ISO/IEC:26300 as the standard plus later revisions and app-specific
> extensions (the OOo code base). 

Yes that is what I would do in the first instance. Why deliberately make
things difficult?

>  Microsoft has a factually strong
> argument that it isn't an issue of file formats but rather of the
> choice in applications.

Ok, they control one application, the other choice is not controlled by
any single company even though one company does put up a lot of the
resources for the engineering effort. So I'd be happy to argue it on the
application freedom ticket or greater scope for interoperability. If it
wasn't for the current lock-in to MS I doubt anyone would choose the MS
route now knowing what is known just as more open architecture hardware
was chosen by the industry for early PCs. IBM PC by name but others able
to clone the machines so a lot of competition and low prices. I see
OpenOffice as a similar situation for productivity software but having
to overcome a massively more entrenched monopoly.

>  The argument has an element of truth precisely
> because ODF implementations are not interoperable and there's this
> incredible stack of special purpose apps built as extensions atop MS
> Office.  It's also yet another reason why migrating from MS Office to
> OOo and clones is so difficult. The OOo code base does not have
> anything even approaching the variety of special purpose extensions
> available for MS Office. E.g., in the law office market, we have only
> two options if you want the competitive advantage of automation, MS
> Office and WordPerfect Office.

To be honest migration from MS Office with 100% fidelity in every
possible aspect in every context is a pipe dream. It's never going to
happen and this is why. For most people OOo needs to be good enough. It
is for the vast majority of users. Once OOo builds up a big enough user
base new businesses will customise the extensions they need and some
highly vertically integrated users will never migrate - just as lawyers
in the US have been locked into WP. Who cares? (Ok they do but there is
a bigger picture) We don't need 100% market share, we simply need enough
to break the monopoly so people have a fair choice. Read Clay
Christensen on disruptive innovation. His theory predicts that MSO would
become a product in highly integrated niche markets where a premium will
be paid for specialist extensions. I think this has to happen first and
then almost certainly as things further mature interoperability
standards will evolve to be more and more comprehensive.

> Take a look at the Finnish Ministry of Justice migration. They were
> able to migrate somewhat over 3/4 of their seats from proprietary
> Notes/Domino and MS Office to OOo but had to keep 1,500 or so MS
> Office seats for compatibility with other branches of government and
> profession-specific apps. And they've got that hard rock of
> non-interoperability between MS Office and OOo remaining.

Change is hard - the key thing is they have made the change. 75% market
share would make a massive difference - 25% would probably signal
breaking the monopolistic hold. Life isn't perfect and never will be.
It's back to is it better for 75% to change than 0%.

> There will be a "practical implementations" of OOXML out probably by
> the end of the year.

If they work reliably then as I said above, back to the pre ISO status
quo but I think the ISO thing was worth doing because it has certainly
damaged MS further in terms of integrity etc. and diverted a lot of
resources from other things.

>  To boot, Novell, Sun, and Corel are working on
> OOXML support in StarOffice/Novell OOo/WordPerfect Office.

Again that is probably inevitable and beyond ISO the more MSO can be
made to interoperate cleanly with OOo the better for OOo. The current
quality of the filters is good enough for a great many people so
improving them just lowers the barriers further.

> >  Great marketing opportunity for odf.
> >
> 
> Only if you are willing to push the myth of ODF interoperability,
> IMHO. Or at least that's the tack taken by the big ODF vendors. I see
> legal, moral, and ethical barriers to joining in that effort.

As I explained above, I think you are too much of a perfectionist. The
world isn't perfect. What matters is whether a is better than b in the
bigger picture not whether a is perfect. 

> The issue I'll ask you to focus on is whether this is unavoidable in
> ODF v. 1.2, or more particularly, whether you will personally do
> anything to change that situation? It isn't just the ODF Foundation
> now saying that ODF is not designed for interoperability. E.g., Rob
> said a few posts back that if you do vendor extensions, you can't do
> interoperability and that there are big issues with ambiguity in the
> ODF spec. I praise Rob for his candor. But he also says he sees ODF as
> a spec for vendors and users like governments to specify their
> requirements, in effect as some sort of blank slate rather than as a
> spec for a uniform and set of file formats for interoperable apps.
> 
> The choice I see is whether folks like you will simply acquiesce to
> ODF remaining a specification not designed for interoperability or
> whether it will become a standard designed for that purpose.

Folks like me fight battles we think we have a chance of winning. I'm
fundamentally a pragmatist. I see no way that we can muster the
resources to fundamentally change the direction odf is going and there
are arguments both ways about the trade off between standardisation and
ability to develop and adapt. So to me in the wider picture, the OOo
code base and odf, backed by a range of big commercial players and grass
roots individuals, are the best practical option for increasing freedom
and choice and really that is my main interest.

>   The only
> reason the ODF TC has got away all these years with not implementing
> the TC charter purpose of developing a standard for the processing and
> interchange of documents among implementations is lack of resistance
> fueled by the pervasive myth of ODF interoperability.

I think that interoperability means different things to different
people. 

>  I know from many years as an environmental activist/organizer that
> the TC cannot withstand the pressure folks on this list could bring to
> bear. You all were willing to become politically active to fight
> OOXML; are you willing to apply some of that activism to improving the
> ODF spec?

Personally I don't think I know enough of the technical details and I
have a business to run so I don't have time to learn them. On technical
issues I rely on people here with obviously greater technical knowledge
than me and so far I don't detect any general consensus in these people
that there is a major winnable battle to have on this.  I know you feel
strongly about it but I know others with equally strong conflicting
opinions. I respect the opinions, I'm just not in a position to do
anything.

> At least I can download OOo for free and use it
> >  on my Linux box. Anyone with the resources can fork the project if they
> >  really want to have their own versions a s Novel, and IBM have done. And
> >  if I want to procure a fully ISO standard editor for my country I can
> >  without having to pay a license fee.
> >
> Sure. but if you follow that line of reasoning, then this organization
> really should change its name to something like the OpenOffice.org
> Fellowship, unless you are willing to become active in fixing
> OpenDocument so it is not just a OOo file format rather than a true
> standard.

If indeed it is fixable in that way. 

>  Isn't it fundamentally misleading to say it's about the file
> formats rather than one code base that implements it?

Not really. At present there is one code base that can fully implement
it but many others that can interoperate with it well enough for
practical purposes with a little effort - they can even warn users about
specific things that might cause them a problem. That is better than
random unpredictable differences like with .doc. Personally I don't
think it was ever *only* about a file format. File formats are important
but they don't exist in a vacuum. FOSS applications with published open
protocols and file formats collectively support digital freedom but
there are always going to be constraints. 

> Do you realize that you are in effect attempting to rationalize having
> a standard that is not a true standard?

Depending on your definition of a true standard. To me the whole ISO
process has shown itself to be political, so again I don't really care
about pedantic definitions of standards, getting bogged down in that
misses the point. I'm more interested in strategies that further
improvements in digital freedom in its widest sense. That is my
political affiliation so no real need to rationalise anything. I simply
accept that we are on a journey towards digital freedom and some
desirable things are going to take time to come to fruition.

>  I.e., it is only because ODF
> violates the law by granting conformance to non-standard
> implementations that you can defensibly say that there are ODF
> implementations that "have been tested widely to show they comply with
> the standard."  In fact, there are no full featured ODF editors that
> have been tested widely to show that they comply with the standard
> because there are very nearly zero conformance requirements in ODF to
> comply with. There is no reference point for evaluating compliance.
> 
> What has actually been shown is that, surprise, surprise, different
> implementations of the OOo code base are interoperable and that
> KOffice is not interoperable with the OOo code base. 

I never expected it would be without some work, but probably that work
is doable given some time and a lot more likely than if each used its
own unique undocumented proprietary binary format.

> Or put more simply, will the myth of ODF interoperability come true in
> ODF v. 1.2? If the South African government report that ODF v. 1.2
> will be out for public comment  in May is accurate, the answer is a
> resounding "no." It can't be done in that little time from where the
> draft spec now stands.

So campaign for improvement and get it into the 1.3 or 2.0 spec. To be
honest I don't have the time or knowledge to make a judgement on the
details of the above and I have to rely on others here. It seems to me
that you are a bit of a loan voice in all this. You would be better to
get some of the technical gurus on your side who would then give
confidence to the not so technically informed that there really is a
better alternative route.

> But the big ODF vendors that dominate the TC have had since 2002 to
> fix those problems. They have not lifted a finger to do so and have
> actually made the situation far worse in ODF v. 1.2, e.g., the
> Metadata SC's work. Interoperability work has been put off time and
> again and folks like me who tried to push the issue, as I did with the
> Metadata SC's work got lectures on manners.rather than
> interoperability.

Maybe it's because they don't see that it's cost-effective to deploy
their resources in a different way. As I said, that might not be
perfect, but if they withdrew their resources and there was no odf at
all and no OOo I think it would be a lot worse.

> If it had been a simple standard, then others could have implemented
> it. 

Or if they really are going to implement in in MSO it means it has to be
complex since MSO is complex.

> Likewise, if ODF were not so ambiguous and permissive, others
> could implement it. The problem is that neither are true standards.
> They are merely legal disguises for a battle between the Microsoft
> Office code base and the OOo code base.

The reality is that that battle began as soon as OOo was born. with
resources, anyone can implement odf in a way that will make it OOo
compatible. Anyone can download OOo freely and install it now. I don't
see a problem with that. If the same becomes true of MSO, fine.  But
there are current problems with a dominant user base that is still
dependent on a single source of supply for its applications and that
doesn't seem likely to change.

> I'll strenuously disagree here. ODF does not level the competitive
> playing field. It in effect  says you must clone the OOo code base to
> achieve interoperability.

But that is doable and has been done. Cloning MSO isn't doable therefore
the playing field is more level.

In fact, given time other apps will use the documented odf (and its
evolutionary changes) as a means to better if not perfect
interoperability and for most users eg on google docs, koffice etc it
will be good enough. 

>  And the complexity of the code base masks
> the ambiguities in the ODF spec itself. E.g., tell me what the full
> functionality is in sufficient detail to implement it for these
> OOo-specific extensions to ODF identified in the OOo code base at
> lines 169-211. <http://lxr.go-oo.org/source/sw/sw/source/ui/uno/SwXDocumentSettings.cxx>.
> 
> The KDE developers have been pushing since 2005 to get those
> OOo-specific extensions into the ODF specification and fully
> specified. No joy yet.

So what is their view on this? Are they backing you and your mission?

> Or put another way, please explain to me how to code an app to achieve
> interoperability in light of this ODF "conformance requirement:"
> "Documents that conform to the OpenDocument specification *may*
> contain elements and attributes not specified within the OpenDocument
> schema."

I don't need to. I just look at a school here in the UK swapping 1000
seats to OOo away from .doc and say that is a step nearer breaking a
monopoly and a notch back in the levelling of the playing field. Seems
to me that you are getting too focussed on the minutae of technical
detail to see the wider picture.


> >  > Both are vendor lock-in
> >  > "standards."
> >
> >  Hardly the same. I can't see how odf actually locks anyone into Sun in
> >  the same way as say .doc locks people into MS. IBM and Novell already
> >  have OOo versions that are interoperable with the core community code
> >  and Star Office so it's not locked into a single vendor. If OOXML was
> >  implemented in a MS product and made available to others it's arguably a
> >  step forward from .doc. I think there are degrees of bad here and odf is
> >  a lot less bad than .doc so lumping all vendor interests as "Lock-in" is
> >  far too simplistic.
> 
> I disagree, at least from a legal and technical perspective.

That is the problem, you are looking through the other end of the
telescope. Let's say for the sake of argument that OOo took over as the
dominant office player would it be better than the current position?
Hell yes. At the simplest level, ordinary citizens, schools and public
sector organisations would not be shelling out on licenses, having to
negotiate license keys or forced upgrades of hardware. Linux users get
the same office suite as Windows users. Anyone can build apps and make
them interoperable because all the things needed are openly published.
Ok it would be nicer to see a range of different applications that could
all interoperate seamlessly. I'd say that would be more likely to evolve
from the OOo dominant position than from the MSO dominant position.

> I think where you go awry in your analysis is that you draw no large
> distinction between standards and apps.

I don't think in practical terms you can at this point in time divorce
things into nice compartments. Law is old and has never changed to fit
the times - software patents etc are evidence - so being too pendantic
about applying the law to technological situations where it is not
really fit for purpose probably isn't going to get too far.

>  What is permissible outside
> the context of standards is quite different from what is permissible
> within that context. Standards must require substitutability of goods.
> Apps, however, may or may not implement standards. You blur the
> distinction between standards and apps and in effect argue that the
> world is better off with ODF when what you really mean is that the
> world is better off with OOo.

I don't see those as mutually exclusive.

>  But if all we have is set of file
> formats for OOo and clones to interoperate, which is the case, then
> ODF is ineligible as a standard.

So screw ISO then. Who cares? The whole ISO debacle was a
political/marketing battle - history will say who won because it's not
at all obvious at the moment. At best MS might have manged to claw back
to a sort of pre-odf ISO status quo. To support odf you don't have to
bet the whole farm on ISO and legal definitions of standards. All you
need is to believe promoting odf is a good idea for whatever reasons. If
ISO provides a strategic lever fine, if not look for something else.

> Let's refine that a bit to say that from the point of view of some
> users ODF is more desirable than proprietary formats. You make the
> error of equating all users' needs to your own.

To me I'm the only user that matters :-) Well I feel entitled to be a
bit selfish since my needs have usually come second to all those who
push .doc and .ppt files at me. Seriously, my judgement is that if we
were starting from scratch and knew what we know now very few people
would vote for proprietary file formats. Only those so locked into them
they are screwed without them really need them.

>  I.e., if I am to
> justify my hourly rates as a lawyer and compete with law offices that
> are well-automated using MS Word or WordPerfect extensions, OOo is not
> an option. 

Fine, don't use it. You and your colleagues took a big risk in handing
control of your documents to proprietary companies and you are now
paying the price of lock-in. Let's free future generations from that in
other industries. 

> It can't even generate the most common legal format, a
> legal style line-numbered pleading format, without extremely
> time-consuming kludges. Integrate with WestLaw or Lexis-Nexis,
> generate a table of authorities in 3 minutes instead of 2-3 hours?
> forgeddaboutit.

So get a group of you together, fund some developers and join OOo
community - or fork OOo if there are enough of you. 

> The productivity gains from using MS Office or WordPerfect Office are
> simply too great. I can't compete if I use OOo. In fact, I was only
> able to switch to Linux after my retirement because of these kinds of
> problems.

I guess you could have used cross over office on Linux. If OOo took say
50% of the office market, don't you think it might be likely that
someone would produce the necessary extensions for the legal profession?
We have to walk before we run, the more complex the implementation, the
longer it is going to take to get open systems in. It might never happen
in some niche industries although never is a long, long time.

>  If you must routinely generate complex documents, OOo sucks
> big time. [Long rant about how long I wasted trying to format
> footnotes the way they are expected to look in legal documents
> omitted. Let's just say that you have to edit settings in 3 separate
> styles that want to override each other. I finally gave up after about
> *70 hours.*.]

Compared with Impression Publisher that I used back in the early 90s,
Word and Writer handle graphics incredibly clumsily. We all have our
crosses to bear. Two years ago I wrote a long paper with references etc
in OOo Writer and it was fine, at least as good as Word. For most people
that use Word processors both Word and Writer are complete overkill
anyway. Yes let's improve OOo - it is being all the time and we need
Sun, IBM, Novell et al to keep that work flow going.


> I see OOo from an entirely different perspective than you do. I see
> OOo as a barrier to the development and implementation of truly
> interoperable standards and applications because of OOo's enormous
> market and mind share in the FOSS community that flows almost entirely
> from the myth of ODF interoperability. Hey, it's FOSS and ODF is open
> and interoperable, right? I need to use this to strike a blow for
> FOSS, open standards, and interoperability. Not

No, most people I talk to say OOo is free I can download it and its
upgrades for free, I can export pdf and run it on Linux and it has very
good .doc filters. Those are the reasons people use OOo not because of
interoperability in general. Personally, I don't use interoperability as
a particular selling point at all. Maybe in years to come I will.

> For writing one-way interop filters.

The .doc filters in OOo work in both directions. They are not perfect
but good enough for most people in most circumstances. Unless all WP
have exactly the same features it's doubtful you would ever get a
perfect match but I could see a developing standard like odf getting
closer and closer to that so in the end it was academic for 99.9% of the
population.


> I think not. A choice between which vendor's code base you will be
> locked into is a battle Microsoft already won.

You are not locked into open source code bases. You can fork the project
if you really feel strongly about it. 

> I am no enemy of continuing progress. In fact, I would love to see ODF
> become a true standard. But unless ODF comes up to the minimum
> requirements for a standard, it has no future worthy of preservation.
> It only creates an obstacle for the real standard when

if - (by what mechanism will it arise?)

>  it finally
> arrives by getting people locked into another non-standard.  It only
> adds yet another layer of complexity in eventually interconnecting a
> ubiquitously networked world, another interoperability barrier.
> 
> The only real question I see is whether you and other folks on the
> list will exert some of the same energy you brought to the OOXML fray
> into bringing ODF up to snuff. The black hat/white hat/ODF is
> interoperable approach to fighting OOXML didn't work.

On the contrary, I don't think it ever mattered that much whether MS got
ISO status. Well not getting it would probably have been a real killer
for them but the process of getting it has certainly raised the odf
profile and damaged MS's integrity further so the outcome is not all
bad.

>  Will you help
> make the ODF interoperability myth come true? Or are you just riding
> on Sun and IBM's coattails and helping them drag the rest of the world
> onto a new set of formats that neither enable nor require
> interoperability?

I just see the world differently from you. Even if I wanted to I simply
haven't the personal resources to commit. At the moment I have at least
to an extent go with the flow.

Ian
-- 
New QCA Accredited IT Qualifications
www.theINGOTs.org

You have received this email from the following company: The Learning
Machine Limited, Reg Office, 36 Ashby Road, Tamworth, Staffordshire, B79
8AQ. Reg No: 05560797, Registered in England and Wales.