[odf-discuss] A story of TIFF

[Daniel Carrera]

On Fri, 2007-02-09 at 18:27 -0800, marbux wrote:
> > I am not trying to make a strawman. Where is the strawman? You said:
> >
> > "My understanding is that the current version using the proposed ODF 1.2
> > changes already achieves near-perfect interop in both directions"
> >
> > I'm saying that I don't see how this is possible, since ODF 1.2 is not
> > implemented elsewhere. Interoperability requires more than one
> > application. You cannot argue interoperability until there is at least
> > one other application with which you can interoperate.
> >
> I'm sorry. I thought it was understood that the Foundation is working
> with their own build of OOo that implements the proposed changes in
> ODF 1.2.

They are? And how are the rest of us supposed to know that? :)

So... they rewrote parts of OOo to implement some changes to ODF, which
they hope will be accepted, and they say that with that they reach
near-perfect interoperability.

The number of surprising and unverifiable claims is increasing. You will
understand if I have reservations. Consider, for example, that it took
about a decade for OOo to reach a mere 85% interop with .doc. It is
surprising that the Foundation achieved better than that, using a
different binary format, in the span of a year or so.

Now, since OOo is open source, and surely whatever they did to OOo has
nothing to do with secret Microsoft APIs, surely they can release their
changes to OOo for public scrutiny. Correct?

> It isn't me that is planning to do that; it's the Foundation
> developers. I am genuinely excited by their approach. But their
> approach does more than flag the dark objects; it also describes them

How can you describe something that is unknown? And back to the OOXML
thought experiment, doesn't OOXML describe tags too?

> I suspect you hadn't realized that the Foundation is working with
> their own build of OOo that implements the proposed changes for ODF
> 1.2. Does that fact resolve many of our differences including this
> one?

It certainly answers my question. But it sort of raises a new one. It
adds one more thing that I'm being asked to take on faith. You will
understand if I hesitate.

> I think you overlook that the "foreign element" tags are limited to
> foreign metadata.

They are not. And they are not "foreign element" tags. Please stop
saying that. Section 1.5 says that you are allowed to create your own
tags, you do not put them inside some <foregin> or <unknown> tag (ODF
has no such thing). All that is required is that you use your own
namespace.

Here is they key paragraph:

"Documents that conform to the OpenDocument specification MAY contain
elements and attributes not specified within the OpenDocument schema.
Such elements and attributes must not be part of a namespace that is
defined within this specification and are called foreign elements and
attributes."

Wrapping an entire OXML document inside <office:document> tags is fully
compliant as per section 1.5.

Even if you _were_ restricted to putting foregin tags inside
<office:meta>, it is trivially easy to get around that requirement. 
Assign an 'id' to every element and have the tag inside <office:meta>
refer to it. For example:

<office:meta>
  <daniel:myblob daniel:refers-to="asdf">...</daniel:myblob>
</office:meta>
<office:document>
  <text:p text:id="asdf">Hello world</text:p>
</office:document>

Voila! Now you can add whatever you want to any tag in the document. You
can take it further and put the entire document inside <office:meta>.

The point is, even if ODF restricted you to putting new tags only inside
<office:meta> (it doesn't) it is trivially simple to wiggle around that
requirement. That's probably why the ODF TC didn't add any such
requirement. Because they know that it achieves nothing useful.

> > > Second, the allowance for wrapping dark objects in
> > > EOOXML tags testifies that MOOXML is not entirely XML but also
> > > includes proprietary binary blobs.
> >
> > Could this same argument not be made for ODF then? "the allowance for
> > wrapping dark objects in ODF tags testifies that ODF is not entirely XML
> > but also includes proprietary binary blobs". Of course, I don't believe
> > this. I'm just offering a word of caution, lest less scrupulous person
> > use your argument against you on a public blog.
> >
> >
> I think that's really been our topic of discussion, hasn't it? :-)

Ok. I'm glad that you see that that argument would cut both ways.

>  I
> am really doing some serious thinking about how to avoid that becoming
> true. E.g., I need feedback on whether it is workable for ODF section
> 1.5 to be changed to instruct that the foreign element tags are only
> for use in interoperability with non-conformant apps;

Bad idea. Too easy to wiggle around that. A better idea would be to
change the last paragraph of section 1.5:

"There are no rules regarding the elements and attributes that actually
have to be supported by conforming applications, except that
applications should not use foreign elements and attributes for features
by the OpenDocument schema. See also appendix D."

In the second-last line, change "should not" to "MUST NOT" (that is,
forcing an absolute prohibition through RFC2119):

"There are no rules regarding the elements and attributes that actually
have to be supported by conforming applications, except that
applications MUST NOT use foreign elements and attributes for features
by the OpenDocument schema. See also appendix D."

This is not easy to enforce, but at least it blocks the very obvious
duplication of functionality.

But this too, can cut but ways (though less). A lot of the features that
the Foundation is storing in binary blobs can easily be implemented in
ODF. The problem is that they just don't know how.

I think that my solution is good. Anything that allows the Foundation to
insert binary bobs for things ODF can do also allows MS to do the same.


>  that a
> conformant app must not use those tags to stash its own dark objects.

So OOo would not be allowed to store document macros? Macros are not
covered in the spec (nor should they be).

> > I think you missed my argument. I posit that an OXML-based approach has
> > the same benefits as the Foundation plugin. Since I certainly do not
> > support an OXML-based approach, and I suspect few people on this list
> > will, I find it hard to support the Foundation plugin. I think that
> > putting OXML inside ODF is *bad*. I think that putting binary blobs
> > inside ODF is *very**bad*. I hope this clarifies the misunderstanding.
> >
> 
> I would not support allowing blobs in ODF were in not for the reality
> that:

The same arguments (valid as they are) can be given for the OXML thought
experiment. That's my point. Extending ODF with OXML has the same
benefits as extending ODF with binary blobs and a few others on top. If
you don't like the former, you shouldn't like the latter. If you like
the latter, you should like the former.

> > Let me see if I understand your position: You claim that EOOXML cannot
> > legitimately (ie, using real tags) represent future documents. That
> > Microsoft will fill it with binary blobs going forward. Therefore,
> > EOOXML cannot be relied on for conversion. Correct?
> >
> Again, I distinguish between MOOXML and EOOXML. As is hopefully better
> explained above, I don't see EOOXML as a trustworthy intermediate
> between IMBR and ODF.

You are beating around the bush. Do you or do you not claim that Ecma
376 cannot legitimately represent future documents. Yes or no?

> > Using OXML as an extension of ODF is an example of a "solution" that has
> > dark objects that are less dark than in the Foundation plugin. If you
> > agree with me that using OXML as an extension of ODF is a bad idea, by
> > my argument, the Foundation plugin is too. Both solutions have the same
> > benefits: 100% fidelity, wrapping dark objects, etc. In the OXML one the
> > dark objects are mostly documented XML, which is an improvement over
> > just binary objects. If you don't support the OXML extension idea (I
> > don't) I don't see how you can logically support a binary object idea.
> >
> I think your premise is doubtful.

What premise? That Ecma 376 can store everything MS Office saves,
possibly by including some binary blobs? This is supported by the fact
that Ecma 376 is already the default file format of the latest Office.

> There is also the fact that we legitimize and promote the adoption of
> EOOXML if we use it as an intermediary in ODF <> IMBR.

Legitimizing the insertion of binary tags in an ODF extension is a worse
precedent. I agree with you that adding Ecma 376 tags to ODF is bad, and
I say that using binary blobs is just as bad and worse. Arguments
against ODF+Ecma also apply to ODF+blobs. Arguments in favour of ODF
+blobs also apply to ODF+Ecma. That's the point.

> > > Again, you've overlooked that the Foundation has proposed a solution
> > > to the blob problem.
> >
> > Gary's "proposed solution" for interop is to flag binary objects and
> > hope that applications will figure out how to deal with them. Have I
> > missed a solution to the dark object problem?
> >
> I think so. The Foundation's solution doesn't just flag the dark
> objects. It also describes them as completely as is possible at any
> moment in time.

How is that a solution if the blob is unknown to begin with?
1. Can you show me an example of an unknown blob where the description
is useful for something?
2. Can you show me an example of a binary blob that the Foundation can
describe better than if they had use Ecma 376?

>  If apps are implemented in such a way that they act on
> what is known, then we're poised to, e.g., do online updates of what
> is known about the dark objects

Huh? How do you figure that would work?

> The problematic dark objects are for the most part among
> those that have no corresponding known XML tags. EOOXML really doesn't
> help that much in understanding them.

Ok, so you have reduced the usefulness of the Foundation plugin to only
a subset of dark objects whose existence you claim, but have presented
no evidence for (dark objects that can't be expressed with Ecma 376).
You ask us to take on faith that these exist. You further argue that the
benefit of flagging these unknown objects that lie outside Ecma 376 with
the very limited non-knowledge the Foundation has about them exceeds the
drawbacks of having to cope with dark binary objects for which there
really is a perfectly valid Ecma 376 tag that could in principle be
understood. You will understand if I have second thoughts about your
solution.


> > An interoperability subset for MS Office has nothing to do with dark
> > objects. In brief:
> >
> > * Dark objects relate to MSO->ODF interop.
> > * MSO subset relates to ODF->MSO interop.
> >
> I think you're mostly right here, except to the extent that the lack
> of definition of the dark objects contributes to the selection of tags
> for the ODF > MSO interop subset.

???


> > >  and five proposed ODF
> > > extensions that allow adequate description of the blobs" content as
> > > more is learned about them.
> >
> > That's fantastic, and kudos for that. Those same extensions can be used
> > to map portions of EOOXML-stored documents.
> >
> Yes, but I think that would require non-conformant EOOXML,

What? No. I mean that whatever cool new tags the Foundation has proposed
can be used just as well whether you are converting from binary blobs or
from Ecma 376. I don't talk about extending Ecma 376 any more than I'm
talking about extending binary blobs.

> > >  Meanwhile, their approach already has
> > > achieved near-perfect fidelity.
> >
> > I thought it was perfect fidelity. Heck, you can get perfect fidelity by
> > just dumping the entire in-memory structure inside a tag.
> >
> No, they say they are above 98 per cent fidelity in both directions
> and the remainder is within grasp.

Did the definition of fidelity just change in the last few emails? You
spent ages talking about 100% fidelity, and now you are down to 98% and
you use it in a context that suggests you are talking about
interoperability instead.

> I've been all over it and have cited it many times.

But you don't seem to act like you understand it. You talk about
<unknown> or <foreign> tags, you talk bout tags being defined in that
section, and you say that foregin elements go in metadata.

> But I disagree that the two paragraphs are the only relevant parts of
> section 1.5. There are more.
> <http://develop.opendocumentfellowship.org/spec/?page=1#1.5>.
> 
> "Conforming applications that read and write documents *may* preserve
> foreign elements and attributes."
> 
> The Metadata SC is expected to change "may" to "must" or "shall."

What does that have to do with the original comment that prompted my
reply? This is what you wrote:

"Phil Boutros of Stellent, who probably knows more about file conversion
issues than anyone else on the planet, pointed the way with the ODF
section 1.5 conformance section."

I replied that this is not rocket science and that the section is
actually quite simple and straight forward. And you reply by listing the
changes that the Metadata SC wants? This is getting really illogical.

Best,
Daniel.
-- 
May you live in interesting times.
May people in high places take notice of you.
May all your wishes come true.
  -- Chinese curse.