Additional appeal against publication of draft-lyon-senderid-* in regards to its recommended use of Resent- header fields in the way that is inconsistant with RFC2822

[william(at)elan.net]

Hello Brian,

With IESG already considering other issues with publication of
draft-lyon-senderid-core as experimental RFC, I'd like to request it
also formerly consider and make determination in regards to issues
raised at the very end of MARID regarding use of Resent- header fields
by draft-lyon-senderid-core and draft-lyon-senderid-pra which appears
to be in violation of the RFC2822 intended use of those fields.

The more fundamental problem in regards to this issue and the issue
raised by Julian Mehnle is that it appears that so-called SenderID
experiment is not limited to only those who wish to participate it,
but has effects on other internet users, in particular those who want
to use only SPF (Classic SPF) as well as those who interpret and use
Resent- fields (they can not be certain when looking at received email
if fields had been added by user action at MUA in according with RFC2822
or in automated manner by forwarding entity as required by senderid)
and this problem must be addressed before such publication is approved
by IESG as official IETF experiment.

The details of the appeal listed below are largely copy of my previous
message to IESG on this subject, those interested can also see:
http://www.imc.org/ietf-mxcomp/mail-archive/msg04713.html
http://www.imc.org/ietf-822/mail-archive/msg05670.html
http://www.imc.org/ietf-mxcomp/mail-archive/msg05774.html


Details of the objection in regards to RFC2822 specified Resent fields:

1. Recommendations for forwarders incompatible with RFC2822

Ref: http://www.ietf.org/internet-drafts/draft-lyon-senderid-core-01.txt

| 7.2 E-Mail Forwarders
|
| In order to pass the PRA variant of the test, a program that forwards
| received mail to other addresses MUST add an appropriate header that
| contains an e-mail address that it is authorized to use. Such
| programs SHOULD use the Resent-From header for this purpose.

The above is is incompatible with the section 3.6.6 of RFC2822:

Ref: http://www.ietf.org/rfc/rfc2822.txt

| Note: Reintroducing a message into the transport system and using
| resent fields is a different operation from "forwarding".
| "Forwarding" has two meanings: One sense of forwarding is that a mail
| reading program can be told by a user to forward a copy of a message
| to another person, making the forwarded message the body of the new
| message. A forwarded message in this sense does not appear to have
| come from the original sender, but is an entirely new message from
| the forwarder of the message. On the other hand, forwarding is also
| used to mean when a mail transport program gets a message and
| forwards it on to a different destination for final delivery. Resent
| header fields are not intended for use with either type of
| forwarding.

This objection was discussed at MARID [1] and was found to be valid based
on the answer by Pete Resnick [2] and in subsequent discussion in jabber
conference [3].

Note: Same objection applies to section 7.3 of senderid-core document
which requires use of Resent- fields by mail lists

References:
[1] http://www.imc.org/ietf-mxcomp/mail-archive/msg04713.html
[2] http://www.imc.org/ietf-mxcomp/mail-archive/msg04972.html
[3] http://www.xmpp.org/ietf-logs/***@ietf.xmpp.org/2004-09-20.html


2. Use Resent-* headers for automatic action, which is incompatible
with RFC2822

Ref: http://www.ietf.org/rfc/rfc2822.txt (section 3.6.6):

| Resent fields are
| strictly informational. They MUST NOT be used in the normal
| processing of replies or other such automatic actions on messages.

I take "automatic action" to include rejection and bouncing of messages
and draft-lyon-senderid-core-01 recommends when PRA address derived from
Resent- header is not authorized based on corresponding SPF record check,
then message is to be rejected by mail system in automated manner.


3. PRA algorithm fails when message is RFC822 compliant,
but not RFC2822 compliant

Ref: http://www.ietf.org/internet-drafts/draft-lyon-senderid-pra-01.txt

| Where messages conform to RFC822 rather than RFC2822, it is possible
| for the algorithm to give unexpected results. An RFC822 message
| should not normally contain more than one set of resent headers;
| however the placement of those headers is not specified, nor are they
| required to be contiguous. It is hence possible that the Resent-From
| header will be selected even though a Resent-Sender header is present.
| Such cases are expected to be rare or non-existent in practice

The problem is acknowledged in the draft, however I disagree that it
should be ignored quite so easily because RFC822 is still listed as
Internet Standard where as RFC2822 is just Proposed Standard.


4. Resent-From can not be used without Resent-Date

Ref: http://www.ietf.org/rfc/rfc2822.txt (section 3.6.6):

| When resent fields are used, the "Resent-From:" and "Resent-Date:"
| fields MUST be sent. The "Resent-Message-ID:" field SHOULD be sent.
| Resent-Sender:" SHOULD NOT be used if "Resent-Sender:" would be
| identical to "Resent-From:".

While not specifically mentioned, I believe draft-lyon-senderid-core
document implies that only Resent-From is to be added by those forwarding
systems that participate in the experiment (this is difficult to confirm
because so far I've not found ANY forwarding system that is actually
adding Resent- as per senderid-core draft).

[Frank Ellermann]

william(at)elan.net wrote:

[...]

Post by william(at)elan.net
http://www.imc.org/ietf-mxcomp/mail-archive/msg05774.html

Yes, that was a very good article.

Post by william(at)elan.net
incompatible with RFC2822

I'm still a bit lost how this could actually _break_ something.
For obvious reasons the author can't say "updates 2822", how
should he fix it ? As you said the 822 issue is mentioned in
the senderid-pra draft.

Do you want more "security considerations", something along
the line of "PRA-participants agree to break an explicit MUST
in 2822" ?

Post by william(at)elan.net
I disagree that it should be ignored quite so easily because
RFC822 is still listed as Internet Standard where as RFC2822
is just Proposed Standard.

Touché. OTOH the author isn't responsible for this "detail" -
in the spirit of TINW (for an IETF-we) it's "our" fault. Bye.

[william(at)elan.net]

Post by Frank Ellermann

Post by william(at)elan.net
incompatible with RFC2822

I'm still a bit lost how this could actually _break_ something.
For obvious reasons the author can't say "updates 2822", how
should he fix it ? As you said the 822 issue is mentioned in
the senderid-pra draft.
Do you want more "security considerations", something along
the line of "PRA-participants agree to break an explicit MUST
in 2822" ?

Its not that easy because participants are not the only ones who
see messages from such forwarders (if they ever come into existence)
and so as I mentioned the draft actions are not limited to those who
agree to participate in the experiment (but at least and unlike
re-purposing of v=spf1 records this action would not cause failures
as you correctly note).

And since this actually breaks standard, this would seem to require
explicit action and consent from IESG with understanding about it
and update to RFC2822 with consensus and last call if properly done
(and there has not been even an attempt by draft authors to ask for
review at ietf-smtp or ietf-822 and do last-call like SPF draft
author has done voluntarily). I guess its also fundamental question
on how seriously does IETF really protect its own standards and
follows its own procedures on how they are to be changed.

As far as fixing it there are two ways to go about it (and you're
correct that I should have mentioned considering its an appeal and
there I have to say, not only what I believe is wrong but how it
can be fixed):

1. As I already mentioned, IETF can do separate draft that updates
RFC2822 and illuminates ambiguities allowing for reuse of Resent-
header fields by forwarders. This new draft (and only that draft)
would have to go to standard track as proposed standard with appropriate
review and last call same as RFC2822.

2. Without changing standard, SID for its experiment (and since
none of the systems adding Resent are adding it the way they want
right now anyway, so no changes to running systems) can just create
new header field and use that - this possibility has been mentioned
at MARID btw.

[Tony Finch]

Post by Frank Ellermann

Post by william(at)elan.net
incompatible with RFC2822

I'm still a bit lost how this could actually _break_ something.
For obvious reasons the author can't say "updates 2822", how
should he fix it ? As you said the 822 issue is mentioned in
the senderid-pra draft.

Regarding RFC 822, the S-ID draft doesn't mention the fact that a large
proportion of software which does something useful with Resent- headers
still implements the 822 syntax, not the 2822 syntax.

Tony.

[Frank Ellermann]

Post by Tony Finch

the author can't say "updates 2822", how should he fix it ?
As you said the 822 issue is mentioned in the senderid-pra
draft.

Regarding RFC 822, the S-ID draft doesn't mention the fact
that a large proportion of software which does something
useful with Resent- headers still implements the 822 syntax,
not the 2822 syntax.

Except from all PRA-purposes and maybe MUAs displaying these
Resent-* header fields somehow (822 or 2822): What other uses
do you have in mind ?

Maybe that was discussed somewhere in MARID last year, in that
case I either missed it or didn't get it.

A related issue, apparently 2822 twisted the syntax (more than
one Resent-* block allowed) _and_ the semantics. Dave's "old"
822 concept could reflect "forwarding" (maybe - I'm not sure).

If that's true 822-Resent-* and PRA are semantically similar,
the only missing piece would be to either allow more than one
"block" (like 2822), or "invalidate" old Resent-* header fields
somehow, e.g. William's Original-* idea.

While I'm at it: How's that supposed to work with RfC 2476bis
8.1 "MAY add Sender" ? Apparently 2476bis 8.1 is still based
on the old 822-concept of Resent-* (?)

Or does it ignore the issue ? Is this part of the PRA-mess in
fact "our" fault (for a definition of TINW including everybody
who reviewed the gellens-submit drafts) ?

Of course I watched 2476 6.1 (and also 8.1), it's essential for
stuff like draft-hutzler-spamops or "op=auth", but something
with Resent-* is very odd, and it's not necessarily PRA. Bye

[Tony Finch]

Post by Frank Ellermann

Post by Tony Finch
Regarding RFC 822, the S-ID draft doesn't mention the fact
that a large proportion of software which does something
useful with Resent- headers still implements the 822 syntax,
not the 2822 syntax.

Except from all PRA-purposes and maybe MUAs displaying these
Resent-* header fields somehow (822 or 2822): What other uses
do you have in mind ?

Ignoring PRA, there are two current sides to the handling of Resent-*:
message submission and message display. An 822 display end can probably
muddle along OK when presented by a 2822 message. An 822 message
submission server will probably do something wrong when fixing up a 2822
message that has been re-sent more than once. A 2822 submission server
will have to have some fairly good heuristics to gracefully handle 822
re-sent messages. Things get even more interesting if an 822-re-sent
message is then 2822-re-sent, because the header order of the original
re-sending has to be fixed.

Sender-ID effecively requires that this kind of submission server fix-up
must be implemented by MTAs and MDAs that do aliasing / redirecting /
forwarding.

Post by Frank Ellermann
While I'm at it: How's that supposed to work with RfC 2476bis
8.1 "MAY add Sender" ? Apparently 2476bis 8.1 is still based
on the old 822-concept of Resent-* (?)

It doesn't address Resent-* at all so one would have to work out for
oneself how a submission server should treat them. This is clearly a bug.
(Is it too late to fix submit-bis now it is in the RFC Editor queue?)
Specifying what to do with them is, however, tricky. Sender-ID lacks this
specification too.

Tony.

[Brian E Carpenter]

William,

We will consider this together with the other appeal.


- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Brian E Carpenter
IETF Chair
Distinguished Engineer, Internet Standards & Technology, IBM

Post by william(at)elan.net
Hello Brian,
With IESG already considering other issues with publication of
draft-lyon-senderid-core as experimental RFC, I'd like to request it
also formerly consider and make determination in regards to issues
raised at the very end of MARID regarding use of Resent- header fields
by draft-lyon-senderid-core and draft-lyon-senderid-pra which appears
to be in violation of the RFC2822 intended use of those fields.
The more fundamental problem in regards to this issue and the issue
raised by Julian Mehnle is that it appears that so-called SenderID
experiment is not limited to only those who wish to participate it,
but has effects on other internet users, in particular those who want
to use only SPF (Classic SPF) as well as those who interpret and use
Resent- fields (they can not be certain when looking at received email
if fields had been added by user action at MUA in according with RFC2822
or in automated manner by forwarding entity as required by senderid)
and this problem must be addressed before such publication is approved
by IESG as official IETF experiment.
The details of the appeal listed below are largely copy of my previous
http://www.imc.org/ietf-mxcomp/mail-archive/msg04713.html
http://www.imc.org/ietf-822/mail-archive/msg05670.html
http://www.imc.org/ietf-mxcomp/mail-archive/msg05774.html
1. Recommendations for forwarders incompatible with RFC2822
Ref: http://www.ietf.org/internet-drafts/draft-lyon-senderid-core-01.txt
| 7.2 E-Mail Forwarders
|
| In order to pass the PRA variant of the test, a program that forwards
| received mail to other addresses MUST add an appropriate header that
| contains an e-mail address that it is authorized to use. Such
| programs SHOULD use the Resent-From header for this purpose.
Ref: http://www.ietf.org/rfc/rfc2822.txt
| Note: Reintroducing a message into the transport system and using
| resent fields is a different operation from "forwarding".
| "Forwarding" has two meanings: One sense of forwarding is that a mail
| reading program can be told by a user to forward a copy of a message
| to another person, making the forwarded message the body of the new
| message. A forwarded message in this sense does not appear to have
| come from the original sender, but is an entirely new message from
| the forwarder of the message. On the other hand, forwarding is also
| used to mean when a mail transport program gets a message and
| forwards it on to a different destination for final delivery. Resent
| header fields are not intended for use with either type of
| forwarding.
This objection was discussed at MARID [1] and was found to be valid
based on the answer by Pete Resnick [2] and in subsequent discussion in
jabber conference [3].
Note: Same objection applies to section 7.3 of senderid-core document
which requires use of Resent- fields by mail lists
[1] http://www.imc.org/ietf-mxcomp/mail-archive/msg04713.html
[2] http://www.imc.org/ietf-mxcomp/mail-archive/msg04972.html
2. Use Resent-* headers for automatic action, which is incompatible
with RFC2822
| Resent fields are
| strictly informational. They MUST NOT be used in the normal
| processing of replies or other such automatic actions on messages.
I take "automatic action" to include rejection and bouncing of messages
and draft-lyon-senderid-core-01 recommends when PRA address derived from
Resent- header is not authorized based on corresponding SPF record
check, then message is to be rejected by mail system in automated manner.
3. PRA algorithm fails when message is RFC822 compliant,
but not RFC2822 compliant
Ref: http://www.ietf.org/internet-drafts/draft-lyon-senderid-pra-01.txt
| Where messages conform to RFC822 rather than RFC2822, it is possible
| for the algorithm to give unexpected results. An RFC822 message
| should not normally contain more than one set of resent headers;
| however the placement of those headers is not specified, nor are they
| required to be contiguous. It is hence possible that the Resent-From
| header will be selected even though a Resent-Sender header is present.
| Such cases are expected to be rare or non-existent in practice
The problem is acknowledged in the draft, however I disagree that it
should be ignored quite so easily because RFC822 is still listed as
Internet Standard where as RFC2822 is just Proposed Standard.
4. Resent-From can not be used without Resent-Date
| When resent fields are used, the "Resent-From:" and "Resent-Date:"
| fields MUST be sent. The "Resent-Message-ID:" field SHOULD be sent.
| Resent-Sender:" SHOULD NOT be used if "Resent-Sender:" would be
| identical to "Resent-From:".
While not specifically mentioned, I believe draft-lyon-senderid-core
document implies that only Resent-From is to be added by those
forwarding systems that participate in the experiment (this is difficult
to confirm because so far I've not found ANY forwarding system that is
actually adding Resent- as per senderid-core draft).