So here it is one year later...

[Gordon Fecyk]

...since the first rumblings of talking about a working group, and there's
been no press on MARID since the breakup of said working group. No press
from Microsoft, no press from the SPF crowd, no press from anyone.

And no instructions included on how to compile, install, or use what little
software is available out there. Except for the few commercial (and
expensive) offerings by GFi.

What gives? Has the whole world lost complete interest in stopping spam? Is
spyware really the next big threat to the Internet that the US Congress is
looking at legislation for it but not bothering with spam anymore?

--
PGP key (0x0AFA039E): <http://www.pan-am.ca/***@pan-am.ca.asc>
Sometimes it's hard to tell where the game ends and where reality bites,
er, begins. <http://vmyths.com/resource.cfm?id=50&page=1>

[Douglas Otis]

On Thu, 2005-01-27 at 09:33 -0600, Gordon Fecyk wrote:
>
> What gives? Has the whole world lost complete interest in stopping spam? Is
> spyware really the next big threat to the Internet that the US Congress is
> looking at legislation for it but not bothering with spam anymore?

There is a signature based effort ongoing with Yahoo and Cisco.

http://www.mipassoc.org/mass/

One hopes details of merging strategies is being discussed, as it does
not appear to be happening much on the mailing list. The technology
looks very promising from the perspective of protecting people.
(Assuming there is a means to revoke authorization granted by way of the
signature.)

http://www.mipassoc.org/clear/

Work is still happening and a new draft is about to be released. This
approach is compatible with mass efforts, but offers a means to protect
networks with similar name based instrument.

The pressure is still very much present. The entire effort is about
improving security by way of authentication. The lack of security
remains the principle enabler. Security places focus upon the
client/server administrator, which is the entity authenticated by way of
efforts in MASS and CLEAR.

-Doug

[Frank Ellermann]

Gordon Fecyk wrote:

> no press from the SPF crowd

The "SPF crowd" has a position on Sender-ID (Dec 2004):
<http://www.OpenSPF.org/>

A council was elected (December 2004): <http://spf.mehnle.net/>

An updated draft was published (January 2005):
<http://www.ietf.org/internet-drafts/draft-schlitt-spf-classic-00.txt>

SpamCop recommends SPF and / or domain keys (yesterday):
<http://www.spamcop.net/fom-serve/cache/329.html>

Other relatively new activities include:
<http://www.ietf.org/internet-drafts/draft-crocker-email-arch-02.txt>
<http://www.ietf.org/internet-drafts/draft-housley-mass-sec-review-00.txt>
<http://www.ietf.org/internet-drafts/draft-irtf-asrg-dnsbl-01.txt>
<http://www.ietf.org/internet-drafts/draft-stumpf-dns-mtamark-03.txt>

Bye, Frank (some remarks about RfC 3865 censored)

[Douglas Otis]

On Thu, 2005-01-27 at 20:37 +0100, Frank Ellermann wrote:
> Gordon Fecyk wrote:
>
> > no press from the SPF crowd

pobox.com still has the slogan:
Sender Policy Framework, an essential part of Sender ID.

And although the link has been modified, it continues to indicate the
current SPF draft is:

http://www.ozonehouse.com/mark/spf/draft-lentczner-spf-00.txt

> The "SPF crowd" has a position on Sender-ID (Dec 2004):

Quote from this faction:

"Many SPF developers and users consider that Microsoft's SenderID
proposal is technically unsound and undermines the progress already
begun by SPF. SPF development and deployment predate Microsoft's entry
into the field and have achieved significant success to date. SenderID,
in its most recent form, appears likely to interfere with the correct
function and deployment of SPF."
...

"Where SenderID breaks the function of existing v=spf1 records, domain
owners will only learn of it when legitimate mail is not delivered. SPF
record publishers made their records public with the expectation that
they would be used with the SPF algorithm. SenderID puts those records
to a different, possibly incompatible use, without any consent from the
publishers."

Agreed; applying a record against different algorithms than that
intended when published is inherently deleterious, even though SPF still
prevents delivery of mail from forwarding and some list servers. : (

> An updated draft was published (January 2005):
> <http://www.ietf.org/internet-drafts/draft-schlitt-spf-classic-00.txt>

Once again the algorithm changes and still this draft uses the same
labels and record identifiers? Classic.

Endorsements from those that block all bounce traffic are not likely
concerned about mail lost as a result of forwarding or being from
various list servers, so it would seem. Based upon these comments,
SPF/Sender-ID reduces mail integrity in the bargain.

-Doug

[Frank Ellermann]

Douglas Otis wrote:

> pobox.com still has the slogan:
> Sender Policy Framework, an essential part of Sender ID.

Yes, the pobox site is not exactly up to date, that's AFAIK on
the "todo" list of the new council. The new SPF draft is more
important.

> it continues to indicate the current SPF draft is:
> http://www.ozonehouse.com/mark/spf/draft-lentczner-spf-00.txt

Meng is one of the 5 SPF council members, they all agree that
the new draft is _the_ draft. He's also one of the 2 authors.

Mark had no time for this project after draft-lentczner-spf-00,
otherwise there'd be probably three authors of the new draft,
if that's allowed by the IETF and supported by xml2rfc.

[Sender-ID position]
> Quote from this faction:

The majority fraction, if you insist on these political terms.
<http://OpenSPF.org/cgi-bin/openspf_pledge.cgi> counts 129
signatures. The SPF council was elected by 161 voters. Not
exactly the same constituency, but roughly the same group of
people. Finding a better text supported by almost all SPF
"fans" is (or was) also on the "todo" list of the new council.

> applying a record against different algorithms than that
> intended when published is inherently deleterious

Indeed.

> Once again the algorithm changes and still this draft uses
> the same labels and record identifiers? Classic.

This "new" draft is technically nearer to the last pre-MARID
"old" draft than draft-lentczner-spf-00 was. The latter was a
rather quick hack salvaging all syntax improvements found here
(= mxcomp) after MARID was killed and the old draft expired.

The "new" draft kept these syntactical improvements, removed
some oddities with the handling of errors, and added all things
from the old draft forgotten in draft-lentczer-spf-00. There
are almost no semantical differences between "old" and "new".

Some of the minor differences between lentczner and new / old
were simply errors, e.g. lentczer would FAIL for a MAIL FROM
domain literal. While that might be seen as a feature it was a
case of "receiver policy" and not "sender policy", and the new
draft fixed it (back to the state of the old draft).

Only side effects of the MARID disaster, no SPF problem, no new
algorithm, no differences for existing policies. Only a better
error handling and more restrictive limits for DNS queries, no
substantial changes.
Bye, Frank

[Douglas Otis]

On Fri, 2005-01-28 at 04:20 +0100, Frank Ellermann wrote:
> Douglas Otis wrote:

> > applying a record against different algorithms than that
> > intended when published is inherently deleterious
>
> Indeed.
>
> > Once again the algorithm changes and still this draft uses
> > the same labels and record identifiers? Classic.
>
> This "new" draft is technically nearer to the last pre-MARID
> "old" draft than draft-lentczner-spf-00 was. The latter was a
> rather quick hack salvaging all syntax improvements found here
> (= mxcomp) after MARID was killed and the old draft expired.

This draft is attempting to make significant algorithmic changes to the
initial draft, as well as how these records are used.

Some examples:

SPF clients MAY check the "HELO" identity by calling the check_host()
function (Section 4) with the "HELO" identity as the <sender>. If
the HELO test returns a "fail", the overall result for the SMTP
session is "fail", and there is no need to test the "MAIL FROM"
identity.

A test against HELO that goes from unknown, to not tested, to fail?
This a change in algorithm, as is the hunt for alternative records,
where a pass may not be based upon address compliance. One wonders how
macros are applied when the same check_host routine is recycled.

An SPF record published at the zone cut for the domain will be used
as a default for all subdomains within the zone (See Section 4.5.)
Domain owners SHOULD publish SPF records for hosts used for the HELO
and MAIL FROM identities instead of using the zone cut default
because the fallback requires additional DNS lookups. The zone cut
default does reduce the need to publish SPF records for non-email
related hosts, such as www.example.com.

Again, another change in algorithm. This also means TXT RR placed at
the zone apex may now be problematic as how it becomes applied and to
which identity.

If no matching records are returned for the <domain;>, the SPF client
MUST find the Zone Cut as defined in [RFC2181] section 6 and repeat
the above steps. The <domain>'s zone origin is then searched for SPF
records. If an SPF record is found at the zone origin, the <domain>
is set to the zone origin as if a "redirect" modifier was executed.

If no matching records are returned for either search, an SPF client
MUST assume that the domain makes no SPF declarations. SPF
processing MUST abort and return "None".

Yet again another change in algorithm isn't it?

This mechanism matches if <ip> is one of the MX hosts for a domain
name.

MX = "mx" [ ":" domain-spec ] [ dual-cidr-length ]

check_host() first performs an MX lookup on the <target-name>. Then
it performs an address lookup on each MX name returned. The <ip> is
compared to each returned IP address. To prevent DoS attacks, a
limit of 10 MX names MUST be enforced (see Section 10). If any
address matches, the mechanism matches.

A limit change is an algorithmic change that could not be possibly
foreseen by earlier publishers. Who is responsible when their mail goes
missing?

Note regarding implicit MXes: If the <target-name> has no MX records,
check_host() MUST NOT pretend the target is its single MX, and MUST
NOT default to an A lookup on the <target-name> directly. This
behavior breaks with the legacy "implicit MX" rule. See [RFC2821]
Section 5. If such behavior is desired, the publisher should specify
an "a" directive.

Would this be yet another algorithm change?

In pseudocode:

sending-domain_names := ptr_lookup(sending-host_IP);
if more than 10 sending-domain_names are found, use at most 10.
for each name in (sending-domain_names) {
IP_addresses := a_lookup(name);
if the sending-domain_IP is one of the IP_addresses {
validated-sending-domain_names += name;
}
}

Yet another change to the algorithm that introduces a new limit.

Pseudocode:

for each name in (validated-sending-domain_names) {
if name ends in <domain-spec>, return match.
if name is <domain-spec>, return match.
}
return no-match.

This mechanism matches if the <target-name> is either an ancestor of
a validated domain name, or if the <target-name> and a validated
domain name are the same. For example: "mail.example.com" is within
the domain "example.com", but "mail.bad-example.com" is not.

Note: Use of this mechanism is discouraged because it is slow, is not
as reliable as other mechanisms in cases of DNS errors and it places
a large burden on the arpa name servers. If used, proper PTR records
must be in place for the domain's hosts and the "ptr" mechanism
should be one of the last mechanisms checked.

One wonders how this mechanism is to be discouraged?

SPF implementations MUST limit the number of mechanism that do DNS
lookups to at most 10, if this number is exceeded, a PermError MUST
be returned. The mechanisms that count against this limit are
"include", "a", "mx", "ptr", "exists" and the "redirect" modifier.
The "all", "ip4" and "ip6" mechanisms do not require DNS lookups and
therefore do not count against this limit. The "exp" modifier
requires a DNS lookup, but it is not counted as it is used only in
the case of errors.

When evaluating the "mx" and "ptr" mechanisms, or the %{p} macro,
there MUST be a limit of no more than 10 MX or PTR RRs looked up and
checked.

I would also describe this as an algorithmic change that could not have
been anticipated by the publishers. And the following regarding
forwarding:

There are several possible ways that this authorization failure can
be ameliorated. If the owner of the external mailbox wishes to trust
the forwarding service, they can direct the external mailbox's MTA to
skip such tests when the client host belongs to the forwarding
service. Tests against some other identity may also be used to
override the test against the "MAIL FROM" identity.

For larger domains, it may not be possible to have a complete or
accurate list of forwarding services used by the owners of the
domain's mailboxes. In such cases, white lists of generally
recognized forwarding services could be employed.

Some other Identity? Generally recognized forwarding services? Would
this open the door for abusing the typical alma mater? This is a mess.

This draft at long last recognizes wildcard labels are a problem. Why
not also recognize a need to _change_ revisions when algorithms change
and the need for a standardized prefix rather than a record tag? These
are rather significant changes being made, why suggest otherwise?

-Doug

[wayne]

The MARID list has become a cesspool of misinformation and trolls.
For that reason, I have not been posting to it and I would encourage
others to ignore the "information" posted here. The following post by
Doug Otis is an prime example of just how bad most posts here are.


In <***@SJC-Office-DHCP-156.mail-abuse.org> Douglas Otis <***@mail-abuse.org> writes:

> On Fri, 2005-01-28 at 04:20 +0100, Frank Ellermann wrote:
>> Douglas Otis wrote:
>
>> > applying a record against different algorithms than that
>> > intended when published is inherently deleterious
>>
>> Indeed.
>>
>> > Once again the algorithm changes and still this draft uses
>> > the same labels and record identifiers? Classic.
>>
>> This "new" draft is technically nearer to the last pre-MARID
>> "old" draft than draft-lentczner-spf-00 was. The latter was a
>> rather quick hack salvaging all syntax improvements found here
>> (= mxcomp) after MARID was killed and the old draft expired.
>
> This draft is attempting to make significant algorithmic changes to the
> initial draft, as well as how these records are used.

Discussions about the SPF specification are best discussed on the
SPF-discuss mailing list.



> Some examples:
>
> SPF clients MAY check the "HELO" identity by calling the check_host()
> function (Section 4) with the "HELO" identity as the <sender>. If
> the HELO test returns a "fail", the overall result for the SMTP
> session is "fail", and there is no need to test the "MAIL FROM"
> identity.
>
> A test against HELO that goes from unknown, to not tested, to fail?
> This a change in algorithm, as is the hunt for alternative records,
> where a pass may not be based upon address compliance. One wonders how
> macros are applied when the same check_host routine is recycled.


This is not an example of changing semantics and/or algorithms. As
pointed out by Frank, this is consistent with the pre-MARID
specification of SPF. The problem is not that draft-schlitt-spf-00
changed things back, but that the IETF, via the MARID WG, allowed it
to change in the first place.


> An SPF record published at the zone cut for the domain will be used
> as a default for all subdomains within the zone (See Section 4.5.)
> Domain owners SHOULD publish SPF records for hosts used for the HELO
> and MAIL FROM identities instead of using the zone cut default
> because the fallback requires additional DNS lookups. The zone cut
> default does reduce the need to publish SPF records for non-email
> related hosts, such as www.example.com.
>
> Again, another change in algorithm. This also means TXT RR placed at
> the zone apex may now be problematic as how it becomes applied and to
> which identity.

This is not an example of changing semantics and/or algorithms. As
pointed out by Frank, this is consistent with the pre-MARID
specification of SPF. The problem is not that draft-schlitt-spf-00
changed things back, but that the IETF, via the MARID WG, allowed it
to change in the first place.


> If no matching records are returned for the <domain;>, the SPF client
> MUST find the Zone Cut as defined in [RFC2181] section 6 and repeat
> the above steps. The <domain>'s zone origin is then searched for SPF
> records. If an SPF record is found at the zone origin, the <domain>
> is set to the zone origin as if a "redirect" modifier was executed.
>
> If no matching records are returned for either search, an SPF client
> MUST assume that the domain makes no SPF declarations. SPF
> processing MUST abort and return "None".
>
> Yet again another change in algorithm isn't it?

This is not an example of changing semantics and/or algorithms. As
pointed out by Frank, this is consistent with the pre-MARID
specification of SPF. The problem is not that draft-schlitt-spf-00
changed things back, but that the IETF, via the MARID WG, allowed it
to change in the first place.


> This mechanism matches if <ip> is one of the MX hosts for a domain
> name.
>
> MX = "mx" [ ":" domain-spec ] [ dual-cidr-length ]
>
> check_host() first performs an MX lookup on the <target-name>. Then
> it performs an address lookup on each MX name returned. The <ip> is
> compared to each returned IP address. To prevent DoS attacks, a
> limit of 10 MX names MUST be enforced (see Section 10). If any
> address matches, the mechanism matches.
>
> A limit change is an algorithmic change that could not be possibly
> foreseen by earlier publishers. Who is responsible when their mail goes
> missing?

While this is, indeed, a change from the last pre-MARID SPF spec, this
limit has been in place in my libspf2 implemenation since long before
MARID existed, and hence is an existing practice. As discussed *many*
times on SPF-discuss, I can not find *any* legitimate emailers who
have come close to this limit. This change only effects abusers and
misconfigured systems.


> Note regarding implicit MXes: If the <target-name> has no MX records,
> check_host() MUST NOT pretend the target is its single MX, and MUST
> NOT default to an A lookup on the <target-name> directly. This
> behavior breaks with the legacy "implicit MX" rule. See [RFC2821]
> Section 5. If such behavior is desired, the publisher should specify
> an "a" directive.
>
> Would this be yet another algorithm change?

No. This has *always* been the case.

> In pseudocode:
>
> sending-domain_names := ptr_lookup(sending-host_IP);
> if more than 10 sending-domain_names are found, use at most 10.
> for each name in (sending-domain_names) {
> IP_addresses := a_lookup(name);
> if the sending-domain_IP is one of the IP_addresses {
> validated-sending-domain_names += name;
> }
> }
>
> Yet another change to the algorithm that introduces a new limit.

While this is, indeed, a change from the last pre-MARID SPF spec, this
limit has been in place in my libspf2 implemenation since long before
MARID existed, and hence is an existing practice. As discussed *many*
times on SPF-discuss, I can not find *any* legitimate emailers who
have come close to this limit. This change only effects abusers and
misconfigured systems.


> Pseudocode:
>
> for each name in (validated-sending-domain_names) {
> if name ends in <domain-spec>, return match.
> if name is <domain-spec>, return match.
> }
> return no-match.
>
> This mechanism matches if the <target-name> is either an ancestor of
> a validated domain name, or if the <target-name> and a validated
> domain name are the same. For example: "mail.example.com" is within
> the domain "example.com", but "mail.bad-example.com" is not.
>
> Note: Use of this mechanism is discouraged because it is slow, is not
> as reliable as other mechanisms in cases of DNS errors and it places
> a large burden on the arpa name servers. If used, proper PTR records
> must be in place for the domain's hosts and the "ptr" mechanism
> should be one of the last mechanisms checked.
>
> One wonders how this mechanism is to be discouraged?

By saying, in the spec "this mechanism is discouraged" and by having
the various SPF wizards not generate it.


> SPF implementations MUST limit the number of mechanism that do DNS
> lookups to at most 10, if this number is exceeded, a PermError MUST
> be returned. The mechanisms that count against this limit are
> "include", "a", "mx", "ptr", "exists" and the "redirect" modifier.
> The "all", "ip4" and "ip6" mechanisms do not require DNS lookups and
> therefore do not count against this limit. The "exp" modifier
> requires a DNS lookup, but it is not counted as it is used only in
> the case of errors.
>
> When evaluating the "mx" and "ptr" mechanisms, or the %{p} macro,
> there MUST be a limit of no more than 10 MX or PTR RRs looked up and
> checked.
>
> I would also describe this as an algorithmic change that could not have
> been anticipated by the publishers.

While this is, indeed, a change from the last pre-MARID SPF spec, this
limit has been in place in my libspf2 implemenation since long before
MARID existed, and hence is an existing practice. As discussed *many*
times on SPF-discuss, I can not find *any* legitimate emailers who
have come close to this limit. This change only effects abusers and
misconfigured systems.


> And the following regarding
> forwarding:
>
> There are several possible ways that this authorization failure can
> be ameliorated. If the owner of the external mailbox wishes to trust
> the forwarding service, they can direct the external mailbox's MTA to
> skip such tests when the client host belongs to the forwarding
> service. Tests against some other identity may also be used to
> override the test against the "MAIL FROM" identity.
>
> For larger domains, it may not be possible to have a complete or
> accurate list of forwarding services used by the owners of the
> domain's mailboxes. In such cases, white lists of generally
> recognized forwarding services could be employed.
>
> Some other Identity? Generally recognized forwarding services? Would
> this open the door for abusing the typical alma mater? This is a mess.

As noted in Section 9, this is is non-normative. If you don't agree
with what is said, you are free to do anything you want. This is just
for your information.



> This draft at long last recognizes wildcard labels are a problem. Why
> not also recognize a need to _change_ revisions when algorithms change
> and the need for a standardized prefix rather than a record tag? These
> are rather significant changes being made, why suggest otherwise?

The section on wildcards was added as part of the MARID process and
remained in the draft-schlitt-spf-00 version because it is useful.
The changes to the algorithms that were added as part of the MARID
process have been removed. Changes to the algorigtms from the
pre-MARID SPF specs have all been implemented by at least one system
and have been checked, via wide surveys, that they do not conflict
the install base of legitimate SPF records.


I find it very amusing that you are now complaining about the process
limits being a change, since you have long complained about the lack
of process limits in previous versions of the SPF spec.


-wayne

[Douglas Otis]

On Fri, 2005-01-28 at 13:20 -0600, wayne wrote:
> > On Fri, 2005-01-28 at 04:20 +0100, Frank Ellermann wrote:
> >> Douglas Otis wrote:
> >
> >> > applying a record against different algorithms than that
> >> > intended when published is inherently deleterious
> >>
> >> Indeed.
> >>
> >> > Once again the algorithm changes and still this draft uses
> >> > the same labels and record identifiers? Classic.
> >>
> >> This "new" draft is technically nearer to the last pre-MARID
> >> "old" draft than draft-lentczner-spf-00 was. The latter was a
> >> rather quick hack salvaging all syntax improvements found here
> >> (= mxcomp) after MARID was killed and the old draft expired.
> >
> > This draft is attempting to make significant algorithmic changes to the
> > initial draft, as well as how these records are used.
> >
> > Some examples:
> >
> > SPF clients MAY check the "HELO" identity by calling the check_host()
> > function (Section 4) with the "HELO" identity as the <sender>. If
> > the HELO test returns a "fail", the overall result for the SMTP
> > session is "fail", and there is no need to test the "MAIL FROM"
> > identity.
> >
> > A test against HELO that goes from unknown, to not tested, to fail?
> > This a change in algorithm, as is the hunt for alternative records,
> > where a pass may not be based upon address compliance. One wonders how
> > macros are applied when the same check_host routine is recycled.
>
> This is not an example of changing semantics and/or algorithms. As
> pointed out by Frank, this is consistent with the pre-MARID
> specification of SPF. The problem is not that draft-schlitt-spf-00
> changed things back, but that the IETF, via the MARID WG, allowed it
> to change in the first place.

Here is a quote for the pre-MARID draft regarding the specified
processing algorithm limits.

6.2 Processing Limits

During processing, an SPF client may perform additional SPF
subqueries due to the Include mechanism and the Redirect modifier.

SPF clients must be prepared to handle records that are set up
incorrectly or maliciously. SPF clients MUST perform loop detection,
limit SPF recursion, or both. If an SPF client chooses to limit
recursion depth, then at least a total of 20 redirects and includes
SHOULD be supported. (This number should be enough for even the most
complicated configurations.)

If a loop is detected, or if more than 20 subqueries are triggered,
an SPF client MAY abort the lookup and return the result "unknown".

Regular non-recursive lookups due to mechanisms like "a" and "mx" or
due to modifiers like "exp" do not count toward this total

This new draft is NOT a compatible change. It should also be noted this
pre-MARID process could entail orders of magnitude more lookups.

> > An SPF record published at the zone cut for the domain will be used
> > as a default for all subdomains within the zone (See Section 4.5.)
> > Domain owners SHOULD publish SPF records for hosts used for the HELO
> > and MAIL FROM identities instead of using the zone cut default
> > because the fallback requires additional DNS lookups. The zone cut
> > default does reduce the need to publish SPF records for non-email
> > related hosts, such as www.example.com.
> >
> > Again, another change in algorithm. This also means TXT RR placed at
> > the zone apex may now be problematic as how it becomes applied and to
> > which identity.
>
> This is not an example of changing semantics and/or algorithms. As
> pointed out by Frank, this is consistent with the pre-MARID
> specification of SPF. The problem is not that draft-schlitt-spf-00
> changed things back, but that the IETF, via the MARID WG, allowed it
> to change in the first place.

You should review which records would be used to evaluate the EHLO/HELO.
And yet another quote from Pre-MARID.

The <responsible-sender> comes from the domain name of the "MAIL
FROM" envelope sender. When the envelope sender has no domain, a
client MUST use the HELO domain instead. If the HELO argument does
not provide an FQDN, SPF processing terminates with "unknown".

Now the record applied against the EHLO/HELO identity could be found at
the zone apex AND at the EHLO/HELO. The results of this check have also
changed and is not consistent.

> > If no matching records are returned for the <domain;>, the SPF client
> > MUST find the Zone Cut as defined in [RFC2181] section 6 and repeat
> > the above steps. The <domain>'s zone origin is then searched for SPF
> > records. If an SPF record is found at the zone origin, the <domain>
> > is set to the zone origin as if a "redirect" modifier was executed.
> >
> > If no matching records are returned for either search, an SPF client
> > MUST assume that the domain makes no SPF declarations. SPF
> > processing MUST abort and return "None".
> >
> > Yet again another change in algorithm isn't it?
>
> This is not an example of changing semantics and/or algorithms. As
> pointed out by Frank, this is consistent with the pre-MARID
> specification of SPF. The problem is not that draft-schlitt-spf-00
> changed things back, but that the IETF, via the MARID WG, allowed it
> to change in the first place.

This changes the pre-MARID, mid-MARID, and post-MARID algorithms! The
publisher can NOT be sure which record will be applied against their
HELO. It goes from virtually a don't care to a failure. The fall-back,
as it is called, will also likely conflict with records at the zone
apex. Complain about mail lost as a result of the algorithms applied by
Sender-ID, but show some compunction regarding the effect of these
changes. All this could be avoided by changing the record identifier.

> > This mechanism matches if <ip> is one of the MX hosts for a domain
> > name.
> >
> > MX = "mx" [ ":" domain-spec ] [ dual-cidr-length ]
> >
> > check_host() first performs an MX lookup on the <target-name>. Then
> > it performs an address lookup on each MX name returned. The <ip> is
> > compared to each returned IP address. To prevent DoS attacks, a
> > limit of 10 MX names MUST be enforced (see Section 10). If any
> > address matches, the mechanism matches.
> >
> > A limit change is an algorithmic change that could not be possibly
> > foreseen by earlier publishers. Who is responsible when their mail goes
> > missing?
>
> While this is, indeed, a change from the last pre-MARID SPF spec, this
> limit has been in place in my libspf2 implemenation since long before
> MARID existed, and hence is an existing practice. As discussed *many*
> times on SPF-discuss, I can not find *any* legitimate emailers who
> have come close to this limit. This change only effects abusers and
> misconfigured systems.

I guess those that are affected are therefore illegitimate? To publish,
discover the limits in the code being used? Code that did not comply to
some or any draft? When are the next changes to the algorithm going to
be made? How can a disruption be avoided when there is NO MEANS to
introduce a new version without removal of the prior?

> > And the following regarding forwarding:
> >
> > There are several possible ways that this authorization failure can
> > be ameliorated. If the owner of the external mailbox wishes to trust
> > the forwarding service, they can direct the external mailbox's MTA to
> > skip such tests when the client host belongs to the forwarding
> > service. Tests against some other identity may also be used to
> > override the test against the "MAIL FROM" identity.
> >
> > For larger domains, it may not be possible to have a complete or
> > accurate list of forwarding services used by the owners of the
> > domain's mailboxes. In such cases, white lists of generally
> > recognized forwarding services could be employed.
> >
> > Some other Identity? Generally recognized forwarding services? Would
> > this open the door for abusing the typical alma mater? This is a mess.
>
> As noted in Section 9, this is is non-normative. If you don't agree
> with what is said, you are free to do anything you want. This is just
> for your information.

In essence, don't expect forwarding to function. May I comment that
this is disruptive and a mess without this comment being called
misinformation or myself as being a troll for taking time to read the
drafts presented on this reflector.

Regarding the debate about who uses SPF, spammers register more domains
than legitimate users, but at any instant, individual spammer domains
will not likely be used once consistently blocked by filters. Looking
at the numbers from the perspective of nominal traffic, versus published
domains with a "bad" record, these ratios _should_ be different.

> > This draft at long last recognizes wildcard labels are a problem. Why
> > not also recognize a need to _change_ revisions when algorithms change
> > and the need for a standardized prefix rather than a record tag? These
> > are rather significant changes being made, why suggest otherwise?
>
> The section on wildcards was added as part of the MARID process and
> remained in the draft-schlitt-spf-00 version because it is useful.
> The changes to the algorithms that were added as part of the MARID
> process have been removed. Changes to the algorigtms from the
> pre-MARID SPF specs have all been implemented by at least one system
> and have been checked, via wide surveys, that they do not conflict
> the install base of legitimate SPF records.
>
> I find it very amusing that you are now complaining about the process
> limits being a change, since you have long complained about the lack
> of process limits in previous versions of the SPF spec.

I have not complained about a reduction in the limits, but rather a
change that imperils the publishers. I was complaining this draft still
does not allow a reasonable method to change processing algorithms
without potentially creating sizeable disruption. This is due, in no
small part, from a false expectation that a wildcard label provided some
utility and is the reason for usurping the use of the TXT RR. This was
wrong and remains wrong. Using a prefix on the record remains a viable
method to ensure SPF offers less disruption to users as well as other
protocols. If I was right once...

-Doug

[wayne]

In <***@SJC-Office-DHCP-156.mail-abuse.org> Douglas Otis <***@mail-abuse.org> writes:

> On Fri, 2005-01-28 at 13:20 -0600, wayne wrote:
>> > On Fri, 2005-01-28 at 04:20 +0100, Frank Ellermann wrote:
>> >
>> >> This "new" draft is technically nearer to the last pre-MARID
>> >> "old" draft than draft-lentczner-spf-00 was. The latter was a
>> >> rather quick hack salvaging all syntax improvements found here
>> >> (= mxcomp) after MARID was killed and the old draft expired.
> [big snip]

> Here is a quote for the pre-MARID draft regarding the specified
> processing algorithm limits.
>
> 6.2 Processing Limits
>
> [stuff from, I'm pretty sure, spf-draft-200406 snipped]
>
> This new draft is NOT a compatible change. It should also be noted this
> pre-MARID process could entail orders of magnitude more lookups.

I disagree. In practice, at least, this is a very compatible change
because almost no one runs into the limits.



> You should review which records would be used to evaluate the EHLO/HELO.
> And yet another quote from Pre-MARID.
>
> The <responsible-sender> comes from the domain name of the "MAIL
> FROM" envelope sender. When the envelope sender has no domain, a
> client MUST use the HELO domain instead. If the HELO argument does
> not provide an FQDN, SPF processing terminates with "unknown".
>
> Now the record applied against the EHLO/HELO identity could be found at
> the zone apex AND at the EHLO/HELO. The results of this check have also
> changed and is not consistent.

Yes, the zone cut stuff, while in spf-draft-200406, is certainly the
thing that I'm most willing to remove from
draft-schlitt-spf-classic-00. It has some nice properties, but it has
some problems also.


> [another big snip]
>
> This changes the pre-MARID, mid-MARID, and post-MARID algorithms!

I'm not sure what you mean by this. The zonecut stuff is in
spf-draft-200406.


>> > [mx processing limit stuff snipped]
>>
>> While this is, indeed, a change from the last pre-MARID SPF spec, this
>> limit has been in place in my libspf2 implemenation since long before
>> MARID existed, and hence is an existing practice. As discussed *many*
>> times on SPF-discuss, I can not find *any* legitimate emailers who
>> have come close to this limit. This change only effects abusers and
>> misconfigured systems.
>
> I guess those that are affected are therefore illegitimate? To publish,
> discover the limits in the code being used? Code that did not comply to
> some or any draft? When are the next changes to the algorithm going to
> be made? How can a disruption be avoided when there is NO MEANS to
> introduce a new version without removal of the prior?

"A difference that makes no difference is no difference" -- spock

Yes, in theory, there are incompatible changes, in practice there
aren't.

> [yet another big snip]
>
> In essence, don't expect forwarding to function.

Yes, SPF and traditional alias/.forward type forwarding don't work
well together. The MAPS DUL list and all end users being able to have
their own MTA don't work well together. DomainKeys and mailing lists
don't work well together. NAT and many games/vpn/etc. don't work well
together. There are *lots* of changes that have happened that cause
other problems.

That section of the SPF-classic spec explains the situation. As an
email receiver, if you don't like it, you don't have to check SPF
records, check the MAPS DUL, check domain keys, or use NAT. Your box,
your rules.


> I have not complained about a reduction in the limits, but rather a
> change that imperils the publishers. I was complaining this draft still
> does not allow a reasonable method to change processing algorithms
> without potentially creating sizeable disruption. This is due, in no
> small part, from a false expectation that a wildcard label provided some
> utility and is the reason for usurping the use of the TXT RR. This was
> wrong and remains wrong. Using a prefix on the record remains a viable
> method to ensure SPF offers less disruption to users as well as other
> protocols. If I was right once...

Lots of stuff here that I guess we will just have to agree to disagree
on. I don't agree with all the choices that were made with SPF, some
due to hind sight, some due to differening opinions. It still remains
far and away the most deployed designated sender system out there. It
doesn't appear that we make any fatal mistakes. IMHO, some of the
stuff you stuggest would be "better" would, actually, have been a
fatal mistake.


-wayne

[Andrew Newton]

On Jan 28, 2005, at 2:20 PM, wayne wrote:

> The problem is not that draft-schlitt-spf-00
> changed things back, but that the IETF, via the MARID WG, allowed it
> to change in the first place.

This statement strikes me as particularly offensive. If the intent was
to have the IETF simply rubber-stamp SPF, making that clear up front
would have saved us all a lot of time.

-andy

[wayne]

In <4DD53697-7189-11D9-9CD0-***@hxr.us> Andrew Newton <***@hxr.us> writes:

> On Jan 28, 2005, at 2:20 PM, wayne wrote:
>
>> The problem is not that draft-schlitt-spf-00
>> changed things back, but that the IETF, via the MARID WG, allowed it
>> to change in the first place.
>
> This statement strikes me as particularly offensive. If the intent
> was to have the IETF simply rubber-stamp SPF, making that clear up
> front would have saved us all a lot of time.

MARID never adopted/considered SPF. When, at the interim meeting,
MARID assigned authors to create proposals, it was to create SenderID
with the use of the PRA. At that point, all compatibility with
SPF-classic was basically thrown out the window. Eventually, enough
people decided there were enough incompatibilities that it was decided
at the ietf-60(?) session that SenderID should use new records. For
some strange reason, the IETF is now considering the SenderID drafts
that still use the same records. Go figure.

I know of no one who asked or assumed that the IETF would rubber-stamp
SPF. I did say, several times, that adopting an existing system would
be a much better idea than trying to create something from scratch.
Both DMP and RMX, for example, would have been a better choices than
SenderID because both had more testing done on them. If MARID had
chosen to adopt either DMP or RMX, that would have been fine with me.


Now, back to message I was replying to, Doug talked about the
SPF-classic spec of draft-lentczner-spf-00. Like all SPF specs, it
was never adopted by MARID. The problem here is that Mark took the
drafts developed during MARID and tried to back-port them to be
SPF-classic drafts. The advantage of doing this is that a great deal
of wordsmithing, in particular work done by Mark, had gone into the
MARID drafts and they were much better language wise. Unfortunately,
they had the many semantic/algorithmic changes that the MARID group
made during the development of SenderID.

draft-schlitt-spf-classic-00 is an attempt to use the much better
language of the MARID drafts, but to restore functionality to match
what the install base of SPF systems is. This draft also has removed
several features that are unused in the wild, and has more strict
error handling/checking added.


So, not rubber-stamping SPF wasn't a problem. Creating a new spec
that changed the semantics of SPF records was.


-wayne

[Andrew Newton]

On Jan 29, 2005, at 12:32 AM, wayne wrote:
> For
> some strange reason, the IETF is now considering the SenderID drafts
> that still use the same records. Go figure.

If by same records, you mean anything TXT then I'm afraid you will have
to live with that... and any other new uses for TXT that come along.

> So, not rubber-stamping SPF wasn't a problem. Creating a new spec
> that changed the semantics of SPF records was.

But I suppose you mean the v=spf1 records and agree that you have a
point.... which is not what I thought you were saying. Sorry for
jumping to conclusions.

-andy

[wayne]

In <8617533D-7210-11D9-9CD0-***@hxr.us> Andrew Newton <***@hxr.us> writes:

> On Jan 29, 2005, at 12:32 AM, wayne wrote:
>> For
>> some strange reason, the IETF is now considering the SenderID drafts
>> that still use the same records. Go figure.
>
> If by same records, you mean anything TXT then I'm afraid you will
> have to live with that... and any other new uses for TXT that come
> along.

Nope, I see no problem with using TXT RRs. (The same goes for CSV's
use of SRV records or MAPS's use of A records.)

>> So, not rubber-stamping SPF wasn't a problem. Creating a new spec
>> that changed the semantics of SPF records was.
>
> But I suppose you mean the v=spf1 records and agree that you have a
> point.... which is not what I thought you were saying. Sorry for
> jumping to conclusions.

Actually, reviewing things, I shouldn't have jumped so hard on MARID
changing things. When MARID decided to do a design-by-committee of a
new system, rarely a wise idea, it didn't change the SPF-classic
specifications. New systems don't have backward compatibility
problems.

The real problem was that the draft-lentczner-spf-00 I-D did not
eliminate all the incompatibilities that the MARID SenderID I-Ds
created. Hence the creation of draft-schlitt-spf-classic-00.


-wayne

[Douglas Otis]

On Sat, 2005-01-29 at 13:03 -0600, wayne wrote:
> In <8617533D-7210-11D9-9CD0-***@hxr.us> Andrew Newton <***@hxr.us> writes:
>
> > On Jan 29, 2005, at 12:32 AM, wayne wrote:
> >> For
> >> some strange reason, the IETF is now considering the SenderID drafts
> >> that still use the same records. Go figure.
> >
> > If by same records, you mean anything TXT then I'm afraid you will
> > have to live with that... and any other new uses for TXT that come
> > along.
>
> Nope, I see no problem with using TXT RRs. (The same goes for CSV's
> use of SRV records or MAPS's use of A records.)

Those publishing SPF records risk causing their mail being forwarded by
various recipients to be blocked or lost, with the same happening to
mail sent through some list servers. For network security and to
establish accurate reputation, an accountable entity for the mail being
sent can be found by way of the HELO domain. SPF however can not
indicate which identity checking is desired by the sender. Is it the
Sender-ID PRA, the SPF MAILFROM, or now the SPF MAILFROM and HELO in
combination?

The design choice of SPF to not use a specific label to access a
potentially large initial TXT RR record, dedicates this TXT RR for use
by one revision of one application. This is wrong. This gets even
worse with use of this same TXT RR by more than one faction. A wildcard
label can not establish policy, so the reasons for avoiding a prefix
proves to have been a major mistake.

While commending efforts that attempt to establish reasonable limits for
the use of DNS, this effort is like starting with a square block and
chipping away at the corners. Eventually, one arrives at the wheel.

The new SPF processing limits are stated differently:

SPF implementations MUST limit the number of mechanism that do DNS
lookups to at most 10, if this number is exceeded, a PermError MUST
be returned. The mechanisms that count against this limit are
"include", "a", "mx", "ptr", "exists" and the "redirect" modifier.
The "all", "ip4" and "ip6" mechanisms do not require DNS lookups and
therefore do not count against this limit. The "exp" modifier
requires a DNS lookup, but it is not counted as it is used only in
the case of errors.

When evaluating the "mx" and "ptr" mechanisms, or the %{p} macro,
there MUST be a limit of no more than 10 MX or PTR RRs looked up and
checked.

A limit of 10 mechanism[s] that do DNS lookups? Each mechanism is then
allowed 10 lookups? Would that be a total of 101 lookups? Will this
prevent attacks? The average number of queries for each lookup could be
around 3. The following text and process definitions seem to allow 10
lookups each to resolve the MX, the PTR, and the %{p} macro (reverse DNS
comparison to forward DNS). Would 10 %{p} count as 20 DNS lookups?

This compares poorly to CSV which uses a single lookup to obtain the
CSV-CSA record. This record also limits the scope of the identity being
checked to that of the HELO/EHLO. This record also uses a prefix as to
not usurp a RR type or interfere with other protocols.

CSV makes it clear ONLY the HELO is to be checked, which avoids the
conflicts present with Sender-ID and SPF algorithms. Sender-ID and SPF
essentially reduce the integrity of mail delivery. The disruption
caused by these algorithms is without significant benefit, as path
registration authorization does not ensure mail is not being spoofed,
nor which identity is being checked at each stage, nor allow accurately
assessing reputation.

Endorsement of SPF by those that also block bounce messages seems to be
indicating a willingness to trade reliability for expediency. In the
long run, this approach will be expensive. There are efforts outside of
SPF that are focusing on the problem. Mail integrity can be retained,
and accountability accurately determined without putting the email and
DNS system at risk. May I encourage some of the energy placed upon SPF
to focus on the MASS and CLEAR work groups?

http://mipassoc.org/mass/
http://mipassoc.org/clear/

-Doug

[Frank Ellermann]

Douglas Otis wrote:

> Those publishing SPF records risk causing their mail being
> forwarded by various recipients to be blocked or lost,

They don't "risk" this, they _want_ it, it's the idea of RMX
or SPF -all that "mail from A" must come from an IP allowed
by the sender policy of A. If "mail from A" arrives at C
from another IP B, then C would reject it.

If B was a spammer or mail worm the spam or worm is in fact
"lost", again that's the very idea of RMX or SPF FAIL. If B
was a legit MTA it would create a bounce message to A, and
the mail is not lost.

Either A screwed up, maybe he forgot to include B in his
sender policy, or B screwed up, maybe a secondary MX forwarded
mail to a primary MX, and the latter tested SPF without
white-listing the secondary MX. SPF tests work as expected
at the border defined by A, but not later.

You can say that you don't like this one and only feature of
SPF, but you can't say that it's a bug, because it's the idea.

> SPF however can not indicate which identity checking is
> desired by the sender. Is it the Sender-ID PRA, the SPF
> MAILFROM, or now the SPF MAILFROM and HELO in combination?

Not only "now", HELO was "always" mandatory for MAIL FROM:<>
and otherwise optional. It's not a new idea. And it's not
necessarily a good idea, actually it was one of the things I
hoped to solve in the former MARID WG, but unfortunately it
wasn't allowed to discuss 2821 and CSV. [The SPF overhead
for simple HELO checks is IMHO too big]

Sender-ID PRA has nothing to do with classic SPF policies, it
only uses a somewhat similar syntax for Sender-ID policies.

> This gets even worse with use of this same TXT RR by more
> than one faction.

If somebody abuses a protocol for a completely different
purpose, and it breaks, then he owns the pieces. As far as
SPF and I'm concerned the goal is a SPF RR replacing TXT in
the future. One of the reasons why I wanted an RfC.

The other reason was to get an "official" document which can't
be modified only because the authors have a new idea. That's
now solved by the formation of the SPF council.

> The new SPF processing limits are stated differently

They are just clear and better. The handling of all errors is
now much more predictable. Minus DNS timeout errors the same
policy should now have the same effect with all conforming SPF
implementations.

> This compares poorly to CSV which uses a single lookup to
> obtain the CSV-CSA record.

Depends, many policies need only ip4 and all, and that causes
no additonal lookup. a, include, exists, and redirect need one
lookup. Ignoring the discouraged ptr only mx is "difficult",
but OTOH looking up MX is something any MTA knows. Yes, there
is a significant potential overhead, that's the price for its
great flexibility like per-user policies.

s/great/too great/ as you see fit. An SMTP where nothing but
the IP is reliable doesn't work. An SMTP where the IP and the
MAIL FROM and the HELO make sense again is a huge step forward.

Bye. Frank

[Douglas Otis]

On Sun, 2005-01-30 at 09:33 +0100, Frank Ellermann wrote:
> Douglas Otis wrote:
>
> > Those publishing SPF records risk causing their mail being
> > forwarded by various recipients to be blocked or lost,
>
> They don't "risk" this, they _want_ it, it's the idea of RMX
> or SPF -all that "mail from A" must come from an IP allowed
> by the sender policy of A. If "mail from A" arrives at C
> from another IP B, then C would reject it.
>
> If B was a spammer or mail worm the spam or worm is in fact
> "lost", again that's the very idea of RMX or SPF FAIL. If B
> was a legit MTA it would create a bounce message to A, and
> the mail is not lost.
>
> Either A screwed up, maybe he forgot to include B in his
> sender policy, or B screwed up, maybe a secondary MX forwarded
> mail to a primary MX, and the latter tested SPF without
> white-listing the secondary MX. SPF tests work as expected
> at the border defined by A, but not later.
>
> You can say that you don't like this one and only feature of
> SPF, but you can't say that it's a bug, because it's the idea.

Those publishing SPF records want their mail to go missing? There is no
means to know which recipient may be using a forwarded account. There
is no means to prevent a "screw up" with SPF. Forwarding is a common
practice within colleges, societies, and many providers. Validating the
legitimacy of an MTA can take place within a single lookup of a small
CSV-CSA record. A single lookup does not increase the risk to DoS
attacks, and also does not create inadvertent loss of mail, as does
SPF.

> > SPF however can not indicate which identity checking is
> > desired by the sender. Is it the Sender-ID PRA, the SPF
> > MAILFROM, or now the SPF MAILFROM and HELO in combination?
>
> Not only "now", HELO was "always" mandatory for MAIL FROM:<>
> and otherwise optional. It's not a new idea. And it's not
> necessarily a good idea, actually it was one of the things I
> hoped to solve in the former MARID WG, but unfortunately it
> wasn't allowed to discuss 2821 and CSV. [The SPF overhead
> for simple HELO checks is IMHO too big]

The draft just submitted changes both when the HELO is checked and the
outcome of this checking. Previously, someone ignoring the use of SPF
for HELO would likely find their bounce messages treated as from an
"unknown" source. With the change being made, the HELO may now be
checked against a record found at the zone apex. This unexpected change
could cause their bounces to be rejected AND also their mail to be
rejected, as the HELO check is to be done in combination with the
MAILFROM rather than as an alternative for handling bounces. These are
changes publishers could not anticipate.

I agree, the overhead for HELO checking would be unacceptably high with
SPF, in addition to the other problems SPF records create.

> Sender-ID PRA has nothing to do with classic SPF policies, it
> only uses a somewhat similar syntax for Sender-ID policies.

Pobox still claims Sender Policy Framework is the essential part of
Sender-ID. It matters how the publishers are basing their decisions. I
have yet to hear any admonishment by proponents that when publishing
SPF, expect some of your mail to be lost or rejected due to problems
with either SPF et al or Sender-ID algorithms. This is shameful.

> > This gets even worse with use of this same TXT RR by more
> > than one faction.
>
> If somebody abuses a protocol for a completely different
> purpose, and it breaks, then he owns the pieces. As far as
> SPF and I'm concerned the goal is a SPF RR replacing TXT in
> the future. One of the reasons why I wanted an RfC.
>
> The other reason was to get an "official" document which can't
> be modified only because the authors have a new idea. That's
> now solved by the formation of the SPF council.

A mechanism built upon a strategy that can not support revision does not
require a council. Nor would it likely find approval by a standards
organization that cares about compatibility and a need to support
orderly change.

> > The new SPF processing limits are stated differently
>
> They are just clear and better. The handling of all errors is
> now much more predictable. Minus DNS timeout errors the same
> policy should now have the same effect with all conforming SPF
> implementations.

This conclusion is guess work. You could be right, but those that have
published what is now considered too many records may find their mail
being rejected. The SPF timeout imposed still ignores UDP fallback.
Normal exponential fallback ensures congestion avoidance. This
oversight for such an intensive use of DNS is a serious concern.

> > This compares poorly to CSV which uses a single lookup to
> > obtain the CSV-CSA record.
>
> Depends, many policies need only ip4 and all, and that causes
> no additonal lookup. a, include, exists, and redirect need one
> lookup. Ignoring the discouraged ptr only mx is "difficult",
> but OTOH looking up MX is something any MTA knows. Yes, there
> is a significant potential overhead, that's the price for its
> great flexibility like per-user policies.

Will imposing the task of implementing the sender's per-user policies
upon the receiving SMTP server scale? Is it reasonable to expect the
receiving SMTP server to perform a hundred lookups per message? I see
this "great" flexibility stemming from an overly ambitious marketing
effort that continues to ignore serious problems. I agree with you,
there is a need to move toward a name basis to establish meaningful
feedback to administrators of MTA systems. A name basis would help
centralize the effort at getting rid of abuse. This can be safely
achieved via the efforts in MASS and CLEAR, but not with SPF.

-Doug

[g***@metro.cx]

On Sun, Jan 30, 2005 at 02:44:27PM -0800, Douglas Otis wrote:
> Those publishing SPF records want their mail to go missing? There is no
> means to know which recipient may be using a forwarded account. There
> is no means to prevent a "screw up" with SPF. Forwarding is a common
> practice within colleges, societies, and many providers. Validating the
> legitimacy of an MTA can take place within a single lookup of a small
> CSV-CSA record. A single lookup does not increase the risk to DoS
> attacks, and also does not create inadvertent loss of mail, as does
> SPF.

The forwarding problem is known and information is on spf.pobox.com
(which still is the primary source for information about spf). It's not
like it is 'the big secret of spf' that there is a forwarding issue. I
suppose people read this site before publishing, so they must know
about the forwarding problem. However, people still publish. I'm not
going to guess at the motivations of these people, i'm not a pyschologist.
Fact is that despite the forwarding issue, people do publish.

I can only elaborate a bit on my own motivations: if a message is
forwarded and then bounces because of spf, the bounce will contain the
actual email address to which the message was forwarded. I can resend,
perhaps attaching a small note about the fact that the forwarding did
not work. I have never had serious problems with this issue.

Koen

--
K.F.J. Martens, Sonologic, http://www.sonologic.nl/
Networking, embedded systems, unix expertise, artificial intelligence.
Public PGP key: http://www.metro.cx/pubkey-gmc.asc
Wondering about the funny attachment your mail program
can't read? Visit http://www.openpgp.org/

[Ian Rogers]

> K.F.J. Martens wrote:
> On Sun, Jan 30, 2005 at 02:44:27PM -0800, Douglas Otis wrote:
>> Those publishing SPF records want their mail to go missing? There is no
>> means to know which recipient may be using a forwarded account. There
>> is no means to prevent a "screw up" with SPF. Forwarding is a common
>> practice within colleges, societies, and many providers. Validating the
>> legitimacy of an MTA can take place within a single lookup of a small
>> CSV-CSA record. A single lookup does not increase the risk to DoS
>> attacks, and also does not create inadvertent loss of mail, as does
>> SPF.
>
> The forwarding problem is known and information is on spf.pobox.com
> (which still is the primary source for information about spf). It's not
> like it is 'the big secret of spf' that there is a forwarding issue.
> [snip]
>
> I can only elaborate a bit on my own motivations: ...

As admin of several dozen domains I publish SPF records to avoid litigation!

Considering that sending a virus as an offence against the Computer Misuse
Act (in the UK at least) every now and then I get irate messages accusing
my servers of sending out spam or viruses. A quick glance through the
header of the accused message confirms that it was spoofed and then I'm
able to write back saying "no, it wasn't my server that sent it and I've
published the SPF records to prove it. Please encourage your ISP to use
SPF" and give the pobox URL for good luck :-)

I'm happy to take the flack for messages not being delivered due to
"anonymous" forwarding. Note there is a difference between a person
forwarding a message from their MUA (which is effectively generating a new
message with the content of another - which SPF handles fine) and
"anonymous remailers" (where a forwarding 'bot effectively spoofs itself
as the original sender in order to forward the message on verbatim to the
ultimate recipient).

In the Good Old Days the anonymous remailer was a useful tool. But in
*this* day and age of forged-sender spam, Joe jobs and phishing that
functionality must (unfortunately) be considered broken.

You (Douglas and other anti-SPF posters) are quite correct in saying that
SPF won't stamp all spam (but then SPF never claimed it would) and that it
breaks anonymous forwarding (which SPF always acknowledged) but SPF+SRS is
the best, current, first-baby-steps solution available for stamping out
forged sender - which *is* IMHO the essential first step for eliminating
joe-jobs, phishing and a significant subset of spam.

Regards,

Ian.

[David Woodhouse]

On Mon, 2005-01-31 at 08:40 +0100, ***@metro.cx wrote:
> The forwarding problem is known and information is on spf.pobox.com
> (which still is the primary source for information about spf). It's not
> like it is 'the big secret of spf' that there is a forwarding issue. I
> suppose people read this site before publishing, so they must know
> about the forwarding problem. However, people still publish. I'm not
> going to guess at the motivations of these people, i'm not a pyschologist.
> Fact is that despite the forwarding issue, people do publish.

The forwarding problem isn't exactly shouted from the rooftops on the
SPF site, and people rarely actually think things through for
themselves, unfortunately. If you point those _same_ people at something
like http://david.woodhou.se/why-not-spf.html they often seem to change
their mind again and stop publishing records, in my experience.

--
dwmw2

[K.F.J. Martens]

On Mon, Jan 31, 2005 at 02:35:13PM +0000, David Woodhouse wrote:
> On Mon, 2005-01-31 at 08:40 +0100, ***@metro.cx wrote:
> > The forwarding problem is known and information is on spf.pobox.com
> > (which still is the primary source for information about spf). It's not
> > like it is 'the big secret of spf' that there is a forwarding issue. I
> > suppose people read this site before publishing, so they must know
> > about the forwarding problem. However, people still publish. I'm not
> > going to guess at the motivations of these people, i'm not a pyschologist.
> > Fact is that despite the forwarding issue, people do publish.
>
> The forwarding problem isn't exactly shouted from the rooftops on the
> SPF site, and people rarely actually think things through for
> themselves, unfortunately. If you point those _same_ people at something
> like http://david.woodhou.se/why-not-spf.html they often seem to change
> their mind again and stop publishing records, in my experience.

Although not correct in some places, it as a funny web page indeed! In
my experience, people have read that page and either don't care, don't
understand or don't believe what is being presented. And given the
incorectness of some of the statements on that page, who can blame them.
Anyway, this thread is going nowhere fast, so let's close it allright?

Kind regards,

Koen Martens

--
K.F.J. Martens, Sonologic, http://www.sonologic.nl/
Networking, hosting, embedded systems, unix, artificial intelligence.
Public PGP key: http://www.metro.cx/pubkey-gmc.asc
Wondering about the funny attachment your mail program
can't read? Visit http://www.openpgp.org/

[Frank Ellermann]

Douglas Otis wrote:

> There is no means to know which recipient may be using a
> forwarded account.

Yes, it's a "sender policy", not a "recipient's policy", and
not a "mediating source policy" (one of the mail-arch terms).

> Forwarding is a common practice within colleges, societies,
> and many providers.

Sure, there's nothing wrong with forwarding, each and every
mail I send is forwarded at least twice, from me to a smart
host, from the smart host to an MX of the recipient, or to
an MDA in the case of local users.

What happens beyond the MX of the recipient is none of my
business, as you say there's no way for me to know what the
recipient does. My "sender policy" cannot cover forwarding
by the recipient. If he wants this for some reason, he has
to use his own MAIL FROM and his own "sender policy".

> Validating the legitimacy of an MTA can take place within
> a single lookup of a small CSV-CSA record.

Fine, I've no problem if my mail providers do this for their
smart hosts, or if they test it for incoming mails at their
MXs. CSV / SPF / SPF+CSV / ... are all okay, as ar as I'm
concerned.

> With the change being made, the HELO may now be checked
> against a record found at the zone apex.

The "zone cut" isn't a new feature, it was in the last SPF
draft published before MARID. BTW, Tony's blog mentions that
CSV now also solved this problem, but I don't find how, it's
apparently not in the last (now expired) CSV draft.

Here's a summary of my SPF experiences in the last 10 months:

Mar 04: some spammer(s) start to abuse "my" vanity domain, I
get about 1000 erroneous bounces / vacation mails /
challenges / etc. per day.
Apr 04: My ISP publishes a sender policy. Intuition or bug,
neither my ISP nor me saw the "minor" problem, that
this sender policy didn't cover "my" vanity domain.
May 04: Wildcard added, now it works even without "zone cut".
"Zone cut" also added to the SPF draft replacing an
older "match_subdomains" construct.
Sep 04: 180,000 bounces etc. so far in my mailbox (JFTR, I'm
a modem user)
Oct 04: My spammer(s) got the SPF idea, again zero bounces
etc. per day
Jan 05: So far not a single problem with SPF from my POV, and
4*30*1000 = 120,000 avoided bounces etc. since Oct 04.

You can discuss "the better is the enemy of the good" in MASS
and CLEAR as long as you like, I have what I wanted, and SPF
was good enough for me. Let them all disable SPF as soon as
something better exists and works, no problem.

> I agree, the overhead for HELO checking would be unacceptably
> high with SPF

Where that's the case (it depends on the sender policy) the
sender could arrange his HELO domains to avoid it. And if CSV
has a better solution than SPF's "zone cut" for spoofed HELOs,
I'd be interested what it is - I'm always curious.

> when publishing SPF, expect some of your mail to be lost or
> rejected due to problems with either SPF et al or Sender-ID
> algorithms. This is shameful.

No legit mail is "lost" with SPF, that's FUD. While MARID was
a shameful disaster, one of the few results was, that Sender-ID
evaluations of SPF sender policies generally do _not_ work.

> those that have published what is now considered too many
> records may find their mail being rejected.

If an erroneous policy "worked" with old SPF implementations,
and new implementations report the error, then it's time to
fix the old error.

> The SPF timeout imposed still ignores UDP fallback.

The overall SPF timeout in old drafts was replaced by clear
processing limits, which don't depend on a vague accumulated
processing time.

| Records that are too long to fit in a single UDP packet MAY
| be silently ignored.

> Is it reasonable to expect the receiving SMTP server to
> perform a hundred lookups per message?

No, that's why it's impossible, you have constructed a worst
case of 1+10+10*10 = 111 DNS lookups for a sender policy with
10 mx directives, each with 10 MX hosts.

The average case is probably more like 1 to 4 lookups, I've no
data to support this guess. And not necessarily per message,
if you'd use CSV or SPF or an RBL to block an IP or to reject
all mails after your HELO-test resulted in a FAIL, then you
won't test the individual messages in this SMTP session.

> This can be safely achieved via the efforts in MASS and
> CLEAR, but not with SPF.

Please inform "us" when that's ready and deployed. Bye, Frank

[Douglas Otis]

On Mon, 2005-01-31 at 10:56 +0100, Frank Ellermann wrote:
> Douglas Otis wrote:
>
> > Forwarding is a common practice within colleges, societies,
> > and many providers.
>
> Sure, there's nothing wrong with forwarding, each and every
> mail I send is forwarded at least twice, from me to a smart
> host, from the smart host to an MX of the recipient, or to
> an MDA in the case of local users.
>
> What happens beyond the MX of the recipient is none of my
> business, as you say there's no way for me to know what the
> recipient does. My "sender policy" cannot cover forwarding
> by the recipient. If he wants this for some reason, he has
> to use his own MAIL FROM and his own "sender policy".

SPF may say, "Here is a comprehensive address list", when in reality it
can never be comprehensive. Either the sender policy allows open-ended
inclusion of other locations, or such sender policies may cause mail to
go missing or be refused. This is a problem created by the sender, and
not the many unknown others using forwarded accounts.

Use of close-ended SPF records presumes excluding mail on the basis of
MAILFROM not matching a sending MTA has value exceeding a disruption it
may cause. SPF does not prevent spoofing of the domain, as many share
common providers and there is no consensus which identities are checked
against such records (when these records are even examined). The high
number of potential DNS lookups ~101-201 (requirements based upon
Schlitt's draft) ensures SPF can not be used to protect the network.
SPF represents an increased risk with respect to network security and
protection.

Protecting the domain from being spoofed is a feature safely offered by
digital signature techniques. I see a real value in this, but yet no
signature scheme offers network protection. CSV is intended to protect
the network using the same name basis.

> > With the change being made, the HELO may now be checked
> > against a record found at the zone apex.
>
> The "zone cut" isn't a new feature, it was in the last SPF
> draft published before MARID. BTW, Tony's blog mentions that
> CSV now also solved this problem, but I don't find how, it's
> apparently not in the last (now expired) CSV draft.

There is not a major change being made to CSV. The design team is
defining use of the Port field for Domain assertions. Such assertions
could include use of signature algorithms, as example. Use of a
wildcard label is unsuitable for publishing policy, so the prefix used
by the CSV-CSA record is not a deterrent toward establishing domain wide
assertions. This domain assertion is not constrained to zone cuts.
After lengthy consideration, it was determined this too would be
inappropriate. This updated draft should be ready very shortly.

> Here's a summary of my SPF experiences in the last 10 months:
>
> Mar 04: some spammer(s) start to abuse "my" vanity domain, I
> get about 1000 erroneous bounces / vacation mails /
> challenges / etc. per day.
> Apr 04: My ISP publishes a sender policy. Intuition or bug,
> neither my ISP nor me saw the "minor" problem, that
> this sender policy didn't cover "my" vanity domain.
> May 04: Wildcard added, now it works even without "zone cut".
> "Zone cut" also added to the SPF draft replacing an
> older "match_subdomains" construct.
> Sep 04: 180,000 bounces etc. so far in my mailbox (JFTR, I'm
> a modem user)
> Oct 04: My spammer(s) got the SPF idea, again zero bounces
> etc. per day
> Jan 05: So far not a single problem with SPF from my POV, and
> 4*30*1000 = 120,000 avoided bounces etc. since Oct 04.

There needs to be a means to discourage spoofing of domains and the
abuse of bounce traffic. The MASS, CLEAR (BATV & CSV) efforts are aimed
at achieving those goals without onerous path registration discovery.
While a small domain may accommodate such related problems, security
considerations dismiss value in path registration and thereby the need
to also disrupt forwarding, various list servers, and user's access to
mail. CSV looks to the administrators of the domain to control access
to their outbound servers, and uses both assertions and reputations to
protect the networks. CSV does not require providers to make specific
DNS record changes to accommodate their customer's use of independent
mailbox domains.

> > I agree, the overhead for HELO checking would be unacceptably
> > high with SPF
>
> Where that's the case (it depends on the sender policy) the
> sender could arrange his HELO domains to avoid it. And if CSV
> has a better solution than SPF's "zone cut" for spoofed HELOs,
> I'd be interested what it is - I'm always curious.

Good to hear.

> > when publishing SPF, expect some of your mail to be lost or
> > rejected due to problems with either SPF et al or Sender-ID
> > algorithms. This is shameful.
>
> No legit mail is "lost" with SPF, that's FUD. While MARID was
> a shameful disaster, one of the few results was, that Sender-ID
> evaluations of SPF sender policies generally do _not_ work.

A well understood problem is not FUD!

> > those that have published what is now considered too many
> > records may find their mail being rejected.
>
> If an erroneous policy "worked" with old SPF implementations,
> and new implementations report the error, then it's time to
> fix the old error.

With SPF, there is no means to prevent disruption and provide an orderly
introduction of newer revisions. A script based language is inherently
complex and rarely stable. : (

> > The SPF timeout imposed still ignores UDP fallback.
>
> The overall SPF timeout in old drafts was replaced by clear
> processing limits, which don't depend on a vague accumulated
> processing time.

>From the Schlitt's draft:
MTAs or other processors MAY also impose a limit on the maximum
amount of elapsed time to evaluate check_host(). Such a limit SHOULD
allow at least 20 seconds. If such a limit is exceeded, the result
of authentication SHOULD be "TempError".

> > Is it reasonable to expect the receiving SMTP server to
> > perform a hundred lookups per message?
>
> No, that's why it's impossible, you have constructed a worst
> case of 1+10+10*10 = 111 DNS lookups for a sender policy with
> 10 mx directives, each with 10 MX hosts.

The concern is not what most use, the concern is what limits must be
permitted. The draft requires these 101 to 201 lookup limits depending
upon the mechanisms.

> The average case is probably more like 1 to 4 lookups, I've no
> data to support this guess. And not necessarily per message,
> if you'd use CSV or SPF or an RBL to block an IP or to reject
> all mails after your HELO-test resulted in a FAIL, then you
> won't test the individual messages in this SMTP session.

By checking per HELO, the rate CSV does a lookup is some multiple less
than SPF, and constrained to a single lookup and not hundreds of
lookups. This aspect is critical.

> > This can be safely achieved via the efforts in MASS and
> > CLEAR, but not with SPF.
>
> Please inform "us" when that's ready and deployed. Bye, Frank

Gladly.

-Doug

[Hector Santos]

Please Doug.

You can't guarantee that an immediate router will be CSV compliant. So you
have the same heterogeneous/mixed policy issues as SPF and all the rest of
the proposals.

In addition, you have a MUCH higher overhead than most as you based on
state point #1 - HELO/EHLO.

Now why would I want to do a HELO CSV check without determining:

- Check to see if the sender is valid,
- Final vs Route

You will say;

"Our new MAPS CSV/DNA service will take of this. We will vouch
for the transaction."

But what if the SMTP operator does not want to use your new MAPS CSV/DNA
service? What if you go out of business? What if you get enough support
headaches from thousands of smaller systems that you decide to raise the
price to filter out these bothersome clientele?

Please, again.

Give me something technically SOUND before even have to bother with the
baloney that will come about with DNA like concepts.

Sincerely,

Hector Santos, CTO
Santronics Software, Inc.
http://www.santronics.com
305-431-2846 Cell
305-248-3204 Office




----- Original Message -----
From: "Douglas Otis" <***@mail-abuse.org>
To: "Frank Ellermann" <***@xyzzy.claranet.de>
Cc: <ietf-***@imc.org>
Sent: Monday, January 31, 2005 4:05 PM
Subject: Re: So here it is one year later...


>
> On Mon, 2005-01-31 at 10:56 +0100, Frank Ellermann wrote:
> > Douglas Otis wrote:
> >
> > > Forwarding is a common practice within colleges, societies,
> > > and many providers.
> >
> > Sure, there's nothing wrong with forwarding, each and every
> > mail I send is forwarded at least twice, from me to a smart
> > host, from the smart host to an MX of the recipient, or to
> > an MDA in the case of local users.
> >
> > What happens beyond the MX of the recipient is none of my
> > business, as you say there's no way for me to know what the
> > recipient does. My "sender policy" cannot cover forwarding
> > by the recipient. If he wants this for some reason, he has
> > to use his own MAIL FROM and his own "sender policy".
>
> SPF may say, "Here is a comprehensive address list", when in reality it
> can never be comprehensive. Either the sender policy allows open-ended
> inclusion of other locations, or such sender policies may cause mail to
> go missing or be refused. This is a problem created by the sender, and
> not the many unknown others using forwarded accounts.
>
> Use of close-ended SPF records presumes excluding mail on the basis of
> MAILFROM not matching a sending MTA has value exceeding a disruption it
> may cause. SPF does not prevent spoofing of the domain, as many share
> common providers and there is no consensus which identities are checked
> against such records (when these records are even examined). The high
> number of potential DNS lookups ~101-201 (requirements based upon
> Schlitt's draft) ensures SPF can not be used to protect the network.
> SPF represents an increased risk with respect to network security and
> protection.
>
> Protecting the domain from being spoofed is a feature safely offered by
> digital signature techniques. I see a real value in this, but yet no
> signature scheme offers network protection. CSV is intended to protect
> the network using the same name basis.
>
> > > With the change being made, the HELO may now be checked
> > > against a record found at the zone apex.
> >
> > The "zone cut" isn't a new feature, it was in the last SPF
> > draft published before MARID. BTW, Tony's blog mentions that
> > CSV now also solved this problem, but I don't find how, it's
> > apparently not in the last (now expired) CSV draft.
>
> There is not a major change being made to CSV. The design team is
> defining use of the Port field for Domain assertions. Such assertions
> could include use of signature algorithms, as example. Use of a
> wildcard label is unsuitable for publishing policy, so the prefix used
> by the CSV-CSA record is not a deterrent toward establishing domain wide
> assertions. This domain assertion is not constrained to zone cuts.
> After lengthy consideration, it was determined this too would be
> inappropriate. This updated draft should be ready very shortly.
>
> > Here's a summary of my SPF experiences in the last 10 months:
> >
> > Mar 04: some spammer(s) start to abuse "my" vanity domain, I
> > get about 1000 erroneous bounces / vacation mails /
> > challenges / etc. per day.
> > Apr 04: My ISP publishes a sender policy. Intuition or bug,
> > neither my ISP nor me saw the "minor" problem, that
> > this sender policy didn't cover "my" vanity domain.
> > May 04: Wildcard added, now it works even without "zone cut".
> > "Zone cut" also added to the SPF draft replacing an
> > older "match_subdomains" construct.
> > Sep 04: 180,000 bounces etc. so far in my mailbox (JFTR, I'm
> > a modem user)
> > Oct 04: My spammer(s) got the SPF idea, again zero bounces
> > etc. per day
> > Jan 05: So far not a single problem with SPF from my POV, and
> > 4*30*1000 = 120,000 avoided bounces etc. since Oct 04.
>
> There needs to be a means to discourage spoofing of domains and the
> abuse of bounce traffic. The MASS, CLEAR (BATV & CSV) efforts are aimed
> at achieving those goals without onerous path registration discovery.
> While a small domain may accommodate such related problems, security
> considerations dismiss value in path registration and thereby the need
> to also disrupt forwarding, various list servers, and user's access to
> mail. CSV looks to the administrators of the domain to control access
> to their outbound servers, and uses both assertions and reputations to
> protect the networks. CSV does not require providers to make specific
> DNS record changes to accommodate their customer's use of independent
> mailbox domains.
>
> > > I agree, the overhead for HELO checking would be unacceptably
> > > high with SPF
> >
> > Where that's the case (it depends on the sender policy) the
> > sender could arrange his HELO domains to avoid it. And if CSV
> > has a better solution than SPF's "zone cut" for spoofed HELOs,
> > I'd be interested what it is - I'm always curious.
>
> Good to hear.
>
> > > when publishing SPF, expect some of your mail to be lost or
> > > rejected due to problems with either SPF et al or Sender-ID
> > > algorithms. This is shameful.
> >
> > No legit mail is "lost" with SPF, that's FUD. While MARID was
> > a shameful disaster, one of the few results was, that Sender-ID
> > evaluations of SPF sender policies generally do _not_ work.
>
> A well understood problem is not FUD!
>
> > > those that have published what is now considered too many
> > > records may find their mail being rejected.
> >
> > If an erroneous policy "worked" with old SPF implementations,
> > and new implementations report the error, then it's time to
> > fix the old error.
>
> With SPF, there is no means to prevent disruption and provide an orderly
> introduction of newer revisions. A script based language is inherently
> complex and rarely stable. : (
>
> > > The SPF timeout imposed still ignores UDP fallback.
> >
> > The overall SPF timeout in old drafts was replaced by clear
> > processing limits, which don't depend on a vague accumulated
> > processing time.
>
> >From the Schlitt's draft:
> MTAs or other processors MAY also impose a limit on the maximum
> amount of elapsed time to evaluate check_host(). Such a limit SHOULD
> allow at least 20 seconds. If such a limit is exceeded, the result
> of authentication SHOULD be "TempError".
>
> > > Is it reasonable to expect the receiving SMTP server to
> > > perform a hundred lookups per message?
> >
> > No, that's why it's impossible, you have constructed a worst
> > case of 1+10+10*10 = 111 DNS lookups for a sender policy with
> > 10 mx directives, each with 10 MX hosts.
>
> The concern is not what most use, the concern is what limits must be
> permitted. The draft requires these 101 to 201 lookup limits depending
> upon the mechanisms.
>
> > The average case is probably more like 1 to 4 lookups, I've no
> > data to support this guess. And not necessarily per message,
> > if you'd use CSV or SPF or an RBL to block an IP or to reject
> > all mails after your HELO-test resulted in a FAIL, then you
> > won't test the individual messages in this SMTP session.
>
> By checking per HELO, the rate CSV does a lookup is some multiple less
> than SPF, and constrained to a single lookup and not hundreds of
> lookups. This aspect is critical.
>
> > > This can be safely achieved via the efforts in MASS and
> > > CLEAR, but not with SPF.
> >
> > Please inform "us" when that's ready and deployed. Bye, Frank
>
> Gladly.
>
> -Doug
>
>

[Douglas Otis]

On Mon, 2005-01-31 at 16:50 -0500, Hector Santos wrote:
> Please Doug.
>
> You can't guarantee that an immediate router will be CSV compliant. So you
> have the same heterogeneous/mixed policy issues as SPF and all the rest of
> the proposals.

If the sending SMTP client publishes a CSV-CSA record, then this client
is both authenticated and authorized by the HELO domain. This permits
assessment of reputation which may supersede evaluation of the IP
address. It also permits direct and meaningful feedback to the
accountable administrator, as a means to rapidly respond to abuse.

Authorizations offered by SPF path registration is insufficient for
reputation assessments, as there is no assurance which administrator is
directly accountable for abusive messages. SPF does not authenticate
the sending domain, it only confirms authorization. Where many domains
have authorized an MTA with SPF, resolving abuse may become an expensive
process.

CSV allows such assessments to be made safely on an opportunistic basis
while not requiring heterogeneous adoption. CSV is also incorporating a
means to assert domain wide use of CSV as a means to discourage HELO
spoofing. Unlike SPF, CSV does not disrupt the normal use of mail nor
depend upon any tests or changes in MTA behavior.

> In addition, you have a MUCH higher overhead than most as you based on
> state point #1 - HELO/EHLO.

Authenticating the HELO is done within a single lookup with CSV. How
can this be a higher overhead?

SPF provides authorization for the MTA referenced by a sending mailbox
domain (without consensus as to which mailbox domain). While SPF could
also be extended to examine the HELO domain, those that have previously
published SPF records may not have ensured HELO domains resolve to an
SPF record. SPF evaluation of HELO was previously applied during the
sending of bounce messages, where the outcome would have been "unknown"
rather than "fail" without specific records in place. It is also likely
that SPF HELO records may require resolving the same large address space
as that needed for mailbox domains and is a DoS concern. : (

Changing SPF to apply a record at the zone cut against the HELO, in
addition to further increasing overhead, may now put some domains in
jeopardy due to a lack of proper record revisioning. Using the same
check_host routine and records, as used to examine mailbox domains, may
also permit unexpected exploits. Although the number of SPF records
could be few on average, the need to register potentially complex paths
always requires an excessive limit for the number of lookups.

> Now why would I want to do a HELO CSV check without determining:
>
> - Check to see if the sender is valid,
> - Final vs Route

MASS is addressing the need to safely authenticate the administrator of
originating sender domain. CSV addresses the need to protect the
network from abuse, and therefore only considers authenticating the
administrator of the immediate sending domain. CSV does not require
customers of various mail providers to have their various provider's
domains entered into their DNS records.

CSV will not inappropriately assess the customers of various providers
for a lapse in access control of some provider's server. The
administrators of the servers are accountable for security, not
customers.

> You will say;
>
> "Our new MAPS CSV/DNA service will take of this. We will vouch
> for the transaction."

Accurately authenticating the accountable domain's administrator (who is
permitting mail access), allows assessing reputation. There is value
from this information alone, as it also means the sending MTA has been
specifically authorized to send mail. This helps with detection of
zombies without the use of a vouching service. Securing the networks
will always require a reputation service, whether IP address based or
extended to using names. There are advantages using names with respect
to ensuring legitimate domains eliminate abusive accounts with the least
expenditure.

> But what if the SMTP operator does not want to use your new MAPS CSV/DNA
> service? What if you go out of business? What if you get enough support
> headaches from thousands of smaller systems that you decide to raise the
> price to filter out these bothersome clientele?

This seems to be putting the cart before the horse.

> Please, again.
>
> Give me something technically SOUND before even have to bother with the
> baloney that will come about with DNA like concepts.

The CLEAR design team is working diligently at ensuring CSV is sound.
There are models where reputation is provided as a service to the SMTP
servers. While CSV has made accommodations for a vouching model, it is
unknown which model will prevail. While bulk mail providers may favor
the vouching approach, a more significant number may rely upon a direct
reputation service model.

I should also note that BATV does not require any outside service to
address abusive bounce messages. Although SES is similar, it introduces
some security concerns by way of its syntax.

-Doug

[David Woodhouse]

On Mon, 2005-01-31 at 18:23 -0800, Douglas Otis wrote:
> I should also note that BATV does not require any outside service to
> address abusive bounce messages. Although SES is similar, it
> introduces some security concerns by way of its syntax.

The syntax of each can be treated as if it's entirely opaque, and SES
without any form of validation at the _receiving_ side is then
indistinguishable from BATV in all but cosmetics, surely?

What are the security concerns of which you speak?

--
dwmw2

[Douglas Otis]

On Mon, 2005-01-31 at 23:57, David Woodhouse wrote:
> On Mon, 2005-01-31 at 18:23 -0800, Douglas Otis wrote:
> > I should also note that BATV does not require any outside service to
> > address abusive bounce messages. Although SES is similar, it
> > introduces some security concerns by way of its syntax.
>
> The syntax of each can be treated as if it's entirely opaque, and SES
> without any form of validation at the _receiving_ side is then
> indistinguishable from BATV in all but cosmetics, surely?
>
> What are the security concerns of which you speak?

This was raised on the CLEAR mailing list by William Leibzon and
responded to by Tony Finch. Follow the thread.

http://mipassoc.org/pipermail/ietf-clear/2004-November/000133.html

To discuss this issue, it would be best done on the CLEAR reflector:

http://mipassoc.org/mailman/listinfo/ietf-clear

-Doug

[wayne]

In <***@bash.adsl-64-142-13-68> Douglas Otis <***@mail-abuse.org> writes:

> This was raised on the CLEAR mailing list [...]
>
> To discuss this issue, it would be best done on the CLEAR reflector:

Wait, you don't want CSV issues discussed on the MARID list even
though CSV was adopted by MARID? But you keep bring up SPF
discussions here instead of on SPF-discuss even though MARID never
adopted SPF?


*sigh*


-wayne

[David Woodhouse]

On Tue, 2005-02-01 at 00:20 -0800, Douglas Otis wrote:
> > What are the security concerns of which you speak?
>
> This was raised on the CLEAR mailing list by William Leibzon and
> responded to by Tony Finch. Follow the thread.
>
> http://mipassoc.org/pipermail/ietf-clear/2004-November/000133.html

Ah yes; I had seen that but discarded it as irrelevant in the context of
BATV -- BATV alone doesn't really offer the _recipient_ much benefit
along those lines anyway. Referring to the examples in message
000146.html, it's obvious that with BATV the attacker could just send a
mail from 'batv=INVALIDBATVTAG/***@domain' anyway, and the recipient
would have no means to validate that.

In the context of SES it's slightly more relevant, but really not by
far. It involves a naïve user actually looking in the message headers
for the reverse-path and _incorrectly_ identifying it as an SES
localpart, thus having faith in it. (If the MUA were to be modified to
show SES/BATV reverse-paths more prominently, of course it wouldn't have
the false positives that wetware can have).

In general, users don't do that. Either they're capable of telling the
difference, or they're not going to be looking in the headers for it
anyway. The middle ground is relatively rare.

It also involves the domain owner/admin actually permitting that kind of
reverse-path to be used in outgoing mail, and authorising it as valid if
it's sent from elsewhere.

--
dwmw2

[Douglas Otis]

On Tue, 2005-02-01 at 03:49 -0600, wayne wrote:
> In <***@bash.adsl-64-142-13-68> Douglas Otis <***@mail-abuse.org> writes:
>
> > This was raised on the CLEAR mailing list [...]
> >
> > To discuss this issue, it would be best done on the CLEAR reflector:
>
> Wait, you don't want CSV issues discussed on the MARID list even
> though CSV was adopted by MARID? But you keep bring up SPF
> discussions here instead of on SPF-discuss even though MARID never
> adopted SPF?

Do you really want me to interact with those on the SPF-discuss mailing
list?

I suggested that the best forum to discuss this BATV/SES syntax was on
the CLEAR list server. I see David made a minor error in his statement,
as example, but I am not the person most conversant on this subject. I
encouraged William to propose his concepts, but Tony raised some
legitimate concerns. It would be beneficial to find consensus on this
syntax.

-Doug

[Hector Santos]

----- Original Message -----
From: "Douglas Otis" <***@mail-abuse.org>
To: "Hector Santos" <***@santronics.com>
Cc: <ietf-***@imc.org>
Sent: Monday, January 31, 2005 9:23 PM
Subject: Re: So here it is one year later...


> On Mon, 2005-01-31 at 16:50 -0500, Hector Santos wrote:
> > Please Doug.
> >
> > You can't guarantee that an immediate router will be CSV compliant. So
you
> > have the same heterogeneous/mixed policy issues as SPF and all the rest
of
> > the proposals.
>
> If the sending SMTP client publishes a CSV-CSA record, then this client
> is both authenticated and authorized by the HELO domain.

The "chain of trust" issues with SPF applies to CSV as well.

What about the downlink? Once the message is received, it is your
responsibility to deliver it. My sender went directly to you, the MDA. If
you decide to route it instead, then you have to maintain the chain of
trust. You can not guarantee this transaction when you become the sender.

> > In addition, you have a MUCH higher overhead than most as you based on
> > state point #1 - HELO/EHLO.
>
> Authenticating the HELO is done within a single lookup with CSV. How
> can this be a higher overhead?

I don't think you really understand the implementation issues of your own
proposal. Maybe you do and are just turning a blind eye.

> > Now why would I want to do a HELO CSV check without determining:
> >
> > - Check to see if the sender is valid,
> > - Final vs Route
>
> MASS is addressing the need to safely authenticate the administrator of
> originating sender domain. ....

You didn't answer the middle-ware, chain of trust issue, and you didn't
address the above.

Delay verification is a REQUIREMENT, not a BCP for all DNS based
authorization ideas.. I have statistic proof with over 1 year of product
operations in thousands of installed sites:
http://www.winserver.com/spamstats.

The functional model of CSV is:

EPV = CSV(helo, ip)

which fails to take into the account the validity of the other process
environment parameters, MailFrom, RcptTo. That my friend, is a overhead
issue.

> > But what if the SMTP operator does not want to use your new MAPS CSV/DNA
> > service? What if you go out of business? What if you get enough
support
> > headaches from thousands of smaller systems that you decide to raise the
> > price to filter out these bothersome clientele?
>
> This seems to be putting the cart before the horse.

No. It is called business and practical product experience. You are
proposing an idea infested (wrongly placed) with capitalistic commercial
concepts, which I don't mind saying, you will be among the first to benefit
from as the first DNA commercial service. PR problem #1. Therefore, if we
were support and implement CSV, we would have to include a disclaimer for
any Anti-Spam CSV support statement; "Batteries Not Included. DNA Service
Subscription Required." We are not going to add what is suppose to be a
"technical" solution with subscription concepts behind it. Period. PR
problem #2. My Advice? Dave should take Doug out of the CSV promotion
picture. You don't do CSV any justice.

Dont' get me wrong. I'm for making a buck as much as the next guy. But the
"Email Problem" needs an open standard technical solution free of 3rd party
dependencies. It is a duel tier process. Transport security must be done
using SMTP first. Main Content Security is an separate issue, however,
there can be a tie in of the two, but not dependent on it.

The point is, if you send me mail, I must be able to trust the transaction
before I accept your payload. Authenticating the payload is an
ADMINISTRATIVE concept.

Using a CSV authorization domain is not sufficient WITHOUT looking at the
2821.Mail From as well as 2821.RCPTTO. Therefore, as it has been
emperically proven, using HELO by itself would be wasteful overhead without
taken all process environment parameters into consideration.

I have PROOF of this issue. Doug. All you are doing is making assertions of
the opposite and you have no proof to show I am wrong.

Sincerely,

Hector Santos, CTO
Santronics Software, Inc.
http://www.santronics.com
305-431-2846 Cell
305-248-3204 Office

[Douglas Otis]

On Tue, 2005-02-01 at 08:23 -0500, Hector Santos wrote:
> On Mon, 2005-1-31 at 9:23 -0500, Douglas Otis wrote:
> > On Mon, 2005-01-31 at 16:50 -0500, Hector Santos wrote:
> > >
> > > Please Doug.
> > >
> > > You can't guarantee that an immediate router will be CSV
> > > compliant. So you have the same heterogeneous/mixed policy issues
> > > as SPF and all the rest of the proposals.
> >
> > If the sending SMTP client publishes a CSV-CSA record, then this client
> > is both authenticated and authorized by the HELO domain.
>
> The "chain of trust" issues with SPF applies to CSV as well.

CSV does not make assurances for MAILFROM or FROM mailbox-domains. The
HELO identifies the administrator of the specific MTA. It is the
administrator trusted and not millions of users that may be using the
server. When there is a problem, ONLY the administrator is able to
rectify the situation. In essence, mail is not secure and the source of
a message is not assured beyond the immediate server. It is wrong to
assume a "chain of trust" is maintained, as this could be injurious to
those assumed to be the source of the message.

MASS avoids the fallacy of a "chain of trust" by using a digital
signature. SPF assumes identities are checked at each stage against a
lengthy resolution of path registrations. There is also no consensus
which mailbox-domain is checked, if any, even assuming these checks were
practical and safe. Schlitt's SPF draft says to use some identity as an
alternative without indicating whether this is the PRA identity or
something else. : (

> What about the downlink? Once the message is received, it is your
> responsibility to deliver it. My sender went directly to you, the MDA. If
> you decide to route it instead, then you have to maintain the chain of
> trust. You can not guarantee this transaction when you become the sender.

Most MTAs only accept mail from anonymous sources being sent to their
immediate domains. Most MTAs also decide whether to accept from an
anonymous source based upon reputations of the IP address. Without such
checks, few could afford the bandwidth needed to permit legitimate mail.
You are describing a practice that could put the reputation of the MTA
IP address at risk. A reputation service should never consider an
unauthenticated identity within the message as being culpable.

> > > In addition, you have a MUCH higher overhead than most as you
> > > based on state point #1 - HELO/EHLO.
> >
> > Authenticating the HELO is done within a single lookup with CSV. How
> > can this be a higher overhead?
>
> I don't think you really understand the implementation issues of your own
> proposal. Maybe you do and are just turning a blind eye.

Information provided by SPF only indicates who may have sent the
message, and not who actually sent it. Resolving issues of abuse by way
of SPF is potentially injurious to mailbox-domain owners. CSV ensures
accountability remains with those administering servers and not a
possibly hapless customer. Essentially SPF is a weapon that does not
always shoot straight and therefore should not be used as an abatement
tool. SPF does not eliminate the need for administrators to control
access to their servers nor remove the need to watch for abusive
accounts. In addition, SPF is a paper tiger easily destroyed. MASS
however is attempting to safely provide a real means to control use of
the sender mailbox-domain.

MASS provides this domain control to protect users, whereas CSV and BATV
protect networks. SPF is unable to offer any real protection for either
users and definitely not networks.

> > > Now why would I want to do a HELO CSV check without determining:
> > >
> > > - Check to see if the sender is valid,
> > > - Final vs Route
> >
> > MASS is addressing the need to safely authenticate the administrator of
> > originating sender domain. ....
>
> You didn't answer the middle-ware, chain of trust issue, and you didn't
> address the above.

The use of a digital signature avoids false assumptions of there being a
"chain of trust" as the mail channel is not secure.

> Delay verification is a REQUIREMENT, not a BCP for all DNS based
> authorization ideas.. I have statistic proof with over 1 year of product
> operations in thousands of installed sites:
> http://www.winserver.com/spamstats.
>
> The functional model of CSV is:
>
> EPV = CSV(helo, ip)
>
> which fails to take into the account the validity of the other process
> environment parameters, MailFrom, RcptTo. That my friend, is a overhead
> issue.

Inordinate overhead must not be placed upon the receiving MTA. CSV
delegates source control overhead onto the administrator accountable for
the server. There are many tools to assist in their effort. Monitoring
the SMTP error logs, unusual traffic patterns, as well as abuse
reporting. CSV helps ensure this reporting is directed to the
accountable administrator. SPF alone will not abate abuse, while it may
produce victims as it is applied in conjunction with reputations.
Spammers will simply adapt and take advantage of these false assumptions
being made by SPF.

> > > But what if the SMTP operator does not want to use your new MAPS
> > > CSV/DNA service? What if you go out of business? What if you
> > > get enough support headaches from thousands of smaller systems
> > > that you decide to raise the price to filter out these bothersome
> > > clientele?
> >
> > This seems to be putting the cart before the horse.
>
> No. It is called business and practical product experience. You are
> proposing an idea infested (wrongly placed) with capitalistic commercial
> concepts, which I don't mind saying, you will be among the first to benefit
> from as the first DNA commercial service.

CSV is offering an authenticated name for the sending MTA that helps in
many ways with respect to dealing with the email abuse problem. The
transition to the use of names will likely be an immediate benefit to
those already providing vouching services.

> PR problem #1. Therefore, if we were support and implement CSV, we
> would have to include a disclaimer for any Anti-Spam CSV support
> statement; "Batteries Not Included. DNA Service Subscription
> Required." We are not going to add what is suppose to be a
> "technical" solution with subscription concepts behind it. Period.

An immediate benefit of CSV comes from being able to associate an
authenticated name with the immediate sending MTA. MASS offers the
feature that allows control over the sender mailbox-domain regardless of
the immediate sending MTA. The blunt instrument provided by CSV
protects the network from grossly negligent administrators. MASS should
be able to protect users from those wishing to spoof sender
mailbox-domains. MASS/CSV can work together in a unified name basis for
guarding mail. SPF will not remove the need for some form of
reputation/vouching service, nor remove the need for some form of
network protection.

> Dont' get me wrong. I'm for making a buck as much as the next guy. But
> the "Email Problem" needs an open standard technical solution free of
> 3rd party dependencies. It is a duel tier process. Transport
> security must be done using SMTP first. Main Content Security is an
> separate issue, however, there can be a tie in of the two, but not
> dependent on it.

There were several companies touting reputations services at the FTC
Email Authentication Summit. I was not promoting such a service. I was
insisting upon adoption of a safe and accurate means to assess abuse. I
remain convinced MASS and CLEAR are considering viable solutions.

> The point is, if you send me mail, I must be able to trust the transaction
> before I accept your payload. Authenticating the payload is an
> ADMINISTRATIVE concept.

Authenticating the payload is the subject of an ongoing effort in MASS.
I want to see MASS succeed in providing an effective solution that can
protect the sender mailbox-domain.

> Using a CSV authorization domain is not sufficient WITHOUT looking at the
> 2821.Mail From as well as 2821.RCPTTO. Therefore, as it has been
> emperically proven, using HELO by itself would be wasteful overhead without
> taken all process environment parameters into consideration.

Holding the sending domain accountable offers substantial relief without
a need to sort through millions of users employing various
mailbox-domains within thousands of various providers. There is a fair
amount of experience that supports this view.

> I have PROOF of this issue. Doug. All you are doing is making assertions of
> the opposite and you have no proof to show I am wrong.

This is not the proper venue for product related data. Most of the
benefits claimed by SPF are related to changes in behavior, and not
cessation of abuse. As an immediate source for relief, BATV would be
more effective in preventing the unwanted bounce traffic as this can be
done unilaterally.

-Doug

[Hector Santos]

Is this a CANNED reply message? It sure reads like it.

Look, CSV does not pass the test for consideration. That might explain why
no commercial vendor has implemented it.

And we were not talking about MASS. We are talking about the fallacy of CSV
being immune to heterogeneous/mix policy topologies and lacks overhead
issues. You are wrong in both cases.

I think you are saying MASS solves these problems for CSV.

Well, that's a PAYLOAD solution so it doesn't address the heterogeneous/mix
policies and overhead issues at the SMTP level. So I believe you are
incorrect here as well.

Anyway, I don't wish to get another canned reply, so I think I'll stop
here.:-)

Thanks for the chat

Sincerely,

Hector Santos, CTO
Santronics Software, Inc.
http://www.santronics.com
305-431-2846 Cell
305-248-3204 Office



----- Original Message -----
From: "Douglas Otis" <***@mail-abuse.org>
To: "Hector Santos" <***@santronics.com>
Cc: <ietf-***@imc.org>
Sent: Tuesday, February 01, 2005 3:46 PM
Subject: Re: So here it is one year later...


>
> On Tue, 2005-02-01 at 08:23 -0500, Hector Santos wrote:
> > On Mon, 2005-1-31 at 9:23 -0500, Douglas Otis wrote:
> > > On Mon, 2005-01-31 at 16:50 -0500, Hector Santos wrote:
> > > >
> > > > Please Doug.
> > > >
> > > > You can't guarantee that an immediate router will be CSV
> > > > compliant. So you have the same heterogeneous/mixed policy issues
> > > > as SPF and all the rest of the proposals.
> > >
> > > If the sending SMTP client publishes a CSV-CSA record, then this
client
> > > is both authenticated and authorized by the HELO domain.
> >
> > The "chain of trust" issues with SPF applies to CSV as well.
>
> CSV does not make assurances for MAILFROM or FROM mailbox-domains. The
> HELO identifies the administrator of the specific MTA. It is the
> administrator trusted and not millions of users that may be using the
> server. When there is a problem, ONLY the administrator is able to
> rectify the situation. In essence, mail is not secure and the source of
> a message is not assured beyond the immediate server. It is wrong to
> assume a "chain of trust" is maintained, as this could be injurious to
> those assumed to be the source of the message.
>
> MASS avoids the fallacy of a "chain of trust" by using a digital
> signature. SPF assumes identities are checked at each stage against a
> lengthy resolution of path registrations. There is also no consensus
> which mailbox-domain is checked, if any, even assuming these checks were
> practical and safe. Schlitt's SPF draft says to use some identity as an
> alternative without indicating whether this is the PRA identity or
> something else. : (
>
> > What about the downlink? Once the message is received, it is your
> > responsibility to deliver it. My sender went directly to you, the MDA.
If
> > you decide to route it instead, then you have to maintain the chain of
> > trust. You can not guarantee this transaction when you become the
sender.
>
> Most MTAs only accept mail from anonymous sources being sent to their
> immediate domains. Most MTAs also decide whether to accept from an
> anonymous source based upon reputations of the IP address. Without such
> checks, few could afford the bandwidth needed to permit legitimate mail.
> You are describing a practice that could put the reputation of the MTA
> IP address at risk. A reputation service should never consider an
> unauthenticated identity within the message as being culpable.
>
> > > > In addition, you have a MUCH higher overhead than most as you
> > > > based on state point #1 - HELO/EHLO.
> > >
> > > Authenticating the HELO is done within a single lookup with CSV. How
> > > can this be a higher overhead?
> >
> > I don't think you really understand the implementation issues of your
own
> > proposal. Maybe you do and are just turning a blind eye.
>
> Information provided by SPF only indicates who may have sent the
> message, and not who actually sent it. Resolving issues of abuse by way
> of SPF is potentially injurious to mailbox-domain owners. CSV ensures
> accountability remains with those administering servers and not a
> possibly hapless customer. Essentially SPF is a weapon that does not
> always shoot straight and therefore should not be used as an abatement
> tool. SPF does not eliminate the need for administrators to control
> access to their servers nor remove the need to watch for abusive
> accounts. In addition, SPF is a paper tiger easily destroyed. MASS
> however is attempting to safely provide a real means to control use of
> the sender mailbox-domain.
>
> MASS provides this domain control to protect users, whereas CSV and BATV
> protect networks. SPF is unable to offer any real protection for either
> users and definitely not networks.
>
> > > > Now why would I want to do a HELO CSV check without determining:
> > > >
> > > > - Check to see if the sender is valid,
> > > > - Final vs Route
> > >
> > > MASS is addressing the need to safely authenticate the administrator
of
> > > originating sender domain. ....
> >
> > You didn't answer the middle-ware, chain of trust issue, and you didn't
> > address the above.
>
> The use of a digital signature avoids false assumptions of there being a
> "chain of trust" as the mail channel is not secure.
>
> > Delay verification is a REQUIREMENT, not a BCP for all DNS based
> > authorization ideas.. I have statistic proof with over 1 year of
product
> > operations in thousands of installed sites:
> > http://www.winserver.com/spamstats.
> >
> > The functional model of CSV is:
> >
> > EPV = CSV(helo, ip)
> >
> > which fails to take into the account the validity of the other process
> > environment parameters, MailFrom, RcptTo. That my friend, is a
overhead
> > issue.
>
> Inordinate overhead must not be placed upon the receiving MTA. CSV
> delegates source control overhead onto the administrator accountable for
> the server. There are many tools to assist in their effort. Monitoring
> the SMTP error logs, unusual traffic patterns, as well as abuse
> reporting. CSV helps ensure this reporting is directed to the
> accountable administrator. SPF alone will not abate abuse, while it may
> produce victims as it is applied in conjunction with reputations.
> Spammers will simply adapt and take advantage of these false assumptions
> being made by SPF.
>
> > > > But what if the SMTP operator does not want to use your new MAPS
> > > > CSV/DNA service? What if you go out of business? What if you
> > > > get enough support headaches from thousands of smaller systems
> > > > that you decide to raise the price to filter out these bothersome
> > > > clientele?
> > >
> > > This seems to be putting the cart before the horse.
> >
> > No. It is called business and practical product experience. You are
> > proposing an idea infested (wrongly placed) with capitalistic commercial
> > concepts, which I don't mind saying, you will be among the first to
benefit
> > from as the first DNA commercial service.
>
> CSV is offering an authenticated name for the sending MTA that helps in
> many ways with respect to dealing with the email abuse problem. The
> transition to the use of names will likely be an immediate benefit to
> those already providing vouching services.
>
> > PR problem #1. Therefore, if we were support and implement CSV, we
> > would have to include a disclaimer for any Anti-Spam CSV support
> > statement; "Batteries Not Included. DNA Service Subscription
> > Required." We are not going to add what is suppose to be a
> > "technical" solution with subscription concepts behind it. Period.
>
> An immediate benefit of CSV comes from being able to associate an
> authenticated name with the immediate sending MTA. MASS offers the
> feature that allows control over the sender mailbox-domain regardless of
> the immediate sending MTA. The blunt instrument provided by CSV
> protects the network from grossly negligent administrators. MASS should
> be able to protect users from those wishing to spoof sender
> mailbox-domains. MASS/CSV can work together in a unified name basis for
> guarding mail. SPF will not remove the need for some form of
> reputation/vouching service, nor remove the need for some form of
> network protection.
>
> > Dont' get me wrong. I'm for making a buck as much as the next guy. But
> > the "Email Problem" needs an open standard technical solution free of
> > 3rd party dependencies. It is a duel tier process. Transport
> > security must be done using SMTP first. Main Content Security is an
> > separate issue, however, there can be a tie in of the two, but not
> > dependent on it.
>
> There were several companies touting reputations services at the FTC
> Email Authentication Summit. I was not promoting such a service. I was
> insisting upon adoption of a safe and accurate means to assess abuse. I
> remain convinced MASS and CLEAR are considering viable solutions.
>
> > The point is, if you send me mail, I must be able to trust the
transaction
> > before I accept your payload. Authenticating the payload is an
> > ADMINISTRATIVE concept.
>
> Authenticating the payload is the subject of an ongoing effort in MASS.
> I want to see MASS succeed in providing an effective solution that can
> protect the sender mailbox-domain.
>
> > Using a CSV authorization domain is not sufficient WITHOUT looking at
the
> > 2821.Mail From as well as 2821.RCPTTO. Therefore, as it has been
> > emperically proven, using HELO by itself would be wasteful overhead
without
> > taken all process environment parameters into consideration.
>
> Holding the sending domain accountable offers substantial relief without
> a need to sort through millions of users employing various
> mailbox-domains within thousands of various providers. There is a fair
> amount of experience that supports this view.
>
> > I have PROOF of this issue. Doug. All you are doing is making assertions
of
> > the opposite and you have no proof to show I am wrong.
>
> This is not the proper venue for product related data. Most of the
> benefits claimed by SPF are related to changes in behavior, and not
> cessation of abuse. As an immediate source for relief, BATV would be
> more effective in preventing the unwanted bounce traffic as this can be
> done unilaterally.
>
> -Doug
>
>

[Douglas Otis]

On Tue, 2005-02-01 at 16:42 -0500, Hector Santos wrote:
> Is this a CANNED reply message? It sure reads like it.

It sure feels like the same issues.

> Look, CSV does not pass the test for consideration. That might
> explain why no commercial vendor has implemented it.

The goal for CSV is to reach consensus on a well considered design, run
tests, and then attempt to pass muster with a standards effort before
widely promoting it. Considering impacts upon existing infrastructure
has been no small part of this process.

It would seem proponents of SPF have concluded problems resulting from
changes to the rules without record versioning is acceptable. Does it
really matter the same view is held by the Sender-ID faction, as well as
the SPF-Classic faction? The spf.org site makes it clear Sender-ID
causes mail delivery problems. Schlitt's draft also makes it clear
forwarded mail may be lost and "Tests against some other identity may
also be used to override the test against the "MAIL FROM" identity."
What other identity? Being this vague ensures there will never be a
"chain of trust". This type of approach makes it impossible to judge,
from the perspective of implementation, who is doing what.

Forgive those that take a longer view. It seems SPF has already lost
control over how these records are used. Breaking mail as a way to
inflict change is not justified for a system that makes consumers rather
than providers accountable. Lack of authenticating the message source,
in the long run, may prove costly as these problem become apparent.

> And we were not talking about MASS. We are talking about the fallacy of CSV
> being immune to heterogeneous/mix policy topologies and lacks overhead
> issues. You are wrong in both cases.

You seem to be obsessing on having SPF replace the role of a responsible
administrator. Alas, SPF can not be trusted to properly indicate the
source of abuse. Please don't say this is not your concern, and that it
is the victim's.

> I think you are saying MASS solves these problems for CSV.

The need to protect the sender's mailbox-domain can be safely provided
by MASS. CSV never attempts to fulfill that role. CSV is the first
protective phalanx for the network, based upon a name rather than an IP
address. This could work in conjunction with a common name based
reputation system. MASS will never offer protection for the network,
whereas CSV can. SPF, with its high overhead, does not afford network
protection advantages.

> Well, that's a PAYLOAD solution so it doesn't address the heterogeneous/mix
> policies and overhead issues at the SMTP level. So I believe you are
> incorrect here as well.

A minimal overhead at the SMTP level is definitely an advantage that CSV
offers over SPF. The disagreement seems to be whether SPF provides
greater benefit by excluding messages that do not comply to an expensive
path registration. Forwarding is now defined as "illegitimate" and
messages lost as a result of transparent interception at hotels and
network kiosks is "acceptable" even though these are effective anti-spam
deterrents. Will customers accept such problems and blame under the
guise of defeating spam?

-Doug

[Frank Ellermann]

Douglas Otis wrote:

> SPF may say, "Here is a comprehensive address list", when in
> reality it can never be comprehensive.

Of course it is. There are at most 512 IPs sending mail from
***@xyzzy to ***@elsewhere. The other 254*256*256*256
IPs don't do this, unless they are forwarding mail from me, and
then it's something arranged by ***@elsewhere. He's free
to do whatever he likes, but of course he shouldn't check one
of his relay IPs against "my" 512 IPs.

And there at most 512 IPs who could say HELO xyzzy.claranet.de.
In reality no MTA should do this, but for this minor detail SPF
isn't good enough, maybe CSV could catch the remaining 512 IPs.
BTW, can it ?

> This is a problem created by the sender

"v=spf1 ip4:212.82.225.0/24 ip4:195.170.96.0/24 -all"
isn't "a problem created by the sender", it's simply the truth.

> SPF does not prevent spoofing of the domain, as many share
> common providers

If somebody authorized to use one of these 512 IPs spoofs "my"
vanity domain, then I'm sure that the abuse desk of my ISP can
solve this problem very quickly, otherwise they'd risk to lose
a happy customer.

And SPF offers solutions for this situation where necessary,
e.g. some existing policies use ? instead of + for shared IPs.

> there is no consensus which identities are checked against
> such records (when these records are even examined)

2821-From and -HELO are checked. Anything else won't work as
expected. Of course I can't force you to do "the right thing",
you could check something else somewhere else, but why bother
with DNS and SPF if you want some random rejects ? Try BLARS,
its cheaper than SPF for this purpose.

> CSV is intended to protect the network using the same name
> basis.

So far I haven't seen any spoofed HELO xyzzy.claranet.de in
bogus "bounces to" me, so probably CSV would have done nothing
at all for my problem.

> This domain assertion is not constrained to zone cuts. After
> lengthy consideration, it was determined this too would be
> inappropriate.

Can it identify HELO xyzzy.claranet.de as rubbish ? I've read
the discussions about SPF's "zone cut" in CLEAR, and the Jan 4
plus Jan 6 entries in <http://www.livejournal.com/users/fanf/>
- but that doesn't explain the details. And I haven't found a
case where `nslookup -q=ns sub.dom.ain.example` fails to find
the "zone cut".

> A well understood problem is not FUD

Claiming that SPF causes mails to be lost is definitely FUD,
Like the collection of monstrous lies in the PDF:
<https://antiphishing.kavi.com/events/Conference_Notes/sender_auth_myths_and_domainkeys.pdf>

BTW, I found that link on the page:
<http://podcast.resource.org/rf-rfc/index.html#item0003>

> A script based language is inherently complex

Yes, the macro language has its drawbacks, and the TINW in "we"
had some problems to get it right for some obscure cases like a
slash in a FQDN used as HELO. Actually that was in one of two
relevant articles in MARID (with thanks to Julia and Margaret)

OTOH Wayne now defined exactly 10 macro letters, 8 are used in
sender policies, anything else is an error. That's not too
complex. Don't try to convince me that the SPF "explanations"
are ridiculous, I wouldn't disagree (one of the minor issues I
hoped to get rid of in the former MARID WG, but here we are).

[overall timeout]
>> From the Schlitt's draft:

ACK, error on my side, this "implementation detail" is still a
part of the draft, and maybe I'll discuss it with Wayne on the
SPF list, it's unnecessary. Of course there must be a way to
report and log "local error conditions".

In the lentczner-drafts the optional overall timeout was a very
important part of "processing limits" and indirectly "security
considerations", now it's on the same level as say a "sigterm".

> The draft requires these 101 to 201 lookup limits depending
> upon the mechanisms.

If you have some local reasons to ignore a sender policy, then
just do it. And if you want a documented reason to state that
a sender policy is erroneous follow the spec. The latter is a
reason to reject the mail with a PermError, the sender policy
has to be fixed.

> By checking per HELO, the rate CSV does a lookup is some
> multiple less than SPF, and constrained to a single lookup
> and not hundreds of lookups. This aspect is critical.

For me it's critical that a forged "mail from me" is rejected,
and CSV won't do this trick. If you think that the two lookups
to check any "mail from me" are too much just don't do it.

SPF worked for me as soon as some "important players" (defined
from the POV of "my" spammer) implemented it, it's not critical
if you don't use it.
Bye, Frank

[Hector Santos]

----- Original Message -----
From: "Frank Ellermann" <***@xyzzy.claranet.de>
To: <ietf-***@imc.org>
Sent: Wednesday, February 02, 2005 8:07 AM
Subject: Re: So here it is one year later...


>
> BTW, I found that link on the page:
> <http://podcast.resource.org/rf-rfc/index.html#item0003>

Ha! How about this new "Think" protocol?

When you think connection-oriented transport services, think TCP!
When you think legacy protocols, think BOOTP!
When bandwidth is cheaper than memory, think MIME!
When you think impedence mismatch, think LDAP!

Hilarious!

Anyway, to the PlayPod radio host. GREAT IDEA!!!

How about getting some developer viewpoints in your shows?

Sincerely,

Hector Santos, CTO
Santronics Software, Inc.
http://www.santronics.com
305-431-2846 Cell
305-248-3204 Office

[Douglas Otis]

On Wed, 2005-02-02 at 14:07 +0100, Frank Ellermann wrote:
> Douglas Otis wrote:
>
> > SPF may say, "Here is a comprehensive address list", when in
> > reality it can never be comprehensive.
>
> Of course it is. There are at most 512 IPs sending mail from
> ***@xyzzy to ***@elsewhere. The other 254*256*256*256
> IPs don't do this, unless they are forwarding mail from me, and
> then it's something arranged by ***@elsewhere. He's free
> to do whatever he likes, but of course he shouldn't check one
> of his relay IPs against "my" 512 IPs.

Today, mail is not checked for a path registration against some
mailbox-domain, and forwarding works as expected. Now SPF allows any
mailbox-domain to be checked against path registration, according to
Schlitt's SPF draft. As a result, there is no assurance that a specific
mailbox-domain was checked. Recipients would be guessing which
mailbox-domain was used to authorize the message, assuming such
authorization was required. The benefits of this process remains highly
dubious.

On the promise of a benefit, this scheme requires administrators to
institute a program of white-listing servers that forward mail on behalf
of their customers. Such white-listing practices permit undetected
abuse, when these servers are excluded from the SPF rule-set (see
draft). This system goes from some mailbox-domain may have been
checked, to perhaps none, even when there is an SPF policy in place for
the sending mailbox-domains. Of course, there is the vast number that
still do not use SPF.

Forwarding of mail involves three parties, the sender-domain, the
recipient-domain, and the forwarding-domain.

A) You want the sender-domain to register all servers used, which is
not in keeping with the normal scale for a DNS response.

B) The recipient domain then must white-list all forwarding-domains
based upon information derived from their users.

Once the recipient-domain also employs the SPF rule-set for accepting
messages, these forwarded messages may become lost or rejected. Don't
forget about the change made with respect to HELO, where now even the
bounce may be refused. The needed feedback to know there is a problem
may have been lost due to the zone-cut change.

Do you still insist you can list all servers? You seem to be admitting
that you are leaving some of this listing work to the recipient-domain,
in that they must white-list forwarded accounts, some mailing-lists,
perhaps some hotel and network kiosks, etc. etc. Why not admit that it
is _impossible_ to include the entire list?

> And there at most 512 IPs who could say HELO xyzzy.claranet.de.
> In reality no MTA should do this, but for this minor detail SPF
> isn't good enough, maybe CSV could catch the remaining 512 IPs.
> BTW, can it ?

CSV does not attempt to include all the IP addresses for an entire
mailbox-domain, but instead limits a response to that of a single
server. There is no difficulty created when more servers are used for a
very large domain, even spanning millions of addresses. Each individual
response is limited to that for a specific server. Even the DNS entry
does not deal with addresses, as this process is automated within the
SRV RR type.

As a mail provider, the HELO could reflect which domain is generating
the traffic (with the cooperation of the sending domain). If the desire
is to isolate accountability, then the HELO technique could satisfy this
need, while not breaking mail to the same extent as attempting to
constrain mail to some mailbox-domain/IP address binding.

Although this is a function provided by the sending domain policy, a bit
within the CSV record could indicate the HELO will "always" be within
the domain of the MAILFROM. The reason not to make this assertion is
the same as why SPF should not make their assertions. It would always
be a lie. The difference is that CSV scales and could achieve the same
silly goal without abusing DNS. : )

> > This is a problem created by the sender
>
> "v=spf1 ip4:212.82.225.0/24 ip4:195.170.96.0/24 -all"
> isn't "a problem created by the sender", it's simply the truth.

I would say it is only half of the truth. The other half is the need
for random white-listing to keep from disrupting current widespread and
legitimate practices.

> > SPF does not prevent spoofing of the domain, as many share
> > common providers
>
> If somebody authorized to use one of these 512 IPs spoofs "my"
> vanity domain, then I'm sure that the abuse desk of my ISP can
> solve this problem very quickly, otherwise they'd risk to lose
> a happy customer.

You may have authorized several providers to send mail on behalf of your
domain, and discovered a problem by way of a poor reputation assessment
or, even worse, your mail being filtered. You will likely find it
difficult to learn where the problem originated and receive little
cooperation from reputation or filtering services, as they protect
sources. The several providers may not care whether you remain a happy
customer when you ask them to do additional work. Perhaps they only
consider Sender-ID a valid choice and you insist that logs be sorted for
SPF specifics. After all, SPF/Sender-ID ensures the problem is yours
and not theirs. Very likely, your recourse will be expensive and
perhaps futile.

> And SPF offers solutions for this situation where necessary,
> e.g. some existing policies use ? instead of + for shared IPs.

Providing a PTR list of CSV authenticated domains compared against a
mailbox-domain would achieve these filtering goals without hundreds of
DNS lookups, as potentially required by SPF. There is much less effort
maintaining names than huge address lists, and this response would fit
within the constraints of DNS.

> > there is no consensus which identities are checked against
> > such records (when these records are even examined)
>
> 2821-From and -HELO are checked. Anything else won't work as
> expected. Of course I can't force you to do "the right thing",
> you could check something else somewhere else, but why bother
> with DNS and SPF if you want some random rejects ? Try BLARS,
> its cheaper than SPF for this purpose.

Schlitt's SPF draft:
There are several possible ways that [an] authorization failure can
be ameliorated. ... Tests against some other identity may also be
used to override the test against the "MAIL FROM" identity.

If MAILFROM doesn't work, then try an assortment of mailbox-domains.
You can not say with any certainty, even for those following Schlitt's
draft, that a specific mailbox-domain was checked. Some chain of trust!

> > CSV is intended to protect the network using the same name
> > basis.
>
> So far I haven't seen any spoofed HELO xyzzy.claranet.de in
> bogus "bounces to" me, so probably CSV would have done nothing
> at all for my problem.

With a field not often checked, there is little reason to lie. Checking
the HELO is not disruptive, as it does not affect the content of the
envelope or the message. Using HELO to establish accountability
protects the owners of the mailbox-domains from being falsely accused
for having sent unauthenticated messages. Unlike other entities within
the message, without a signature, only the HELO can be authenticated.
SPF assertions assume a fallacious "chain of trust." Schlitt's draft
makes a mockery of that concept.

> > This domain assertion is not constrained to zone cuts. After
> > lengthy consideration, it was determined this too would be
> > inappropriate.
>
> Can it identify HELO xyzzy.claranet.de as rubbish ? I've read
> the discussions about SPF's "zone cut" in CLEAR, and the Jan 4
> plus Jan 6 entries in <http://www.livejournal.com/users/fanf/>
> - but that doesn't explain the details. And I haven't found a
> case where `nslookup -q=ns sub.dom.ain.example` fails to find
> the "zone cut".

There were several reasons for not using the zone-cut. There is some
blood on the floor regarding this topic. Complying with the way domains
and zones are used, how software is structured, and what is reasonable
to expect of the server in some odd cases, were the major reasons in my
view. For the most part, this draft is in its final stages. The design
team wishes to present their finished version rather than a series of
interim drafts. CSV is very much a team effort.

> > A well understood problem is not FUD
>
> Claiming that SPF causes mails to be lost is definitely FUD,

Odd, I would think that the mention of lost mail in Schlitt's SPF draft
would have removed the uncertainty and doubt. I still think fear is
well advised. : )

> Like the collection of monstrous lies in the PDF:
> <https://antiphishing.kavi.com/events/Conference_Notes/sender_auth_myths_and_domainkeys.pdf>

I would be interested to know what you find in error, besides too many
slides. : )

> BTW, I found that link on the page:
> <http://podcast.resource.org/rf-rfc/index.html#item0003>

Marshall and Andy fail to consider anything other than path registration
in this bit of nostalgia. They are consistent.

> > A script based language is inherently complex
>
> Yes, the macro language has its drawbacks, and the TINW in "we"
> had some problems to get it right for some obscure cases like a
> slash in a FQDN used as HELO. Actually that was in one of two
> relevant articles in MARID (with thanks to Julia and Margaret)
>
> OTOH Wayne now defined exactly 10 macro letters, 8 are used in
> sender policies, anything else is an error. That's not too
> complex. Don't try to convince me that the SPF "explanations"
> are ridiculous, I wouldn't disagree (one of the minor issues I
> hoped to get rid of in the former MARID WG, but here we are).
>
> [overall timeout]
> >> From the Schlitt's draft:
>
> ACK, error on my side, this "implementation detail" is still a
> part of the draft, and maybe I'll discuss it with Wayne on the
> SPF list, it's unnecessary. Of course there must be a way to
> report and log "local error conditions".

Let me remind you. A lookup may entail several queries to resolve the
resource. The timeout for each query typically involves a timeout of 5
seconds with perhaps 2 or 3 retries which double the timeout after each.
Waiting for a timeout to expire is how UDP is prevented from flooding a
network during congestion. With the 101 lookup limit (higher for
reverse DNS resolution), should an implementation attempt to resolve SPF
in parallel, there may be many outstanding DNS queries not completed
when everything starts anew for the next message. A slow DNS server
could entail an average of 3 queries per lookup which totals 303 queries
or about 25 minutes per message, without invoking a single timeout or
lookup limit! This time could be more than doubled by using reverse DNS
resolution. : (

> In the lentczner-drafts the optional overall timeout was a very
> important part of "processing limits" and indirectly "security
> considerations", now it's on the same level as say a "sigterm".

This view overlooks the need for UDP exponential fallback! Think of the
network and not just the program.

> > The draft requires these 101 to 201 lookup limits depending
> > upon the mechanisms.
>
> If you have some local reasons to ignore a sender policy, then
> just do it. And if you want a documented reason to state that
> a sender policy is erroneous follow the spec. The latter is a
> reason to reject the mail with a PermError, the sender policy
> has to be fixed.

Why write a spec and then imply that it should not be followed? Don't
you see a problem?

> > By checking per HELO, the rate CSV does a lookup is some
> > multiple less than SPF, and constrained to a single lookup
> > and not hundreds of lookups. This aspect is critical.
>
> For me it's critical that a forged "mail from me" is rejected,
> and CSV won't do this trick. If you think that the two lookups
> to check any "mail from me" are too much just don't do it.

SPF does NOT prevent mail from being mistaken as being from you. SPF
normally checks the MAILFROM. MAILFROM is rarely seen by the user. CSV
can authenticate the HELO domain, SPF can only determine that the
message has been authorized by a domain.

When one considers the need to stop spam, reputation is an important
aspect of that effort. No reputation service should hold a domain
culpable just because a message had been authorized. With the HELO
authenticated, any abuse will be attributed to the correct entity. With
SPF, you are using a weapon that does not always shoot straight. Unlike
SPF, CSV is as accurate as the underlying infrastructure.

> SPF worked for me as soon as some "important players" (defined
> from the POV of "my" spammer) implemented it, it's not critical
> if you don't use it.

Take the longer view and considering what is needed to improve security
and to better protect networks. There will be less mess than what may
be created by the many potential exploits enabled by SPF. Although SPF
may temporarily achieve some reductions in bounce traffic, BATV is a
better immediate solution. MASS and CLEAR still represent viable and
well considered solutions. SPF still should be discouraged.

-Doug

[Alan DeKok]

Douglas Otis <***@mail-abuse.org> wrote:
> Those publishing SPF records want their mail to go missing?

Any mail they didn't send.

> There is no means to know which recipient may be using a forwarded
> account.

Yup. Likewise, there's no way to know who on the net will be
sending spam forged to be "from" your domain.

> Forwarding is a common practice within colleges, societies, and many
> providers.

As is spamming. As is drunk driving. Having something a "common
practice" doesn't mean it's right.

> Validating the legitimacy of an MTA can take place within a single
> lookup of a small CSV-CSA record.

Which is a great idea, and has been integrated into SPF.

> A single lookup does not increase the risk to DoS attacks, and also
> does not create inadvertent loss of mail, as does SPF.

No, "intentional" loss of mail. I intend every single spam which
uses my domain name to be discarded. I would rather have others
discard them than have those people complain to me that I'm spamming
them.


This is not to say SPF is perfect. Other methods do much of what
SPF does, without it's problems.

But as a concept, the domain owner should be able to control the use
of that name. If we can't agree on that, then the only logical
alternative is that third parties can use someones name without their
consent, and therefore forged spam is OK.

Alan DeKok.

[wayne]

In <***@mail.nitros9.org> "Alan DeKok" <***@ox.org> writes:

> Douglas Otis <***@mail-abuse.org> wrote:
>
>> There is no means to know which recipient may be using a forwarded
>> account.
>
> Yup. Likewise, there's no way to know who on the net will be
> sending spam forged to be "from" your domain.

If the sending domain uses SES, then it can detect the forwarder
situation and not cause an SPF failure. (This can also be used for
roaming users.) The cost is an extra DNS lookup.

The receiving MTA can know if email is being sent through a
forwarder. For example, the user can tell the system via a whitelist
request. Spamassassin's auto-whitelists will automatically help out a
great deal because it detects the overall spamminess of the forwarder and
that will be enough to override the SPF failures.

Better systems could be built to automatically detect forwarders by
looking to see source usually fails the 2821.MAILFROM SPF check, but
usually passes when the 2822.To: (or cc:) gets an SPF check done on
it.


> This is not to say SPF is perfect. Other methods do much of what
> SPF does, without it's problems.

I certainly agree that SPF is not perfect. I am, however, curious
which systems you think do much better than SPF and what data you have
to back up that opinion. So far, I haven't found any data to back
that claim.



-wayne

[Alan DeKok]

wayne <***@schlitt.net> wrote:
> I certainly agree that SPF is not perfect. I am, however, curious
> which systems you think do much better than SPF

I did not say "better".

There are schemes which, individually, do less than what SPF does.
As a result, they do not have the issues that SPF has, because SPF has
a wider scope and therefore wider side-effects.

Alan DeKok.

[Frank Ellermann]

Andrew Newton wrote:

> If the intent was to have the IETF simply rubber-stamp SPF,
> making that clear up front would have saved us all a lot of
> time.

My intention was always to get a RfC for LMAP (with SPF as the
most promising candidate at this time, but also including CSV),
i.e. to find and fix all bugs and oddities in these concepts.

You never allowed this to happen here. You always pressed for
2822 and the IMHO broken Sender-ID concept, and when that failed
you let the A-D close this WG without prior consultation.
--
<http://purl.net/xyzzy/home/test/MARID.appeal.txt>

[Alan DeKok]

Frank Ellermann <***@xyzzy.claranet.de> wrote:
> You never allowed this to happen here. You always pressed for
> 2822 and the IMHO broken Sender-ID concept, and when that failed
> you let the A-D close this WG without prior consultation.

The chairs asked for consensus from the group. The consensus was to
look at 2822 and Sender-Id.

Once that happened, further consensus did not occur.

Alan DeKok.

[wayne]

In <***@mail.nitros9.org> "Alan DeKok" <***@ox.org> writes:

> Frank Ellermann <***@xyzzy.claranet.de> wrote:
>> You never allowed this to happen here. You always pressed for
>> 2822 and the IMHO broken Sender-ID concept, and when that failed
>> you let the A-D close this WG without prior consultation.
>
> The chairs asked for consensus from the group. The consensus was to
> look at 2822 and Sender-Id.
>
> Once that happened, further consensus did not occur.


No.

The chairs/AD asked for consensus from the group. The consensus was to
look at the 2821 identities first and then look at the 2822
identities. The charter promised that once this decision was reached,
that all further discussions on other identities would be ruled out of
scope.

This did not happen.

The chairs/AD, at the interim meeting, decided to ignore this and
switched to 2822 with SenderID and the PRA as the only option. This
complete change of direction was never confirmed on the mailing list,
despite the IETF rules on the subject.

(Ok, CSV was left on the table, to be considered after SenderID.
That, however, completely ignored SPF's 2821.HELO checking, and all
forms of 2821.MAILFROM checking.)


MARID showed just how much more important political considerations are
in making decisions than technical considerations. The end run that
the IETF is making, right now, around the technical problems with
SenderID is just another great example. The SenderID I-Ds have now
been sent off to a secret directorate for review, with no place to
make public comments. It is not even clear if the IESG will allow an
IETF wide last-call for the SenderID I-Ds before they are promoted to
being RFCs.

But, the people on this MARID mailing list don't even care about
that. They would much rather discuss the SPF-classic I-Ds, something
that was never adopted by the MARID working group.


-wayne

[Alan DeKok]

wayne <***@schlitt.net> wrote:
> The chairs/AD asked for consensus from the group. The consensus was to
> look at the 2821 identities first and then look at the 2822
> identities. The charter promised that once this decision was reached,
> that all further discussions on other identities would be ruled out of
> scope.

i.e. http://www.imc.org/ietf-mxcomp/mail-archive/msg01161.html

My recollection was that there was a later consensus call, but I
can't find the post. The closest is:

http://www.imc.org/ietf-mxcomp/mail-archive/msg01864.html

Where you note that the consensus of the WG (simple hum, unconfirmed
on the list) was changed to RFC 2822.

> MARID showed just how much more important political considerations are
> in making decisions than technical considerations.

Almost all spam/anti-spam discussion is political. We know how to
stop all spam: turn off all email. Past that, the "technical"
decision to turn off limited portions of email for some reason
(e.g. message/host validation) quickly becomes political. Some people
will not implement the scheme themselves, and others oppose a scheme
in general.

Alan DeKok.

[william(at)elan.net]

On Sat, 29 Jan 2005, Alan DeKok wrote:

> Frank Ellermann <***@xyzzy.claranet.de> wrote:
> > You never allowed this to happen here. You always pressed for
> > 2822 and the IMHO broken Sender-ID concept, and when that failed
> > you let the A-D close this WG without prior consultation.
>
> The chairs asked for consensus from the group. The consensus was to
> look at 2822 and Sender-Id.

That is not exactly true. Prior to May interim meeting there was general
consensus that we're going to look and work with RFC2821 identities and
RFC2821 MAIL FROM in particular.

On that meeting it was announced that Meng & Microsoft agreed to merge
into what was essentially CallerID (xml dns record, rfc2822 identities,
pra), but its syntax would be closer to supporting spf with clients
also support spf1 records. It was presented to the group in the way
that left no serious alternative except CSV for checking HELO which
was also worked by the WG.

There were people on that meeting who did not like the kind of merger
between spf and callerid that was announced but we were willing to at
least try to see what this would turn out into especially since IETF
was willing to put its hand into making it work. Within month however
it became clear that XML was not the way to go (not for DNS TXT record
at the very least) and within another month it was also seen that
RFC2822 identities is really not the right thing to do per-hop
authentication of incoming mail server by ip.

Besides that Microsoft intentions and its willingness to work every party
that had interest in MARID was seriously in question and it was seen
that Microsoft and its PRA was more of a burden then help.

So by IETF60 there was really no longer consensus, but parties most
interested in SID were still trying hard to push it through. The
result was not unexpected that when actual poll on consensus (last-call
on sid documents) was done it fall miserably.

The correct thing to do would have been to re-asses the situation and check
for consensus again to see if any alternative solution would be have
consensus to go forward and would not have the same technical problems.
But it was also from the start felt by many that IETF was under a lot of
pressure from corporate interests to standardize CallerID/SenderID and
that MARID was nothing but a show to slightly polish it before putting
ietf stamp on it and that that abandoning in favor of another alternative
would be seen as unacceptable to very very large corporate parties who
would take it as a sign that they should not come and work with IETF again.

So unfortunately, instead of choosing to clearly disregard what was
technically unacceptable proposal that does not have consensus of the
community,IETF decided to just disband the WG and make decisions about
the same proposal privately at the level of IESG and select group of
individuals who we don't even know - DEA Directorate had been formed but
its membership seems to be kept private even though Ted Hardie said here
in message http://www.imc.org/ietf-mxcomp/mail-archive/msg05054.html
that it will be made public. I'm hoping that Ted Hardie and IESG will
follow through on its promise and that there is no official information
on IETF pages about DEA is just an oversight due to very overworked staff
who had not yet had time to create appropriate information page.

--
William Leibzon
Elan Networks
***@elan.net

[Frank Ellermann]

Alan DeKok wrote:

> The consensus was to look at 2822 and Sender-Id.

That was a decree and no consensus. CSV was declared off topic
until the 2822 discussions were ready, and as a result CSV was
never discussed. Same problem for classic SPF, minus the one
week when the former WG chair asked for a spf2.0/mfrom draft,
and let the WG be closed three (?) days after it was published.

There's a serious problem with RfC 3710 (4.3) vs. RfC 2418 (4).
It's a bad sign that 3710 and 2418 are now among the relatively
few RfCs that I know, thanks but no thanks to MARID. :-(

Bye, Frank

[Dean Anderson]

On Thu, 27 Jan 2005, Gordon Fecyk wrote:

>
> ...since the first rumblings of talking about a working group, and there's
> been no press on MARID since the breakup of said working group. No press
> from Microsoft, no press from the SPF crowd, no press from anyone.

The working group broke up because of unresolvable technical problems.

> And no instructions included on how to compile, install, or use what little
> software is available out there. Except for the few commercial (and
> expensive) offerings by GFi.

If you aren't a developer of SPF, probably you shouldn't be using SPF.

> What gives? Has the whole world lost complete interest in stopping spam? Is
> spyware really the next big threat to the Internet that the US Congress is
> looking at legislation for it but not bothering with spam anymore?

No, the world has just realized that SPF doesn't work, and won't stop
spam, nor stop forgery. In its present form, SPF does nothing except
create more opportunities for email abuse, and promotes spam and email
abuse. Most people are interested in things that work. Fewer are
interested in making things worse.

--Dean

--
Av8 Internet Prepared to pay a premium for better service?
www.av8.net faster, more reliable, better service
617 344 9000

[t***@ashtonwoodshomes.com]

> -----Original Message-----
> From: owner-ietf-***@mail.imc.org
> [mailto:owner-ietf-***@mail.imc.org]On Behalf Of Dean Anderson
> Sent: Thursday, January 27, 2005 8:48 PM
> To: Gordon Fecyk
> Cc: IETF MXCOMP (E-mail)
> Subject: Re: So here it is one year later...
>
>
>
> On Thu, 27 Jan 2005, Gordon Fecyk wrote:
>
> >
> > ...since the first rumblings of talking about a working
> group, and there's
> > been no press on MARID since the breakup of said working
> group. No press
> > from Microsoft, no press from the SPF crowd, no press from anyone.
>
> The working group broke up because of unresolvable technical problems.

Get specific Dean, the unresolvable technical issues was the the PRA is broken, NOT SPF. So it is SenderID's PRA
component not its SPF component that caused the technical failure.

And the senderid venture was doomed to failure anyway because of the M$ IPR crap.

>
> > And no instructions included on how to compile, install, or
> use what little
> > software is available out there. Except for the few commercial (and
> > expensive) offerings by GFi.
>
> If you aren't a developer of SPF, probably you shouldn't be using SPF.

Wrong again. And the proof is that there are many domains that implement SPF, successfully.

>
> > What gives? Has the whole world lost complete interest in
> stopping spam? Is
> > spyware really the next big threat to the Internet that the
> US Congress is
> > looking at legislation for it but not bothering with spam anymore?
>
> No, the world has just realized that SPF doesn't work, and won't stop
> spam, nor stop forgery.

SPF never pretended to stop spam. It does prevent forgery, it does not prevent phishing, but then no technical solution
will ever solve phishing as long as MUA's like Outlook show "pretty names" for everything, suppressing the try
underlying identities of everything from email addresses to attachment types.

> In its present form, SPF does nothing except
> create more opportunities for email abuse, and promotes spam and email
> abuse. Most people are interested in things that work. Fewer are
> interested in making things worse.

I cannot see how you come to that conclusion, (especially not on your lack of facts you presented). It does however
well illustrate that there was/is an effort to scuttle SPF. Mostly by those with alternative products. Which is
interesting, because if SPF was not a threat to the alternative products, they wouldn't try to scuttle SPF.

Terry Fielder

>
> --Dean
>
> --
> Av8 Internet Prepared to pay a premium for better service?
> www.av8.net faster, more reliable, better service
> 617 344 9000
>
>

[Dean Anderson]

On Thu, 27 Jan 2005 ***@ashtonwoodshomes.com wrote:

> > -----Original Message-----
> > From: owner-ietf-***@mail.imc.org
> > [mailto:owner-ietf-***@mail.imc.org]On Behalf Of Dean Anderson
> > Sent: Thursday, January 27, 2005 8:48 PM
> > To: Gordon Fecyk
> > Cc: IETF MXCOMP (E-mail)
> > Subject: Re: So here it is one year later...
> >
> >
> >
> > On Thu, 27 Jan 2005, Gordon Fecyk wrote:
> >
> > >
> > > ...since the first rumblings of talking about a working
> > group, and there's
> > > been no press on MARID since the breakup of said working
> > group. No press
> > > from Microsoft, no press from the SPF crowd, no press from anyone.
> >
> > The working group broke up because of unresolvable technical problems.
>
> Get specific Dean, the unresolvable technical issues was the the PRA is
> broken, NOT SPF. So it is SenderID's PRA component not its SPF
> component that caused the technical failure.

That was _a_ problem. It wasn't the only problem.

> And the senderid venture was doomed to failure anyway because of the M$ IPR crap.

I agree the IPR crap was a bad thing. (I am the president of the LPF,
after all). However, I don't think it doomed standardization to failure.
The IETF has standardized RFCs with patented algorithms before. People
made an informed choice, and decided that anti-spam technology is
something that needs to be pervasive, free, and unencumbered. This has to
be a major concern for the privateers interested "making a fortune in
anti-spam"

> > > And no instructions included on how to compile, install, or
> > use what little
> > > software is available out there. Except for the few commercial (and
> > > expensive) offerings by GFi.
> >
> > If you aren't a developer of SPF, probably you shouldn't be using SPF.
>
> Wrong again. And the proof is that there are many domains that implement SPF, successfully.

Most them are spammers.

> > > What gives? Has the whole world lost complete interest in
> > stopping spam? Is
> > > spyware really the next big threat to the Internet that the
> > US Congress is
> > > looking at legislation for it but not bothering with spam anymore?
> >
> > No, the world has just realized that SPF doesn't work, and won't stop
> > spam, nor stop forgery.
>
> SPF never pretended to stop spam.

That isn't what Meng Weng said it Linux World. He said it would end spam.
There are many other similar articles. Indeed, Microsoft also announced
it would "end spam".

> It does prevent forgery,

Demonstrably, it does not. Excluding DNS spoofing, which is also trivial,
one only needs an account at the same ISP as the forgery target. Many
ISPs will not use SPF because of the DOS vulnerabilities, and unless
everyone use SPF, it will not stop forgery, even given an assumption that
every domain has a unique IP (which isn't a realistic assumption)

> it does not prevent phishing, but then no technical solution will ever
> solve phishing as long as MUA's like Outlook show "pretty names" for
> everything, suppressing the try underlying identities of everything from
> email addresses to attachment types.

I didn't hear anyone say it would prevent phishing. Phishing is only
tangentially related to forgery, anyway. Even if you could stop forgery,
it would not alter phishing.

> > In its present form, SPF does nothing except create more opportunities
> > for email abuse, and promotes spam and email abuse. Most people are
> > interested in things that work. Fewer are interested in making things
> > worse.
>
> I cannot see how you come to that conclusion, (especially not on your
> lack of facts you presented).

Facts were already presented. The working group dissolved. Look at the
IETF-MXCOMP archives. About 15 different critical, insurmountable problems
were listed by me and others.

> It does however well illustrate that there was/is an effort to scuttle
> SPF. Mostly by those with alternative products. Which is interesting,
> because if SPF was not a threat to the alternative products, they
> wouldn't try to scuttle SPF.

None of the people speaking about SPF problems have alternative products.
Some of the people in favor of SPF however have significant investments in
products, or have interests in its other properties, such as preventing
email outsourcing by binding together email and access. AOL, MSN, etc for
example, have interests in the latter.

The SPF steamroller attempts to move on anyway.

--
Av8 Internet Prepared to pay a premium for better service?
www.av8.net faster, more reliable, better service
617 344 9000

[g***@metro.cx]

On Thu, Jan 27, 2005 at 10:19:36PM -0500, Dean Anderson wrote:
> On Thu, 27 Jan 2005 ***@ashtonwoodshomes.com wrote:
> > Wrong again. And the proof is that there are many domains that implement SPF, successfully.
>
> Most them are spammers.

May we see some proof of that please? And with proof i don't mean that
report that is by now almost a year old and which was found to be
incorrect a while ago.

I don't have proof of the opposite, but i do seem to experience that
many non-spammers are implementing spf (i'm saying this based on the
requests the spf 'tech support team' gets in and activity on spf mailing
lists).

Koen

--
K.F.J. Martens, Sonologic, http://www.sonologic.nl/
Networking, embedded systems, unix expertise, artificial intelligence.
Public PGP key: http://www.metro.cx/pubkey-gmc.asc
Wondering about the funny attachment your mail program
can't read? Visit http://www.openpgp.org/

[Dean Anderson]

I don't do the usage tracking. It was reported on the mxcomp and other
lists. I've never heard anyone dispute it before. I'd have to dig through
my mail archives to find the reports, but my recollection is that they
came from people doing the usage tracking.

While I'm not disputing that there are non-spammers using SPF, the
spammers rushed to it for fairly obvious reasons. Anything that even
suggests that mail might be accepted or subjected to less checks is
definitely a big win for spammers.

--Dean

On Fri, 28 Jan 2005 ***@metro.cx wrote:

>
> On Thu, Jan 27, 2005 at 10:19:36PM -0500, Dean Anderson wrote:
> > On Thu, 27 Jan 2005 ***@ashtonwoodshomes.com wrote:
> > > Wrong again. And the proof is that there are many domains that implement SPF, successfully.
> >
> > Most them are spammers.
>
> May we see some proof of that please? And with proof i don't mean that
> report that is by now almost a year old and which was found to be
> incorrect a while ago.
>
> I don't have proof of the opposite, but i do seem to experience that
> many non-spammers are implementing spf (i'm saying this based on the
> requests the spf 'tech support team' gets in and activity on spf mailing
> lists).
>
> Koen
>
>

--
Av8 Internet Prepared to pay a premium for better service?
www.av8.net faster, more reliable, better service
617 344 9000

[Justin Mason]

***@metro.cx writes:
> On Thu, Jan 27, 2005 at 10:19:36PM -0500, Dean Anderson wrote:
> > On Thu, 27 Jan 2005 ***@ashtonwoodshomes.com wrote:
> > > Wrong again. And the proof is that there are many domains that implement SPF, successfully.
> >
> > Most them are spammers.
>
> May we see some proof of that please? And with proof i don't mean that
> report that is by now almost a year old and which was found to be
> incorrect a while ago.

FWIW, here's the results of a check of 54725 spams and 6680 nonspam mails,
from SpamAssassin's weekly mass-check of network rules (at
http://www.pathname.com/~corpus/NET.age ).

All these messages were received less than 1 month ago, and are taken from
5 people's hand-classified corpora.

SPF records passing HELO strings: 4.98% of spam, 13.29% of ham
SPF records passing the MAIL FROM: 3.72% spam, 18.90% of ham

So it certainly looks like that statement is untrue.

- --j.

[Dean Anderson]

On Fri, 28 Jan 2005, Justin Mason wrote:

>
> FWIW, here's the results of a check of 54725 spams and 6680 nonspam mails,
> from SpamAssassin's weekly mass-check of network rules (at
> http://www.pathname.com/~corpus/NET.age ).
>
> All these messages were received less than 1 month ago, and are taken from
> 5 people's hand-classified corpora.
>
> SPF records passing HELO strings: 4.98% of spam, 13.29% of ham
> SPF records passing the MAIL FROM: 3.72% spam, 18.90% of ham
>
> So it certainly looks like that statement is untrue.

Err, no:

OVERALL% SPAM% HAM% S/O RANK SCORE NAME:0-1
OVERALL% SPAM% HAM% S/O RANK SCORE NAME:1-3
OVERALL% SPAM% HAM% S/O RANK SCORE NAME:3-6

5.377 3.7259 18.9072 0.165 0.23 -0.00 SPF_PASS:0-1
1.361 0.9087 3.3508 0.213 0.25 -0.00 SPF_PASS:1-3
1.749 0.5116 18.4304 0.027 0.34 -0.00 SPF_PASS:3-6

As you see, spam + ham does not add up to overall. Its not clear what
these statistics mean, nor how they were calculated. But your
interpretation is clearly either wrong or at least not supported by the
page.

FYI, this is the original post:

---------- Forwarded message ----------
Date: Thu, 9 Sep 2004 15:18:42 +0200
From: Markus Stumpf <maex-lists-email-ietf-***@Space.Net>
To: ietf-***@vpnc.org
Subject: SPF abused by spammers


Justin Murdock posted this link on the qmail list:
http://news.bbc.co.uk/1/hi/technology/3631350.stm
"CipherTrust [...] found that 34% more spam is passing SPF checks than
legitimate e-mail."

\Maex

--
SpaceNet AG | Joseph-Dollinger-Bogen 14 | Fon: [phone]
Research & Development | D-80807 Muenchen | Fax: +49 (89)
32356-299
"The security, stability and reliability of a computer system is
reciprocally
proportional to the amount of vacuity between the ears of the admin"



--Dean

--
Av8 Internet Prepared to pay a premium for better service?
www.av8.net faster, more reliable, better service
617 344 9000

[Dean Anderson]

On Fri, 28 Jan 2005, Dean Anderson wrote:
>
> As you see, spam + ham does not add up to overall. Its not clear what
> these statistics mean, nor how they were calculated. But your
> interpretation is clearly either wrong or at least not supported by the
> page.

Sorry to reply to my own post, but there is another point I forgot: when
dealing with a very sample from a very small domain, that deals with a
limited part of the internet, E.G. a domain with relatively few people who
develop SPF, and who therefore exchange disproportionately more mail with
other SPF users, then their email statistics on SPF usage will also be
skewed. In other words, this is not a random sample.

One needs to get better sample from someplace that doesn't particularly
cater to SPF email users, and interacts more broadly with the internet.

--Dean

--
Av8 Internet Prepared to pay a premium for better service?
www.av8.net faster, more reliable, better service
617 344 9000

[Justin Mason]

Dean Anderson writes:
> On Fri, 28 Jan 2005, Dean Anderson wrote:
> >
> > As you see, spam + ham does not add up to overall. Its not clear what
> > these statistics mean, nor how they were calculated. But your
> > interpretation is clearly either wrong or at least not supported by the
> > page.
>
> Sorry to reply to my own post, but there is another point I forgot: when
> dealing with a very sample from a very small domain, that deals with a
> limited part of the internet, E.G. a domain with relatively few people who
> develop SPF, and who therefore exchange disproportionately more mail with
> other SPF users, then their email statistics on SPF usage will also be
> skewed. In other words, this is not a random sample.
>
> One needs to get better sample from someplace that doesn't particularly
> cater to SPF email users, and interacts more broadly with the internet.

FWIW, those figures come from contributors at 5 different sites. It's not
just one domain. Some are not even spam filter developers. ;)

Ciphertrust's figures are also biased -- their customers in turn will be a
certain cross-section of email users, rather than "truly representative"
of the internet at large.

I don't think anyone has yet come up with a way to get a hand-classified
sample that can reflect everyone's use of email...

- --j.

[Justin Mason]

Dean Anderson writes:
> On Fri, 28 Jan 2005, Justin Mason wrote:
>
> >
> > FWIW, here's the results of a check of 54725 spams and 6680 nonspam mails,
> > from SpamAssassin's weekly mass-check of network rules (at
> > http://www.pathname.com/~corpus/NET.age ).
> >
> > All these messages were received less than 1 month ago, and are taken from
> > 5 people's hand-classified corpora.
> >
> > SPF records passing HELO strings: 4.98% of spam, 13.29% of ham
> > SPF records passing the MAIL FROM: 3.72% spam, 18.90% of ham
> >
> > So it certainly looks like that statement is untrue.
>
> Err, no:
>
> OVERALL% SPAM% HAM% S/O RANK SCORE NAME:0-1
> OVERALL% SPAM% HAM% S/O RANK SCORE NAME:1-3
> OVERALL% SPAM% HAM% S/O RANK SCORE NAME:3-6
>
> 5.377 3.7259 18.9072 0.165 0.23 -0.00 SPF_PASS:0-1
> 1.361 0.9087 3.3508 0.213 0.25 -0.00 SPF_PASS:1-3
> 1.749 0.5116 18.4304 0.027 0.34 -0.00 SPF_PASS:3-6
>
> As you see, spam + ham does not add up to overall. Its not clear what
> these statistics mean, nor how they were calculated. But your
> interpretation is clearly either wrong or at least not supported by the
> page.

Actually, you're wrong there. This is SpamAssassin's "hit-frequencies"
tool output. Those are percentages, not message counts, so simply summing
SPAM%+HAM% will not add up to OVERALL%.

Here's a quick walk through the pertinent parts. (I'm discarding the 1-3
and 3-6 month ranges -- those are old mails so that data isn't very useful
for network tests -- and just concentrate on the 0-1 month range.)

OVERALL% SPAM% HAM% S/O RANK SCORE NAME:0-1
61405 54725 6680 0.891 0.00 0.00 (all messages):0-1

This means that there were 61405 messages mass-checked in total,
with 54725 spams, and 6680 "hams" (non-spam messages).

5.377 3.7259 18.9072 0.165 0.23 -0.00 SPF_PASS:0-1

looking at the SPAM% and HAM% columns, that means that 3.7259% of the spams
checked had SPF_PASS, and 18.9072% of the hams. that means

((3.7259 / 100) * 54725) = 2038.998775

I'd suspect that rounding error means that 2039 spam messages passed the
SPF check, so round to 2039.

((18.9072 / 100) * 6680) = 1263.00096

and 1263 hams passed SPF. Total those, and you get 3302 messages
from the overall corpus passing SPF; to express that as a percentage
of the total overall corpus, in other words "OVERALL%", you compute
(3302 / 61405) * 100 = 5.377.

If you have any more questions on the hit-frequencies format, I'll
be happy to fill you in -- I wrote the tool in question ;)

> FYI, this is the original post:
>
> ---------- Forwarded message ----------
> Date: Thu, 9 Sep 2004 15:18:42 +0200
> From: Markus Stumpf <maex-lists-email-ietf-***@Space.Net>
> To: ietf-***@vpnc.org
> Subject: SPF abused by spammers
>
> Justin Murdock posted this link on the qmail list:
> http://news.bbc.co.uk/1/hi/technology/3631350.stm
> "CipherTrust [...] found that 34% more spam is passing SPF checks than
> legitimate e-mail."

Sure. But this was guaranteed to change over time, and vary depending on
corpus composition. It's pretty radically different now, from where I and
the other SpamAssassin corpus contributors are viewing it.

- --j.

> \Maex
>
> --
> SpaceNet AG | Joseph-Dollinger-Bogen 14 | Fon: [phone]
> Research & Development | D-80807 Muenchen | Fax: +49 (89)
> 32356-299
> "The security, stability and reliability of a computer system is
> reciprocally
> proportional to the amount of vacuity between the ears of the admin"
>
> --Dean

[Dean Anderson]

On Fri, 28 Jan 2005, Justin Mason wrote:

> Actually, you're wrong there. This is SpamAssassin's "hit-frequencies"
> tool output. Those are percentages, not message counts, so simply summing
> SPAM%+HAM% will not add up to OVERALL%.
>
> Here's a quick walk through the pertinent parts. (I'm discarding the 1-3
> and 3-6 month ranges -- those are old mails so that data isn't very useful
> for network tests -- and just concentrate on the 0-1 month range.)
>
> OVERALL% SPAM% HAM% S/O RANK SCORE NAME:0-1
> 61405 54725 6680 0.891 0.00 0.00 (all messages):0-1
>
> This means that there were 61405 messages mass-checked in total,
> with 54725 spams, and 6680 "hams" (non-spam messages).
>
> 5.377 3.7259 18.9072 0.165 0.23 -0.00 SPF_PASS:0-1
>
> looking at the SPAM% and HAM% columns, that means that 3.7259% of the spams
> checked had SPF_PASS, and 18.9072% of the hams. that means
>
> ((3.7259 / 100) * 54725) = 2038.998775
>
> I'd suspect that rounding error means that 2039 spam messages passed the
> SPF check, so round to 2039.
>
> ((18.9072 / 100) * 6680) = 1263.00096
>
> and 1263 hams passed SPF. Total those, and you get 3302 messages
> from the overall corpus passing SPF; to express that as a percentage
> of the total overall corpus, in other words "OVERALL%", you compute
> (3302 / 61405) * 100 = 5.377.
>
> If you have any more questions on the hit-frequencies format, I'll
> be happy to fill you in -- I wrote the tool in question ;)

Ah. That certainly explains the computation. Thanks. But the particular
stats are from only 4 people, who, being spamassassin contributors,
probably have more SPF-using friends than most people.

> > FYI, this is the original post:
> >
> > ---------- Forwarded message ----------
> > Date: Thu, 9 Sep 2004 15:18:42 +0200
> > From: Markus Stumpf <maex-lists-email-ietf-***@Space.Net>
> > To: ietf-***@vpnc.org
> > Subject: SPF abused by spammers
> >
> > Justin Murdock posted this link on the qmail list:
> > http://news.bbc.co.uk/1/hi/technology/3631350.stm
> > "CipherTrust [...] found that 34% more spam is passing SPF checks than
> > legitimate e-mail."
>
> Sure. But this was guaranteed to change over time, and vary depending on
> corpus composition. It's pretty radically different now, from where I and
> the other SpamAssassin corpus contributors are viewing it.

Everything changes over time.

--
Av8 Internet Prepared to pay a premium for better service?
www.av8.net faster, more reliable, better service
617 344 9000

[Dean Anderson]

On Fri, 28 Jan 2005, Dean Anderson wrote:

> > ((3.7259 / 100) * 54725) = 2038.998775
> >
> > I'd suspect that rounding error means that 2039 spam messages passed the
> > SPF check, so round to 2039.
> >
> > ((18.9072 / 100) * 6680) = 1263.00096
> >
> > and 1263 hams passed SPF. Total those, and you get 3302 messages
> > from the overall corpus passing SPF; to express that as a percentage
> > of the total overall corpus, in other words "OVERALL%", you compute
> > (3302 / 61405) * 100 = 5.377.

BTW, I'd say that 38% of the SPF use was ham, and 62% was spam.
1263 ham / 3302 = ~38%
2039 spam / 3302 = ~62%

So, 24% versus the 34% in September. Either slightly better than
September, or perhaps the sample is too small and skewed to be useful. Or
both. Its still better to block whenever you see SPF.

Still, I'd think the 3.7% of total spam using SPF is still fairly
significant, and probably reflects the relative proportion of genuine
commercial spam to non-commercial spam. It would be interesting to know,
of those spams that pass SPF, how many are CAN-SPAM compliant? How many
of the non-SPF spams were CAN-SPAM compliant? I'd conjecture a strong
correlation.

--Dean

--
Av8 Internet Prepared to pay a premium for better service?
www.av8.net faster, more reliable, better service
617 344 9000

[Justin Mason]

Dean Anderson writes:
> On Fri, 28 Jan 2005, Dean Anderson wrote:
>
> > > ((3.7259 / 100) * 54725) = 2038.998775
> > >
> > > I'd suspect that rounding error means that 2039 spam messages passed the
> > > SPF check, so round to 2039.
> > >
> > > ((18.9072 / 100) * 6680) = 1263.00096
> > >
> > > and 1263 hams passed SPF. Total those, and you get 3302 messages
> > > from the overall corpus passing SPF; to express that as a percentage
> > > of the total overall corpus, in other words "OVERALL%", you compute
> > > (3302 / 61405) * 100 = 5.377.
>
> BTW, I'd say that 38% of the SPF use was ham, and 62% was spam.
> 1263 ham / 3302 = ~38%
> 2039 spam / 3302 = ~62%
>
> So, 24% versus the 34% in September. Either slightly better than
> September, or perhaps the sample is too small and skewed to be useful. Or
> both. Its still better to block whenever you see SPF.

hmm! not sure if that's a good assumption -- it's very much dependent on
the comparative ham:spam ratio a domain would see. This set of corpora is
heavily skewed towards receiving more spam than ham; 87.8% of our messages
being spam.

Let's say those corpora were only receiving a third as much spam as they
currently do, possibly because they were younger email addresses or
whatever. In that case, we'd see 1263 ham, 680 spam (2039/3 ~= 680), in
which case the proportion of SPF-using-spam vs SPF-using-ham would be on
its head: 34% being spam, 65% being ham.

What I'm trying to illustrate here is that it's important to compare
figures using figures that compensate for the comparative ham:spam ratio,
because that varies wildly. Hence, comparing (SPF-bearing-ham / all-ham)
to (SPF-bearing-spam / all-spam) is safer than comparing the message
counts of SPF-bearing-ham and SPF-bearing-spam directly.

> Still, I'd think the 3.7% of total spam using SPF is still fairly
> significant, and probably reflects the relative proportion of genuine
> commercial spam to non-commercial spam. It would be interesting to know,
> of those spams that pass SPF, how many are CAN-SPAM compliant? How many
> of the non-SPF spams were CAN-SPAM compliant? I'd conjecture a strong
> correlation.

now that's something I don't have time to get into, CAN-SPAM compliance
not being something that's easy to automate checking for (more's the
pity).

- --j.

[Dean Anderson]

On Fri, 28 Jan 2005, Justin Mason wrote:

> >
> > BTW, I'd say that 38% of the SPF use was ham, and 62% was spam.
> > 1263 ham / 3302 = ~38%
> > 2039 spam / 3302 = ~62%
> >
> > So, 24% versus the 34% in September. Either slightly better than
> > September, or perhaps the sample is too small and skewed to be useful. Or
> > both. Its still better to block whenever you see SPF.
>
> hmm! not sure if that's a good assumption -- it's very much dependent on
> the comparative ham:spam ratio a domain would see. This set of corpora is
> heavily skewed towards receiving more spam than ham; 87.8% of our messages
> being spam.
>
> Let's say those corpora were only receiving a third as much spam as they
> currently do, possibly because they were younger email addresses or
> whatever. In that case, we'd see 1263 ham, 680 spam (2039/3 ~= 680), in
> which case the proportion of SPF-using-spam vs SPF-using-ham would be on
> its head: 34% being spam, 65% being ham.

Err, no. If there was less total spam, then instead of some 60k messages,
you might have 50k messages. However, very likely, there would still be
3302 messages that had correct SPF records. Your calculation, while a
useful number, adds in all sorts of other noise, not related to SPF. This
sort of misunderstanding is unfortunately, all too frequent.

If you decrease the number of spams by 10k messages (roughly 15%), and
3.7% of total spam has SPF, and all spammers do the same thing, then the
SPF spam number would only decrease by 3.7%. Assuming that, we'd expect
our SPF-using spams to drop by 75 messages. (neglecting variance)

So now we'd have:

1263 ham / 3227 = ~39%
1964 spam / 3227 = ~61%

Barely a difference, yet a roughly 15% change in spam volume.

However, the SPF spammers are not doing the same thing as all other
spammers. We have no reason to think so. So a change in the behavior of
the non-SPF using spammers would _not_ affect the behavior of the
SPF-using spammers, so you'd expect the same 3302 messages as before. Of
course, virus joe-jobbers probably think that by generating more spam,
they alter the statistics of real spammers. That's only true if there is
no way to distinguish joe-jobs from real spam. But of course, we can make
such distinctions with CAN-SPAM. No doubt, this is precisely why the DMA
was behind CAN-SPAM.

Fake spam violates CAN-SPAM: no real product or service, etc. Genuine
spam MOSTLY doesn't: There is a strong incentive for genuine spammers to
comply, and compliance isn't hard for a real company. But it is
practically impossible for fake spammers to comply, because if they had
real products or services, they'd be real spammers, but of course, they
aren't real commercial operations.

SPF added another interesting element to all this, in that genuine
spammers leapt onto the bandwagon early, and essentially added a label to
their spam, a lable that fake spammers can't easilly add at the moment.
Eventually, viruses such will be able to start using ISP relays identified
by SPF, but for now, they can't do that easily, and so don't have SPF
"protection" or labeling. The unexpected segregation created during this
transition period should yield a great deal of insight into genuine spam
and virus activity.

BTW, I noticed that John Levine published a diatribe on the "failure of
CAN-SPAM" on circleid a few days ago. He is wrong again, asserting
incorrectly that CAN-SPAM was meant to outlaw spam. It legalized spam, and
gave a way to distinguish the real (ie DMA member) spammers, from the fake
(and I suspect anti-spam radical) abusers. I suspect that if you go back
to Congress, the DMA will simply push for enforcement of the criminal
provisions of CAN-SPAM, which should reveal the identities of our virus
operators. I wonder who that will reveal, and if it will be another ISP
abuse admin, or radical anti-spammer. I forsee the end of a certain type
of spam, in the same way that open relay abuse ended: The abusers will
just give up. It will be nice if this accounts for 96.3% of all spam.
But, I've digressed enough.

> What I'm trying to illustrate here is that it's important to compare
> figures using figures that compensate for the comparative ham:spam ratio,
> because that varies wildly. Hence, comparing (SPF-bearing-ham / all-ham)
> to (SPF-bearing-spam / all-spam) is safer than comparing the message
> counts of SPF-bearing-ham and SPF-bearing-spam directly.

Safer? Its different. Safer has some other implication thats not clear. I
don't know what "safer" means. It could mean that since 18% of total ham
has SPF records, that perhaps deleting based on the roughly 2 to 1 chance
that "SPF records mean spam", is an "unsafe" bet. More rules to cover the
ham would be important.

However, when you ask the question "what is the ratio of SPF use that is
spam and SPF use that ham?", the Sum of the percents have to add to 100,
otherwise you didn't answer the question. You answered some other
question. Possibly, that question is also useful, but it wasn't the
question asked. The assertion was the spammers jumped on SPF. The percent
SPF use that is spam is 62%. The percent of SPF use that ham is 38%. The
total SPF use is composed of spam and ham. Spam SPF use + Ham SPF use must
equal 100%.

Your corpus supports the Ciphertrust assertion. But you said it didn't.
That was incorrect. The mistake was that you answered a different
question.

> > Still, I'd think the 3.7% of total spam using SPF is still fairly
> > significant, and probably reflects the relative proportion of genuine
> > commercial spam to non-commercial spam. It would be interesting to know,
> > of those spams that pass SPF, how many are CAN-SPAM compliant? How many
> > of the non-SPF spams were CAN-SPAM compliant? I'd conjecture a strong
> > correlation.
>
> now that's something I don't have time to get into, CAN-SPAM compliance
> not being something that's easy to automate checking for (more's the
> pity).

You should have accepted the IEMCC proposal back in 1997. Wallace and co.
proposed everything that's in CAN-SPAM, with the benefit of a special
X-spam header (I think it was called something else, like X-advertisement
or some such). That sort of labeling would have made the problem easy.
But the radicals didn't want to be reasonable at the time--thought they
could end spam by techincal means.

--Dean

> - --j.
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.2.5 (GNU/Linux)
> Comment: Exmh CVS
>
> iD8DBQFB+sdSMJF5cimLx9ARAhvgAKCdDpROw4/yqVxCwOwMipg46uh/LACbBD/6
> HsR9l0+iPdHRG4RDM/zOSmQ=
> =lLX4
> -----END PGP SIGNATURE-----
>
>
>

--
Av8 Internet Prepared to pay a premium for better service?
www.av8.net faster, more reliable, better service
617 344 9000

[Gordon Fecyk]

> If the intent was
> to have the IETF simply rubber-stamp SPF, making that clear up front
> would have saved us all a lot of time.

Sokath, his eyes uncovered!

And in total fairness, s/SPF/SenderID.

--
PGP key (0x0AFA039E): <http://www.pan-am.ca/***@pan-am.ca.asc>
Sometimes it's hard to tell where the game ends and where reality bites,
er, begins. <http://vmyths.com/resource.cfm?id=50&page=1>

[Hallam-Baker, Phillip]

The main point of the Web page seems to be that Domain Keys/IIM does the
whole thing much better.

Like we did not know that already, the point of SPF/Sender-Id is that it
will be a while yet before we have a merged spec for Domain-Keys/IIM we are
still in trial stages. It makes a lot of sense to go forward with the low
hanging fruit authentication that SPF provides.

SPF now includes the HELO check so CSV is utterly redundant, it is an
unhelpful contribution to the debate at this point. The promoters of CSV
have not been working to win friends and influence people. 'nuff said.


For DK to work we need a policy layer, that will almost certainly be linked
to the deployed SPF/Sender-ID standard.


> -----Original Message-----
> From: owner-ietf-***@mail.imc.org
> [mailto:owner-ietf-***@mail.imc.org] On Behalf Of David Woodhouse
> Sent: Monday, January 31, 2005 9:35 AM
> To: ***@metro.cx
> Cc: Douglas Otis; Frank Ellermann; ietf-***@imc.org
> Subject: Re: So here it is one year later...
>
>
>
> On Mon, 2005-01-31 at 08:40 +0100, ***@metro.cx wrote:
> > The forwarding problem is known and information is on spf.pobox.com
> > (which still is the primary source for information about spf). It's
> > not like it is 'the big secret of spf' that there is a forwarding
> > issue. I suppose people read this site before publishing,
> so they must
> > know about the forwarding problem. However, people still
> publish. I'm
> > not going to guess at the motivations of these people, i'm not a
> > pyschologist. Fact is that despite the forwarding issue, people do
> > publish.
>
> The forwarding problem isn't exactly shouted from the
> rooftops on the SPF site, and people rarely actually think
> things through for themselves, unfortunately. If you point
> those _same_ people at something like
> http://david.woodhou.se/why-not-spf.html they > often seem to
> change their mind again and stop publishing records, in my experience.
>
> --
> dwmw2
>

[wayne]

In <***@mou1wnexm05.vcorp.ad.vrsn.com> "Hallam-Baker, Phillip" <***@verisign.com> writes:

> The main point of the Web page seems to be that Domain Keys/IIM does the
> whole thing much better.
>
> Like we did not know that already, [...]

Actually, that is something I don't know.

Despite asking several times on the MASS mailing list, I have yet to
see any data on the false positive rates for DK, IIM, William's DK-IIM
merged system, CSV, or SES. So far, all I've seen is people claiming
that their systems work better than SPF, with no data to back it up.
(Often, there is data to back up the claim that SPF has false
positives, but we *do* know that.)

SPF breaks forwarding unless the sender uses SES, the forwarder uses
SRS, or the receiver uses a whitelist.

DK breaks mailing lists, and from what I can tell from reading MASS,
the DK folks don't see that as a problem. The other crypto systems at
least *try* to not break mailing lists, but it not at all clear how
well they do in practice. (SES looks like it will do the best, but it
requires some sort of call-back to work.)


CSV breaks very little, but no one really seems to care. Despite
having *FAR* more publicity in the 9(?) months since it was first
announced, far fewer people are using its HELO checking than were
using SPF's HELO checking in the first 6 months since it was
announced. Of course, the real growth of SPF didn't happen until
after 6 months since it took that long to get a slushy spec done.


I really like the ideas behind the crypto systems, but they have a
bunch of technical problems with them that haven't been figured out
yet. Considering that the technical problems with both crypto systems
and IP systems have been pretty obvious since the spring of 2003, I'm
not sure that good solutions will be found for either type of system.
It was based on my analysis of the various technical problems will all
systems that I decided that IP based solutions would have the most
potential and therefore I started helping out with SPF. 18 months
later, it still looks like I made the right choice.


So, please, before you go claiming that DK/IIM is "better", provide
some data to back up that claim.


-wayne

[Jim Fenton]

wayne wrote:

>Despite asking several times on the MASS mailing list, I have yet to
>see any data on the false positive rates for DK, IIM, William's DK-IIM
>merged system, CSV, or SES. So far, all I've seen is people claiming
>that their systems work better than SPF, with no data to back it up.
>(Often, there is data to back up the claim that SPF has false
>positives, but we *do* know that.)
>
>
We are starting to get some data on false positives (signature breakage)
for DK and IIM, but since the number of verifying domains at this point
is very small, it would raise more questions than it would answer if we
were to publish it. Success rates depend a lot on what the originating
and terminating domains' MTAs do with their messages, so having data
from a couple of domains is far from representative. It also depends on
things like whether the signer, in the case of DK, uses the optional h=
parameter to sign specific headers.

>SPF breaks forwarding unless the sender uses SES, the forwarder uses
>SRS, or the receiver uses a whitelist.
>
>DK breaks mailing lists, and from what I can tell from reading MASS,
>the DK folks don't see that as a problem. The other crypto systems at
>least *try* to not break mailing lists, but it not at all clear how
>well they do in practice. (SES looks like it will do the best, but it
>requires some sort of call-back to work.)
>
>
The real answer is to have mailing lists re-sign their messages. While
IIM does try not to break mailing lists, and does in some cases, it's
imperfect and intended primarily as a expedient until they do.

If you'd like to discuss this more, let's move to the ietf-mailsig list.

-Jim

[Hector Santos]

In order words, SPF and most so with CSV and DK/IIM presume across the board
compliance.

Thanks for confirming the obvious. The question is, which one is more
plausible in addressing the real industry concerns with minimum
intrafracture changes?

Personally, there are two form of mail integrity concepts:

One that remain outside of transports, and one that independently controlled
by SMTP with the possibility linkage of the two.

The point being is that SMTP (the transport system) should not depend on
SPF, CSV or DK/IIM clients. You either do it across the board or you don't
because ultimately you have to deal with the questions of compatibility and
legacy issues.

In layman terms:

What if my SMTP mail server detects a non-DK/IIM ready message?

You need to answer that question otherwise what is the use of using DK/IIM
if a system still needs to work and be ready for non-DK/IIM messages?

Now of course, it might work well in an exclusive CISCO corporate/enterprise
settings, a big pat on the back, an exciting marketing and promotion largely
based on "hope" I can imagine for your big customer base. But this is not
a standard across the board.

In short, we need to get back to basic and address the real issue - SMTP
with Transport Negotiated security and to satisfy CANSPAM - Topic
identification.

Regardless of the EPV (Endpoint Validation) method used, SMTP "3821" must
be remodeled as an advanced secured system with some level of backward
compatibility. This will allow for the fastest endorsement and adoption.

As a side comment, the failure of MARID was 100% due to the philosophical
conflicts between developers and administrators. I had seen it before when
the last original big public email network "FIDONET" faced the then new
internet migration technical design issues. It is now all deja-vu with the
security issues of the internet. Everyone has half-baked ideas. Everyone is
protecting their turfs especially with the old school people around here,
and not many, except for myself and a few others, are not pointing to the
obvious - Where is the IETF-SMTP input in all this? Where is author of
2821?

In all honesty, I could only hope that a SMTP, POP, IMAP vendor/developer's
working group get together to work out the a Generic/General specifications
for the new transport system - independent of any specific SPF, CSV, DK/IIM
proposals. This will allow the new system to STILL work when new inventors
have new EVP methods. We are doing it backwards based on some half washed
"semi-proprietary" ideas and we wonder there isn't censensus.

I'm out of here.....

Sincerely,

Hector Santos, CTO
Santronics Software, Inc.
http://www.santronics.com
305-431-2846 Cell
305-248-3204 Office


----- Original Message -----
From: "Jim Fenton" <***@cisco.com>
To: "IETF MARID WG" <ietf-***@imc.org>
Sent: Monday, January 31, 2005 3:18 PM
Subject: Re: So here it is one year later...


>
> wayne wrote:
>
> >Despite asking several times on the MASS mailing list, I have yet to
> >see any data on the false positive rates for DK, IIM, William's DK-IIM
> >merged system, CSV, or SES. So far, all I've seen is people claiming
> >that their systems work better than SPF, with no data to back it up.
> >(Often, there is data to back up the claim that SPF has false
> >positives, but we *do* know that.)
> >
> >
> We are starting to get some data on false positives (signature breakage)
> for DK and IIM, but since the number of verifying domains at this point
> is very small, it would raise more questions than it would answer if we
> were to publish it. Success rates depend a lot on what the originating
> and terminating domains' MTAs do with their messages, so having data
> from a couple of domains is far from representative. It also depends on
> things like whether the signer, in the case of DK, uses the optional h=
> parameter to sign specific headers.
>
> >SPF breaks forwarding unless the sender uses SES, the forwarder uses
> >SRS, or the receiver uses a whitelist.
> >
> >DK breaks mailing lists, and from what I can tell from reading MASS,
> >the DK folks don't see that as a problem. The other crypto systems at
> >least *try* to not break mailing lists, but it not at all clear how
> >well they do in practice. (SES looks like it will do the best, but it
> >requires some sort of call-back to work.)
> >
> >
> The real answer is to have mailing lists re-sign their messages. While
> IIM does try not to break mailing lists, and does in some cases, it's
> imperfect and intended primarily as a expedient until they do.
>
> If you'd like to discuss this more, let's move to the ietf-mailsig list.
>
> -Jim
>
>

[Jim Fenton]

I think I'll stick with the "in layman terms" part of the message
because that's most clearly stated:

Hector Santos wrote:

>In layman terms:
>
>What if my SMTP mail server detects a non-DK/IIM ready message?
>
>
IMO, it will be a very long time (if ever) before a "typical" mail
server will be able to take some action such as rejecting a message
based on a missing signature, except possibly when the originating
domain advertises that it signs all of its mail.

>You need to answer that question otherwise what is the use of using DK/IIM
>if a system still needs to work and be ready for non-DK/IIM messages?
>
>
It's another message classification criterion, and when present enables
reputation and accreditation based on who the signer is.

>Now of course, it might work well in an exclusive CISCO corporate/enterprise
>settings, a big pat on the back, an exciting marketing and promotion largely
>based on "hope" I can imagine for your big customer base. But this is not
>a standard across the board.
>
>
I hope it's obvious that's not Cisco's intent. But it should be
possible with standards-based message authentication for anyone to more
accurately whitelist their suppliers, customers, and partners, to make
sure that their messages never get lost.

-Jim

[Hector Santos]

Jim, these are legit questions:

1) Who is the signer? MUA or MSA?

2) How should a MDA handle the following:

2a) A domain with DK/IIM published information but a non-DK/IIM payload?
2b) A non-DK/IIM domain?
2c) Broken DK/IIM message?

3) What is the role of the intermediatary router MTA?

4) What is the relationship of the transport parameters; IP, HELO, MAILFROM
and RCPTO with a DK/IIM ready payload? What upfront logic can be used?

5) Since a MDA requires to view the PAYLOAD to validate the DK/IIM, what is
the policy for POST-SMTP failed DK/IIM results?

6) Finally, how should gateway transformation systems handle DK/IIM payloads
which may get stripped. For example, a console based mail system only needs
two parts:

The main headers:

From:
To:
Subject:
Date:

and the body:

text body

6a) How does this change the mail reader, online and offline? For example,
should the presentation software display some "Red Warning Icon" suggesting
"Possible Invalid Message?"

Thanks

Sincerely,

Hector Santos, CTO
Santronics Software, Inc.
http://www.santronics.com
305-431-2846 Cell
305-248-3204 Office



----- Original Message -----
From: "Jim Fenton" <***@cisco.com>
To: "Hector Santos" <***@santronics.com>
Cc: "IETF MARID WG" <ietf-***@imc.org>
Sent: Tuesday, February 01, 2005 12:57 PM
Subject: Re: So here it is one year later...


>
> I think I'll stick with the "in layman terms" part of the message
> because that's most clearly stated:
>
> Hector Santos wrote:
>
> >In layman terms:
> >
> >What if my SMTP mail server detects a non-DK/IIM ready message?
> >
> >
> IMO, it will be a very long time (if ever) before a "typical" mail
> server will be able to take some action such as rejecting a message
> based on a missing signature, except possibly when the originating
> domain advertises that it signs all of its mail.
>
> >You need to answer that question otherwise what is the use of using
DK/IIM
> >if a system still needs to work and be ready for non-DK/IIM messages?
> >
> >
> It's another message classification criterion, and when present enables
> reputation and accreditation based on who the signer is.
>
> >Now of course, it might work well in an exclusive CISCO
corporate/enterprise
> >settings, a big pat on the back, an exciting marketing and promotion
largely
> >based on "hope" I can imagine for your big customer base. But this is
not
> >a standard across the board.
> >
> >
> I hope it's obvious that's not Cisco's intent. But it should be
> possible with standards-based message authentication for anyone to more
> accurately whitelist their suppliers, customers, and partners, to make
> sure that their messages never get lost.
>
> -Jim
>
>

[Jim Fenton]

This is almost entirely about message signing, so I'm going to at least
copy ietf-mailsig, and we should probably move this thread off ietf-mxcomp.

Hector Santos wrote:

>Jim, these are legit questions:
>
>1) Who is the signer? MUA or MSA?
>
>
MUA, MSA, or some other MTA along the way.

>2) How should a MDA handle the following:
>
>
What the recipient does with the results of message signing are really
up to him/her, but here is what I would probably do:

>2a) A domain with DK/IIM published information but a non-DK/IIM payload?
>
>
If by "published information" you mean that the domain has published a
policy that it signs all messages, then an unsigned message should at
the very least be considered very suspect. I would probably just put it
in the "spam folder".

>2b) A non-DK/IIM domain?
>
>
Do what you do today.

>2c) Broken DK/IIM message?
>
>
Same as an unsigned message. It will be trivial for attackers to create
"broken" messages, so a broken signature is really not an indication of
anything.

>3) What is the role of the intermediatary router MTA?
>
>
If it's just providing a message routing function, it does not need to
be signature-aware. If it is manipulating the message, it may need to
verify and re-sign the message.

>4) What is the relationship of the transport parameters; IP, HELO, MAILFROM
>and RCPTO with a DK/IIM ready payload? What upfront logic can be used?
>
>
No relationship. DK/IIM deal only with the payload, and not SMTP itself
(unlike the first revision of IIM which used MAILFROM).

>5) Since a MDA requires to view the PAYLOAD to validate the DK/IIM, what is
>the policy for POST-SMTP failed DK/IIM results?
>
>
The policy I use is to color code the messages in my MUA using a filter
that looks at the authentication results header.

>6) Finally, how should gateway transformation systems handle DK/IIM payloads
>which may get stripped. For example, a console based mail system only needs
>two parts:
>
>The main headers:
>
> From:
> To:
> Subject:
> Date:
>
>and the body:
>
> text body
>
>
It could do any number of things, like modify the subject line (much as
I hate this) or put a warning line at the beginning of the body.

>6a) How does this change the mail reader, online and offline? For example,
>should the presentation software display some "Red Warning Icon" suggesting
>"Possible Invalid Message?"
>
>
That would be nice, when we can get it. For now, I'm happy with
coloring the summary line in my MUA, but I suspect that MUAs will come
along that can do much more.

-Jim

[Hallam-Baker, Phillip]

> [mailto:owner-ietf-***@mail.imc.org] On Behalf Of wayne

> Despite asking several times on the MASS mailing list, I have
> yet to see any data on the false positive rates for DK, IIM,
> William's DK-IIM merged system, CSV, or SES. So far, all
> I've seen is people claiming that their systems work better
> than SPF, with no data to back it up. (Often, there is data
> to back up the claim that SPF has false positives, but we
> *do* know that.)

They certainly work better in certain circumstances, but at this point I
would not want to publish any data because I don't know how typical those
circumstances and in any case the issue is irrelevant.

DK and SPF have different failure modes. I don't think this is a competition
situation. A system with both schemes is much more effective than either on
its own.

Sure if we started from scratch we would only have one auth scheme. We are
retrofitting a legacy system so expect some pain and duplicated effort.

Both systems can do much better if we present them as a coherent integrated
plan than if we make the mistake of having a standards war. If you think
that is a good idea then Jon Callas and I can tell you just how great the
PGP/SMIME standards war was for both of our companies.


Phill

[wayne]

In <***@mou1wnexm05.vcorp.ad.vrsn.com> "Hallam-Baker, Phillip" <***@verisign.com> writes:

>> [mailto:owner-ietf-***@mail.imc.org] On Behalf Of wayne
>
>> Despite asking several times on the MASS mailing list, I have
>> yet to see any data on the false positive rates for DK, IIM,
>> William's DK-IIM merged system, CSV, or SES. [...]
>
> They certainly work better in certain circumstances, but at this point I
> would not want to publish any data because I don't know how typical those
> circumstances and in any case the issue is irrelevant.

I can certainly believe this.


> DK and SPF have different failure modes. I don't think this is a competition
> situation. A system with both schemes is much more effective than either on
> its own.


I completely, 100% agree and I can not emphisized this enough. I see
things like SPF and crypto systems as complementary systems, not
competing systems. I'm sorry I forgot to mention that.


-wayne

[Hector Santos]

----- Original Message -----
From: "wayne" <***@schlitt.net>
To: "IETF MARID WG" <ietf-***@imc.org>
Sent: Monday, January 31, 2005 3:53 PM
Subject: Re: So here it is one year later...


> I completely, 100% agree and I can not emphisized this enough. I see
> things like SPF and crypto systems as complementary systems, not
> competing systems. I'm sorry I forgot to mention that.

Vendors with a suite of solutions know this already. No one system is good
enough, especially if its all half-baked into a legacy SMTP system.

We made our system "hook" ready at each state of the SMTP state machine.
That way, if someone invents something new tomorrow, its plug in plug.

MARID should of focused on the generic security aspect of the transport
system rather than just to force one or more of the half-based,
semi-proprietary proposals down our throats.

Sincerely,

Hector Santos, CTO
Santronics Software, Inc.
http://www.santronics.com
305-431-2846 Cell
305-248-3204 Office

[william(at)elan.net]

On Mon, 31 Jan 2005, wayne wrote:

> > DK and SPF have different failure modes. I don't think this is a
> > competition situation. A system with both schemes is much more
> > effective than either on its own.
>
> I completely, 100% agree and I can not emphisized this enough. I see
> things like SPF and crypto systems as complementary systems, not
> competing systems. I'm sorry I forgot to mention that.

The are complimentary but they work and protect different parts of email
message. That means each one must be able to work on its own independant
of the other one and authentication should work properly on each layer.
You can not have failure scenario of one system being depdending on the
authentication in another layer - this is just a bad security architecture.

That means if we want to use SPF and mail signatures for anything other
then whitelisting (i.e. to get rid of actual bad messages and find bad
senders), we must find ways to deal with SPF forwarding problems on the
SMTP session layer and must have MASS signatures that work with mail lists.

--
William Leibzon
Elan Networks
***@elan.net

[wayne]

In <Pine.LNX.4.44.0501311413490.27005-***@sokol.elan.net> "william(at)elan.net" <***@elan.net> writes:

> On Mon, 31 Jan 2005, wayne wrote:
>
>> > DK and SPF have different failure modes. I don't think this is a
>> > competition situation. A system with both schemes is much more
>> > effective than either on its own.
>> [snip]
>
> The are complimentary but they work and protect different parts of email
> message. That means each one must be able to work on its own independant
> of the other one and authentication should work properly on each layer.
> You can not have failure scenario of one system being depdending on the
> authentication in another layer - this is just a bad security architecture.

You certainly have a point here and but I think you over state it a
little bit.

If both SPF and the crypto systems (DK/IIM/SES/etc.) agree on the
results, then you have much more reliable whitelisting *or*
blacklisting.

If SPF passes and the crytpo system fails, then that is good evidence
that the email came from a mailing list. Combined with other
evidence, and you can get pretty good results.

If SPF fails and the crypto system passes, then that is good evidence
that the email came from a forwarder. Combined with other evidence,
and you can get pretty good results.


> That means if we want to use SPF and mail signatures for anything other
> then whitelisting (i.e. to get rid of actual bad messages and find bad
> senders), we must find ways to deal with SPF forwarding problems on the
> SMTP session layer and must have MASS signatures that work with mail lists.

Yes, SPF *MUST* continue to work on the forwarding problem so that you
don't have just "good evidence", and the same for the crypto systems.
However, I don't think either kind of system will ever be able to
completely solve their weaknesses.


I still think, and have said so many times over the last year or so,
that these systems can complement each other.


-wayne

[Hector Santos]

Wayne,

A good well-though out system would factor out "mailing list" or other
considerations.

What is a mailing list anyway? It is just a new "agent" into mail stream.

In my view, the real problem is the unfortunate direction where more and
more systems are disrespecting the decades old tradition of not screwing
around with the Mail Integrity.

Sincerely,

Hector Santos, CTO
Santronics Software, Inc.
http://www.santronics.com
305-431-2846 Cell
305-248-3204 Office

----- Original Message -----
From: "wayne" <***@schlitt.net>
To: "IETF MARID WG" <ietf-***@imc.org>
Sent: Tuesday, February 01, 2005 5:02 AM
Subject: Re: So here it is one year later...


>
> In <Pine.LNX.4.44.0501311413490.27005-***@sokol.elan.net>
"william(at)elan.net" <***@elan.net> writes:
>
> > On Mon, 31 Jan 2005, wayne wrote:
> >
> >> > DK and SPF have different failure modes. I don't think this is a
> >> > competition situation. A system with both schemes is much more
> >> > effective than either on its own.
> >> [snip]
> >
> > The are complimentary but they work and protect different parts of email
> > message. That means each one must be able to work on its own independant
> > of the other one and authentication should work properly on each layer.
> > You can not have failure scenario of one system being depdending on the
> > authentication in another layer - this is just a bad security
architecture.
>
> You certainly have a point here and but I think you over state it a
> little bit.
>
> If both SPF and the crypto systems (DK/IIM/SES/etc.) agree on the
> results, then you have much more reliable whitelisting *or*
> blacklisting.
>
> If SPF passes and the crytpo system fails, then that is good evidence
> that the email came from a mailing list. Combined with other
> evidence, and you can get pretty good results.
>
> If SPF fails and the crypto system passes, then that is good evidence
> that the email came from a forwarder. Combined with other evidence,
> and you can get pretty good results.
>
>
> > That means if we want to use SPF and mail signatures for anything other
> > then whitelisting (i.e. to get rid of actual bad messages and find bad
> > senders), we must find ways to deal with SPF forwarding problems on the
> > SMTP session layer and must have MASS signatures that work with mail
lists.
>
> Yes, SPF *MUST* continue to work on the forwarding problem so that you
> don't have just "good evidence", and the same for the crypto systems.
> However, I don't think either kind of system will ever be able to
> completely solve their weaknesses.
>
>
> I still think, and have said so many times over the last year or so,
> that these systems can complement each other.
>
>
> -wayne
>
>

[Hallam-Baker, Phillip]

> [mailto:owner-ietf-***@mail.imc.org] On Behalf Of Douglas Otis

> A well understood problem is not FUD!

You may believe you understand the problem well. I don't understand any of
your explanations and I don't think that is due to a lack of expertise or
ability on my part.

I remain Sir yours, etc.

[m***@brycer.com]

What an amusing thread. Following along, it suggest this mailing list
is secretly a humor list. At the least, it should be obvious why the WG
accomplished nothing. "He's dead, Jim."


Bryce Ryan