Date: Sat, 9 Oct 93 04:30:02 PDT
From: Advanced Amateur Radio Networking Group <tcp-group@ucsd.edu>
Errors-To: TCP-Group-Errors@UCSD.Edu
Reply-To: TCP-Group@UCSD.Edu
Precedence: Bulk
Subject: TCP-Group Digest V93 #262
To: tcp-group-digest
TCP-Group Digest Sat, 9 Oct 93 Volume 93 : Issue 262
Today's Topics:
Found messages (was: lost messages)
SMTP transparency
Send Replies or notes for publication to: <TCP-Group@UCSD.Edu>.
Subscription requests to <TCP-Group-REQUEST@UCSD.Edu>.
Problems you can't solve otherwise to brian@ucsd.edu.
Archives of past issues of the TCP-Group Digest are available
(by FTP only) from UCSD.Edu in directory "mailarchives".
We trust that readers are intelligent enough to realize that all text
herein consists of personal comments and does not represent the official
policies or positions of any party. Your mileage may vary. So there.
----------------------------------------------------------------------
Date: Fri, 08 Oct 1993 14:21:42 +0100
From: Geert Jan de Groot <geertj@ica.philips.nl>
Subject: Found messages (was: lost messages)
To: tcp-group@ucsd.edu
I received several copies, thanks everone. I also received quite some
'would like to read' requests. It's not too long, it is an amazing
archievement and it's fun to read so I'm re-posting it. Enjoy!
==============================================================================
Subject: For your amusement
From: Phil R. Karn
Date: Mon, 4 Dec 89 22:17:20 EST
==============================================================================
> Posted: Mon, Dec 4, 1989 5:14 AM GMT Msg: FGIJ-4109-8426
> From: BMCGWIER
> To: amsat, arrl, tclark, hprice
> Subj: STAR TREK IV PACKETS
Several months ago, Harold Price, NK6K, challenged me to demodulate what he
thought might be HF packets in Star Trek IV. During the scenes where Scotty
is valiantly trying to beam both Chekov and Uhura back from the U.S.S.
Enterprise, where they have been stealing Nuclear vessel high speed photons,
Scotty is having a hard time hearing them. One of the sources of
interference is what appeared to Harold to be HF packet. Always being one
to rise to a challenge, I took on the job of doing some fancy Digital Signal
Processing footwork. Almost from the first I was certain that it must be an
HF packet since my very first demodulator attempt clearly revealed flags
before the start of a frame and end of frame was also clear. I knew it was
HDLC of some variety. Several things impeded the effort, including Scotty's
voice on top of the packets, some SSB from 20 meters was also nearly on top
of the signal. All of this had to be filtered out. I spent an hour of time
on the Cray-2 at work and used the fanciest FSK demodulator I could write
and I finally had noisy baseband signal plotted on paper in front of me. I
did my best to get an integral number of samples per baud as the signal was
very noisy, and though the bits could be made out by eye, I could tell that
it was going to take another hour of Cray-2 time to get the clock recovered
and to make good bit decisions. In a couple of places, HDLC showed me what
were clearly bit errors, and these could be done by eye as well. After the
filtering, and building a demodulator for the badly mistuned signal (it was
almost 900 Hz below `normal'), I took the bits to Phil Karn, KA9Q and he
decoded the NRZI data and proved beyond a shadow of a doubt that it was
indeed an HF amateur radio packet. It was WA8ZCN-0 sending an RR for NR-3
to N6AEZ on 20 meters. I got Bill Harrigill, WA8ZCN on the phone and he
agrees that it was probably him. Thanks Harold for the challenge and Phil
for the help.
Bob N4HY
==============================================================================
Subject: more on Star Trek
From: Phil R. Karn
Date: Mon, 4 Dec 89 22:55:11 EST
==============================================================================
I just posted Bob's note to rec.ham-radio.packet after adding the following
commentary:
[I should comment here that what Bob gave me was 9 pages of hard-copy
waveform plots, representing about 360 bits at 300 baud, or about 1.2
seconds of real time. These waveforms showed the raw, unsliced output of
his software FM demodulator. Using pencil, paper, ASCII chart and a copy
of the AX.25 protocol spec, I first recovered the bit clock using a
manual method similar in principle to that used in the WA4DSY 56k modem
demodulator. That is, I marked off sampling points where the waveform
slope went rapidly through zero, ie., at the center of an alternating
mark - space - mark or space - mark - space sequence. From these I could
extrapolate the proper sampling points on the other nearby bits
consisting of mark-mark or space-space sequences.
Then I was ready to "slice" the data into digital bits, perform the NRZI
decoding, and decode the bit stream into an AX.25 frame. The only
really hard part of the job was discovering that Bob had printed page 3
of his waveform plots offset by about 5/8 of a bit time from the other
pages; this really screwed up my clock recovery until I recognized and
corrected for the jump from the A/D sample numbers Bob had printed
underneath each bit cell.
One could actually write a fairly educational piece about the principles
of "maximum likelihood decoding" from this exercise. For example,
knowing that each byte of an AX.25 address field except the last always
had bit zero set to 0, that the callsigns themselves always contained
capital letters, digits or spaces (but never lower case letters,
punctuation or control characters) and so on helped me to determine when
I had probably made an error in decoding a particular bit, or if I was
off in the bit count. Even the online callbook came in handy - I could
determine if a particular callsign was valid and whether the operator
was authorized to transmit packet on HF. All this illustrates the key
role of redundant information in encoding information to tolerate noise
and interference. (And before somebody points out that I just wrote a
flame about the inefficiency of the AX.25 address field encodings, I
should point out that there are *much* more efficient ways to improve
performance through redundancy than by sprinkling zeros into the data in
fixed locations. :-) --KA9Q]
==============================================================================
Subject: Update on Star Trek packet
From: Phil R. Karn
Date: Tue, 5 Dec 89 19:01:26 est
==============================================================================
An update on the "Star Trek IV" packet decoding project:
Last night I finished decoding the second half of the graphs N4HY gave
me. I had made an earlier mistake in the interpretation of the control
field. The actual control field was that of an I frame with N(S)=1,
N(R)=1, P/F=0. The Level 3 protocol ID was F0, which means "no upper
layer protocol" (a very common value). My first attempt at decoding the
text field yielded the following:
>Qt takes 4 program
(At that point, Scotty started talking over the packet, so I was unable
to decode any further.) Since this didn't look quite right, I looked at
the second byte a little closer. I found that if I flipped one
weak-looking bit, I got
>It takes 4 program
which makes much more sense.
As reported earlier, the sender of the packet was WA8ZCN and the
destination N6AEZ. Bob, N4HY, has confirmed that WA8ZCN was indeed very
active on 20m packet until he moved to Arkansas a few years ago, but he
has been unable to reach N6AEZ for confirmation as yet. Bob reports that
Mr. Harrigill (WA8ZCN) was quite tickled to learn of our discovery, and
that he would run right out to rent the videotape again so he could hear
his callsign immortalized in the Final Frontier. :-)
Phil
------------------------------
Date: Fri, 8 Oct 93 09:48:12 EDT
From: "Ross Patterson" <n4yyh@wa2hee.ampr.org>
Subject: SMTP transparency
To: tcp-group@UCSD.EDU
On Fri, 8 Oct 93 01:06:38 EDT, Barry Siegfried wrote:
>(Note that the above doesn't specify just precisely where this first
>dot character comes from, that is, was it originally in the mail, or
>was it put there by the sending SMTP client?)
Sorry, sometimes I presume too much upon the reader. Yes, the first period
was originally in the mail. It's the responsibility of the sender-SMTP
(i.e. the client) to transform "<CRLF>." into "<CRLF>..". I guess I should
have included the preceeding paragraph in RFC 821 ss. 4.5.2:
"1. Before sending a line of mail text the sender-SMTP checks the first
character if the line. If it is a period, one additional period is
inserted at the beginning of the line."
>But why not a stand alone dot? If I deliberately put a stand alone dot,
>or multiple dots at the beginning of lines in my messages, I want them
>delivered to the other end precisely the way I entered them (therein
>the definition of "transparency").
Again, if you put in a standalone period (i.e. "<CRLF>.<CRLF>"), the
sender-SMTP is supposed to send that to the receiver-SMTP (i.e. the server)
as "<CRLF>..<CRLF>", to prevent it from being interpreted as the end-of-mail
sequence (i.e. "<CRLF>.<CRLF>", a standalone period). Thus if a
receiver-SMTP sees a standalone period, it isn't supposed to de-double
it, it's supposed to throw it out and exit DATA mode. Transparency rules.
>Upon checking just precisely what the original code in the SMTP server
>does, I came up with these results:
God, I love science! Gedanken followed by experimentation!
>Actually, Ross was on the right track when he said:
>
>> It would seem the thing to do is to delete:
>>
>> else if (!(*p == '.' && *(p+1) == '\0'))
>> p--;
>>
>> thus removing any period that starts a line (i.e. "<CRLF>.").
>
>And that's probably where he should have stopped, since a careful
>examination of the problem will indicate that the problem is
>actually elsewhere!
My sentiments exactly. I agree that you need to always remove a period
that follows a <CRLF>, because that's what RFC 821 says to do. I almost
sent the note off without the "more conservative" option, but decided to
include it because it protects against SMTP clients that don't double
the periods, like NOS.
>Here's my take on this:
>
>An examination of the corresponding code in the SMTP client indicates
>that a dot is added only to lines that consist of a single dot,
>followed by <CRLF>, and that is where the REAL problem is !
I didn't look at the client, but if it fails to transform "<CRLF>."
into "<CRLF>..", you're right that it needs fixing.
>Definition
>----------
>The SMTP client will always add a dot in any line which begins
>with a dot, and the SMTP server will always delete the first
>character of a line if it is a dot.
>
>This would seem to preserve "transparency".
Not to mention bringing it into conformance with the spec.
>Now of course, until all NOS SMTP clients and servers have these
>mods in them, when a modified SMTP client talks to an unmodified
>SMTP server, most lines beginning with a dot will have an extra
>dot in the received message, and, when an unmodified SMTP client
>talks to a modified SMTP server, most lines beginning with a dot
>will be reduced by one dot in the received message.
Remember also that not all connections to NOS SMTP servers come from NOS
SMTP clients. We went through this on BITNET with a buggy SMTP
implementation about 10 years ago, and there's only one way to do it. Fix
your code as soon as you can. You'll be better off because, except perhaps
in the purely RF environment, there are more non-NOS SMTP systems out there
than there are NOS SMTP systems.
73,
Ross
N4YYH
------------------------------
End of TCP-Group Digest V93 #262
******************************
******************************