Date: Thu, 20 May 93 04:30:13 PDT From: Advanced Amateur Radio Networking Group <tcp-group@ucsd.edu> Errors-To: TCP-Group-Errors@UCSD.Edu Reply-To: TCP-Group@UCSD.Edu Precedence: Bulk Subject: TCP-Group Digest V93 #130 To: tcp-group-digest TCP-Group Digest Thu, 20 May 93 Volume 93 : Issue 130 Today's Topics: Any ***phunnies*** with JNOS1.09??? (3 msgs) BPQ Documenting NetROM Net/Rom standards (2 msgs) New POP client setup NOS support for other drives (2 msgs) On the subject of LZW Phurther Clairification of Phunnies with JNOS1.09 dumping... Protocol Question Re: NOS support for other tcp packet size problem (3 msgs) USING POP Send Replies or notes for publication to: <TCP-Group@UCSD.Edu>. Subscription requests to <TCP-Group-REQUEST@UCSD.Edu>. Problems you can't solve otherwise to brian@ucsd.edu. Archives of past issues of the TCP-Group Digest are available (by FTP only) from UCSD.Edu in directory "mailarchives". We trust that readers are intelligent enough to realize that all text herein consists of personal comments and does not represent the official policies or positions of any party. Your mileage may vary. So there. ---------------------------------------------------------------------- Date: Wed, 19 May 1993 11:56:51 -0400 (EDT) From: MIKEBW@ids.net (Mike Bilow, <MIKEBW@ids.net>) Subject: Any ***phunnies*** with JNOS1.09??? To: ka7oei@uugate.wa7slg.ampr.org, tcp-group@ucsd.edu It is and has been an intentional feature of the NOS memory allocator for some time that it will force open sessions to shut down in an attempt to recover during memory exhaustion. When you reach a point such that coreleft drops below threshold, then the memory allocator enters its "Yellow Alert" state and tries to begin garbage collects. When available memory drops below half of threshold, then the memory allocator enters its "Red Alert" state and starts shutting down active processes in an attempt to recover. This usually results in crashing. -- Mike Bilow, <mikebw@ids.net> (Internet) N1BEE @ WA1PHY.#EMA.MA.USA.NA (AX.25) ------------------------------ Date: Wed, 19 May 1993 19:21:47 -0500 From: miltonm@inetnode.austin.ibm.com (Milton Miller) Subject: Any ***phunnies*** with JNOS1.09??? To: MIKEBW@ids.net In message <930519115652.609e@ids.net>, Mike Bilow writes: > >It is and has been an intentional feature of the NOS memory allocator >for some time that it will force open sessions to shut down in an >attempt to recover during memory exhaustion. Hhuhhhh?? Since when has NOS forced existing sessions to close in order to recover???? Immediately close new sessions, yes. Discard unacknowledged data, yes. But the appilication code decides what to discard or merge, not the kernel. The socket code collapses mbufs if possible, but I it doesn't close sockets. > When you reach a point >such that coreleft drops below threshold, then the memory allocator >enters its "Yellow Alert" state and tries to begin garbage collects. >When available memory drops below half of threshold, then the memory >allocator enters its "Red Alert" state and starts shutting down active >processes in an attempt to recover. This usually results in crashing. I won't dispute that Red Alert usually results in crashing. But the only process close down as a result of low memory that I am aware of is when the servers start a new connection, and there it is an explicit check. Question: Has anyone tried experimenting with the threshold? Are there over-agressive garbage collectors, so that any Yellow alert will result in a crash? Do realize that Garbage collection takes significant work space! I am espically intrested in alert tests since the switch to a seperate alert check process (as opposed to the call from malloc). There was too much stack usage for the old alert code. The more likely reason to get reset connections is Johans TCP Retry code which limits retries to a (too small default IMHO) value. If nos gets this many retries on a tcp socket, it gives up. This was added sometime between 1.04 and 1.07 inclusive. milton -- Milton Miller KB5TKF miltonm@inetnode.austin.ibm.com miltonm@austin.ibm.com These are not official IBM words, they are my words. ------------------------------ Date: Wed, 19 May 1993 20:59:05 -0400 (EDT) From: MIKEBW@ids.net (Mike Bilow, <MIKEBW@ids.net>) Subject: Any ***phunnies*** with JNOS1.09??? To: miltonm@inetnode.austin.ibm.com, tcp-group@ucsd.edu The "Garbstate" value (none, Yellow, or Red) is passed to the respective Gcollect functions in each process. It is up to them to decide what to do in each case; all the memory allocator does is call the functions. If the function are set up to close down processes, they can do so. It is a programming option. The JNOS TCP timeout business is a whole problem of its own. I have been meaning to bring that up in a longer message and try to propose a solution, but I have not had the chance to write up the message. -- Mike Bilow, <mikebw@ids.net> (Internet) N1BEE @ WA1PHY.#EMA.MA.USA.NA (AX.25) ------------------------------ Date: Wed, 19 May 1993 12:08:02 -0400 (EDT) From: MIKEBW@ids.net (Mike Bilow, <MIKEBW@ids.net>) Subject: BPQ To: w6swe@w6swe.tapr.org, tcp-group@ucsd.edu I can't sat if G8BPQ ever does anything which I would consider prohibited. I know that one G8BPQ system talking to another G8BPQ system can detect this and behave slightly differently. Most of my debugging of the NOS netrom code was done against G8BPQ, since it is handy and generally reliable. I don't have any reason to suspect that there are hidden hooks in G8BPQ which would operate to the detriment of other systems, at least not NOS. In everything I looked at, G8BPQ sets don't care fields to zero when talking to NOS netrom. It may be that G8BPQ uses these fields for something else when talking exclusively between G8BPQ systems, but I can't speak to that. One of the problems with netrom as a protocol is that there is no clear point where fields are or are not significant. For example, in TCP, there is an ACK flag that tells you that the contents of the ACK counter field are valid. Most implementations place valid data in the counter field whether or not the ACK flag is set, but that is merely good manners. Netrom has no similar method for gauranteeing the significance of a field, so you are often forced to treat all fields as always significant or to place explicit dummy data in them. It is also very difficult to test these things. Most software elects not to display netrom fields which are treated as not significant, including G8BPQ. I modified the NOS code so that tracing will show fields that are not significant with parentheses around them, but they will be displayed. Previous implementations of NOS netrom either did not show the fields (as in G8BPQ) or showed them without indicating that were to be considered not significant. -- Mike Bilow, <mikebw@ids.net> (Internet) N1BEE @ WA1PHY.#EMA.MA.USA.NA (AX.25) ------------------------------ Date: Thu May 20 11:07:30 1993 From: iiitac@pyr.swan.ac.uk Subject: Documenting NetROM To: tcp-group@ucsd.edu I don't see that the tcp community will gain much from documenting what is a best a just about workable standard like netrom. At most all that should be documented is the tcp/ip over netrom, since for most tcp/ip users that is all that matters. Alan ------------------------------ Date: Wed, 19 May 1993 10:10:59 PDT From: "Jeffrey D. Angus" <jangus@skyld.tele.com> Subject: Net/Rom standards To: tcp-group@ucsd.edu Well, there are several ways to solve the Net/Rom standard problem. 1. Talk to Ron Raikes, WA8DED who wrote the original code. "Ron Raikes, WA8DED" <76337.727@compuserve.com> 2. Talk to Nord><Link and use their source code for the "copy" (no, I don't want no *^&%&^% flames about this) of net/rom called TheNet. No, I don't know who exactly to get in touch with them. If someone else knows I would appreciate a e-mail note back. 3. Publish a standard. Right wrong or whatever, once it's in print and circulated, then it can be codified and corrected. 4. Scream loudly. The correctness of your position, like a civil suit, is dependent on a preponderance of evidence. 73 es GM from Jeff -- J. Angus: jangus@skyld.tele.com -- "Als ik Kan", Gustav Stickley US Mail: PO Box 4425 Carson, CA 90749-4425 1 (310) 324-6080 ------------------------------ Date: Wed, 19 May 1993 20:32:47 -0400 (EDT) From: MIKEBW@ids.net (Mike Bilow, <MIKEBW@ids.net>) Subject: Net/Rom standards To: jangus@skyld.tele.com, tcp-group@ucsd.edu Your basic points are valid. However, I think Ron Raikes, having done the original NET/ROM (TM) implementation, probably wants nothing to do with it at this point. I can't speak for him, but I understand that he was none too pleased at the commercial implications of the NORD><LINK affair. As for TheNet, it now seems to be virtually in the public domain with a dozen different variations, some of them actually incompatible with each other. (NOS also has a lot of people working on it, but the ethos of NOS causes them to share source code and pool effort, which does not appear to be true of TheNet.) This leaves your proposal to actually write a protocol specification for netrom. I considered this, but I am not sure how well received it would be in the mainstream community of netrom protocol implementors, especially if were issued unilaterally by the TCP/IP community. I am also not too sure how far we could get away with standardizing. For example, I doubt that any attempt to define and specify the function of the acknowledgment timers would fly, since no matter what we did, all but one of the existing implementations would be declared non-compliant. At a minimum, a specification document would allow us to standardize on terminology. There are a lot of subtle issues that we could at least document across implementations. For example, G8BPQ decrements the obsolescence counter on a route when the NODES broadcast is sent, locking the timing intervals together, while NOS runs a separate "netrom obsotimer" timer for this purpose. Nevertheless, this threatens to be a horrendous task if the document is to be accurate enough to attain credibility. -- Mike Bilow, <mikebw@ids.net> (Internet) N1BEE @ WA1PHY.#EMA.MA.USA.NA (AX.25) ------------------------------ Date: Wed, 19 May 1993 20:42:28 -0400 (EDT) From: MIKEBW@ids.net (Mike Bilow, <MIKEBW@ids.net>) Subject: New POP client setup To: moerner@almaden.ibm.com, tcp-group@ucsd.edu Before someone tries it and gets confused, let me just correct one minor point in Weo's modified version of Doug's and Karl's POP client setup. The "pop addserver..." command is now used for BOTH POP2 and POP3 clients, not just POP3. The only difference in syntax is that "pop2" or "pop3" are to be specified in the proper field of the command. In other words, Weo's fix (not specifying ".txt") on POP3 also applies to POP2. While it is possible to compile a version using the old POP2 code rather than the new POP2 and POP3 code from my source, none of my executables do this, nor can I see any legitimate reason to build the executable that way. If someone did this, then the old style POP2 client setup would work, and the "pop addserver..." command would be unsupported. Using my 921225v0.85, you can run the POP2 and POP3 servers simultaneously. You can also have multiple POP2 and POP3 client events posted on the list with multiple "pop addserver..." commands in any combination. This should also apply to JNOS, since I believe Johan's POP code was synced to mine around his v1.07 or so. Thanks to Weo for illuminating a previously unknown user interface error. NOS really should generate a message if you make the mistake he did. -- Mike Bilow, <mikebw@ids.net> (Internet) N1BEE @ WA1PHY.#EMA.MA.USA.NA (AX.25) ------------------------------ Date: Wed, 19 May 1993 11:35:25 -0400 (EDT) From: MIKEBW@ids.net (Mike Bilow, <MIKEBW@ids.net>) Subject: NOS support for other drives To: karn@qualcomm.com, tcp-group@ucsd.edu The restriction on the JOIN command is considerably worse under DOS 6. You no loner get the JOIN command with the distribution set of DOS. If you want JOIN, you have to send away for the "Supplemental Utilities" disk from Microsoft, which costs a few dollars for shipping. Of course, you also get other useful ultities in addition to JOIN, such as EDLIN. The word is that you can download JOIN and its friends from the Microsoft BBS, but I haven't bothered to check that. -- Mike Bilow, <mikebw@ids.net> (Internet) N1BEE @ WA1PHY.#EMA.MA.USA.NA (AX.25) ------------------------------ Date: Wed, 19 May 93 17:00:47 EDT From: "John R. Ackermann" <jra@law7.daytonoh.ncr.com> Subject: NOS support for other drives To: Mike Bilow <MIKEBW@ids.net> You (Mike Bilow,) write: > > The restriction on the JOIN command is considerably worse under DOS 6. > You no loner get the JOIN command with the distribution set of DOS. > If you want JOIN, you have to send away for the "Supplemental Utilities" > disk from Microsoft, which costs a few dollars for shipping. Of course, > you also get other useful ultities in addition to JOIN, such as EDLIN. > > The word is that you can download JOIN and its friends from the > Microsoft BBS, but I haven't bothered to check that. The supplement disk is available via ftp from a bunch of places... look for dos6supp.zip (or .exe). John ------------------------------ Date: Thu May 20 11:01:55 1993 From: iiitac@pyr.swan.ac.uk Subject: On the subject of LZW To: tcp-group@ucsd.edu Has anyone tried applying the new gnu zip code to a stream of radio material yet. On big text files I'm getting another 15-20% of compression over compress so on air that kind of compression would be more than welcome. It would also bury for good any potential patent problems with lzw. Alan ------------------------------ Date: Wed, 19 May 93 14:06:11 mdt From: ka7oei@uugate.wa7slg.ampr.org Subject: Phurther Clairification of Phunnies with JNOS1.09 dumping... To: tcp-group@ucsd.edu I may have not been too clear on when JNOS1.09 seems to be dumping. Firstly, we are running it on an AT (DOS loaded high, about 620k or so of free memory, etc.). When is starts, 'coreleft' is about 90k and 'avail' is about 3k. The first activity on a newly-restarted NOS session causes 'coreleft' to shrink rapidly and the memory known by 'avail' rapidly grows. It ***seems*** that the dumping occurs when 'avail' hits zero and NOS grabs more from 'coreleft'. That is, a newly-started NOS will dump people (occasionally) until memory is "stabilized" (about 20-30k or 'coreleft', normally) and then it will behave itself. That occurs if 3 or so people have logged on at once and are doing "normal" things. Occasionally, a surge of activity will draw coreleft down to 200 or so (NOS just seems to 'hang' THAT session for the moment, until memory is released by whatever it is that hogging it at that second) and everything is fine. ONLY then do things like yellow and red (red are rare...) garbage collections occur, as well as alloc failures... In short, the "dumping" seems to occur ONLY when NOS has recently been started... Any "ideas"? de <Clint> KA7OEI "We look for things.... Things that make us go..." *** ------------------------------ Date: Wed, 19 May 93 14:48:15 MDT From: jr@upl.com (J.R. Westmoreland) Subject: Protocol Question To: tcp-group@ucsd.edu When a packet is sent over the radio link using NOS, KA9Q, WAMPES, etc. is that packet a encapsulated IP datagram within AX25? If so, what kind of AX25 packet is sent? Is it a unconnected packet? Is there a description of the way the encapsulation is done? Thanks, J.R. Westmoreland (N7MFF) ------------------------------ Date: Thu, 20 May 1993 08:42:39 +1000 From: Terry Dawson <terryd@extro.ucc.su.OZ.AU> Subject: Re: NOS support for other To: tcpgroup@ucsd.edu >>that being for the drive to be prepended to the filepath as top level >>directory. >>much more elegant in my opinion, and it doesn't break any existing >>software at all. > >For ordinary local DOS drives, you can use the DOS "join" command to >do exactly this. Unfortunately, CD-ROM drivers are implemented as >network drives, and you can't join this (at least under DOS 5.0). >Anybody know if this same restriction exists in 6.0? Absolutely, I've used 'join' quite successfully on a couple of occasions, but my thought was that if such a scheme were employed within NOS such that it performed the necessary translations, then the limitations of 'join' and its genre, with respect to networked and other special or virtual drives would be overcome without exception. To take this one step further, and while it seems a little kludgy, perhaps a 'mount' command could be built into NOS such that you could 'mount' a device:filesystem onto anywhere else. All I guess this would really entail is for NOS to keep a translation record somewhere, and for it to parse each filname it is given, and performing the necessary translations to real drives:subdirectory. If noone else wants to do it, and there is enough demand for it, I'll look at coming up with an implementation. It seems to me that it should be fairly trivial to do. Terry ------------------------------ Date: Wed, 19 May 93 06:40:09 cst From: kf5mg@vnet.IBM.COM Subject: tcp packet size problem To: TCP-Group@UCSD.Edu I've added LZW compression to POP3. It appears to work.... the data looks like compressed data when it is sent to the pop3client and the data looks ok when you view it on the remote box. There's a serious problem though. The LZW compressed data is larger that the uncompressed data. This only occurs when I use the lzw compression with the POP3 sockets. When I use lzw compression with SMTP, the lzw data is smaller. I've noticed that when I do the POP3 data transfer with the non-lzw sockets, the packet sizes are 512 bytes. This is what my TCP MSS is set for. When I turn on the lzw compression, I start getting packet size of 10, 20, 30, 50, 100 bytes... Nothing over 150 bytes. I'm pretty sure that this is why I'm getting more data. Each packet has overhead associated with the lzw compression. Since I have smaller packets, there's more of them and they have more overhead. On SMTP transfers using lzw compression, even the compressed sockets use packets of 500 bytes or more. Any one have any ideas why POP3, lzw compressed sockets would use a lot of really small packets? Thanks. p.s. I'm planning on adding lzw compression to ftp and telnet transfers too. If someone has already done this, please let me know. If anyone else in interested in this, let me know and I'll post my status every so often and make the code diffs available. 73's de Jack - kf5mg AMPRnet - kf5mg@kf5mg.ampr.org - 44.28.0.14 AX25net - kf5mg@kf5mg.#dfw.tx.usa.na - work (817) 962-4409 Internet - kf5mg@vnet.ibm.com - home (817) 488-4386 ------------------------------------------------------------------- | "I am Homer, of Borg...prepare to be assim -- ooo, donuts." | ------------------------------------------------------------------- ------------------------------ Date: Wed, 19 May 93 15:34:20 +0100 From: Mike Chace <mikec@praxis.co.uk> Subject: tcp packet size problem To: Jack Snodgrass <kf5mg@vnet.IBM.COM> >>>>> Regarding tcp packet size problem; kf5mg@vnet.IBM.COM adds: kf5mg> p.s. I'm planning on adding lzw compression to ftp and telnet kf5mg> transfers too. If someone has already done this, please let me kf5mg> know. If anyone else in interested in this, let me know and kf5mg> I'll post my status every so often and make the code diffs kf5mg> available. Jack, WNOS has long implemented compression of telnet sockets both for interactive use and for convers interlinks. The source code for WNOS is available on ucsd.edu in hamradio/packet/tcpip/wnos as far as I remember. 73, Mike (G6DHU) ------------------------------ Date: Wed, 19 May 93 13:22:03 CDT From: andyw@aspen.cray.com (Andy Warner) Subject: tcp packet size problem To: tcp-group@ucsd.edu (TCP Group) Mike wrote: > [...] > Jack, > > WNOS has long implemented compression of telnet sockets both > for interactive use and for convers interlinks. The source code > for WNOS is available on ucsd.edu in hamradio/packet/tcpip/wnos > as far as I remember. Has anyone else thought about adding LZW compression to other TCP implementations (eg SunOS, 386BSD...) ? I strongly disagree with the "NOS-centric" way LZW was added to the socket code, not the application(s), this has the end result of making it useless to anyone who runs something other than NOS.. Now it's a defacto standard, I don't expect much joy trying to change things (anyone for XLZW2 ?). Sigh. -- andyw. N0REN/G1XRL andyw@aspen.cray.com Andy Warner, Cray Research, Inc. (612) 683-5835 ------------------------------ Date: 2 Mar 93 19:45:24 GMT From: (null) Subject: USING POP From: crompton@NADC.NADC.NAVY.MIL (D. Crompton) Newsgroups: mail.tcp-group Subject: USING POP Message-ID: <37525@ucsd.Edu> Date: 16 Jul 91 16:09:54 GMT It seems that some group members missed an earlier message I sent out on POP. I am repeating it here for there benefit. Hope this answers most questions on the usage of POP. Since Doug put this out in 1991 POP3 has become the protocol to use and is part of almost all current nos. As it turns out the method to set up as a POP USER has changed with POP3 and I will add that information to Doug's for a complete statement on using POP. How to use POP in NOS --------------------- The HOST should establish a 'POPUSERS.' file in root with the following format: username:password: username:password: etc. There should be an entry for each user of your POP system. We generally use call letters for both entries. I.E. wa3dsp:wa3dsp: The HOST must also start the pop server 'start pop' which should go in your NOS autoexec. *******************If your NOT using POP3 ************************ Each USER must add the following lines to there autoexec: 'pop mailbox CALL' Where CALL is the name of the mailbox on the host to retrieve mail from. The /spool/mail/CALL.txt file. Usually the users call. 'pop mailhost hostname' This specifies what host to pop from. I.E. 'pop mailhost wa3dsp.ampr.org' 'pop userdata user password' This data should match the data in the hosts /popuser file. I.E. 'pop userdata wa3dsp wa3dsp' 'pop timer 3600' For stations that are on for extended periods and receive there mail via pop from a mailhost this timer must be set to interrogate the host on a regular basis. Alternatively they could do a 'pop kick' to check for mail. Time should be set to probably no less than 1/2 hour on a radio circuit. 'pop kick' This should be entered at the end of your autoexec to check for mail from your mailhost at startup. So the autoexec entries would look like this for USER w3iwi... pop mailbox w3iwi pop mailhost wa3dsp.ampr.org pop userdata w3iwi w3iwi pop timer 3600 pop kick and HOST wa3dsp's autoexec... start pop HOST wa3dsp's popusers. in root.... w3iwi:w3iwi: ******************** If you ARE using POP3 ************* Verify that your using POP3 this way; At the nos prompt type 'pop add' and if your using POP3 you will see: net> pop add net> Usage: popmail addserver <mailserver> [<seconds>] [hh:mm- hh:mm] <protocol> <mailbox> <username> <password> net> Now the way to set up as a USER is to put in autoexec.nos a line that gives nos all the required info. Using the example above: <mailserver> = wa3dsp [<seconds>] = 3600 = 1 hour between pop requests <protocal> = POP3 <mailbox> = w3iwi.txt (what to call the file from the Host) CORRECTION - DO NOT SPECIFY .txt EXTENSION, otherwise pop client will never start. Correct way: <mailbox> = w3iwi <username> = w3iwi <password> = w3iwi So the line you will type in autoexec.nos will be: pop add wa3dsp 3600 POP3 w3iwi.txt w3iwi w3iwi CORRECTION: should be pop add wa3dsp 3600 POP3 w3iwi w3iwi w3iwi Different than what earlier pop used but still simple. A few other points... If a pop users wants mail to be delivered to the host for them to pick up via POP they should enter a 'reply to' field in BM.RC to direct mail to the host and not back to them. POP is a very good service for Amateur Radio. It is especially good when a flood of messages are sent out to all users. This is a condition that often causes crashes on memory marginal systems using SMTP. Also alot of unnecessary traffic is sent out to stations that are not on the air. With POP the user asks for and gets mail. This is naturally a random operation. Lowering channel congestion and NOS memory usage. Mail that is POP'ed from the host is deleted from the /spool/mail directory upon successful transfer. The USER is notified that new mail has arrived at the completion of the entire transfer. One drawback that I notice with POP is that the messages (many could build up) for a user are sent as a group. If the circuit fails with a hard error halfway thru a POP xfer of a message group, no messages are saved at the user end, even though some got thru. It is an all or none with POP. This reminds me of the stupidity of BBS's in this regard. Hopefully users will not let messages build up. I have some users who let the mail build up to 30 or 40K over a few weeks. Doug Karl ------------------------------ Date: Wed, 19 May 93 09:01:18 PDT From: "W. E. Moerner (MOERNER@ALMADEN.IBM.COM)" <moerner@almaden.ibm.com> To: tcp-group@ucsd.edu Message-Id: <051993.090119.moerner@ibm.com> Subject: new POP users BEWARE! As a new POP user with the latest N1BEE 0.85 rev. of the PA0GRI code using POP3, I learned the hard way that you must NOT specify the .txt extension in the command line to set up pop3 on the client machine! If you do, the pop3 client will never start, with no error messages of any kind or any clue as to what is wrong. I appreciate the detailed pop instructions posted earlier here by Doug and Karl, and I reproduce a corrected version of the instructions below. Thanks to Mike Bilow for attempting to keep me sane while I was trying to figure out what was wrong. 73, wn6i ================================================================ CORRECTED by W. E. Moerner 19 May 93 ------------------------------ End of TCP-Group Digest V93 #130 ****************************** ******************************