Searching \ for '[PIC] 18F4550 in-circuit serial programming circui' in subject line. ()
Make payments with PayPal - it's fast, free and secure! Help us get a faster server
FAQ page: techref.massmind.org/techref/microchip/devprogs.htm?key=programming
Search entire site for: '18F4550 in-circuit serial programming circui'.

Exact match. Not showing close matches.
PICList Thread
'[PIC] 18F4550 in-circuit serial programming circui'
2006\07\12@032718 by Craig

flavicon
face
Hello all, we are trying to avoid the need for a gang programmer for using the 18F4550 in TQFP package.  So we want to add the in-circuit serial option to our PCB so the chips can be programmed in-circuit.  The problem is that there is no part value spcified for the resistor between VDD and the VDD line on the PIC in the reference design in App Note 91016B.

Does anyone know the part value to use?  The part that we need to know the value of is in Figure 1.

Anyone have success with this, and can you tell me the part value?

Thanks all :)

2006\07\12@040455 by Wouter van Ooijen

face picon face
> The problem is that there is
> no part value spcified for the resistor between VDD and the
> VDD line on the PIC in the reference design in App Note 91016B.

Search Results For: 91016B
No documents match your search criteria.

URL?

Wouter van Ooijen

-- -------------------------------------------
Van Ooijen Technische Informatica: http://www.voti.nl
consultancy, development, PICmicro products
docent Hogeschool van Utrecht: http://www.voti.nl/hvu


2006\07\12@043620 by Bob Axtell

face picon face
craig@ivibe.com wrote:
> Hello all, we are trying to avoid the need for a gang programmer for using the 18F4550 in TQFP package.  So we want to add the in-circuit serial option to our PCB so the chips can be programmed in-circuit.  The problem is that there is no part value spcified for the resistor between VDD and the VDD line on the PIC in the reference design in App Note 91016B.
>
> Does anyone know the part value to use?  The part that we need to know the value of is in Figure 1.
>
> Anyone have success with this, and can you tell me the part value?
>
> Thanks all :)
>  
Microchip references say 10K. But I always use 22K, because some
programmers can't drive that much current thru the 10K reliably.

--

I don't think anyone uses a gang-programmer for TQFP because its too
easy to mangle the pins, as a good handler is costly. Sometimes bits get
dropped in the SMT heating chamber, too. You'll be happy with ICSP. I've
used nothing but this for as long as the new flash devices have been
out. You can use very inexpensive programmers at multiple
tech stations.

--Bob

2006\07\12@065943 by olin piclist

face picon face
craig@ivibe.com wrote:
> Hello all, we are trying to avoid the need for a gang programmer for
> using the 18F4550 in TQFP package.  So we want to add the in-circuit
> serial option to our PCB so the chips can be programmed in-circuit.
> The problem is that there is no part value spcifi
>
> Does anyone know the part value to use?  The part that we need to know
> the value of is in Figure 1.

As you can see, your email message makes no sense because you sent each
paragraph as a single long line, and it got truncated at 256 characters
somewhere between your mailer and mine.  All transports and mail clients can
handle messages sent as plain text with lines not exceeding 80 characters.

I can probably help with in-circuit programming, but you've got to fix your
mailer settings first (you should anyway, whether you want my help or not).


******************************************************************
Embed Inc, Littleton Massachusetts, (978) 742-9014.  #1 PIC
consultant in 2004 program year.  http://www.embedinc.com/products

2006\07\12@070632 by Xiaofan Chen

face picon face
On 7/12/06, Wouter van Ooijen <spam_OUTwouterTakeThisOuTspamvoti.nl> wrote:
> > The problem is that there is
> > no part value spcified for the resistor between VDD and the
> > VDD line on the PIC in the reference design in App Note 91016B.
>
> Search Results For: 91016B
> No documents match your search criteria.
>
> URL?
>

TB016:
ww1.microchip.com/downloads/en/AppNotes/91016b.pdf

2006\07\12@071332 by olin piclist

face picon face
Bob Axtell wrote:
> You'll be happy with ICSP. I've
> used nothing but this for as long as the new flash devices have been
> out. You can use very inexpensive programmers at multiple
> tech stations.

You can, but in a production environment where time = money going for the
cheapest programmers is probably not the best overall answer.  I had a
customer a few years ago faced with this problem.  Out of discussions about
the programmer requirements and adding some generalization of my own came
the ProProg (http://www.embedinc.com/products).  It was specifically
designed as the "brick outhouse" solution for production fixtures.  It's
still relatively small and has holes in the corners for mounting.  It's also
1/3 the price of a Microchip PM3 with much less bulk and clunkiness.

By the way, that customer has meanwhile bought over a dozen of them as they
create more fixtures and have people that need a ICSP programmer on their
desks.


******************************************************************
Embed Inc, Littleton Massachusetts, (978) 742-9014.  #1 PIC
consultant in 2004 program year.  http://www.embedinc.com/products

2006\07\12@092930 by Wouter van Ooijen

face picon face
> The problem is that there is
> no part value spcified for the resistor between VDD and the
> VDD line on the PIC in the reference design in App Note 91016B.
>
> Microchip references say 10K. But I always use 22K, because some
> programmers can't drive that much current thru the 10K reliably.

I think that's the MCLR-Vdd resistor, the OP asked about the Vdd-Vdd
resistor, which is less easy to determine.

> TB016: http://ww1.microchip.com/downloads/en/AppNotes/91016b.pdf

thanks Xiaofan :)

This resistor
- prevents accidental shorting of the power (when the rest of the
circuit is powered) when someone does something wrong (I am not sure
that is the intention, but it is a welcome side-effect)
- makes it possible for the programmer to short the PICs power, for
those PICs that use/require the Vpp-before-Vdd sequence (while the rest
of the circuit is powered), and/or for the programmer to vary the PICs
Vdd, for multi-level verification (aka production programming), all fast
enough to meet the programming specs. Note that there will likely be a
big capacitor on the Vdd (upper Vdd, not the PICs Vdd), which does not
exactly help the programmer circuit to create the required Vdd rise
time.

This resistor must be low enough to cause no trouble (due to voltage
drop due to the PICs current consumption) when the PIC is doing its
intended job, and must be high enough to allow the programmer to do its
thing. So now you know why no value is specified: it depends very much
on your particular situation.

My Wisp628 programmer (not a 'production' grade programmer) uses a
different (some would - rightly - say: brute) method to handle the PICs
Vdd for in-circuit programming: it assumes the target circuit is powered
by a short-circuit protected 5V power with a fast recovery (read: a 7805
or somthing similar) and simply shorts this power for a very brief time.
Probably not the best solution under all circumstances, but maybe worth
considering.

Wouter van Ooijen

-- -------------------------------------------
Van Ooijen Technische Informatica: http://www.voti.nl
consultancy, development, PICmicro products
docent Hogeschool van Utrecht: http://www.voti.nl/hvu


2006\07\12@141123 by Bob Axtell

face picon face
Olin Lathrop wrote:
{Quote hidden}

I never considered your programmers expensive. Olin. I think they are
priced correctly for the quality
and support.

--Bob

2006\07\14@153219 by Hector Martin [PIClist] n/a

flavicon
face
Olin Lathrop wrote:
> .....craigKILLspamspam@spam@ivibe.com wrote:
>> Hello all, we are trying to avoid the need for a gang programmer for
>> using the 18F4550 in TQFP package.  So we want to add the in-circuit
>> serial option to our PCB so the chips can be programmed in-circuit.
>> The problem is that there is no part value spcifi
>>
>> Does anyone know the part value to use?  The part that we need to know
>> the value of is in Figure 1.
>
> As you can see, your email message makes no sense because you sent each
> paragraph as a single long line, and it got truncated at 256 characters
> somewhere between your mailer and mine.  All transports and mail clients can
> handle messages sent as plain text with lines not exceeding 80 characters.
I would've thought any recent mailer could handle that. It certainly got
fine to me (I can read the whole line), although I don't know what the
standard requires these days. It's still a good practice sending them
wrapped at 80 chars though.

If anyone cares, my MTA is Postfix, and the message does something like
Internet->Postfix->Maildrop->[maildir storage]->Courier-IMAP. Works fine
here.

--
Hector Martin (hectorspamKILLspammarcansoft.com)
Public Key: http://www.marcansoft.com/hector.asc

2006\07\14@173153 by olin piclist

face picon face
Hector Martin [PIClist] wrote:
> I would've thought any recent mailer could handle that.

We keep having this discussing, and I don't know why it's so hard to
understand that IT'S NOT THE MAILER.  I and others have explained this a
bunch of time.  Lines are getting truncated somewhere between the sending
mailer and the receiving mailer.

> It's still a good practice sending them
> wrapped at 80 chars though.

Exactly.  I don't know why that seems so hard to understand for too many
people either.


******************************************************************
Embed Inc, Littleton Massachusetts, (978) 742-9014.  #1 PIC
consultant in 2004 program year.  http://www.embedinc.com/products

2006\07\14@182239 by Hector Martin [PIClist] n/a

flavicon
face
Olin Lathrop wrote:
> Hector Martin [PIClist] wrote:
>> I would've thought any recent mailer could handle that.
>
> We keep having this discussing, and I don't know why it's so hard to
> understand that IT'S NOT THE MAILER.  I and others have explained this a
> bunch of time.  Lines are getting truncated somewhere between the sending
> mailer and the receiving mailer.

So replace mailer with MTA. Fact is, either one of the sender/receiver
mailers, or a server in between (either SMTP MTA or the destination
IMAP/POP server), can't handle them, which I think is quite bad.

>
>> It's still a good practice sending them
>> wrapped at 80 chars though.
>
> Exactly.  I don't know why that seems so hard to understand for too many
> people either.
Agreed. Long lines DO cause problems sometimes, although it's still
pretty bad for an MTA somewhere to be so old/crappy that it can't handle
them.

So, yeah, people should wrap lines at 80 chars for safety, but that is
no excuse for servers being unable to handle longer lines safely. We
should have both, for extra safety.

By the way, I'm fairly sure the culprit is one of your provider's
servers, Olin, since mail usually gets routed straight from the endpoint
source server to the endpoint target server (and then to your inbox),
and it got fine to me, so it isn't the PICList's server's fault (and
mail will get straight from there to the initial server that handles
your domain's MX, if I'm not mistaken, so it has to be on your side).
It's quite possible that it's out of your control, but you might want to
bug them about it since they really should fix it.


--
Hector Martin (.....hectorKILLspamspam.....marcansoft.com)
Public Key: http://www.marcansoft.com/hector.asc

2006\07\14@204631 by Jan-Erik Soderholm

face picon face
Hector Martin [PIClist] wrote :

> By the way, I'm fairly sure the culprit is one of your provider's
> servers, Olin, since mail usually gets routed straight from
> the endpoint source server to the endpoint target server (and
> then to your inbox),

I'm not sure what you mean with "gets routed straight", but
the mail where you wrote the text I quoted above, was first
routed 3 times between your client and the MIT list-server,
then another 6 times from the MIT list-server until it reached
my Exchange mailbox.

A lot of servers there that could interfere.

B.t.w, 80 chars is not *always* enough, I usualy
try to keep it within 70 or so.

Jan-Erik.



2006\07\15@102045 by olin piclist

face picon face
Hector Martin [PIClist] wrote:
> So, yeah, people should wrap lines at 80 chars for safety, but that is
> no excuse for servers being unable to handle longer lines safely.

It appears it is handling up to 256 characters per line.  That is quite
safely beyond 80 or 132.  Many servers will have some limit, although
perhaps not as low as 256.  But that still means sending arbitrarily long
lines is a really bad idea.


******************************************************************
Embed Inc, Littleton Massachusetts, (978) 742-9014.  #1 PIC
consultant in 2004 program year.  http://www.embedinc.com/products

2006\07\15@103410 by olin piclist

face picon face
Jan-Erik Soderholm wrote:
> B.t.w, 80 chars is not *always* enough,

Really!!?

Can you give an example where a 80 character line gets corrupted or
mishandled?


******************************************************************
Embed Inc, Littleton Massachusetts, (978) 742-9014.  #1 PIC
consultant in 2004 program year.  http://www.embedinc.com/products

2006\07\15@134126 by Jan-Erik Soderholm

face picon face
Olin Lathrop wrote :

> Really!!?
>
> Can you give an example where a 80 character line gets corrupted or
> mishandled?

Sure.

Here is one example of (part of) a mail which has a encoded ZIP
file in it. The ZIP file was encoded with a tool that creates 80 char
lines. The 2'nd example is what it should look like.

I'm not sure if the lines was "corrupted" or "mishandled", but they
was split on multiple lines at least.

The tool is now changed to produce 70 char lines, and we don't get
this
"problem" now. Best looked at using a fixed space font, of course.

I can not go into to many details, but this mail was sent from a site
of a major US electronic outsourcing contractor to a server at a
major Swedish telecom company starting on "E"...

Regards,
Jan-Erik.



Encoding of file DISK_M2T:[TMP]TRCY2006071518501000.ZIP;1
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
(@@QIM5R_5$LT)3VT5DTfQ%TC%%L0@#M0eSL5DCN5@SL0@CL.(URP-
SLaam@C@5ag@DP@@Dah@HA
ag@H
B@$FQA5Cb@Dam@+GCb@B<??ag@B@@Hah@Hb@m@PM1AB@5DGan@H@TKM@ATc@m@@FY9;4(8
(7-=AF
b@,d
R@@mAc@f2MVg2@CL6@3M1TSL8TCL1@CL082e!YW3dG#"$eEEDE5W+"gG(L"_%UUO3)
GOIPBZF-?7
J:IF



Encoding of file DISK_M2T:[TMP]TRCY2006071518501000.ZIP;1
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
(@@QIM5R_5$LT)3VT5DTfQ%TC%%L0@#M0eSL5DCN5@SL0@CL.(URP-
SLaam@C@5ag@DP@@Dah@HAag@H
B@$FQA5Cb@Dam@+GCb@B<??ag@B@@Hah@Hb@m@PM1AB@5DGan@H@TKM@ATc@m@@FY9;4(8
(7-=AFb@,d
R@@mAc@f2MVg2@CL6@3M1TSL8TCL1@CL082e!YW3dG#"$eEEDE5W+"gG(L"_%UUO3)
GOIPBZF-?7J:IF



2006\07\15@135202 by Jan-Erik Soderholm

face picon face
Replying to myself...

Well, now the lines got a second split from what I sent... :-(

In my first example, line 1 and 2 should be on one line, the line
with "ag@H" was the original split part. The same for the other
two tripplets of line.

In the second example there should be 3 full 80 char lines.

Anyway, I guess this mail was another example in itself... :-)

Regards,
Jan-Erik.



> {Original Message removed}

2006\07\15@172914 by Tamas Rudnai

face picon face
Some mailer splits messages up (I can't remember exactly why and what was
the size, but it was around 70 chars). Most e-mail client uses some
encodings to get around so they encode it when sending and decode it when
receiving and because it is a standard all e-mail client will handle it.

Anyway, if you know that your message is encoded in 80 chars, then just drop
all CRLFs and reconstruct it. Or you can use those encodings
(quoted-printable or Base64 or uuencode for example). These work pretty well
in all MTAs and clients.

Regards,
Tamas



On 15/07/06, Jan-Erik Soderholm <EraseMEjan-erik.soderholmspam_OUTspamTakeThisOuTtelia.com> wrote:
{Quote hidden}

> > {Original Message removed}

2006\07\15@181749 by Jan-Erik Soderholm

face picon face
Tamas Rudnai wrote :

> Most e-mail client uses some encodings to get around so they
> encode it when sending and decode it when receiving and
> because it is a standard all e-mail client will handle it.

In *this* case both sender and reveiver are batch jobs
running on OpenVMS ("VMS", "VAX/VMS", whatever) systems.

There are no such thing as a "standard e-mail client" here. The
content is generated as a plain textfile and sent using a
mail-command from the batch script.

> Or you can use those encodings  (quoted-printable or
> Base64 or uuencode for example). These work pretty well
> in all MTAs and clients.

Yes, tool used is basicly a Base64 encode/decode tool.
And since I changed the line lenght from 80 to 70, it
all works mutch better now.

Anyway, my point was that 80 char lines *do* get mangled
by smtp servers sometimes.

This system handles several 1000's of mails a day and
works just fine otherwise.

Jan-Erik.



2006\07\15@183947 by Gerhard Fiedler

picon face
Olin Lathrop wrote:

> It appears it is handling up to 256 characters per line.  That is quite
> safely beyond 80 or 132.  Many servers will have some limit, although
> perhaps not as low as 256.  But that still means sending arbitrarily
> long lines is a really bad idea.

(I tried to change the tag and subject of this thread, but it seems nobody
is following my lead, so I follow the pack and post this back on the PIC
tag with the 18F4550 subject :)


What I never understood is why mail servers have to peek into my message
lines. They shouldn't really care about what is there, where the CRs and/or
LFs are -- they should just pass it on, please, the way it was sent...

Does anybody know why mail servers look for line ends in messages? What are
they trying to find?

Gerhard

2006\07\15@185648 by Jan-Erik Soderholm

face picon face
Gerhard Fiedler wrote :

> What I never understood is why mail servers have to peek into
> my message lines. They shouldn't really care about what is
> there, where the CRs and/or LFs are -- they should just pass
> it on, please, the way it was sent...
> Does anybody know why mail servers look for line ends in
> messages? What are they trying to find?

Since the smtp protocol is *text-based* (that is works by
sending *lines* using the printable part of the US-ASCII
character set), there have to be some standard.

It's not a full 8-bit binary "channel" where you can send
just about anything.

Since the smtp protocol is based on sending *lines*, I guess
the servers *have* to listen for EOL's. How would they
keep in synk with each other if not ?

Remember, this standard was set when everyone had
a 24 x 80 char VT100 (or compatible) on their desks. Noone had
any need to send anything > 80 chars in lenght. So it's was just
logical for the smtp servers to either "cut" or "split" lines at
80 chars.

Later came different encoding systems to enable other files
then "plain text" to be transferd. Such as uuencode and
Base64. MIME and so on, but that was long after the base
smtp protocol was decided.

Jan-Erik.



2006\07\15@190629 by olin piclist

face picon face
Gerhard Fiedler wrote:
> Does anybody know why mail servers look for line ends in messages?
> What are they trying to find?

Both SMTP and POP3 are text base protocols, and therefore work on *lines* of
text.  Most implementations read a line into some fixed size buffer, and
truncate if the input line exceeds the buffer size.  Frankly, truncating at
256 characters when you really should be sending lines no longer than 80 or
possible 132 doesn't seem so unreasonable to me.

Modern programs written when a few Mb isn't of much consequence might
reserver 1024 or more for a line.  If you really wanted to do it "right",
you would dynamically allocate a string as it comes in.  That's a lot more
trouble than most (all ?) SMTP server authors want to bother with,
especially since sending really long lines is silly in the first place.

I've always thought that the list server should automatically reject any
message with lines exceeding 256 characters, using HTML, base64 or other
fancy encoding, and that contained a previous footer added by the server
(indicating someone too lazy to properly trim a reply).  However I didn't
write the list server, so we have to put up with the occasional obnoxiously
formatted message.


******************************************************************
Embed Inc, Littleton Massachusetts, (978) 742-9014.  #1 PIC
consultant in 2004 program year.  http://www.embedinc.com/products

2006\07\16@102841 by Gerhard Fiedler

picon face
Olin Lathrop wrote:

>> Does anybody know why mail servers look for line ends in messages?
>> What are they trying to find?
>
> Both SMTP and POP3 are text base protocols,

... and Jan-Erik Soderholm wrote:

> Since the smtp protocol is *text-based* (that is works by sending *lines*
> using the printable part of the US-ASCII character set), there have to
> be some standard.

Agreed. If I'm not mistaken, the standard for SMTP is described in RFC 2821
http://www.ietf.org/rfc/rfc2821.txt. When reading your messages, I wonder
whether we're actually talking about the same SMTP...

> and therefore work on *lines* of text.  

Right. Based on the current standard for SMTP (and the preceding RFC 821,
in place since 1982), that's lines with up to 1000 characters per content
line (4.5.3.1 "text line"). A server that doesn't do that is not quite an
SMTP server. If we want to be able to work with the internet, we /need/
standards; I think that was what you were saying, and I agree
wholeheartedly. I think we should bug those that use not compliant servers
on the network and try to get them to adhere to the existing standards.
That's for the benefit of everybody. Maybe the mailer that sent out that
360 character line was actually designed to spec and would not send out
lines longer than 1000 characters?

The standard also says "To the maximum extent possible, implementation
techniques which impose no limits on the length of these objects should be
used." I read that to say that if there's not a good /technical/ reason for
imposing line length limits, it shouldn't be done.


And so far, I still fail to see a technical reason for limiting the line
length inside the message content part. The message content of a message
transmitted through the SMTP protocol is everything that follows the DATA
command up to the "<CRLF>.<CRLF>" sequence that indicates "end of message".
That's 5 characters the server needs to look for... there's no need for any
buffer to parse a message for that, just a bit of state machine logic (or
some other means).

I also don't see any technical reason to handle the content between start
and end as lines. The argument that it is a "text-based" protocol doesn't
really explain anything. The content part has clear start and end markers,
and there's no need for line-based handling. (There's the requirement to
insert an additional '.' after a "<CRLF>." sequence that's not followed by
a '<CR>'; but again, there's no humongous buffer necessary to do that, and
no need to read the data line-by-line, and no need to truncate any lines.)


> Most implementations read a line into some fixed size buffer, and
> truncate if the input line exceeds the buffer size.  

Well... bad implementation then (according to the standard), with an
arbitrary limitation (and a non-compliant one if that's less than 1000
chars), it seems to me, and not a technical reason. Is there any reason why
they couldn't read the characters (no matter whether CR/LF or other) into
the buffer until the buffer is full, do with the buffer content whatever
needs to be done, and continue reading the following characters? I'm sure
you've done something like that on systems much smaller than typical SMTP
servers.


> Frankly, truncating at 256 characters when you really should be sending
> lines no longer than 80 or possible 132 doesn't seem so unreasonable to
> me.

Well... you like it 80 chars wide, so that's why you find that reasonable.
We could reason without reason all time about this, but most text
processing has long ago abandoned fixed line widths and moved on to
free-flowing paragraphs -- for good reasons. And most email clients go to
all sorts of lengths to implement "solutions" to a problem that shouldn't
be one in the first place  (like "re-flowing" oddly broken text, for
example).

> Modern programs written when a few Mb isn't of much consequence might
> reserver 1024 or more for a line.  If you really wanted to do it "right",
> you would dynamically allocate a string as it comes in.

I don't quite agree. Why do you think in terms of "line" when talking about
message content? What is the technical reason to process message content
line by line? It doesn't have to be parsed, other than for the "<CRLF>."
and "<CRLF>.<CRLF>" sequences. Probably nobody on this list would need a
1024 character buffer to find a 3 or 5 character sequence in an arbitrary
character stream.


> I've always thought that the list server should automatically reject any
> message with lines exceeding 256 characters,

We had discussions about standards before, and it seems we have a different
attitude towards standards. I tend to think that when there's a standard,
especially an international one, it's usually a good thing to try to adhere
to it if there's not a special reason not to. Same with SMTP... it plain
and simply defines 1000 characters as supported line length. (It's actually
1001 characters for the transmitted data including the trailing CRLF and an
optional leading dot inserted by the mailer.)

So IMO any SMTP server that truncates after 256 characters should get a
compliance update. (Is 20+ years enough to get something like that
straight?) Considering human nature and the implementation inertia theorem
that says that every implementation in place stays the way it is, broken as
it may be, until enough people bug the responsible maintainer for enough
time, that's probably the way to go :)  And it's not that there would be a
shortage of compliant (and free) servers.

I, at least, am glad that the list server seems to try to be standard
compliant :)


> using HTML, base64 or other fancy encoding,

Many of the printable characters in widespread use are outside the 7-bit
ASCII set and need encoding for transmitting as 7-bit ASCII. There are
solid and widely accepted and implemented internet standards for that in
place. Why create a non-compliant bubble?

Gerhard

2006\07\16@105940 by Jan-Erik Soderholm

face picon face
Another thing...

Gerhard,

When quoting, one should be carefully so each quote
is refered to the right poster...

I think 2 of your quotes was from me, the othes
from Olin (I think so, at least)...

Regards,
Jan-Erik.




2006\07\16@114028 by Gerhard Fiedler

picon face
Jan-Erik Soderholm wrote:

> When quoting, one should be carefully so each quote is refered to the
> right poster...
>
> I think 2 of your quotes was from me, the othes from Olin (I think so,
> at least)...

Yes; sorry if that came over the wrong way. I thought that since both you
and Olin were talking about basically the same thing, I might respond in
one message. I will try to attribute the quotes less ambiguously if I do
that again.

For the record: the first quote was Olin's and the second quote was yours
(both correctly attributed, I think). The rest was all from Olin, and that
was not clearly marked.

Gerhard

2006\07\16@122854 by olin piclist

face picon face
Gerhard Fiedler wrote:
> Right. Based on the current standard for SMTP (and the preceding RFC
> 821, in place since 1982), that's lines with up to 1000 characters
> per content line (4.5.3.1 "text line").

OK, I couldn't remember what the minimum guaranteed line length is.  So it's
1000 instead of 256, but you still have the same issue.  Actually it is
probably the POP3 server that truncated the lines, not a SMTP server.  I
think the POP3 standard does not mention a specific line length, so
implementers can and apparently did chose different values.

> The standard also says "To the maximum extent possible, implementation
> techniques which impose no limits on the length of these objects
> should be used." I read that to say that if there's not a good
> /technical/ reason for imposing line length limits, it shouldn't be
> done.

Or if it's more hassle.  There is no excuse for rediculously long lines, and
reading lines into a fixed size buffer is by far the easiest thing to do.  I
would look at the standard that says 1000 and probably type 1000 into the
code.  Compliant is compliant.  "Encourage" is really no standard at all and
therefore meaningless.  It doesn't do any good for a mailer to know that
larger than 1000 character lines *might* get thru.

> And so far, I still fail to see a technical reason for limiting the
> line length inside the message content part. The message content of a
> message transmitted through the SMTP protocol is everything that
> follows the DATA command up to the "<CRLF>.<CRLF>" sequence that
> indicates "end of message". That's 5 characters the server needs to
> look for... there's no need for any buffer to parse a message for
> that, just a bit of state machine logic (or some other means).

You've obviously never written a SMTP server.  Good software is structured
in layers.  Since SMTP is a text based protocol, you have low level routines
that read and write lines of text.  SMTP is also a store and forward
protocol.  Most likely the SMTP server will write the message to a file for
later processing and forwarding, and there is good reason to make this a
text file.  It also may interpret some of the header lines.  All this means
you're going to inherently have text line handling routines.  I'd be rather
surprised if there are implementations out there that deal with the message
body data as a special caes of a binary dump.

> I also don't see any technical reason to handle the content between
> start and end as lines. The argument that it is a "text-based"
> protocol doesn't really explain anything. The content part has clear
> start and end markers, and there's no need for line-based handling.

Srictly speaking no need, but it makes the software simpler if it uses the
same text I/O mechanism it already has to use for other parts of the
protocol.  Again, you're forgetting about implementation reality.

>> Most implementations read a line into some fixed size buffer, and
>> truncate if the input line exceeds the buffer size.
>
> Well... bad implementation then

No, probably just about all implementations, and quite reasonably so.

> Same with SMTP... it plain and simply defines 1000 characters as
> supported line length.

Bot POP3 doesn't, which is apparently the real issue here.

> Many of the printable characters in widespread use are outside the
> 7-bit ASCII set

But none of the ones I use and are required for conversing in english, which
is the standard for this list.  Actually with the normal 8 bit encoding you
get all the few special characters used by english speakers in other parts
of the world, like the british pound sterling symbol.


******************************************************************
Embed Inc, Littleton Massachusetts, (978) 742-9014.  #1 PIC
consultant in 2004 program year.  http://www.embedinc.com/products

2006\07\16@134126 by Peter

picon face

On Sat, 15 Jul 2006, Olin Lathrop wrote:

> truncate if the input line exceeds the buffer size.  Frankly, truncating at
> 256 characters when you really should be sending lines no longer than 80 or
> possible 132 doesn't seem so unreasonable to me.

Most RFCs refer to 1000 character maximum line length.

> Modern programs written when a few Mb isn't of much consequence might
> reserver 1024 or more for a line.  If you really wanted to do it "right",
> you would dynamically allocate a string as it comes in.  That's a lot more
> trouble than most (all ?) SMTP server authors want to bother with,
> especially since sending really long lines is silly in the first place.

Most higher level formatting systems do not care about line length.
Incliding html.

> I've always thought that the list server should automatically reject any
> message with lines exceeding 256 characters, using HTML, base64 or other
> fancy encoding, and that contained a previous footer added by the server
> (indicating someone too lazy to properly trim a reply).  However I didn't
> write the list server, so we have to put up with the occasional obnoxiously
> formatted message.

The list server is standards compliant and truncates long lines longer
than 1000 characters to less than that (by inserting newlines I think).
I have audited my archives of the piclist and there are several
offenders who have pushed the line lengths to above 900 characters. In
all cases they were generated by M$ software.

There is no point in being upset with users who set up their software
with manufacturer (M$) defaults, and then end up uncompliant.

Peter

2006\07\16@194433 by Gerhard Fiedler

picon face
Olin Lathrop wrote:

>> Right. Based on the current standard for SMTP (and the preceding RFC
>> 821, in place since 1982), that's lines with up to 1000 characters per
>> content line (4.5.3.1 "text line").
>
> OK, I couldn't remember what the minimum guaranteed line length is.  So
> it's 1000 instead of 256, but you still have the same issue.  

One part of the issue, yes -- some limitation remains. But the other part,
and the main topic, was that a mail client sent out a line with 360
characters, you received it truncated to 256 characters, and you thought
that this was a non-standard message, that truncating it to 256 was
acceptable behavior and that the problem was on the sender's client side.
It is not -- it's a completely standard-compliant behavior of a mail client
to send out a line with 360 characters. Whatever truncated that line was
out of line, standard-wise, so to speak.

Maybe the mail client used to send this line prevents sending lines longer
than 1000 characters and therefore is quite nicely standard-compliant
(probably more so than the server truncating at 256 characters); we don't
know. (AFAIR it was OE, the same you usually use, so you could check on
that.)


> Actually it is probably the POP3 server that truncated the lines, not a
> SMTP server.  I think the POP3 standard does not mention a specific line
> length, so implementers can and apparently did chose different values.

The POP3 standard RFC1939 defines that messages are assumed to conform to
RFC822. RFC822 (and RFC2822, which now supersedes RFC822) defines that
lines must be limited to 998 characters (excluding the trailing CRLF; which
not surprisingly is consistent with SMTP). So I guess it's safe to say that
any POP3 server that truncates standard-conform SMTP messages is not a
standard-conform POP3 server. Which seems to make sense, if you think about
it. After all, the main purpose of a POP3 server is to handle SMTP
messages. Would you really choose a maximum line length for a POP3
implementation that's smaller than the one required for SMTP? I didn't
think so... :)

So if it really was the POP3 server, there's something to fix. And it's
close to home, too, so you know who to talk to.


> I would look at the standard that says 1000 and probably type 1000 into
> the code.  Compliant is compliant.  

Exactly... that's the whole point. On the other side of the coin is written
"not compliant is not compliant" :)


>>> I've always thought that the list server should automatically reject
>>> any message [...] using HTML, base64 or other fancy encoding,
>>
>> Many of the printable characters in widespread use are outside the 7-bit
>> ASCII set and need encoding for transmitting as 7-bit ASCII.
>
> But none of the ones I use ...

Are you seriously saying that because you don't use these characters, the
list server should reject messages using the methods required to transmit
them in a standard-conform way?

> ... and are required for conversing in english, which is the standard for
> this list.  

People may have names with accents which they may want to write correctly,
or may want to use one of the non-ASCII symbols like £ or °. Is there a
good reason to reject a message because it uses the appropriate symbol for
"degree" instead of writing it out?

> Actually with the normal 8 bit encoding you get all the few special
> characters used by english speakers in other parts of the world, like
> the british pound sterling symbol.

Is this really what you are suggesting? Using 8-bit character sets without
encoding into 7-bit ASCII? You are complaining about lines longer than 80
characters, but suggesting the use of 8-bit characters in SMTP
transmissions? Are you serious with this?

SMTP standard requires that all characters in the message content part (the
part that follows the DATA command) be part of the (7-bit) ASCII character
set. Which in turn means that every message that contains characters not in
that set (like £) needs to be transmitted using an appropriate standard
encoding (quoted-printable, base64 etc.)

Since I used two non-ASCII characters in this message (£ and °), this
message of mine will come with content in the ISO-8859-1 charset (see the
Content-Type header) that's encoded for SMTP transmission in 7-bit ASCII
using quoted-printable (see the Content-Transfer-Encoding header).
Perfectly standard-compliant, differently from your suggestion.

Is there a reason why you propose a non-standard behavior for the list
server? Don't you think that we're all better off with a standard-compliant
server (and standard-compliant clients, of course)?

Gerhard

2006\07\16@201824 by olin piclist

face picon face
Gerhard Fiedler wrote:
> you received it truncated to 256 characters, and you
> thought that this was a non-standard message,

I never said that, only that it was a bad idea to send long lines.  Like it
or not, things do truncate lines out there.  You can argue about standards
all you want, but if you want your mail to get thru, use 80 character or
less lines.

> It is not -- it's a completely standard-compliant behavior of a
> mail client

You seem to be confusing "standards compliant" with "good idea".

> The POP3 standard RFC1939 defines that messages are assumed to conform
> to RFC822.

But the line length is defined in RFC 821 (STD 10).

> RFC822 (and RFC2822, which now supersedes RFC822) defines
> that lines must be limited to 998 characters (excluding the trailing
> CRLF;

OK, I wasn't aware of that.  Last time I looked into this, there was no
RFC2822.

> So if it really was the POP3 server, there's something to fix. And it's
> close to home, too, so you know who to talk to.

With some poking around I could probably find and fix it, but then I can't
imagine not having something else more important to do for a long time to
come.

> Are you seriously saying that because you don't use these characters,
> the list server should reject messages using the methods required to
> transmit them in a standard-conform way?

Geesh, calm down Gerhard.  That was just a flip response.  I would love to
enforce responsible email formatting but I know that would never happen,
unless I own the server of course.  Some of the phancy encoding methods are
handled better by clients now, but there are still some message I get that
when I do a reply it switches to some wierd font and generally makes a mess.
It seems this is prevalent for email originating from China.  I can fix it
by saving the raw message, blowing away all the MIME bs in the header, and
sending the message back to myself.


******************************************************************
Embed Inc, Littleton Massachusetts, (978) 742-9014.  #1 PIC
consultant in 2004 program year.  http://www.embedinc.com/products

2006\07\16@211445 by Robert Ammerman

picon face
I always thought that a good way to handle protocols is to be:

1: As permissive as possible
2: As undemanding as possible

In other words, try to handle the worst case from others, but don't push the
envelope yourself.

Bob Ammerman
RAm Systems

{Original Message removed}

2006\07\17@095605 by Sergey Dryga

face picon face
Olin Lathrop <olin_piclist <at> embedinc.com> writes:

<SNIP>
> I would love to
> enforce responsible email formatting but I know that would never happen,
> unless I own the server of course.  Some of the phancy encoding methods are

The responsible email formatting is a very vague term.  One man's responsible
formattin is another man's absolute nightmare.  I, for example, prefer line
length not truncated at 80 chars, but as a single line per paragraph.  
Sometimes I like to read email in smaller window and it will wrap text to 64
chars/line.  If there are newline chars after every 80, it looks ugly and hard
to read.  Maybe instead of forcing an arbitrary formatting it would be more
convenient to let people (or their mail clients) to do any formatting THEY
like?

Sergey
http://beaglerobotics.com

2006\07\17@100742 by olin piclist

face picon face
Sergey Dryga wrote:
> Maybe instead of
> forcing an arbitrary formatting it would be more convenient to let
> people (or their mail clients) to do any formatting THEY like?

That's of course the problem.  What is convenient for one person is annoying
to another.  Obviously in the end everyone does what they want, but you have
to recognize that that causes problems too.


******************************************************************
Embed Inc, Littleton Massachusetts, (978) 742-9014.  #1 PIC
consultant in 2004 program year.  http://www.embedinc.com/products

More... (looser matching)
- Last day of these posts
- In 2006 , 2007 only
- Today
- New search...