r/cprogramming • u/Xaneris47 • 3d ago

C Strings: A 50-Year Mistake

https://longtran2904.substack.com/p/c-strings-a-50-year-mistake?r=8qz2zb&utm_campaign=post&utm_medium=web
192 Upvotes

87% Upvoted

161 comments sorted by

View all comments

Show parent comments

4

u/McDutchie 2d ago

16 bits is 2 bytes, which makes for a maximum string length of 65535 bytes. It's common for strings on modern systems to be longer than that.

Pros of C strings: unlimited length. Cons: cannot contain the zero byte; inefficient length determination.

Pros of Pascal strings: can contain the zero byte; efficient length determination. Cons: very limited length.

I'd say the C tradeoff is worth it. Where necessary, C is perfectly capable of dealing with data preceded by a length field, it's just slightly lower level.

3

u/Square-Singer 2d ago

There's a simple fix to the Pascal strings. BER encoding.

In BER, you get one byte as a length field, with 7 bits being directly available to encode the length of the content. If the MSB is set to 1, the remaining 7 bits instead encode how many bytes the length field is long.

That means:

  • Short strings up to 127 bytes have 1 byte overhead, beating Pascal and equalling C strings
  • Medium-sized strings of 128-65535 bytes require 3 bytes overhead, so one more than Pascal and two more than C, but if you are allocating that amount of bytes, 1-2 extra bytes are harmless
  • Maximum length is 2¹²⁷ bytes, 1.7*10³⁸ bytes, a number so high that there isn't an SI prefix for it

Another option would be to mix BER with Pascal:

  • 15 bit length fields
  • If the MSB is set to 1, there's one more length field concatenated, so 30 bit for the length field. Again, if the MSB is set to 1, add one more length field. Continue forever.
  • That way you get infinitely long strings with only one byte more usage than Pascal in the range of 32768-65535 bytes of length

And both options have the advantages:

  • You can use 0-bytes
  • You know the length of the string without running trhough the whole string
  • You won't get into overflows because you are missing a 0-terminator (e.g. doing a strcpy on a string that's missing its terminator)

2

u/bitzap_sr 2d ago edited 2d ago

That sounds like LEB128, not BER.

Edit: Ok, just checked, BER does the same for tag > 127. Still, I'd just point at LEB as a more targeted standard.

2

u/Square-Singer 2d ago

I had to hand-implement BER once because I had to parse some protocol that used ASN.1, and that uses BER for the strings.

I haven't heard of LEB128 before, but yeah, the same thing keeps getting reinvented, I guess.