Resuming [vf]?nprintf after reaching the limit

I have an application which prints strings to a buffer using snprintf and vsnprintf. Currently, if it detects an overflow, it appends a > to the end of the string as a sign that the string was chopped and prints a warning to stderr. I'm trying to find a way to have it resume the string [from where it left off] in another buffer.

If this was using strncpy, it would be easy; I know ho开发者_开发知识库w many bytes were written, and so I can start the next print from *(p+bytes_written); However, with printf, I have two problems; first, the formatting specifiers may take up more or less space in the final string as in the format string, and secondly, my valist may be partially parsed.

Does anyone have an easy-ish solution to this?

EDIT: I should probably clarify that I'm working on an embedded system with limited memory + no dynamic allocation [i.e., I don't want to use dynamic allocation]. I can print messages of 255 bytes, but no more, although I can print as many of those as I want. I don't, however, have the memory to allocate lots of memory on the stack, and my print function needs to be thread-safe, so I can't allocate just one global / static array.

I don't think you can do what you're looking for (other than by the straightforward way of reallocating the buffer to the necessary size and performing the entire operation again).

The reasons you listed are a couple contributors to this, but the real killer is that the formatter might have been in the middle of formatting an argument when it ran out of space, and there's no reasonable way to restart that.

For example, say there's 3 bytes left in the buffer, and the formatter starts working on a "%d" conversion for the value -1234567. It ll put "-1\0" into the buffer then do whatever else it needs to do to return the size of buffer you really need.

In addition to you being able to determine which specifier the formatter was working on, you'd need to be able to figure out that instead of passing in -1234567 on the second round you need to pass in 234567. I defy you to come up with a reasonable way to do that.

Now if there's a real reason you don't want to restart the operation from the top, you probably could wrap the snprintf()/vsnprintf() call with something that breaks down the format string, sending only a single conversion specifier at a time and concatenating that result to the output buffer. You'd have to come up with some way for the wrapper to keep some state across retries so it knows which conversion spec to pick up from.

So maybe it's doable in a sense, but it sure seems like it would be an awful lot of work to avoid the much simpler 'full retry' scheme. I could see maybe (maybe) trying this on a system where you don't have the luxury of dynamically allocating a larger buffer (an embedded system, maybe). In that case, I'd probably argue that what's needed is a much simpler/restricted scope formatter that doesn't have all the flexibility of printf() formatters and can handle retrying (because their scope is more limited).

But, man, I would try very hard to talk some sense into whoever said it was a requirement.

Edit:

Actually, I take some of that back. If you're willing to use a customized version of snprintf() (let's call it snprintf_ex()) I could see this being a relatively simple operation:

int snprintf_ex( char* s, size_t n, size_t skipChars, const char* fmt, ...);

snprintf_ex() (and its companion functions such as vsnprintf()) will format the string into the provided buffer (as usual) but will skip outputting the first skipChars characters.

You could probably rig this up pretty easy using the source from your compiler's library (or using something like Holger Weiss' snprintf()) as a starting point. Using this might look something like:

int bufSize = sizeof(buf);
char* fmt = "some complex format string...";

int needed = snprintf_ex( buf, bufSize, 0, fmt, arg1, arg2, etc, etc2);

if (needed >= bufSize) {
    // dang truncation...

    // do whatever you want with the truncated bits (send to a logger or whatever)

    // format the rest of the string, skipping the bits we already got
    needed = snprintf_ex( buf, bufSize, bufSize - 1, fmt, arg1, arg2, etc, etc2);

    // now the buffer contains the part that was truncated before. Note that 
    //  you'd still need to deal with the possibility that this is truncated yet
    //  again - that's an exercise for the reader, and it's probably trickier to
    //  deal with properly than it might sound...
}

One drawback (that might or might not be acceptable) is that the formatter will do all the formatting work over again from the start - it'll just throw away the first skipChars characters that it comes up with. If I had to use something like this, I'd think that would almost certainly be an acceptable thing (it what happens when someone deals with truncation using the standard snprintf() family of functions).

The C99 functions snprintf() and vsnprintf() both return the number of characters needed to print the whole format string with all the arguments.

If your implementation conforms to C99, you can create an array large enough for your output strings then deal with them as needed.

int chars_needed = snprintf(NULL, 0, fmt_string, v1, v2, v3, ...);
char *buf = malloc(chars_needed + 1);
if (buf) {
  snprintf(buf, chars_needed + 1, fmt_string, v1, v2, v3, ...);
  /* use buf */
  free(buf);
} else {
  /* no memory */
}

If you're on a POSIX-ish system (which I'm guessing you may be since you mentioned threads), one nice solution would be:

First try printing the string to a single buffer with snprintf. If it doesn't overflow, you've saved yourself a lot of work.

If that doesn't work, create a new thread and a pipe (with the pipe() function), fdopen the writing end of the pipe, and use vfprintf to write the string. Have the new thread read from the reading end of the pipe and break the output string into 255-byte messages. Close the pipe and join with the thread after vfprintf returns.