How Do C++ Compilers Merge Identical String Literals

How does compiler (MS Visual C++ 2010) combine identical string literals in different cpp source files? For example, if I have the string literal "hello world\n" in src1.cpp and src2.cpp respectively. The compiled exe file will have only 1 "hello world" string literal probably in the constant/readonly section. Is this task done by the linker?

What I hope to achieve is that I got some modules written in assembly to be used by C++ modules. And these assembly modules contain many long string literal definitions. I know the string literals are identical to some other string literals in the C++ source.开发者_C百科 If I link my assembly generated obj code with the compiler generated obj code, would these string literals be merged by the linker to remove redundant strings as is the case when all modules are in C++?

(Note the following applies only to MSVC)

My first answer was misleading since I thought that the literal merging was magic done by the linker (and so that the /GF flag would only be needed by the linker).

However, that was a mistake. It turns out the linker has little special involvement in merging string literals - what happens is that when the /GF option is given to the compiler, it puts string literals in a "COMDAT" section of the object file with an object name that's based on the contents of the string literal. So the /GF flag is needed for the compile step, not for the link step.

When you use the /GF option, the compiler places each string literal in the object file in a separate section as a COMDAT object. The various COMDAT objects with the same name will be folded by the linker (I'm not exactly sure about the semantics of COMDAT, or what the linker might do if objects with the same name have different data). So a C file that contains

char* another_string = "this is a string";

Will have something like the following in the object file:

SECTION HEADER #3
  .rdata name
       0 physical address
       0 virtual address
      11 size of raw data
     147 file pointer to raw data (00000147 to 00000157)
       0 file pointer to relocation table
       0 file pointer to line numbers
       0 number of relocations
       0 number of line numbers
40301040 flags
         Initialized Data
         COMDAT; sym= "`string'" (??_C@_0BB@LFDAHJNG@this?5is?5a?5string?$AA@)
         4 byte align
         Read Only

RAW DATA #3
  00000000: 74 68 69 73 20 69 73 20 61 20 73 74 72 69 6E 67  this is a string
  00000010: 00      

with the relocation table wiring up the another_string1 variable name to the literal data.

Note that the name of the string literal object is clearly based on the contents of the literal string, but with some sort of mangling. The mangling scheme has been partially documented on Wikipedia (see "String constants").

Anyway, if you want literals in an assembly file to be treated in the same manner, you'd need to arrange for the literals to be placed in the object file in the same manner. I honestly don't know what (if any) mechanism the assembler might have for that. Placing an object in a "COMDAT" section is probably pretty easy - getting the name of the object to be based on the string contents (and mangled in the appropriate manner) is another story.

Unless there's some assembly directive/keyword that specifically supports this scenario, I think you might be out of luck. There certainly might be one, but I'm sufficiently rusty with ml.exe to have no idea, and a quick look at the skimpy MSDN docs for ml.exe didn't have anything jump out.

However, if you're willing to put the sting literals in a C file and refer to them in your assembly code via externs, it should work. However, that's essentially what Mark Ransom advocates in his comments to the question.

Yes, the process of merging the resources is done by the linker.

If your resources in your compiled assembly code are properly tagged as resources, the linker will be able to merge them with compiled C code.

Much may depend on the specific compiler, linker, and how you drive them. For example, this code:

// s.c
#include <stdio.h>

void f();

int main() {
    printf( "%p\n", "foo" );
    printf( "%p\n", "foo" );
    f();
}

// s2.c
#include <stdio.h>

void f() {
    printf( "%p\n", "foo" );
    printf( "%p\n", "foo" );
}

when compiled as:

gcc s.c s2.c

produces:

00403024
00403024
0040302C
0040302C

from which you can see the strings have only been coalesced in individual translation units.

Identical literals, within the same translation unit, are processed during the parsing phase. The compiler converts literals in tokens and stores them into a table (for simplicity, assume [token ID, value]). When the compiler encounters the literal the first time, the value is entered into the table. The next encounters use the same literal. When generating code, this value is placed into memory and then each access reads this single value (except for those cases where placing the value in the executable code more than once speeds up execution or shortens executable length).

Duplicate literals in more than one translation unit may be consolidated by the linker. All identifiers tagged with global access (i.e. visible from outside the translation unit) will be consolidated if possible. That means that the code will access only version of the symbol.

Some build projects place common or global identifiers into (resource) tables, which allow the identifiers to change without changing the executable. This is a common practice for GUIs that need to present text translated into different languages.

Be aware that with some compilers and linkers, they may not perform the consolidation by default. Some may require a command line switch (or an option). Check your compiler documentation to see how it handles duplicate identifiers or text strings.

"/GF (Eliminate Duplicate Strings)"

http://msdn.microsoft.com/en-us/library/s0s0asdt.aspx

Assembly language doesn't provide any way to work directly with an anonymous string literal like C or C++ does.

As such, what you almost certainly want to do is define the strings in your assembly code with names. To use those from C or C++, you want to put an extern declaration of the array into a header that you can #include in whatever files need access to them (and in your C++ code, you'll use the names, not the literals themselves):

foo.asm

.model flat, c

.data
    string1 db "This is the first string", 10, 0
    string2 db "This is the second string\n", 10, 0

foo.h:

extern char string1[];
extern char string2[];

bar.cpp

#include "foo.h"

void baz() { std:::cout << string1; }