Compile lua-wrapped C functions using LLVM JIT?

I have compiled some C functions into LLVM bytecode. Now I'd like to make these functions accessible to a Lua script engine and then compile a Lua script into native machine code.

I've found the llvm-lua project that compiles lua script using llvm. I am now wondering if it is possible to perform jit compilation and optimization of the C functions that are called from a lua script.

E.g., I have these two C functions:

void func1() {
  for(int i = 1; i < 5; ++i)
    printf("hello from func1");
}
void func2() {
  for(int i = 1; i < 5; ++i)
    printf("hello from func2");
}

and I expose them to a Lua script engine and run a lua script like that:

func1()
func2()

th开发者_Go百科en I'd like the llvm-lua compiler to optimize it and compile it into a program that corresponds to

for(int i = 1; i < 5; ++i) {
  printf("hello from func1");
  printf("hello from func2");
}

and not into

for(int i = 1; i < 5; ++i)
  printf("hello from func1");
for(int i = 1; i < 5; ++i)
  printf("hello from func2");

Is there any possibility to implement that?

Cheers,

Manuel

For any kind of complex program transformation like the one you're trying to achieve here, it's best to remove as many intermediate steps adding complexity as possible. First prove that it works on the simplest case, and then add the complexity back in step by step. For your particular problem this translates to: try to get the desired optimization to happen on pure C code with all code in the same file, then on pure C code in different files, etc.. If you can't make it happen in all of the simpler cases, then it's fairly unlikely that you can make it work for the original goal with all additional complexity (and by doing it step by step you'll also have a much better idea about the possible causes of any problems you encounter).

If you follow the above advice, I'm pretty confident (although I haven't tried it) that your desired optimization will not be done by the LLVM optimizer, even in the simplest case of having everything in a single C file and running with full optimizations. The reason is that you're expecting the optimizer to change the semantics of your code, because two sequential for loops aren't guaranteed to have identical side effects (observable changes) as a single for loop with the two bodies executed in sequence (the code you provided is a good example of that). For the optimization to be safe, the compiler would have to be able to guarantee (prove) various properties about the side effects of all code executed from the loop bodies. Although not impossible, it's extremely hard to do in the general case in a language with uncontrolled side effects such as C, and in most cases not feasible if you're crossing any library boundaries (as you're likely to do here), since you don't actually have one unifying optimization step which (at least in theory) could take all necessary code into consideration. If you really want to dig deeper into LLVM and its optimizer framework, I recommend you start by reading this excellent article outlining the motivations and design of LLVM, and then figure out which code the optimizer will have to be able to look at in a single step to make it possible.

I'd recommend thinking about what your motivation is for trying to get Lua to compile to LLVM bitcode and be optimized together with LLVM bitcode from C. I'm sure there are legitimate reasons, but unless you're absolutely convinced that this is the only way to achieve your goals, then I'd personally try a different approach.

Lets say your main motivation is performance. As Andrew Y already mentioned, I'd recommend having a good look at luajit. It enables pure (decently written) Lua to perform close to C and many times better than standard Lua, and it also includes a Foreign Function Interface (FFI) which might be helpful for your problem. From the FFI page:

The FFI library allows calling external C functions and using C data structures from pure Lua code.

The FFI library largely obviates the need to write tedious manual Lua/C bindings in C. No need to learn a separate binding language — it parses plain C declarations! These can be cut-n-pasted from C header files or reference manuals. It's up to the task of binding large libraries without the need for dealing with fragile binding generators.

The FFI library is tightly integrated into LuaJIT (it's not available as a separate module). The code generated by the JIT-compiler for accesses to C data structures from Lua code is on par with the code a C compiler would generate. Calls to C functions can be inlined in JIT-compiled code, unlike calls to functions bound via the classic Lua/C API.