Embedded IronPython Memory Leak

I need some help finding a solution to a memory leak I'm having. I have a C# application (.NET v3.5) that allows a user to run IronPython scripts for testing purposes. The scripts may load different modules from the Python standard library (as included with IronPython binaries).开发者_JS百科 However, when the script is completed, the memory allocated to the imported modules is not garbage collected. Looping through multiple runs of one script (done for stress testing) causes the system to run out of memory during long term use.

Here is a simplified version of what I'm doing.

Script class main function:

public void Run()
{
    // set up iron python runtime engine
    this.engine = Python.CreateEngine(pyOpts);
    this.runtime = this.engine.Runtime;
    this.scope = this.engine.CreateScope();

    // compile from file
    PythonCompilerOptions pco = (PythonCompilerOptions)this.engine.GetCompilerOptions();
    pco.Module &= ~ModuleOptions.Optimized;
    this.script = this.engine.CreateScriptSourceFromFile(this.path).Compile(pco);

    // run script
    this.script.Execute(this.scope);

    // shutdown runtime (run atexit functions that exist)
    this.runtime.Shutdown();
}

An example 'test.py' script that loads the random module (adds ~1500 KB of memory):

import random
print "Random number: %i" % random.randint(1,10)

A looping mechanism that will cause the system to run out of memory:

while(1)
{
    Script s = new Script("test.py");
    s.Run();
    s.Dispose();
}

I added the section to not optimize the compilation based on what I found in this thread, but the memory leak occurs either way. Adding the explicit call to s.Dispose() also makes no difference (as expected). I'm currently using IronPython 2.0, but I've also tried upgrading to IronPython 2.6 RC2 without any success.

How do I get the imported modules in the embedded IronPython script to be garbage collected like normal .NET objects when the scripting engine/runtime goes out of scope?

using Iron Python 2.6 RC 2, and C# 3.5

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Microsoft.Scripting.Hosting;
using IronPython.Hosting;

namespace IPmemTest {
    class IPy {
        private string script = "import random; random.randint(1,10)";

        public IPy() {
        }

        public void run() {
            //set up script environment
            Dictionary<String, Object> options = new Dictionary<string, object>();
            options["LightweightScopes"] = true;
            ScriptEngine engine = Python.CreateEngine(options);
            ScriptRuntime runtime = engine.Runtime;
            ScriptScope scope = runtime.CreateScope();
            var source = engine.CreateScriptSourceFromString(this.script);
            var comped = source.Compile();
            comped.Execute(scope);
            runtime.Shutdown();
            }
    }
}

and my loop is

class Program {
        static void Main(string[] args) {
            while (true) {
                var ipy = new IPy();
                ipy.run();
            }
        }
    }

memory usage increases to about 70,000K, but then levels off.

Have you tried running the Iron python in its own appDomain? Python.CreateEngine allows you to pass it an AppDomain, which can then be unloaded when the script completes.

Or, based on this discussion, use LightweightScopes option, like so

Dictionary<String, Object> options = new Dictionary<string, object>();
options["LightweightScopes"] = true;
ScriptEngine engine = Python.CreateEngine(options);
ScriptRuntime runtime = engine.Runtime;

The other answer did not help me very much, so im posting my workaround. What I did is I set up the Scripting Scope and the Engine OUTSIDE my while loop. My memory leak was increasing very fast before, but with this fix, I was able to stop it.

The only thing I left inside the loop was the actual call from my Python script. This is the beginning of my script.

var engine = Python.CreateEngine();
        scope = engine.CreateScope();
        ICollection<string> paths = engine.GetSearchPaths();
        paths.Add(@"C:\Python27");
        paths.Add(@"C:\Python27\Lib");
        paths.Add(@"C:\Python27\Lib\site-packages");
        engine.SetSearchPaths(paths);
        paths = engine.GetSearchPaths();
        engine.ExecuteFile(path, scope);


        while (!stoppingToken.IsCancellationRequested)
        {
           Run();
        }

Also I declared a private ScriptScrope in my class so i could call the scope from other methods without passing the scope.

public class Worker : BackgroundService
{
    private readonly ILogger<Worker> _logger;
    private ScriptScope scope;

Then inside my "Run" Method i just added:

var result= scope.GetVariable("MyPythonMethod");

Now my memory usage stays at 130 MB instead of 1GB+.