Hosting/Embedding an Ironpython 2.x app in your C# app

We have a C# library we exercise with some python scripts. We also have the ability to execute our scripts from inside the C# app we use with our library. So instead of writing python scripts and executing them with ipy.exe we created a custom script runner with a bit of a framework attached to it. I set out to upgrade us from IPy 1.1.2 to the latest stable, 2.6.

I had a hell of a time finding all the right resources to figure out how to host/embed Ironpython inside a C# app so I'm writing this post with the aim of showing how to do it.

First things first, when you're looking for help, you need to look for help on Microsoft's Dynamic Language Runtime (DLR) just as much as you need to look for help on Ironpython. In fact, if you only look for Ironpython you'll find tons of examples for the Ironpython 1.x api which don't work at all in the 2.x framework.

To embed in your C# app you should know about three parts of the DLR, ScriptingHost, ScriptEngine, and ScriptScope.

ScriptingHost is tied to an AppDomain (must admit here that I don't know what an AppDomain is, I'm not a C# guy). ScriptEngine is analogous to an instance of an interpreter. ScriptScope is analogous to a context for the execution of bits of code. An instance of a ScriptScope provides a way to separate execution context for multiple scripts on one ScriptEngine instance. A ScriptScope instance also contains global variables and probably lots of other stuff I don't know about.

Here's an outline of what we do when we embed and execute scripts.
1. Load assemblies so that they are accessible from script code
2. Add CPython libraries to the path
3. Execute common script functionality
4. Allow execution of script files
5. Exit scripts cleanly
6. Provide detailed stack trace with file and line number when errors occur

All of this functionality is wrapped up inside a class.

Here's the code.

public class Host
{
    private static Host ScriptingHost;

    private ScriptEngine engine;
    private ScriptScope scope;

    public static void InitializeHosting() 
    {
        ScriptingHost = HostFactory(); 
    }

    public static Host GetScriptingHost()
    {
        return ScriptingHost; 
    }

    private Host()
    {
        ScriptRuntime runtime = IronPython.Hosting.Python.CreateRuntime();

        runtime.LoadAssembly( System.Reflection.Assembly.GetExecutingAssembly() );  // reference to current assembly
        runtime.LoadAssembly(typeof(System.Diagnostics.Debug).Assembly);  //reference to clr

        engine = runtime.GetEngine("Python");

        if (Directory.Exists("C:\\Python26\\Lib"))
            AddToPath("C:\\Python26\\Lib\\");
        
        scope = engine.CreateScope();

        engine.Execute("import clr", scope);
        engine.Execute("clr.AddReference('MyLib')", scope);
        engine.Execute("clr.AddReference('System')", scope);
        engine.Execute("import System", scope);
        engine.Execute("import MyLib", scope);
    }

    public void AddToPath(string path)
    {
        ICollection paths = engine.GetSearchPaths();
        paths.Add(path);
        engine.SetSearchPaths(paths);
    }

    public int ExecuteFile(string s, string [] args, bool rethrow)
    {
        int retCode = 0;

        scope.SetVariable("args", args);  // we access this variable in our scripts
        try
        {
            engine.ExecuteFile(s, scope);
        }
        catch (Microsoft.Scripting.SyntaxErrorException e)
        {
            Console.WriteLine(string.Format("{0}:{1}:{2}", e.SourcePath, e.Line, e.Message));

            retCode = 1;

            if (rethrow)
            {
                Exception e2 = new Exception(e.Message, e);
                throw e2;
            }
        }
        catch (IronPython.Runtime.Exceptions.SystemExitException e)
        {
            Object nonIntegerExitCode;
            int exitCode = e.GetExitCode(out nonIntegerExitCode);
            if (nonIntegerExitCode != null)
            {
                Console.WriteLine(nonIntegerExitCode.ToString());
            }
            else
            {
                retCode = exitCode;
            }
        }
        catch (Exception e)
        {
            string pyTrace = "";

            // get the stackframes associated with the exception
            Microsoft.Scripting.Runtime.DynamicStackFrame[] stackframes;
            stackframes = Microsoft.Scripting.Runtime.ScriptingRuntimeHelpers.GetDynamicStackFrames(e);

            // store off each line
            foreach (Microsoft.Scripting.Runtime.DynamicStackFrame frame in stackframes)
            {
                pyTrace += "   " + frame.ToString() + "\n";
            }

            Console.WriteLine(string.Format("Exception caught while running {0}:\n{1}\nStack Trace:\n{2}", s, e.Message, pyTrace));

            retCode = 1;

            if (rethrow)
            {
                Exception e2 = new Exception(e.Message, e);
                throw e2;
            }
        } 

        return retCode;
    }
}

Pay special attention to the bit about how to grab the python stacktrace from the exception and that the CPython libs were added to the search path.

If you're like me you'll fire this framework up and run an old script only to get stopped dead on the error 'type' object has no attribute 'CreateArray'. CreateArray was never part of System.Array, but through the use of some reflection I did not take the time to understand, it was there via IronPython.Runtime.Operations.ArrayOps. Thankfully, we don't even need the CreateArray function anymore. Code that used to say:

newData = [random.randint(0,255) for k in range(1024)]
newData = System.Array.CreateArray(System.Byte,newData)

Can now be written as:

newData = System.Array[System.Byte]([random.randint(0,255) for k in range(1024)])

Now you should be able to upgrade your IronPython 1.x code or simply embed IronPython 2.x in your app. There might be other stuff I run into but this was the meat of it.

Here's some links that were helpful to me:
http://devhawk.net/CategoryView,category,IronPython.aspx - This guy discusses using the SetTrace feature of the engine to implement your own debugger or enable PDB.
http://dlr.codeplex.com/wikipage?title=Docs%20and%20specs - The DLR spec docs (can be found by searching for dlr-spec-hosting.doc, some older links point you to dlr-spec-hosting.pdf which no longer exists)
http://blogs.msdn.com/seshadripv/default.aspx - MSDN blog that has lots of good examples and help. Doesn't seem to have been updated for quite some time though.
http://www.mail-archive.com/users@lists.ironpython.com/ - Archive of the ironpython mailing list, where your question has probably already been debated and answered.