Is unloading an assembly such a complex task?

[Squirm]

Greetings,

 

 

My aim is to provide a "plugins" system for an application I am developing. Having worked with plugins in COM I already knew the basic concepts involved and divil's articles on the subject meant I was very quickly able to get a working plugins system working. Or so I thought...

 

The problem came when I wanted to unload a plugin, and in such a way that the DLL which contains it would no longer be locked by the framework - to allow for recompilation (and subsequent reloading of the modified plugin). In the absence of an Assembly.Unload method I expected that as soon as no types from the assembly were still instantiated, and there were no more references to anything within the assembly (including the assembly itself), that it would be automatically unloaded.

 

Suffice to say this is not the case. It would appear that once an assembly is loaded, it is loaded forever, until the AppDomain which contains it terminates. Consider the following code:

 

foreach (FileInfo f in dlls) {
   Assembly   dll = Assembly.LoadFrom(f.FullName);
}

GC.Collect();

 

This is what I have come down to, and even this code will keep the assemblies loaded. This is doubly annoying since this basic operation is required even to just query the assembly for type information, even if you later discover that it does not contain any useful types (ie plugins).

 

My attention is therefore turned to the AppDomain class. It seems I could create a new AppDomain every time I want to query the assemblies, and upon unloading (ending? terminating?) the AppDomain the DLLs containing the assemblies should be unlocked. When a plugin is to be instantiated, it can be loaded into its own AppDomain and when finished with, unloading the AppDomain should free the plugin assembly.

 

But just looking at the AppDomain class (which I have never looked at before) suggests this is a daunting task and will break a lot of my code, since it wont be so simple to just pass an object reference between the application AppDomain and the plugin AppDomain.

 

So, a few questions (finally!).

 

1. Can an individual assembly be dynamically unloaded in the same way it is loaded, if so how?
   
2. If AppDomains are the way to go, what sorts of things should I be looking to change in the program - how do AppDomains interact?
   
3. Has anybody else been in a similar situation, and if so how did you solve it if you did?
   

 

On one hand I can't believe it would be so involved to create a plugins system, especially with the lightness of divil's tutorials and all other information I can find on the subject. On the other hand I can see why the framework might be so restrictive, but it sure is annoying.

 

Regards

 

Squirm

 

[edit]Found this on MSDN:

There is no way to unload an individual assembly without unloading all of the application domains that contain it. Even if the assembly goes out of scope, the actual assembly file will remain loaded until all application domains that contain it are unloaded

 

:-\[/edit]

Edited August 13, 2005 by Squirm

[Wraith]

The problem came when I wanted to unload a plugin, and in such a way that the DLL which contains it would no longer be locked by the framework - to allow for recompilation (and subsequent reloading of the modified plugin). In the absence of an Assembly.Unload method I expected that as soon as no types from the assembly were still instantiated, and there were no more references to anything within the assembly (including the assembly itself), that it would be automatically unloaded.

The Type tree for the AppDomain in which the assembly is loaded contains those types, if any code is executing those types could be referenced in any part of the current stack frame, any object residing on the heaps could reference the Types loaded from the assembly, and those are just the static references. With reflection a Type could be required at any time even by code which does not explicity reference the assembly, consider Activator.CreateInstance construction for example.

 

You, as the programmer, may know that it is no longer referenced in any way but unless you can provide a method to determine this for the CLR it cannot allow assemblies to be unloaded even when it appears there are no references to the types.

 

Suffice to say this is not the case. It would appear that once an assembly is loaded, it is loaded forever, until the AppDomain which contains it terminates. Consider the following code:

 

foreach (FileInfo f in dlls) {
   Assembly   dll = Assembly.LoadFrom(f.FullName);
}

GC.Collect();

Why the GC.Collect()? You're really not supposed to explicitly call collect unless you know you need to and if simply loading assemblies causes you to require a collection you've got a seriously messed up set of .cctors defined.

 

This is what I have come down to, and even this code will keep the assemblies loaded. This is doubly annoying since this basic operation is required even to just query the assembly for type information, even if you later discover that it does not contain any useful types (ie plugins).

It will also allow an error to occur if you try to load a plugin which contains a Type already present in the parse tree, there are a host of other problems that can occur as well.

 

My attention is therefore turned to the AppDomain class. It seems I could create a new AppDomain every time I want to query the assemblies, and upon unloading (ending? terminating?) the AppDomain the DLLs containing the assemblies should be unlocked. When a plugin is to be instantiated, it can be loaded into its own AppDomain and when finished with, unloading the AppDomain should free the plugin assembly.

My advice would be to create an AppDomain for probing, locate the plugin types present (if any) and pass them back to the main domain in a serialized form (the TypeName string for example, just not Type classes), then create a new AppDomain and specifically load the plugin. This way if there are no plugins present you unload the probing domain cleanly and with no loss, if there is an error you know it comes from that specific assembly because your pobing domain is cleanly created each time instead of reusing a domain which contains the waste from other load operations.

 

But just looking at the AppDomain class (which I have never looked at before) suggests this is a daunting task and will break a lot of my code, since it wont be so simple to just pass an object reference between the application AppDomain and the plugin AppDomain.

Objects which inherit from MarshalByRefObject or ContextBoundObject are moved between AppDomains as references, ValueTypes are copied. Interaction/function calls between objects in different AppDomains are handled by transparent proxies (see Remoting docs for details) and serialization of messages. Unless you're getting complicated its pretty much like using objects normally because the proxies pretend to be their target. Events can be more complicated to work with because or marshalling subtleties and performance can suffer because of all the bits of code that you're not seeing but overall its not too hard to use.

 

So, a few questions (finally!).

1. Can an individual assembly be dynamically unloaded in the same way it is loaded, if so how?

No.

 

2. If AppDomains are the way to go, what sorts of things should I be looking to change in the program - how do AppDomains interact?

You need to isolate your plugin system from the core. Better still is to create a functional plugin system and have your core application be a plugin itself, this means that you get a very rich plugin system and you adequately test the entire plugin system because you'll be using it all.

 

3. Has anybody else been in a similar situation, and if so how did you solve it if you did?

 

I decided i needed plugins, i found the problems you've outlined and worked up a functional plugin system which allows me to probe assemblies for plugins and load them into appdomains in a number of configurations,monolithic (single domain), shared plugin domain, plugin per-domain, assembly per domain. It took some reading and coding but i've got a nice system that if i ever need i can just drop into my solution as a namespace and have work.

 

On one hand I can't believe it would be so involved to create a plugins system, especially with the lightness of divil's tutorials and all other information I can find on the subject. On the other hand I can see why the framework might be so restrictive, but it sure is annoying.

A basic plugin system is basic, a complex one is not. If you want complex behaviors then you have to take the time to write the code needed i'm afraid. The framework limitations are very apparent in this scenario its true but these same limitations vasly increase ease of use elsewhere.

 

 

One other thing you need to be aware of is that the error handling is going to be an absolute cow to write, a serious porportion of my plugin system is catching and handling Exceptions at various levels and events so i can cleanly report and recover from misconfigurations, bad dlls or plugins. Done properly this is a big chunk of code, done properly its done once and you never have to look at it ever again. :)

[Squirm]

Thanks for the response. I have a shakey system working, which is a start. Thanks for the tip about returning an array of typenames rather than actual Type objects. This is my probing code:

 

public Type[] FindPlugins(string dir)
{
AppDomain	querydom;
PluginFinder	finder;
Type[]		types;

querydom = AppDomain.CreateDomain("PluginQueryDomain",
		new System.Security.Policy.Evidence(AppDomain.CurrentDomain.Evidence));
querydom.Load(Assembly.GetExecutingAssembly().GetName());
finder = (PluginFinder) querydom.CreateInstanceAndUnwrap(
		Assembly.GetExecutingAssembly().FullName, "BlueFish.Plugins.PluginFinder");

types = finder.FindPlugins(dir);

AppDomain.Unload(querydom);
return types;
}

 

(I will now convert that to use type strings). It was actually much simpler than I had expected, since as you point out the MarshalByRef class deals with all the nasties. I think I can see how to get it all working now, the major hurdle I have at the moment is my large (mis?)use of delegates to private methods, which apparently are not allowed across appdomains, so I'm going to have to rethink on that issue.

 

Thankyou for the indepth reply. It's reassuring to know I'm not the only person to have dealt with this problem.

 

:)

 

[edit]Re: GC.Collect(), I never normally use it, I just threw it in to be sure that if assemblies could be automatically unloaded, they would be. And the answer is that they can't. :)[/edit]

Edited August 13, 2005 by Squirm

[Wraith]

Thanks for the tip about returning an array of typenames rather than actual Type objects.

I now realise that i didn't explain why i think this is useful. A Type contains information about the type, that information includes the information aobut the filesystem or web location of the containing assembly, if you pass a Type back to another domain it may well use that information to load the assembly. If that occurs you've lost the separation between your AppDomains which was what you wanted in the first place. If you pass the TypeName around you can use it to locate the type but it is only a string and won't force an assembly load unless you manually use it to do so.

 

I created a simple [serializable] marked class called LoadableType which contains only two strings, the TypeName and ModuleName, with those i can load the correct assembly but i'm only using strings.

[bri189a]

Nice set of post guys and very informative for those who might not of done dynamic assembly loading before. But I have a question regarding your design and maybe I can learn something here.

 

I've worked on an application that uses plug-ins and in it, any new plug-in that are created are deployed in a package that installs to specific underlying folder of the application it is for. This way all I really do is load all the plug-ins that are in that folder.

 

While true an user could put a dll in there that doesn't belong, my app would load it, but it would never be used because it doesn't lie in a certain namespace or doesn't have a certain base class or implement a certain interface (depending on the context), so yes it would tie up some memory but it wouldn't 'hurt' anything.

 

My deployment app ensures that there are no instances of the targed app running before installing new plug-ins, which I don't really think is a big deal. Users are use to having to reboot when windows issues an update, or restart MS Office when a service pack is installed, so I didnt' figure it mattered.

 

But since you've seem to put a lot of thought into 'unloading' assemblies (outside of the app domain closing), I'm curious as to the thought process behind it... like I said maybe there's something to be learned here.

 

Thanks folks...

[Wraith]

One scenario you've not considered is dynamicly emitted assemblies, runtime code generation etc. If you for some reason need to emit some code to do an operation then you can put it in a new AppDomain, once your emitted code has been run you now have the choice of removing it from memory entirely leaving your application free to create a new version of the function, perhaps a faster one, to operate on more data. If you're only using a single load domain you're stuck with the first set of code you generated and each time you generate a new version you irreversably increase the memory footprint of your application.

 

In the Plugin model we've already discussed you may wich to replace a running plugin because you've made a new version. Now you can force the application to a full stop replace the plugin and restart but if the application is doing a long running task then you may lose a lot of work. Enabling plugins to be unloaded and replaced without the parent application going down allows for a more modular design and is less intrusive than forcing a full application closure each time.

 

Having a flexible plugin system is a lot more work than a simple one, and the gains seem few unless you're the type of person who just has to have those really nifty features. On the other hand you may find that the abilities provided by a rich plugin architecture improve the usability of your application far more than you anticipated, some problems you just don't notice until they're removed, then you wonder how you ever coped.

[bri189a]

Intersting. Haven't run into a situation like that before but I will definitely keep in mind. Thanks for the input.

[BrianBender]

I seem to be having the same problems. I can load assemblies all day in the current AppDomain without references and without interfaces if need be. But try as I may they will ot unload. I have been working on this problem for weeks. I have seen other apps using Remoting but I know there has got to be a way to create a child AppDomain and reference obkect via reflection and uload the child domain when finished.

 

My Scenario. I have windows services which are basically libraries called obviously by a service wrapper. Since the wrapper does the instansiation, I want to update these dlls without stopping and restarting my service.

 

I have created a mockup using a form and a separate dll

 

Here is the mockup simple dll the form will call:

using System;

namespace TestClass
{
public class Test : MarshalByRefObject
{
	public string GetResponse()
	{
		return "Test Response 1";
	}
}
}

 

Here is the form code:

 

using System.ComponentModel;
using System.Windows.Forms;
using System.Data;
using System.Reflection;

namespace DomainTest
{

public class Form1 : System.Windows.Forms.Form
{
             private System.Windows.Forms.Button btnLoad;
	private System.Windows.Forms.Button btnUnload;
	private System.Windows.Forms.Button btnCall;
	private System.Windows.Forms.TextBox txtResponse;

	private AppDomain _runDomain;
	private Assembly _assembly;
	private Type _typTest;
	Object	_objTest;
	AppDomainSetup _appSetup;

	private System.ComponentModel.Container components = null;

	public Form1()
	{
		InitializeComponent();
		_appSetup = new AppDomainSetup();
		_appSetup.ApplicationBase = @"E:\Development\.Net\CSharp\Examples\AppDomain\AppDomainEnvironment";
		_runDomain = AppDomain.CreateDomain("MyDomain", null, _appSetup);

	
	}
	
	protected override void Dispose( bool disposing )
	{
		if( disposing )
		{
			if (components != null) 
			{
				components.Dispose();
			}
		}
		base.Dispose( disposing );
	}

	#region Windows Form Designer generated code
	/// <summary>
	/// Required method for Designer support - do not modify
	/// the contents of this method with the code editor.
	/// </summary>
	private void InitializeComponent()
	{
		this.btnLoad = new System.Windows.Forms.Button();
		this.btnUnload = new System.Windows.Forms.Button();
		this.btnCall = new System.Windows.Forms.Button();
		this.txtResponse = new System.Windows.Forms.TextBox();
		this.SuspendLayout();
		// 
		// btnLoad
		// 
		this.btnLoad.Location = new System.Drawing.Point(30, 35);
		this.btnLoad.Name = "btnLoad";
		this.btnLoad.Size = new System.Drawing.Size(70, 25);
		this.btnLoad.TabIndex = 0;
		this.btnLoad.Text = "Load";
		this.btnLoad.Click += new System.EventHandler(this.btnLoad_Click);
		// 
		// btnUnload
		// 
		this.btnUnload.Enabled = false;
		this.btnUnload.Location = new System.Drawing.Point(300, 35);
		this.btnUnload.Name = "btnUnload";
		this.btnUnload.Size = new System.Drawing.Size(70, 25);
		this.btnUnload.TabIndex = 1;
		this.btnUnload.Text = "Unload";
		this.btnUnload.Click += new System.EventHandler(this.btnUnload_Click);
		// 
		// btnCall
		// 
		this.btnCall.Enabled = false;
		this.btnCall.Location = new System.Drawing.Point(166, 35);
		this.btnCall.Name = "btnCall";
		this.btnCall.Size = new System.Drawing.Size(70, 25);
		this.btnCall.TabIndex = 2;
		this.btnCall.Text = "Call";
		this.btnCall.Click += new System.EventHandler(this.btnCall_Click);
		// 
		// txtResponse
		// 
		this.txtResponse.Location = new System.Drawing.Point(30, 85);
		this.txtResponse.Name = "txtResponse";
		this.txtResponse.Size = new System.Drawing.Size(340, 20);
		this.txtResponse.TabIndex = 3;
		this.txtResponse.Text = "";
		// 
		// Form1
		// 
		this.AutoScaleBaseSize = new System.Drawing.Size(5, 13);
		this.ClientSize = new System.Drawing.Size(402, 441);
		this.Controls.Add(this.txtResponse);
		this.Controls.Add(this.btnCall);
		this.Controls.Add(this.btnUnload);
		this.Controls.Add(this.btnLoad);
		this.Name = "Form1";
		this.Text = "Form1";
		this.ResumeLayout(false);

	}
	#endregion

	
	[sTAThread]
	static void Main() 
	{
		Application.Run(new Form1());
	}

	private void btnLoad_Click(object sender, System.EventArgs e)
	{
		
		
		_assembly = AppDomain.CurrentDomain.Load("TestClass");
		_typTest = _assembly.GetType("TestClass.Test");
		_objTest = _runDomain.CreateInstance("TestClass", "TestClass.Test");
		System.Diagnostics.Debug.WriteLine(_objTest.GetType());

		btnCall.Enabled = _objTest != null;
		btnUnload.Enabled = _objTest != null;
		btnLoad.Enabled = _objTest == null;
	}
       
	private void btnCall_Click(object sender, System.EventArgs e)
	{
		MethodInfo method = _typTest.GetMethod("GetResponse");
		txtResponse.Text = (string) method.Invoke(_objTest, null);
	}

	private void btnUnload_Click(object sender, System.EventArgs e)
	{
		AppDomain.Unload(_runDomain);
		btnCall.Enabled = false;
		btnLoad.Enabled = true;
		btnUnload.Enabled = false;
	}

	
}
}

 

The Results:

 

The issue is that when calligna method off of the newly created object I get the following error:

 

Object does not match target type.

 

I looked at the actual object type from the object created with this line of code:

 

_objTest = _runDomain.CreateInstance("TestClass", "TestClass.Test");

 

The object is not a TestClass.Test Type object, it is a System.Runtime.Remoting.ObjectHandle type object.

 

Soooooooooooooooooooo....

 

How to get around this? Am I even close here?

Edited January 13, 2006 by PlausiblyDamp

[Wraith]

I have seen other apps using Remoting but I know there has got to be a way to create a child AppDomain and reference obkect via reflection and uload the child domain when finished.

There is? Then why is no-one else using it? Why is it that everyone who has approached this problem pretty much ends up doing the same thing, AppDomains and marshalling between then.

 

My Scenario. I have windows services which are basically libraries called obviously by a service wrapper. Since the wrapper does the instansiation, I want to update these dlls without stopping and restarting my service.

Sounds like you want to set the shadow copy property of the appdomain and use a FilesystemWatcher, theres an example on the microsoft site (an article iirc) that does just this, by Eric Gunnerson i think.

 

How to get around this? Am I even close here?

You appear to be trying to get this working without doing the required background reading, you don't understand what you're doing so you can't cope with the errors you're making or the exception message you're getting. Do your reading.

[BrianBender]

You appear to be trying to get this working without doing the required background reading, you don't understand what you're doing so you can't cope with the errors you're making or the exception message you're getting. Do your reading.

 

All I do is read. If you have any decent documentation on App Domains I would certainly love to see it.

Edited August 23, 2005 by BrianBender

[Wraith]

My reference is the SDK documentation and the example code and articles i found on the internet with simple searches. Its a bit obtuse in places but it is entirely documented and accessible.