You make a very important point, mskeel. It is something that never occured to me until recently; it is very difficult to understand why a tool exists until you have used it in a practical situation yourself. To that end, how about some examples of when you would each said feature. It might not be as education as certain books and tutorials, but for anyone who is merely interested in a quick read, here goes:
1. Overloading
Nerseus gave an example of this. This is very nice any time you want to have a function, but there are different possiblilites of parameters that could be applicable. Instead of choosing one possible set of parameters, or writing a dozen different functions that do the same exact thing with different input, you make a dozen overloads for the same function. This is ideal for constructors, for example. Look at the System.Version class. When you create a System.Version object, you may want to pass a string (new Version("1.2")). You may want to pass integers (new Version(1, 2)). It might seem useless to be able to type "1.2" instead of 1, 2, but it is anything but useless when the data could either come from a textbox or a function that returns an integer. You may or may not want to specify revision and build. So you create overloads for each possibility. And, unlike normal functions, you don't have the option of creating a number of differently named constructors, so overloading makes life much, much easier.
2. Interfaces
In programming we see the term "class" so often that we seem forget what it really means. A class defines a "class," or category, or type, of object. Stream defines a type of object. FileStream is a Stream. MemoryStream is a stream. They are all objects of the same "class" (or category, or type). A Bitmap is an Image and a Metafile is an Image. Interfaces define common functionality between objects that serve different purposes. Streams are a means of communicating data. Images are a means of representing visual data. But both of these different categories of objects have a certain common functionality: they both have a function to free unmanaged memory: Dispose(). This is the most common .Net Interface because it is a very commonly implemented functionality, but is not the fundamental function of the class.
A TreeNode collection and an integer array are two very different things, and there is no reason why they should inherit the same base class, but they both represent some type of collection that can be enumerated, so they both implement IEnumerable. You can operate on an IEnumerable interface and not care about what class the object actually is.
3. Abstract Classes
Abstract classes exist to define some, but not all, of a classes function. It is similar to an interface, but unlike an interface, it can implement some functionality. For example, the Control class handles keyboard and mouse input, but does not do any drawing. That is left up to the deriving class. The control class calls the OnPaint method when something needs to be drawn, but the Button class, or the CheckBox class, or whatever class, actually defines the OnPaint method. This can't be done with interfaces, and unlike regular inheritable classes, abstract classes don't have to implement everything.
4. Polymorphism
Suppose you write a function to take data for input, encrypt it, and then output it in its original form. Suppose that you write this function to operate over an array of bytes. Then you want to use it for and array of integers. And then for a text file. And then for downloaded web content. And you end up writing four different functions. Suppose you wanted to be able to output in any of the four forms and output in any of the four forms. Now you need sixteen functions, or at least the very least four read functions, four write functions, and one encrpyt function.
Now what if you write a function that accepts a stream as input, encrypts the data, and outputs the data to a stream. You can now use a MemoryStream object to encrypt a byte array, and a FileStream object to encrypt a file, and you don't need to write extra code to put the encrypted data in a different place from where it came. And then, if someone wants to encrypt data coming in through a serial port, they can write a SerialStream class. And not only can they encrypt the serail port data, but they can use that serial port data for any function that operates on streams.
With polymorphism, you can write one function and that one function is incredibly flexible because it operates on polymorphic objects. Not only that, but you can expand what the function can do without any modification to the function by exercising polymorphism. Create your own stream class, and like magic, the EncryptStream function is capable of doing more.