You're trying to deserialize a list of objects into a single object. It seems list you're also using XML that you created, not XML created by the serialization process. Try:
Dim lst As New List(Of EmployeeAdder)
Dim ser As New Xml.Serialization.XmlSerializer(GetType(List(Of EmployeeAdder)))
Dim reader As New System.IO.StreamReader(New System.IO.FileStream(Application.StartupPath & "\test.xml", IO.FileMode.Open))
lst = CType(ser.Deserialize(reader), List(Of EmployeeAdder))
using XML formatted like this:
<?xml version="1.0" encoding="utf-8"?>
<ArrayOfEmployeeAdder xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<EmployeeAdder>
<ID>1</ID>
<Name>Test</Name>
<ActivityID>1</ActivityID>
</EmployeeAdder>
<EmployeeAdder>
<ID>2</ID>
<Name>Test 2</Name>
<ActivityID>2</ActivityID>
</EmployeeAdder>
</ArrayOfEmployeeAdder>