Doesn't look like it can be removed. To clarify, this is the equivalent of the "Properties" node in the C# solution explorer. Unlike the C# "Properties," and characteristic of VB8, the files that ultimately compose "My Project" are hidden in the solution explorer and only editable through the designer. To rectify this and view the components of "My Project" click "Show All Files" under the project menu.
"My Project" must be there because it represents files that are part of the solution. (I would expect it to dissappear if all said files were to be deleted, but it doesn't). It doesn't take up extra space, however, since it replaces (and includes) AssemblyInfo.vb, and additionally includes the new Settings.settings, Application.MyApp, Application.MyApp.designer, and Resources.Resx files.