You can put the images in the same folder as the executable, sure. They won't be embedded in the EXE, but that's fine (probably better, actually). For development, just put the image in \bin\Debug or \bin\Release then reference the picture without any path information, such as:
button1.Image = New Bitmap("blah.bmp")
Or you could store them in a subfolder (such as \bin\Debug\images) and use:
button1.Image = New Bitmap("images\\blah.bmp")
(The double backslash is used in C#, not sure about VB)
When you give someone your EXE, just create a subfolder named "images" under the location of the EXE.
Yet another alternative is to have a dummy form that contains the pictures embedded already (or in a PictureBox or multiple PictureBoxes on your form). Set the Image property at design time and .NET will embed the picture in your RESX file. When you compile to an EXE, the picture will get embedded automatically. Then you can do something like:
button1.Image = picBox1.Image
Hope that helps!
-nerseus