Not sure MS Front Page will upload to your web server without having ftp access.
For my web server, I upload files via SCP or SFTP, as I do not run a FTP server.
You will need to check into whether you can simply upload using MS FrontPage, without additional configurations.
For uploading via SCP from a MS Windows system, you can find a freeware utility called,
WinSCP. It works just like Windows Explorer file manager.
Any reason your using a router instead of a switch? If the router has packet filtering, be sure to disable it. An improperly configured router may cause you some hang-ups.
Since you're trying to eliminate any snags, you may be better off connecting up your machines using a ethernet cross-over cable. This will allow you to plug each machine directly to each other.
If you are using the Apache webserver, you should be able to find enough information on a simple setup. Always check the developers site for howto's.