The local key pair can be left as is..deleting them causes a problem (ie, the remote host's authorized_keys file has nothing to work with...
). The local copy of the id_dsa.pub file is meaningless to the local (or client) side of the authentication, while the remote copy of the id_dsa.pub is part of the authorized_keys file. This serves to authenticate against the local key file (ie, id_dsa).
You can/should get rid of the .pub file on the remote side, once you've cat-ted it into the remote account's ~/.ssh/authorized_keys file. Leaving it around is fine, but best practice's would suggest that you scratch it..just to be safe(r).