Update!
I of course did a little more digging after posting this and found a solution that seems to be working for me. It was another how-to, but a very simple one that was easier for me to follow.
http://www.arches.uga.edu/~pkeck/ssh/
There is a link to the guide I used. Below is the note I made up afterwards for my own future reference. Hopefully this may help someone else running into the same confusion I was:
SSH Key How-To
Client tasks:
ssh-keygen -t dsa
cp id_dsa.pub authorized_keys2
Then copy the authorized_keys2 to file to the .ssh directory of the user you will
be using for ssh access, on the server you will be accessing. For example "/home/sysera/.ssh/authorized_keys2".
Then perform this command on the client to start the ssh-agent on the client machine:
ssh-agent sh -c 'ssh-add < /dev/null && bash'
This will open a new bash shell. From this shell you should be able to access the server machine
without being prompted:
ssh servername
Note: I also have not been prompted for the password after logged out of the current bash session and starting a new one.
I now feel a little silly with this being solved, seemingly so simply, but the entire subject did seem to completely baffle me in the past when I tried to broach it.
Thanks!
-Sys