UNIX for Dummies Questions & Answers

See: man bc

then try
$
$a=5
$b=6
$echo "$a + $b"|bc

Obviously what is inside the quotes will be more complex

You could also be more specific about what you mean by "in Unix"? Do you need a kernel driver, an OpenOffice spreadsheet with the same formula (ugh!), a script which reads the coordinates from the user, a script which reads the coordinates from the command line, or something else?

I pointed you to a program you can download and install, others have shown how to do it by hand as the need arises. You could even script bc if you are good. But almost any programming language at your disposal will have the necessary trig functions and the ability to add, subtract, multiplicate and divide. It's not like it's hard to do if you know the basics. If you promise you won't be using it to launch missiles, maybe somebody will take the challenge, but more detailed requirements would help.