There's two main types of backups:
Bare Metal Recovery backup: You backup the entire system so that you can recover everything back in one hit if you totally fry the system.

Application / Data Recovery backup: You backup only the customised config and data for your app (optionally backing up the application binaries too). If you have to recover from a totally stuffed system, you rebuild the base OS, the apply your backup over the top to put the custom stuff back.

Bare metal recoveries are generally difficult to do and notoriously unreliable as they are hard to test properly. (But I am a bit biased there, so a 2nd opinion might be in order).

Based on my above (biased) opinion, I'd recommend you look at backing up the data only - ie the contents fo the website and webserver config files (including any authentication databases it uses). In the event of a problem, reinstall your OS, reinstall the webserver, restore your backup.

If your webserver isn't installed via a linux package (eg .rpm or .deb etc), you could consider backup up the webserver application directory too - it'll save a step in your recovery in the event of a failure at the expense of longer backup times and more space used.

The final decision should really come down to the following questions:
How fast do you need to recover?
How much storage can you afford?
How are you planning on testing your backups periodically?

Then choose a model that best meets your needs.

As an aside to think about, consider backing up over the network to another server instead, it'll save tapes and time. The drawback is that a total site failure could destroy your backups as well as the original.