It depends how much X.

You could turn the box into an X terminal and front another server, or you could run a minimal X installation. Ironically I count Motif as lightweight these days, don't attempt Kde or Gnome.

Note that Xfree86 and Xorg tend to deprecate older video hardware support, I had an old machine that I had to compile an older version of an X server in order to support it.