Installing packages...need help with the basics


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Installing packages...need help with the basics
# 1  
Old 04-27-2014
Installing packages...need help with the basics

Hello,

I am working in terminal on a Mac OS X 10.9.2. I need to install a series of bioinformatics tools and packages (currently I want to install fastq-tools but downstream I'll need other such as samtools, bwa etc etc.) and will eventually have to be able to do this not only on my mac but also under my user account on a university linux cluster (to a place on the cluster where I have file permissions I guess?).

The instructions for installing fastq-tools reads:

"on most systems, installation is as simple as ./configure && make && make install"..."the only external dependencies are PCRE and zlib"....

I could use clarification on the following:

1) I assume that the steps are to
- download the package from whatever website is hosting it
- unzip it
- cd to that folder in Terminal
- enter the command <<./configure && make && make>>

I assume that I need to do this for each of the dependencies first...

Is this correct?

2) If I follow these steps, where are the executables actually installed? I'm really confused about the whole /usr/bin versus /usr/local/bin thing. How can I see where packages are installed as a default? (I might have inadvertently changed this messing around by now). In fact, when I do this for the dependancies I get errors...

3) I read that if I want to specify a different directory to store the programs in, I can use:

./configure --prefix=/myUnixStuff/programs
make
sudo make install

(why sudo?)

I've done this for one of the dependancies and a bunch of stuff happened but when I open the /myUnixStuff/programs folder, it's empty. Does this mean it failed (I didn't seem to get error messages) or that stuff went somewhere else?

4) Following up from question 3, how does one see all the programs available in PATH to check that an install worked? For some programs it seems you check by typing <<program -V>> but that's not working in this case.

5) Also following up from question 3, if I DO manage to install all the tools I need to one place on my mac, can I simply scp that file over to the linux cluster and expect it to work? (I'm being optimistic here I know)

Sorry for all the questions. I've been searching online for hours and am very lost. If anyone has a straightforward system for installing packages, I'm eager to hear it. I've come across fink and a few other related tools but my comprehension of linux is too poor for me to really understand what those things do....

Thanks
# 2  
Old 04-28-2014
Typically (ie you can do what you like within reason but most people follow this guideline):
  • /usr/bin is where the OS has it's executable code, it's also where software that is installed via the OS's package management system (eg .dmg files on Mac).
  • /usr/local/bin is where custom executable code goes, things that aren't managed by any sort of version control in the OS or packages (eg download thing, compile thing, run "make install")
  • You can fulfil dependancies by either getting the right precompiled package, or by compiling yourself (make). You can mix and match, but typically I prefer to use only one method to supply dependancies (especially so if they are only going to be used by that one tool I'm trying to get going) A good rule of thumb is to try really hard to find a 'proper' package for the thing you want, then revert to compiling if you really have to (or if you really need a special version of it or compile option)
  • When you run a "make install" it will follow whatever instructions are in the makefile. Usually, when you run ./configure, it'll set up a few variables automatically to sensible defaults, including the install path.
  • If you want to copy files into protected arts of the OS (/usr/bin and /usr/local/bin for example), you need root privs, sudo grants these to the command you specify immediately after the word "sudo" (eg sudo echo "I am root for this command only")
  • Hitting tab should expand all available commands, but that's probably not what you really want to do. Use ls on the install dir to look for it:
    Code:
    ls /usr/local/bin

    or
    Code:
    ls /usr/bin

  • Generally speaking, you compile things for the local environment. You can cross-compile but it's a bit fiddly to explain in a forum post.
# 3  
Old 04-28-2014
Hi.

This thread is similar to one at Installing a command/program to a remote computer

Noted there is information from one source of fastq-dump that there are pre-compiled binaries for some platforms.

If I were you, I would practice installing to an area under my home directory on the Mac because when you go to the campus Linux cluster, you will not likely have adminstration rights there, so that you will need to do a number of tasks differently.

In addition, simply copying from the Mac to the Linux cluster will probably not work. You could test this out beforhand by copying, say the Mac version of system command ls to your home directory on the Linux cluster and try to execute it. Mac hardware is (now) Intel, but the OS is essentailly FreeBSD, and I don't see that as running under Linux. Here's an example: I copied /bin/ls from a PCBSD machine to a Debian machine:
Code:
OS, ker|rel, machine: FreeBSD, 9.2-RELEASE-p8, amd64

OS, ker|rel, machine: Linux, 3.2.0-4-amd64, x86_64
Distribution        : Debian 7.4 (wheezy, workstation-vm)

working with that file on the original machine:
Code:
$ cp /bin/ls ~
$ ls -ln ./ls
-r-xr-xr-x  1 1001  1001  32256 Apr 28 09:26 ./ls
$ file ./ls
./ls: ELF 64-bit LSB executable, x86-64, version 1 (FreeBSD), dynamically linked (uses shared libs), for FreeBSD 9.2, stripped
$ ./ls
Desktop		Music		Videos		log		try
...

transferring and working with that file ls on Debian::
Code:
$ ls -lgG ./ls
-r-xr-xr-x 1 32256 Apr 28 09:21 ./ls*
$ file ./ls
./ls: ELF 64-bit LSB executable, x86-64, version 1 (FreeBSD), dynamically linked (uses shared libs), for FreeBSD 9.2, stripped
$ ./ls
-bash: ./ls: No such file or directory

However, I do not know from where you are getting fastq-dump, so things might be different for you.

Best wishes ... cheers, drl
# 4  
Old 04-28-2014
Quote:
Originally Posted by jullee
"on most systems, installation is as simple as ./configure && make && make install"..."the only external dependencies are PCRE and zlib"....
In fact this is not an installation in the sense you would expect on a Windows system. Here is some context to make you understand better what you attempt to do:

Software is written in source code (the programs text, written by the programmer) and then translated into the code the machine is able to understand ("binary"). The programs which do this translation are called "compilers". Most compilers have an awful lot of configuration options, which is why compiling a complete software package is an awful lof commands. To make this easier there is (standardized) tool in all UNIX systems, which is called "make". "make" is rule-based and although it was developed with compiling programs in mind it can be used for other things (like installing software) too.

Having said this: what you do with "configure" is calling a special script (called "configure", which is a quasi-standard) which attempts to best-guess the special surroundings it operates in and then makes adjustments to the rule-files "make" will operate on in the second step. Have a look at a file called "Makefile" in the directory you unpacked your source code to to see it.

"Make" called without anything compiles the program from the source and if it is successful it will be called in the form "make install", which will, instead of compiling the software, install it on the computer.

Quote:
Originally Posted by jullee
I could use clarification on the following:

1) I assume that the steps are to
- download the package from whatever website is hosting it
- unzip it
- cd to that folder in Terminal
- enter the command <<./configure && make && make>>
Yes. instead of the last line you might want to enter the commands separately:

Code:
./configure
make
make install

and watch its output. You do not need to invoke "command" as long as you do it from the command line, which i suggest you do.

Quote:
Originally Posted by jullee
I assume that I need to do this for each of the dependencies first...
Yes. Maybe the dependencies ahve dependencies itself, so you might have to do that several times, but basically you are correct.


Quote:
Originally Posted by jullee
2) If I follow these steps, where are the executables actually installed? I'm really confused about the whole /usr/bin versus /usr/local/bin thing. How can I see where packages are installed as a default? (I might have inadvertently changed this messing around by now). In fact, when I do this for the dependancies I get errors...
UNIX-systems follow a general plan where certain files should go. This is not a standard and there are minor deviations from one system to the other, but in general all UNIX-like systems follow this plan. See the "file systems hierarchy standard", which tries to lay down this standard for Linux systems. More than 95% of what is in there is correct for any other system too, even if there is no "law" compelling someone to do it this way.

"/usr/bin" is the place where system binaries go and you should NOT put your programs there. You "might" put a softlink there pointing at your binary, but thats about all. I'd recommend refraining even from that.

"/usr/local/bin" is the place where you can put things, but you should only put administrative binaries there. If you write a script for trimming log files or creating user accounts and similar things this is the place to put these. Application programs should get a separate directory in "/opt" and a binary (better yet: a link to the binary) in "/opt/bin". Read the aforementioned FHS-document and you will understand what i talk about.

Quote:
Originally Posted by jullee
sudo make install

(why sudo?)
A UNIX system has only one user account which really is allowed to do everything - "root". One should NOT work as root if it is not absolutely necessary. To compile a program can without problems be done as ordinary user. Only to install the program (which is a change to the system) one needs the user "root" and its special power and therefore you switch the effective user for this one command. For this you prepend it by "sudo", which is a software package to do this. Find out more about the "sudo" software by reading its man page or searching the net.

Quote:
Originally Posted by jullee
5) Also following up from question 3, if I DO manage to install all the tools I need to one place on my mac, can I simply scp that file over to the linux cluster and expect it to work? (I'm being optimistic here I know)
No. Unix (MacOS is a UNIX with some graphical gimmicks on top) systems are "source-compatible", not "binary-compatible" like Windows. If a software runs on a Windows machine you can copy it (its binary, the "*.exe" file), put it on another Windws system and it will run there too. Because UNIX runs on a wide range of hardware with different processors, different bus systems, etc.., it is "source-compatible". If you write a software text and it compiles and runs on one system you can take the program text (notice: the text, not the binary!), transfer it to another UNIX system, compile it there and the resulting binary will run there (ahem: in theory :-)) ). But because the processors on the two systems might be different and therefore the binaries will be different even if they come from the same program source.

I hope this helps.

bakunin

Last edited by bakunin; 04-28-2014 at 02:08 PM..
 
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Red Hat

Installing Linux packages

hi, I wants to customize the Linux packages after installting the Linux OS... I am not able to install the packages packages in GUI mode System-> Administration->add/remove software.i want to install some desktop and dabase related packages how do i install those packages... I use RHEL6.2... (1 Reply)
Discussion started by: Rahulne25
1 Replies

2. Solaris

Installing packages in Solaris 11

I want to install EMCpower (EMC Powerpath package) in Solaris 11. At most of the places, I can see procedure to install packages which comes with repository. This is third party tool, I have downloaded it to /var/tmp. How should I install it ? root@orapdps11 # pkg publisher PUBLISHER ... (4 Replies)
Discussion started by: solaris_1977
4 Replies

3. UNIX for Dummies Questions & Answers

help me with basics

hello everyone i have to start with unix as it is a part of my training programme and i have to do a self study, i dont know where to start from. i need some basic questions to be answerd like why we use unix ? what is a terminal? what is an editor? why we write commands inside terminal? these... (4 Replies)
Discussion started by: aryancool
4 Replies

4. Red Hat

Installing rpm packages

Hi guys, I am trying to install some packages for my oracle 11g r2 installation, the below error shows up when I try below: warning: glibc-devel-2.5-24.i386.rpm: Header V3 DSA signature: NOKEY, key ID 37017186 error: Failed dependencies: glibc-headers is needed by... (8 Replies)
Discussion started by: messi777
8 Replies

5. UNIX for Dummies Questions & Answers

Installing deb packages from Ubuntu Server CD

Hi, I have mounted the Ubuntu server edition 10.10 ISO on my server under a directory media/servercd. I would like to install some services from this. I edited the sources.list file to say: deb file:/media/servercd maverick main restricted and it's properly mounted but when I try... (1 Reply)
Discussion started by: shadowcat
1 Replies

6. Emergency UNIX and Linux Support

Installing packages on Redhat 5.5: rpmlib issues

Hi there I'm having trouble with a remote Red Hat server. We are busy with an Oracle 11g installation on this box and going through the list of required packages, etc. The installation required elfutils-libelf-devel-0.148. When I try to install that I get the following error; rpm -i... (6 Replies)
Discussion started by: notreallyhere
6 Replies

7. Slackware

Find Slackware Packages - packages.acl.org.ua

Hi! Let me introduce a project for find and download Slackware packages and browse Slackware repositories. The site provides following features: * Large, daily updated database with RPM, DEB, TGZ, TXZ packages for well-known repositories of the Slackware, Fedora, CentOS, RHEL, Debian,... (2 Replies)
Discussion started by: lystor
2 Replies

8. UNIX for Dummies Questions & Answers

installing packages

hi Guys, relatively new to Unix. i have a list of Unix packages to install... how do i install only what is on that list? can someone help? Kind regards Brian (1 Reply)
Discussion started by: brian112
1 Replies

9. Solaris

Installing European Locale Packages

I have installed the locale package for en_GB and if i do a locale -a (after reboot) it will display: en_GB en_GB.ISO8859-1 en_GB.ISO8859-15 en_GB.ISO8859-15@euro (short version) and i get this when typing only 'locale': # locale LANG=en_GB LC_CTYPE="C" LC_NUMERIC=C LC_TIME="C"... (2 Replies)
Discussion started by: xqtor
2 Replies
Login or Register to Ask a Question