Sponsored Content
Top Forums Shell Programming and Scripting Get common lines from multiple files Post 302437733 by dr.house on Friday 16th of July 2010 03:52:12 AM
Old 07-16-2010
Possibly a point to start from:

Code:
[house@leonov] sed -i fileA -e 's/.*$/& \t fileA/g'
[house@leonov] sed -i fileB -e 's/.*$/& \t fileB/g'
[house@leonov] cat fileA fileB >> common
[house@leonov] sort common
chr1    110751878    NP_006393.2    HBXIP    P45L        fileB
chr1    116944164    NP_001533.2    IGSF3    R671W       fileB
chr1    246803952    NP_001001821.1    OR2T34    A244T   fileB
chr1    31237964    NP_001018494.1    PUM1    M340L      fileA
chr1    31237964    NP_055491.1    PUM1    M340L         fileA
chr1    33251518    NP_001616.1    AK2    H191D          fileA
chr1    33251518    NP_001616.1    AK2    H191D          fileB
chr1    33251518    NP_037543.1    AK2    H191D          fileA
chr1    57027345    NP_001004303.2    C1orf168    P270S          fileA
chr1    57027345    NP_001004303.2    C1orf168    P270S          fileB
chr1    89606840    NP_940862.2    GBP6    R48C          fileB

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

To find all common lines from 'n' no. of files

Hi, I have one situation. I have some 6-7 no. of files in one directory & I have to extract all the lines which exist in all these files. means I need to extract all common lines from all these files & put them in a separate file. Please help. I know it could be done with the help of... (11 Replies)
Discussion started by: The Observer
11 Replies

2. Shell Programming and Scripting

Common lines from files

Hello guys, I need a script to get the common lines from two files with a criteria that if the first two columns match then I keep the maximum value of the 3rd column.(tab separated columns) Sample input: file1: 111 222 0.1 333 444 0.5 555 666 0.4 file 2: 111 222 0.7 555 666... (5 Replies)
Discussion started by: jaysean
5 Replies

3. Shell Programming and Scripting

Common lines from files

Hello guys, I need a script to get the common lines from two files with a criteria that if the first two columns match then I keep the maximum value of the 5th column.(tab separated columns) . 3rd and 4th columns corresponds to the row which has highest value for the 5th column. Sample... (2 Replies)
Discussion started by: jaysean
2 Replies

4. Shell Programming and Scripting

Merge multiple lines in same file with common key using awk

I've been a Unix admin for nearly 30 years and never learned AWK. I've seen several similar posts here, but haven't been able to adapt the answers to my situation. AWK is so damn cryptic! ;) I have a single file with ~900 lines (CSV list). Each line starts with an ID, but with different stuff... (6 Replies)
Discussion started by: protosd
6 Replies

5. Shell Programming and Scripting

Find common lines between multiple files

Hello everyone A few years Ago the user radoulov posted a fancy solution for a problem, which was about finding common lines (gene variation names) between multiple samples (files). The code was: awk 'END { for (R in rec) { n = split(rec, t, "/") if (n > 1) dup = dup ?... (5 Replies)
Discussion started by: bibb
5 Replies

6. Shell Programming and Scripting

Compare multiple files, and extract items that are common to ALL files only

I have this code awk 'NR==FNR{a=$1;next} a' file1 file2 which does what I need it to do, but for only two files. I want to make it so that I can have multiple files (for example 30) and the code will return only the items that are in every single one of those files and ignore the ones... (7 Replies)
Discussion started by: castrojc
7 Replies

7. UNIX for Dummies Questions & Answers

Filter lines common in two files

Thanks everyone. I got that problem solved. I require one more help here. (Yes, UNIX definitely seems to be fun and useful, and I WILL eventually learn it for myself. But I am now on a different project and don't really have time to go through all the basics. So, I will really appreciate some... (6 Replies)
Discussion started by: latsyrc
6 Replies

8. Shell Programming and Scripting

Join common patterns in multiple lines into one line

Hi I have a file like 1 2 1 2 3 1 5 6 11 12 10 2 7 5 17 12 I would like to have an output as 1 2 3 5 6 10 7 11 12 17 any help would be highly appreciated Thanks (4 Replies)
Discussion started by: Harrisham
4 Replies

9. Shell Programming and Scripting

Join columns across multiple lines in a Text based on common column using BASH

Hello, I have a file with 2 columns ( tableName , ColumnName) delimited by a Pipe like below . File is sorted by ColumnName. Table1|Column1 Table2|Column1 Table5|Column1 Table3|Column2 Table2|Column2 Table4|Column3 Table2|Column3 Table2|Column4 Table5|Column4 Table2|Column5 From... (6 Replies)
Discussion started by: nv186000
6 Replies

10. Shell Programming and Scripting

Find common lines between all of the files in one folder

Could it be possible to find common lines between all of the files in one folder? Just like comm -12 . So all of the files two at a time. I would like all of the outcomes to be written to a different files, and the file names could be simply numbers - 1 , 2 , 3 etc. All of the file names contain... (19 Replies)
Discussion started by: Eve
19 Replies
File::Copy(3pm) 					 Perl Programmers Reference Guide					   File::Copy(3pm)

NAME
File::Copy - Copy files or filehandles SYNOPSIS
use File::Copy; copy("file1","file2") or die "Copy failed: $!"; copy("Copy.pm",*STDOUT); move("/dev1/fileA","/dev2/fileB"); use File::Copy "cp"; $n = FileHandle->new("/a/file","r"); cp($n,"x"); DESCRIPTION
The File::Copy module provides two basic functions, "copy" and "move", which are useful for getting the contents of a file from one place to another. copy The "copy" function takes two parameters: a file to copy from and a file to copy to. Either argument may be a string, a FileHandle reference or a FileHandle glob. Obviously, if the first argument is a filehandle of some sort, it will be read from, and if it is a file name it will be opened for reading. Likewise, the second argument will be written to (and created if need be). Trying to copy a file on top of itself is a fatal error. If the destination (second argument) already exists and is a directory, and the source (first argument) is not a filehandle, then the source file will be copied into the directory specified by the destination, using the same base name as the source file. It's a failure to have a filehandle as the source when the destination is a directory. Note that passing in files as handles instead of names may lead to loss of information on some operating systems; it is recommended that you use file names whenever possible. Files are opened in binary mode where applicable. To get a consistent behaviour when copying from a filehandle to a file, use "binmode" on the filehandle. An optional third parameter can be used to specify the buffer size used for copying. This is the number of bytes from the first file, that will be held in memory at any given time, before being written to the second file. The default buffer size depends upon the file, but will generally be the whole file (up to 2MB), or 1k for filehandles that do not reference files (eg. sockets). You may use the syntax "use File::Copy "cp"" to get at the "cp" alias for this function. The syntax is exactly the same. The behavior is nearly the same as well: as of version 2.15, <cp> will preserve the source file's permission bits like the shell utility cp(1) would do, while "copy" uses the default permissions for the target file (which may depend on the process' "umask", file ownership, inherited ACLs, etc.). If an error occurs in setting permissions, "cp" will return 0, regardless of whether the file was successfully copied. move The "move" function also takes two parameters: the current name and the intended name of the file to be moved. If the destination already exists and is a directory, and the source is not a directory, then the source file will be renamed into the directory specified by the destination. If possible, move() will simply rename the file. Otherwise, it copies the file to the new location and deletes the original. If an error occurs during this copy-and-delete process, you may be left with a (possibly partial) copy of the file under the destination name. You may use the "mv" alias for this function in the same way that you may use the <cp> alias for "copy". syscopy File::Copy also provides the "syscopy" routine, which copies the file specified in the first parameter to the file specified in the second parameter, preserving OS-specific attributes and file structure. For Unix systems, this is equivalent to the simple "copy" routine, which doesn't preserve OS-specific attributes. For VMS systems, this calls the "rmscopy" routine (see below). For OS/2 systems, this calls the "syscopy" XSUB directly. For Win32 systems, this calls "Win32::CopyFile". Special behaviour if "syscopy" is defined (OS/2, VMS and Win32): If both arguments to "copy" are not file handles, then "copy" will perform a "system copy" of the input file to a new output file, in order to preserve file attributes, indexed file structure, etc. The buffer size parameter is ignored. If either argument to "copy" is a handle to an opened file, then data is copied using Perl operators, and no effort is made to preserve file attributes or record structure. The system copy routine may also be called directly under VMS and OS/2 as "File::Copy::syscopy" (or under VMS as "File::Copy::rmscopy", which is the routine that does the actual work for syscopy). rmscopy($from,$to[,$date_flag]) The first and second arguments may be strings, typeglobs, typeglob references, or objects inheriting from IO::Handle; they are used in all cases to obtain the filespec of the input and output files, respectively. The name and type of the input file are used as defaults for the output file, if necessary. A new version of the output file is always created, which inherits the structure and RMS attributes of the input file, except for owner and protections (and possibly timestamps; see below). All data from the input file is copied to the output file; if either of the first two parameters to "rmscopy" is a file handle, its position is unchanged. (Note that this means a file handle pointing to the output file will be associated with an old version of that file after "rmscopy" returns, not the newly created version.) The third parameter is an integer flag, which tells "rmscopy" how to handle timestamps. If it is < 0, none of the input file's timestamps are propagated to the output file. If it is > 0, then it is interpreted as a bitmask: if bit 0 (the LSB) is set, then timestamps other than the revision date are propagated; if bit 1 is set, the revision date is propagated. If the third parameter to "rmscopy" is 0, then it behaves much like the DCL COPY command: if the name or type of the output file was explicitly specified, then no timestamps are propagated, but if they were taken implicitly from the input filespec, then all timestamps other than the revision date are propagated. If this parameter is not supplied, it defaults to 0. Like "copy", "rmscopy" returns 1 on success. If an error occurs, it sets $!, deletes the output file, and returns 0. RETURN
All functions return 1 on success, 0 on failure. $! will be set if an error was encountered. AUTHOR
File::Copy was written by Aaron Sherman <ajs@ajs.com> in 1995, and updated by Charles Bailey <bailey@newman.upenn.edu> in 1996. perl v5.16.3 2013-03-04 File::Copy(3pm)
All times are GMT -4. The time now is 03:17 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy