Sponsored Content
Top Forums UNIX for Dummies Questions & Answers Merging tables: identifiying common and unique elements Post 302822491 by lsantome on Monday 17th of June 2013 05:09:20 PM
Old 06-17-2013
Hi MadeInGermany, thank you for your quick reply!

A1: Yes, every table is contained in a single file. I merge them two by two, based on their filename (pattern) with the following code:

Code:
for sample in `for file in *.tab; do echo ${file/_*/}; done | sort | uniq`; do
    cat $sample* \
    | cut -f1-33 \
    | sort -u -k2,2 \
    > $sample.tab
done

Explanation:
- The pattern defines which files are going to be merged
- Open files and select columns 1 to 33
- Sort rows based on column 2, removing duplicates
- Create an output file based on the pattern used in step one.

A2: No, identical lines do not have the same line number

Thank you again

Best,

lsantome
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

find common elements in 2 files (for loop)

Hi, i'm new here (and to scripting too). I was hoping for some help in comparing two files. i have a file called 'file1' with a list of names in the following format: adam jones paul higgins kelly lowe i also have another file which may contain some of the names but with a lot of... (4 Replies)
Discussion started by: ibking
4 Replies

2. Shell Programming and Scripting

Merging two files with a common column

Hi, I have two files file1 and file2. I have to merge the columns of those two files into file3 based on common column of two files. To be simple. file1: Row-id name1 13456 Rahul 16789 Vishal 18901 Karan file2 : Row-id place 18901 Mumbai ... (2 Replies)
Discussion started by: manneni prakash
2 Replies

3. UNIX for Dummies Questions & Answers

Merging Tables by a column

Dear Friends, I really do not know Linux and I really would like to understand it because it does help to work with large data. I am reading this forum for 1 week to try a solution for my problem. I think that, using others post informations, I was almost there... I have 2 big tables... (4 Replies)
Discussion started by: lColli
4 Replies

4. Shell Programming and Scripting

Merging 2 files based on a common column

Hi All, I do have 2 files file 1 has 4 tab delimited columns 234 a c dfgyu 294 b g fih 302 c h jzh 328 z c san 597 f g son File 2 has 2 tab delimted columns 234 23 302 24 597 24 I want to merge file 2 with file 1 based on the data common in both files which is the first column so... (6 Replies)
Discussion started by: Lucky Ali
6 Replies

5. Shell Programming and Scripting

Creating array with non-duplicate / unique elements in ksh

Hi all, I have created 3 arrays which can have common elements in each like- arr_a contains str1 str2 str3 str4 str5 arr_b contains str3 str6 str7 str1 str8 arr_c contains str4 str9 str10 str2 each array is created with "set -A arr_name values" command. I want to create a resultant array-say... (1 Reply)
Discussion started by: sanzee007
1 Replies

6. Shell Programming and Scripting

Merging files with common IDs without JOIN

Hi, I am trying to merge information across 2 files. The first file is a "master" file, with all IDS. File 2 contains a subset of IDs of those in File 1. I would like to match up individuals in File 1 and File 2, and add information in File 2 to that of File 1 if they appear. However, if an... (3 Replies)
Discussion started by: hubleo
3 Replies

7. Shell Programming and Scripting

Merging two files without any common pattern

Hi I have file1 as IJU_NSOMOW; SOWWOD_TWUIQ; and file2 as how are you?; fine there; Now my problem is i need the output file as IJU_NSOMOW; how are you?; SOWWOD_TWUIQ; fine there; (2 Replies)
Discussion started by: Priya Amaresh
2 Replies

8. Shell Programming and Scripting

Count common elements in a column

HI, I have a 3-column tab separated column (approx 1GB) in which I would like to count and output the frequency of all of the common elements in the 1st column. For instance: If my input was the following: dot is-big 2 dot is-round 3 dot is-gray 4 cat is-big 3 hot in-summer 5 My... (4 Replies)
Discussion started by: owwow14
4 Replies

9. Shell Programming and Scripting

Merging two tables including multiple ocurrence of column identifiers and unique lines

I would like to merge two tables based on column 1: File 1: 1 today 1 green 2 tomorrow 3 red File 2: 1 a lot 1 sometimes 2 at work 2 at home 2 sometimes 3 new 4 a lot 5 sometimes 6 at work (4 Replies)
Discussion started by: BSP
4 Replies

10. Shell Programming and Scripting

Get unique elements from Array

I have an array code and output is below: echo $1 while read -r fline; do echo "%%%%%%$fline%%%%%" fmy_array+=("$fline") done <<< "$1" Output: CR30903 YU0007 SRIL CR30903 Yogesh SRIL %%%%%%CR30903 YU0007 SRIL%%%%% %%%%%%CR30903 Yogesh SRIL%%%%% ... (8 Replies)
Discussion started by: mohtashims
8 Replies
GIT-MERGE-FILE(1)						    Git Manual							 GIT-MERGE-FILE(1)

NAME
git-merge-file - Run a three-way file merge SYNOPSIS
git merge-file [-L <current-name> [-L <base-name> [-L <other-name>]]] [--ours|--theirs|--union] [-p|--stdout] [-q|--quiet] [--marker-size=<n>] <current-file> <base-file> <other-file> DESCRIPTION
git merge-file incorporates all changes that lead from the <base-file> to <other-file> into <current-file>. The result ordinarily goes into <current-file>. git merge-file is useful for combining separate changes to an original. Suppose <base-file> is the original, and both <current-file> and <other-file> are modifications of <base-file>, then git merge-file combines both changes. A conflict occurs if both <current-file> and <other-file> have changes in a common segment of lines. If a conflict is found, git merge-file normally outputs a warning and brackets the conflict with lines containing <<<<<<< and >>>>>>> markers. A typical conflict will look like this: <<<<<<< A lines in file A ======= lines in file B >>>>>>> B If there are conflicts, the user should edit the result and delete one of the alternatives. When --ours, --theirs, or --union option is in effect, however, these conflicts are resolved favouring lines from <current-file>, lines from <other-file>, or lines from both respectively. The length of the conflict markers can be given with the --marker-size option. The exit value of this program is negative on error, and the number of conflicts otherwise. If the merge was clean, the exit value is 0. git merge-file is designed to be a minimal clone of RCS merge; that is, it implements all of RCS merge's functionality which is needed by git(1). OPTIONS
-L <label> This option may be given up to three times, and specifies labels to be used in place of the corresponding file names in conflict reports. That is, git merge-file -L x -L y -L z a b c generates output that looks like it came from files x, y and z instead of from files a, b and c. -p Send results to standard output instead of overwriting <current-file>. -q Quiet; do not warn about conflicts. --ours, --theirs, --union Instead of leaving conflicts in the file, resolve conflicts favouring our (or their or both) side of the lines. EXAMPLES
git merge-file README.my README README.upstream combines the changes of README.my and README.upstream since README, tries to merge them and writes the result into README.my. git merge-file -L a -L b -L c tmp/a123 tmp/b234 tmp/c345 merges tmp/a123 and tmp/c345 with the base tmp/b234, but uses labels a and c instead of tmp/a123 and tmp/c345. GIT
Part of the git(1) suite Git 1.7.10.4 11/24/2012 GIT-MERGE-FILE(1)
All times are GMT -4. The time now is 04:55 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy