Sponsored Content
Full Discussion: Compare and Merge files
Top Forums Shell Programming and Scripting Compare and Merge files Post 302752769 by nua7 on Monday 7th of January 2013 01:20:46 PM
Old 01-07-2013
Thanks LMHmedchem But I need to do this in awk or sed.

Also found another way of doing it , incase it is helpful for others.


Code:
awk -v input_file=file1 'BEGIN { FS="|"; while((getline<input_file)>0){sub1=substr($0,1,11); subArr[sub1]=""}}{ if ($1 in subArr) print $0;}' file2  > file3


.

Last edited by Scrutinizer; 01-07-2013 at 02:28 PM.. Reason: icode tags to code tags
This User Gave Thanks to nua7 For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Compare two files and merge columns in a third

Hi, I'm working with snmp, with a little script I'm able to obtain from a switch a list with a couple of values with this format Port Mac 1 00:0A:0B:0C:0D:0E .... (hundred of entries) Now with a simple arp on a router I am able to obtain another list 00:0A:0B:0C:0D:0E... (20 Replies)
Discussion started by: CM64
20 Replies

2. Shell Programming and Scripting

compare the column from 3 files and merge that line

I have 3 file, each of has got 80000 records. file1.txt ----------------------- ABC001;active;modify;accept; ABC002;notactive;modify;accept; ABC003;notactive;no-modify;accept; ABC004;active;modify;accept; ABC005;active;no-modify;accept; file2.txt ---------------------------... (8 Replies)
Discussion started by: ganesh_mak
8 Replies

3. Shell Programming and Scripting

Merge files of differrent size with one field common in both files using awk

hi, i am facing a problem in merging two files using awk, the problem is as stated below, file1: A|B|C|D|E|F|G|H|I|1 M|N|O|P|Q|R|S|T|U|2 AA|BB|CC|DD|EE|FF|GG|HH|II|1 .... .... .... file2 : 1|Mn|op|qr (2 Replies)
Discussion started by: shashi1982
2 Replies

4. UNIX for Dummies Questions & Answers

compare columns from 2 files and merge

Dear all, Being new to Unix i have a problem. I have 2 files: File 1: 118,1,0,2,3,0,5,0.3,0,0.3,0.6,1 118,2,1,2,2,0,5,0.4,0,0.4,0.4,1 118,4,2,0,3,0,5,0.7,0,0.3,0.6,1 118,6,4,1,0,0,5,0.8,0,0.2,0,1 File 2: 118,1,BFGL-NGS-109695,3610326,0,18,1,0.556,0.389,0.056,0.25,0.8183... (2 Replies)
Discussion started by: samwilkinson
2 Replies

5. Shell Programming and Scripting

Require compare command to compare 4 files

I have four files, I need to compare these files together. As such i know "sdiff and comm" commands but these commands compare 2 files together. If I use sdiff command then i have to compare each file with other which will increase the codes. Please suggest if you know some commands whcih can... (6 Replies)
Discussion started by: nehashine
6 Replies

6. Shell Programming and Scripting

Shell Scripting: Compare pattern in two files and merge the o/p in one.

one.txt ONS.1287677000.820.log 20Oct2010 ONS.1287677000.123.log 21Oct2010 ONS.1287677000.456.log 22Oct2010 two.txt ONS.1287677000.820.log:V AC CC EN ONS.1287677000.123.log:V AC CC EN ONS.1287677000.820.log:V AC CC EN In file two.txt i have to look for pattern which column one... (17 Replies)
Discussion started by: saluja.deepak
17 Replies

7. Shell Programming and Scripting

AWK compare/merge

File1 2917,`0722,RDF1+TDEV,90(6),33,03E:0_12E:0,10000000c96c4af3_10000000c96c6e88,BL_db00p01e 2917,`0781,RDF1+TDEV,100(5),33,03E:0_12E:0,10000000c96c4af3_10000000c96c6e88,BL_db00p01e File2 2917,`0722,RDF1+TDEV,90(6),03E:0_12E:0,10000000c96c4af3_10000000c96c6e88,BL_db00p01e... (4 Replies)
Discussion started by: greycells
4 Replies

8. Shell Programming and Scripting

Checking in a directory how many files are present and basing on that merge all the files

Hi, My requirement is,there is a directory location like: :camp/current/ In this location there can be different flat files that are generated in a single day with same header and the data will be different, differentiated by timestamp, so i need to verify how many files are generated... (10 Replies)
Discussion started by: srikanth_sagi
10 Replies

9. Shell Programming and Scripting

Compare two files and merge into third

Hello: Newbie with Awk. Trying to compare two files and merge data based on CID. Please see the input file format and desired output. Any help is appreciated. TIA Input File1 CID1 --- TYP1 --- DCN1 --- INDATE1 --- IN-DATA1 CID2 --- TYP2 --- DCN2 --- INDATE2 --- IN-DATA2 CID3 ---... (6 Replies)
Discussion started by: wincrazy
6 Replies

10. Shell Programming and Scripting

Compare and merge two big CSV files

Hi all, i need help. I have two csv files with a huge amount of data. I need the first column of the first file, to be compared with the data of the second, to have at the end a file with the data not present in the second file. Example File1: (only one column) profile_id 57036226... (11 Replies)
Discussion started by: SirMannu
11 Replies
funmerge(1)							SAORD Documentation						       funmerge(1)

NAME
funmerge - merge one or more Funtools table files SYNOPSIS
funmerge [-w|-x] -f [colname] <iname1> <iname2> ... <oname> OPTIONS
-f # output a column specifying file from which this event came -w # adjust position values using WCS info -x # adjust position values using WCS info and save old values DESCRIPTION
funmerge merges FITS data from one or more FITS Binary Table files or raw event files. The first argument to the program specifies the first input FITS table or raw event file. If "stdin" is specified, data are read from the standard input. Use Funtools Bracket Notation to specify FITS extensions and row filters. Subsequent arguments specify additional event files and tables to merge. (NB: Stdin cannot not be used for any of these additional input file arguments.) The last argument is the out- put FITS file. The columns in each input table must be identical. If an input file begins with the '@' character, it is processed as an include file, i.e., as a text file containing event file names (as well as blank lines and/or comment lines starting with the '#' sign). If standard input is specified as an include file ('@stdin'), then file names are read from the standard input until EOF (^D). Event files and include files can be mixed on a command line. Rows from each table are written sequentially to the output file. If the switch -f [colname] is specified on the command line, an addi- tional column is added to each row containing the number of the file from which that row was taken (starting from one). In this case, the corresponding file names are stored in the header parameters having the prefix FUNFIL, i.e., FUNFIL01, FUNFIL02, etc. Using the -w switch (or -x switch as described below), funmerge also can adjust the position column values using the WCS information in each file. (By position columns, we mean the columns that the table is binned on, i.e., those columns defined by the bincols= switch, or (X,Y) by default.) To perform WCS alignment, the WCS of the first file is taken as the base WCS. Each position in subsequent files is adjusted by first converting it to the sky coordinate in its own WCS coordinate system, then by converting this sky position to the sky position of the base WCS, and finally converting back to a pixel position in the base system. Note that in order to perform WCS alignment, the appropriate WCS and TLMIN/TLMAX keywords must already exist in each FITS file. When performing WCS alignment, you can save the original positions in the output file by using the -x (for "xtra") switch instead of the -w switch (i.e., using this switch also implies using -w) The old positions are saved in columns having the same name as the original positional columns, with the added prefix "OLD_". Examples: Merge two tables, and preserve the originating file number for each row in the column called "FILE" (along with the corresponding file name in the header): [sh] funmerge -f "FILE" test.ev test2.ev merge.ev Merge two tables with WCS alignment, saving the old position values in 2 additional columns: [sh] funmerge -x test.ev test2.ev merge.ev This program only works on raw event files and binary tables. We have not yet implemented image and array merging. SEE ALSO
See funtools(7) for a list of Funtools help pages version 1.4.2 January 2, 2008 funmerge(1)
All times are GMT -4. The time now is 06:37 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy