Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

I want to find the difference between two files, only for the header (column names)

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting I want to find the difference between two files, only for the header (column names)
# 8  
Old 09-10-2014
Hi Rudic,

Sorry for the confusion. I apologize for the same. Yes, I was trying with a different input file and what I specified above initially was an example.

But I still have a difficulty in deleting the column with its values after identifying the new column.

Please help.

Thanks & regards
Praveen
# 9  
Old 09-10-2014
Help with what?
# 10  
Old 09-10-2014
Hi Rudic,

Basically I want to identify all the new columns in the feed.txt and delete them along with its values (which means finding column position for all the values) and deleting them so that the data can be loaded for the day without any issues.

Thanks & regards
Praveen
# 11  
Old 09-10-2014
For which I presented a proposal that works with the data you presented.
This User Gave Thanks to RudiC For This Post:
Praveen Pandit
# 12  
Old 09-10-2014
Hi Dave,

Columns can be added anywhere in the data. It may be one or more columns.

Regards
Praveen

---------- Post updated at 08:07 AM ---------- Previous update was at 08:02 AM ----------

Hi RudiC,

Thanks for your proposal and help. Please correct me if I am wrong. When I tried to execute your proposal. It reported only the new column name but did not delete the new column and the associated values.

Best Regards
Praveen
# 13  
Old 09-10-2014
OK. Please show your input file, output file and any error message that may occur.
# 14  
Old 09-10-2014
Hi,

If you want to delete the additional data from the new input file, what is the reason for capturing what the additional column headings were?

Regards

Dave

---------- Post updated at 04:41 PM ---------- Previous update was at 02:34 PM ----------

Hi,

You could try this, using awk - a and b are the file names I used for the test.

Code:
awk -F, '
FILENAME == "a" && FNR == 1 {
        col[$1] = 1
        col[$2] = 1
        col[$3] = 1
        col[$4] = 1
# print "saved col1 = " col[$1], $1
# print "saved col2 = " col[$2], $2
# print "saved col3 = " col[$3], $3
# print "saved col4 = " col[$4], $4
}
FILENAME != "a" && FNR == 1 {
        for ( i=1; i<=NF; i++)
        {
                if ($i in col)
                {
                        # print i " saving to col1"
                        col1 = i
                        i++
                        break
                }
        }
        for ( ;i<=NF;i++)
        {
                if ($i in col)
                {
                        # print $i " saving to col2"
                        col2 = i
                        i++
                        break
                }
        }
        for ( ;i<=NF;i++)
        {
                if ($i in col)
                {
                        # print $i " saving to col3"
                        col3 = i
                        i++
                        break
                }
        }
        for ( ;i<=NF;i++)
        {
                if ($i in col)
                {
                        # print $i " saving to col4"
                        col4 = i
                        i++
                        break
                }
        }
}
FILENAME != "a" && FNR > 1 {
        printf "%s,%s,%s,%s\n", $col1, $col2, $col3, $col4
}
' a b

If you need any further help I'll have to see your input files.

Regards

Dave
This User Gave Thanks to gull04 For This Post:
Praveen Pandit
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Prefix a variable in the first column of all the records of the files with and without header

In a bash shell, I have to prefix a variable to two .CSV files File1.CSV and File2.CSV. One of the files has a header and the other one is with no header in the below format: "value11","value12","value13","value14","value15","value16" "value21","value22","value23","value24","value25","value26"... (7 Replies)
Discussion started by: dhruuv369
7 Replies

2. Shell Programming and Scripting

How to get difference of the same column between two files when other column matches?

File 1: 20130416,235800,10.78.25.104,BR2-loc,60.0,1624,50.0,0,50.0,0 20130416,235800,10.78.25.104,BR1-LOC,70.0,10,50.0,0,70.0,0 20130416,235800,10.78.25.104,Hub_None,60.0,15,60.0,0,50.0,0 File 2: 20130417,000200,10.78.25.104,BR2-loc,60.0,1626,50.0,0,50.0,0... (3 Replies)
Discussion started by: Lakshmikumari
3 Replies

3. Homework & Coursework Questions

Script to find difference between 2 files by column

Hi , i am newbie to shell scripting and am trying to do the below job, A shell script to be run with a command like sh Compare.ksh file1.txt file2.txt 1 2 > file3.txt 1 2-are the key columns Consider the delimiter would be Tab or comma File 1: SK TEST NAME MATHS PHYSICS 21 1 AAA... (1 Reply)
Discussion started by: shakthi666
1 Replies

4. Shell Programming and Scripting

Script to find difference between 2 files by column

Hi , i am newbie to shell scripting and am trying to do the below job, A shell script to be run with a command like sh Compare.ksh file1.txt file2.txt 1 2 > file3.txt 1 2-are the key columns Consider the delimiter would be Tab or comma File 1: SK TEST NAME MATHS PHYSICS 21 1... (1 Reply)
Discussion started by: shakthi666
1 Replies

5. UNIX for Dummies Questions & Answers

Find the average based on similar names in the first column

I have a table, say this: name1 num1 num2 num3 num4 name2 num5 num6 num7 num8 name3 num1 num3 num4 num9 name2 num8 num9 num1 num2 name2 num4 num5 num6 num4 name4 num4 num5 num7 num8 name5 num1 num3 num9 num7 name5 num6 num8 num3 num4 I want a code that will sort my data according... (4 Replies)
Discussion started by: FelipeAd
4 Replies

6. UNIX for Dummies Questions & Answers

Rename a header column by adding another column entry to the header column name

Hi All, I have a file example.csv which looks like this GrpID,TargetID,Signal,Avg_Num CSCH74_1_1,2007,61,256 CSCH74_1_1,212007,647,679 CSCH74_1_1,12007,3,32 CSCH74_1_1,207,299,777 I want the output as GrpID,TragetID,Signal-CSCH74_1_1,Avg_Num CSCH74_1_1,2007,61,256... (1 Reply)
Discussion started by: Vavad
1 Replies

7. Shell Programming and Scripting

Rename a header column by adding another column entry to the header column name URGENT!!

Hi All, I have a file example.csv which looks like this GrpID,TargetID,Signal,Avg_Num CSCH74_1_1,2007,61,256 CSCH74_1_1,212007,647,679 CSCH74_1_1,12007,3,32 CSCH74_1_1,207,299,777 I want the output as GrpID,TragetID,Signal-CSCH74_1_1,Avg_Num CSCH74_1_1,2007,61,256... (4 Replies)
Discussion started by: Vavad
4 Replies

8. Shell Programming and Scripting

find difference in file column...

Hi All, i have a file that is tab delimited. i need help to find the rows which are having same price based on the site code but some times, there are difference so i need to find only the records which are different in all site code. Dept Sec Barcode 10001 10002 10003 10004... (1 Reply)
Discussion started by: malcomex999
1 Replies

9. Shell Programming and Scripting

script to compare first column of two files and find difference

Hi, I want to write a script which will compare the 1st column of both the files and will give the difference. e.g:- my 1st file contains: 89 /usr 52 /usr/local 36 /tmp 92 /opt 96 /home 27 /etc/opt/EMCom 1 ... (3 Replies)
Discussion started by: adityam
3 Replies

10. Shell Programming and Scripting

Column names in flat files

Hi all, I want to create column names in a flat file and then load the data through some other application. For example, I have a file with emp.txt and I need column names as eno,ename,sal in the first line. The delimiter here is comma and record delimiter is end of line or unix new line. Could... (1 Reply)
Discussion started by: srivsn
1 Replies
Login or Register to Ask a Question