Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Insert FileName to headers

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Insert FileName to headers
# 1  
Old 11-20-2014
Insert FileName to headers
hi there

I have a lot of files with same number of rows and columns.$2 and $3 are the same in all files .
I run awk script and merge $2,$3,$1 from first file and $1 from another files.

I would like to know how to modify awk script to insert filename as header for merged columns ( Expected Result).

Thanks in advance.

Code:
awk 'BEGIN{OFS = "\t"}
  NR == FNR {a[$2 " " $3] = $1; next}
  {a[$2 " " $3] = (a[$2 " " $3] OFS $1)}
  END {for(x in a) print (x OFS a[x])}' *  > Result.csv

FileName:SG1
Code:
  76   3.62945 101.54361  2.000
  76   3.62945 101.54862  2.000
  75   3.62945 101.55362  2.000
  73   3.62945 101.55863  2.000
  76   3.62446 101.54361  2.000
  77   3.62446 101.54862  2.000
  77   3.62446 101.55362  2.000
  76   3.62446 101.55863  2.000
  77   3.61946 101.54361  2.000
  78   3.61946 101.54862  2.000
  78   3.61946 101.55362  2.000
  79   3.61946 101.55863  2.000
  77   3.61447 101.54361  2.000
  78   3.61447 101.54862  2.000
  78   3.61447 101.55362  2.000
  80   3.61447 101.55863  2.000


FileName:SG2
Code:
  74   3.62945 101.54361  2.000
  74   3.62945 101.54862  2.000
  74   3.62945 101.55362  2.000
  74   3.62945 101.55863  2.000
  74   3.62446 101.54361  2.000
  74   3.62446 101.54862  2.000
  74   3.62446 101.55362  2.000
  76   3.62446 101.55863  2.000
  75   3.61946 101.54361  2.000
  75   3.61946 101.54862  2.000
  75   3.61946 101.55362  2.000
  78   3.61946 101.55863  2.000
  75   3.61447 101.54361  2.000
  76   3.61447 101.54862  2.000
  76   3.61447 101.55362  2.000
  79   3.61447 101.55863  2.000


FileName:SG3
Code:
  71   3.62945 101.54361  2.000
  71   3.62945 101.54862  2.000
  71   3.62945 101.55362  2.000
  74   3.62945 101.55863  2.000
  72   3.62446 101.54361  2.000
  72   3.62446 101.54862  2.000
  72   3.62446 101.55362  2.000
  76   3.62446 101.55863  2.000
  72   3.61946 101.54361  2.000
  73   3.61946 101.54862  2.000
  74   3.61946 101.55362  2.000
  78   3.61946 101.55863  2.000
  74   3.61447 101.54361  2.000
  75   3.61447 101.54862  2.000
  75   3.61447 101.55362  2.000
  79   3.61447 101.55863  2.000

Result AWK
Code:
3.62945 101.55362	75	74	71
3.61447 101.55863	80	79	79
3.61447 101.55362	78	76	75
3.61946 101.54862	78	75	73
3.61946 101.54361	77	75	72
3.62945 101.55863	73	74	74
3.61946 101.55863	79	78	78
3.61946 101.55362	78	75	74
3.62446 101.54862	77	74	72
3.62446 101.54361	76	74	72
3.62446 101.55863	76	76	76
3.62446 101.55362	77	74	72
3.62945 101.54862	76	74	71
3.62945 101.54361	76	74	71
3.61447 101.54862	78	76	75
3.61447 101.54361	77	75	74

Expected Result:
Code:
                        SG1     SG2     SG3
3.62945 101.55362	75	74	71
3.61447 101.55863	80	79	79
3.61447 101.55362	78	76	75
3.61946 101.54862	78	75	73
3.61946 101.54361	77	75	72
3.62945 101.55863	73	74	74
3.61946 101.55863	79	78	78
3.61946 101.55362	78	75	74
3.62446 101.54862	77	74	72
3.62446 101.54361	76	74	72
3.62446 101.55863	76	76	76
3.62446 101.55362	77	74	72
3.62945 101.54862	76	74	71
3.62945 101.54361	76	74	71
3.61447 101.54862	78	76	75
3.61447 101.54361	77	75	74

# 2  
Old 11-20-2014
Try (untested):
Code:
awk 'BEGIN          {OFS = "\t"; HD=OFS OFS}
     FNR == 1       {HD=HD OFS FILENAME}
     FNR == NR      {a[$2 " " $3] = $1; next}
                    {a[$2 " " $3] = (a[$2 " " $3] OFS $1)}
     END            {print HD
                     for(x in a) print (x OFS a[x])}
    ' *  > Result.csv

This User Gave Thanks to RudiC For This Post:
ali.seifaddini
# 3  
Old 11-20-2014
Quote:
Originally Posted by RudiC
Try (untested):
Code:
awk 'BEGIN          {OFS = "\t"; HD=OFS OFS}
     FNR == 1       {HD=HD OFS FILENAME}
     FNR == NR      {a[$2 " " $3] = $1; next}
                    {a[$2 " " $3] = (a[$2 " " $3] OFS $1)}
     END            {print HD
                     for(x in a) print (x OFS a[x])}
    ' *  > Result.csv

Thank you for your responding. But your code didn`t work well.

the result:
Code:
SG1130113000247.CAPNFXS.csv.csv    SG1130113001247.CAPNFXT.csv.csv     SG1130113002247.CAPNFXU.csv.csv
76,3.62945,101.54361,2.000 76,3.62945,101.54862,2.000     80,3.61447,101.55863,2.000
         75,3.62945,101.54361,2.000    75,3.62945,101.54862,2.000     75,3.62945,101.55362,2.000    76,3.62945,101.55863,2.000     76,3.62446,101.54361,2.000    76,3.62446,101.54862,2.000     76,3.62446,101.55362,2.000    78,3.62446,101.55863,2.000     77,3.61946,101.54361,2.000    77,3.61946,101.54862,2.000     77,3.61946,101.55362,2.000    81,3.61946,101.55863,2.000     77,3.61447,101.54361,2.000    78,3.61447,101.54862,2.000     78,3.61447,101.55362,2.000    80,3.61447,101.55863,2.000     76,3.62945,101.54361,2.000    76,3.62945,101.54862,2.000     75,3.62945,101.55362,2.000    73,3.62945,101.55863,2.000     76,3.62446,101.54361,2.000    77,3.62446,101.54862,2.000     77,3.62446,101.55362,2.000    76,3.62446,101.55863,2.000     77,3.61946,101.54361,2.000    78,3.61946,101.54862,2.000     78,3.61946,101.55362,2.000    79,3.61946,101.55863,2.000     77,3.61447,101.54361,2.000    78,3.61447,101.54862,2.000     78,3.61447,101.55362,2.000    80,3.61447,101.55863,2.000     74,3.62945,101.54361,2.000    74,3.62945,101.54862,2.000     74,3.62945,101.55362,2.000    74,3.62945,101.55863,2.000     74,3.62446,101.54361,2.000    74,3.62446,101.54862,2.000     74,3.62446,101.55362,2.000    76,3.62446,101.55863,2.000     75,3.61946,101.54361,2.000    75,3.61946,101.54862,2.000     75,3.61946,101.55362,2.000    78,3.61946,101.55863,2.000     75,3.61447,101.54361,2.000    76,3.61447,101.54862,2.000     76,3.61447,101.55362,2.000    79,3.61447,101.55863,2.000
SG1130113001247.CAPNFXT.csv.csv SG1130113002247.CAPNFXU.csv.csv     SG1130113000247.CAPNFXS.csv.csv

# 4  
Old 11-20-2014
If you are using a Solaris/SunOS System, change awk in the script to /usr/xpg4/bin/awk, /usr/xpg6/bin/awk, or nawk.

No matter what system you're using, change the last line of the script from:
Code:
    ' *  > Result.csv

to:
Code:
    ' SG[1-3]  > Result.csv

or, if your filenames aren't really SG1, SG2, and SG3 as you said, but are instead named SG1130113000247.CAPNFXS.csv.csv, SG1130113001247.CAPNFXT.csv.csv, and SG1130113002247.CAPNFXU.csv.csv, respectively, use something like:
Code:
    ' SG*.csv.csv  > Result.csv

If this doesn't fix your problem, let us know what OS (including the version) and shell you're using to give us some hint as to what might be wrong.
This User Gave Thanks to Don Cragun For This Post:
ali.seifaddini
# 5  
Old 11-20-2014
Quote:
Originally Posted by ali.seifaddini
. . . Thank you for your responding. But your code didn`t work well.
. . .

I'd say my code does work VERY well.

Using your sample code on your sample files, I get
Code:
awk 'BEGIN{OFS = "\t"}
  NR == FNR {a[$2 " " $3] = $1; next}
  {a[$2 " " $3] = (a[$2 " " $3] OFS $1)}
  END {for(x in a) print (x OFS a[x])}' SG1 SG2 SG3
3.61447 101.55362    78    76    75
3.61946 101.54361    77    75    72
3.62446 101.54361    76    74    72
3.61946 101.55362    78    75    74
3.62446 101.55362    77    74    72
3.61447 101.54862    78    76    75
3.62945 101.54361    76    74    71
3.61447 101.55863    80    79    79
3.62945 101.55362    75    74    71
3.61946 101.54862    78    75    73
3.62446 101.54862    77    74    72
3.61946 101.55863    79    78    78
3.62446 101.55863    76    76    76
3.62945 101.54862    76    74    71
3.62945 101.55863    73    74    74
3.61447 101.54361    77    75    74

Applying my proposal to your sample files, I get
Code:
awk 'BEGIN          {OFS = "\t"; HD=OFS OFS}
     FNR == 1       {HD=HD OFS FILENAME}
     FNR == NR      {a[$2 " " $3] = $1; next}
                    {a[$2 " " $3] = (a[$2 " " $3] OFS $1)}
     END            {print HD
                     for(x in a) print (x OFS a[x])}
    ' SG1 SG2 SG3
                     SG1   SG2   SG3
3.61447 101.55362    78    76    75
3.61946 101.54361    77    75    72
3.62446 101.54361    76    74    72
3.61946 101.55362    78    75    74
3.62446 101.55362    77    74    72
3.61447 101.54862    78    76    75
3.62945 101.54361    76    74    71
3.61447 101.55863    80    79    79
3.62945 101.55362    75    74    71
3.61946 101.54862    78    75    73
3.62446 101.54862    77    74    72
3.61946 101.55863    79    78    78
3.62446 101.55863    76    76    76
3.62945 101.54862    76    74    71
3.62945 101.55863    73    74    74
3.61447 101.54361    77    75    74

Whatever strange results you show above are from applying the code to data not adhering to your specifications.

EDIT: BTW, looks like your data files are comma delimited, which you did not mention in your spec.
This User Gave Thanks to RudiC For This Post:
ali.seifaddini
# 6  
Old 11-20-2014
Quote:
Originally Posted by RudiC
Try (untested):
Code:
awk 'BEGIN          {OFS = "\t"; HD=OFS OFS}
     FNR == 1       {HD=HD OFS FILENAME}
     FNR == NR      {a[$2 " " $3] = $1; next}
                    {a[$2 " " $3] = (a[$2 " " $3] OFS $1)}
     END            {print HD
                     for(x in a) print (x OFS a[x])}
    ' *  > Result.csv

I added comma in script, and it works fine. Thanks you so much.

Code:
awk 'BEGIN          {OFS = "\t"; HD=OFS","OFS}
     FNR == 1       {HD=HD OFS FILENAME}
     FNR == NR      {a[$2 " " $3] = $1; next}
                    {a[$2 " " $3] = (a[$2 " " $3] OFS $1)}
     END            {print HD
                     for(x in a) print (x OFS a[x])}
    ' *  > Result2.csv

# 7  
Old 11-20-2014
Quote:
Originally Posted by ali.seifaddini
I added comma in script, and it works fine. Thanks you so much.

Code:
awk 'BEGIN          {OFS = "\t"; HD=OFS","OFS}
     FNR == 1       {HD=HD OFS FILENAME}
     FNR == NR      {a[$2 " " $3] = $1; next}
                    {a[$2 " " $3] = (a[$2 " " $3] OFS $1)}
     END            {print HD
                     for(x in a) print (x OFS a[x])}
    ' *  > Result2.csv

That change only affects the 1st line of the output. With your sample input, it changes the header from:
Code:
			SG1	SG2	SG3

to:
Code:
	,		SG1	SG2	SG3

If your input file is comma delimited rather than whitespace delimited, the way to fix it would be to change the 1st line of the script to something like:
Code:
awk 'BEGIN          {FS=","; OFS = "\t"; HD=OFS OFS}

This User Gave Thanks to Don Cragun For This Post:
ali.seifaddini
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Insert the line number from text file to filename output

Hi everyone :) I have a file "words.txt" containing hundreds of lines of text. Each line contains a slogan. Using the code below i am able to generate an image with the slogan text from each line. The image filename is saved matching the last word on each line. Example: Line 1: We do... (2 Replies)
Discussion started by: martinsmith
2 Replies

2. Shell Programming and Scripting

Insert Filename into Multiple Files

Hi, We have a folder that has files in the following structure abc.sql def.sql efg.sql . . . xyz.sql I have to find a certain string (say "test") in each file and replace it with the name of the file. For eg. if "test" is present in abc.sql, I want to replace it with "test abc". If... (8 Replies)
Discussion started by: jerome_rajan
8 Replies

3. Shell Programming and Scripting

insert part of a filename in a textfile

Hi All, I've got a textfile that i've stripped and edited using sed and want to insert part of the filename (original filename consists of: fw0204.txt.ncat_report.txt) but there are multiple files being delivered by an application to the data directory The part of the filename that I need to... (0 Replies)
Discussion started by: etquart
0 Replies

4. Shell Programming and Scripting

BASH Batch renaming insert additional zero into filename

Hi all, Wondering how this could be accomplished........ a directory contains sequentially numbered files from fw01 to fw999. How would I insert an additional zero so that the directory lists these files in a proper manner? (i.e. all double digit files from fw01 to fw99 would become... (3 Replies)
Discussion started by: putter1900
3 Replies

5. Shell Programming and Scripting

insert filename into each line of multiple files

I need to insert <filename + comma> into each line of multiple files. Any idea how to script that? Regards, Manu (5 Replies)
Discussion started by: linux.yahoo
5 Replies

6. Shell Programming and Scripting

Filename from splitting files to have the same filename of the original file with counter value

Hi all, I have a list of xml file. I need to split the files to a different files when see the <ko> tag. The list of filename are B20090908.1100-20090908.1200_CDMA=1,NO=2,SITE=3.xml B20090908.1200-20090908.1300_CDMA=1,NO=2,SITE=3.xml B20090908.1300-20090908.1400_CDMA=1,NO=2,SITE=3.xml ... (3 Replies)
Discussion started by: natalie23
3 Replies

7. Shell Programming and Scripting

Merging of files with different headers to make combined headers file

Hi , I have a typical situation. I have 4 files and with different headers (number of headers is varible ). I need to make such a merged file which will have headers combined from all files (comman coluns should appear once only). For example - File 1 H1|H2|H3|H4 11|12|13|14 21|22|23|23... (1 Reply)
Discussion started by: marut_ashu
1 Replies

8. Shell Programming and Scripting

Remove text between headers while leaving headers intact

Hi, I'm trying to strip all lines between two headers in a file: ### BEGIN ### Text to remove, contains all kinds of characters ... Antispyware-Downloadserver.com (Germany)=http://www.antispyware-downloadserver.c om/updates/ Antispyware-Downloadserver.com #2... (3 Replies)
Discussion started by: Trones
3 Replies

9. Shell Programming and Scripting

insert filename into file using SED (or AWK)

Hi, I would like to insert a file's filename into the first line of that file - for a batch of files. Is this possible using SED? Thanks in advance. (2 Replies)
Discussion started by: USER#5
2 Replies

10. UNIX for Dummies Questions & Answers

Insert date/time within a filename

Hi Guys, I need to script the renaming of files as followins: files: firstjd secondjo thirdjv My script needs to insert the date/time infront of the last 2 characters of the filenames above, any ideas greatly received :) the letters before the last 2 characters could change, I'm only... (7 Replies)
Discussion started by: cooperman
7 Replies
Login or Register to Ask a Question