Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Take out First Line and merge all files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Take out First Line and merge all files
# 1  
Old 08-28-2012
Take out First Line and merge all files
Hi Gurus,
I have n number of files. Data which is in the files have column headers. I need to take them out and merge into one file.
Can you help please?
I need to do that little urgent.
Thanks
# 2  
Old 08-28-2012
Code:
for file in $listOfNfiles
do
  tail +2 $file >> outputfile
done

This will create "outputfile" containing everything from line 2 onwards from each of your files.
This User Gave Thanks to Smiling Dragon For This Post:
raopatwari
# 3  
Old 08-29-2012
Try this...
Code:
for file in $file_list
do
  awk ' NR != 1 ' $file >> New_File
done

This User Gave Thanks to pamu For This Post:
raopatwari
# 4  
Old 08-29-2012
A small nitpick, there is a small difference in performance between
Code:
for ...;do cmd >> file;done

and
Code:
for ...;do cmd; done > file

The latter only opens the output file once.
With GNU sed, you can do
Code:
sed -s 1d $file_list > merged-file

This User Gave Thanks to binlib For This Post:
Smiling Dragon
# 5  
Old 08-29-2012
one more using awk

Code:
for file in $file_list
do 
  awk ' NR>1 ' $file >> New_File 
done

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Merge files with multi-line stanzas

Hello. I try to merge few /etc/qconfig files in AIX (from few servers into one file on new server). Could you please help me what would be the best way to merge files that contain multi-line stanzas, like this: stanza1: attr1 = value1 attr2 = value2 attr3 = value3 stanza2: attr1 =... (7 Replies)
Discussion started by: kareem33
7 Replies

2. UNIX for Dummies Questions & Answers

Merge two files line by line

Hello, i want to merge two files via terminal. After three lines of file one i will append a line of file two: Example: File 1: @chr1_15953153_15953093_1_0_0_0_1:0:0_0:0:0_0/1 TGTCGTAAAATCACAACGTCCCCATCTTTCAAGCCATAAGCAGCCAATGATCTGTGGTTGTCTGTGAGAGGTCTTTCCGCATAGACGATCTACAACAAGA +... (3 Replies)
Discussion started by: Innsbrucker
3 Replies

3. Shell Programming and Scripting

Merge multiple lines to one line when line starts with and ends with

example: comment Now_TB.table column errac is for error messages 1 - first 2 - second 3 -third ; in this example I need to be able to grab the comment as first word and ; as the last word and it might span a few lines. I need it to be put all in one line without line breaks so I can... (4 Replies)
Discussion started by: wambli
4 Replies

4. Shell Programming and Scripting

Merge two files line by line and column by column

Hi All, I have two files having oracle query result. I want to merge to files line by line and also with column File1 23577|SYNC TYPE 23578|Order Number|ConnectionState 23585|Service State|Service NameFile2 23577|AR Alarm Sync 23578|A5499|9 23585|7|test_nov7Result... (18 Replies)
Discussion started by: Harshal22
18 Replies

5. UNIX for Dummies Questions & Answers

loop? print max column in each line for 800 files and merge

Hello, I have 800 or so files with 3 columns each and >10000 lines each. For each file and each line I would like to print the maximum column number for each line. Then I would like to 'paste' each of these files together (column-wise) so that the file with expression in label '_1' is the... (6 Replies)
Discussion started by: peanuts48
6 Replies

6. UNIX for Dummies Questions & Answers

Merge files and add file name to the end of each line

Hello everybody, I'm trying to merge a lot of files, but I want to include the filename to the end of each line. I've tried to use cat, but I got stuck. My files are for example: file01.001 123456 aaa ddd ee 458741 eee fff ee file02.003 478596 uuu ddd ee 145269 ttt fff ee ... (4 Replies)
Discussion started by: ernesto561
4 Replies

7. Shell Programming and Scripting

Merge 2 files, line by line

Hi all, I've been looking for a way to merge 2 simple text files, line by line using the shell: For example: File1: aaaa bbbb cccc File2: 1111 2222 3333 The 2 files have the same number of lines. I'm trying to obtain: aaaa 1111 bbbb 2222 cccc 3333 (0 Replies)
Discussion started by: venezia
0 Replies

8. Shell Programming and Scripting

merge files based on line by line

Hi, lets assume the following details.. file 1 has below details abc|1234|xyz abc1|1234|xyz1 abc2|1234|xyz2 and file 2 has below details pqr|124|lskd ebwer|325|dfb wf|234|sdb I need out put shown below abc|1234|xyz pqr|124|lskd abc1|1234|xyz1 ebwer|325|dfb abc2|1234|xyz2... (4 Replies)
Discussion started by: alnhk
4 Replies

9. Shell Programming and Scripting

How to use Perl to merge multi-line into single line

Hi, Can anyone know how to use perl to merge the following multi-line information which beginning with "BAM" into one line. For each line need to delete the return and add a space. Please see the red color line. ******Org. Multi-line) BAM admin 101.203.57.22 ... (3 Replies)
Discussion started by: happyday
3 Replies

10. Shell Programming and Scripting

compare the column from 3 files and merge that line

I have 3 file, each of has got 80000 records. file1.txt ----------------------- ABC001;active;modify;accept; ABC002;notactive;modify;accept; ABC003;notactive;no-modify;accept; ABC004;active;modify;accept; ABC005;active;no-modify;accept; file2.txt ---------------------------... (8 Replies)
Discussion started by: ganesh_mak
8 Replies
Login or Register to Ask a Question