Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

awk dynamically append columns into file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk dynamically append columns into file
# 1  
Old 09-03-2014
awk dynamically append columns into file
Hi, I have a large data file, want to separate it into 100 part and export one specific field as a file; then I want to append each part's column into one file. How to realize that?

Code:
1 2 3 1
2 3 4 2
2 3 4 3
3 3 4 4
3 4 5 5
3 4 5 6

I want the last column of the data file, e.g divide it into 3 parts, so the final file I want is like
Code:
1 3 5
2 4 6

Last edited by Scrutinizer; 09-03-2014 at 03:32 PM.. Reason: code tags
# 2  
Old 09-03-2014
Not sure what you're looking for...can you be more clear about your requirements...
This User Gave Thanks to shamrock For This Post:
wanliushao
# 3  
Old 09-03-2014
Not sure either. To just get the last column en put it in three columns, try:
Code:
awk '{print $NF}' file | pr -s -t3

This User Gave Thanks to Scrutinizer For This Post:
wanliushao
# 4  
Old 09-03-2014
Code:
awk '{print $NF}' file | pr -s -t3

Thanks~ This works, but the data file is very large, when try to use this to divide it into 20000 parts, the error shows: width too narrow. What should be the solutions?
Moderator's Comments:
Mod Comment Please use CODE tags when displaying sample input, sample output, and sample code segments.
Last edited by Don Cragun; 09-03-2014 at 04:12 PM.. Reason: Add CODE tags.
# 5  
Old 09-03-2014
What do you mean with 20000 parts?
# 6  
Old 09-03-2014
I think pr needs a page width to match you number of fields if you use -t20000 you may need to specify a width to match

Code:
awk '{print $NF}' file | pr -s -t20000 -W300000

---------- Post updated at 10:38 AM ---------- Previous update was at 10:22 AM ----------

After some playing around with a test file this might get you what you require:

Code:
awk '{print $NF}' infile | pr -s" " -t -20000 -W100000 | sed -r 's/[[:space:]]+/ /g'

sed command removes extra spaces that pr seems to insert
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

How to print multiple required columns dynamically in a file using the header name?

Hi All, i am trying to print required multiple columns dynamically from a fie. But i am able to print only one column at a time. i am new to shell script, please help me on this issue. i am using below script awk -v COLT=$1 ' NR==1 { for (i=1; i<=NF; i++) { ... (2 Replies)
Discussion started by: balu1234
2 Replies

2. UNIX for Beginners Questions & Answers

awk to update file with partial matching line in another file and append text

In the awk below I am trying to cp and paste each matching line in f2 to $3 in f1 if $2 of f1 is in the line in f2 somewhere. There will always be a match (usually more then 1) and my actual data is much larger (several hundreds of lines) in both f1 and f2. When the line in f2 is pasted to $3 in... (4 Replies)
Discussion started by: cmccabe
4 Replies

3. Shell Programming and Scripting

Shell script to apply functions to multiple columns dynamically

Hello, I have a requirement to apply hashing algorithm on flat file on one or more columns dynamically based on header sample input file ID|NAME|AGE|GENDER 10|ABC|30|M 20|DEF|20|F say if i want multiple columns based on the header example id,name or id,age or name,gender and hash and... (13 Replies)
Discussion started by: mkathi
13 Replies

4. Shell Programming and Scripting

awk to Sum columns when other column has duplicates and append one column value to another with Care

Hi Experts, Please bear with me, i need help I am learning AWk and stuck up in one issue. First point : I want to sum up column value for column 7, 9, 11,13 and column15 if rows in column 5 are duplicates.No action to be taken for rows where value in column 5 is unique. Second point : For... (1 Reply)
Discussion started by: as7951
1 Replies

5. Shell Programming and Scripting

Append awk results into file or array

for a in {1..100} do awk '{ sum+=$a} END {print sum}' a=$a file1 > file2 done I know I will get only one number if following the code above, how can I get 100 sum numbers in file2? (2 Replies)
Discussion started by: wanliushao
2 Replies

6. Shell Programming and Scripting

awk to append data to a file

Hi I search for certain values in a file across many directories using the following awk code awk '/Sl.*thickness/ {Sl=$3;Tt=$NF}END{print Sl, Tt}' DILAT.DAT What I would like to do is write out Sl and Tt obtained from these files from many directories to a single file. So for example if... (2 Replies)
Discussion started by: lost.identity
2 Replies

7. Shell Programming and Scripting

Append file from ref file AWK

FILE1 abc:xxx:abc:123:wer:AAA:12 csf:xxx:123:aeg:sar:BBB:13 asq:yer:321:wsa:qqq:CCC:14 FILE2 AAA:12:SET1:R1 AAA:12:SSS1:RR1 AAA:11:SET4:R3 BBB:13:SET2:R2 OUTPUT abc:xxx:abc:123:wer:AAA:12:SET1:R1:SSS1:RR1 csf:xxx:123:aeg:sar:BBB:13:SET2:R2::... (4 Replies)
Discussion started by: greycells
4 Replies

8. Shell Programming and Scripting

append an output file with two columns

Hi All, can you help me with this: grep XXX dir/*.txt|wc -l > newfile.txt - this put the results in the newfile.txt, but I want to add another column in the newfile.txt, string 'YYYYY', separated somehow, which corresponds on the grep results? For example grep will grep XXX dir/*.txt|wc -l >... (5 Replies)
Discussion started by: apenkov
5 Replies

9. UNIX for Dummies Questions & Answers

Using awk (or whatever) to pull and append data in a new file

One of the fortunate things about posting in a "Dummies" forum is you probably aren't expecting a lot of out me... I'm trying to pull fields from two lines in the same file(s), and then append them together in a new file. So...I get a nice line-by-line of the first bit of data I'm looking... (6 Replies)
Discussion started by: Milano_EH3
6 Replies

10. Shell Programming and Scripting

sendmail subject to append dynamically.

Hi, I have to use sendmail from command line, and we need to append a static string in subject to a new value from file.Is there any way to do it. Thanks Shrikrishna (1 Reply)
Discussion started by: shrikrishna
1 Replies
Login or Register to Ask a Question