Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: awk, string as record separator, transposing rows into columns
Top Forums Shell Programming and Scripting awk, string as record separator, transposing rows into columns Post 302513539 by spindoctor on Wednesday 13th of April 2011 01:54:59 PM
Old 04-13-2011
awk, string as record separator, transposing rows into columns
I'm working on a different stage of a project that someone helped me address elsewhere in these threads.
The .docs I'm cycling through look roughly like this:

Code:
1 of 26 DOCUMENTS


Copyright 2010 The Age Company Limited
All Rights Reserved 
The Age (Melbourne, Australia)

November 27, 2010 Saturday  
First Edition

SECTION: NEWS; In Brief Overseas; Pg. 16

LENGTH: 114 words

There are about four .doc files I'm cycling through and I need to get them to look like this:
Code:
1 of 26 Documents, Copyright 2010 etc, All Rights Reserved, The Age (Melbourne, Australia), November 27, 2010
2 of 26 Documents, Copyright 2010 etc, All Rights Reserved, The Age (Melbourne, Australia), November 27, 2010
3 of 26 Documents, Copyright 2010 etc, All Rights Reserved, The Age (Melbourne, Australia), November 27, 2010

I'm working with the following script
Code:
#!/usr/bin/sh
for i in /Users/simon/canadian/*
do
  textutil -convert txt $i 
done

for i in /Users/simon/canadian/*
do
  sed -i "" '/^$/d' $i  
done

rm /Users/simon/canadian/*.doc

for i in /Users/simon/canadian/*
do
  grep -A5 'DOCUMENTS$' $i > $i.tmp
  mv $i.tmp $i
done

for i in /Users/simon/canadian/*
do
  gawk '$1=$1' RS="DOCUMENTS$" FS="\n" OFS=, $i >$i.tmp
  mv $i.tmp $i
done

It's doing what I want, EXCEPT, the output is all being put on one line:
1 of 26 DOCUMENTS,Copyright 2010 The Age Company Limited,All Rights Reserved ,--,2 of 26 DOCUMENTS,Copyright 2010 Nationwide News Pty Limited,All Rights Reserved ,--,3 of 26 DOCUMENTS,Copyright 2010 John Fairfax Publications Pty Ltd,All Rights Reserved ,--

I know that the double-dash is inserted by AWK when it encounters a a new record separator, so it is encountering the "DOCUMENTS$" record separator properly, but it's not outputting the text on separate lines.
Can someone provide any insight?
Simon
Last edited by Franklin52; 04-13-2011 at 03:11 PM.. Reason: Please indent your code and use code tags
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Rows to columns transposing and reformating.

----File attached. Input file =========== COL_1 <IP Add 1> COL_2 <Service1> COL_3 <ABCDEFG> COL_4 <IP ADD:PORT> COL_4 <IP ADD:PORT> COL_1 <IP Add 2> COL_2 <Service2> COL_2 <Service3> COL_2 <Service4> COL_3 <AAAABBB> COL_4 <IP ADD:PORT> COL_4 <IP ADD:PORT> COL_4 <IP... (27 Replies)
Discussion started by: bluethunder
27 Replies

2. Shell Programming and Scripting

Transposing columns with awk

I want a sweet simple time efficient awk script in online which gets output 001_r 0.0265185 0.0437049 0.0240642 0.0310264 0.0200482 0.0146746 0.0351344 0.0347856 0.036119 1.49 firstcoloumnvalue allvaluesof 'c' in one row 001_r : 002_r c: 0.0265185 N: 548 001_r : 007_r c:... (5 Replies)
Discussion started by: phoenix_nebula
5 Replies

3. Shell Programming and Scripting

Transposing rows into columns

I have a file like the one given below P1|V1|V2 P1|V1|V3 P1V1|V2 P2|V1|V4 P2|V2|V6 P2|V1|V4 I want it convert to P1|V1|V2|V2|V3 P2|V1|V4|V2|V6 2nd and 3rd column should be considered as together and so the tird row is duplicate Any ideas? (3 Replies)
Discussion started by: prasperl
3 Replies

4. Shell Programming and Scripting

Transposing Repeated Rows to Columns.

I have 1000s of these rows that I would like to transpose to columns. However I would like the transpose every 3 consecutive rows to columns like below, sorted by column 3 and provide a total for each occurrences. Finally I would like a grand total of column 3. 21|FE|41|0B 50\65\78 15... (2 Replies)
Discussion started by: ravzter
2 Replies

5. Shell Programming and Scripting

Help for a Perl newcomer! Transposing data from columns to rows

I have to create a Perl script which will transpose the data output from my experiment, from columns to rows, in order for me to analyse the data. I am a complete Perl novice so any help would be greatly appreciated. The data as it stands looks like this: Subject Condition Fp1 ... (12 Replies)
Discussion started by: Sarah_W
12 Replies

6. Shell Programming and Scripting

transposing columns into rows

Hi, I need to transpose columns of my files into rows and save it as individual files. sample contents of the file below. 0.9120 0.7782 0.6959 0.6904 0.6322 0.8068 0.9082 0.9290 0.7272 0.9870 0.7648 0.8053 0.8300 0.9520 0.8614 0.6734 0.7910 0.6413 0.7126 0.7364 0.8491 0.8868 0.7586 0.8949... (8 Replies)
Discussion started by: ida1215
8 Replies

7. Shell Programming and Scripting

Transposing rows and columns (pivoting) using shell scripting

Here is the contents of an input file. A,1,2,3,4 10,aaa,bbb,ccc,ddd 11,eee,fff,ggg,hhh 12,iii,jjj,lll,mmm 13,nnn,ooo,ppp I wanted the output to be A 10 1 aaa 10 2 bbb 10 3 ccc 10 4 ddd 11 1 eee 11 2 fff 11 3 ggg 11 4 hhh ..... and so on How to do it in ksh... (9 Replies)
Discussion started by: ksatish89
9 Replies

8. Shell Programming and Scripting

Transposing rows to columns with multiple similar lines

Hi, I am trying to transpose rows to columns for thousands of records. The problem is there are records that have the same lines that need to be separated. the input file as below:- ID 1A02_HUMAN AC P01892; O19619; P06338; P10313; P30444; P30445; P30446; P30514; AC Q29680; Q29837;... (2 Replies)
Discussion started by: redse171
2 Replies

9. Shell Programming and Scripting

Use string as Record separator in awk

Hello to all, Please some help on this. I have the file in format as below. How can I set the record separator as the string below in red "No. Time Source Destination Protocol Length Info" I've tried code below but it doesn't seem to... (6 Replies)
Discussion started by: cgkmal
6 Replies

10. Shell Programming and Scripting

Pivot Rows to Columns, with field separator

Hi All, I have a requirement to flatten data out, based on the value in COL_2. Our file is pipe delimited, however COL_2 contains a comma separated string, which we would like to pivot out from one row into multiple rows. Please see my example input data below: Input Data ... (4 Replies)
Discussion started by: RichZR
4 Replies

LEARN ABOUT LINUX

igawk

IGAWK(1)							 Utility Commands							  IGAWK(1)

NAME

       igawk - gawk with include files

SYNOPSIS

       igawk [ all gawk options ] -f program-file [ -- ] file ...
       igawk [ all gawk options ] [ -- ] program-text file ...

DESCRIPTION

       Igawk is a simple shell script that adds the ability to have ``include files'' to gawk(1).

       AWK programs for igawk are the same as for gawk, except that, in addition, you may have lines like

	      @include getopt.awk

       in your program to include the file getopt.awk from either the current directory or one of the other directories in the search path.

OPTIONS

       See gawk(1) for a full description of the AWK language and the options that gawk supports.

EXAMPLES

       cat << EOF > test.awk
       @include getopt.awk

       BEGIN {
	    while (getopt(ARGC, ARGV, "am:q") != -1)
		 ...
       }
       EOF

       igawk -f test.awk

SEE ALSO

       gawk(1)

       Effective AWK Programming, Edition 1.0, published by the Free Software Foundation, 1995.

AUTHOR

       Arnold Robbins (arnold@skeeve.com).

Free Software Foundation					    Nov 3 1999								  IGAWK(1)
All times are GMT -4. The time now is 07:02 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy