Login or Register to Ask a Question and Join Our Community


UNIX for Advanced & Expert Users

Help using awk with a text file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users Help using awk with a text file
# 1  
Old 03-27-2013
Help using awk with a text file
Basically I have a text file that is made up of customer statements. I want to be able to separate the customer statements into their own file. The file is consistant in that the account # (part of the key) is always on line 10 then on every 55th line from there on out so my logic would be like: Read file put 10th line for charcters 1-5 in variable (account #), compare that to line 65 and if the same, read the next key (65 + 55), if they are different then take 10 lines less and write output using key plus date so key would be 90313.txt for Act 9 on March 2013 statement. I know how to use awk to get it to show x line like awk NR==10 example.txt but I need to know the script to keep this ongoing and to output each statement to its own file.
example.txt (4.3 KB) 
# 2  
Old 03-27-2013
Can you post some highlighted input data for people to consider please. Expected output would be useful along with any code you have tried so far.

Feel free to sanitise the data, of course.



Robin
Liverpool/Blackburn
UK
# 3  
Old 03-27-2013
sample data and example of result in 90313.txt for act 9
Quote:
Originally Posted by rbatte1
Can you post some highlighted input data for people to consider please. Expected output would be useful along with any code you have tried so far.

Feel free to sanitise the data, of course.



Robin
Liverpool/Blackburn
UK
ABC PARTS COMPANY
100 WILDFLOWER TRAIL
MT PLEASANT SC 29579
800-555-1212

03/27/13
9 ABC TOWING CO.
123 ANY LANE ST
ACME PA 15610-1084


PAGE 1 OF 1
02/05/13 PB 1PB 15.45 15.45
02/05/13 PB 2PB 1029.87 1045.32














1045.32
15.45 1029.87



ABC PARTS COMPANY
100 WILDFLOWER TRAIL
MT PLEASANT SC 29579
800-555-1212

03/27/13
103 TONY CEILA
100 ROCKY ROAD
LATROBE PA 15650


PAGE 1 OF 1
02/05/13 PB 4PB 460.00 460.00















460.00
460.00



ABC PARTS COMPANY
100 WILDFLOWER TRAIL
MT PLEASANT SC 29579
800-555-1212

03/27/13
105 RICHARD FELTS
R D 1 BOX 196
MIDDLETOWN NY 10925


PAGE 1 OF 1
02/05/13 PB 4PB 721.56 721.56















721.56
721.56



ABC PARTS COMPANY
100 WILDFLOWER TRAIL
MT PLEASANT SC 29579
800-555-1212

03/27/13
119 A&J HAULING
300 LANE ST
LANCASTER SC 29578


PAGE 1 OF 1
02/05/13 PB 1PB 188.17 188.17















188.17
188.17



ABC PARTS COMPANY
100 WILDFLOWER TRAIL
MT PLEASANT SC 29579
800-555-1212

03/27/13





PAGE 1 OF 1
















2415.05
203.62 1029.87 1181.56




FIN CHG
CB
90313.txt (667 Bytes) 
# 4  
Old 03-28-2013
Update on AWK
Quote:
Originally Posted by rbatte1
Can you post some highlighted input data for people to consider please. Expected output would be useful along with any code you have tried so far.

Feel free to sanitise the data, of course.



Robin
Liverpool/Blackburn
UK
I don't have anything written, I am not sure how to write awk statement that will read multiple lines and then output to a file. Basically I am reading line 10, characters c1-5 and putting into a variable ($A) to compare to line 65 characters c1-5 ($NA) and if different then I want to write lines 1 to 55 to file using $A plus month year so file would be 90313.txt. If $A matches on line 65 then read line 120 and compare, if different output lines 1 to 110 to file 90313.txt if not read next 175 line and compare...continue on and on until EOF.
# 5  
Old 04-01-2013
You aren't using CODE tags (which are required when posting code and data to this forum), so we can't identify the lines and characters you're talking about.

You still haven't highlighted the 5 characters on line 10 that you want to compare to line 65 even though this was requested 4 days ago.

Line 10 in all of your input samples appears to be an empty line following the address line:
Code:
ACME PA 15610-1084

If we can't match your description of what you're trying to do to the input you're giving us to work with; we can't help.
This User Gave Thanks to Don Cragun For This Post:
jim mcnamara
# 6  
Old 04-01-2013
Example of file with CODE around example
This is a sample of the file with line numbers assigned for easier reading, acct number is on line 10 for the first statement in positions 1 thru 5 then the account # appears on every 55th line after the first account. Each statement is 55 lines long. The goal is to read each statement and write to a separate file for each new statement using the account number as part of the key to the file, the first statement in the example below would be 90313.txt, 2nd statement would be 1000313.txt and so on. I am a newbie at posting so sorry for the first posts...thanks for the assistances!
Code:
 1  ^M                 ABC PARTS COMPANY             ^M
 2                   100 WILDFLOWER TRAIL          ^M
 3                   MT PLEASANT SC 29579          ^M
 4                          800-555-1212
 5
 6
 7
 8  ^M                                       04/01/13^M
 9  ^M
10      9    ABC TOWING CO.                  ^M
11           123 ANY LANE ST                 ^M
12           ACME PA 15610-1084              ^M
13                                           ^M
14  ^M
15  ^M
16                                                    PAGE   1 OF   1^M
17  ^M
18  02/05/13         PB  1PB           15.45                    15.45 ^M
19  02/05/13         PB  2PB         1029.87                  1045.32
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48  ^M^M
49                                                            1045.32 ^M
50  ^M
51                   15.45     1029.87                         ^M
52  ^M
53                                   STATEMENT MESSAGE HERE          ^M
54                                   PLEASE RETURN TO US             ^M
55  ^M
56  ^L^M^M                 ABC PARTS COMPANY             ^M
57                   100 WILDFLOWER TRAIL          ^M
58                   MT PLEASANT SC 29579          ^M
59                          800-555-1212
60
61
62
63  ^M                                       04/01/13^M
64  ^M
65    100    MIRIAM ACKERMAN                 ^M
66           193 FESS ROAD                   ^M
67           LATROBE PA  15650               ^M
68                                           ^M
69  ^M
70  ^M
71                                                    PAGE   1 OF   1^M
72  ^M
73  03/29/13         PD    1234        50.00                    50.00 ^M
74  03/29/13         PC                           25.00-        25.00
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103  ^M^M
104                                                              25.00 ^M
105  ^M
106      25.00                                                  ^M
107  ^M
108                                   STATEMENT MESSAGE HERE          ^M
109                                   PLEASE RETURN TO US             ^M
110  ^M
111  ^L^M^M                 ABC PARTS COMPANY             ^M
112                   100 WILDFLOWER TRAIL          ^M
113                   MT PLEASANT SC 29579          ^M
114                          800-555-1212
115
116
117
118  ^M                                       04/01/13^M
119  ^M
120    103    TONY CEILA                      ^M
121           100 ROCKY ROAD                  ^M
122           LATROBE PA 15650                ^M
123                                           ^M
124  ^M
125  ^M
126                                                    PAGE   1 OF   1^M
127  ^M
128  02/05/13         PB  4PB          460.00                   460.00
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158  ^M^M
159                                                             460.00 ^M
160  ^M
161                                                      460.00 ^M
162  ^M
163                                   STATEMENT MESSAGE HERE          ^M
164                                   PLEASE RETURN TO US             ^M
165  ^M
166  ^L^M^M                 ABC PARTS COMPANY             ^M
167                   100 WILDFLOWER TRAIL          ^M
168                   MT PLEASANT SC 29579          ^M
169                          800-555-1212
170
171
172
173  ^M                                       04/01/13^M
174  ^M
175    105    RICHARD FELTS                   ^M
176           R D 1 BOX 196                   ^M
177           MIDDLETOWN NY 10925             ^M
178                                           ^M
179  ^M
180  ^M
181                                                    PAGE   1 OF   1^M
182  ^M
183  02/05/13         PB  4PB          721.56                   721.56
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
207
208
209
210
211
212
213  ^M^M
214                                                             721.56 ^M
215  ^M
216                                                      721.56 ^M
217  ^M
218                                   STATEMENT MESSAGE HERE          ^M
219                                   PLEASE RETURN TO US             ^M
220  ^M

# 7  
Old 04-01-2013
Quote:
Originally Posted by ziggy6
This is a sample of the file with line numbers assigned for easier reading, acct number is on line 10 for the first statement in positions 1 thru 5 then the account # appears on every 55th line after the first account. Each statement is 55 lines long. The goal is to read each statement and write to a separate file for each new statement using the account number as part of the key to the file, the first statement in the example below would be 90313.txt, 2nd statement would be 1000313.txt and so on. I am a newbie at posting so sorry for the first posts...thanks for the assistances!
Code:
 1  ^M                 ABC PARTS COMPANY             ^M
 2                   100 WILDFLOWER TRAIL          ^M
 3                   MT PLEASANT SC 29579          ^M
 4                          800-555-1212
 5
 6
 7
 8  ^M                                       04/01/13^M
 9  ^M
10      9    ABC TOWING CO.                  ^M
11           123 ANY LANE ST                 ^M
12           ACME PA 15610-1084              ^M
13                                           ^M
14  ^M
15  ^M
16                                                    PAGE   1 OF   1^M
17  ^M
18  02/05/13         PB  1PB           15.45                    15.45 ^M
19  02/05/13         PB  2PB         1029.87                  1045.32
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48  ^M^M
49                                                            1045.32 ^M
50  ^M
51                   15.45     1029.87                         ^M
52  ^M
53                                   STATEMENT MESSAGE HERE          ^M
54                                   PLEASE RETURN TO US             ^M
55  ^M
56  ^L^M^M                 ABC PARTS COMPANY             ^M
57                   100 WILDFLOWER TRAIL          ^M
58                   MT PLEASANT SC 29579          ^M
59                          800-555-1212
60
61
62
63  ^M                                       04/01/13^M
64  ^M
65    100    MIRIAM ACKERMAN                 ^M
66           193 FESS ROAD                   ^M
67           LATROBE PA  15650               ^M
68                                           ^M
69  ^M
70  ^M
71                                                    PAGE   1 OF   1^M
72  ^M
73  03/29/13         PD    1234        50.00                    50.00 ^M
74  03/29/13         PC                           25.00-        25.00
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103  ^M^M
104                                                              25.00 ^M
105  ^M
106      25.00                                                  ^M
107  ^M
108                                   STATEMENT MESSAGE HERE          ^M
109                                   PLEASE RETURN TO US             ^M
110  ^M
111  ^L^M^M                 ABC PARTS COMPANY             ^M
112                   100 WILDFLOWER TRAIL          ^M
113                   MT PLEASANT SC 29579          ^M
114                          800-555-1212
115
116
117
118  ^M                                       04/01/13^M
119  ^M
120    103    TONY CEILA                      ^M
121           100 ROCKY ROAD                  ^M
122           LATROBE PA 15650                ^M
123                                           ^M
124  ^M
125  ^M
126                                                    PAGE   1 OF   1^M
127  ^M
128  02/05/13         PB  4PB          460.00                   460.00
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158  ^M^M
159                                                             460.00 ^M
160  ^M
161                                                      460.00 ^M
162  ^M
163                                   STATEMENT MESSAGE HERE          ^M
164                                   PLEASE RETURN TO US             ^M
165  ^M
166  ^L^M^M                 ABC PARTS COMPANY             ^M
167                   100 WILDFLOWER TRAIL          ^M
168                   MT PLEASANT SC 29579          ^M
169                          800-555-1212
170
171
172
173  ^M                                       04/01/13^M
174  ^M
175    105    RICHARD FELTS                   ^M
176           R D 1 BOX 196                   ^M
177           MIDDLETOWN NY 10925             ^M
178                                           ^M
179  ^M
180  ^M
181                                                    PAGE   1 OF   1^M
182  ^M
183  02/05/13         PB  4PB          721.56                   721.56
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
207
208
209
210
211
212
213  ^M^M
214                                                             721.56 ^M
215  ^M
216                                                      721.56 ^M
217  ^M
218                                   STATEMENT MESSAGE HERE          ^M
219                                   PLEASE RETURN TO US             ^M
220  ^M

This is a major step in the right direction, but the output file names still have not been specified. Do you really want leading spaces on the output file names when customer numbers are less than five digits? It is easy to put them in (as requested), but it will make handling these files harder for you. Would you prefer to have leading zeroes added so the filename always starts with a 5 digit customer number?

Where do the last 4 digits of the output file names come from? They don't come from the dates on lines 8, 18, or 19 in your 55 line statements. Are they always supposed to be 0313? Are they supposed to be the two digit month and two digit year corresponding to the date when the script is run?

Are the multiple occurrences of the strings "^L" and "^M" in you input file literal characters that you want kept in the output files? Are they a graphic representation of form feed and carriage return characters that you want kept in the output files? Or, are they a graphic representation of form feed and carriage return characters that you want to be stripped from the output?

If there are multiple statements for a given customer number, are they always adjacent records in the input file?

What shell do you want to use and what operating system are you using?
This User Gave Thanks to Don Cragun For This Post:
ziggy6
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Match text to lines in a file, iterate backwards until text or text substring matches, print to file

hi all, trying this using shell/bash with sed/awk/grep I have two files, one containing one column, the other containing multiple columns (comma delimited). file1.txt abc12345 def12345 ghi54321 ... file2.txt abc1,text1,texta abc,text2,textb def123,text3,textc gh,text4,textd... (6 Replies)
Discussion started by: shogun1970
6 Replies

2. UNIX for Beginners Questions & Answers

awk to update file with partial matching line in another file and append text

In the awk below I am trying to cp and paste each matching line in f2 to $3 in f1 if $2 of f1 is in the line in f2 somewhere. There will always be a match (usually more then 1) and my actual data is much larger (several hundreds of lines) in both f1 and f2. When the line in f2 is pasted to $3 in... (4 Replies)
Discussion started by: cmccabe
4 Replies

3. Shell Programming and Scripting

Splitting a text file into smaller files with awk, how to create a different name for each new file

Hello, I have some large text files that look like, putrescine Mrv1583 01041713302D 6 5 0 0 0 0 999 V2000 2.0928 -0.2063 0.0000 N 0 0 0 0 0 0 0 0 0 0 0 0 5.6650 0.2063 0.0000 N 0 0 0 0 0 0 0 0 0 0 0 0 3.5217 ... (3 Replies)
Discussion started by: LMHmedchem
3 Replies

4. Shell Programming and Scripting

awk to reformat text file

Howdy. AWK beginner here. I need to reformat a text file in the following format: TTGS08-2014001 6018.00 143563.00 ... (2 Replies)
Discussion started by: c47v3770
2 Replies

5. Shell Programming and Scripting

Search and replace from file in awk using a 16 bit text file

Hello, Some time ago a helpful awk file was provided on the forum which I give below: NR==FNR{A=$0;next}{for(j in A){split(A,P,"=");for(i=1;i<=NF;i++){if($i==P){$i=P}}}}1 While it works beautifully on English and Latin characters i.e. within the ASCII range of 127, the moment a character beyond... (6 Replies)
Discussion started by: gimley
6 Replies

6. Shell Programming and Scripting

Awk to convert a text file to CSV file with some string manipulation

Hi , I have a simple text file with contents as below: 12345678900 971,76 4234560890 22345678900 5971,72 5234560990 32345678900 71,12 6234560190 the new csv-file should be like: Column1;Column2;Column3;Column4;Column5 123456;78900;971,76;423456;0890... (9 Replies)
Discussion started by: FreddyDaKing
9 Replies

7. UNIX for Advanced & Expert Users

Help using Awk and cut with a text file

Looking for some help on using awk and cut I have a text file that has fixed information and want to write a script that will prompt the user for an account to search for and pint the output The sample line that has the key information looks like this: Statement to: ... (5 Replies)
Discussion started by: ziggy6
5 Replies

8. Shell Programming and Scripting

search text file in file if this file contains necessary text (awk,grep)

Hello friends! Help me pls to write correct awk and grep statements for my task: I have got files with name filename.txt It has such structure: Start of file FROM: address@domen.com (12...890) abc DATE: 11/23/2009 on Std SUBJECT: any subject End of file So, I must check, if this file... (4 Replies)
Discussion started by: candyme
4 Replies

9. Shell Programming and Scripting

awk to reformat a text file

I am definitely not an expert with awk, and I want to reformat a text file like the following. This is probably a very easy one for an expert out there. I would like to keep the lines in the same order, but move the heading to only be listed once above the lines. This is what the text file... (7 Replies)
Discussion started by: linux4life
7 Replies

10. Shell Programming and Scripting

process text file with awk

I have a text file which represent a http packet: header1 haeder2 ..... ..... headern payload I need to count bytes in the payload. How can I get it using awk? Thanks in advance Andrea Musella (2 Replies)
Discussion started by: littleboyblu
2 Replies
Login or Register to Ask a Question