Shell script not parsing complete file using AWK


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Shell script not parsing complete file using AWK
# 1  
Old 07-19-2012
Shell script not parsing complete file using AWK

Hi,

I have shell script which will read single edi document and break data between ST & SE to separate files.Below example should create 3 separate files. I have written script with the below command and it is working fine for smaller files.

Code:
awk -F\| -vt=`date +%m%d%y%H%M%S%s` \
  '$1=="ST",$1=="SE"{if($1=="ST"){close(f);f="edi850."t"" ++i} ; $1=$1; print>f}' OFS=\| inputfile

If i run it using large file around 10 MB, then it is not working. It is writing partial file. Please help me with the correct code.

example

Code:
ST|850
BEG|PO|1234
LIN|1|23
SE|4
ST|850
BEG|PO|234
LIN|1|43
LIN|1|41
SE|5
ST|850
BEG|PO|345
LIN|1|53
SE|4


Last edited by jim mcnamara; 07-19-2012 at 03:24 PM.. Reason: Please wrap data and sripts with CodeTags
# 2  
Old 07-19-2012
Some versions of awk do not close files correctly.

If
Code:
ulimit -n

is less than the total number of possible output files then this may be the problem.

What OS and version of awk?
Code:
# some awks
what `which awk`
# GNU awk
awk --version

will give you version information.
# 3  
Old 07-19-2012
OS Version - AIX 3

aix:/home $ what `which awk`
/usr/bin/awk:
61 1.15 src/bos/usr/ccs/lib/libc/__threads_init.c, libcthrd, bos53H, h2006_10B1 3/5/06 16:44:58
62 1.29 src/bos/usr/bin/awk/main.c, cmdawk, bos53V, v2009_02A7 1/5/09 23:22:25

---------- Post updated at 01:44 PM ---------- Previous update was at 01:41 PM ----------

In my file it is having only one ST & SE, so it should create only one file...Please advice the code i need add.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

DB2 in awk or PARSING VALUE in Shell Script

Guys, My Motive is, There are some View names in the file with the format of SCHEMA.VIEWNAMe, I want to read the data from file and parse it to SCHEMA and VIEWNAME and execute DB2 command to get the base Table name. I can parse easily through AWK but i could not execute db2 commands in... (1 Reply)
Discussion started by: Nandy
1 Replies

2. Shell Programming and Scripting

Issue with awk script parsing log file

Hello All, I am trying to parse a log file and i got this code from one of the good forum colleagues, However i realised later there is a problem with this awk script, being naive to awk world wanted to see if you guys can help me out. AWK script: awk '$1 ~ "^WRITER_" {p=1;next}... (18 Replies)
Discussion started by: Ariean
18 Replies

3. Shell Programming and Scripting

awk script and parsing

Hello I am trying to find how to do the following with awk and other tools. Find a line in the file where the element 1 is equal to 12345, from the first occurrence, in that line find the element 11 what is equal to and print all lines in file where their element 11 is equal to the value of... (2 Replies)
Discussion started by: koutroul
2 Replies

4. Shell Programming and Scripting

Wait for one processes to complete in a shell script

Let's say I start process A.sh, then start process B.sh. I call both of them in my C.sh How can I make sure that B starts its execution only after A.sh finishes. I have to do this in loop.Execution time of A.sh may vary everytime. It is a parameterized script. (17 Replies)
Discussion started by: rafa_fed2
17 Replies

5. Shell Programming and Scripting

Waiting for a process to complete in shell script

Hi, I need to initiate a process script which will start and do some processing and then shuts down. Then i need to other verifications. But the the process takes around 25 to 3o minutes. One thing i can monitor the nohup.out file for this process where i can wait for shutting down statement to... (3 Replies)
Discussion started by: Prashanth19
3 Replies

6. Shell Programming and Scripting

parsing using shell script

I have a file parameters.txt which contains 151524 151525 I have another file OID.csv which contains NE Version Object Type ID SDK param name Object OID test1 Start: 4.2 End: 4.2 pan 151524 speed ... (5 Replies)
Discussion started by: LavanyaP
5 Replies

7. Shell Programming and Scripting

Help in parsing a CSV file with Shell script

I have a CSV file which contains number series as one of the fields. Some of the records of the field look like : 079661/3 I have to convert the above series as 079661 079662 079663 and store it as 3 different records. Looking for help on how to achieve this. Am a newbie at Shell... (10 Replies)
Discussion started by: mihirk
10 Replies

8. Shell Programming and Scripting

Shell script for parsing 300mb log file..

am relatively new to Shell scripting. I have written a script for parsing a big file. The logic is: Apart from lot of other useless stuffs, there are many occurances of <abc> and corresponding </abc> tags. (All of them are properly closed) My requirement is to find a particular tag (say... (3 Replies)
Discussion started by: gurpreet470
3 Replies

9. Shell Programming and Scripting

shell script takes long time to complete

Hi all, I wrote this shell script to validate filed numbers for input file. But it take forever to complete validation on a file. The average speed is like 9mins/MB. Can anyone tell me how to improve the performance of a shell script? Thanks (12 Replies)
Discussion started by: ozzman
12 Replies

10. Shell Programming and Scripting

Parsing a file in Shell Script

Hi, I have a requirement. I have an application which can take a file as inputs. Now the file can contain any number of lines. The tool has to pick up the first uncommented line and begin processing it. For example the file could be like this: #MANI123|MANI1234 #MANI234|MANI247... (4 Replies)
Discussion started by: sendhilmani123
4 Replies
Login or Register to Ask a Question