Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Awk: System command not working in awk

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Awk: System command not working in awk
# 1  
Old 11-19-2014
Awk: System command not working in awk
Hi,
I have around 10 files in a folder in which I want to change the file format from tab(\t) to pipe(|) with some changes in the fields as well. Below is the code, while tmp file is getting generated but move command is not working, please help

Following is the code

Code:
 awk -F"\t" '{print $1,$2,substr($3,1,2)substr($3,4,2)substr($3,7,2),substr($4,7,4)substr($4,1,2)substr($4,4,2),$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18,$19,$20,$21,$22,$23,$24,$25,$26,$27,$28,$29,$30,$31 >> FILENAME".tmp" }END{system("mv " FILENAME".tmp" " " FILENAME)} ' OFS="|" *.PRC

Sample Input file
Code:
 65535   1558    14:46:55        11/19/2014      8926877659      30239933826126  8926877659      45      0       0       0       0       20272   00000   2   802DBD5B 405891369800923 postpaid        null    null    20272   20272   OUT     14:47:40        561     561     144     null    405899153999998 405899153999998      63581439
65535   65535   14:46:21        11/19/2014      9143110710      8926328080      9143110710      88      0       0       0       0       23702   15930   2   8076C355 405891373813936 postpaid        null    919153881217    23702   23702   OUT     14:47:49        295     295     159     null    405899153999998 405899153999998      58123387
65535   302     14:46:30        11/19/2014      8642035475      999     8642035475      84      HTLN    0       0       0       21170   00000   2       80CD9C6C     405891961388656 postpaid        null    null    21170   21170   OUT     14:47:54        562     562     144     null    405899153999998 405899153999998      47000903
65535   311     14:47:50        11/19/2014      9143110721      31809143110663  9143110721      28      0       0       0       0       23702   00000   2   8063898E 405891373813947 postpaid        null    null    23702   23702   OUT     14:48:18        295     295     144     null    405899153999998 405899153999998      63842634
65535   302     14:48:05        11/19/2014      9143110266      30618906570722  9143110266      21      0       0       0       0       22912   00000   1   80D295D1 405891350843985 postpaid        null    null    22912   22912   OUT     14:48:26        561     561     144     null    405899153999998 405899153999998      45469437

# 2  
Old 11-19-2014
It's perilous to alter your originals. One mistake and you've trashed all your input data. I would begin by renaming all your files to FILENAME".tmp" and create new ones with the original filenames instead; delete your originals once you've tested the new data and have made sure it's worked to your satisfaction.

END does not work that way, it runs once and only once, after the very last file is read. You need to watch for the filename changing instead. Or, better yet, move that function outside of awk.

It's safe and normal to rename a file while you have it open, on UNIX. I've tested this and it will end up with the originals in *.tmp and the new files in *.prc. It will warn you about running it twice in a row without checking the data.

Code:
for FILE in *.tmp
do
        [ -f "$FILE" ] || break
        echo ".tmp files remaining from last run -- check and delete if originals OK" >&2
        exit 1
done

awk -v OFS="|" '(!F) || (F != FILENAME) { if(F) close(F) ; F=FILENAME ; system("mv "F" "F".tmp"); }  { print ... > F }' *.ext

# 3  
Old 11-19-2014
I think I found the "bug".
Your awk command incl. system() call will work fine if you apply it to only one file at a time.

See what happens if you have more files:
Code:
$ touch file{1..9}
$ ls file{1..9}
file1  file2  file3  file4  file5  file6  file7  file8  file9
$ awk '{next;}END{print FILENAME}' file{1..9}
file9
$

Do you recognize the dilemma?

Possible solution:
Code:
for file in *.PRC; do
 awk ... $file
done

Hope this helps.

---------- Post updated at 08:34 PM ---------- Previous update was at 08:29 PM ----------

Doh! Smilie Too late by two hours Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk command not working as expected

Following one line of awk code removes first 3 characters from each line but when I run the same code on another linux platform it doesn't work and only prints blank lines for each record. Can anyone please explain why this doesn't work? (31 Replies)
Discussion started by: later_troy
31 Replies

2. Shell Programming and Scripting

Using sprintf and system command in awk

Hello Friends, I'm trying something hard (for me) to create a report script,normally the following script works: Echos are just for cosmetic touch, echo -n "\n-----\t----------\t-------------\t\t--------------\t\t--------\n COUNT\tEVENT_TYPE\tRESPONSE_CODE\t\tINTERNAL_ERROR\t\tFLOWNAME... (7 Replies)
Discussion started by: EAGL€
7 Replies

3. Shell Programming and Scripting

awk command on .DAT file not working?

Hi All, I am trying to run awk command on .DAT file and it is not working. The same command is working on .txt file: Contents of the file ZZ_55555555_444444_ZZZZZZ_7777777_888_99.DAT: HEADER|ZZ_55555555_444444_ZZZZZZ_7777777_888_99.DAT... (10 Replies)
Discussion started by: sagar.cumar
10 Replies

4. Shell Programming and Scripting

awk - System command not working

dear All, my awk system command isn't working or rather I'm missing something in my command. Appreciated , if anyone can assist me what exactly I'm missing ?? awk ' /^/ { > c=split($3,a,"/") ;for(n=1; n<=c; ++n) > { > if (system("test -d" /home/cubedata/20120104/"$1"/"a")) { > print... (5 Replies)
Discussion started by: manas_ranjan
5 Replies

5. Shell Programming and Scripting

AWK command working different in Linux

Hi All I have fired a command in linux table=`echo ${file_name} | awk '{FS="/"; print $NF}' | awk '{FS="."; print $1}'` where file_name has /data/ds/dpr_ebicm_uat/backfill/temp/etl_app_info.csv /data/ds/dpr_ebicm_uat/backfill/temp/etl_app_jobs.csv ... (10 Replies)
Discussion started by: vee_789
10 Replies

6. Shell Programming and Scripting

awk command not working from the shell script

Hi, When i run the below command i am able to get the output. awk '/BEGIN DSSUBRECORD/{c=3;next}c-->0' abc.txt | awk '/END DSSUBRECORD/{exit}{print}' | awk '/Owner/{exit}{print}' | awk '{n2=n1;n1=n;n=$0;if(NR%3==0){printf"%s,%s,%s\n",n2,n1,n}}' Output: Name "file_name", ... (5 Replies)
Discussion started by: onesuri
5 Replies

7. Shell Programming and Scripting

awk command not working

Hi all, Trying to write a script that reads a file and prints everything after a certain string is found to the end of the file. Awk is giving me an error and not sure why it doesn't work: # cat test_file Mon Nov 16 2009 16:11:08 abc def Tue Nov 17 2009 16:08:06 ghi jkl Wed Nov 18... (8 Replies)
Discussion started by: jamie_collins
8 Replies

8. Shell Programming and Scripting

system command within awk

I was trying commands within awk. i got stuck here. i take 2 files. cat input first second third fourth fifth sixth seventh eighth ninht tenthcat add *************** Nirbhay ***************i run the command awk '{ if ( NR == 5 ) { print system("cat add") "\n" $0 } else {... (4 Replies)
Discussion started by: nirbhay
4 Replies

9. Shell Programming and Scripting

awk system() command not working

I am using Sun Solaris 5.8 I am trying to run a system command such as ls and echo inside awk, but when I run the following code system echo is not displayed. bash-2.03$ ls | awk 'BEGIN { print "first print" system("echo system echo") print "second print" ... (1 Reply)
Discussion started by: rakeshou
1 Replies

10. Shell Programming and Scripting

can I pass awk variable to system command?

I wanna use a system function to deal with several data. So I use awk variable FILENAME to transfer the file directory to system command, but it does not work. I use a shell function "out_function" to deal with data and save the result in another directory with the same file name. How can I... (2 Replies)
Discussion started by: zhynxn
2 Replies
Login or Register to Ask a Question