awk command on .DAT file not working?

Login or Register to Ask a Question and Join Our Community

awk command on .DAT file not working?

Tags

Login to Discuss or Reply to this Discussion in Our Community

Top Forums Shell Programming and Scripting awk command on .DAT file not working?

02-06-2013

Registered User

19, 0

Join Date: Dec 2012

Last Activity: 20 February 2013, 6:42 PM EST

Posts: 19

Thanks Given: 0

Thanked 0 Times in 0 Posts

awk command on .DAT file not working?

Hi All,

I am trying to run awk command on .DAT file and it is not working. The same command is working on .txt file:
Contents of the file ZZ_55555555_444444_ZZZZZZ_7777777_888_99.DAT:

Code:

HEADER|ZZ_55555555_444444_ZZZZZZ_7777777_888_99.DAT
XXXXXXX|YYYYYYY|ZZZZZZZZZZ|12334446456|A|YY|111111111|111111111|111111111||111111111|15|3|NNNNNN|Y||1|AAA|ZZZ|Y|111111111
XXXXXXX|YYYYYYY|ZZZZZZZZZZ|12334446456|A|AA|111111111|111111111|111111111||111111111|30|3|NNNNNN|Y||1|AAA|ZZZ|Y|111111111
XXXXXXX|YYYYYYY|ZZZZZZZZZZ|12334446456|B|YY|111111111|111111111|111111111|111111111|15|3|NNNNNN|Y|3|AAA|111111111
XXXXXXX|YYYYYYY|ZZZZZZZZZZ|12334446456|B|AA|111111111|111111111|111111111|111111111|15|3|NNNNNN|Y|3|AAA|111111111
XXXXXXX|YYYYYYY|ZZZZZZZZZZ|12334446456|C|YY|111111111|111111111|111111111|111111111|15|1|NNNNNN|N|4|AAA|BBB|N|111111111
XXXXXXX|YYYYYYY|ZZZZZZZZZZ|12334446456|C|AA|111111111|111111111|111111111|111111111|15|1|NNNNNN|N|4|AAA|BBB|N|111111111
XXXXXXX|YYYYYYY|ZZZZZZZZZZ|12334446456|D|YY|111111111|111111111|111111111|111111111|45|NNNNNN|Y|IND|111111111
XXXXXXX|YYYYYYY|ZZZZZZZZZZ|12334446456|D|AA|111111111|111111111|111111111|111111111|45|NNNNNN|Y|IND|111111111
XXXXXXX|YYYYYYY|ZZZZZZZZZZ|12334446456|E|YY|111111111|111111111|111111111||111111111|3|N|N|Y|111111111
XXXXXXX|YYYYYYY|ZZZZZZZZZZ|12334446456|E|AA|111111111|111111111|111111111||111111111|3|N|N|Y|111111111
XXXXXXX|YYYYYYY|ZZZZZZZZZZ|12334446456|F|YY|111111111|111111111|111111111|111111111|3|N|Y|111111111
XXXXXXX|YYYYYYY|ZZZZZZZZZZ|12334446456|F|AA|111111111|111111111|111111111|111111111|3|N|Y|111111111
TRAILER|0000000012

Command I am using is below:

Code:

awk -F'|' -v FN=DAT_FILENAME '/^[0-9]/{o=FN "_" $5 ".DAT";print $0 >>o}' ZZ_55555555_444444_ZZZZZZ_7777777_888_99.DAT

Could any one please advise.

Thanks a lot in advance.

Sagar.

sagar.cumar

View Public Profile for sagar.cumar

Find all posts by sagar.cumar

02-06-2013

Registered User

11,728, 1,345

Join Date: Feb 2004

Last Activity: 8 May 2020, 9:07 AM EDT

Location: NM

Posts: 11,728

Thanks Given: 903

Thanked 1,345 Times in 1,201 Posts

This

Code:

FN=DAT_FILENAME

has a problem, I am guessing that DAT_FILENAME is a variable.

Code:

FN="$DAT_FILENAME"

If that is correct make the change.

A general note - we can help people much better when they give us sample input (you did that, thanks) and expected output. "Not working" is very unhelpful....

jim mcnamara

View Public Profile for jim mcnamara

Find all posts by jim mcnamara

02-06-2013

Registered User

19, 0

Join Date: Dec 2012

Last Activity: 20 February 2013, 6:42 PM EST

Posts: 19

Thanks Given: 0

Thanked 0 Times in 0 Posts

Hi Jim,

No luck...

But the same command is working on text file.

Sagar

sagar.cumar

View Public Profile for sagar.cumar

Find all posts by sagar.cumar

02-06-2013

Registered User

11,728, 1,345

Join Date: Feb 2004

Last Activity: 8 May 2020, 9:07 AM EDT

Location: NM

Posts: 11,728

Thanks Given: 903

Thanked 1,345 Times in 1,201 Posts

Please read my edit. Please. Post expected output.

jim mcnamara

View Public Profile for jim mcnamara

Find all posts by jim mcnamara

02-06-2013

Registered User

19, 0

Join Date: Dec 2012

Last Activity: 20 February 2013, 6:42 PM EST

Posts: 19

Thanks Given: 0

Thanked 0 Times in 0 Posts

Hi Jim,

Apologies and below is expected output. The input file should be splitted based on the column 5 and accordingly it should create output files.

Please find below output scripts.

Code:

DAT_FILENAME_A.DAT
XXXXXXX|YYYYYYY|ZZZZZZZZZZ|12334446456|A|AA|111111111|111111111|111111111||111111111|30|3|NNNNNN|Y||1|AAA|ZZZ|Y|111111111
XXXXXXX|YYYYYYY|ZZZZZZZZZZ|12334446456|A|YY|111111111|111111111|111111111||111111111|15|3|NNNNNN|Y||1|AAA|ZZZ|Y|111111111
DAT_FILENAME_B.DAT
XXXXXXX|YYYYYYY|ZZZZZZZZZZ|12334446456|B|AA|111111111|111111111|111111111|111111111|15|3|NNNNNN|Y|3|AAA|111111111
XXXXXXX|YYYYYYY|ZZZZZZZZZZ|12334446456|B|YY|111111111|111111111|111111111|111111111|15|3|NNNNNN|Y|3|AAA|111111111
DAT_FILENAME_C.DAT
XXXXXXX|YYYYYYY|ZZZZZZZZZZ|12334446456|C|AA|111111111|111111111|111111111|111111111|15|1|NNNNNN|N|4|AAA|BBB|N|111111111
XXXXXXX|YYYYYYY|ZZZZZZZZZZ|12334446456|C|YY|111111111|111111111|111111111|111111111|15|1|NNNNNN|N|4|AAA|BBB|N|111111111
DAT_FILENAME_D.DAT
XXXXXXX|YYYYYYY|ZZZZZZZZZZ|12334446456|D|AA|111111111|111111111|111111111|111111111|45|NNNNNN|Y|IND|111111111
XXXXXXX|YYYYYYY|ZZZZZZZZZZ|12334446456|D|YY|111111111|111111111|111111111|111111111|45|NNNNNN|Y|IND|111111111
DAT_FILENAME_E.DAT
XXXXXXX|YYYYYYY|ZZZZZZZZZZ|12334446456|E|AA|111111111|111111111|111111111||111111111|3|N|N|Y|111111111
XXXXXXX|YYYYYYY|ZZZZZZZZZZ|12334446456|E|YY|111111111|111111111|111111111||111111111|3|N|N|Y|111111111
DAT_FILENAME_F.DAT
XXXXXXX|YYYYYYY|ZZZZZZZZZZ|12334446456|F|AA|111111111|111111111|111111111|111111111|3|N|Y|111111111
XXXXXXX|YYYYYYY|ZZZZZZZZZZ|12334446456|F|YY|111111111|111111111|111111111|111111111|3|N|Y|111111111

Thanks a lot in advance.

Sagar

Last edited by Franklin52; 02-07-2013 at 06:07 AM.. Reason: Please use code tags for data and code samples

sagar.cumar

View Public Profile for sagar.cumar

Find all posts by sagar.cumar

02-07-2013

Moderator

3,689, 1,352

Join Date: Jan 2012

Last Activity: 22 August 2020, 11:29 PM EDT

Location: Galactic Empire

Posts: 3,689

Thanks Given: 268

Thanked 1,352 Times in 1,258 Posts

Code:

awk -F\| '!/HEADER/&&!/TRAILER/{ f="DAT_FILENAME_"$5".DAT"; print $0 > f; }' ZZ_55555555_444444_ZZZZZZ_7777777_888_99.DAT

Yoda

View Public Profile for Yoda

Visit Yoda's homepage!

Find all posts by Yoda

02-07-2013

Registered User

19, 0

Join Date: Dec 2012

Last Activity: 20 February 2013, 6:42 PM EST

Posts: 19

Thanks Given: 0

Thanked 0 Times in 0 Posts

Hi Bipin,

Thanks a lot for your solution. But, May I know what is wrong in my command.

Can I get any link to get explanation of awk command.

Sagar.

sagar.cumar

View Public Profile for sagar.cumar

Find all posts by sagar.cumar

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Using sed command to replace "|" with ^ for all *.dat files in a folder not working

I am trying to use the below sed command to replace all "|" to ^, in a folder had 50 dat files. when i tried with 1 file it worked but when i tried with wild card, is not working. sed -i 's/"|"/\^/g' *.dat Is this the proper way to use sed command thank you very much for help.

2. Shell Programming and Scripting

Execute a Command in a .Dat File and use it in other Files

We have a process where we store the database password in a config file like below from where the password is picked up and used in Database Scripts ID, Password But we now have a Audit Requirement not to have the passwords in Config Files directly. We have a command which could fetch the...

3. Shell Programming and Scripting

How to use 'ls' command to list files like .dat, not .*.dat?

How to use 'ls' command to list files like *.dat, not *.*.dat

4. Shell Programming and Scripting

Help with Getting distinct record count from a .dat file using UNIX command

Hi, I have a .dat file with contents like the below: Input file ============SEQ NO-1: COLUMN1========== 9835619 7152815 ============SEQ NO-2: COLUMN2 ========== 7615348 7015548 9373086 ============SEQ NO-3: COLUMN3=========== 9373086 Expected Output: (I just...

5. Shell Programming and Scripting

awk command not working as expected

Following one line of awk code removes first 3 characters from each line but when I run the same code on another linux platform it doesn't work and only prints blank lines for each record. Can anyone please explain why this doesn't work?

6. Shell Programming and Scripting

Awk: System command not working in awk

Hi, I have around 10 files in a folder in which I want to change the file format from tab(\t) to pipe(|) with some changes in the fields as well. Below is the code, while tmp file is getting generated but move command is not working, please help Following is the code awk -F"\t" '{print...

7. Shell Programming and Scripting

Compute average ignoring outliers of different segments within a dat file using awk

I have data files that look like this, say data.txt 0.00833 6.34 0.00833 6.95 0.00833 7.08 0.00833 8.07 0.00833 8.12 0.00833 8.26 0.00833 8.70 0.00833 9.36 0.01667 20.53 0.01667 6.35 0.01667 6.94 0.01667 7.07 0.01667 8.06 0.01667 8.10 0.01667 8.25 0.01667 8.71 0.01667 9.31...

8. Shell Programming and Scripting

Remove interspersed headers in .dat file with AWK

Heya there, A small selection of my data is shown below. DATE TIME FRAC_DAYS_SINCE_JAN1 2011-06-25 08:03:20.000 175.33564815 2011-06-25 08:03:25.000 175.33570602 2011-06-25 ...

9. Shell Programming and Scripting

awk command not working

Hi all, Trying to write a script that reads a file and prints everything after a certain string is found to the end of the file. Awk is giving me an error and not sure why it doesn't work: # cat test_file Mon Nov 16 2009 16:11:08 abc def Tue Nov 17 2009 16:08:06 ghi jkl Wed Nov 18...

10. Shell Programming and Scripting

awk system() command not working

I am using Sun Solaris 5.8 I am trying to run a system command such as ls and echo inside awk, but when I run the following code system echo is not displayed. bash-2.03$ ls | awk 'BEGIN { print "first print" system("echo system echo") print "second print" ...

Login or Register to Ask a Question