[AWK] read lines with \x00 symbol


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting [AWK] read lines with \x00 symbol
# 1  
Old 11-01-2008
Java [AWK] read lines with \x00 symbol

I want to read a large (~1-4Gb) txt file with fields separated by "," and line separator "\n". Unfortunately, file contains \x00 (zero ASCII) symbols
AWK treats them as end of line + it ignores reminder of the line after the \x00.

As a simple example:
echo "\0060\0061\000\0060\0063" | nawk '{print $0}'
Output: 01
I would like to get: 0103

Is there a way to ignore \x00 in (n)awk?

Thank you.
Dmitriy
# 2  
Old 11-01-2008
On Linux and FreeBSD
Code:
# echo "\0060\0061\000\0060\0063" | nawk '{print $0}'
\0060\0061\000\0060\0063

# 3  
Old 11-02-2008
At SuSe with gawk:
echo "\0060\0061\000\0060\0063" | gawk '{print $0}'
\0060\0061\000\0060\0063


but at Solaris 8 it behaves as you described. So maybe use some dummy tool before '[n|g]awk' like 'tr' or 'sed'?
# 4  
Old 11-02-2008
On Solaris you should use /usr/xpg4/bin/awk
# 5  
Old 11-02-2008
Thank you, Guys. Your comments help me a lot.

I should have specify my OS from the begining. It is SUN with 5.9 OS (very old but my department is happy with it Smilie )
finally, I've decided to use perl to replace \x00, smthng like

perl -p -i -w -e 's/\x{00}//g' 20030417.txt

takes a while (~1min) but does what I want
# 6  
Old 11-02-2008
Well it did not help:

echo "\0060\0061\000\0060\0063" | /usr/xpg4/bin/awk '{print $0}'
01

How come you know that there would be other version of awk?

Regards
# 7  
Old 11-02-2008
Quote:
Originally Posted by SebaM6
How come you know that there would be other version of awk?
I just search this forum. Smilie

Last edited by danmero; 11-02-2008 at 11:51 PM.. Reason: Ops, I just notice my posts number
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk or a combination of commands to read and calculate nth lines from pattern

Two numerical lines, found by either header line, need to be added and the total placed in a new-header section. Also the total should should be rounded or cut to a two decimal anynumber.XX format with the AB string added on the end. For example: The numerical lines from headers 2 and 3 are... (3 Replies)
Discussion started by: jessandr
3 Replies

2. Shell Programming and Scripting

awk to add lines with symbol to output file

In the awk below which does execute I get output that is close, except for all the lines that start with a # are removed. Some lines have one others two or three and after the script adds the ID= to the fields below the pattern in the awk, I can not seem to add the # lines back to the output. ... (5 Replies)
Discussion started by: cmccabe
5 Replies

3. Shell Programming and Scripting

Printf transforms \x0a into \x00

If printf '\x0a' | xxd -cols 1produces 0000000: 0a .then, why does printf '%c' "`printf '\x0a'`" | xxd -cols 1produce 0000000: 00 .?? (2 Replies)
Discussion started by: vomv1988
2 Replies

4. Shell Programming and Scripting

Read a file using awk for a given no of lines.

Hi, how do i read a file using awk for a given no of line? e.g 1. read only first 50 line. 2. read starting from line 20 to line 60.. thanks in advance. -alva (8 Replies)
Discussion started by: alvagenesis
8 Replies

5. Shell Programming and Scripting

AWK getline command to read more then two lines

Hello, Am trying to print three lines in a single line using AWK getline command. Below is the command am trying and its displaying only two lines in a single line. Command: awk '{getline n; printf($0,t);next}' Can you please help me ? Thanks, Prince (1 Reply)
Discussion started by: prince1987
1 Replies

6. UNIX for Dummies Questions & Answers

When reading a csv file, counter to read 20 lines and wait for minute then read next 20 till end

Hello All, i am a newbie and need some help when reading a csv file in a bourne shell script. I want to read 10 lines, then wait for a minute and then do a reading of another 10 lines and so on in the same way. I want to do this till the end of file. Any inputs are appreciated ... (3 Replies)
Discussion started by: victor.s
3 Replies

7. Virtualization and Cloud Computing

Install Xen vm on centos5.5 crashed "x00\x00\x00\x00\"

Hi, everyone: I'm new to xen. When I install a vm on centos5.5, I got xen crashed: # virt-install -n centos5 -r 512 --vcpus=1 --disk path=/home/mycoy/centos5.img,size=8 --nographics -l http://mirror01.idc.hinet.net/CentOS/5.5/isos/i386/CentOS-5.5-i386-netinstall.iso when... (1 Reply)
Discussion started by: mycoy
1 Replies

8. Solaris

/usr/lib/passwdutil.so.1: symbol __nsl_fgetspent_r: referenced symbol not found

deleteing post (0 Replies)
Discussion started by: dshakey
0 Replies

9. Shell Programming and Scripting

merge lines into single line based on symbol \t

The symbols are \t and \t\t (note: not tab) If the line starts with \t merge them into a single line upto symbol \t\t \t\t to end and start new line I able to join in a single line but not ending at \t\t and I completely confused help would be appreciated:b::D Input \ta tab XXXXXXXXXX \te... (5 Replies)
Discussion started by: repinementer
5 Replies

10. Shell Programming and Scripting

Join Every 5 Lines With $ Symbol

Hi All, I have output like this in one file. IFName: aust00m1.mis.amat.com ] ObjID: 5eceea48-0d59-71dd-1512-9887a1f10000 IFAlias: Dest: AMNA austkchr1-ser0/0/0:0.315 Type: FRASI CID: DHEC.559128 IFDescription: ATM9/1/0.315-atm subif Status: Normal IFName: aust00m1.mis.amat.com ] ObjID:... (3 Replies)
Discussion started by: ntgobinath
3 Replies
Login or Register to Ask a Question