Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Removing inline binary data from txt file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Removing inline binary data from txt file
# 8  
Old 04-04-2013
Quote:
Originally Posted by Yoda
I don't think nawk supports that character class.
I'm not sure it matters, since these don't look like control characters to me.
# 9  
Old 04-04-2013
found this with more googling

perl -pi2 -e 'BEGIN{$rep{chr($_)}=q() for 0..31,127;$rep{chr(10)}=chr(10)}s/([[:cntrl:]])/$rep{$1}/g'

this seems to work but not sure what this does though Smilie
credit to http://compgroups.net/comp.lang.perl...n-a-file/12755
---------- Post updated at 01:14 PM ---------- Previous update was at 12:49 PM ----------

@Corona688 tr did not work in this case
Code:
sandbox]$>tr -d '\200-\377'  < bin4lines.txt
app_id:1936 pgm_num:0 branch:TBNY ord_num:0500012(QMK) deal_num:0 ref_num:R
app_id:1396 pgm_num:0 branch:TBNY ord_num:0000016(Ì11S) deal_num:0 ref_num:R
app_id:1996 pgm_num:0 branch:TBNY ord_num:0260012(ûPAK) deal_num:0 ref_num:R
app_id:1936 pgm_num:0 branch:TBNY ord_num:F016 deal_number:0 ref_num:R
added the line with issue here
Q6I) deal_number:0

@yoda
sandbox]$>awk '{gsub(/[[:cntrl:]]/,x)}1' < bin4lines.txt awk: syntax error near line 1
awk: illegal statement near line 1

Am on a solaris box
Last edited by tasmac; 04-04-2013 at 04:04 PM..
# 10  
Old 04-04-2013
Quote:
Originally Posted by tasmac
@yoda
sandbox]$>awk '{gsub(/[[:cntrl:]]/,x)}1' < bin4lines.txt awk: syntax error near line 1
awk: illegal statement near line 1

Am on a solaris box
I said to run /usr/xpg4/bin/awk not awk

awk is broken on SunOS / Solaris.
This User Gave Thanks to Yoda For This Post:
tasmac
# 11  
Old 04-04-2013
Is there a way to change this so that it does not chnage the same file but writes into a new file instead of changing the same file
Code:
perl -pi2 -e 'BEGIN{$rep{chr($_)}=q() for 0..31,127;$rep{chr(10)}=chr(10)}s/([[:cntrl:]])/$rep{$1}/g'

---------- Post updated at 01:50 PM ---------- Previous update was at 01:46 PM ----------

@yoda this works when I use the awk you suggested

Code:
sandbox]$>/usr/xpg4/bin/awk '{gsub(/[[:cntrl:]]/,x)}1' < bin4lines.txt
app_id:1936 pgm_num:0 branch:TBNY ord_num:0500012(QMK) deal_num:0 ref_num:R
app_id:1396 pgm_num:0 branch:TBNY ord_num:0000016(Ì11S) deal_num:0 ref_num:R
app_id:1996 pgm_num:0 branch:TBNY ord_num:0260012(ûPAK) deal_num:0 ref_num:R
app_id:1936 pgm_num:0 branch:TBNY ord_num:F016 deal_number:0 ref_num:R
added the line with issue here
app_id:1996 pgm_num:0 branch:TBNY ord_num:0440010(Q6I) deal_number:0

---------- Post updated at 01:54 PM ---------- Previous update was at 01:50 PM ----------

combining @rtrdx1 and @yoda I get this

Code:
sandbox]$>/usr/xpg4/bin/awk '{gsub(/[[:cntrl:]]/,x)}1' < bin4lines.txt | perl -pe 's/[^[:ascii:]]//g'
app_id:1936 pgm_num:0 branch:TBNY ord_num:0500012(QMK) deal_num:0 ref_num:R
app_id:1396 pgm_num:0 branch:TBNY ord_num:0000016(11S) deal_num:0 ref_num:R
app_id:1996 pgm_num:0 branch:TBNY ord_num:0260012(PAK) deal_num:0 ref_num:R
app_id:1936 pgm_num:0 branch:TBNY ord_num:F016 deal_number:0 ref_num:R
added the line with issue here
app_id:1996 pgm_num:0 branch:TBNY ord_num:0440010(Q6I) deal_number:0

# 12  
Old 04-04-2013
So am I correct in assuming that you got the desired output?

If yes, you can redirect the output to another file to have a new file created rather than modifying the original file.
# 13  
Old 04-04-2013
@yoda
yes thats what I ended up doing thanks
This User Gave Thanks to tasmac For This Post:
Yoda
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Removing PATTERN from txt without removing lines and general text formatting

Hi Everybody! First post! Totally noobie. I'm using the terminal to read a poorly formatted book. The text file contains, in the middle of paragraphs, hyphenation to split words that are supposed to be on multiple pages. It looks ve -- ry much like this. I was hoping to use grep -v " -- "... (5 Replies)
Discussion started by: AxeHandle
5 Replies

2. Windows & DOS: Issues & Discussions

Xterm logging on Cygwin/X - binary data in log file.

I have Cygwin/X installed on Windows 7. In an xterm, I turned on logging via Main Options > Log to File. When I open my log file with Vim I get a warning that it might be binary. Looking through the file I see what I think are VT datastream escape characters. It makes it hard to use the... (1 Reply)
Discussion started by: gctaylor
1 Replies

3. Shell Programming and Scripting

Data extraction from .txt file

Hey all, i´ve got the following problem: i´m aquiring data with an instrument and i get data in a .txt file. This is how the txt file looks like: Report of AU program poptau F1P=-49.986ppm F2P=-110.014ppm Target directory for serfile: D:/data/Spect500/nmr/Thoma/882 Linear... (17 Replies)
Discussion started by: expikx
17 Replies

4. Shell Programming and Scripting

To log binary file output to a txt file

Hi, I wrote a small script whose function is to execute the postemsg provided if the threshold breaches. I want to log this postemsg messages to a log file. But I am not able to do. Can someone throw some light on how to log the output of this. I am pasting a snippet of that code. ... (2 Replies)
Discussion started by: dbashyam
2 Replies

5. Programming

writing binary/struct data to file

I am trying to write binary data to a file. My program below: #include <stdlib.h> #include <stdio.h> struct tinner { int j; int k; }; struct touter { int i; struct tinner *inner; }; int main() { struct touter data; data.i = 10; struct tinner... (4 Replies)
Discussion started by: radiatejava
4 Replies

6. Shell Programming and Scripting

how to check the file data type(ascii or binary)

hi i am receiving a file from one system , i have to verify the format of the file data i.e whether the data is in acii format or binary format, please help thanks in advance satya (1 Reply)
Discussion started by: Satyak
1 Replies

7. Shell Programming and Scripting

Script for removing text from a txt file

Hello, So I wanted to write a very simple script to remove some information from a text file and save it as something else. For example I have a text file (let's call it txt) with three rows of numbers: 0 0 1 9 8 7 5 0 6 7 9 0 0 7 9 8 1 1 6 4 0 6 0 0 9 8 4 6 0 9 2 8 1 And I want to... (2 Replies)
Discussion started by: hertingm
2 Replies

8. UNIX for Dummies Questions & Answers

Binary txt file received when i use uuencode to send txt file as attachment

Hi, I have already read a lot of posts on sending attachments in unix...but none of them were of help for my problem...so here goes.. i wanna attach a text file and send to a mail id..used the following code : uuencode "$File1" "$File1" ;|mail -s "$Mail_sub" abc@abc.com it works... (2 Replies)
Discussion started by: ash22
2 Replies

9. UNIX for Advanced & Expert Users

get data from .txt file

Hi I have a file diskspace.txt the data in that file is asFilesystem kbytes used avail %used Mounted on /dev/vg06/lvol1 18870272 12099836 6673336 64% /dist know i am writing a script to get the above bold data and that data need to be used as a input. any idea... (4 Replies)
Discussion started by: gkrishnag
4 Replies

10. UNIX for Dummies Questions & Answers

Binary data to text file conversion

Dear Sir; i want to know how the binary data convert to text file or readablw format (ASCII).If possible pl. help me for the software and where it is available for download. i.e. (1 Reply)
Discussion started by: auro123
1 Replies
Login or Register to Ask a Question