Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Unix Shell scripting, removing hex 0d 0a

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Unix Shell scripting, removing hex 0d 0a
# 1  
Old 12-28-2010
Bug Unix Shell scripting, removing hex 0d 0a
hi,
I have a file with data like this :
Code:
5963491,11926750,Policy Endorsement 1
Policy Endorsement 2
Policy Endorsement 3
Policy Endorsement 4
Policy Endorsement 5
Policy Endorsement 6
Policy Endorsement 7
5963492,11926751,Product[0].Quote Options[0].CoPolicy[0].CoLobs[0].CoLob[0].LwPolInput[0].LwXsLayerInfoRpt[0].LwXsLayerInfo[1].LwXsLayerCode
5963492,11926752,2
5963493,11926753,Product[0].Quote Options[0].SW Selected Forms[2].SW Selected Form Default

I want it to be like this:
Code:
5963491,11926750,Policy Endorsement 1 Policy Endorsement 2 Policy Endorsement 3 Policy Endorsement 4 Policy Endorsement 5 Policy Endorsement 6 Policy Endorsement 7
5963492,11926751,Product[0].Quote Options[0].CoPolicy[0].CoLobs[0].CoLob[0].LwPolInput[0].LwXsLayerInfoRpt[0].LwXsLayerInfo[1].LwXsLayerCode
5963492,11926752,2
5963493,11926753,Product[0].Quote Options[0].SW Selected Forms[2].SW Selected Form Default


There are few lines of data which needs to be append to the previous line. That is, the line should commence by a 7 or 8 digit number.
When i see the hex format of file (od -x file 1 ) , it contains 0d 0a. I want it to be only 0d. dos2unix wasn't much of a help. Appreciate your help. thanks


Moderator's Comments:
Mod Comment Please use code tags when posting data and code samples!
Last edited by Franklin52; 12-29-2010 at 09:23 AM..
# 2  
Old 12-28-2010
Code:
awk 'ORS=/ Endorsement [0-6]$/?FS:RS' file

# 3  
Old 12-28-2010
Hi,

Using 'sed':
Code:
$ cat infile
5963491,11926750,Policy Endorsement 1
Policy Endorsement 2
Policy Endorsement 3
Policy Endorsement 4
Policy Endorsement 5
Policy Endorsement 6
Policy Endorsement 7
5963492,11926751,Product[0].Quote Options[0].CoPolicy[0].CoLobs[0].CoLob[0].LwPolInput[0].LwXsLayerInfoRpt[0].LwXsLayerInfo[1].LwXsLayerCode
5963492,11926752,2
5963493,11926753,Product[0].Quote Options[0].SW Selected Forms[2].SW Selected Form Default
$ sed -n '1 H; 2,$ { /^[^0-9]\{7,8\}/ H; /^[0-9]\{7,8\}/ { x; s/^\n//; s/\n/ /g; p} }; $ { /^[0-9]\{7,8\}/ {x;p} }' infile
5963491,11926750,Policy Endorsement 1 Policy Endorsement 2 Policy Endorsement 3 Policy Endorsement 4 Policy Endorsement 5 Policy Endorsement 6 Policy Endorsement 7
5963492,11926751,Product[0].Quote Options[0].CoPolicy[0].CoLobs[0].CoLob[0].LwPolInput[0].LwXsLayerInfoRpt[0].LwXsLayerInfo[1].LwXsLayerCode
5963492,11926752,2
5963493,11926753,Product[0].Quote Options[0].SW Selected Forms[2].SW Selected Form Default

Regards,
Birei
# 4  
Old 12-29-2010
Hi, What i meant is I have got a lot of data following the 7 digit number. Like this:
Code:
5834563,11133336,djkfhdfksdkfl
aaaaahhh 12 No
5834564,11133337,iorueureir rierei rere
qqqqrerr r
ruerirei reoprixm cm reopie jkldjas  kls
woewio

I want all the data to be appended to the previous 7 digit number. Like this :
Code:
5834563,11133336,djkfhdfksdkfl aaaaahhh 12 No
5834564,11133337,iorueureir rierei rere qqqqrerr r ruerirei reoprixm cm reopie jkldjas  kls woewio

Thanks. (please note that that 1st column value can be either 7 or 8 digit number)
Last edited by Franklin52; 12-29-2010 at 09:24 AM.. Reason: Please use code tags
# 5  
Old 12-29-2010
Since I like slicing and dicing my files with PERL:
Code:
$/ = "\r\n";
$\ = "\r";

my $line = undef;

while (<>) {
    chomp;

    if (m{^\d{7,8},}) {
        print $line if defined $line;
        $line = $_;
        next;
    }

    $line .= $_;
}

print $line if defined $line;

will generate:
Code:
963491,11926750,Policy Endorsement 1Policy Endorsement 2Policy Endorsement 3Policy Endorsement 4Policy Endorsement 5Policy Endorsement 6Policy Endorsement 7<CR>
5963492,11926751,Product[0].Quote Options[0].CoPolicy[0].CoLobs[0].CoLob[0].LwPolInput[0].LwXsLayerInfoRpt[0].LwXsLayerInfo[1].LwXsLayerCode<CR>
5963492,11926752,2<CR>
5963493,11926753,Product[0].Quote Options[0].SW Selected Forms[2].SW Selected Form Default<CR>

Now I explicitly set the end-of-line character to "\r" since I am running Linux on an x86 based system. Ymmv on a macos based system.
# 6  
Old 12-29-2010
Quote:
0d 0a. I want it to be only 0d.
Very strange request.

Please show a representative sample before and after file displayed with "sed" to make control codes visible.
Code:
sed -n l filename

# 7  
Old 12-30-2010
oops i read your second reply after posting this !! still you can go through this reply
--------------------------------------------------------------------------------

0d 0a - Hex representation of Control-M character / <C-R> / /r/n

I faced same kind of problem where i had to remove only /r & keep /n. if its only1 time thing then open file in vim
Code:
# In command mode 
:se list       # list all characters 
:se ff=unix  # set file format to unix, will remove /r from /r/n
:w!            # save the file 
:se ff?        # verify file format

or

Code:
perl -pi.back -e "s/\x0d0a/\x0a/g" <file_name>

above should do the trick, i did something like that but i can not recall it completely you can play with above perl expression see if it work! BTW take a backup of file in some other directory before you run this.
Last edited by zedex; 12-30-2010 at 02:46 AM.. Reason: read 2nd reply from author later on
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

UNIX Shell Scripting

Describe in short the word completion feature of the tcsh Completion works anywhere in the command line, not at just the end, for both commands and filenames. Type part of a word and hit the Tab key, and the shell replaces the incomplete word with the complete one in the input buffer. The... (1 Reply)
Discussion started by: Elena Lauren
1 Replies

2. Programming

unix Shell scripting

Hi All, need help to complete the automation but stuck at a perticular situation below is the code <code> fixed_function_name { code.... code.... variable_map= { a="/a" b="/b" c="/c" so on... } (7 Replies)
Discussion started by: yadavricky
7 Replies

3. UNIX for Dummies Questions & Answers

Unix Shell Scripting

I'm sorry if this doesn't go here, but I'm in depserate need of help with my last unix homework. Anyways, I'm taking summer classes, and one of them is UNIX. I've understood everything thus far, but I'm having a killer time with how my instructor has worded the problems for shell scripting. I... (3 Replies)
Discussion started by: dw15
3 Replies

4. UNIX for Dummies Questions & Answers

Unix Shell Scripting( Calling from Unix to PLSQL)

Hello Experts, I have the following questions to be discussed here at this esteemed discussion forum. I have two Excel sheets which contain Unix Commands llike creating directory the structure/ftp/Copy/Zip etc to basically create an environment. I need help in understanding some of... (1 Reply)
Discussion started by: faizsaadq
1 Replies

5. Shell Programming and Scripting

Unix shell scripting

Hi All, I have one file called date1.txt and it contains dates like 130112 140112 150112 160112 170112 180112 190112 201012 so i need a script to read this file line by line and find out the day of each date and assign this value in one variable. And validate Weekday="Mon" then... (4 Replies)
Discussion started by: vichuelaa
4 Replies

6. Shell Programming and Scripting

Removing the entire file contents using unix shell script.

I need to remove the entire file contents in file using the shell script. Actually the grap -v command will create one more file and it occupy the space also. I need to remove the entire file contents without creating new file using the shell scripting. Please help me. (5 Replies)
Discussion started by: praka
5 Replies

7. UNIX for Advanced & Expert Users

Need your Help on Unix Shell Scripting.........

Hi Friends, 1. Bash Shell Scrpt to take backup at evening 2. I need a bash shell script for killing all processes. (5 Replies)
Discussion started by: vinayraj
5 Replies

8. Shell Programming and Scripting

Unix shell scripting

Hi, we are writing this fields dynamically retrieved from database and writing into the file. $bmpRec = $bmpRec.'|'.$cust_id; # sp4 $bmpRec = $bmpRec.'|'.$serv_id; # sp5 $bmpRec = $bmpRec.'|'.$site_id; # sp6 $bmpRec = $bmpRec.'|'.$loc_id; # sp7 ... (4 Replies)
Discussion started by: Maruthi Kunnuru
4 Replies

9. Shell Programming and Scripting

difference between AIX shell scripting and Unix shell scripting.

please give the difference between AIX shell scripting and Unix shell scripting. (2 Replies)
Discussion started by: haroonec
2 Replies

10. UNIX for Dummies Questions & Answers

Unix shell scripting

I need to write a script which analyses an invoice file, counting the amount of pages in the file to be printed per account number and per invoice. The account numbers are stored in another file which has instructions on what do with ach customers invoice as per their account number. please... (6 Replies)
Discussion started by: la_burton
6 Replies
Login or Register to Ask a Question