File formatting in unix


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting File formatting in unix
# 1  
Old 06-11-2010
Network File formatting in unix

Hi ,

I have a text file noname.txt containing 1000+ records like this. One of the record I have given below.

Input will b e like this



Code:
BOT:
2010/06/01 00:25:59        
      21 = "private"        
      Access-Method = 31        
     NCC = GBR        
      01 = "340806@osiris.fr.ft"        
      04 = 57.250.0.210        
      05 = 337        
      06 = "Framed"        
      07 = "PPP"        
      1E = "08001961001"        
      1F = "441224615080"        
      28 = Stop        
      29 = 0        
      2A = 9321        
      2B = 5409        
      2C = "00045DDF"        
      2D = "RADIUS"        
      2E = 125        
      2F = 151        
      30 = 114        
      31 = "Lost Carrier"        
      3D = "Async"
:EOT

Output what I am expecting is


Code:
20100601#002559#private#31#GBR#osiris.fr.ft#340806#57.250.0.210#337#Framed#PPP#08001961001#441224615080#Stop#0#9321#5409#00045DDF#RADIUS#125#151#114#Lost Carrier#Async

Please let me know is this something possible in UNIX...
Please help I am in trouble.

Moderator's Comments:
Mod Comment Use code tags please, ty.

Last edited by zaxxon; 06-16-2010 at 10:21 AM.. Reason: code tags; helping poster with stucked caps key in subject
# 2  
Old 06-11-2010
Yes it is possible in Unix.

If you know some scripting language you should be able to do it very easily.
# 3  
Old 06-16-2010
Solution in Perl

I am very new to Perl and I have tried to write a very lay man code in Perl, hope it helps.

It would be excellent if someone can reduce the code length.
Code:
#!/usr/bin/perl

use strict;
use warnings;

open (FILE,"noname.txt")|| die "file does not exist";

my @arr = ();
while (<FILE>)
{
    my $LINE = $_;
        chomp($LINE);   
    if (/\//)
        {
        my @arr1 = split(/ /,$LINE);
        push @arr, join('',split(/\//,$arr1[0]));
        push @arr, join('',split(/:/,$arr1[1]));
        }       
    if (/=/)
        {
        my @arr2;
        if (/"/)
            {
            @arr2 = split(/"/,$LINE);
           
            if (/@/)
                {
                my @arr3 = split(/@/,$arr2[1]);
                push @arr, $arr3[1];
                push @arr, $arr3[0];
                }
            else
                {
                push @arr, $arr2[1];
                }
            }
        else
            {
            @arr2 = split(/= /,$LINE);
            push @arr, $arr2[1];
            }
        }
    if (/EOT/)
        {
        print join('#',@arr);
        @arr = ();
        print "\n";
        }

}

close (FILE);

This code can be easily converted to shell script.

Last edited by Franklin52; 06-16-2010 at 08:39 AM.. Reason: Please use code tags, thank you!
# 4  
Old 06-16-2010
Code:
awk '{if ( $1 ~ "BOT") {printf "\n"} else if ( $1 ~ "EOT" ) {printf "\n"} else {printf $3"#"}}' file.txt | sed -e s/\"//g

#private#31#GBR#340806@osiris.fr.ft#57.250.0.210#337#Framed#PPP#08001961001#441224615080#Stop#0#9321#5409#00045DDF#RADIUS#125#151#114#Lost#Async#

#private#31#GBR#340806@osiris.fr.ft#57.250.0.210#337#Framed#PPP#08001961001#441224615080#Stop#0#9321#5409#00045DDF#RADIUS#125#151#114#Lost#Async#

Moderator's Comments:
Mod Comment Please use code tags.

Last edited by Scott; 06-16-2010 at 10:08 AM.. Reason: Code tags
# 5  
Old 06-16-2010
Something like this:

Code:
 
awk -F"=" '/BOT|EOT/ { next } $0 !~ /=/ { print ;next } { print $2 ;next } END { print "\n"}' ORS="#" input_file | sed -e 's/#[        ]*/#/g' -e 's/[     ]*#/#/g' -e 's/"//g'

Surely this code lenght can be reduced further.
# 6  
Old 06-16-2010
Here's a pass at this in perl:

Code:
#!/usr/bin/perl

use strict;
use warnings;

open (FILE,"noname.txt")|| die "file does not exist";
my @results;
while (<FILE>) {
        if ($_ =~ m/:EOT/) {
                # end of a record, print and reset
                print join("#", @results);
                @results = ();
                print "\n";
        } else {
                chomp($_);
                next if ($_ =~ m/^BOT:/); # skip start of record marker
                if ($_ =~ m%[:/]%) {
                        $_ =~ s%\s+%#%;  # replace spaces in datestamp with #
                }
                $_ =~ s%[/:]%%g; # strip / and :
                $_ =~ s%^\s+%%; # strip leading whitespace
                $_ =~ s%\s+$%%;  # strip trailing whitespace
                $_ =~ s%"%%g;    # strip quotes
                $_ =~ s%.*\s+=\s+(.*)%$1%g; # strip everything before the data
                push @results, $_;
        }
}
close (FILE);

Edited to note: I just noticed that the OP wants to have the data string "340806@osiris.fr.ft" transposed in the output, as "osiris.fr.ft#340806". That makes the above incorrect, and provides a more interesting challenge.

---------- Post updated at 08:53 AM ---------- Previous update was at 07:50 AM ----------

OK, here's another pass that handles the field with the @ correctly:
Code:
#!/usr/bin/perl

use strict;
use warnings;

open(FILE, "noname.txt") || die "Can't open file: $!\n";

while(<FILE>) {
        chomp($_);
        next if $_ =~ m/BOT:/;
        if ($_ =~ m/:EOT/) {
                print "\n";
        } else {
                #handle the time/date stamp
                if ($_ =~ s%^(\d{4})/(\d{2})/(\d{2})\s+(\d{2}):(\d{2}):(\d{2})\s+%$1$2$3#$4$5$6%) {
                        print "$_";
                        next;
                }
                # handle the transposition of what looks like an email address
                if ($_ =~ s%.*=\s+"(.*)@(.*)"\s+%$2#$1%) {
                        print "#$_";
                        next;
                }
                # handle everything else
                # assumes a pretty standard format of <whitespace>ID<whitespace>=<data><maybe whitespace>
                $_ =~ s/.*=\s+(.*)\s*/$1/; # pull out the data after the =
                $_ =~ s/"//g; # strip quotes
                $_ =~ s/^\s+//g; # strip leading spaces
                $_ =~ s/\s+$//g; # strip trailing spaces
                print "#$_";
        }
}

close(FILE);


Last edited by malcolmpdx; 06-16-2010 at 11:55 AM..
This User Gave Thanks to malcolmpdx For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Formatting the output in UNIX

Hi All, Here is an oracle command to list out the bug fix patches applied on Oracle Software. (hostname)/home/oracle-> $ORACLE_HOME/OPatch/opatch lsinventory -bugs_fixed The output of this command is as follows(Pasting only relevant section): List of Bugs fixed by Installed Patches: ... (3 Replies)
Discussion started by: veeresh_15
3 Replies

2. Shell Programming and Scripting

UNIX Date Formatting

Hi I have a shell variable storing DATE in YYYY-MM-DD format is there a way i extract required field say only DD Also, would be great if there is a way which could take date format as well so that code is generic for any date format eg DDMMYYYY or DD/MM/YYYY or YYYY/MM/DD etc. Thanks (4 Replies)
Discussion started by: skyineyes
4 Replies

3. Shell Programming and Scripting

Help in UNIX to Windows file formatting

Hi Techies, i need in n UNIX to Windows file formatting. i have created a csv file using a script and mailed it to my email, when i open the file with Excel sheet it looks like below Collector_no|_|Collector_Name|_|SE_Count|_|Installed_Domains|_|CPU_Sys|_|CPU_Idle|_|Memory_Util... (9 Replies)
Discussion started by: mahi_mayu069
9 Replies

4. Shell Programming and Scripting

Formatting sql in UNIX

hi folks am running the shell script below #!/bin/bash sqlplus 'scott@orcl/tiger'<<ENDSQL >> outputlogfile.csv SET PAGES 0 SET HEAD ON SET ECHO OFF SET FEEDBACK OFF SET LINESIZE 100 SET PAGESIZE 100 SET SERVEROUTPUT ON --# Fire the query on database select * from employee; ... (2 Replies)
Discussion started by: coolboy98699
2 Replies

5. Shell Programming and Scripting

Need help with formatting file in UNIX

hi, please help me with the below 1)i have a file like below with data 389096 389097 389098 ..and i need some thing like below '389096','ZZ1111' '389097','ZZ1111' '389098','ZZ1111' 2)i have a file like below with data 8300003225 8300003223 8300006494 (7 Replies)
Discussion started by: kr123
7 Replies

6. Shell Programming and Scripting

Formatting csv file in Unix script

Hi, Am using the following command to create a csv file paste File_1.csv File_4.csv File_7.csv >Out1.csv paste File_2.csv File_5.csv File_8.csv >>Out1.csv paste File_3.csv File_4.csv File_9.csv >>Out1.csv Input Data: Expected Output: Column 1 Column 2 ... (2 Replies)
Discussion started by: meva
2 Replies

7. Shell Programming and Scripting

Formatting date time in unix

while read l do vTimeCreated=`perl -e '@d=localtime ((stat(shift))); printf "%02d-%02d-%04d %02d:% 02d:%02d\n", $d,$d+1,$d+1900,$d,$d,$d' ${l}` echo "${l} || ${vTimeCreated}" >> ${fPrefx}_Output_Files_${vDate}.txt done < servername.txt Using the above code to format date time for each of the... (0 Replies)
Discussion started by: HeadBang
0 Replies

8. Shell Programming and Scripting

Formatting text file in unix

Hi, I am using the following format command for formatting my text file in unix. awk -F":" '{ printf "%-50s%-1s%-50s\n", $1,":", $2}' filename > targetfile The target file is then sent as an attachment via email. When I view the target file in notepad multiple lines get spanned as a... (2 Replies)
Discussion started by: AAA
2 Replies

9. Shell Programming and Scripting

data formatting in Unix shell Scripting

#! /bin/ksh ############################ # AFI Monitor Script ############################ . /db2/uszlad48/sqllib/db2profile export mondir=/home/script/krishna export monlog=$mondir/Error_Report_`date +%Y%m%d`.log echo "connect to database r2pdev" >>$monlog db2 connect to r2pdev user... (1 Reply)
Discussion started by: regnumber
1 Replies

10. UNIX for Dummies Questions & Answers

formatting of file in unix

i have a txt file in which it contains some lines each line contains some 350-400 characters but when i read the file line by line in ksh it is telling the max characters in each line is between 100-127 characters in each line it contains some values with some spaces in between when i try to... (2 Replies)
Discussion started by: trichyselva
2 Replies
Login or Register to Ask a Question