Sponsored Content
Top Forums Shell Programming and Scripting Shell script change new format on the file. Post 302998940 by Aia on Friday 9th of June 2017 11:06:21 PM
Old 06-10-2017
Quote:
Originally Posted by dotran
[...]
---------- Post updated at 08:39 PM ---------- Previous update was at 05:33 PM ----------

I have this perl code and when run see extra "|" at the end of each line. And not sure how to code remove it or can write shell script easier.....can someone help? Thanks
Code:
 
 #!/usr/bin/perl 
 my $filename = 'abc.txt';
open(my $fh, '<:encoding(UTF-8)', $filename)  or die "Could not open file '$filename' $!";
print "FILE_NAME,Filename|Data Format|Compression|GZIP Bytes|Unzipped Bytes|Records|Record Length";
while (my $row = <$fh>) {
  chomp $row;
  my ($label, $value) = split /: /, $row;
  if ($row eq '') {
    print "\n";
  }
  else
  {
     print "$value|";
  }
}

/test1> ./test.pl
Code:
FILE_NAME,Filename|Data Format|Compression|GZIP Bytes|Unzipped Bytes|Records|Record Length|||||
SW_PP_CTRL_20170505.txt.gz|ASCII with carriage returns and linefeeds|GZIP|2019064|11413730|95788|157|
SW_PP_DATA_20170505.txt.gz|ASCII with carriage returns and linefeeds|GZIP|691778058|8316153069|60400481|158|
SW_PP_DEMO_20170505.txt.gz|ASCII with carriage returns and linefeeds|GZIP|26240709|77053000|543250|227|
SW_PP_PLANXREF_20170505.txt.gz|ASCII with carriage returns and linefeeds|GZIP|557904|3061930|16262|310|
SW_PP_PRODUCT_20170505.txt.gz|ASCII with carriage returns and linefeeds|GZIP|21375|229431|1264|211|
SW_PP_REASSIGN_20170505.txt.gz|ASCII with carriage returns and linefeeds|GZIP|32681|69399|802|130|


A possible fix.

Code:
#!/usr/bin/perl
my $filename = 'abc.txt';
open(my $fh, '<:encoding(UTF-8)', $filename)  or die "Could not open file '$filename' $!";
print "FILE_NAME,Filename|Data Format|Compression|GZIP Bytes|Unzipped Bytes|Records|Record Length\n";
my $tmp;
while(<$fh>){
  
  if(/^Filename/../^Record L/){
    my $value = (split /:\s+/)[1];
    chomp $value;
    if($tmp) {
      $tmp .= "|$value";
    }
    else {
      $tmp .= "$value";
    }
    if(/^Record L/){
      print "$tmp\n";
      undef $tmp;
    }
  }
}
close $fh;

Slighter improved version.

Code:
#!/usr/bin/perl
my $filename = 'abc.txt';
open(my $fh, '<:encoding(UTF-8)', $filename)  or die "Could not open file '$filename' $!";
print "FILE_NAME,Filename|Data Format|Compression|GZIP Bytes|Unzipped Bytes|Records|Record Length\n";
my @tmp;
$" = '|';
while(<$fh>){
  if(/^Filename/../^Record L/){
    my $value = (split /:\s+/)[1];
    chomp $value;
    push @tmp, $value;
    if(/^Record L/){
      print "@tmp\n";
      undef @tmp;
    }
  }
}
close $fh;

Another version that dynamically takes care of the header.

Run as perl transform.pl example_file
Code:
#!/usr/bin/perl
use strict;
use warnings;

my @tmp;
my @header;
my $build_header = 1;
$"="|";
while(<>){
  if(/^Filename/../^Record L/){
    my ($id, $value) = split /\s+:\s+|\n/;
    push @tmp, $value;
    push @header, $id if $build_header;
    if($id =~ /^Record L/){
      print "FILE_NAME,@header\n" if $build_header;
      print "@tmp\n";
      undef @tmp;
      undef $build_header;
    }
  }
}


Last edited by Aia; 06-10-2017 at 12:32 AM.. Reason: Add another example
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

format csv file using shell script

i have a report.csv file from oracle datavase In that file data is like this with report heading and date SALES DAILY REPORT DATE: 06-26-2007 REPORT NAME: SALES DATA AA.BB.CCCC.DDDD,BBBBB,06-26-2007,0,BEGIN,END ZZ.VV.DDDD.XXXXXXX,MMMMMM,06-25-2007,18,BEGIN,END... (3 Replies)
Discussion started by: raosurya
3 Replies

2. Shell Programming and Scripting

script to change the date format in a file

i have many files with date format of 6-9-2008 and i want a script that can change the format to 2008-06-09 Thanks (15 Replies)
Discussion started by: shehzad_m
15 Replies

3. Solaris

format file using shell script

Hi All, I am new to shell scripts. I have a requirement to change the format of a file. Here is the original file: #student layout student_name student_class student_subject david 5 chemistry paul 4 physics steve 6 mathematics This is the format i need: k1,david,5,chemistry... (1 Reply)
Discussion started by: balajim
1 Replies

4. Solaris

format file using shell script

my question "format file using shell script " is not a homework. bad guess. my actual file is much more complex. the requirement is to format the file before i can read it from SAP. so i'd appreciate if any inputs can be provide. i've tried most of the commands like tr and sed and nawk, no... (2 Replies)
Discussion started by: balajim
2 Replies

5. Shell Programming and Scripting

How to check file name format using shell script?

Hi, I am writting a script, which accepts input file as parameter. Input file name is aa_bb_cc_dd_ee.<ext> I need to check that input file name should be of 5 fileds. Please help me out. :confused: (7 Replies)
Discussion started by: Poonamol
7 Replies

6. Shell Programming and Scripting

Help with shell script for know when a file change it

Hi, IŽd like to know how to program a shell script for know when a file changes and based on that make another tasks all this in real time.. Thanks (2 Replies)
Discussion started by: mrios7
2 Replies

7. Shell Programming and Scripting

Conversion of below Tabs Tex file into CSV format file : shell script needed

Request if some one could provide me shell script that converts the below "input file" to "CSV format file" given Name Domain Contact Phone Email Location ----------------------- ------------------------------------------------ ------- ----- ---------------------------------... (7 Replies)
Discussion started by: sreenath1037
7 Replies

8. Shell Programming and Scripting

Change value in a file using perl or shell script

hi, I have a local.conf file which has the first line TOPDIR = "/home/mvdev/workspace/boxer". I want to replace the value to "/home/common/workspace/mirror". I tried the following perl command that is perl -p -i -e 's/Path/path1/g' myfile.txt then sed... (7 Replies)
Discussion started by: amvarma77
7 Replies

9. Shell Programming and Scripting

Change date format in shell script

Plz help me To display date in the mm/dd/yyyy. Eg. if date is 28-09-2012 the output of the shell script should be date 09/28/2012. (1 Reply)
Discussion started by: shivasaini
1 Replies

10. Shell Programming and Scripting

Format CSV file from a shell script

I have a shell script which creates a CSV files. there are 3 fields, field1, field2 and comp. I will generates the values for field1 and field2 and Want to compare both. If field1>filed2 then comp should be success written in green in CSV file, else it should fail in red color. How can I change the... (5 Replies)
Discussion started by: sauravrout
5 Replies
All times are GMT -4. The time now is 11:34 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy