Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Add delimiter to a file from layout file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Add delimiter to a file from layout file
# 1  
Old 05-13-2016
Add delimiter to a file from layout file
I need to add delimiter in file abc.txt using layout.txt
for eg 1 to 2 is category field so after that I need to add delimiter ~

abc.txt
Code:
110101160315100000000000000
 110101160315100000000000000

Layout.txt
Code:
CATEGORY                      POSITION  (1:2)       char,     2
GROUP_ID                      POSITION  (3:4)       char,     2
 DOR_MONTH                   POSITION  (9:10)      char,     2

abc.txt with delimiter
Code:
11~01~01160~315100000000000000
 11~01~01160~315100000000000000

the way I am trying is
1)Read the file Layout.txt line by line
2)Get position of ( , ) ,:
3)set the position of start and end position
4)read abc.txt and append the file with ~ at above given location.

However I got stuck in step 3 which is as below , variable in index gives error.
also Is there any better and efficient method to do it ?
Code:
 
  
 #!/usr/bin/ksh 
 input="./Emi11LayoutAsDefinedInCLEC.TXT"
start_character="("
end_character=")"
tmp=":"
 function getPosition
{   
   echo "getPosition : $var"
   S=`echo $var |awk '{print index($0,$start_character)}'`
   E=`echo $var |awk '{print index($0,$end_character)}'`
   echo "START POSITION of $start_position --------------------->: " $S
   echo "END POSITION of $end_position --------------------->: " $E    
   
}
 count=0
while IFS= read -r var
do
   echo "Reading : Starts **********"
   echo $var
   getPosition $var   
   echo "Reading : Finish **********"
   ((count=$count+1))
done < "$input"



Moderator's Comments:
Mod Comment Please use code tags as required by forum rules!
Last edited by RudiC; 05-13-2016 at 05:06 AM.. Reason: Added code tags
# 2  
Old 05-13-2016
what about field(s) from position 4 to 9 . Could you please provide your input file

I am assuming layout.txt will have all the fields in ascending order with no. of characters seperated by ","(comma).
Please try below code.

Code:
awk -F"," 'NR==FNR{a[NR]=$2;ln=NR;next;}
{ s=1
for(i=1;i<=ln;i++) {
str1=substr($0,s,a[i])
s=s+a[i]
printf i==ln ? str1"\n":str1"~"
}
 }' layout.txt abc.txt

Last edited by pravin27; 05-13-2016 at 05:01 AM..
# 3  
Old 05-13-2016
It's not quite clear what you want, do you want a delimiter
- before the respective field (but not for the first field)
- after the respective field (but not for the last field)
- both but removing double delimiters
?


Please be aware that the sample output is not consistent - the delimiter is in the middle of DOR_MONTH.
# 4  
Old 05-13-2016
Quote:
Originally Posted by RudiC
It's not quite clear what you want, do you want a delimiter
- before the respective field (but not for the first field)
- after the respective field (but not for the last field)
- both but removing double delimiters
?


Please be aware that the sample output is not consistent - the delimiter is in the middle of DOR_MONTH.
Ok let me put it again
Sample file
abc.txt
-------------
Code:
110101160315100000000000000
110101160315100000000000000

Layout.txt
--------------
Code:
CATEGORY                      POSITION  (1:2)       char,     2
GROUP_ID                      POSITION  (3:4)       char,     2
DOR_MONTH                     POSITION  (9:10)      char,     2

About layout means position
-from 1 to 2 is Category and hence we need separator
-from 3 to 4 is GROUP_ID and hence we need separator
-from 9 to 10 is DOR_MONTH and hence we need separator
Final output
--------------
Code:
11~01~0116~03~15100000000000000
11~01~0116~03~15100000000000000

Last edited by RudiC; 05-13-2016 at 12:41 PM.. Reason: Added code tags (again).
# 5  
Old 05-13-2016
using perl
Code:
my $sourceFile="/v/global/user/b/bh/bharade/Programs/Forum_Scripts/Layout.txt";
my $targetFile="/v/global/user/b/bh/bharade/Programs/Forum_Scripts/abc.txt";

open(FH1,"$sourceFile") or die "$!\n";
open(FH2,"$targetFile") or die "$!\n";
my $i=1;
while  (<FH1>) {
  if (/(\d+)\:(\d+)/) {
    my $start=$1;
    my $end=$2;
    if ($. == 1) {
      if ( $start != 1) {
        my $newEnd=$start - 1;
        my $newStart=0;
        $hash{$i}=[$newStart,$newEnd];
        ++$i;
      }
    }
    $hash{$i}=[$start - 1,$end];
    ++$i;
  }
}
my $prevEnd=0;
while (<FH2>) {
  foreach ($j=1;$j<$i;$j++){
    if ( ($hash{$j}->[0] - $prevEnd ) > 1 ) {
      $gapstart=$hash{$j}->[0] - $prevEnd;
      $gapEnd=$hash{$j}->[0] - 1;
      print substr ($_,$gapstart,$gapstart),"~";
    }

    print substr ($_,$hash{$j}->[0],$hash{$j}->[1] - $hash{$j}->[0] ),"~";
    $prevEnd = $hash{$j}->[1];
  }
  if (length($_) != $prevEnd ) {
    print substr ($_,$prevEnd);
  }
}

close (FH1);
close (FH2);

# 6  
Old 05-13-2016
Quote:
Originally Posted by pravin27
using perl
Code:
my $sourceFile="/v/global/user/b/bh/bharade/Programs/Forum_Scripts/Layout.txt";
my $targetFile="/v/global/user/b/bh/bharade/Programs/Forum_Scripts/abc.txt";

open(FH1,"$sourceFile") or die "$!\n";
open(FH2,"$targetFile") or die "$!\n";
my $i=1;
while  (<FH1>) {
  if (/(\d+)\:(\d+)/) {
    my $start=$1;
    my $end=$2;
    if ($. == 1) {
      if ( $start != 1) {
        my $newEnd=$start - 1;
        my $newStart=0;
        $hash{$i}=[$newStart,$newEnd];
        ++$i;
      }
    }
    $hash{$i}=[$start - 1,$end];
    ++$i;
  }
}
my $prevEnd=0;
while (<FH2>) {
  foreach ($j=1;$j<$i;$j++){
    if ( ($hash{$j}->[0] - $prevEnd ) > 1 ) {
      $gapstart=$hash{$j}->[0] - $prevEnd;
      $gapEnd=$hash{$j}->[0] - 1;
      print substr ($_,$gapstart,$gapstart),"~";
    }

    print substr ($_,$hash{$j}->[0],$hash{$j}->[1] - $hash{$j}->[0] ),"~";
    $prevEnd = $hash{$j}->[1];
  }
  if (length($_) != $prevEnd ) {
    print substr ($_,$prevEnd);
  }
}

close (FH1);
close (FH2);

Is it possible to do in only unix Korn shell file???
# 7  
Old 05-13-2016
Try this pure shell (bash):
Code:
while IFS=" 	(:)" read _ _ P _ _ L; do ((IX++)); POS[$IX]=$((P-1)); LEN[$IX]=$L; done < layout.txt

while read T
  do    unset CRS L
        for i in ${!POS[@]}
          do    L=$L${T:$CRS:$((${POS[$i]}-CRS))}"~"${T:${POS[$i]}:${LEN[$i]}}"~"
                CRS=$((${POS[$i]}+${LEN[$i]}))
          done
        L=${L#\~}
        L=${L//~~/\~}
        L=$L${T:$CRS}
        printf "%s\n" $L
  done < abc.txt
11~01~0116~03~15100000000000000
11~01~0116~03~15100000000000000

Last edited by RudiC; 05-16-2016 at 05:48 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Perl Code to change file delimiter (passed as argument) to bar delimiter

Hi, Extremely new to Perl scripting, but need a quick fix without using TEXT::CSV I need to read in a file, pass any delimiter as an argument, and convert it to bar delimited on the output. In addition, enclose fields within double quotes in case of any embedded delimiters. Any help would... (2 Replies)
Discussion started by: JPB1977
2 Replies

2. UNIX for Dummies Questions & Answers

Getting the folder name and file name after delimiter

Hi, I have a input /dev/cm/test1.txt /qa/tm/hmkr/cc/test2.txt and I need an out like below foldername, filename /dev/cm/,test1.txt /qa/tm/hmkr/cc/,test2.txt I tried with awk $NF, but I'm getting the filenames and not folder names. Please let me know how to achive the above... (5 Replies)
Discussion started by: somu_june
5 Replies

3. Shell Programming and Scripting

Shell script to put delimiter for a no delimiter variable length text file

Hi, I have a No Delimiter variable length text file with following schema - Column Name Data length Firstname 5 Lastname 5 age 3 phoneno1 10 phoneno2 10 phoneno3 10 sample data - ... (16 Replies)
Discussion started by: Gaurav Martha
16 Replies

4. Shell Programming and Scripting

Help with changing text file layout

Hi there, I am with this one column input text file to change layout, please help. Thanks. I have awk, sed. $ cat input Median 1.0 2.3 3.0 Median 35.0 26.3 45.7 10.1 63.1 Median 1.2 2.3 (8 Replies)
Discussion started by: cwzkevin
8 Replies

5. UNIX for Advanced & Expert Users

File Delimiter

Hi All, I woul like to know with out opening a file in unix ,how we can find out what is the delemeter in that file... Thanks.. edit by bakunin: changed thread title to "delimiter" so it can be found. (4 Replies)
Discussion started by: raju4u
4 Replies

6. Shell Programming and Scripting

Delimiter in output file

Hello, I am trying to find the record count in a specific folder, Here is the part of the code =========================== STARTDATE=`date +"%y%m%d%H%M"` for i in `ls *.DAT` do wc -l $i >> /XYZ/SrcFiles/"Record_counts"$STARTDATE.csv ... (2 Replies)
Discussion started by: Shanks
2 Replies

7. Shell Programming and Scripting

Creating delimiter file

#/bin/sh sysdate=`date +"%m/%d/%Y"` systime=`date +%r` ps_per=`lsps -s | nawk '{print $2+0}'|tail -1` ps_tot=`lsps -s | nawk '{print $1+0}'|tail -1` lcpu=`vmstat | nawk -F= '/lcpu/ {print $2+0}'` mem_tot=`vmstat | nawk -F= '/mem=/ {print $3+0}'` avm=`vmstat|awk '{print... (7 Replies)
Discussion started by: Daniel Gate
7 Replies

8. UNIX for Dummies Questions & Answers

How to change delimiter in my file ?

Hi I have a file in which delimiter is ';' However if the delimiter is within "" it is a part of the string and not delimiter. How to get the fields ? I want to replace the delimiter ';' to '|'. The file contains data like this : 11111; “2222 2222”; “3333; 3333”; “4444 ""44444” The file... (2 Replies)
Discussion started by: dashing201
2 Replies

9. Shell Programming and Scripting

extracting delimiter from a file.

hi, pls someone tell me how to extract delimiters from any file and pass it to a unix script.since, im a beginner in unix i find it little bit difficult.how to use awk to do this? (9 Replies)
Discussion started by: sureshmit
9 Replies

10. Shell Programming and Scripting

splitting file with more than one delimiter

Hi, I just wandering how to split a record which has more than one delimiter, i have a file which contains pattern as group separtor and ~ as field separtor, Ultimately I need consider even the groups as a field, So i need to make this multi-delimited file into ~ delimited file. My record... (4 Replies)
Discussion started by: braindrain
4 Replies
Login or Register to Ask a Question