Program to insert Delimiters at fixed locations in a file, Can you please Debug it for me??


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Program to insert Delimiters at fixed locations in a file, Can you please Debug it for me??
# 8  
Old 06-16-2010
nawk -f jd.awk text1.txt fixed.txt

jd.awk:
Code:
function setFieldsByWidth(   i,n,FWS,start,copyd0) {
  # Licensed under GPL Peter S Tillier, 2003
  # NB corrupts $0
  copyd0 = $0                             # make copy of $0 to work on

  n = split(FIELDWIDTHS,FWS)

  if (n == 1) {
    print "Warning: FIELDWIDTHS contains only one field width." > "/dev/stderr"
    print "Attempting to continue." > "/dev/stderr"
  }

  start = 1
  for (i=1; i <= n; i++) {
    $i = substr(copyd0,start,FWS[i])
    start = start + FWS[i]
  }
}

#Note that the "/dev/stderr" entries in some lines have wrapped.

#I then call setFieldsByWidth() in my main awk code as follows:

BEGIN {
  #FIELDWIDTHS="1 3 8 8 5 9 1 9" # for example
  OFS=" "
}
FNR==NR {
  FIELDWIDTHS=(FIELDWIDTHS)?FIELDWIDTHS FS $0: $0
  next
}
!/^[  ]*$/ {
  saveDollarZero = $0 # if you want it later
  setFieldsByWidth()
  # now we can manipulate $0, NF and $1 .. $NF as we wish
  print $0 OFS
  next
}

# 9  
Old 06-17-2010
Code:
#!/bin/ksh

typeset SEP=','
typeset in=input.dat
typeset out=output.dat
typeset lay=layout.dat

typeset compare
typeset replace
typeset i=1

## Parse the layout and build perl regex strings
cat $lay | while read size
do
  compare="$compare(.{$size})"
  replace="$replace\${$i}${SEP}"
  i=$((i+1))
done

cat $in | perl -pe "s/$compare/${replace%,}/g" > $out

If you need to parse large files, I would recommend a C program. If you do want to use shell script, you might want to take care of some performance stats. The following post I wrote might be helpful with that:
Golden Rule for Super Fast Shell Scripts! | A Programmer's Journal
# 10  
Old 06-17-2010
Another approach:
Code:
awk '
NR==FNR{a[++c]=$0; next}
{ j=1
  for(i=1;i<=c;i++) {
    printf("%s ",substr($0,j,a[i]))
    j+=a[i]
  }
  print ""
}' text1.txt $fixedwidthfile

# 11  
Old 06-17-2010
And here's a Perl-only solution -

Code:
$
$
$ cat layout.dat
2
4
6
3
$
$ cat input.dat
AABBBBCCCCCC
DDDEEFFFFGGGGGG
HHHIIJJJJKKKKKKLLL
$
$
$ ##
$ perl -ne 'chomp;
          if ($ARGV eq "layout.dat") {
            $n += $_;
            $pattern .= "A$_ ";
            $format .= "%s ";
          } else {
            $x .= $_;
            do {$x=~/^(.{$n})(.*)$/ and $y=$1 and $x=$2;
                printf("$format\n", unpack($pattern, $y));
            } until (length($x) < $n)
          }' layout.dat input.dat
 
AA BBBB CCCCCC DDD
EE FFFF GGGGGG HHH
II JJJJ KKKKKK LLL
$
$

tyler_durden
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Insert data between comma delimiters-large file

Having a huge file in the following format. 2,3,1,,,4 1,2,3,,,,,5, 8,7,3,4,,,, Output needed is: 2,3,1,0.0,0.0,4 1,2,3,0.0,0.0,0.0,0.0,5, 8,7,3,4,0.0,0.0,0.0, I have tried reading the file each line, using AWK to parse to find out ",," and then insert 0.0 . It works but very slow. Need... (8 Replies)
Discussion started by: wincrazy
8 Replies

2. Shell Programming and Scripting

Insert a complete file into another file after certain fixed line

Hi Friends, I am writing a shell script where it is required to insert a file say file1 into file2 after certain number of fixed lines in file2.. Please help me how this could be done.. Please suggest me any useful links i need to go through to achieve this.... Thanks in advance .. (2 Replies)
Discussion started by: amr89
2 Replies

3. Shell Programming and Scripting

Insert a variable to a text file after fixed number of lines

Hi, I am new to unix. I need to insert a variable which contains some lines of text into a text file after fixed number of lines.. Please help me on this.. Thanks in Advance, Amrutha (3 Replies)
Discussion started by: amr89
3 Replies

4. Shell Programming and Scripting

running C program to output in multiple locations

Hi Guys, What I am looking at doing is to run a C program in my home directory, but output files in multiple directories BUT not at the same instance. For e.g. 1st instance: Run program.c and output results in path /aaa/bbb/ccc/ 2nd instance: Run program.c again and output results... (9 Replies)
Discussion started by: Jatsui
9 Replies

5. Shell Programming and Scripting

Getting a string without fixed delimiters

I have a line of text for example aaaa bbbb cccc dddd eeee ffffff I would need to get the cccc however bbbb could be there or not. So whether bbbb is in the line or not I need cccc. I was looking at either awk or sed....and trying to start at c and end until the next space. Also... (11 Replies)
Discussion started by: bombcan1
11 Replies

6. UNIX for Advanced & Expert Users

Insert Delimiter at fixed locations in a flat file

Hi Can somebody help me with solution for this PLEASE? I have a flat file and need to insert delimiters at fixed positions in all the lines so that I can easily convert into EXCEL with columns defined as per their width. For Example Here is the file { kkjhdhal sdfewss sdtereetyw... (7 Replies)
Discussion started by: jd_mca
7 Replies

7. Shell Programming and Scripting

Fetch the rows with match string on a fixed lenth text file - NO delimiters

Hi I am trying to fetch the rows with match string "0000001234" Input file looks like below: 09 0 XXX 0000001234 Z 1 09 0 XXX 0000001234 Z 1 09 0 XXX 0000001234 Z 1 09 0 XXX 0000001234 Z 1 09 0 XXX 0000001234 Z 1... (6 Replies)
Discussion started by: nareshk
6 Replies

8. Shell Programming and Scripting

string deletion, variable contents, fixed delimiters

Hi, I need to mass delete the following string(s) from my files weight=100, However the '100' is variable e.g Current: ---------------- moretext=bar, weight=100, moreinfo=blah extrastuff=hi, weight=9999, extrainfo=foo Desired: ------------------ moretext=bar, moreinfo=blah... (2 Replies)
Discussion started by: rebelbuttmunch
2 Replies

9. Shell Programming and Scripting

Insert lines between delimiters

I'm working with a file like: somestuff somemorestuff ... someadditionalstuff STARTTAG ENDTAG someotherstuff somecoolstuff ... somefinalstuffI've got some text (either in a file or piped) to put between STARTTAG and ENDTAG. I was thinking something like grepping for the line number of... (2 Replies)
Discussion started by: BMDan
2 Replies

10. UNIX for Dummies Questions & Answers

How do I insert commas/delimiters in File

Hi, Newbie here. Need to convert a txt file to .csv format. There's no character to replace so not sure if I can use sed :confused: . The comma is to be inserted after every certain number of characters in each line... Help! Thanks. (4 Replies)
Discussion started by: mbelen
4 Replies
Login or Register to Ask a Question