Split and add header and trailer from input file Post: 302908320

Sponsored Content

Top Forums Shell Programming and Scripting Split and add header and trailer from input file Post 302908320 by Chubler_XL on Sunday 6th of July 2014 06:32:06 PM

07-06-2014

Moderator

How about having your replace.txt file formatted like this:

Code:

ED:20:
E9:20:
D9:20:
C2:20:
80:20:
99:20:
E2:20:
0D24::
C3:20:
C4:20:
C8:20:
C9:20:
A0:20:
FF:20:
B1:20:
83:20:
C1:20:
E1:20:
B7:20:
F6:20:
F1:20:
F3:20:
E3:20:
1A:20:

Note how I replaced your OD0A value with OD24 this is because awk strips the CR characters from each line. To match end-of-line we use the dollar sign (0x24), this being the RE meta-character for end-of-string.

This also means if you want to replace any RE meta-characters (eg star or dot) you will need to backslash them so to replace star with hash we would use 5C2A:23

Below is the code which does rely on the gawk --non-decimal-data command line switch:

Code:

awk --non-decimal-data \
-v stamp=$(date +%Y%m%d%H%M%S) \
    -v f="TOM PAT SAM BOB KIM" \
    -v t="TOM TOM TOM BOB KIM" '
function hextoascii(str,i,ret) {
   for(i=1;i<length(str);i+=2)
      ret=ret sprintf("%c", 0 + ("0x" substr(str,i,2)))
   return ret
}
BEGIN {
   split(f,from)
   for(i=split(t,to);i;i--) CONV[from[i]]=to[i]
}
FNR==NR{
   v=split($0,vals,":")
   repl[hextoascii(vals[1])]=hextoascii(vals[2])
   next
}
{ for(i in repl) gsub(i, repl[i]) }
/^H/ {header=$0 ; next}
/^T/ {trailer=$0 ; next}
length>3{
   typ=substr($0,34,3)
   if (typ in CONV) {
       fname="xyz_" CONV[typ] "_" stamp ".txt"
       if (!(fname in A)) {A[fname]; print header > fname}
   }
   else fname="xyz_error_" stamp ".txt"
   print $0 >> fname
   close(fname)
}
END {
  for (fname in A) print trailer >> fname
}' replace.txt Test.txt

Chubler_XL

View Public Profile for Chubler_XL

Find all posts by Chubler_XL

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Split large file and add header and footer to each file

I have one large file, after every 200 line i have to split the file and the add header and footer to each small file? It is possible to add different header and footer to each file?

2. Shell Programming and Scripting

Split large file and add header and footer to each small files

I have one large file, after every 200 line i have to split the file and the add header and footer to each small file? It is possible to add different header and footer to each file?

3. Shell Programming and Scripting

Removing Header & Trailer from a file

Hi All, I am karthik. I am new to this forum. I have one requirement. I have a file with header and footer. Header may be like HDR0001 or FILE20090110 (Assume it is unknown so far, but i am sure there is a header in the file) likewise file has the trailer too. I just...

4. Shell Programming and Scripting

Creating Header & Trailer for bulk volume data file

Hi all, I have a requirement to create a Header &Trailer for a file which is having 20 millions of records. If I use the following method, i think it will take more time. cat "Header"> file1.txt cat Data_File>>file1.txt cat "Trailer">>file1.txt since second CAT command has to read all...

5. Shell Programming and Scripting

Adding Header and Trailer records to a appended file

How can we a shell script and pass date parameters .I have 3 files comming from Datastage with |" delimited I need append 3 files as above: File1: P0000|"47416954|"AU|"000|"INS|"0000|"|"20060601|"99991231|"|"|"|"|"01 File 2:...

6. UNIX for Dummies Questions & Answers

Adding header and trailer into a file

Hi, I want to add the below Header to all the files in sequence File1,File2,File3...etc "ABC,<number of chracter in the file>" e,g - If File1 is as below pqrstuvdt abcdefgh then I want to add the above header into it ,So that File1 becomes as below ABC,17 pqrstuvdt abcdefgh ...

7. Shell Programming and Scripting

Remove last few characters in a file but keeping Header and trailer intact

Hi All, I am trying write a simple command using AWK and SED to this but without any success. Here is what I am using: head -1 test1.txt>test2.txt|sed '1d;$d' test1.txt|awk '{print substr($0,0,(length($0)-2))}' >>test2.txt|tail -1 test1.txt>>test2.txt Input: Header 1234567 abcdefgh...

8. Shell Programming and Scripting

Script to validate file header and trailer

9. Shell Programming and Scripting

Verify the header and trailer in file

please see my requirement, I hope I am clear.

10. UNIX for Advanced & Expert Users

Removing Header and Trailer record of a EBCDIC file

I have a EBCDIC multi layout file which has a header record which is 21 bytes, The Detail records are 2427 bytes long and the trailer record is 9 bytes long. Is there a command to remove the header as well as trailer record and read only the detail records while at the same time not altering...

LEARN ABOUT CENTOS

pmloadderivedconfig

PMLOADDERIVEDCONFIG(3)					     Library Functions Manual					    PMLOADDERIVEDCONFIG(3)

NAME

       pmLoadDerivedConfig - load derived metric definitions from a file

C SYNOPSIS

       #include <pcp/pmapi.h>

       int pmLoadDerivedConfig(char *fname);

       cc ... -lpcp

DESCRIPTION

       Each  line of the file fname is either a comment line (with a ``#'' in the first position of the line) or the declaration of a derived per-
       formance metric, specified as:

       * the name of the derived metric, using the same ``dot notation'' syntax that is used for PCP  performance  metrics,  see  PCPIntro(1)  and
	 pmns(5).

       * an equals sign (``='')

       * a valid expression for a derived metric, as described in pmRegisterDerived(3).

       White space is ignored in the lines.

       For each line containing a derived metric definition, pmRegisterDerived(3) is called to register the new derived metric.

       The  result  from pmLoadDerivedConfig will be the number of derived metrics loaded from fname else a value less than zero in the case of an
       error.

EXAMPLE

       # sample derived metric definitions
       bad_in_pkts = network.interface.in.errors + network.interface.in.drops
       # note the following would need to be on a single line ...
       disk.dev.read_pct = 100 * delta(disk.dev.read) /
		   (delta(disk.dev.read) + delta(disk.dev.write))

SEE ALSO

       PCPIntro(1), PMAPI(3) and pmRegisterDerived(3).

Performance Co-Pilot													    PMLOADDERIVEDCONFIG(3)