Sponsored Content
Top Forums Shell Programming and Scripting Masking data for different file format Post 302529473 by birei on Thursday 9th of June 2011 11:32:52 AM
Old 06-09-2011
Hi,

Using 'perl':
Code:
$ cat script.pl
use strict;                                                                                                                                                                         
use warnings;                                                                                                                                                                       
                                                                                                                                                                                    
@ARGV || die "Usage: perl $0 file1 file2 ...\n";                                                                                                                                    
                                                                                                                                                                                    
while ( my $file = shift @ARGV ) {                                                                                                                                                  
    # Open files. In failure, warn the problem and read next file.                                                                                                                  
    open my $ifh, "<", $file or do { warn "Cannot open $file for reading: $!\n"; next };                                                                                            
    open my $ofh, ">", $file . ".out" or do { warn "Cannot open $file.out for writing: $!\n"; next };                                                                               
                                                                                                                                                                                    
    # Check type of file.                                                                                                                                                           
    while ( <$ifh> ) {                                                                                                                                                              
        # File 1.                                                                                                                                                                   
        if ( /^DTM/ ) {                                                                                                                                                             
            s/(DTM\+\d*:)\d{12}/$1 . ("'" x 12)/eg;                                                                                                                                 
        # File 2.                                                                                                                                                                   
        } elsif ( /^(?:EXIT|DISC)/ ) {                                                                                                                                              
            s/^(EXIT|DISC)\d{8}/$1 . ("'" x 8)/e;                                                                                                                                   
        # File 3.                                                                                                                                                                   
        } elsif ( /^HHDR/ ) {                                                                                                                                                       
            s/(HHDR\s+\d{6})\d{8}/$1 . ("'" x 8)/eg;                                                                                                                                
        }                                                                                                                                                                           
                                                                                                                                                                                    
        print $ofh $_;                                                                                                                                                              
    }                                                                                                                                                                               
                                                                                                                                                                                    
    close $ifh or warn "Error found closing $file: $!\n";                                                                                                                           
    close $ofh or warn "Error found closing $file.out: $!\n";                                                                                                                       
} 
$ perl script
Usage: perl script.pl file1 file2 ...
$ perl script.pl file1 file2 file3
(Output files will be appended with '.out' -> file1.out, file2.out and file3.out in this sample).

Regards,
Birei
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

how to number format a data file without using SED?

Hi I have a file which contains data (list of data) and I want to put a number with bracket 1) 2) 3) etc at the beginning of every successive line I can do it with SED and I can also do it using the nl route but am looking for a different method. I'm guessing I would need some sort of loop... (3 Replies)
Discussion started by: Cactus Jack
3 Replies

2. UNIX for Advanced & Expert Users

Sed to format data in a file

Hi , i need help with formatting a file i am generating which is to be used in mainframe app so the file length has to be 80 for each rows. The file that m able to generate looks like this (consists of two rows only) E 1006756 1006756 Active T E 0551055 0551055 Active T I... (2 Replies)
Discussion started by: cnilashis
2 Replies

3. Shell Programming and Scripting

Data Masking

I have a pipe delimited file that I need to 'mask' to before loading to keep some data confidential. I need to maintain the first 4 bytes of certain columns and replace the remaining bytes with an 'x'. I would like to maintain spaces but it's not a requirement. Example, need to mask columns 2... (2 Replies)
Discussion started by: 1superdork
2 Replies

4. UNIX for Dummies Questions & Answers

Format/Fix Timestamp Data in a File.

Hello Experts, I have a timestamp(6) column in a .csv data file , format of the data is as below:- ETCT,P,Elec, Inc.,abc,11/5/2010 4:16:09.000000 PM,Y,Y,Y I want the timestamp column to be properly formatted like 11/05/2010 04:16:09.000000 PM Currently the "0" is missing with... (3 Replies)
Discussion started by: mtlrsk
3 Replies

5. Shell Programming and Scripting

Masking off data for different kind of files

Hi everyone, I am required to mask off the date data for 3 kinds of files. Please help me to find a most generic way to handle the files. Example of the content of each kind(the dates are in red): 1. (1-line files). Possible solution: Date needed to be masked off start with DTM+, end with :203 ... (1 Reply)
Discussion started by: Alvin123
1 Replies

6. UNIX for Dummies Questions & Answers

How to Open a data format file?

Hi, Am having a file. I checked that file format by the following command file filename Output is filename: data So the file is data format file Am trying to view that file so i have used some commands like cat,more so on but it showing the contents like compressed form(full of Symbols). How... (4 Replies)
Discussion started by: Adhi
4 Replies

7. UNIX for Dummies Questions & Answers

Format the data file

hi all, i am new to unix forum,i want do some large data format work. pls help me to format the file. i have attched data file to be formated in that two set of line item is there . first set header & second set header to be come in to single header item same as like all the data also. ... (2 Replies)
Discussion started by: dhamu
2 Replies

8. UNIX for Advanced & Expert Users

Identifing The Format of Data File

Hello All, We are receiving UTF-16 formatted file and when i vi the file in Linux OS i am seeing some weird characters, is this because in my .bashrc profile the language variable has been set to LC_ALL=en_US.UTF-8 and hence the reason i am not able to read the UTF-16 formatted file properly? or... (2 Replies)
Discussion started by: Ariean
2 Replies

9. Shell Programming and Scripting

Script to generate Excel file or to SQL output data to Excel format/tabular format

Hi , i am generating some data by firing sql query with connecting to the database by my solaris box. The below one should be the header line of my excel ,here its coming in separate row. TO_CHAR(C. CURR_EMP_NO ---------- --------------- LST_NM... (6 Replies)
Discussion started by: dani1234
6 Replies

10. UNIX for Dummies Questions & Answers

Masking data

How Can I mask one particular columns using some unix command? (4 Replies)
Discussion started by: dsa
4 Replies
All times are GMT -4. The time now is 04:58 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy