Flat File Validation


 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Flat File Validation
# 1  
Old 08-28-2019
Flat File Validation

Hi Team

there is a requirement to do validate on Flat files using Shell Scripting.

Suppose we have two flat files ( F1, F2).

Validation

1. Row count between F1 and F2
a) Write it in a file with results of count of each file and differences if any

2. Apply checksum or any other efficient way of hashing each row in given data in both F1 and F2
a) compare each rows in F1 and F2 using hashing technique, log the exact mismatched column value in a row or entire row based on mismatches found if any

3. Have a provision to exclude columns in a file such that condition mentioned above 1 * 2 will be excluded during comparison check.

any help would be greatly appreciated.

thank you
# 2  
Old 08-28-2019
Do not post classroom or homework problems in the main forums. Homework and coursework questions can only be posted in this forum under special homework rules.

Please review the rules, which you agreed to when you registered, if you have not already done so.

More-than-likely, posting homework in the main forums has resulting in a forum infraction. If you did not post homework, please explain the company you work for and the nature of the problem you are working on.

If you did post homework in the main forums, please review the guidelines for posting homework and repost.

Thank You.

The UNIX and Linux Forums.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Datatype file validation

I have a sourcefile which contains data as below.I want to check whether datatype,structure and date format looks good as mentioned. Data is delemited by cydila . Source file-Emp.txt sno name phoneno dept joineddate 1 vivek 0861 CSE 2013-05-29 00:00:00 2 dinesh 123456 ECE ... (2 Replies)
Discussion started by: katakamvivek
2 Replies

2. Shell Programming and Scripting

File validation

Hi there, As a part of file validation, I needed to check for delimiter count in the file. My aim is to find, how many records have failed to have predefined numbers of delimiters in the file. My code looks like below i=`awk -F '|' 'NF != 2 {print NR, $0} ' ${pinb_fldr}/${pfile}DAT |... (3 Replies)
Discussion started by: anandapani
3 Replies

3. Shell Programming and Scripting

Unicode file validation

I don't want HTML_CONTENT,RICH_CONTENT,TEXT_CONTENT columns data in the file and reset of data we need to extract. Find the attached file. Need to extract date in between DI_UX_ROW_END tag. Can help me using unix command using AWK. Thanks, (2 Replies)
Discussion started by: bmk
2 Replies

4. Shell Programming and Scripting

File validation

Hello, File contains 1,3 and 5 are required columns. it's working use this command. awk -F\| '$1 && $3 && $5' test1.txt > test2.txt How can use this unix programming.while using runnign this scirpt,it's raising the error. awk: ^ syntax error #!/usr/bin/ksh `awk -F\| '$1 &&... (3 Replies)
Discussion started by: bammidi
3 Replies

5. Shell Programming and Scripting

File Name Validation

Hi All, I am trying to validate the file name against the naming convention mentioned in configuration file. In the configuration file, the file name convention is mentioned as: Myfile_SQ<NN>_<NN>_YYYYMMDD_HHMMSS.xml The actual file received is Myfile_SQ10_30_20110423_073002.xml How do... (1 Reply)
Discussion started by: angshuman
1 Replies

6. Shell Programming and Scripting

File and Data Validation.

Hello, I am working on an interface between a legacy system and an ERP system.The format of the data extracted into the staging folder from the legacy system is a follows. One control file named ExtractDataControl.txt Multiple entity files eg R1001.txt, R1002.txt, R1020.txt The control... (5 Replies)
Discussion started by: Savio_Saldanha
5 Replies

7. UNIX for Dummies Questions & Answers

File Validation

Hi All, I am new to UNIX scripting. I need to validate a file. I need to check whether the file has a header , detail records and footer. If all the file is good I need to create a status file with the status 'Y' else 'N'. I have pasted the an example of the file below: ... (5 Replies)
Discussion started by: kumar66
5 Replies

8. Shell Programming and Scripting

Unix File Validation! Help

Hi All, I got a file with 3 fields delimited by hyphen "-". I have to validate & cleanse the data before i begine the processing Requirements 1. No record should contain more than 2 delimiters 2. No record should even contain less than 2 delimiters 3. Any records that matches rule 1 &... (8 Replies)
Discussion started by: minnuverma
8 Replies

9. UNIX for Dummies Questions & Answers

Unix File Validation! Help

Hi All, I got a file with 3 fields delimited by hyphen "-". I have to validate & cleanse the data before i begine the processing Requirements 1. No record should contain more than 2 delimiters 2. No record should even contain less than 2 delimiters 3. Any records that matches rule 1 &... (1 Reply)
Discussion started by: minnuverma
1 Replies

10. Shell Programming and Scripting

validation :file <filename>

My user is ftp a CSV file from window. There is possiblity to doing FTP through different mode. This file is then picked by another program, but I want to make sure the FTP file would be always ascii format, bu releasing a commaon File <file name> Can any one provide input , abut how to put a... (2 Replies)
Discussion started by: u263066
2 Replies
Login or Register to Ask a Question