Sponsored Content
Full Discussion: File validation
Top Forums Shell Programming and Scripting File validation Post 302746533 by anandapani on Wednesday 19th of December 2012 02:51:39 PM
Old 12-19-2012
Wrench File validation

Hi there,

As a part of file validation, I needed to check for delimiter count in the file.
My aim is to find, how many records have failed to have predefined numbers of delimiters in the file.
My code looks like below

Code:
i=`awk -F '|'  'NF != 2 {print NR, $0} ' ${pinb_fldr}/${pfile}DAT | wc -l | awk '{print $1}'`

echo "number of records with failed delimiter count is $i"

But however, I need to parameterise the delimter count value (in this case 2).
So I am trying to do like this, but it fails..

Code:
dil=2
i=`awk -F '|'  'NF != `echo $dil` {print NR, $0} ' ${pinb_fldr}/${pfile}DAT | wc -l | awk '{print $1}'`
echo "number of records with failed delimiter count is $i"

Appreciate any suggestion in this regard. Thanks .....
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

validation :file <filename>

My user is ftp a CSV file from window. There is possiblity to doing FTP through different mode. This file is then picked by another program, but I want to make sure the FTP file would be always ascii format, bu releasing a commaon File <file name> Can any one provide input , abut how to put a... (2 Replies)
Discussion started by: u263066
2 Replies

2. UNIX for Dummies Questions & Answers

File Validation

Hi All, I am new to UNIX scripting. I need to validate a file. I need to check whether the file has a header , detail records and footer. If all the file is good I need to create a status file with the status 'Y' else 'N'. I have pasted the an example of the file below: ... (5 Replies)
Discussion started by: kumar66
5 Replies

3. Shell Programming and Scripting

File name and format validation

Hi Gurus, I used unix long time back. I need help for writing a unix script which can be automated and execute every day on specific time. 1.) This is the actual functional requirement. Informatica should reject incoming files that have invalid filenames or file formats 2.) My File... (6 Replies)
Discussion started by: vsmeruga
6 Replies

4. UNIX for Dummies Questions & Answers

file name with timestamp validation

Hi , I am trying to write a shell script which will validate all .csv file names in a directory , my file name format is as below. CDR_SCAN_Report_YYYYMMDDHHMI.csv 1. I need to validate the name should start with CDR_SCAN_Report. 2. And the time stamp part is a valid time stamp,if it is... (1 Reply)
Discussion started by: shruthidwh
1 Replies

5. Shell Programming and Scripting

File Name Validation

Hi All, I am trying to validate the file name against the naming convention mentioned in configuration file. In the configuration file, the file name convention is mentioned as: Myfile_SQ<NN>_<NN>_YYYYMMDD_HHMMSS.xml The actual file received is Myfile_SQ10_30_20110423_073002.xml How do... (1 Reply)
Discussion started by: angshuman
1 Replies

6. Shell Programming and Scripting

Input validation of file

Hi, I'm trying to do input validation for a file. If the user fails to enter a file name, the script should prompt for the file name or exit. I'm using this script to integrate with a bigger script which does ftp of the files which are "read". #!/bin/ksh echo "enter a file name" if then... (16 Replies)
Discussion started by: sam_bd
16 Replies

7. Shell Programming and Scripting

File validation

Hello, File contains 1,3 and 5 are required columns. it's working use this command. awk -F\| '$1 && $3 && $5' test1.txt > test2.txt How can use this unix programming.while using runnign this scirpt,it's raising the error. awk: ^ syntax error #!/usr/bin/ksh `awk -F\| '$1 &&... (3 Replies)
Discussion started by: bammidi
3 Replies

8. Shell Programming and Scripting

XML file validation

Hi, i am new to unix script. i have two xml files. one is orderinfo.xml and another one is xsd file(EmailMessage.xml). we need to compare the both file for poper nodes exists are not. for example: <xsd:element name="EmailMessage"> tag in the EmailMessage.xml file(xsd sheet),this tag... (2 Replies)
Discussion started by: bmk
2 Replies

9. Shell Programming and Scripting

Unicode file validation

I don't want HTML_CONTENT,RICH_CONTENT,TEXT_CONTENT columns data in the file and reset of data we need to extract. Find the attached file. Need to extract date in between DI_UX_ROW_END tag. Can help me using unix command using AWK. Thanks, (2 Replies)
Discussion started by: bmk
2 Replies

10. UNIX for Dummies Questions & Answers

Validation before moving the File

Hi, Before moving the files from source directory to target directory I need to check if the files have these strings 'Error', 'Not Valid' ,' YTF-' or if the file is a zero byte file, if the file contains these strings or if it is a zero byte file i should log a entry in the log file and fail... (5 Replies)
Discussion started by: gaur.deepti
5 Replies
rl(1)								   User Commands							     rl(1)

NAME
rl - Randomize Lines. SYNOPSIS
rl [OPTION]... [FILE]... DESCRIPTION
rl reads lines from a input file or stdin, randomizes the lines and outputs a specified number of lines. It does this with only a single pass over the input while trying to use as little memory as possible. -c, --count=N Select the number of lines to be returned in the output. If this argument is omitted all the lines in the file will be returned in random order. If the input contains less lines than specified and the --reselect option below is not specified a warning is printed and all lines are returned in random order. -r, --reselect When using this option a single line may be selected multiple times. The default behaviour is that any input line will only be selected once. This option makes it possible to specify a --count option with more lines than the file actually holds. -o, --output=FILE Send randomized lines to FILE instead of stdout. -d, --delimiter=DELIM Use specified character as a "line" delimiter instead of the newline character. -0, --null Input lines are terminated by a null character. This option is useful to process the output of the GNU find -print0 option. -n, --line-number Output lines are numbered with the line number from the input file. -q, --quiet, --silent Be quiet about any errors or warnings. -h, --help Show short summary of options. -v, --version Show version of program. EXAMPLES
Some simple demonstrations of how rl can help you do everyday tasks. Play a random sound after 4 minutes (perfect for toast): sleep 240 ; play `find /sounds -name '*.au' -print | rl --count=1` Play the 15 most recent .mp3 files in random order. ls -c *.mp3 | head -n 15 | rl | xargs --delimiter=' ' play Roll a dice: seq 6 | rl --count 2 Roll a dice 1000 times and see which number comes up more often: seq 6 | rl --reselect --count 1000 | sort | uniq -c | sort -n Shuffle the words of a sentence: echo -n "The rain in Spain stays mainly in the plain." | rl --delimiter=' ';echo Find all movies and play them in random order. find . -name '*.avi' -print0 | rl -0 | xargs -n 1 -0 mplayer Because -0 is used filenames with spaces (even newlines and other unusual characters) in them work. BUGS
The program currently does not have very smart memory management. If you feed it huge files and expect it to fully randomize all lines it will completely read the file in memory. If you specify the --count option it will only use the memory required for storing the specified number of lines. Improvements on this area are on the TODO list. The program uses the rand() system random function. This function returns a number between 0 and RAND_MAX, which may not be very large on some systems. This will result in non-random results for files containing more lines than RAND_MAX. Note that if you specify multiple input files they are randomized per file. This is a different result from when you cat all the files and pipe the result into rl. COPYRIGHT
Copyright (C) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008 Arthur de Jong. This is free software; see the license for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Version 0.2.7 Jul 2008 rl(1)
All times are GMT -4. The time now is 01:42 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy