Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Verify a flat file with UNIX function or script.
Top Forums UNIX for Dummies Questions & Answers Verify a flat file with UNIX function or script. Post 302340975 by duke0001 on Tuesday 4th of August 2009 09:25:52 PM
Old 08-04-2009
Verify a flat file with UNIX function or script.
I re-post my question to seek your help and critique. I was required to verify a very large and tab-delimited file, named 'MyFile'.

1. The each line in 'MyFile' has 7 columns, and that the values in the 5th column are integers. I need to use shell functions (and standard LINUX/UNIX filters) to verify that these conditions were satisfied in 'MyFile'. My code looks like this:

Code:
#! /bin/sh
#  The nested function to verify columns in file and data type in 5th column.
echo “ Check the columns in the file”
func_x () {
awk ‘ {print $0} ‘ MyFile’
}
echo “ Check the data type in 5th column”
func_int () {
awk – F “|” ‘ $5 ~ “^ [0 – 9] [0 – 9] *$” {print $5}’ MyFile.
}

2. 2. In the same file, the each value in 1st column is unique. How to verify this with UNIX function? My code is:

Code:
#!/bin/ksh
# Verify the unique value in the 1st column
func_u () {
cat MyFile | awk  ‘ {print $1} ‘ | sort | uniq  | wc –l
}

3. Write a shell function that counts the number of occurrences of the word “SpecWord” in the file 'MyFile'. My code is:

Code:
#! /bin/sh
# counts the number of SpecWord in the file
Func_n () {
grep –io “SpecWord” MyFile | wc –l
}

Thanks for your help in advance.
Last edited by duke0001; 08-05-2009 at 11:09 AM..
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

XML to flat file in Unix

Hello, How can I take a file in XML format and convert it to a comma separated format? Is there any scripts or programs that can do this for Unix? I tried surfing the net for such an application, but everything seems to be for Windows OS. Any help or suggestions are greatly appreciated. ... (2 Replies)
Discussion started by: oscarr
2 Replies

2. Shell Programming and Scripting

How To create Flat Files using Unix Shell Script?

Hi all, How to create Flat Files using Unix Shell Script. The Script is supposed to be sheduled to run at a particular time? Thanks in advance Appu (4 Replies)
Discussion started by: Aparna_k82
4 Replies

3. Shell Programming and Scripting

(Urgent):Creating flat file using sql script and sqlplus from UNIX Shell Script

Hi, I need help urgently for following issue. Pls help me to resolve this issue. I am calling sql script file(file1.sql) from UNIX Shell Script(script1.ksh) using sql plus and trying to create flat file that contains all records returned from SQL query in SQL script(file1.sql) I given... (6 Replies)
Discussion started by: praka
6 Replies

4. Shell Programming and Scripting

Verify large file with Unix function

I am seeking help on one UNIX function writting. Please help. I have a large file, named 'MyFile', It was tab-delmited. I am told that each line in 'MyFile' has 7 columns, and that the values in the 5th column are integers. How do I use shell functions (and standard LINUX/UNIX filters) to verify... (1 Reply)
Discussion started by: duke0001
1 Replies

5. UNIX for Dummies Questions & Answers

Verify the data type in a file with UNIX function

I am seeking help on this UNIX function, please help. Thanks in advance. I have a large file, named as 'MyFile'. It was tab-delmited, I am told that each record in column 1 is unique. How would I verify this with UNIX function or command? (1 Reply)
Discussion started by: duke0001
1 Replies

6. Shell Programming and Scripting

Can someone verify the code of two function

Input=D123,S1234,D345 | kESTREL PRODUCTION SUPPORT echo -en "Enter the logmsg=" read logmsg logmsg1=${logmsg%%|*}; echo "$logmsg1"|tr ',' '\n' | sed 's/*$ | sed '/^$/d'//'>pre-commit.config Char() { while read line do if ] || ];then echo "Success" else exit 1;... (1 Reply)
Discussion started by: rohit22hamirpur
1 Replies

7. Solaris

How to generate graph from flat file by using unix script

Hi, I need to generate graph from the text file in Unix server. The file contains the following data, Fri Feb 03 07:01:00 PST 2012|3325 Fri Feb 03 07:02:00 PST 2012|3349 Fri Feb 03 07:03:00 PST 2012|3290 Fri Feb 03 07:04:00 PST 2012|3496 Fri Feb 03 07:05:00 PST 2012|3362 Fri Feb 03 07:06:00... (2 Replies)
Discussion started by: gkn
2 Replies

8. UNIX for Dummies Questions & Answers

Read a flat file, evaluate and create output. UNIX SCRIPT.

Hi all, I have a flat file as below; 470423495|1||TSA-A000073800||1|||1 471423495|1||TSA-A000073800||5|||5 472423495|1||TSA-A000073800||2|||7 473423495|1||TSA-A000073800||3|||3 I like to create a Unix script. The script have to valuate the last two columns, if the values are... (4 Replies)
Discussion started by: mrreds
4 Replies

9. Shell Programming and Scripting

SQL query in UNIX script - output in flat file

Hi, I never did this before... what I want to do is execute a SQL query from a unix script and redirect sql query's output to a flat file (comma separated one) without the header info (no column names). I would also want not to print the query's output to the screen. snapshot of my script:... (13 Replies)
Discussion started by: juzz4fun
13 Replies

10. Programming

Writing a UNIX shell script to call a C function and redirecting data to a .txt file

Hi, I am complete new to C programming and shell scripting. I just wrote a simple C code to calculate integral using trapezoid rule. I am prompting user to pass me No. of equally spaced points , N , upper and lower limit. My code looks as follows so far: #include<stdio.h> #include<string.h>... (2 Replies)
Discussion started by: bjhjh
2 Replies

LEARN ABOUT DEBIAN

numsum

NUMSUM(1)						User Contributed Perl Documentation						 NUMSUM(1)

NAME

       numsum - numsum program file

SYNOPSIS

       numsum [-iIcdhrsvxy] <FILE>

       | numsum [-iIcdhrsvxy]	(Input on STDIN from pipeline.)

       numsum [-iIcdhrsvxy]	(Input on STDIN.  Use Ctrl-D to stop.)

DESCRIPTION

       numsum will take all the numbers on stdin and return the sum of those numbers.  Currently it only processes the first number on each line.
       Besides positive numbers, it also handles negative numbers and numbers with decimals.

OPTIONS

	   -i  Only return the integer portion of the final sum.
	   -I  Only return the decimal portion of the final sum.

	   -c	   Print out the sum of each column.
	   -r	   Print out the sum of each row.

	   -x <n>  Specify a comma seperated list of columns to print.
	   -y <n>  Specify a comma seperated list of rows to print.

	   -s <string> Specify a string to use as a seperator for columns.
		       This defaults to be consecutive whitespace (s+).

	   -h  Help: You're looking at it.
	   -V  Increase verbosity.
	   -d  Debug mode.  For developers
	   -q  Quiet mode, don't print any warnings.

EXAMPLES

       Simply add up the numbers in a file.
	   $ numsum numbers.txt
	   4315

       Enter your own numbers on STDIN. The last number is the answer.
	   $ numsum
	   4
	   21
	   98
	   100
	   223

       Use it in a command pipeline.
	   $ ls -1s | grep .mp3 | numsum -c -x 5
	   72288

       Add up the total byte count in a http log file.
	   $ cat access_log | awk {'print $10'} numsum

	   or

	   numsum -c -x 10 access_log

       Add up the columns of numbers of a file.

	   $ cat columns
	   1 6 11 16 21
	   2 7 12 17 22
	   3 8 13 18 23
	   4 9 14 19 24
	   5 10 15 20 25
	   $ numsum -c columns
	   15 40 65 90 115

       Add up the 1st, 2nd and 5th columns only.

	   $ numsum -c -x 1,2,5 columns
	   15 40 115

       Add up the rows of numbers of a file.

	    $ numsum -r columns
	    55
	    60
	    65
	    70
	    75

       Add up the 2nd and 4th rows.

	    $ numsum -r -y 2,4 columns
	    60
	    70

SEE ALSO

       numaverage(1), numbound(1), numinterval(1), numnormalize(1), numgrep(1), numprocess(1), numrandom(1), numrange(1), numround(1)

COPYRIGHT

       numsum is part of the num-utils package, which is copyrighted by Suso Banderas and released under the GPL license.  Please read the COPYING
       and LICENSE files that came with the num-utils package

	 Developers can read the GOALS file and contact me about providing
       submitions or help for the project.

MORE INFO

       More info on numsum can be found at:

       http://suso.suso.org/programs/num-utils/

perl v5.10.1							    2009-10-31								 NUMSUM(1)
All times are GMT -4. The time now is 08:34 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy