Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Shell script to read lines in a text file and filter user data

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Shell script to read lines in a text file and filter user data
# 1  
Old 10-21-2009
Question Shell script to read lines in a text file and filter user data
hi all,

I have this file [myfile.txt] with some user data.

example:
Code:
$cat myfile.txt
FName|LName|Gender|Company|Branch|Bday|Salary|Age
aaaa|bbbb|male|cccc|dddd|19900814|15000|20|
eeee|asdg|male|gggg|ksgu|19911216|||
aara|bdbm|male|kkkk|acke|19931018||23|
asad|kfjg|male|kkkc|gkgg|19921213|14000|24|
aera|bprb|male|cccc|pppp||15000|20|
.
.
. // and so on


So what I want to do is to take out (to a file) the missing fields as following format:

Code:
<FName> <LName> <Company> Missing Field/s:<> <>

example output:

Code:
eeee asdg gggg Missing Field/s: Salary Age
aara bdbm kkkk Missing Field/s: Salary

CAN ANYONE HELP ME PLEASE?
Last edited by Franklin52; 10-21-2009 at 08:32 AM.. Reason: Please use code tags!
# 2  
Old 10-21-2009
Code:
grep \|\|  file > new_file

Last edited by danmero; 10-21-2009 at 08:45 AM..
# 3  
Old 10-21-2009
hi all,
I found the answer.Go through the link
==========================================
Shell script to read lines in a text file and filter user data - LinuxQuestions.org
===========================================

And thank you danmero for U'r co-operation.
# 4  
Old 10-21-2009
Another one:

Code:
awk -F"|" '
BEGIN{h="FName|LName|Gender|Company|Branch|Bday|Salary|Age";split(h,a,"|")}
{
  s=$1" "$2" "$4" Missing Field/s: "
  for(i=1;i<NF;i++) {
    if (!$i) {s=s" "a[i];f=1}
  }
}
f{print s;f=0}' file

This is the output:

Code:
$ cat file
FName|LName|Gender|Company|Branch|Bday|Salary|Age
aaaa|bbbb|male|cccc|dddd|19900814|15000|20|
eeee|asdg|male|gggg|ksgu|19911216|||
aara|bdbm|male|kkkk|acke|19931018||23|
asad|kfjg|male|kkkc|gkgg|19921213|14000|24|
aera|bprb|male|cccc|pppp||15000|20|
$
$ awk -F"|" '
BEGIN{h="FName|LName|Gender|Company|Branch|Bday|Salary|Age";split(h,a,"|")}
{
  s=$1" "$2" "$4" Missing Field/s: "
  for(i=1;i<NF;i++) {
    if (!$i) {s=s" "a[i];f=1}
  }
}
f{print s;f=0}' file
eeee asdg gggg Missing Field/s:  Salary Age
aara bdbm kkkk Missing Field/s:  Salary
aera bprb cccc Missing Field/s:  Bday
$

# 5  
Old 10-23-2009
Code:
while(<DATA>){
  if($. == 1){
    my @tmp=split("[|]",$_);
    for(my $i=0;$i<=$#tmp;$i++){
    	$hash{$i}=$tmp[$i];
    }    
  }
  else{
  	if(/[|][|]+/){
  		my @tmp=split("[|]",$_);
  		print $tmp[0]," ",$tmp[1]," ",$tmp[3]," Missing Field/s:";
  		for(my $i=0;$i<=$#tmp;$i++){
  			print $hash{$i}," " if $tmp[$i] eq "";
  		}
  		print "\n";
  	}
  }
}
__DATA__
FName|LName|Gender|Company|Branch|Bday|Salary|Age|
aaaa|bbbb|male|cccc|dddd|19900814|15000|20|
eeee|asdg|male|gggg|ksgu|19911216|||
aara|bdbm|male|kkkk|acke|19931018||23|
asad|kfjg|male|kkkc|gkgg|19921213|14000|24|
aera|bprb|male|cccc|pppp||15000|20|

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Shell script to filter certain lines from a file

hi~ i need script on AIX. and have a text file following : create aa 1 2 3 from a@a; create bb from b; create cc 3 4 5 6 6 7 from c@c; (7 Replies)
Discussion started by: tomato00
7 Replies

2. Shell Programming and Scripting

How to get the shell script to read the .txt file as an input/data?

i have written my shell script in notepad however i am struggling to pass the data file to be read to the script the data file is of .txt format. My target is to run the shell script from the terminal and pass 3 arguments e.g. polg@DESKTOP-BVPDC5C:~/CS1420/coursework$ bash valsplit.sh input.txt... (11 Replies)
Discussion started by: Gurdza32
11 Replies

3. Shell Programming and Scripting

Read csv file, convert the data and make one text file in UNIX shell scripting

I have input data looks like this which is a part of a csv file 7,1265,76548,"0102:04" 8,1266,76545,"0112:04" I need to make the output data should look like this and the output data will be part of text file: 7|1265000 |7654899 |A| 8|12660000 |76545999 |B| The logic behind the... (6 Replies)
Discussion started by: RJG
6 Replies

4. UNIX for Beginners Questions & Answers

Shell - Read a text file with two words and extract data

hi I made this simple script to extract data and pretty much is a list and would like to extract data of two words separated by commas and I would like to make a new text file that would list these extracted data into a list and each in a new line. Example that worked for me with text file... (5 Replies)
Discussion started by: dandaryll
5 Replies

5. UNIX for Dummies Questions & Answers

Shell script to read lines in a text file and filter user data Shell Programming and Scripting

sxsaaas (3 Replies)
Discussion started by: VikrantD
3 Replies

6. Shell Programming and Scripting

Read data from .csv file through shell script & modify

I need to read data from a file called "test.csv" through shell script where the file contains values like name,price,descriptor etc. There are rows where descriptor (& in some rows name) are written as string & other characters like "car_+" OR "bike*" etc where it should contains strings like... (3 Replies)
Discussion started by: raj100
3 Replies

7. Shell Programming and Scripting

How to read the data from the text file in shell script?

I am having one text file and i need to read that data from my shell script. I will expain you the scenario: Script look like: For name type 1: For age type 2: For Salary type3: echo "Enter the input:" read the data if input is 1 then go to the Text file and print the... (2 Replies)
Discussion started by: dineshmurs
2 Replies

8. Shell Programming and Scripting

shell script to read data from text file and to load it into a table in TOAD

Hi....can you guys help me out in this script?? Below is a portion text file and it contains these: GEF001 000093625 MKL002510 000001 000000 000000 000000 000000 000000 000001 GEF001 000093625 MKL003604 000001 000000 000000 000000 000000 000000 000001 GEF001 000093625 MKL005675 000001... (1 Reply)
Discussion started by: pallavishetty
1 Replies

9. Windows & DOS: Issues & Discussions

Filter data from text file

Hi All We have got a text file, which has data dumped from 60 tables. From these 60 tables of data we need data from 4 tables only. I tried assigning line numbers to filter out data, but it is not working as intended. below is the sample file ----Table1----- 3,dfs,43,df 4,sd,5,edd... (18 Replies)
Discussion started by: b_sri
18 Replies

10. Shell Programming and Scripting

Shell Script to read specific lines in a file

I have a file with contents as follows Record 1: Rejected - Error on table "DWO"."P2G_CUST_EVENTS". ORA-00001: unique constraint (DWO.CUST_EVENTS_PK) violated Record 5: Rejected - Error on table "DWO"."P2G_CUST_EVENTS". ORA-00001: unique constraint (DWO.CUST_EVENTS_PK) violated Record 6:... (5 Replies)
Discussion started by: varshanswamy
5 Replies
Login or Register to Ask a Question