problem with bad records


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting problem with bad records
# 1  
Old 04-16-2007
problem with bad records

I have a data file with around 1 million records and i have 12 data fileds in each
record seperated by 11 pipes. The file also has some bad records where there is
only one pipe in some of the records. I want to print all this records with only
one pipe in them. These bad records are broken into two lines and hence they are bad.

Can someone let me know how i can print these to a new file.

sample file with good and bad records;

Quote:
|WATER & POWER TECHNOLOGIES-B||||||||||
123456|GE FINANCIAL ASSURANCE-R||||||||||
|HICKAM AFB-R
||||||||||
I want tp print the bad records in the file to a new file.
# 2  
Old 04-16-2007
Code:
awk -F"|" ' NF == 2 { print; getline ; print } ' file > newfile

# 3  
Old 04-16-2007
dsravan,
Try this:
grep -v '.*|.*|.*|.*|.*|.*|.*|.*|.*|.*|.*|' input_file
# 4  
Old 04-16-2007
Code:
sed -n '/^[^|]*|[^|]*$/p' file > newfile

# 5  
Old 04-16-2007
Quote:
Originally Posted by dsravan
I want to print all this records with only one pipe in them.

I want tp print the bad records in the file to a new file.

Code:
grep '^[^|]*|[^|]*$'  FILE > BADFILE

If you want to join all such lines with the following line:

Code:
awk -F'|' 'NF == 2 { printf "%s|", $0; next }
                            { print }' FILE > NEWFILE

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Problem running zip from cron - bad zipfile offset

I've created a pretty straightforward shell script to do backups via rsync. Every night it rsyncs a copy of a website (approx 60GB) to the local machine. Once a week, after the rsync step, it zips the local copy of the whole site into 2GB chunks and places them in another folder. When running... (3 Replies)
Discussion started by: Agreppa
3 Replies

2. Shell Programming and Scripting

Remove bad records from file and move them into a file then send those via email

Hi my requirement is that i want pull the bad records from input file and move those records in to a seperate file. that file has to be sent via email.. any suggentions please (1 Reply)
Discussion started by: sxk4999
1 Replies

3. Shell Programming and Scripting

remove bad records.

HI I have a problem in a file .The file was generated with the wrong data in it. MAL 005158UK473BBTICK1120722 A9999999ADASCD 1120722ADD_SECURIADD_SECURI MAL 005158UK473BBU 1120722 A9999999FF000EA0B9C 1120722ADD_SECURIADD_SECURI MAL 005158UK473ISN 1120722 A9999999US005158UK43... (5 Replies)
Discussion started by: ptappeta
5 Replies

4. Shell Programming and Scripting

problem in redirecting records using nawk

I am trying to redirect record to two files using nawk if-else. #Identify good and bad records and redirect records using if-then-else nawk -F"|" '{if(NF!=14){printf("%s\n",$0) >> "$fn"_bad_data}else{printf("%s\n",$0) >> $fn}}' "$fn".orig "$fn".orig is the source file name bad... (7 Replies)
Discussion started by: siteregsam
7 Replies

5. Programming

Problem with DNS-Records in C

Hey community, I'm coding an application in ANSI-C that needs to locate IP-Adresses. I fount that a good and very cheap way to do this would be to make DNS-queries against Blacklist provider UCEPROTECT's Country zone (country.uceprotect.net) They seem to offer a TXT-Record and also an A... (0 Replies)
Discussion started by: warpdrive
0 Replies

6. Shell Programming and Scripting

Removing bad records from a text file

Hi, I have an requirement where i need to remove few bad records(bad records I mean email id's are part of the 1st field, where a numeric value expected) from the text file delimited by ",". file1.txt --------- 1234,,DAVID,MAX abc@email.com,,JOHN,SMITH 234,,ROBERT,SEN I need to remove... (3 Replies)
Discussion started by: naveen_sangam
3 Replies

7. UNIX for Dummies Questions & Answers

Bash: bad substitution problem...pls help!

I have this situation in my script (simplified): A=C C=10 I need to get number 10 using just A variable. I tried with : echo $`echo $A` - but i get $C string (i need number) Thanks very much for any help! (1 Reply)
Discussion started by: xfouxs
1 Replies

8. Shell Programming and Scripting

AWK Multi-Line Records Numbering Problem

I have a set of files of multi-line records with the records separated by a blank line. I needed to add a record number to the front of each line followed by a colon and did the following: awk 'BEGIN {FS = "\n"; RS = ""}{for (i=1; i<=NF; i++)print NR,":",$i}' ~/Desktop/data98-1-25.txt >... (3 Replies)
Discussion started by: RacerX
3 Replies

9. UNIX for Dummies Questions & Answers

SCO Backup Problem (bad Sblock Magic Number)

Hi guys, First I have to say that I'm not Unix expert, I just have medium level experince in Unix scripting and some knowledge with a little of hands on experience of unix administration (Solaris only). I have my plans to move ahead in that field but this is a different story. I have a client... (0 Replies)
Discussion started by: SolarisProToBe
0 Replies

10. UNIX for Dummies Questions & Answers

bad substitution problem (easy question)

hi, what i want to do is to convert all the txt file under my directory to the properties file using the native2ascii command. however, when i run my script, i got bad substitution error. what's wrong with my script ? pls help. thanks #!/bin/sh curDIR=`pwd` oldExt='txt'... (10 Replies)
Discussion started by: champion
10 Replies
Login or Register to Ask a Question