Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Please help !!!!Problem with data file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Please help !!!!Problem with data file
# 8  
Old 04-07-2007
my process throws out erro syaing that record doesn't have enough number of vartext records in that file. I am not able to see with naked eye whats wrong with that line which is the bad record.
# 9  
Old 04-07-2007
Don't know what to tell you. Maybe your count is off a bit, so try:

sed -n 19995,20005p
# 10  
Old 04-07-2007
Here is a awk one-liner to find records without 12 fields..

Code:
$ cat data
321916335|CITY OF JASPER-R GSM AL WSCA CRU||||||4994||||
321916337|CITY OF JASPER-N GSM AL WSCA IRU||||||4994||||
321916341|NEW GENERATION CONSTRUCTION-N CBE CRU||||||4994|||||
321916343|NEW GENERATION CONSTRUCTION-N CBE IRU||||||4994||||
309514681|TRACFONE|TRACFONE|||||4994||||
$ awk -F'|' 'NF != 12 {print "record ", NR, " is ", $0 }'  < data
record  3  is  321916341|NEW GENERATION CONSTRUCTION-N CBE CRU||||||4994|||||
$

I added an extra field to a record for testing.
# 11  
Old 04-07-2007
here is the file contenets:

Quote:
295468075|GOLDKIST|||||||0|||
295468077|SIEMENS MEDICAL CSG-INV|||||||0|||
295468079|KOSA/AREVA SPCLTY SARL|||||||0|||
295468081|IBM
|||||||0|||
295468083|COMCAST - INVOICING|||||||0|||
see that one record has span into two lines. I want to make them on to singlr line. It's a 1 millino file and i can't do that manually. please let me know how i can starighten them out.
# 12  
Old 04-07-2007
Quote:
Originally Posted by dsravan
my process throws out erro syaing that record doesn't have enough number of vartext records in that file. I am not able to see with naked eye whats wrong with that line which is the bad record.
assuming all the records/lines in your file are supposed to have the same number of fields, what IS this number? In other words, what is the number of '|' separated fields that your 'application' is expecting?
# 13  
Old 04-07-2007
Get the line numbers for the bad two records... let's say 12001 and 12002...

sed -n 1,12000 > piece1 < oldfile
sed -n 12001,12002 > piece2 < oldfile
sed -n 12003,1000000 > piece3 < oldfile

vi piece2

cat piece1 piece2 piece3 > newfile
# 14  
Old 04-07-2007
vgresh,

Each record has 11 pipes in it for sure. And there are 12 data values in each record.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Replace data of a file with data from another file using shell scripting.

Dears, I'm new to shell scripting and i was wondering if you can help me with following matter. I have a file containing 400,000 records. The file contains two columns like: 00611291,0270404000005453 25262597,1580401000016155 25779812,1700403000001786 00388934,1200408000000880... (1 Reply)
Discussion started by: paniklas
1 Replies

2. Shell Programming and Scripting

Extract header data from one file and combine it with data from another file

Hi, Great minds, I have some files, in fact header files, of CTD profiler, I tried a lot C programming, could not get output as I was expected, because my programming skills are very poor, finally, joined unix forum with the hope that, I may get what I want, from you people, Here I have attached... (17 Replies)
Discussion started by: nex_asp
17 Replies

3. UNIX for Dummies Questions & Answers

Mapping a data in a file and delete line in source file if data does not exist.

Hi Guys, Please help me with my problem here: I have a source file: 1212 23232 343434 ASAS1 4 3212 23232 343434 ASAS2 4 3234 23232 343434 QWQW1 4 1134 23232 343434 QWQW2 4 3212 23232 343434 QWQW3 4 and a mapping... (4 Replies)
Discussion started by: kokoro
4 Replies

4. Shell Programming and Scripting

Three Difference File Huge Data Comparison Problem.

I got three different file: Part of File 1 ARTPHDFGAA . . Part of File 2 ARTGHHYESA . . Part of File 3 ARTPOLYWEA . . (4 Replies)
Discussion started by: patrick87
4 Replies

5. Shell Programming and Scripting

Scan and change file data content problem

Input file >Read_1 XXXXXXXXXXSDFXXXXXDS (condition 1: After the last "X" per line, if the distance is less than or equal to 3 letter, replace those not "X" letter with "X") TREXXXXXXXSDFXXXXXDS (condition 2: Before the first "X" per line, if the distance is less than or equal to 3 letter,... (12 Replies)
Discussion started by: patrick87
12 Replies

6. Shell Programming and Scripting

Compare and print out data only appear in file 1 problem

Below is the data content of file_1 and file_2: file_1 >sample_1 FKGJGPOPOPOQA ASDADWEEWERE ASDAWEWQWRW ASDASDASDASDD file_2 >sample_1 DRTOWPFPOPOQA ASDADWEEASDF ASDADRTYWRW ASDASDASDASDD I got try the following perl script. Unfortunately, it can't give my desired output result... (7 Replies)
Discussion started by: patrick87
7 Replies

7. Shell Programming and Scripting

Find and replace data in text file with data in same file

OK I will do my best to explain what I need help with. I am trying to format an ldif file so I can import it into Oracle oid. I need the file to look like this example. Keep in mind there are 3000 of these in the file. changetype: modify replace: userpassword dn:... (0 Replies)
Discussion started by: timothyha22
0 Replies

8. Shell Programming and Scripting

C Shell problem: using a key from one file to find data in another

I've never written scripts (just switched from Ada to C++). I have a book that's over my head and a few examples, other then that I'm floundering. Everything here at work is being done in C Shell. None of the C++ programmers are experienced in shell scripting. I have a data file with the... (2 Replies)
Discussion started by: bassmaster
2 Replies

9. Shell Programming and Scripting

Problem in writing the data to a file in one row

Hi All I am reading data from the database and writing to temporary file in the below format. 1=XP|external_component|com.adp.meetingalertemail.processing.MeetingAlertEmail|EMAILALERTPUSH|32|4#XP |classpath|/usr/home/dfusr/lib/xalan.jar: /usr/home/dfusr/lib/xerces.jar: ... (2 Replies)
Discussion started by: rajeshorpu
2 Replies

10. Shell Programming and Scripting

Problem getting data to a report file.

Hi all, I'm trying in vain to workout how I can generate a report from a months worth of files that get created every day. There is one file per day and each daily file contain the output from a df -v command. With the following section of code ... for xdffile in $1$2/df?? do ... (4 Replies)
Discussion started by: Cameron
4 Replies
Login or Register to Ask a Question