Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Please help !!!!Problem with data file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Please help !!!!Problem with data file
# 1  
Old 04-07-2007
Please help !!!!Problem with data file
I have a 1 million record file and in there i have 580 bad records meaning like they were spread on to 2 lines and is making my process fail as it is expecting single line records. How can i correct this records spread into 2 lines into single line or how i can writer these records into a new file and delete from the original flle so that i can worry about the bad records later. Please let me know.
# 2  
Old 04-07-2007
dsravan,
Without a sample of the data we cannot help you.
Please, display a sample with the "good" and the "bad" records with no more than 10 or 15 records total.
# 3  
Old 04-07-2007
how i can display a reocrd namely 20000 in a million reocrd file. I am getting the error temp file too large when trying to open in vi. Can i know the sed command to display that record.
# 4  
Old 04-07-2007
Try using "less" or "more" commands, or give a sample of the first 10 as:
head -10 file_name
# 5  
Old 04-07-2007
Iused had and tail commands and got the file

Quote:
321916335|CITY OF JASPER-R GSM AL WSCA CRU||||||4994||||
321916337|CITY OF JASPER-N GSM AL WSCA IRU||||||4994||||
321916341|NEW GENERATION CONSTRUCTION-N CBE CRU||||||4994||||
321916343|NEW GENERATION CONSTRUCTION-N CBE IRU||||||4994||||
309514681|TRACFONE|TRACFONE|||||4994||||
with naked eye it seems the file is ok. But the reocrd that is having problem is the 3rd one from above.
# 6  
Old 04-07-2007
dsravan,
You said:
" have 580 bad records meaning like they were spread on to 2 lines"
The 3rd record in your sample is not spread on to 2 lines.
Could you please clarify?
Thanks.
# 7  
Old 04-07-2007
Quote:
Originally Posted by dsravan
how i can display a reocrd namely 20000 in a million reocrd file. I am getting the error temp file too large when trying to open in vi. Can i know the sed command to display that record.

sed -n 20000p
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Replace data of a file with data from another file using shell scripting.

Dears, I'm new to shell scripting and i was wondering if you can help me with following matter. I have a file containing 400,000 records. The file contains two columns like: 00611291,0270404000005453 25262597,1580401000016155 25779812,1700403000001786 00388934,1200408000000880... (1 Reply)
Discussion started by: paniklas
1 Replies

2. Shell Programming and Scripting

Extract header data from one file and combine it with data from another file

Hi, Great minds, I have some files, in fact header files, of CTD profiler, I tried a lot C programming, could not get output as I was expected, because my programming skills are very poor, finally, joined unix forum with the hope that, I may get what I want, from you people, Here I have attached... (17 Replies)
Discussion started by: nex_asp
17 Replies

3. UNIX for Dummies Questions & Answers

Mapping a data in a file and delete line in source file if data does not exist.

Hi Guys, Please help me with my problem here: I have a source file: 1212 23232 343434 ASAS1 4 3212 23232 343434 ASAS2 4 3234 23232 343434 QWQW1 4 1134 23232 343434 QWQW2 4 3212 23232 343434 QWQW3 4 and a mapping... (4 Replies)
Discussion started by: kokoro
4 Replies

4. Shell Programming and Scripting

Three Difference File Huge Data Comparison Problem.

I got three different file: Part of File 1 ARTPHDFGAA . . Part of File 2 ARTGHHYESA . . Part of File 3 ARTPOLYWEA . . (4 Replies)
Discussion started by: patrick87
4 Replies

5. Shell Programming and Scripting

Scan and change file data content problem

Input file >Read_1 XXXXXXXXXXSDFXXXXXDS (condition 1: After the last "X" per line, if the distance is less than or equal to 3 letter, replace those not "X" letter with "X") TREXXXXXXXSDFXXXXXDS (condition 2: Before the first "X" per line, if the distance is less than or equal to 3 letter,... (12 Replies)
Discussion started by: patrick87
12 Replies

6. Shell Programming and Scripting

Compare and print out data only appear in file 1 problem

Below is the data content of file_1 and file_2: file_1 >sample_1 FKGJGPOPOPOQA ASDADWEEWERE ASDAWEWQWRW ASDASDASDASDD file_2 >sample_1 DRTOWPFPOPOQA ASDADWEEASDF ASDADRTYWRW ASDASDASDASDD I got try the following perl script. Unfortunately, it can't give my desired output result... (7 Replies)
Discussion started by: patrick87
7 Replies

7. Shell Programming and Scripting

Find and replace data in text file with data in same file

OK I will do my best to explain what I need help with. I am trying to format an ldif file so I can import it into Oracle oid. I need the file to look like this example. Keep in mind there are 3000 of these in the file. changetype: modify replace: userpassword dn:... (0 Replies)
Discussion started by: timothyha22
0 Replies

8. Shell Programming and Scripting

C Shell problem: using a key from one file to find data in another

I've never written scripts (just switched from Ada to C++). I have a book that's over my head and a few examples, other then that I'm floundering. Everything here at work is being done in C Shell. None of the C++ programmers are experienced in shell scripting. I have a data file with the... (2 Replies)
Discussion started by: bassmaster
2 Replies

9. Shell Programming and Scripting

Problem in writing the data to a file in one row

Hi All I am reading data from the database and writing to temporary file in the below format. 1=XP|external_component|com.adp.meetingalertemail.processing.MeetingAlertEmail|EMAILALERTPUSH|32|4#XP |classpath|/usr/home/dfusr/lib/xalan.jar: /usr/home/dfusr/lib/xerces.jar: ... (2 Replies)
Discussion started by: rajeshorpu
2 Replies

10. Shell Programming and Scripting

Problem getting data to a report file.

Hi all, I'm trying in vain to workout how I can generate a report from a months worth of files that get created every day. There is one file per day and each daily file contain the output from a df -v command. With the following section of code ... for xdffile in $1$2/df?? do ... (4 Replies)
Discussion started by: Cameron
4 Replies
Login or Register to Ask a Question