need help on manipulating a file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting need help on manipulating a file
# 1  
Old 03-11-2011
Bug need help on manipulating a file

Hi,

I need a shell/command to achieve this task.

I've a delimited unloaded file from oracle in a scrambled format as shown below with many blank lines in it, I'm just trying to tailor it in a format that would be compatible to view and load it to a IDS db.

Here is the problem

Infile-

asd,sdfgsdf,,
<blank line 1>
fgsdfg, , , ,asfa,
<blank line 2>
<blank line 3>
sdfd,sdfd, ,
asd,asdd,
asdf, , , ,dsfd,
afdd,dsd

blank line 1 - potential data
blank line 2 - potential data
blank line 3 - record seperator

In the above ex, blank line1,2 is not a record seperator but it has a NULL data which is identified in the ASCII format. I'm trying to make the file into each data record in a single row as below, replacing the non-viewable data fields into some string (*$& here)

Output-
asd,sdfgsdf,, *$&,*$& ,fgsdfg, , , ,asfa, *$&
sdfd,sdfd, ,asd,asdd,asdf, , , ,dsfd,afdd,dsd

Please note that each blank line can have one or more fields in it

Thanks
# 2  
Old 03-11-2011
I'd try to look at those blank lines a little closer. Try to pipe them to 'od' : they should differ, if they contain different whitespace characters.
Then, you could use e.g. sed to replace the white chars with something else:
Code:
[MIRO@stroke]$ echo -e "\r"  | od -x
0000000 0a0d
0000002

[MIRO@stroke]$ echo -e "\r" | sed 's/\x0D/blah/g'
blah

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Manipulating column in a file in UNIX

Hi I have a file content.txt where I want to replace the value of second column of the file with half of the value of that column. I only have to replace if the second column starts with COM_, rest all values have to be same eg, cat content.txt |COM_A|123|JHV|8475 |ABC|2765|BV|876... (7 Replies)
Discussion started by: infyanurag
7 Replies

2. Shell Programming and Scripting

Manipulating csv file

We need to convert a field in a csv file which is in cents to dollars.(divide by 100) in our shell script. Can some body help me? (3 Replies)
Discussion started by: Deepthz
3 Replies

3. Shell Programming and Scripting

Help Manipulating Large Csv File

Hello everyone, I am trying to manipulate a large .csv file where I have output similar to the following - http://imgur.com/TEXD8.png The result that I am looking for would be to consolidate the first column, but combine the second and third column so it still relates to the first. I... (8 Replies)
Discussion started by: xxwohxx
8 Replies

4. Shell Programming and Scripting

Manipulating a file

Hi everybody, I need an urgent help with a BASH script. I have file which contains (besides the other data) the lines with the following structure identified by with keyword PCList: <PARAMETER NAME="PCList" TYPE="LIST_STRUCTURE" MODEL="{,}" ... (1 Reply)
Discussion started by: sameucho
1 Replies

5. Shell Programming and Scripting

Manipulating the etc/passwd file with sed

How can i use sed to extract the user name and home directory from the /etc/passwd/ file on my server. (11 Replies)
Discussion started by: Pauline mugisha
11 Replies

6. Shell Programming and Scripting

parsing a file and manipulating the contents

Hi I have a text file as follows BOB 14/14 TOM 94/94 SAM 3/3 CRIS 13/13 TOM 6/6 CRIS 27/27 SAM 2/2 JACK 25/25 CRIS (6 Replies)
Discussion started by: shellignorant
6 Replies

7. UNIX for Dummies Questions & Answers

Help!! manipulating file

Hi all, I need help manipulating the file below. Here is what I needed to do. First, I have to replace INSUPD to DELETE. Then I need to change the content of the file around by flipping the contents in the file from the bottom to the top (start from "CMD") How should I attack this? Here... (2 Replies)
Discussion started by: sirrtuan
2 Replies

8. Solaris

Manipulating File

Help...please. I have a log that contains Warning Authentication Failed: User GHDT88998HS doesn't exit: The User GHDT88998HS could not be found Mar 22, 2008 5:22:22AM com.hometel.ttm.auth.userlogin. about maybe a thousand entries failed user acct message How can I grab just the username... (2 Replies)
Discussion started by: rivendell500
2 Replies

9. Shell Programming and Scripting

Manipulating a text file

hey guys, need ur expert help. m a core banker got stuck in someting techie and cant find a solution have manged to extract a file from oracle apps in a format that looks something like this... REC- A b c d x INV- A b... (6 Replies)
Discussion started by: komalkg
6 Replies

10. UNIX for Advanced & Expert Users

Manipulating output file

I have a file containing two fields, Name and Time, with about 57 lines in this file. I am struggling to create a loop that will cut out the first ten lines of this file and echo it to the screen. Can anybody help me please. (1 Reply)
Discussion started by: mariner
1 Replies
Login or Register to Ask a Question