Remove the footer from a flat file by searching a pattern


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Remove the footer from a flat file by searching a pattern
# 1  
Old 10-02-2011
MySQL Remove the footer from a flat file by searching a pattern

Hi,
I have more than 30,000 records in a flat file. I want to remove footer from the file by searching a string pattern for the footer.
Example.... let the flat file test.dat contains below records.

record1
record2
record3
..
..
..
record31000
Total records 31000

I want to remove the last line ("Total records 31000") from the file by searching for the string "Total records". I want to modify the input file by removing the line, i don't want to use a separate file as output file due to the large size of the input file. please suggest me.

Thanks in advance.
# 2  
Old 10-02-2011
Try this...
Code:
sed -i '/Total records/ d' input_file

--ahamed
# 3  
Old 10-02-2011
Thanks Ahamed.
But I think your code needs an output file for writing the output. I need to modify the input file as the size of the input file is large.
# 4  
Old 10-02-2011
No, the "-i" option will modify the original file itself!

--ahamed
# 5  
Old 10-03-2011
using awk ..
Code:
$ awk '!/Total records/' infile

# 6  
Old 10-03-2011
Hi ahamed,
I think the option "-i" is shell specific. This command is working fine in bash, but not working in ksh. Please suggest me if you have any suggestions(code). thanks...

Hi jayan_jay,
This command is fine but needs an output file for writing the output. I need to modify the input file as the size of the input file is large. Please suggest me if you have any idea for this. Thanks...
# 7  
Old 10-03-2011
which OS are you using? If Solaris, try using /usr/xpg4/bin/sed

--ahamed
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove first NULL Character in Flat File

We have a flat file with below data : ^@^@^@^@00000305^@^@^@^@^@^@430^@430^@^@^@^@^@^@^@^@^@09079989530As we can see ^@ is Null character in this file I want to remove only the first few null characters before string 00000305 How can we do that, any idea. I want a new file without first few... (5 Replies)
Discussion started by: simpltyansh
5 Replies

2. UNIX for Advanced & Expert Users

Remove duplicates in flat file

Hi all, I have a issues while loading a flat file to the DB. It is taking much time. When analyzed i found out that there are duplicates entry in the flat file. There are 2 type of Duplicate entry. 1) is entire row is duplicate. ( i can use sort | uniq) to remove the duplicated entry. 2) the... (4 Replies)
Discussion started by: samjoshuab
4 Replies

3. Shell Programming and Scripting

Searching for pattern and remove the lines

Hi , I want to remove the specific pattern and remove those lines from file using shell script. i want to remove these lines <?xml version='1.0' encoding='UTF-8'?> <row_set> </row_set> my input file has content like this. file name: sample.xml <?xml version='1.0'... (4 Replies)
Discussion started by: nukala_2
4 Replies

4. UNIX for Dummies Questions & Answers

How to remove numeric characters in the flat file

HI, can any one help me please .. i have flat file like qwer123rt ass3242ccf jjk654 kjh838ppp nhdg453ok hdkk34 i want remove numeric characters in the flat file i want output like this qwerrt assccf jjk kjhppp nhdgok hdkk help me... (4 Replies)
Discussion started by: rafimd1985
4 Replies

5. Shell Programming and Scripting

Remove somewhat Duplicate records from a flat file

I have a flat file that contains records similar to the following two lines; 1984/11/08 7 700000 123456789 2 1984/11/08 1941/05/19 7 700000 123456789 2 The 123456789 2 represents an account number, this is how I identify the duplicate record. The ### signs represent... (4 Replies)
Discussion started by: jolney
4 Replies

6. Shell Programming and Scripting

Searching for Log / Bad file and Reading and writing to a flat file

Need to develop a unix shell script for the below requirement and I need your assistance: 1) search for file.log and file.bad file in a directory and read them 2) pull out "Load_Start_Time", "Data_File_Name", "Error_Type" from log file 4) concatinate each row from bad file as... (3 Replies)
Discussion started by: mlpathir
3 Replies

7. Shell Programming and Scripting

Searching a pattern in file and deleting th ewhole line containing the pattern

Hi All, Please can someone assist in the script I have made that searches a pattern in a file and delete the whole line containing the pattern. #!bin/sh # The pattern that user want to add to the files echo "Enter the pattern of the redirect" read value # check if the user has... (1 Reply)
Discussion started by: Shazin
1 Replies

8. UNIX for Dummies Questions & Answers

how to remove the first line from a flat file ?

Hi, I want to remove the first line from a flat file using unix command as simple as possible. Can anybody give me a hand ? Thanks in advance. xli (21 Replies)
Discussion started by: xli
21 Replies

9. Shell Programming and Scripting

Adding a footer to a flat file

I wanted to know if there is a way to add a footer to a flat file that is being extracted by a sql query. so if the query extracts 1000 rows, in the flat file it appends the last line as 1000. Is there a way to do so? Thanks (1 Reply)
Discussion started by: rudoraj
1 Replies

10. Shell Programming and Scripting

How to remove page breaks from a flat file???

Hi All, I get a flat file with its last field data splitting onto a new line.I got this program from Vgersh which when run would cancatenate the split data back to the end of the previous records.But this program fails when it encounters a page break between the split data and the previous... (5 Replies)
Discussion started by: kumarsaravana_s
5 Replies
Login or Register to Ask a Question