Sponsored Content
Top Forums Shell Programming and Scripting To check Blank Lines, Blank Records and Junk Characters in a File Post 302943348 by chatwithsaurav on Thursday 7th of May 2015 02:08:09 PM
Old 05-07-2015
To check Blank Lines, Blank Records and Junk Characters in a File

Hi All

Need Help

I have a file with the below format (ABC.TXT) :

Code:
®¿¿ABCDHEJJSJJ|XCBJSKK01|M|7348974982790
HDFLJDKJSKJ|KJALKSD02|M|7378439274898
KJHSAJKHHJJ|LJDSAJKK03|F|9898982039999
(cont......)

I need to write a script where it will check for : blank lines (between rows,before first line and after last line) and remove them. Also if there are any blank records between the "|" delimeter it will identify the row number and send a mail. Finally in case there are "junk characters" it will identify and remove them. How should I go about it?

Last edited by Scrutinizer; 05-07-2015 at 05:18 PM.. Reason: CODE tags
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Deleting the blank line in a file and counting the characters....

Hi, I am trying to do two things in my script. I will really appreciate any help in this regards. Is there a way to delete a last line from a pipe delimited flat file if the last line is blank. If the line is not blank then do nothing..... Is there a way to count a word that are starting... (4 Replies)
Discussion started by: rkumar28
4 Replies

2. Shell Programming and Scripting

how to check weather file is blank or not?

Dear All I want to do following task. Kindly suggest. In my script every hour one file is genarated. say xyz.txt. Now if this file contain some data then i want to do task A and if file is blank then i want to do nothing. Kindly help me to do this. regards jaydeep (5 Replies)
Discussion started by: jaydeep_sadaria
5 Replies

3. Shell Programming and Scripting

Can't remove blank lines from a file

Hi Guys, I have been trying to remove blank lines from a file with no success. I tried using all the following options on the file: tr -s '\n' < abc.txt grep -v "^$" abc.txt sed '/^$/d' abc.txt sed '/./!d' abc.txt awk '/./' abc.txt The file is a text file. (11 Replies)
Discussion started by: npatwardhan
11 Replies

4. Shell Programming and Scripting

Delete blank lines, if blank lines are more than one using shell

Hi, Consider a file named "testfile" The contents of file are as below first line added for test second line added for test third line added for test fourth line added for test fifth line added for test (5 Replies)
Discussion started by: anil8103
5 Replies

5. Shell Programming and Scripting

Delete blank lines in a file

Hi All, I have a file and I need to delete the lines that are blank and is starting with some characters below. Something like below: Regular Ascii File: Line1: AGODA1 BUSAN||SK Lord Beach 4/6/2012 4/7/2012 68060 Line2: AGODA2 BUSAN||SK Beach Hotel 4/6/2012 4/7/2012 610200 Line3: ... (4 Replies)
Discussion started by: rkumar28
4 Replies

6. Shell Programming and Scripting

Reform Lines in File without blank lines and spaces

Hello All, I have a file with data as below. Each line consists of 21 fields. I am not able to load them back to the database. 50733339,"834","834 ","005010X279A1","N","Y","007977163","0001 ",30,"2110D ","EB ","EB007 ","2 ","Conditional Required Data Element Miss ing... (3 Replies)
Discussion started by: Praveenkulkarni
3 Replies

7. Shell Programming and Scripting

Removing blank lines from a file

Hi All, How do i remove continuos blank lines from a file. I have a file with data: abc; def; ghi; jkl; mno; pqr; In the above file, there are two blank lines. I want to remove, one out of them. My output should look like: (2 Replies)
Discussion started by: raosr020
2 Replies

8. Shell Programming and Scripting

How to check if the file is empty or has blank space.?

Hi, I am using KSH. I am trying to check if the output file is empty or not. I tried with ] but what i see is my file is empty but still manages to have a size of 1 instead of 0. But my file doesnot have anything its empty. I am not sure how to check this. can any one help? (10 Replies)
Discussion started by: Sharma331
10 Replies

9. UNIX for Advanced & Expert Users

Delete blank spaces and blank lines in a file

Hi Gurus, Somebody can say me how to delete blank spaces and blank lines in a file unix, please. Thank you for advanced. (10 Replies)
Discussion started by: systemoper
10 Replies

10. Shell Programming and Scripting

Script to find blank records in a file except for few columns

I have a file with the following format: X|High|2|GIC|DM||XHM|||6 Months X|Moderate|2|GIC|DM||XHM|||6 Months X|High|2|GCM|DM||XSF|||6 Months X|Med|2|GCM|DM||XSF|||6 Here there are ten columns but I need to print rows having blank records in any of the rows (except for 6th,8th and 9th... (10 Replies)
Discussion started by: chatwithsaurav
10 Replies
mbox(5)                                                         File Formats Manual                                                        mbox(5)

NAME
mbox - file containing mail messages INTRODUCTION
The most common format for storage of mail messages is mbox format. An mbox is a single file containing zero or more mail messages. MESSAGE FORMAT
A message encoded in mbox format begins with a From_ line, continues with a series of non-From_ lines, and ends with a blank line. A From_ line means any line that begins with the characters F, r, o, m, space: From god@heaven.af.mil Sat Jan 3 01:05:34 1996 Return-Path: <god@heaven.af.mil> Delivered-To: djb@silverton.berkeley.edu Date: 3 Jan 1996 01:05:34 -0000 From: God <god@heaven.af.mil> To: djb@silverton.berkeley.edu (D. J. Bernstein) How's that mail system project coming along? The final line is a completely blank line (no spaces or tabs). Notice that blank lines may also appear elsewhere in the message. The From_ line always looks like From envsender date moreinfo. envsender is one word, without spaces or tabs; it is usually the envelope sender of the message. date is the delivery date of the message. It always contains exactly 24 characters in asctime format. moreinfo is optional; it may contain arbitrary information. Between the From_ line and the blank line is a message in RFC 822 format, as described in qmail-header(5), subject to >From quoting as described below. HOW A MESSAGE IS DELIVERED
Here is how a program appends a message to an mbox file. It first creates a From_ line given the message's envelope sender and the current date. If the envelope sender is empty (i.e., if this is a bounce message), the program uses MAILER-DAEMON instead. If the envelope sender contains spaces, tabs, or newlines, the program replaces them with hyphens. The program then copies the message, applying >From quoting to each line. >From quoting ensures that the resulting lines are not From_ lines: the program prepends a > to any From_ line, >From_ line, >>From_ line, >>>From_ line, etc. Finally the program appends a blank line to the message. If the last line of the message was a partial line, it writes two newlines; oth- erwise it writes one. HOW A MESSAGE IS READ
A reader scans through an mbox file looking for From_ lines. Any From_ line marks the beginning of a message. The reader should not attempt to take advantage of the fact that every From_ line (past the beginning of the file) is preceded by a blank line. Once the reader finds a message, it extracts a (possibly corrupted) envelope sender and delivery date out of the From_ line. It then reads until the next From_ line or end of file, whichever comes first. It strips off the final blank line and deletes the quoting of >From_ lines and >>From_ lines and so on. The result is an RFC 822 message. COMMON MBOX VARIANTS
There are many variants of mbox format. The variant described above is mboxrd format, popularized by Rahul Dhesi in June 1995. The original mboxo format quotes only From_ lines, not >From_ lines. As a result it is impossible to tell whether From: djb@silverton.berkeley.edu (D. J. Bernstein) To: god@heaven.af.mil >From now through August I'll be doing beta testing. Thanks for your interest. was quoted in the original message. An mboxrd reader will always strip off the quoting. mboxcl format is like mboxo format, but includes a Content-Length field with the number of bytes in the message. mboxcl2 format is like mboxcl but has no >From quoting. These formats are used by SVR4 mailers. mboxcl2 cannot be read safely by mboxrd readers. UNSPECIFIED DETAILS
There are many locking mechanisms for mbox files. qmail-local always uses flock on systems that have it, otherwise lockf. The delivery date in a From_ line does not specify a time zone. qmail-local always creates the delivery date in GMT so that mbox files can be safely transported from one time zone to another. If the mtime on a nonempty mbox file is greater than the atime, the file has new mail. If the mtime is smaller than the atime, the new mail has been read. If the atime equals the mtime, there is no way to tell whether the file has new mail, since qmail-local takes much less than a second to run. One solution is for a mail reader to artificially set the atime to the mtime plus 1. Then the file has new mail if and only if the atime is less than or equal to the mtime. Some mail readers place Status fields in each message to indicate which messages have been read. SEE ALSO
maildir(5), qmail-header(5), qmail-local(8) mbox(5)
All times are GMT -4. The time now is 04:27 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy