01-14-2011
Find line number of bad data in large file
Hi Forum.
I was trying to search the following scenario on the forum but was not able to.
Let's say that I have a very large file that has some bad data in it (for ex: 0.0015 in the 12th column) and I would like to find the line number and remove that particular line.
What's the easiest way to do so?
For a smaller file, I could use the "vi editor", edit the file, search for the bad data in the specific column and then just delete the row.
But for a larger file where using the "vi editor" is out of the question.
Cannot really use the grep -v "0.0015" option since "0.0015" value could be valid for other rows and which is not in the 12th column.
I do not know what is the line number where the bad data resides.
Thanks.
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
Hi all;
I'm having a problem when want to list a large number of files in current directory using find together with the prune option.
First i used this command but it list all the files including those in sub directories:
find . -name "*.dat" | xargs ls -ltr
Then i modified the command... (2 Replies)
Discussion started by: ashikin_8119
2 Replies
2. Shell Programming and Scripting
Hi Everybody,
I am trying to write a script that will get some perticuler data from a file and redirect to a file.
My Question is,
I have a Very huge file,In that file I have my required data is started from 25th line and it will ends in 100th line.
I know the line numbers, I need to get all... (9 Replies)
Discussion started by: Anji
9 Replies
3. Solaris
System Solaris 8
When I open a CONSOLE window the following starts scrolling:
"ServiceCommand: :write: Bad FIle Number"
This will continue to scroll without stopping. However, you can type while it is scrolling and login into root and even conduct business within the CONSOLE window. The... (1 Reply)
Discussion started by: Kevin1166
1 Replies
4. Programming
Hi All,
I don't need any code for this just some advice. I have a large collection of heterogeneous data (about 1.3 million) which simply means data of different types like float, long double, string, ints. I have built a linked list for it and stored all the different data types in a structure,... (5 Replies)
Discussion started by: shoaibjameel123
5 Replies
5. Shell Programming and Scripting
Hi All,
I have searched this forum for related posts but could not find one that fits mine. I have a shell script which removes all the XML tags including the text inside the tags from some 4 million XML files.
The shell script looks like this (MODIFIED):
find . "*.xml" -print | while read... (6 Replies)
Discussion started by: shoaibjameel123
6 Replies
6. Shell Programming and Scripting
Hi
I have requirement to find nth occurrence in a file and capture data from with in lines (between lines)
Data in File.
<QUOTE>
<SESSION>
<ATTRIBUTE NAME='Parameter Filename' VALUE='file1.parm'/>
<ATTRIBUTE NAME='Service Name' VALUE='None'/>
</SESSION>
<SESSION>
<ATTRIBUTE... (6 Replies)
Discussion started by: tmalik79
6 Replies
7. Shell Programming and Scripting
Hello. I was wondering if anyone could help. I have a file containing a large table in the format:
marker1 marker2 marker3 marker4
position1 position2 position3 position4
genotype1 genotype2 genotype3 genotype4
with marker being a name, position a numeric... (2 Replies)
Discussion started by: davegen
2 Replies
8. Shell Programming and Scripting
Hi
I want to use awk to match where field 3 contains a number within string - then print the line and just the number as a new field.
The source file is pipe delimited and looks something like
1|net|ABC Letr1|1530|||
1|net|EXP_1040 ABC|1121|||
1|net|EXP_TG1224|1122|||
1|net|R_North|1123|||... (5 Replies)
Discussion started by: Mudshark
5 Replies
9. Shell Programming and Scripting
Hi i have some large data files that contain several fields and rows the data in a field have a numeric value that is in a sine wave pattern what i would like todo is locate each peak and pick the highest value and print that complete line. the data looks something like this it is field nr4 which... (4 Replies)
Discussion started by: ninjaunx
4 Replies
10. Shell Programming and Scripting
I have a .csv file that has been create from a google form and I need to extract the data from it that has been entered by users.
The CSV will have anywhere between 100 and 1000 lines which comprise entr data for a sports carnival
A few typical line is shown here to show the problem I have
... (19 Replies)
Discussion started by: kcpoole
19 Replies
LEARN ABOUT FREEBSD
badsect
BADSECT(8) BSD System Manager's Manual BADSECT(8)
NAME
badsect -- create files to contain bad sectors
SYNOPSIS
badsect bbdir sector ...
DESCRIPTION
The badsect utility makes a file to contain a bad sector. Normally, bad sectors are made inaccessible by the standard formatter, which pro-
vides a forwarding table for bad sectors to the driver. If a driver supports the bad blocking standard it is much preferable to use that
method to isolate bad blocks, since the bad block forwarding makes the pack appear perfect, and such packs can then be copied with dd(1).
The technique used by this program is also less general than bad block forwarding, as badsect cannot make amends for bad blocks in the i-list
of file systems or in swap areas.
On some disks, adding a sector which is suddenly bad to the bad sector table currently requires the running of the standard DEC formatter.
Thus to deal with a newly bad block or on disks where the drivers do not support the bad-blocking standard badsect may be used to good
effect.
The badsect utility is used on a quiet file system in the following way: First mount the file system, and change to its root directory. Make
a directory BAD there. Run badsect giving as argument the BAD directory followed by all the bad sectors you wish to add. (The sector num-
bers must be relative to the beginning of the file system, but this is not hard as the system reports relative sector numbers in its console
error messages.) Then change back to the root directory, unmount the file system and run fsck(8) on the file system. The bad sectors should
show up in two files or in the bad sector files and the free list. Have fsck(8) remove files containing the offending bad sectors, but do
not have it remove the BAD/nnnnn files. This will leave the bad sectors in only the BAD files.
The badsect utility works by giving the specified sector numbers in a mknod(2) system call, creating an illegal file whose first block
address is the block containing bad sector and whose name is the bad sector number. When it is discovered by fsck(8) it will ask ``HOLD BAD
BLOCK ?''. A positive response will cause fsck(8) to convert the inode to a regular file containing the bad block.
DIAGNOSTICS
The badsect utility refuses to attach a block that resides in a critical area or is out of range of the file system. A warning is issued if
the block is already in use.
SEE ALSO
fsck(8)
HISTORY
The badsect utility appeared in 4.1BSD.
BUGS
If more than one sector which comprise a file system fragment are bad, you should specify only one of them to badsect, as the blocks in the
bad sector files actually cover all the sectors in a file system fragment.
BSD
June 5, 1993 BSD