11-28-2008
find and remove rows from file where multi occurrences of character found
I have a '~' delimited file of 6 - 7 million rows. Each row should contain 13 columns delimited by 12 ~'s. Where there are 13 tildes, the row needs to be removed. Each row contains alphanumeric data and occasionally a ~ ends up in a descriptive field and therefore acts as a delimiter, resulting in the row looking like it has 14 columns instead of 13. I have tried a combination of grep and awk but it is running very slowly. I suspect it is the way I am using it.
tried this to print the bad rows with line numbers to a file:
grep -n '~.*~.*~.*~.*~.*~.*~.*~.*~.*~.*~.*~.*~' inputfile | awk {print} > outputfile
also tried this to create a file with only the good rows in it:
grep -v '~.*~.*~.*~.*~.*~.*~.*~.*~.*~.*~.*~.*~' inputfile > outputfile
Both are extremely slow. The input file is approx. 800 meg
thanks
10 More Discussions You Might Find Interesting
1. UNIX for Advanced & Expert Users
I have a '~' delimited file of 6 - 7 million rows. Each row should contain 13 columns delimited by 12 ~'s. Where there are 13 tildes, the row needs to be removed. Each row contains alphanumeric data and occasionally a ~ ends up in a descriptive field and therefore acts as a delimiter, resulting... (7 Replies)
Discussion started by: kpd
7 Replies
2. UNIX for Dummies Questions & Answers
Hi to all,this is my first post here.
I've a file as
name
89
78
09
67
othername
how I can remove the word name and othername from this file, and an eventually blank row in it?Thanks in advance. (2 Replies)
Discussion started by: cv313x
2 Replies
3. UNIX for Dummies Questions & Answers
Hello,
I have "mastered" the counting of occurrences with uniq -c. :D
Now I need to keep that file neat and nice and remove only the occurrence number and the space after it.
Example:
1286456 aaa
164758 aab aaa
112345 aac aaf
should be turned into:
aaa
aab aaa
aac aaf
I... (6 Replies)
Discussion started by: hobbiecat
6 Replies
4. Shell Programming and Scripting
I have a file that contains several thousands rows. Here is an example.
^411912$
^487267$
^643776$
^682249$
^687737$
^692328$
^693767$
^695483$
^697289$
^757411$
^776688$
^778953$
^806123$
^872262$
^877877$
^839837$
^76666$
^72018$
^23330$ (3 Replies)
Discussion started by: justinb_155
3 Replies
5. Shell Programming and Scripting
Hi there,
i need some help to remove all occurrences of a certain character at the beginning of a string.
Example: my string is 00102030 and i want to remove all zeros from beginning of string so the result is 102030 (3 Replies)
Discussion started by: gigagigosu
3 Replies
6. Shell Programming and Scripting
*******************************************
* ROW *
*******************************************
CODE:CODE1
FILE: FILE1
FIELD: FIELD1
KEY: KEY1
ORA-00001: unique constraint (ETL.KEY_PK) violated
*******************************************
* ROW *
*******************************************... (7 Replies)
Discussion started by: Shanks
7 Replies
7. Shell Programming and Scripting
Hi - I have a file "file1" of below format. Its a comma seperated file. Note that each string is enclosed in double quotes.
"abc","-0.15","10,000.00","IJK"
"xyz","1,000.01","1,000,000.50","OPR"
I want the result as:
"abc","-0.15","10000.00","IJK"
"xyz","1,000.01","1000000.50","OPR"
I... (8 Replies)
Discussion started by: njny
8 Replies
8. UNIX for Advanced & Expert Users
I was thinking something like this but it always gets rid of the file location.
grep -roh base. | wc -l
find . -type f -exec grep -o base {} \; | wc -l
Would this be a job for awk? Would I need to store the file locations in an array? (3 Replies)
Discussion started by: cokedude
3 Replies
9. Shell Programming and Scripting
QL10169_SAUJANA%SubNetwork=ONRM_ROOT_MO_R,SubNetwork=ERBS_KCRN11,MeContext=QL10169_SAUJANA_5
%External_Link_Failure
%X2_link_problem_to_one_or_several_neighbouring_eNodeBs.
QL10187_MATANG_JAYA_2%SubNetwork=ONRM_ROOT_MO_R,SubNetwork=ERBS_KUCHING,MeContext=QL10187_MATANG_JAY A_2_3... (2 Replies)
Discussion started by: Ankit Vyas
2 Replies
10. Shell Programming and Scripting
I have a file which comes every day and the file data look's as below.
Vi abc.txt
a|b|c|d\n
a|g|h|j\n
Some times we receive the file with only a new line character in the file like
vi abc.txt
\n (8 Replies)
Discussion started by: rak Kundra
8 Replies
LEARN ABOUT SUSE
tk::table
Table(3) User Contributed Perl Documentation Table(3)
NAME
Tk::Table - Scrollable 2 dimensional table of Tk widgets
SYNOPSIS
use Tk::Table;
$table = $parent->Table(-rows => number,
-columns => number,
-scrollbars => anchor,
-fixedrows => number,
-fixedcolumns => number,
-takefocus => boolean);
$widget = $table->Button(...);
$old = $table->put($row,$col,$widget);
$old = $table->put($row,$col,"Text"); # simple Label
$widget = $table->get($row,$col);
$cols = $table->totalColumns;
$rows = $table->totalRows;
$table->see($widget);
$table->see($row,$col);
($row,$col) = $table->Posn($widget);
DESCRIPTION
Tk::Table is an all-perl widget/geometry manager which allows a two dimensional table of arbitary perl/Tk widgets to be displayed.
Entries in the Table are simply ordinary perl/Tk widgets. They should be created with the Table as their parent. Widgets are positioned in
the table using:
$table->put($row,$col,$widget)
If $widget is not a reference it is treated as a string, and a Lable widget is created with the string as its text.
All the widgets in each column are set to the same width - the requested width of the widest widget in the column. Likewise, all the
widgets in each row are set to the same height - the requested height of the tallest widget in the column.
A number of rows and/or columns can be marked as 'fixed' - and so can serve as 'headings' for the remainder the rows which are scrollable.
The requested size of the table as a whole is such that the number of rows specified by -rows (default 10), and number of columns specified
by -columns (default 10) can be displayed.
If the Table is told it can take the keyboard focus then cursor and scroll keys scroll the displayed widgets.
The Table will create and manage its own scrollbars if requested via -scrollbars.
The table can be emptied using
$table->clear
the widgets which were in the table are destroyed.
The Tk::Table widget is derived from a Tk::Frame, so inherits all its configure options.
BUGS
/ Snags / Possible enhancements
o Very large Tables consume a lot of X windows.
o No equivalent of pack's -anchor/-pad etc. options
SEE ALSO
Tk::grid, Tk::HList, Tk::TableMatrix, Tk::MListbox, Tk::Columns
perl v5.12.1 2007-11-08 Table(3)