12-01-2009
Deleting columns by list or file
Dear specialists out there, please help a poor awk newbie:
I have a very huge file to process consisting of 300000 columns and 1500 rows. About 20000 columns shall be deleted from that file. So it is clear, that I can't do this by writing down all the columns in an awk command like $1, $x etc.
I have a text-file containing one column with the identifiers for these 20000 columns, that shall be removed (corresponding to the identifiers in the header/first line in file to process).
Can anyone give me a hint ho to do this with awk or a shell script?
I'd appreciate any kind of help very much!
Best regards, Felix
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi Friends,
I want to delete specific columns from a file.
Say my file content is as follows:
"1","a","ww1",1234"
"2","b","wwr3","2222"
"3","c","erre","3333"
Now i want to delete the column 2 and 4 from this file.
That is I want the file content to be:
"1","ww1"
"2","wwr3"... (11 Replies)
Discussion started by: premar
11 Replies
2. Shell Programming and Scripting
Hi ,
I want to delete some rows & columns from file.
can someone please help me on this?
Regards. (2 Replies)
Discussion started by: code19
2 Replies
3. Shell Programming and Scripting
Hi All,
Am working on perl script which should delete columns in existing CSV file.
If my file is :
AA,BB,CC,DD
00,11,22,33
00,55,66,77
00,99,88,21
AA,BB... are all my headers can come in any order (e.g AA,CC,BB...) and rest are values. I want to delete column CC...
Can anybody help... (2 Replies)
Discussion started by: darshakraut
2 Replies
4. Shell Programming and Scripting
This should follow with my last post but I think it's better to start a new one. Now I have a list of numbers stored in pos.txt:
2
6
7
.
.
.
n
.
.
.
And I need to extract (2n-1, 2n) columns from matrix.txt:
ind1 A B C D E F G H I J K L M N O P Q R ...
ind2 B C D E F G H... (3 Replies)
Discussion started by: Zoho
3 Replies
5. UNIX for Dummies Questions & Answers
I have a space delimited text file with 1,000,000+ columns and 100 rows. I want to delete columns 2 through 5 (2 and 5) included from the text file. How do I do that? Thanks. (3 Replies)
Discussion started by: evelibertine
3 Replies
6. UNIX for Dummies Questions & Answers
I have a tab limited text file with 10000+ columns. I want to delete columns 6 through 23, how do I go about doing that? Thanks! (1 Reply)
Discussion started by: evelibertine
1 Replies
7. UNIX for Advanced & Expert Users
Hi All,
i have a .Csv file in the below format
startTime, endTime, delta, gName, rName, rNumber, m2239max, m2239min, m2239avg, m100016509avg, m100019240max, metric3min, m100019240avg, propValues
11-Mar-2012 00:00:00, 11-Mar-2012 00:05:00, 300.0, vma3550a, a-1_CPU Index<1>, 200237463, 0.0,... (9 Replies)
Discussion started by: mahi_mayu069
9 Replies
8. Shell Programming and Scripting
Hi group,
Can you please tell how to delete specific columns from a file.
I know something like
awk -F, '{ print $1" "$2" "15 }' input.txt > output.txt will delete all other columns. But this is in a way to copy some particular columns.
But is there any other way to select just some... (11 Replies)
Discussion started by: smitra
11 Replies
9. Shell Programming and Scripting
Hi Friends,
I have come across some files where some of the columns don not have data.
Key, Data1,Data2,Data3,Data4,Data5
A,5,6,,10,,
A,3,4,,3,,
B,1,,4,5,,
B,2,,3,4,,
If we see the above data on Data5 column do not have any row got filled. So remove only that column(Here Data5) and... (4 Replies)
Discussion started by: ks_reddy
4 Replies
10. Shell Programming and Scripting
Hello,
I have a list of words separated by spaces I am trying to delete from a text file, and I could not figure out what is the best way to do this.
what I tried (does not work) :
delete="password key number verify"
arr=($delete)
for i in arr
{
sed "s/\<${arr}\>]*//g" in.txt
}
>... (5 Replies)
Discussion started by: Hawk4520
5 Replies
IGAWK(1) Utility Commands IGAWK(1)
NAME
igawk - gawk with include files
SYNOPSIS
igawk [ all gawk options ] -f program-file [ -- ] file ...
igawk [ all gawk options ] [ -- ] program-text file ...
DESCRIPTION
Igawk is a simple shell script that adds the ability to have ``include files'' to gawk(1).
AWK programs for igawk are the same as for gawk, except that, in addition, you may have lines like
@include getopt.awk
in your program to include the file getopt.awk from either the current directory or one of the other directories in the search path.
OPTIONS
See gawk(1) for a full description of the AWK language and the options that gawk supports.
EXAMPLES
cat << EOF > test.awk
@include getopt.awk
BEGIN {
while (getopt(ARGC, ARGV, "am:q") != -1)
...
}
EOF
igawk -f test.awk
SEE ALSO
gawk(1)
Effective AWK Programming, Edition 1.0, published by the Free Software Foundation, 1995.
AUTHOR
Arnold Robbins (arnold@skeeve.com).
Free Software Foundation Nov 3 1999 IGAWK(1)