Sponsored Content
Top Forums Shell Programming and Scripting Filtering first file columns based on second file column Post 302715741 by ks_reddy on Monday 15th of October 2012 10:15:11 AM
Old 10-15-2012
Filtering first file columns based on second file column

Hi friends,

I have one file like below. (.csv type)
Code:
SNo,data1,data2
1,1,2
2,2,3
3,3,2

and another file like below.
Code:
Exclude
data1

where Exclude should be treated as column name in file2.
I want the output shown below.
Code:
SNo,data2
1,2
2,3
3,2

Where my data1 column got removed from my first file as it is mentioned in next file.
In reality i have 1000's of columns in my file 1 and want to remove some of them by keep updating in second file.
I got a one liner R code, but to load the first file and writing the result into another file is taking lot of time as well as more memory commit.

Regards
Sidda
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Filtering records of a file based on a value of a column

Hi all, I would like to extract records of a file based on a condition. The file contains 47 fields, and I would like to extract only those records that match a certain value in one of the columns, e.g. COL1 COL2 COL3 ............... COL47 1 XX 45 ... (4 Replies)
Discussion started by: risk_sly
4 Replies

2. Shell Programming and Scripting

filtering one file based on results from other

Can anybody help me with writing a script for the data that I want to use from one file based on the data from another file. I have file1 in this form; (the first field represents a well name and the second field represents the depth of interest) FILE1 -------- DATA_35_0 ... (2 Replies)
Discussion started by: digipak
2 Replies

3. Shell Programming and Scripting

filtering one file based on results from other- AGAIN

I have asked this question here before and got the answer too. Unfortunately I used only one record as an example and the script works fine for one record but not for more than one record. Can anybody help me with writing a script for the data that I want to use from one file based on the... (13 Replies)
Discussion started by: digipak
13 Replies

4. Shell Programming and Scripting

Filtering issues with multiple columns in a single file

Hi, I am new to unix and would greatly appreciate some help. I have a file containing multiple colums containing different sets of data e.g. File 1: John Ireland 27_December_69 Mary England 13_March_55 Mike France 02_June_80 I am currently using the awk... (10 Replies)
Discussion started by: crunchie
10 Replies

5. UNIX for Dummies Questions & Answers

Filtering records from 1 file based on some manipulation doen on second file

Hi, I am looking for an awk script which should help me to meet the following requirement: File1 has records in following format INF: FAILEd RECORD AB1234 INF: FAILEd RECORD PQ1145 INF: FAILEd RECORD AB3215 INF: FAILEd RECORD AB6114 ............................ (2 Replies)
Discussion started by: mintu41
2 Replies

6. Shell Programming and Scripting

Filtering lines for column elements based on corresponding counts in another column

Hi, I have a file like this ACC 2 2 21 aaa AC 443 3 22 aaa GCT 76 1 33 xxx TCG 34 2 33 aaa ACGT 33 1 22 ggg TTC 99 3 44 wee CCA 33 2 33 ggg AAC 1 3 55 ddd TTG 10 1 22 ddd TTGC 98 3 22 ddd GCT 23 1 21 sds GTC 23 4 32 sds ACGT 32 2 33 vvv CGT 11 2 33 eee CCC 87 2 44... (1 Reply)
Discussion started by: polsum
1 Replies

7. Linux

To get all the columns in a CSV file based on unique values of particular column

cat sample.csv ID,Name,no 1,AAA,1 2,BBB,1 3,AAA,1 4,BBB,1 cut -d',' -f2 sample.csv | sort | uniq this gives only the 2nd column values Name AAA BBB How to I get all the columns of CSV along with this? (1 Reply)
Discussion started by: sanvel
1 Replies

8. Shell Programming and Scripting

Replacing 12 columns of one file by second file based on mapping in third file

i have a real data prod file with 80+ fields containing 1k -2k records. i have to extract say 12 columns out of this which are sensitive fields along with one primary key say SEQ_ID (like DOB,account no, name, SEQ_ID, govtid etc) in a lookup file. i have to replace these sensitive fields in... (11 Replies)
Discussion started by: megh12
11 Replies

9. UNIX for Beginners Questions & Answers

Filtering based on column values

Hi there, I am trying to filter a big file with several columns using values on a column with values like (AC=5;AN=10;SF=341,377,517,643,662;VRT=1). I wont to filter the data based on SF= values that are (bigger than 400) ... (25 Replies)
Discussion started by: daashti
25 Replies

10. UNIX for Beginners Questions & Answers

Filtering records of a csv file based on a value of a column

Hi, I tried filtering the records in a csv file using "awk" command listed below. awk -F"~" '$4 ~ /Active/{print }' inputfile > outputfile The output always has all the entries. The same command worked for different users from one of the forum links. content of file I was... (3 Replies)
Discussion started by: sunilmudikonda
3 Replies
kst(1)								       kst2								    kst(1)

NAME
kst -- plots scientific data SYNOPSIS
kst kst [ -F DATASOURCE ] [ -f STARTFRAME ] [ -n NUMFRAMES ] [ -s FRAMESPERSAMPLE [ -a ] ] kstfile kst datasource [ -f STARTFRAME ] [ -n NUMFRAMES ] [ -s FRAMESPERSAMPLE [ -a ] ] [ -P PLOTNAME ] [ -A] [ -d ] [ -l ] [ -b ] [ -x FIELD ] [ -e FIELD ] [ -r RATE ] [ -y FIELD ] [ -p FIELD ] [ -h FIELD ] [ -z FIELD ] [ --png filename ] [ --print filename [ --landscape | --portrait ] [ --Letter | --A4 ] ] DESCRIPTION
kst is a graphical data plotting tool, which includes the ability to plot live data (contiuously update the plot as new data becomes avail- able). kst may be run entirely from within its GUI, in which case no command line arguments are needed. kst has a powerful wizard for easily set- ting up new plots. In the second invocation, kst loads pre-saved plot specifications from the file kstfile and optionally may override some of the settings in that file. In the third invocation, kst displays plots of the data as directed in the default manner. Additional datasources may be specified on the command line with options for that datasource following it. Command line options generally apply to all subsequent options unless later overridden. OPTIONS
-A plot all curves in separate plots -a average frames (simple mean) when plotting with -s. --A4 use A4 sized paper for printing. Requires --print. -b plot as bargraphs. -d use points when plotting curves. -e FIELD use FIELD as the magnitude of error bars applied to the next plot. -F datasource override the datasource indicated in the kstfile and instead use datasource. -f STARTFRAME begin plotting at STARTFRAME frames into the data. Set to -1 to start at NUMFRAMES from the end of the data. -h FIELD plot FIELD as a histogram. -l use lines when plotting curves. --landscape use landscape orientation for printing. Requires --print. --Letter use US Letter sized paper for printing. Requires --print. -n NUMFRAMES plot at most NUMFRAMES frames of data. Set to -1 to indicate all of the data. -P PLOTNAME plot all plots in the plot named PLOTNAME. -p FIELD plot FIELD as a power spectrum plot. --png filename render to a portable network graphics (png) image with file name filename and quit. --portrait use portrait orientation for printing. Requires --print. --print filename print the plot to file filename and quit. -s FRAMESPERSAMPLE plot every FRAMESPERSAMPLE frames when plotting. -x FIELD use FIELD as the X-axis vector. -y FIELD plot FIELD as an XY plot. -z FIELD plot FIELD as an image. EXAMPLES
Plot all data in column 2 from data.dat. kst data.dat -y 2 Same as above, except only read 20 lines, starting at line 10. kst data.dat -f 10 -n 20 -y 2 also read col 1. One plot per curve. kst data.dat -f 10 -n 20 -y 1 -y 2 Read col 1 from data2.dat and col 1 from data.dat kst data.dat -f 10 -n 20 -y 2 data2.dat -y 1 Same as above, except read 40 lines starting at 30 in data2.dat kst data.dat -f 10 -n 20 -y 2 data2.dat -f 30 -n 40 -y 1 Specify the X vector and error bars: Plot x = col 1 and Y = col 2 and error flags = col 3 from data.dat kst data.dat -x 1 -e 3 -y 2 Get the X vector from data1.dat, and the Y vector from data2.dat. kst data1.dat -x 1 data2.dat -y 1 Placement: Plot column 2 and column 3 in plot P1 and column 4 in plot P2 kst data.dat -P P1 -y 2 -y 3 -P P2 -y 4 BUGS
Please report bugs to either the kst mailing list at kst@kde.org or to the kde bugzilla under the kst product at http://bugs.kde.org/. If you can run kst, you can easily report bugs by selecting Bug Report Wizard from the Help menu. AUTHOR
Matthew Truch <matt@truch.net> LICENSE
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is included in the `COPYING.DOC' file as part of the kst distribution. Version 2.0.3 22 February 2011 kst(1)
All times are GMT -4. The time now is 11:30 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy