Sponsored Content
Top Forums Shell Programming and Scripting Extract columns into seperate file Post 303000208 by Ads89 on Thursday 6th of July 2017 09:30:58 AM
Old 07-06-2017
Extract columns into seperate file

I have a comma delimited file as per the one below and I am currently extracting the values in 2 columns (COL1 & COL6) to produce a smaller trimmed down version of the file which only contains the columns we need;

Code:
COL1,COL2,COL3,COL4,COL5,COL6,COL7,COL8,COL9
1111,AAAA,AAAA,AAAA,AAAA,X100,AAAA,AAAA,XXXX
2222,AAAA,AAAA,AAAA,AAAA,X200,AAAA,AAAA,
3333,AAAA,AAAA,AAAA,AAAA,X300,AAAA,AAAA,XXXX
4444,AAAA,AAAA,AAAA,AAAA,X400,AAAA,AAAA,XXXX
5555,AAAA,AAAA,AAAA,AAAA,X500,AAAA,AAAA,

I now have an additional requirement to only extract the values of COL1 & COL6 when COL9 has value present(could be anything) i.e. lines 1,3,4
The output produced would therefore look something like;

Code:
COL1,COL2
1111,X100
3333,X300
4444,X400

I have the below code which extracts only COL1 & COL2, but need to additional functionality


Code:
 
 awk -F, 'BEGIN {OFS=","} {gsub(/^[ \t]+/, "", $1); gsub(/[ \t]+$/, "", $1); gsub(/^[ \t]+/, "", $6); gsub(/[ \t]+$/, "", $6)} {if (NR>1) {print $1,$6}}' input.csv > output.csv

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

how to extract columns from a text file

Hi, In ksh, I have a file with similar rows as follows: Department = 1234 G/L Asset Acct No = 12.0000. 2/29/2008 Department = 1234 G/L Asset Acct No = 13.0000. 3/29/2008. I want to create a new text file that contains only the numbers and date: 1234 12.0000. 2/29/2008 1234 13.0000. ... (16 Replies)
Discussion started by: ihot
16 Replies

2. Shell Programming and Scripting

Help, need to extract columns from file

I have huge fixed width, text file in unix box and I need to extract columns found between the width 105 and 200 and output it to a new file. Can anyone tell me how to extract it? Thanks for your help. (1 Reply)
Discussion started by: kiran2k
1 Replies

3. UNIX for Dummies Questions & Answers

extract only the "numbers" that are present in this file to a seperate file..

It may sounds too easy but appreciate any help on this.. i have a file with the below details: ****************************************** Please find the locked pernr details for the Time evaluation Personnel number could not be locked Person rejected: 04552737 Personnel number could not... (4 Replies)
Discussion started by: rohit.shetty84
4 Replies

4. Shell Programming and Scripting

extract columns from 2 different files and create new file

Hi All, I have 2 issues while working with file. 1. I have 2 delimited(~) files. I want to extract column numbner 3 from file1 and column number 8 from file2 and paste it into file3. I have tried using cut, but not able to get answer. 2. i have 2 filxed-width file. I wanted to do same... (1 Reply)
Discussion started by: Amit.Sagpariya
1 Replies

5. Shell Programming and Scripting

Extract Columns from file

Hi All, Could you please help me with following: I have to parse a .csv file. For example: If the csv file contains 3 columns, then i have to print the column names. The field separator is (comma). example.csv (contains 2 lines as follows) This is,a test file, for validation... (2 Replies)
Discussion started by: vfrg
2 Replies

6. Shell Programming and Scripting

fileutility to extract columns from source file

Hi experts,Please help me for the below requirement.i have a source file.(lets say contains 50 columns). I am extarcting five columns from the source file by using pattern file.for exampleinput file:--------a,b,c,d,"a,g","v b",s,koutputfile=======a,"a,g","v b",s,kThanks in advancesubhendu (1 Reply)
Discussion started by: subhendu81
1 Replies

7. Shell Programming and Scripting

extract columns from file and send mail

Hi I have a file of the form name1,lastname1,email1@gmail.com,9.08243E+12,team1,role1,username1,password1 name2,lastname2,email2@gmail.com,9.08243E+11,team2,role2,username2,password2 I need to extract the email (column 3) and send a mail to each person, with their details ( specifically... (3 Replies)
Discussion started by: pkabali
3 Replies

8. UNIX for Dummies Questions & Answers

Seperate columns according to delimiters

Hi all I need your help to separate colomns based on "-" delimiter for a very big file 30 millions rows I have a colmun looking like this : clomun 1 1-100000989-A_ATC 1-10000179-AAAAA 1-100002154-TGTTA 1-100002155-GTTAG 1-100002443 1-100002490 1-100002619 I need to separte in three... (5 Replies)
Discussion started by: biopsy
5 Replies

9. Shell Programming and Scripting

Command to extract all columns except the last few from a txt file

hello, i have publicly available txt file with little less than 300000 rows. i want to extract from column 1 to column 218 and save it in another text file. i use the cut command but the file is saved with multiple rows from the source file onto a single row in the destination. basically it is... (6 Replies)
Discussion started by: madrazzii
6 Replies

10. Shell Programming and Scripting

Match Columns in one file and extract columns from another file

Kindly help merging information from two files with the following data structure. I want to match for the CHR-SNP in Foo and get the columns that match from CHROM-rsID Fields 1 & 2 of foo may have duplicates, however, a joint key of Fields $1$2$3$4 is unique. Also would be helpful to clean up... (4 Replies)
Discussion started by: genehunter
4 Replies
Smokeping_probes_EchoPingDNS(3) 				     SmokePing					   Smokeping_probes_EchoPingDNS(3)

NAME
Smokeping::probes::EchoPingDNS - an echoping(1) probe for SmokePing OVERVIEW
Measures DNS roundtrip times for SmokePing with the echoping_dns plugin. SYNOPSIS
*** Probes *** +EchoPingDNS binary = /usr/bin/echoping forks = 5 offset = 50% step = 300 # The following variables can be overridden in each target section dns_request = example.org # mandatory dns_tcp = yes dns_type = AAAA extraopts = -some-letter-the-author-did-not-think-of ipversion = 4 pings = 5 plugin = /path/to/dns.so pluginargs = -p plugin_specific_arg priority = 6 timeout = 1 tos = 0xa0 waittime = 1 # [...] *** Targets *** probe = EchoPingDNS # if this should be the default probe # [...] + mytarget # probe = EchoPingDNS # if the default probe is something else host = my.host dns_request = example.org # mandatory dns_tcp = yes dns_type = AAAA extraopts = -some-letter-the-author-did-not-think-of ipversion = 4 pings = 5 plugin = /path/to/dns.so pluginargs = -p plugin_specific_arg priority = 6 timeout = 1 tos = 0xa0 waittime = 1 VARIABLES
Supported probe-specific variables: binary The location of your echoping binary. Default value: /usr/bin/echoping forks Run this many concurrent processes at maximum Example value: 5 Default value: 5 offset If you run many probes concurrently you may want to prevent them from hitting your network all at the same time. Using the probe- specific offset parameter you can change the point in time when each probe will be run. Offset is specified in % of total interval, or alternatively as 'random', and the offset from the 'General' section is used if nothing is specified here. Note that this does NOT influence the rrds itself, it is just a matter of when data acqusition is initiated. (This variable is only applicable if the variable 'concurrentprobes' is set in the 'General' section.) Example value: 50% step Duration of the base interval that this probe should use, if different from the one specified in the 'Database' section. Note that the step in the RRD files is fixed when they are originally generated, and if you change the step parameter afterwards, you'll have to delete the old RRD files or somehow convert them. (This variable is only applicable if the variable 'concurrentprobes' is set in the 'General' section.) Example value: 300 Supported target-specific variables: dns_request The DNS request (domain name) to be queried. Example value: example.org This setting is mandatory. dns_tcp The echoping_dns '--tcp' option: use only TCP ('virtual circuit'). Enabled if specified with a value other than 'no' or '0'. Example value: yes dns_type The echoping_dns '-t' option: type of data requested (NS, A, SOA etc.) Example value: AAAA Default value: A extraopts Any extra options specified here will be passed unmodified to echoping(1). Example value: -some-letter-the-author-did-not-think-of ipversion The IP protocol used. Possible values are "4" and "6". Passed to echoping(1) as the "-4" or "-6" options. Example value: 4 pings How many pings should be sent to each target, if different from the global value specified in the Database section. Note that the number of pings in the RRD files is fixed when they are originally generated, and if you change this parameter afterwards, you'll have to delete the old RRD files or somehow convert them. Example value: 5 plugin The echoping plugin that will be used. See echoping(1) for details. This can either be the name of the plugin or a full path to the plugin shared object. Example value: /path/to/dns.so Default value: dns pluginargs Any extra arguments needed by the echoping plugin specified with the pluginname variable. These are generally provided by the subclass probe. Example value: -p plugin_specific_arg priority The "-p" echoping(1) option. Example value: 6 timeout The "-t" echoping(1) option. Example value: 1 Default value: 5 tos The "-P" echoping(1) option. Example value: 0xa0 waittime The "-w" echoping(1) option. Example value: 1 AUTHORS
Niko Tyni <ntyni@iki.fi> NOTES
The fill, size and udp EchoPing variables are not valid. Plugins, including echoping_dns, are available starting with echoping version 6. SEE ALSO
Smokeping::probes::EchoPing, Smokeping::probes::EchoPingPlugin 2.6.8 2013-03-17 Smokeping_probes_EchoPingDNS(3)
All times are GMT -4. The time now is 05:55 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy