Sponsored Content
Top Forums Shell Programming and Scripting Egrep or awk for removing values within CSV file? Post 302936478 by macurdy on Wednesday 25th of February 2015 09:56:23 PM
Old 02-25-2015
Egrep or awk for removing values within CSV file?

Hello,

I have a large CSV file that contains values all on the same column, and in one very long row (e.g. no line breaks till end, with all data values separated by a comma).

The file has two types of data for the values. One begins with the letters rs and some numbers. The other begins with the letter i and some numbers. An example is below (id's are genome identifiers).

Code:
rs28931576,rs11542040,rs28931577,rs429358,i6007484,i6007510,rs28931578,i6007500,i6007489,i5000217,i6007504,i6007493,rs769455,i6007507,i6007497,i6007512,i6007495,i6007485,i6007492,i5000216,i5000205,rs7412

My Unix command line knowledge was enough to use the cat and cut commands to get the above data to this point.

I can't seem to figure out how to remove all of the values that begin with the letter i. I've tried some awk and egrep commands, but don't have the mastery yet to get this figured out.

I also need a way to get rid of duplicate commas after the i values are removed.

Right now, I'm using Find-Replace with TextEdit on mac to do these steps, however I'd love to be able to script this.

Any help is much appreciated!

Last edited by Don Cragun; 02-26-2015 at 01:29 AM.. Reason: Add CODE tags.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need to compare two csv files values and write into another csv file

Hi all, Am new to scripting. So i just need your ideas to help me out. Here goes my requirement. I have two csv files 1.csv 2.csv abc,1.24 abc,1 def,2.13 def,1 I need to compare the first column of 1.csv with 2.csv and if matches then need to compare... (2 Replies)
Discussion started by: chinnahyd
2 Replies

2. UNIX for Dummies Questions & Answers

Help removing strings from one file that match any of the values in a second file.

Hello, I have a file that lists a few hundred values. Example: abca abcb abcc abcd I have a 2nd file with a few thousand lines. I need to remove every line from the 2nd file that contains any of the values listed in first file. Example of strings to delete: line1 *abca* end of... (1 Reply)
Discussion started by: upstate_boy
1 Replies

3. Shell Programming and Scripting

How to grep/awk/egrep two values for given output?

Dear Friends, I have a command which can result following output. Packet is: /var/adm/yyyy/pkt6043 Intended for network : /vob/repo I would like to retrive pkt6043 and /vob/repo using single command. Blue color test will be always contstant and red color text will be dynamic ... (2 Replies)
Discussion started by: baluchen
2 Replies

4. Linux

Removing non printing characters from a csv file

Hi, I have an csv file and there are some non printable characters(extended ascii) so I am trying to create a clean copy of the csv file . I am using this command: tr -cd "" < /opt/informatica/PowerCenter8.6.0/server/infa_shared/SrcFiles/ThirdParty/locations.csv > ... (4 Replies)
Discussion started by: gerkus
4 Replies

5. Shell Programming and Scripting

Removing lines of a .csv file

Hello, Does anyone have a one-liner to remove lines of a csv file if the value in a specific column is zero? For example, I have this file, 12345,COM,5,0,N,29.95,Y 12345,MOM,1,0,N,29.95,Y 12345,COM,4,0,N,9.99,Y 12345,MOM,0,2,N,9.99,Y 12345,REN,0,1,N,9.99,Y and I want to remove lines... (4 Replies)
Discussion started by: palex
4 Replies

6. Shell Programming and Scripting

Removing zero values from text file

Hi all, I wrote the following code to remove the value which are 0 in the input file (a columns if numbers). awk 'BEGIN { for (i=1; i<=NF; i++) if ($i) printf("%13.6e\n",$i) }' $1 >> $2 The script works if the zeros are written as 0.0000 but not as 0.000000e+00 In... (10 Replies)
Discussion started by: f_o_555
10 Replies

7. Shell Programming and Scripting

removing lines with similar values from file

Hello, got a file with this structure: 33274 171030 02/29/2012 37897 P_GEH 2012-02-29 10:31:26 33275 171049 02/29/2012 38132 P_GEH 2012-02-29 10:35:27 33276 171058 02/29/2012 38515 P_GEH 2012-02-29 10:43:26 33277 170748 02/29/2012 40685 P_KOM ... (3 Replies)
Discussion started by: krecik28
3 Replies

8. Shell Programming and Scripting

Removing \n from .csv file

Hi, I have a requirement like my .csv file is generating from a db2 table using export command like below: file format: ----------- 2011 4 0 0 N S C C "BHPC BHPC" 0 0 0 2011 5 0 0 N S C C "BHPC BHPC" 0 0 0 here BHPC is having new line character and because this when i am trying... (4 Replies)
Discussion started by: RRVARMA
4 Replies

9. Shell Programming and Scripting

Bash script help - removing certain rows from .csv file

Hello Everyone, I am trying to find a way to take a .csv file with 7 columns and a ton of rows (over 600,000) and remove the entire row if the cell in forth column is blank. Just to give you a little background on why I am doing this (just in case there is an easier way), I am pulling... (3 Replies)
Discussion started by: MrTuxor
3 Replies

10. UNIX for Beginners Questions & Answers

Removing commas from CSV file

Hi I'm creating a sh script to generate a csv file. The CSV contains the values from a sql table. The content looks this: a,b,c,c2,c3,,,,,,,,,,,d,e I have some code that can separate the fields using the comma as delimiter, but some values actually contain commas, such as... (2 Replies)
Discussion started by: preema
2 Replies
rc2(8)							      System Manager's Manual							    rc2(8)

NAME
rc2 - Run command script executed when entering a multiuser run level SYNOPSIS
rc2 DESCRIPTION
The rc2 script contains run commands that enable initialization of the system to a multiuser state; run level 2. In addition to commands listed within the script itself, rc2 contains instructions to run certain commands found in the /sbin/rc2.d directory. The script defines the conditions under which the commands execute; some commands run if the system is booting, other commands execute if the system is chang- ing run levels. By convention, files in the /sbin/rc2.d directory begin with either the letter "K" or the letter "S" and are followed by a two-digit number and a filename; for example: K00lpd S00savecore S25uucp In general, the system starts commands that begin with the letter "S" and stops commands that begin with the letter "K." Commands that begin with the letter "K" run only when the system is changing run levels from a higher to a lower level. Commands that begin with the letter "S" run in all cases. The numbering of commands in the /sbin/rc2.d directory is important since the numbers are sorted and the com- mands are run in ascending order. Files in the /sbin/rc2.d directory are normally links to files in the /etc/init.d directory. An entry in the inittab file causes the system to execute the rc2 run commands, for example: s2:23:wait:/sbin/rc2 < /dev/console > /dev/console 2>&1 The following operations are typical of those that result from executing the rc2 script and the commands located in the /sbin/rc2.d direc- tory. The operation depends on which state the system is entering or exiting. Setting the time zone Checking the current run level Stop- ping network services and daemons Starting (or stopping) system services and daemons Starting the cron daemon Setting up paging and dump facilities Setting up uucp files Setting the TIMEZONE variable is one of the first operations completed by the rc2 script. This action provides the default time zone for subsequent commands. FILES
Specifies the command path Specifies the directory of commands that correspond to the run level RELATED INFORMATION
Commands: init(8), rc0(8), rc3(8) delim off rc2(8)
All times are GMT -4. The time now is 02:39 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy