Sponsored Content
Top Forums Shell Programming and Scripting Count the delimeter from a file and delete the row if delimeter count doesnt match. Post 302449801 by Akumar1 on Tuesday 31st of August 2010 03:04:58 PM
Old 08-31-2010
Count the delimeter from a file and delete the row if delimeter count doesnt match.

I have a file containing about 5 million rows, in the file there are some records which has extra delimiter at random position. (we dont know the positions), now we have to Count the delimeter from each row and if the count of delimeter is not matching then I want to delete those rows from the file.

What is the most efficient way of doing it. rather how can I do it.

the actual file file has 105 column whith delimeter as open bracket.


thanks for your help in advance.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to read and parse the content of csv file containing # as delimeter into fields using Bash?

#!/bin/bash i=0 cat 1.csv | while read fileline do echo "$fileline" IFS="#" flds=( $fileline ) nrofflds=${#flds} echo "noof fields$nrofflds" fld=0 while do echo "noof counter$fld" echo "$nrofflds" #fld1="${flds}" trying to store the content of line to fields but i... (4 Replies)
Discussion started by: barani75
4 Replies

2. Shell Programming and Scripting

how to display column value in a row with space as delimeter

Hi I need to write a small script to kill the process id of particular job in one shot , Example > ps PID TTY TIME COMMAND 16280 pts/70 0:00 sh 16278 pts/70 0:00 rlogind 16197 pts/70 0:00 ps 1234 pts/70 0:00 runflow 2341 pts/70 0:00 runflow 12673 pts/70 ... (6 Replies)
Discussion started by: mani_isha
6 Replies

3. Shell Programming and Scripting

count identical strings print last row and count

I have a sorted file like: Apple 3 Apple 5 Apple 8 Banana 2 Banana 3 Grape 31 Orange 7 Orange 13 I'd like to search $1 and if $1 is not the same as $1 in the previous row print that row and print the number of times $1 was found. so the output would look like: Apple 8 3 Banana... (2 Replies)
Discussion started by: dcfargo
2 Replies

4. Shell Programming and Scripting

append delimeter count for each line in text file

Hi guys, plz tell me how to achieve this how to delete the lines in a file using sed command (6 Replies)
Discussion started by: hari908
6 Replies

5. Shell Programming and Scripting

Needed value after the last delimeter in a file with varying number of delimited columns

Hi All, My file has the records as below: aaa\bbb\c\dd\ee\ff\gg zz\vv\ww pp\oo\ii\uu How can I get the value after the last delimeter. My o/p: gg ww uu Thanks in Advance, (5 Replies)
Discussion started by: HemaV
5 Replies

6. Shell Programming and Scripting

Replacing the delimeter with other delimeter

Hi Friends, I have a file1.txt as below 29123973Ç2012-0529Ç35310124Ç000000000004762Ç00010Ç20Ç390ÇÇÇÇF 29123974Ç20120529Ç35310125Ç0000000000046770Ç00010Ç20Ç390ÇÇÇÇF 29123975Ç20120529Ç35310126Ç0000000000046804Ç00010Ç20Ç390ÇÇÇÇF 29123976Ç20120529Ç35310127Ç0000000000044820Ç00010Ç20Ç390ÇÇÇÇF i have a file2.txt... (4 Replies)
Discussion started by: i150371485
4 Replies

7. Shell Programming and Scripting

Split line of file from delimeter.

I have a below file. INPUT FILE select * from customer MERGE INTO Archive; delete from Employee; using select * from customer; delete from employee; select * from Employee; insert into employee(1,1); OUTPUT FILE select * from customer MERGE INTO Archive delete from Employee using... (5 Replies)
Discussion started by: Mohin Jain
5 Replies

8. UNIX for Beginners Questions & Answers

Count the number of files to delete doesnt match

Good evening, need your help please Need to delete certain files before octobre 1 2016, so need to know how many files im going to delete, for instance ls -lrt file_20160*.lis!wc -l but using grep -c to another file called bplist which contains the list of all files backed up doesn match... (7 Replies)
Discussion started by: alexcol
7 Replies

9. Shell Programming and Scripting

How to change comma delimeter in the file to number?

I have a file H,20180624200732,VPAS,TRANS_HDR,20180724, VPAS.TRANS_HDR.20180724.01.txt, ,93, T,1, I have to change and instead first comma put ",1" like below H,20180624200732,VPAS,TRANS_HDR,20180724, VPAS.TRANS_HDR.20180724.01.txt,1,93, T,1, I made sed "2s/, /,1/"... (8 Replies)
Discussion started by: digioleg54
8 Replies

10. Shell Programming and Scripting

Large File masking incorrectly happening Ç delimeter issue

The OS version is Red Hat Enterprise Linux Server release 6.10 I have a script to mask some columns with **** in a data file which is delimeted with Ç , I am using awk for the masking , when I try to mask a small file the awk works fine and masks the required column , but when the file is... (6 Replies)
Discussion started by: LinuxUser8092
6 Replies
PG_COPY_FROM(3) 														   PG_COPY_FROM(3)

pg_copy_from - Insert records into a table from an array

SYNOPSIS
bool pg_copy_from (resource $connection, string $table_name, array $rows, [string $delimiter], [string $null_as]) DESCRIPTION
pg_copy_from(3) inserts records into a table from $rows. It issues a COPY FROM SQL command internally to insert records. PARAMETERS
o $connection - PostgreSQL database connection resource. o $table_name - Name of the table into which to copy the $rows. o $rows - An array of data to be copied into $table_name. Each value in $rows becomes a row in $table_name. Each value in $rows should be a delimited string of the values to insert into each field. Values should be linefeed terminated. o $delimiter - The token that separates values for each field in each element of $rows. Default is TAB. o $null_as - How SQL NULL values are represented in the $rows. Default is N ("\N"). RETURN VALUES
Returns TRUE on success or FALSE on failure. EXAMPLES
Example #1 pg_copy_from(3) example <?php $db = pg_connect("dbname=publisher") or die("Could not connect"); $rows = pg_copy_to($db, $table_name); pg_query($db, "DELETE FROM $table_name"); pg_copy_from($db, $table_name, $rows); ?> SEE ALSO
pg_copy_to(3). PHP Documentation Group PG_COPY_FROM(3)
All times are GMT -4. The time now is 03:44 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy