Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: File Containing Extra delimiter should be removed
Top Forums UNIX for Beginners Questions & Answers File Containing Extra delimiter should be removed Post 303009819 by bakunin on Wednesday 20th of December 2017 01:35:19 PM
Old 12-20-2017
Quote:
Originally Posted by ikdKunal
Now the 2nd row & last row has an extra delimiter, How to remove that ? In some large file having 100K data , there can be 100 such rows having extra pipe , how to remove it ?
Please, as we do not know your environment as you do, tell us about it:

your shell?
your OS?
the version of your OS?

Furthermore: i guess that your file can contain only ONE extra delimiter per line and the delimiter will contain no data, like:

Code:
a|b|c|d|e|||

Otherwise, you will have to explain what to do with such cases.

If this is so and you have a run-of-the-mill UNIX system you can try the following:

Code:
sed 's/^\(\([^|]*|\)\{3\}[^|]*\).*/\1/' /path/to/your/file

This will display the changed file only to screen. If you are satisfied with the outcome use:

Code:
sed 's/^\(\([^|]*|\)\{3\}[^|]*\).*/\1/' /path/to/your/file > /path/to/newfile

to save these results.

Explanation of the regexp:

[^|]*| matches a single cell, an arbitrary number of non-delimiters followed by a delimiter. This regexp is repeated three times:\([^|]*|\)\{3\}, then followed by an optional field content of non-delimiters: \([^|]*|\)\{3\}[^|]*.

All this is surrounded by brackets to use it as a back-reference. Any further content of the line is then included only to replace everything by the back-reference above so that effectively the rest of the line is deleted.

I hope this helps.

bakunin
Last edited by bakunin; 12-20-2017 at 02:49 PM.. Reason: corrected typo
 

10 More Discussions You Might Find Interesting

1. Solaris

after init all /tmp file has been removed

I'm new in Solaris server After the system support reboot the Solaris server, all the files in /tmp has been removed, is that normal under Solaris or under different init level will get different result? which init level will do that? (5 Replies)
Discussion started by: yesthomas
5 Replies

2. Shell Programming and Scripting

Please Help. Strings in file 1 need to be searched and removed from file 2

Please help. Here is my problem. I have 9000 lines in file a and 500,000 lines in file b. For each line in file a I need to search file b and remove that line. I am currently using the grep -v command and loading the output into a new file. However, because of the size of file b this takes an... (4 Replies)
Discussion started by: mjs3221
4 Replies

3. UNIX for Dummies Questions & Answers

how to removed chr(10) characters in a file

Hi, How do we remove an extra new line in a file. New line in ascii is called chr(10). Suppose we have a file as: 12345 98765 ------ ------ From the above i represented new line with dashed lines. Basically i have 2 new lines with white space at the end of the file. How do i removes... (1 Reply)
Discussion started by: sandeep_1105
1 Replies

4. Linux

file removed

Hi Team, I have deleted a file accidentally by using rm command. I am not the root(admin) user. Can you please let me know how to get that .tex file? (2 Replies)
Discussion started by: darling
2 Replies

5. Shell Programming and Scripting

Shell script to put delimiter for a no delimiter variable length text file

Hi, I have a No Delimiter variable length text file with following schema - Column Name Data length Firstname 5 Lastname 5 age 3 phoneno1 10 phoneno2 10 phoneno3 10 sample data - ... (16 Replies)
Discussion started by: Gaurav Martha
16 Replies

6. UNIX for Dummies Questions & Answers

Remove Extra Delimiter

Hi , I have file like this.. aaa|bbbb|cccc|dddd|fff|dsaaFFDFD| Adsads|sas|sa|as|asa|saddas|dsasd|sdad| dsas|dss|sss|sss|ddd|dssd|rrr|fddf| www|fff|refd|dads|fsdf|00sd| 5fgdg|dfs00|d55f|sfds55|445fsd|55ds|sdf| so I do no have any fix pattern and I want to remove extra... (11 Replies)
Discussion started by: pankajchaudhari
11 Replies

7. Shell Programming and Scripting

Perl Code to change file delimiter (passed as argument) to bar delimiter

Hi, Extremely new to Perl scripting, but need a quick fix without using TEXT::CSV I need to read in a file, pass any delimiter as an argument, and convert it to bar delimited on the output. In addition, enclose fields within double quotes in case of any embedded delimiters. Any help would... (2 Replies)
Discussion started by: JPB1977
2 Replies

8. Shell Programming and Scripting

Error removed from file

Below is a flowchart of a program. Most everything works as expected, but there are a couple of issues that I need some expert help on. The check function was setup initially for a single user input. The input has been modified to allow for multiple inputs, so the code below does not work. My... (15 Replies)
Discussion started by: cmccabe
15 Replies

9. UNIX for Beginners Questions & Answers

File Management: Removing of files from Server2 IF the same file is removed from Server1.

Hi Folks, I have a requirement of file management on different servers. Source Server is SERVER-A. Two servers will fetch files from SERVER-A: SERVER1 and SERVER2. 4th SERVER is SERVER-B, It will fetch files from SERVER1. If SERVER1 goes DOWN, SERVER-B will fetch pending files from... (2 Replies)
Discussion started by: Raza Ali
2 Replies

10. Shell Programming and Scripting

Recover the original file once removed

Hi All, Is there is any machanisim, once delete the file can we restore it. Thanks (8 Replies)
Discussion started by: bmk123
8 Replies

LEARN ABOUT DEBIAN

psc

PSC(1)							      General Commands Manual							    PSC(1)

NAME

       psc - prepare sc files

SYNOPSIS

       psc [-fLkrSPv] [-s cell] [-R n] [-C n] [-n n] [-d c]

DESCRIPTION

       Psc  is used to prepare data for input to the spreadsheet calculator sc(1).  It accepts normal ascii data on standard input.  Standard out-
       put is a sc file.  With no options, psc starts the spreadsheet in cell A0.  Strings are right justified.  All data on a line is entered	on
       the  same row; new input lines cause the output row number to increment by one.	The default delimiters are tab and space.  The column for-
       mats are set to one larger than the number of columns required to hold the largest value in the column.

OPTIONS

       -f     Omit column width calculations.  This option is for preparing data to be merged with an existing spreadsheet.  If the option is  not
	      specified, the column widths calculated for the data read by psc will override those already set in the existing spreadsheet.

       -L     Left justify strings.

       -k     Keep  all  delimiters.   This  option  causes the output cell to change on each new delimiter encountered in the input stream.   The
	      default action is to condense multiple delimiters to one, so that the cell only changes once per input data item.

       -r     Output the data by row first then column.  For input consisting of a single column, this option will result in  output  of  one  row
	      with multiple columns instead of a single column spreadsheet.

       -s cell
	      Start  the  top  left  corner  of the spreadsheet in cell.  For example, -s B33 will arrange the output data so that the spreadsheet
	      starts in column B, row 33.

       -R n   Increment by n on each new output row.

       -C n   Increment by n on each new output column.

       -n n   Output n rows before advancing to the next column.  This option is used when the input is  arranged  in  a  single  column  and  the
	      spreadsheet is to have multiple columns, each of which is to be length n.

       -d c   Use the single character c as the delimiter between input fields.

       -P     Plain numbers only.  A field is a number only when there is no imbedded [-+eE].

       -S     All numbers are strings.

       -v     Print the version of psc

SEE ALSO

       sc(1)

AUTHOR

       Robert Bond

PSC 7.16							 19 September 2002							    PSC(1)
All times are GMT -4. The time now is 04:53 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy