Remove duplicates based on a column in fixed width file Post: 302437447

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Changing one column of delimited file column to fixed width column

Hi, Iam new to unix. I have one input file . Input file : ID1~Name1~Place1 ID2~Name2~Place2 ID3~Name3~Place3 I need output such that only first column should change to fixed width column of 15 characters of length. Output File: ID1<<12 spaces>>Name1~Place1 ID2<<12...

2. Shell Programming and Scripting

How to split a fixed width text file into several ones based on a column value?

Hi, I have a fixed width text file without any header row. One of the columns contains a date in YYYYMMDD format. If the original file contains 3 dates, I want my shell script to split the file into 3 small files with data for each date. I am a newbie and need help doing this.

3. Shell Programming and Scripting

need to remove duplicates based on key in first column and pattern in last column

Given a file such as this I need to remove the duplicates. 00060011 PAUL BOWSTEIN ad_waq3_921_20100826_010517.txt 00060011 PAUL BOWSTEIN ad_waq3_921_20100827_010528.txt 0624-01 RUT CORPORATION ad_sade3_10_20100827_010528.txt 0624-01 RUT CORPORATION ...

4. Shell Programming and Scripting

remove duplicates based on single column

Hello, I am new to shell scripting. I have a huge file with multiple columns for example: I have 5 columns below. HWUSI-EAS000_29:1:105 + chr5 76654650 AATTGGAA HHHHG HWUSI-EAS000_29:1:106 + chr5 76654650 AATTGGAA B@HYL HWUSI-EAS000_29:1:108 + ...

5. Shell Programming and Scripting

Removing duplicates in fixed width file which has multiple key columns

Hi All , I have a requirement where I need to remove duplicates from a fixed width file which has multiple key columns .Also , need to capture the duplicate records into another file . File has 8 columns. Key columns are col1 and col2. Col1 has the length of 8 col 2 has the length of 3. ...

6. Shell Programming and Scripting

Fixed width file search based on position value

Hi, I am unable to find the right option to extract the data in the fixed width file. sample data abcd1234xgyhsyshijfkfk hujk9876 io xgla loki8787eljuwoejroiweo dkfj9098 dja Search based on position 8-9="xg" and print the entire row output ...

7. Shell Programming and Scripting

To replace the value of the column in a fixed width file

I have a fixed with file with header & trailer length having the same length of the detail record file. The details record length of this file is 24, for Header and Trailer the records will be padded with spaces to match the record length of the file Currently I am adding 3 spaces in header...

8. Shell Programming and Scripting

UNIX command -Filter rows in fixed width file based on column values

Hi All, I am trying to select the rows in a fixed width file based on values in the columns. I want to select only the rows if column position 3-4 has the value AB I am using cut command to get the column values. Is it possible to check if cut -c3-4 = AB is true then select only that...

9. Shell Programming and Scripting

Search and replace value based on certain conditions in a fixed width file

Hi Forum. I tried searching for a solution using the internet search but I haven't been able to find any solution for what I'm trying to accomplish. I have a fixed width column file where I need to search for any occurrences of "D0" in col pos.#1-2, 10-11, 20-21 and replaced it with "XD". ...

LEARN ABOUT REDHAT

mac2unix

dos2unix(1)						      General Commands Manual						       dos2unix(1)

NAME

       dos2unix - DOS/MAC to UNIX text file format converter

SYNOPSYS

       dos2unix [options] [-c convmode] [-o file ...] [-n infile outfile ...]

       Options:

       [-hkqV] [--help] [--keepdate] [--quiet] [--version]

DESCRIPTION

       This manual page documents dos2unix, the program that converts plain text files in DOS/MAC format to UNIX format.

OPTIONS

       The following options are available:

       -h --help
	      Print online help.

       -k --keepdate
	      Keep the date stamp of output file same as input file.

       -q --quiet
	      Quiet mode. Suppress all warning and messages.

       -V --version
	      Prints version information.

       -c --convmode convmode
	      Sets conversion mode. Simulates dos2unix under SunOS.

       -o --oldfile file ...
	      Old file mode. Convert the file and write output to it. The program default to run in this mode. Wildcard names may be used.

       -n --newfile infile outfile ...
	      New  file  mode.	Convert the infile and write output to outfile. File names must be given in pairs and wildcard names should NOT be
	      used or you WILL lost your files.

EXAMPLES

       Get input from stdin and write output to stdout.

	      dos2unix

       Convert and replace a.txt. Convert and replace b.txt.

	      dos2unix a.txt b.txt

	      dos2unix -o a.txt b.txt

       Convert and replace a.txt in ASCII conversion mode.  Convert and replace b.txt in ISO conversion mode.  Convert	c.txt  from  Mac  to  Unix
       ascii format.

	      dos2unix a.txt -c iso b.txt

	      dos2unix -c ascii a.txt -c iso b.txt

	      dos2unix -c mac a.txt b.txt

       Convert and replace a.txt while keeping original date stamp.

	      dos2unix -k a.txt

	      dos2unix -k -o a.txt

       Convert a.txt and write to e.txt.

	      dos2unix -n a.txt e.txt

       Convert a.txt and write to e.txt, keep date stamp of e.txt same as a.txt.

	      dos2unix -k -n a.txt e.txt

       Convert and replace a.txt. Convert b.txt and write to e.txt.

	      dos2unix a.txt -n b.txt e.txt

	      dos2unix -o a.txt -n b.txt e.txt

       Convert c.txt and write to e.txt. Convert and replace a.txt.  Convert and replace b.txt. Convert d.txt and write to f.txt.

	      dos2unix -n c.txt e.txt -o a.txt b.txt -n d.txt f.txt

DIAGNOSTICS

BUGS

       The program does not work properly under MSDOS in stdio processing mode.  If you know why is that so, please tell me.

AUTHORS

       Benjamin Lin - <blin@socs.uts.edu.au>

       Bernd Johannes Wuebben (mac2unix mode) <wuebben@kde.org>

MISCELLANY

       Tested environment:

	      Linux 1.2.0 with GNU C 2.5.8

	      SunOS 4.1.3 with GNU C 2.6.3

	      MS-DOS 6.20 with Borland C++ 4.02

       Suggestions and bug reports are welcome.

SEE ALSO

       unix2dos(1) mac2unix(1)

1995.03.31							   dos2unix v3.0						       dos2unix(1)