Sponsored Content
Top Forums Shell Programming and Scripting delete duplicated characters in each line Post 302382025 by daptal on Monday 21st of December 2009 10:15:29 PM
Old 12-21-2009
Code:
cat abc.txt |  perl -e '
while(<>){ 
         chomp;
         my %hash;
         map { print "$_/" } grep(!$hash{$_}++, split(//));
         print "\n";
}'

HTH,
PL
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

grep and delete 2nd duplicated of txt... -part2

Hi, I find out one problem is...the main point is we must delete 2nd duplicated of word in txt file. For example apple orange pink green orange yellow orange red output should be: apple orange pink green yellow orange (16 Replies)
Discussion started by: happyv
16 Replies

2. Shell Programming and Scripting

Delete new line characters from a file

Hi, I have a file with about 25 colums separated with '~', but few of the lines have extra tabs ('^') and new line characters ('$'). Is there a way I can delete those characters if they are anywhere before the 25th column in a line? example: CLUB000650;12345678;0087788667;NOOP MEMBER ... (4 Replies)
Discussion started by: rudoraj
4 Replies

3. Shell Programming and Scripting

Delete characters from each line

Hi, I have a file that has data in the following manner, tt_0.00001.dat 123.000 tt_0.00002.dat 124.000 tt_0.00002.dat 125.000 This is consistent for all the entries in the file. I want to delete the 'tt_' and '.dat' from each line. Could anyone please guide me how to do this using awk or... (2 Replies)
Discussion started by: lost.identity
2 Replies

4. Shell Programming and Scripting

delete first 2 characters for each line, please help

hi, ./R1_970330_210505.sard ./R1_970403_223412.sard ./R1_970626_115235.sard ./R1_970626_214344.sard ./R1_970716_234214.sard ... ... ... for these strings, i wanna remove the ./ for each line how can i do that? i know it could possibly be done by sed, but i really have not idea how... (4 Replies)
Discussion started by: sunnydanniel
4 Replies

5. Shell Programming and Scripting

Delete characters from each line until meet character ":"

Hello, I have file that looks like this : 765327564:line1 94:line2 7865:line3 ..... 765322:linen I want to cut all the digits from the beginning of each line up to ":" character and to have everything like this : line1 line2 line3 ..... linen P.S : content of line1 ...... (8 Replies)
Discussion started by: black_fender
8 Replies

6. Shell Programming and Scripting

How delete characters of specific line with sed?

Hi, I have a text file with some lines like this: /MEDIA/DISK1/23568742.MOV /MEDIA/DISK1/87456321.AVI /MEDIA/DISK2/PART1/45753131.AVI /IMPORT/44452.WAV ... I want to remove the last 12 characters in each line that it ends "AVI". Should look like this: /MEDIA/DISK1/23568742.MOV... (12 Replies)
Discussion started by: inaki
12 Replies

7. UNIX for Dummies Questions & Answers

How to delete text between two characters in line?

Hi, I have a large text file with the following format: >gi|347545744|gb|JN204951.1| Dismorphia spio voucher 5 ATCAAATTCCTTCCTCTCCTTAAA >gi|17544664774|gb|WN204922.32| Rodapara nigens gene region CCGGGCAAATTCCTTCCTCTCCTTAAA >gi|555466400|gb|SG255122.8| Bombyx mandariana genbank 3... (1 Reply)
Discussion started by: euspilapteryx
1 Replies

8. Shell Programming and Scripting

Delete duplicated fields in a line

Hi, I have files with this kind of format (separator is space): A1 B1 C1 D1 E1 F1 D1 C1 G1 H1 A2 B2 C2 D2 E2 F2 D2 C2 G2 H2 A3 B3 C3 D3 E3 F3 G3 D3 C3 H3 A4 B4 C4 D4 E4 F4 G4 D4 C4 H4 I want the output to be: A1 B1 E1 F1 G1 H1 A2 B2 E2 F2 G2 H2 A3 B3 E3 F3 G3 H3 A4 B4 E4 F4 G4... (12 Replies)
Discussion started by: Gr4wk
12 Replies

9. Shell Programming and Scripting

How to delete 'duplicated' column values and make a delimited file too?

Hi, I have the following output from an Oracle SQL statement and I want to remove duplicated column values. I know it is possible using Oracle analytical/statistical functions but unfortunately I don't know how to use any of those. So now, I've gone to PLAN B using awk/sed maybe or any... (5 Replies)
Discussion started by: newbie_01
5 Replies

10. Shell Programming and Scripting

Remove duplicated records and update last line record counts

Hi Gurus, I need to remove duplicate line in file and update TRAILER (last line) record count. the file is comma delimited, field 2 is key to identify duplicated record. I can use below command to remove duplicated. but don't know how to replace last line 2nd field to new count. awk -F","... (11 Replies)
Discussion started by: green_k
11 Replies
GRPCK(8)						    System Management Commands							  GRPCK(8)

NAME
grpck - verify integrity of group files SYNOPSIS
grpck [options] [group [ shadow ]] DESCRIPTION
The grpck command verifies the integrity of the groups information. It checks that all entries in /etc/group and /etc/gshadow have the proper format and contain valid data. The user is prompted to delete entries that are improperly formatted or which have other uncorrectable errors. Checks are made to verify that each entry has: o the correct number of fields o a unique and valid group name o a valid group identifier (/etc/group only) o a valid list of members and administrators o a corresponding entry in the /etc/gshadow file (respectively /etc/group for the gshadow checks) The checks for correct number of fields and unique group name are fatal. If an entry has the wrong number of fields, the user will be prompted to delete the entire line. If the user does not answer affirmatively, all further checks are bypassed. An entry with a duplicated group name is prompted for deletion, but the remaining checks will still be made. All other errors are warnings and the user is encouraged to run the groupmod command to correct the error. The commands which operate on the /etc/group and /etc/gshadow files are not able to alter corrupted or duplicated entries. grpck should be used in those circumstances to remove the offending entries. OPTIONS
The -r and -s options cannot be combined. The options which apply to the grpck command are: -h, --help Display help message and exit. -r, --read-only Execute the grpck command in read-only mode. This causes all questions regarding changes to be answered no without user intervention. -R, --root CHROOT_DIR Apply changes in the CHROOT_DIR directory and use the configuration files from the CHROOT_DIR directory. -s, --sort Sort entries in /etc/group and /etc/gshadow by GID. By default, grpck operates on /etc/group and /etc/gshadow. The user may select alternate files with the group and shadow parameters. CONFIGURATION
The following configuration variables in /etc/login.defs change the behavior of this tool: MAX_MEMBERS_PER_GROUP (number) Maximum members per group entry. When the maximum is reached, a new group entry (line) is started in /etc/group (with the same name, same password, and same GID). The default value is 0, meaning that there are no limits in the number of members in a group. This feature (split group) permits to limit the length of lines in the group file. This is useful to make sure that lines for NIS groups are not larger than 1024 characters. If you need to enforce such limit, you can use 25. Note: split groups may not be supported by all tools (even in the Shadow toolsuite). You should not use this variable unless you really need it. FILES
/etc/group Group account information. /etc/gshadow Secure group account information. /etc/passwd User account information. EXIT VALUES
The grpck command exits with the following values: 0 success 1 invalid command syntax 2 one or more bad group entries 3 can't open group files 4 can't lock group files 5 can't update group files SEE ALSO
group(5), groupmod(8), gshadow(5), passwd(5), pwck(8), shadow(5). shadow-utils 4.1.5.1 05/25/2012 GRPCK(8)
All times are GMT -4. The time now is 06:27 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy