Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Unique account id file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Unique account id file
# 1  
Old 05-14-2010
Unique account id file
I have a fixed length file (each record of 288 characters).

The unique id in each record is account_id which is from position 1:30

My file is having duplicate account ids ( records having same account id). However the information except account_id can be different. ( Most of the records having difference balance { position 60:80}).

I want to extract only 1 record of each account id. It can be first occurrence of that account id.So the new file will have unique account id records only.

How can I achieve this.
# 2  
Old 05-14-2010
so you need the unique account numbers ?

how about using the cut command to extract columns 1-30 ? then do a sort unique command?
Code:
 cut -c1-30 file | sort -u

# 3  
Old 05-14-2010
need the entire record
I want the new file to have the entire record ( all 288 characters) of unique account ids.

If account id 00001 is having 5 recordsI want 1 whole record of this account id.
# 4  
Old 05-14-2010
try this ?
a[] is an array keeping track how many times the account has appeared. if it appears the first time, we print it

Code:
awk '{account=substr($0,1,30); a[account]++; if(a[account]==1){print $0;}}' file

Last edited by Leion; 05-14-2010 at 05:26 AM.. Reason: typo error
This User Gave Thanks to Leion For This Post:
varunrbs
# 5  
Old 05-14-2010
Code:
#!/bin/sh
while read L
do
    [ "$L" = "$M" ] || echo "$L"
    M=$L
done < file

# 6  
Old 05-19-2010
Quote:
Originally Posted by Leion
try this ?
a[] is an array keeping track how many times the account has appeared. if it appears the first time, we print it

Code:
awk '{account=substr($0,1,30); a[account]++; if(a[account]==1){print $0;}}' file


Thanks this worked perfectly.
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Windows & DOS: Issues & Discussions

What happens to your skype account if you close outlook.com email account?

Hello, Does anyone know what happens to your skype account if you close the outlook.com email account which are linked together? As you know they are both owned by Microsoft. Thanks (0 Replies)
Discussion started by: milhan
0 Replies

2. UNIX for Beginners Questions & Answers

sed awk: split a large file to unique file names

Dear Users, Appreciate your help if you could help me with splitting a large file > 1 million lines with sed or awk. below is the text in the file input file.txt scaffold1 928 929 C/T + scaffold1 942 943 G/C + scaffold1 959 960 C/T +... (6 Replies)
Discussion started by: kapr0001
6 Replies

3. How to Post in the The UNIX and Linux Forums

Simultaneously try to execute commands after connecting to remote account to one account

I have made password less connection to my remote account. and i tried to execute commands at a time. but i am unable to execute the commands. ssh $ACCOUNT_DETAILS@$HOST_DETAILS cd ~/JEE/*/logs/ (1 Reply)
Discussion started by: kishored005
1 Replies

4. UNIX for Dummies Questions & Answers

Print unique lines without sort or unique

I would like to print unique lines without sort or unique. Unfortunately the server I am working on does not have sort or unique. I have not been able to contact the administrator of the server to ask him to add it for several weeks. (7 Replies)
Discussion started by: cokedude
7 Replies

5. Shell Programming and Scripting

Change unique file names into new unique filenames

I have 84 files with the following names splitseqs.1, spliseqs.2 etc. and I want to change the .number to a unique filename. E.g. change splitseqs.1 into splitseqs.7114_1#24 and change spliseqs.2 into splitseqs.7067_2#4 So all the current file names are unique, so are the new file names.... (1 Reply)
Discussion started by: avonm
1 Replies

6. Shell Programming and Scripting

Create shell script to extract unique information from one file to a new file.

Hi to all, I got this content/pattern from file http.log.20110808.gz mail1 httpd: Account Notice: close igchung@abc.com 2011/8/7 7:37:36 0:00:03 0 0 1 mail1 httpd: Account Information: login sastria9@abc.com proxy sid=gFp4DLm5HnU mail1 httpd: Account Notice: close sastria9@abc.com... (16 Replies)
Discussion started by: Mr_47
16 Replies

7. Shell Programming and Scripting

get part of file with unique & non-unique string

I have an archive file that holds a batch of statements. I would like to be able to extract a certain statement based on the unique customer # (ie. 123456). The end for each statement is noted by "ENDSTM". I can find the line number for the beginning of the statement section with sed. ... (5 Replies)
Discussion started by: andrewsc
5 Replies

8. Linux

Apply disk quota to account(dedicate 3 GB to account).

Hi , I am faceing lot of problem due to "disk space is not enough". senerio is like as, In system has 5 account. a,b,c,d,e say account c if very critical. Due to other user's data, user 'c' is faceing disk space issue. I want to dedicate 3 GB for user 'c'. No user... (1 Reply)
Discussion started by: ashokd009
1 Replies

9. UNIX for Dummies Questions & Answers

Change Account to not lock account if password expires

I have access to 15+ UNIX boxes at work, and I do not consistently log onto all of them over time. When I do try to access one I havent been on in awhile, my account is locked as the password has expired. I need to request to the UNIX SA's that the password expiration is 90 days and that if it... (1 Reply)
Discussion started by: stringzz
1 Replies
Login or Register to Ask a Question