Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Shel program file handling

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Shel program file handling
# 1  
Old 02-13-2008
Shel program file handling
Hi,

Iam having the file as follows:
Code:
QWASEDRF1234567890098765     abc@quebex.com            000000000-932333       678394-56=3   9033894
QWASEDRF1234567890098765     abc@quebex.com            000000000-932333       678394-56=3   9033894
OPIUYTREE0986666544443322     dcsx@olivaa.net              123456678-rererrrde      09655584-35=6   77778
OPIUYTREE0986666544443322     dcsx@olivaa.net              123456678-rererrrde      09655584-35=6   77778

I need to take the lines that occurs more than once.

Actually i have used uniq -d filename. It is taking corectly with the file having lines that are not having the spaces.

Here our file has lines with lots of spaces inbetween.

Can anybody help me in this ??

Thanks
nivas
# 2  
Old 02-13-2008
Code:
cat filename | tr -s '[:space:]' | uniq -d

That should help.
# 3  
Old 02-13-2008
Quote:
Originally Posted by nivas
I need to take the lines that occurs more than once. Actually i have used uniq -d filename. It is taking corectly with the file having lines that are not having the spaces. Here our file has lines with lots of spaces inbetween.
Do before you uniq -d, you want to alter the input to make them have less white space? Your example doesn't seem to suffer from this problem or your not showing the lines that don't suffer from too much white space.

None the less "tr --squeeze-repeats" might be the answer for something like this:

Code:
$ cat >tmp
one two
one      two
three four
three four
^d

$ cat tmp | tr -s " " | uniq -d
one two
three four

# 4  
Old 02-13-2008
Iam having the file as follows:

Code:
KILOPJUY23415000000003537   roaringrat@hscd_com                                                                                  2008-02-020014243.99 0000758 00284.00 2008-03-01SEMENZA, CATHY E                              000000000000016.03 000000000000000.00 CBB00010000000920
KILOPJUY23415000000003537   roaringrat@hscd_com                                                                                  2008-02-020014243.99 0000758 00284.00 2008-03-01SEMENZA, CATHY E                              000000000000016.03 000000000000000.00 CBB00010000000920
AQWSEDFR298099000003537   poiuty@hpoiyuj-usa_com                                                                                2008-02-020000823.15 0011676 00017.00 2008-03-01HJORTH,ROGER C                                000000000000051.66 000000000000040.00 CBB00010000000906
AQWSEDFR298099000003537   poiuty@hpoiyuj-usa_com                                                                                2008-02-020000823.15 0011676 00017.00 2008-03-01HJORTH,ROGER C                                000000000000051.66 000000000000040.00 CBB00010000000906


The abouve 2 methods are not working here...pls help..
nivas
# 5  
Old 02-13-2008
Quote:
Originally Posted by nivas
Iam having the file as follows: The abouve 2 methods are not working here...pls help..
squeeze those characters are they tabs or something? You said spaces if its whitespace then you need you squeeze accordingly.
# 6  
Old 02-13-2008
Its all white spaces and not tabs.

how to sqeeze???
nivas
# 7  
Old 02-13-2008
try this also
Try this out.....

awk '{ a[$1]=$0 }
END{
n = asort(a)
for (i=1;i<=n;i++) print a[i]
}' <filename>
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to simulate ''Esc'' keystroke in a bash shel script?

I am using Ubuntu 12.4 I am running the following script: ++++++++ #!/bin/bash while ; do xdotool mousemove 248 539 click 1 & sleep 3 done +++++++++ This moves the mouse on the specified position on the screen and clicks that pauses the script for 3 seconds and repeats the... (2 Replies)
Discussion started by: alfarnume
2 Replies

2. Solaris

How to create core through program at the time of crash by handling signals?

I am in process of writing a library which can make any application of my product capable of creating core in the application's log folder with a product friendly core file name programatically. In my library I am registering for certain signals e.g. SIGILL, SIGFPE, SIGBUS, SIGSEGV, SIGSYS, SIGABRT... (5 Replies)
Discussion started by: rajeev_ks
5 Replies

3. AIX

How to create core through program at the time of crash by handling signals?

I am in process of writing a library which can make any application of my product capable of creating core in the application's log folder with a product friendly core file name programatically. In my library I am registering for certain signals e.g. SIGILL, SIGFPE, SIGBUS, SIGSEGV, SIGSYS, SIGABRT... (1 Reply)
Discussion started by: rajeev_ks
1 Replies

4. Homework & Coursework Questions

need help with a shel script plzz

hey how u doing guys ... I'm a student @ EMU and taking a beginner Linux class and I have those two extra credit project that im struggling with so I hope to get some correction and help write a shell script that will 1-prompt the user for a file name 2-check to see if the file exists... (2 Replies)
Discussion started by: flyman
2 Replies

5. Programming

Please help:program hang stuck there signal handling on POSIX Message Queue UNIX C programming

in a single main() function,so need signal handling. Use Posix Message Queue IPC mechanism , can ignore the priority and other linked list message,to implement the scenario: client:Knock Knock server:who's there client: Eric Server:Eric,Welcome. client:exit all process terminated ... (1 Reply)
Discussion started by: ouou
1 Replies

6. UNIX for Advanced & Expert Users

File handling

my input for a script is another csv file but in that file say 7 lines are there ... how can i get line by line to that input for example : >cat link.csv www.yahoo.com,yahoo www.google.com,google www.unix.com,unix another file in that file i need to ping the above links ... (2 Replies)
Discussion started by: ponmuthu
2 Replies

7. Shell Programming and Scripting

need some clarification on for loop in shel script

for ( ( $i=0 ; $i<=6 ; $i++ ) ) do p=/RSA-Data/PE-data00"${i}" echo "$p" done --- i need output as below /RSA-Data/PE-data000 . . . /RSA-Data/PE-Data006 (1 Reply)
Discussion started by: mail2sant
1 Replies

8. UNIX for Advanced & Expert Users

File Handling

Hi, I have a log file which runs into 3 to 5 GB. We store this typically for 6 months. When a new month starts we move the previous month into a 9 month back up log (file.9m) and delete the last month of the 9 month back up. Iam using awk to find the data and cat to join the files like... (3 Replies)
Discussion started by: baanprog
3 Replies

9. Shell Programming and Scripting

remotely installing apache server using Shel scripting

Hi guys, I am trying to install apache server onto a remote machine. I file transferred the .gz file and extracted it on the remote server using shell scripts. Then I used the make and make install commands. Everything worked fine until now. I know that i had to change the httpd.conf... (2 Replies)
Discussion started by: tadi18
2 Replies

10. Shell Programming and Scripting

Shel Script doesn't work from Exceed

Hi, I am using this script to load up my Oracle Databases, but when I log in through Exceed, it hangs. Can anyone tell me what else I need to add to make this work?? Details ****************************************************************************************************... (11 Replies)
Discussion started by: dnkansah
11 Replies
Login or Register to Ask a Question