Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

sdiff discrepancies

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting sdiff discrepancies
# 1  
Old 05-08-2011
sdiff discrepancies
Hi All

I have two large (similar) files that have a unique number in the first column...(the trade id) and then a value for that trade in the second column...delimited by "~". Here is a snippet of what the files look like
Code:
10030372~-410682.73
10030490~-4052867.73
10030646~-352592.21
10030720~-2385197.26

One file is from yesterday and one is from today. I was using sdiff in the hope that only matching trade ids would get picked up....eg
Code:
sdiff $fname-yday $REPT_RPT_DIR/$fname-today | grep " | " | tr -d " " | tr "|" "~" | cut -d"~" -f1,2,4 > $filename-final

For some reason IDs that are only unique to one of the files are creeping into the final output...

I can write a workaround (which will be longer and messier as I am pretty new to this) however if anyone can suggest a way of matching the fields on the tradeIds but omitting ones that are in one file and not the other it will be forever grateful!! Smilie

Thanks
Kevin
Last edited by Scott; 05-08-2011 at 01:11 PM.. Reason: Please use code tags
# 2  
Old 05-08-2011
so, you want the lines which is having the trade id in both the files ?

please provide the sample input and output format
Last edited by itkamaraj; 05-08-2011 at 12:47 PM..
# 3  
Old 05-08-2011
Ideally I would like to compare $fname-yday to $fname-today based on the trade id....and have my output file as the tradeId (only if it is in both files) and the corresponding col2 values from both inital files....for example

The deal 10030372 had a value of -410682.73 yday and has a value of -200682.73 today so the final file should read
Code:
10030372~-410682.73~-200682.73

Apologies if im not making myself clear
# 4  
Old 05-08-2011
Try this
Code:
sdiff file1 file2 | awk -F"[|~]" '{if ($1=$3) print $1"~"$2"~"$4}'

# 5  
Old 05-08-2011
The above partially works...if i write it like this:
Code:
sdiff test1 test2 | awk -F"~" '{if ($1=$3) print $1"~"$2"~"$4}'

I get an output similar to this..
Code:
45194.92~30993.55                                               |  26179117~
-21373.59~-31471.20                                              |  26179715~
-214211.51~-192570.34                                             |  26179716~
166588.84~258942.67                                              |  26179717~
71886.30~69687.78                                               |  26179718~
21373.59~31471.20                                               |  26179719~
-166588.84~-258942.67                                             |  26179720~
214211.51~192570.34                                              |  26179721~
-71886.30~-69687.78                                              |  26179722~
34974.83~33149.26                                               |  26182586~
-34974.83~-33149.26                                              |  26182606~
19663.33~15499.63                                               |  26183060~

The numbers and corresponding values do seem to be correct (whereas before some werent) however this time I am only getting 9k matching values out of 15k...and I know that there are only 41 non matching deals
Last edited by Scott; 05-08-2011 at 01:12 PM.. Reason: Please use code tags
# 6  
Old 05-08-2011
This will work for you
Code:
sdiff file1 file2 | tr -d "[\t ]" | awk -F"[~|]" '{ if($1==$3) print ($1"~"$2"~"$4)}'

# 7  
Old 05-08-2011
Hi

The 1st part of the command still gives discrepencies

When I run this sdiff file1 file2 | tr -d "[\t ]" | the end of my file looks like this

26250326~-1782.25|26253872~434.06
26253872~444.07|26253873~108.52
26253873~111.02|26253874~-434.06
26253874~-444.07|26253875~-108.52
26253875~-111.02|26260133~71618.06


Lets take 26253875 for an example - even though this exists in both files...it will get missed.....

What I am looking for is something like a VLOOKUP excel function.....thought that sdiff was the answer but it looks as if i am mistaken
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Sdiff

file1: USER:XACME\cn3937 User:XACME\z01220 User:XACME\z01404 User:XACME\z02102 User:XACME\U17234 File2: USER:XACME\lawson USER:XACME\cn7913 USER:XACME\cn8037 USER:XACME\cn8042 USER:XACME\cn8046 USER:XACME\u31645 USER:XACME\u19050 USER:XACME\U28715 USER:XACME\U17234 USER:XACME\cn3937... (2 Replies)
Discussion started by: loktamann
2 Replies

2. Web Development

Discrepancies Between Our Mobile Detection Code and Google Analytics Regarding Chinese User Agents

Update: Last two days the number of users on the site has peaked (normally around 10AM US Eastern Time) between 4,300 and 4,500. This is the highest number of consistent concurrent users in at least 3 years. ... and the traffic continues to rise week-over-week. (5 Replies)
Discussion started by: Neo
5 Replies

3. Shell Programming and Scripting

SDiff Two files with space problem

Hello guys, I have a problem. I'm trying to use SDiff with two files which are containing spaces. My problem is that I want to save the output with > in an extra file. If I try to use it like this. sdiff "test file1" "test file2" > OutputfileI get this message: usage: diff ... (11 Replies)
Discussion started by: Mariopart
11 Replies

4. Shell Programming and Scripting

Sdiff command

Hi i am comparing file on 2 different machine with the help of script. however i am get below o/p ======= /usr/tmp ========= ======= /usr/tmp not a regular file i am not sure what does "not a regular file mean" . is it something serious, if yes then what i need to check or we can... (1 Reply)
Discussion started by: scriptor
1 Replies

5. Shell Programming and Scripting

Sdiff query

I wanted to ask can sdiff be used for comparing files which are present in different network?? If yes then how? Thanks in Advance!!!!!!! (1 Reply)
Discussion started by: bhavanabahety
1 Replies

6. Shell Programming and Scripting

Using sdiff without files

Hi, I'm trying to use sdiff by parsing the output of another command instead of the filename: sdiff <(echo test1) <(echo test2)However, this seems to cause my terminal session to stop working. If I use it with normal diff it works fine: ~$ diff <(echo test1) <(echo test2) 1c1 < test1... (4 Replies)
Discussion started by: Subbeh
4 Replies

7. UNIX for Dummies Questions & Answers

sdiff issue - not gathering correct difference

Hi there, I am hoping someone can help! I am doing a simple difference between two files using the sdiff command.. sdiff -s file1 file2 this works, but for some words it doesn't... Here is an example: fha 2B01 fha 2B01 fmaa 3A01 fmaa 3A01 fox 4A46 | fmx 3A01 ... (1 Reply)
Discussion started by: thegant
1 Replies

8. UNIX for Dummies Questions & Answers

A more intelligent SDIFF

Hi all I have two files which are essentially the same. However the way an exponent is written is different (i.e. in 1 file, a particular number might be written as 1.43230000E+02 whereas in another it might be 1.4323E2). If I use SDIFF then the program will merely check the ASCII characters... (1 Reply)
Discussion started by: robbiegregg
1 Replies

9. UNIX for Dummies Questions & Answers

Sdiff ? problem

Hello, I'm using Sdiff to compare 2 files, I've used this before and it works fine and still does in some cases. But it seems to trip up when using combinations of alpha-numeric text. I created two simple files to test and as you can see it seems to trip up on the "gr55a" text, any ideas ? ... (2 Replies)
Discussion started by: cowpoke
2 Replies

10. Shell Programming and Scripting

sdiff problem

Hi, I'm having the following problem with the unix sdiff command. for example if I try sdiff <filename 1> <filename 2> Where filename 1 and filename 2 contain really long pathnames that I need to compare. The output I get will be 139 characters long which is the default. But an output of... (0 Replies)
Discussion started by: ruudrio
0 Replies
Login or Register to Ask a Question