Sponsored Content
Top Forums Shell Programming and Scripting Joining lines from two files - please help Post 302081624 by phrazz on Wednesday 26th of July 2006 03:09:08 PM
Old 07-26-2006
how to diff sizes of two directories (check mirror)

This is an intriguing problem, and a common one at that. Basically we want to know if two files differ by size with the same names in two directories (or if files exist in one directory but not the other). So in essence we have a compound key = filename+filesize. This is a perfect job for uniq as long as we can build a good key.

So we generate two outfiles with the filenames and sizes. Take a long ls, discard the first line (total) and awk the 5th (size) and last (filename) columns:

ls -l $DirA | awk '{print $NF, $5}' | tail +2 > /tmp/ls.a
ls -l $DirB | awk '{print $NF, $5}' | tail +2 > /tmp/ls.b


Then we sort and count the number of "key" lines, then filter those which are unique:

sort ls.a ls.b | uniq -c | awk '($1 == 1)'
1 bad.jpg 384
1 fred.jpg 19580
1 fred.jpg 384


So this spits out files that only exist on one side or files that aren't the same size. By grepping these in both tmp files we learn which is A versus B side.

Make sure you remove the tmp files when you're done. Smilie

-Phrazz

P.S.: For two boxes that don't share a drive mount, use SSH and $PWD if the paths are the same. If not, there's a bit more trickery involved.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Joining 2 lines in a file together

Hi guys, I've got a log file which has entries that look like this: ------------------------------------------------------------------------------- 06/08/04 07:57:57 AMQ9002: Channel program started. EXPLANATION: Channel program 'INSCCPQ1.HSMTSPQ1' started. ACTION: None. ... (3 Replies)
Discussion started by: m223464
3 Replies

2. Shell Programming and Scripting

Joining 3 lines at a time

Hi, I have a file which has the contents as below : 07:38:36 EST date 20041117 07:39:06 EST 07:00:29 EDT date 20050504 07:25:16 EDT 07:00:40 EDT date 20050505 07:23:12 EDT I need to delete the new line character from all lines except 3rd,6th,9th etc. lines so that i get the output... (14 Replies)
Discussion started by: Sabari Nath S
14 Replies

3. UNIX for Dummies Questions & Answers

Conditionally joining lines in vi

I've done this before but I can't remember how. Too long away from vi. I want to do a search are replace, but I want the replace to be a join. Example see spot run see spot walk see spot run fast see spot hop %s/run$/<somehow perform a join with the next line>/g so the results... (0 Replies)
Discussion started by: ifermon
0 Replies

4. Shell Programming and Scripting

joining two lines

Hi , I want to join two lines in a file, where the second line contain query string. if it doesn't contain that string i don't want to join e.g. Input file is as following: name fame game none none none name fame game cat eat mice I need output file as name fame game none none... (2 Replies)
Discussion started by: ashrafonics
2 Replies

5. Shell Programming and Scripting

Issue with Joining lines from two files

Hi, I have two text files, that need their data joining/concatenation. 'Paste' works for this. But have an issue when there is mismatch in number of rows in each file. E.g. (main file) File1 - has 20 rows File2 - has 30 rows. Command 'paste file1 file2 > file3' joins all lines. I want the... (4 Replies)
Discussion started by: sharath160
4 Replies

6. Shell Programming and Scripting

Need help joining lines

Hi All, I need the command to join 2 lines into one. I found lots of threads but none give me the sollution. Probably because unix scripting is one of my best features ;) I got a logfile where line 2 needs to be joined with line 1, lines 4 needs to be joined with line 3 etc If you need... (16 Replies)
Discussion started by: rene21976
16 Replies

7. Shell Programming and Scripting

pattern matching lines using the date, and then joining the lines

Hi Guys, Was trying to attempt the below using awk and sed, have no luck so far, so any help would be appreciated. Current Text File: The first line has got an "\n", and the second line has got spaces/tabs then the word and "\n" TIME SERVER/CLIENT TEXT... (6 Replies)
Discussion started by: eo29
6 Replies

8. Shell Programming and Scripting

Joining broken lines

I have a plain test file with a delimeter ''. In this file some lines are broken into two. The first part of these broken line will have 6 columns and the second part will have 4. These broken lines will be consicutive. I want to join the two consicutive lines which are having 6 fields and 4... (8 Replies)
Discussion started by: ratheeshjulk
8 Replies

9. Shell Programming and Scripting

Joining lines in different way

Hi all, I'm excited to the part of unix.com forum, and noob to it. I have an query, where I have an file and it contains data like this use thread when posting do no I was expecting the result as use thread thread when when posting posting do do no use thread when thread when... (6 Replies)
Discussion started by: Jose Nirmal
6 Replies

10. Shell Programming and Scripting

Joining broken lines and removing empty lines

Hi - I have req to join broken lines and remove empty lines but should NOT be in one line. It has to be as is line by line. The challenge here is there is no end of line/start of line char. thanks in advance Source:- 2003-04-34024|04-10-2003|Claims|Claim|01-13-2003|Air Bag:Driver;... (7 Replies)
Discussion started by: Jackceasar123
7 Replies
VIPSTHUMBNAIL(1)					      General Commands Manual						  VIPSTHUMBNAIL(1)

NAME
vipsthumbnail - make thumbnails of image files SYNOPSIS
vipsthumbnail [flags] imagefile1 imagefile2 ... DESCRIPTION
vipsthumbnail(1) processes each imagefile in turn, shrinking each image to fit within a 128 by 128 pixel square. The shrunk image is writ- ten to a new file named tn_imagefile.jpg. This program is typically faster and uses less memory than other image thumbnail programs. For example: $ vipsthumbnail fred.png jim.tif will read image files fred.png and jim.tif and write thumbnails to the files tn_fred.jpg and tn_jim.jpg. $ vipsthumbnail --size=64 -o thumbnails/%s.png fred.jpg will read image file fred.jpg and write a 64 x 64 pixel thumbnail to the file thumbnails/fred.png. On Unix machines, vips will create temporary files in "/tmp" by default. Use the environment variable TMPDIR to change this location. On Windows, vips uses GetTempPath() to pick a location, see the MS documentation. Use the --vips-disc-threshold command-line switch, or the IM_DISC_THRESHOLD environment variable, to make vipsthumbnail use memory rather than temporary files. OPTIONS
-s N, --size=N Set the output thumbnail size to N x N pixels. The image is shrunk so that it just fits within this area, Images which are smaller than this are expanded. -o FORMAT, --output=FORMAT Set the output format string. The input filename has any file type suffix removed, then that value is substitued into FORMAT replac- ing %s. The default value is tn_%s.jpg meaning JPEG output, with tn_ prepended. You can add format options too, for example tn_%s.jpg:20 will write JPEG images with Q set to 20. -p I, --interpolator=I Resample with interpolator I. Use vips --list classes to see a list of valid interpolators. The default is bilinear. -n, --nosharpen By default, vipsthumbnail(1) will sharpen thumbnails slightly to make them look more pleasing. This option disables this sharpening. -e PROFILE, --eprofile=PROFILE Export thumbnails with this ICC profile. Images are only colour-transformed if there is both an output and an input profile avail- able. The input profile can either be embedded in the input image or supplied with the --iprofile option. -i PROFILE, --iprofile=PROFILE Import images with this ICC profile, if no profile is embdedded in the image. Images are only colour-transformed if there is both an output and an input profile available. The output profile should be supplied with the --oprofile option. -l, --nodelete Don't delete the profile from the output image. Since all output images will generally have the same profile, vipsthumbnail(1) will usually delete it. This option leaves the profile inside the image. -v, --verbose vipsthumbnail(1) normally runs silently, except for warning and error messages. This option makes it print a list of the operations it performs on each image. RETURN VALUE
returns 0 on success and non-zero on error. SEE ALSO
header(1) 13 May 2010 VIPSTHUMBNAIL(1)
All times are GMT -4. The time now is 07:53 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy