Joining lines from two files - please help


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Joining lines from two files - please help
# 22  
Old 07-25-2006
FILE_BASE="${i}" | awk -F: '{print $2}' doesn't make sense. This is what you need:
FILE_BASE=$(print "${i}" | nawk -F: '{print $2}')
Also, I am confused by FILE_BASE=${i##*:${HOME_DIR}/}. Where id HOME_DIR come from?

I can't accout for the zeros unless your find command is outputing additional fields.
# 23  
Old 07-26-2006
tmarikle,
Thank you once again. I got it going, but with the other commands that you gave me.
Do you think me not having nawk functionality and due to which I am using awk, it might be the issue with your initial script?
# 24  
Old 07-26-2006
I use nawk, which is a newer awk than the earlier awks, because my Solaris system's awk is mostly useless to me. Your system's awk may actually be nawk anyway.

...and yes, it could have contributed to the problems that you experienced if it's not there. Smilie
# 25  
Old 07-26-2006
how to diff sizes of two directories (check mirror)

This is an intriguing problem, and a common one at that. Basically we want to know if two files differ by size with the same names in two directories (or if files exist in one directory but not the other). So in essence we have a compound key = filename+filesize. This is a perfect job for uniq as long as we can build a good key.

So we generate two outfiles with the filenames and sizes. Take a long ls, discard the first line (total) and awk the 5th (size) and last (filename) columns:

ls -l $DirA | awk '{print $NF, $5}' | tail +2 > /tmp/ls.a
ls -l $DirB | awk '{print $NF, $5}' | tail +2 > /tmp/ls.b


Then we sort and count the number of "key" lines, then filter those which are unique:

sort ls.a ls.b | uniq -c | awk '($1 == 1)'
1 bad.jpg 384
1 fred.jpg 19580
1 fred.jpg 384


So this spits out files that only exist on one side or files that aren't the same size. By grepping these in both tmp files we learn which is A versus B side.

Make sure you remove the tmp files when you're done. Smilie

-Phrazz

P.S.: For two boxes that don't share a drive mount, use SSH and $PWD if the paths are the same. If not, there's a bit more trickery involved.
# 26  
Old 07-27-2006
You can run the following script to print the file name and size.
Assuming your directories contain the same number of files and same file names.

script name = 1.sh
----------------------
\rm -rf _out1 2>/dev/null
for f in `ls -1 | sort`
do
if [ "$f"xx = "_out"xx -o "$f"xx = "1.sh"xx ]
then
continue
fi
# executables have * at the end
f=`echo $f | cut -d'*' -f1`
#directories have "/" at the end of the file name
f=`echo $f | cut -d'/' -f1`
ls -s $f | awk -F" " '{ print $NF " " $1 }' >>_out1
done
You can run this script in 2 directories and redirect the ouput to 2 different files , say _out1 and _out2..
then use "paste _out1 _out2 " this gives you sorted list with file names and sizes.

Hope this helps.
Suman

Last edited by suman_jakkula; 07-27-2006 at 03:40 AM.. Reason: script name missing
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Joining broken lines and removing empty lines

Hi - I have req to join broken lines and remove empty lines but should NOT be in one line. It has to be as is line by line. The challenge here is there is no end of line/start of line char. thanks in advance Source:- 2003-04-34024|04-10-2003|Claims|Claim|01-13-2003|Air Bag:Driver;... (7 Replies)
Discussion started by: Jackceasar123
7 Replies

2. Shell Programming and Scripting

Joining lines in different way

Hi all, I'm excited to the part of unix.com forum, and noob to it. I have an query, where I have an file and it contains data like this use thread when posting do no I was expecting the result as use thread thread when when posting posting do do no use thread when thread when... (6 Replies)
Discussion started by: Jose Nirmal
6 Replies

3. Shell Programming and Scripting

Joining broken lines

I have a plain test file with a delimeter ''. In this file some lines are broken into two. The first part of these broken line will have 6 columns and the second part will have 4. These broken lines will be consicutive. I want to join the two consicutive lines which are having 6 fields and 4... (8 Replies)
Discussion started by: ratheeshjulk
8 Replies

4. Shell Programming and Scripting

pattern matching lines using the date, and then joining the lines

Hi Guys, Was trying to attempt the below using awk and sed, have no luck so far, so any help would be appreciated. Current Text File: The first line has got an "\n", and the second line has got spaces/tabs then the word and "\n" TIME SERVER/CLIENT TEXT... (6 Replies)
Discussion started by: eo29
6 Replies

5. Shell Programming and Scripting

Need help joining lines

Hi All, I need the command to join 2 lines into one. I found lots of threads but none give me the sollution. Probably because unix scripting is one of my best features ;) I got a logfile where line 2 needs to be joined with line 1, lines 4 needs to be joined with line 3 etc If you need... (16 Replies)
Discussion started by: rene21976
16 Replies

6. Shell Programming and Scripting

Issue with Joining lines from two files

Hi, I have two text files, that need their data joining/concatenation. 'Paste' works for this. But have an issue when there is mismatch in number of rows in each file. E.g. (main file) File1 - has 20 rows File2 - has 30 rows. Command 'paste file1 file2 > file3' joins all lines. I want the... (4 Replies)
Discussion started by: sharath160
4 Replies

7. Shell Programming and Scripting

joining two lines

Hi , I want to join two lines in a file, where the second line contain query string. if it doesn't contain that string i don't want to join e.g. Input file is as following: name fame game none none none name fame game cat eat mice I need output file as name fame game none none... (2 Replies)
Discussion started by: ashrafonics
2 Replies

8. UNIX for Dummies Questions & Answers

Conditionally joining lines in vi

I've done this before but I can't remember how. Too long away from vi. I want to do a search are replace, but I want the replace to be a join. Example see spot run see spot walk see spot run fast see spot hop %s/run$/<somehow perform a join with the next line>/g so the results... (0 Replies)
Discussion started by: ifermon
0 Replies

9. Shell Programming and Scripting

Joining 3 lines at a time

Hi, I have a file which has the contents as below : 07:38:36 EST date 20041117 07:39:06 EST 07:00:29 EDT date 20050504 07:25:16 EDT 07:00:40 EDT date 20050505 07:23:12 EDT I need to delete the new line character from all lines except 3rd,6th,9th etc. lines so that i get the output... (14 Replies)
Discussion started by: Sabari Nath S
14 Replies

10. Shell Programming and Scripting

Joining 2 lines in a file together

Hi guys, I've got a log file which has entries that look like this: ------------------------------------------------------------------------------- 06/08/04 07:57:57 AMQ9002: Channel program started. EXPLANATION: Channel program 'INSCCPQ1.HSMTSPQ1' started. ACTION: None. ... (3 Replies)
Discussion started by: m223464
3 Replies
Login or Register to Ask a Question