Dynamic Array Issue


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Dynamic Array Issue
# 1  
Old 03-11-2007
Dynamic Array Issue

Could one of you, please, provide some input regarding my problem below and it is as follows:

I have 2 files that I need to make sure are identical before processing:

First, I sort both files

Second, I do a diff file1 file2 > File 3
This provides me with the difference.

Now, I need to read File 3 - into an array and remove the difference from both files - if any

filename = /.../.../.../... #input file
set -A <array_name> [ < $filename]

Could I use the sed '/expression/' to remove the line???


Please advise.

Don
# 2  
Old 03-11-2007
Quote:
Originally Posted by ddedic
I have 2 files that I need to make sure are identical before processing:
Just:
cp file1 file2
or copy the larger one to the smaller one (or vice versa).
# 3  
Old 03-11-2007
dynamic

sorry, for not being more specific:

I am processing 2 large files - 10mil records

One is master and the other is detail. Because there is no uniqueness between them I have to rely on the row count of each file before processing.

With that said, I cut -d "|" -f2 from both files into > aa.out and ab.out

Once this is complete, I sort both files and do a diff to get the records in one and not the other and vice versa.

Now that I have a diff.out (most of the time it will only be a record or two)
I need to be able to remove them from the original file before application processing.

I will attempt to illustrate and for your review:

File 1
111111
222222
333333
555555

File 2
111111
222222
333333
444444

cat diff.out
03c4400
444444
03c5556
555555

I now need to remove those two records from both files - please let me know if any ideas.
# 4  
Old 03-11-2007
I assume you have something like file1.orig and file2.orig with more fields but using | as a field separator. Here is a way to do this with comm and sed:
Code:
$
$ cat file2.orig
111111|sdjksd
222222|sdjksd
333333|sdjksd
444444|sdjksd
$ comm -13 file1 file2 | sed 's=^=/^=;s=$=\|/d=' > file2.sed
$ sed -f file2.sed < file2.orig
111111|sdjksd
222222|sdjksd
333333|sdjksd
$

# 5  
Old 03-11-2007
thank you for your prompt response. This is exactly what I was looking for. Thank you.
# 6  
Old 03-12-2007
Data dynamic arrays

correct - i am using a "|" delimited file and the only common field between the 2 files is -f3 that i cut before comparing - the rest of the fields are different.

Specifically, I am struggling with step to remove those records from the individual files once difference has been determined.

Please let me know if you can help.


Example.

File 1 - master
ab|dc|11111|2|3|4|5|6|
ab|dc|22222|2|3|4|5|6|
ab|dc|33333|2|3|4|5|6|
ab|dc|55555|2|3|4|5|6|

File 2 - detail
xa|zd|qs|11111|w|3|4|66|b
xa|zd|qs|22222|w|3|4|8|d
xa|zd|qs|33333|w|3|4|3|s
xa|zd|qs|44444|w|3|4|1|f

===============================

After compare Smilie result should be:
File 1 - master
ab|dc|11111|2|3|4|5|6|
ab|dc|22222|2|3|4|5|6|
ab|dc|33333|2|3|4|5|6|


File 2 - detail
xa|zd|qs|11111|w|3|4|66|b
xa|zd|qs|22222|w|3|4|8|d
xa|zd|qs|33333|w|3|4|3|s
# 7  
Old 03-12-2007
Code:
$ cat file2.orig
aa|bb|111111|sdjksd
aa|bb|222222|sdjksd
aa|bb|333333|sdjksd
aa|bb|444444|sdjksd
$ comm -13 file1 file2 | sed 's=^=/^[^|]*|[^|]*|=;s=$=\|/d='  > file2.sed
$ sed -f file2.sed < file2.orig
aa|bb|111111|sdjksd
aa|bb|222222|sdjksd
aa|bb|333333|sdjksd
$

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Programming

Memory corruption in dynamic array of strings

I put together a C function to add strings to a dynamic array of strings (mostly for educational purpose to explain pointers to my kid). It works, but sometimes one or two strings in the array becomes corrupted. Running example on 64 bit Ubuntu, gcc ver. 4.8.4 Hope my code is self-explanatory: ... (2 Replies)
Discussion started by: migurus
2 Replies

2. Shell Programming and Scripting

Help w/ Reading Matrix & Storing in dynamic array

First of I would just like to state that I am not looking for you guys to just do my work for me, I do want to learn and actually understand everything that is happening. Hey all, I am having trouble on this. What I need to do is... Write an executable C file that will take a text file (not a... (8 Replies)
Discussion started by: innvert
8 Replies

3. Programming

readdir and dynamic array memory corruption

Hi everyone I am developing an utility. At some part of it I read directory entries to a dynamic array: struct list It stores pointers to items: list.entries, which are structures: struct entry If a number of files in a directory is greater then number of elements an array was initially... (11 Replies)
Discussion started by: torbium
11 Replies

4. Shell Programming and Scripting

Assigning values for a dynamic array for an input

Hello, Can somebody please give me a snippet for the below requirement. I want to assign the values separeted by a comma to be assigned to a dynamic array. If I give an input (read statement) like abc1,abc2,abc3,abc4,abc5, all these strings abc* should be assigned to an array like below... (2 Replies)
Discussion started by: suneelj
2 Replies

5. Shell Programming and Scripting

dynamic index for array in while loop

Hi, I'm just trying to use a dynamic index for some array elements that I'm accessing within a loop. Specifically, I want to access an array at variable position $counter and then also at location $counter + 1 and $counter + 2 (the second and third array positions after it) but I keep getting... (0 Replies)
Discussion started by: weak_code-fu
0 Replies

6. Programming

array dynamic allocation

Hi, I have the following problem: i must allocate a dynamic array from a subroutine which should return such array to main function. The subroutine has already a return parameter so i thought of pass the array as I/O parameter. I tried the following program but it doesn't work (segmentation... (11 Replies)
Discussion started by: littleboyblu
11 Replies

7. Programming

Creating an array to hold posix thread ids: Only dynamic array works

I am facing a strange error while creating posix threads: Given below are two snippets of code, the first one works whereas the second one gives a garbage value in the output. Snippet 1 This works: -------------- int *threadids; threadids = (int *) malloc (num_threads * sizeof(int)); ... (4 Replies)
Discussion started by: kmehta
4 Replies

8. Shell Programming and Scripting

creating a dynamic array

i want to create an array the array elements are populated depending upon the number of entries present in a data file The data file is created dynamically how to achieve the same thanks (1 Reply)
Discussion started by: trichyselva
1 Replies

9. Programming

(MPI) Segmentation fault with dynamic allocated 2D array

Hi all, I'm getting segmentation fault errors when I try to send/receive 2 rows of a matrix as a block, and I can't figure out why. Basically, I have a 4x5 matrix with an extra top row and an extra leftmost column, making it a (4+1)x(5+1) matrix stored in P0. I'm trying to send the 2nd and 3rd... (1 Reply)
Discussion started by: lancer6238
1 Replies

10. Shell Programming and Scripting

creating a dynamic array in ksh

Hi, Is it possible to create a dynamic array in shell script. I am trying to get the list of logfiles that created that day and put it in a dynamic array. I am not sure about it. help me New to scripting Gundu (3 Replies)
Discussion started by: gundu
3 Replies
Login or Register to Ask a Question