Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Shell script to filter records in a zip file that contains matching columns from another file
Top Forums Shell Programming and Scripting Shell script to filter records in a zip file that contains matching columns from another file Post 302962566 by anil.v on Tuesday 15th of December 2015 10:38:02 PM
Old 12-15-2015
Shell script to filter records in a zip file that contains matching columns from another file
Not sure if this is the correct forum for this question. I have two files. file1.zip, file2

Input:

file1.zip
Code:
col1,    col2 ,   col3
a ,      b ,  0:0:0:0:0:c436:9346:d40b
x,       y,   0:0:0:0:0:880:39f9:c9a7
m,      n ,  0:0:0:0:0:80c7:9161:fe00

file2.txt
Code:
col1
c4:36:93:46:d4:0b
08:80:39:f9:c9:a7
48:1d:60:62:f5:9d

I need to
  • Convert col3 in file1 to col1 in file2 format i.e., ab:cd:ef:gh:ij:kl by padding 0's for and using semicolon':' for every 2 characters after trimming first 10 char.
  • compare it with col1 in table B and store the matching columns required output for the above scenario

Code:
col1 ,   col2,    col3
a  ,    b  , c4:36:93:46:d4:0b
x ,     y ,  08:80:39:f9:c9:a7

I have 1500 zip files and one reference file. Not sure how to achieve this in unix. Can someone please help.

for direct files, I used

ls *.zip | awk '{ print "zipgrep -f file2.txt "$0" >> result.csv"}' | xargs -I {}
Last edited by Scrutinizer; 12-16-2015 at 12:26 AM.. Reason: formatting; CODE tags
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Count records in a zip file

Hello, I searched the forums on the keywords in the title I used above, but I did not find the answer: Is it possible to count records in a .zip file on an AIX machine if i don't have pkunzip installed? From all the research I'm reading in google and the reading of pkunzip in Unix.com,... (3 Replies)
Discussion started by: tekster757
3 Replies

2. Shell Programming and Scripting

Filter records in a file using AWK

I want to filter records in one of my file using AWK command (or anyother command). I am using the below code awk -F@ '$1=="0003"&&"$2==20100402" print {$0}' $INPUT > $OUTPUT I want to pass the 0003 and 20100402 values through a variable. How can I do this? Any help is much... (1 Reply)
Discussion started by: gpaulose
1 Replies

3. Shell Programming and Scripting

Shell script matching similar records

hello all, I have requirement to identify similar records matching about 80% to 90%.I have to black list customers with multiple accounts. The data is in the Oracle Database, but is there any way I can get the data into flat file and compare the strings and fetch similar matching records? ... (2 Replies)
Discussion started by: kashik786
2 Replies

4. Shell Programming and Scripting

Filter records based on 2nd file

Hello, I want to filter records of a file if they fall in range associated with a second file. First the chr number (2nd col of 1st file and 1st col of 2nd file) needs to be matched. Then if the 3rd col of the first file falls within any of the ranges specified by the 2nd and 3rd cols , then... (4 Replies)
Discussion started by: ritakadm
4 Replies

5. UNIX for Dummies Questions & Answers

Shell script to read lines in a text file and filter user data Shell Programming and Scripting

sxsaaas (3 Replies)
Discussion started by: VikrantD
3 Replies

6. UNIX for Dummies Questions & Answers

Filter records in a huge text file from a filter text file

Hi Folks, I have a text file with lots of rows with duplicates in the first column, i want to filter out records based on filter columns in a different filter text file. bash scripting is what i need. Data.txt Name OrderID Quantity Sam 123 300 Jay 342 498 Kev 78 2500 Sam 420 50 Vic 10... (3 Replies)
Discussion started by: tech_frk
3 Replies

7. Shell Programming and Scripting

Script to find blank records in a file except for few columns

I have a file with the following format: X|High|2|GIC|DM||XHM|||6 Months X|Moderate|2|GIC|DM||XHM|||6 Months X|High|2|GCM|DM||XSF|||6 Months X|Med|2|GCM|DM||XSF|||6 Here there are ten columns but I need to print rows having blank records in any of the rows (except for 6th,8th and 9th... (10 Replies)
Discussion started by: chatwithsaurav
10 Replies

8. Shell Programming and Scripting

Shell script not able to zip the file using gzip

Hi all, I am calling Temp.sh and it is has simple line $gpath=`which gzip` $gpath $FilePath/My_temp.log if I run this script, logging to server then its works fine. But when I send this script over the SSH it does not work at it. gzip is command is not execute. I am using gzip 1.6... (2 Replies)
Discussion started by: girijajoshi
2 Replies

9. Shell Programming and Scripting

Help with Shell Scrip in Masking particular columns in .csv file or .txt file using shell script

Hello Unix Shell Script Experts, I have a script that would mask the columns in .csv file or .txt file. First the script will untar the .zip files from Archive folder and processes into work folder and finally pushes the masked .csv files into Feed folder. Two parameters are passed ... (5 Replies)
Discussion started by: Mahesh G
5 Replies

10. UNIX for Beginners Questions & Answers

Shell script to filter certain lines from a file

hi~ i need script on AIX. and have a text file following : create aa 1 2 3 from a@a; create bb from b; create cc 3 4 5 6 6 7 from c@c; (7 Replies)
Discussion started by: tomato00
7 Replies

LEARN ABOUT DEBIAN

zipmerge

ZIPMERGE(1)						      General Commands Manual						       ZIPMERGE(1)

NAME

       zipmerge - merge zip archives

SYNOPSIS

       zipmerge [-DhIiSsV] target-zip source-zip Op source-zip ...

DESCRIPTION

       zipmerge  merges  the  source zip archives source-zip into the target zip archive target-zip.  By default, files in the source zip archives
       overwrite existing files of the same name in the target zip archive.

       Supported options:

	      -D   Ignore directory components in file name comparisons.

	      -h   Display a short help message and exit.

	      -I   Ignore case in file name comparisons

	      -i   Ask before overwriting files.  See also -s.

	      -S   Do not overwrite files that have the same size and CRC32 in both the source and target archives.

	      -s   When -i is given, do not before overwriting files that have the same size and CRC32.

	      -V   Display version information and exit.

EXIT STATUS

       zipmerge exits 0 on success and 1 if an error occurred.

SEE ALSO

       zipcmp(1), ziptorrent(1), libzip(3)

AUTHORS

       Dieter Baron <dillo@giga.or.at> and Thomas Klausner <tk@giga.or.at>

NiH								   June 4, 2008 						       ZIPMERGE(1)
All times are GMT -4. The time now is 05:29 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy