Sponsored Content
Top Forums Shell Programming and Scripting Grouping files according to certain fields in their name Post 302603552 by kristinu on Thursday 1st of March 2012 09:56:59 AM
Old 03-01-2012
Grouping files according to certain fields in their name

I have a list of fils stored insortedLst, and want to select certain fields to group specific files together:

Example of the files would be as below:

Code:
n02-z30-dsr65-ndelt0.25-varp0.002-16x12drw-run1.log
n02-z30-dsr65-ndelt0.25-varp0.002-16x12drw-run2.log
n02-z30-dsr65-ndelt0.25-varp0.002-16x12drw-run3.log
n02-z30-dsr65-ndelt0.25-varp0.004-16x12drw-run1.log
n02-z30-dsr65-ndelt0.25-varp0.004-16x12drw-run2.log
n02-z30-dsr65-ndelt0.25-varp0.004-16x12drw-run3.log
n02-z30-dsr65-ndelt0.25-varp0.006-16x12drw-run1.log
n02-z30-dsr65-ndelt0.25-varp0.006-16x12drw-run2.log
n02-z30-dsr65-ndelt0.25-varp0.006-16x12drw-run3.log
n02-z30-dsr65-ndelt0.25-varp0.008-16x12drw-run1.log
n02-z30-dsr65-ndelt0.25-varp0.008-16x12drw-run2.log
n02-z30-dsr65-ndelt0.25-varp0.008-16x12drw-run3.log
n02-z30-dsr65-ndelt0.25-varp0.010-16x12drw-run1.log
n02-z30-dsr65-ndelt0.25-varp0.010-16x12drw-run2.log
n02-z30-dsr65-ndelt0.25-varp0.010-16x12drw-run3.log

I use the following command to group similar files according to similar fields

Code:
        echo $sortedLst | tr ' ' '\n' \
          | awk -F- '{ c=($4$5$6!=p && FNR!=1)?ORS:""; p=$4$5$6 } { printf("%c%s\n",c,$0) }'

I now want the user to be able to define the grouping fields himself rather than hardwiring '$4$5$6' in the awk script.

Using the code above, the output would then be shown like this:

Code:
n02-z30-dsr65-ndelt0.25-varp0.002-16x12drw-run1.log
n02-z30-dsr65-ndelt0.25-varp0.002-16x12drw-run2.log
n02-z30-dsr65-ndelt0.25-varp0.002-16x12drw-run3.log
   
n02-z30-dsr65-ndelt0.25-varp0.004-16x12drw-run1.log
n02-z30-dsr65-ndelt0.25-varp0.004-16x12drw-run2.log
n02-z30-dsr65-ndelt0.25-varp0.004-16x12drw-run3.log
   
n02-z30-dsr65-ndelt0.25-varp0.006-16x12drw-run1.log
n02-z30-dsr65-ndelt0.25-varp0.006-16x12drw-run2.log
n02-z30-dsr65-ndelt0.25-varp0.006-16x12drw-run3.log
   
n02-z30-dsr65-ndelt0.25-varp0.008-16x12drw-run1.log
n02-z30-dsr65-ndelt0.25-varp0.008-16x12drw-run2.log
n02-z30-dsr65-ndelt0.25-varp0.008-16x12drw-run3.log
   
n02-z30-dsr65-ndelt0.25-varp0.010-16x12drw-run1.log
n02-z30-dsr65-ndelt0.25-varp0.010-16x12drw-run2.log
n02-z30-dsr65-ndelt0.25-varp0.010-16x12drw-run3.log


Last edited by kristinu; 03-01-2012 at 11:10 AM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Grouping files into tars

Hi all, I have a problem where i have several files in a directory which I SCP from a server to my local machine and i would like to periodically tar/gzip them based on their naming convention. Here is the scenario: I SCP files (which all end with the same ending) periodically across to a... (3 Replies)
Discussion started by: muay_tb
3 Replies

2. Shell Programming and Scripting

parsing file names and then grouping similar files

Hello Friends, I have .tar files which exists under different directories after the below code is run: find . -name "*" -type f -print | grep .tar > tmp.txt cat tmp.txt ./dir1/subdir1/subdir2/database-db1_28112009.tar ./dir2/subdir3/database-db2_28112009.tar... (2 Replies)
Discussion started by: EAGL€
2 Replies

3. Shell Programming and Scripting

Comparing fields in two files

Hi, i want to compare two files by one field say $3 in file1 needs to compare with $2 in file2. sample file1 - reqd_charge_code 2263881188,24570896,439 2263881964,24339077,439 2263883220,22619162,228 2263884224,24631840,442 2263884246,22612161,442 sample file2 - rg_j ... (2 Replies)
Discussion started by: raghavendra.cse
2 Replies

4. Shell Programming and Scripting

combine 3 files by grouping

I have a file, which is really large but i shortened it: A3059GVS 1 A 01 Plate_1 40 25.37016 14.6298 A3059GVS 2 A 01 Plate_2 40 26.642002 13.3583 A3059GVS 3 A 02 Plate_1 40 25.381462 ... (4 Replies)
Discussion started by: mykey242
4 Replies

5. Shell Programming and Scripting

Add fields in different files only if some fields between them match

Hi everybody (first time posting here) I have a file1 that looks like > 1,101,0.1,0.1 1,26,0.1,0.1 1,3,0.1,0.1 1,97,0.5,0.5 1,98,8.1,0.218919 1,99,6.2,0.248 2,101,0.1,0.1 2,24,3.1,0.147619 2,25,23.5,0.559524 2,26,34,0.723404with 762 lines.. I have another 'similar' file2 > ... (10 Replies)
Discussion started by: murpholinox
10 Replies

6. UNIX for Dummies Questions & Answers

Please help me to find out maximum value of a field based on grouping of other fields.

Please help me to find out maximum value of a field based on grouping of other fields, as we do in SQL. Like in SQL if we are having below records : Client_Name Associate_Name Date1 Value C1111 A1111 2012-01-17 10 C1111 A1111 ... (1 Reply)
Discussion started by: KamalKumarKalra
1 Replies

7. Shell Programming and Scripting

Compare fields in files

Hi, I need the most efficient way of comparing the following and arriving at the result I have a file which has entries like, File1: 1|2|5|7|8|2|3|6|3|1 File2: 1|2|3|1|2|7|9|2 I need to compare the entries in these two file with those of a general file, 1|2|3|5|2|5|6|9|3|1... (7 Replies)
Discussion started by: pradebban
7 Replies

8. Shell Programming and Scripting

grouping log files based on counter

I have my log file as below 00:18:02 - Nothing normal; Garbage Collection kicked off & running from last 3 min... 00:19:02 - Nothing normal; Garbage Collection kicked off & running from last 4 min... 00:19:02 - Nothing normal; Garbage Collection kicked off & running from last 4 min...... (11 Replies)
Discussion started by: manas_ranjan
11 Replies

9. Shell Programming and Scripting

Grouping files on pattern

I have this Requirement where i have to group the files, I have a folder say "temp" where many files resides...files are like this; 010020001_S-ABC-Sort-DEFAW_YYYYMMDD_HHMMSS.txt 010020004_S-PQR-Sort-DRTON_YYYYMMDD_HHMMSS.txt 010020009_S-JKL-Sort_MNOLO_YYYYMMDD_HHMMSS.txt... (8 Replies)
Discussion started by: gnnsprapa
8 Replies

10. UNIX for Beginners Questions & Answers

Is there a UNIX command that can compare fields of files with differing number of fields?

Hi, Below are the sample files. x.txt is from an Excel file that is a list of users from Windows and y.txt is a list of database account. $ head -500 x.txt y.txt ==> x.txt <== TEST01 APP_USER_PROFILE USER03 APP_USER_PROFILE TEST02 APP_USER_EXP_PROFILE TEST04 APP_USER_PROFILE USER01 ... (3 Replies)
Discussion started by: newbie_01
3 Replies
DOVEADM-LOG(1)							      Dovecot							    DOVEADM-LOG(1)

NAME
doveadm-log - Locate, test or reopen Dovecot's log files SYNOPSIS
doveadm [-Dv] log errors [-s min_timestamp] doveadm [-Dv] log find [directory] doveadm [-Dv] log reopen doveadm [-Dv] log test DESCRIPTION
The doveadm log commands are used to locate and reopen the log files of dovecot(1). It's also possible to test the configured targets of the *log_path settings. OPTIONS
Global doveadm(1) options: -D Enables verbosity and debug messages. -v Enables verbosity, including progress counter. COMMANDS
log errors doveadm log errors [-s min_timestamp] The log errors command is used to show the last - up to 1,000 - errors and warnings. If no output is generated, no errors have occurred since the last start. -s min_timestamp An integer value, representing seconds since the epoch - also known as Unix timestamp. When a min_timestamp was given, doveadm(1) will only show errors occurred since that point in time. log find doveadm log find [directory] The log find command is used to show the location of the log files, to which dovecot(1) sends its log messages. If dovecot(1) logs its messages through syslogd(8) and doveadm(1) could not find any log files, you can specify the directory where your syslogd writes its log files. log reopen doveadm log reopen This command causes doveadm to reopen all log files, configured in the log_path, info_log_path and debug_log_path settings. These settings are configured in /etc/dovecot/conf.d/10-logging.conf. This is for example useful after manually rotating the log files. log test doveadm log test This command causes doveadm to write the message "This is Dovecot's priority log (timestamp)" to the configured log files. The used prior- ities are: debug, info, warning, error and fatal. EXAMPLE
This example shows how to locate the log files used by dovecot(1). doveadm log find Looking for log files from /var/log Debug: /var/log/dovecot.debug Info: /var/log/mail.log Warning: /var/log/mail.log Error: /var/log/mail.log Fatal: /var/log/mail.log REPORTING BUGS
Report bugs, including doveconf -n output, to the Dovecot Mailing List <dovecot@dovecot.org>. Information about reporting bugs is avail- able at: http://dovecot.org/bugreport.html SEE ALSO
doveadm(1) Dovecot v2.2 2013-11-24 DOVEADM-LOG(1)
All times are GMT -4. The time now is 04:22 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy