Sponsored Content
Top Forums Shell Programming and Scripting Picking up files conditionally Post 302953180 by Saanvi1 on Tuesday 25th of August 2015 04:07:41 PM
Old 08-25-2015
It is one of the ETL servers that loads these files into databases. Each file pertains to different load process. So passing filename in parrallel will load various tables based on filename simultaneously.
Currently it is working fine in production without any issues. But ocassionally now we started receiving multiple files( not high in number) with same name but different dates, hence kicking off same file will execute the same ETL code causing it to fail multiple times. I am trying to avoid a situation of failure and want to keep the parallel execution in place for individual files and for the ones that are more than one file with same name that needs to be sequential load one after the other.

Code:
 
 1.    app-cnd-imp-20150820.txt
 2.    app-cxyzm-imp-20150820.txt
 3.    app-petco-imp-20150820.txt
 4.    app-mobility-imp-20150820.txt
 5.    app-mobility-imp-20150821.txt
 6.    app-mobility-imp-20150822.txt
 7.    app-cellular-imp-20150824.txt


So in the file list above I can run file number 1,2,3,4,7 in one pass of loop and wait for completion and number 5 file in second pass of the loop and wait for completion and number 6 file in third pass of loop as the 4, 5,6 pertains to the same ETL code and will fail the load process.

Thanks

Last edited by Saanvi1; 08-25-2015 at 05:11 PM.. Reason: spelling correction
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Email from script conditionally

I have a script that is run from the Cron 3 times an hour, here is the cron line: 02,22,42 7-18 * * 1-5 /hci/TEST/bin/myscript.ksh TEST 1>/hci/TEST/logs/myscript.info 2>/hci/TEST/logs/myscript I am curious as to whether the time parameters from cron, ( 02, 22, 42 etc) can be accessed from the... (2 Replies)
Discussion started by: dfb500
2 Replies

2. UNIX for Dummies Questions & Answers

Conditionally joining lines in vi

I've done this before but I can't remember how. Too long away from vi. I want to do a search are replace, but I want the replace to be a join. Example see spot run see spot walk see spot run fast see spot hop %s/run$/<somehow perform a join with the next line>/g so the results... (0 Replies)
Discussion started by: ifermon
0 Replies

3. Shell Programming and Scripting

Get min from a column conditionally

hi, i have a file with folowing content: STORAGE PERCENTAGE FLAG: /storage_01 64% 0 /storage_02 17% 1 /storage_03 10% 0 /storage_04 50% 1 I need to get the value of STORAGE from those with FLAG=0 and which has the min PERCENTAGE i am able to get the STORAGE corresponding to... (8 Replies)
Discussion started by: kichu
8 Replies

4. Shell Programming and Scripting

Conditionally delete last X lines

delete last X lines,which start with + example file: test1 test2 remove1 remove2 one liner shell is preferred. (8 Replies)
Discussion started by: honglus
8 Replies

5. Shell Programming and Scripting

conditionally combine text from two files into one

Hi! I'm trying to take multiple text files (6), which have text on some lines but not others, and combine them. I'd also like to make the values in one column of some of the files (files 4-6) negative. I'm trying to write a short script (see below) as I have to do this with a large number of... (2 Replies)
Discussion started by: felix.echidna
2 Replies

6. UNIX for Dummies Questions & Answers

How to conditionally replace a pattern?

Hi, How to replace only the function calls with a new name and skip the function definition and declarations. consider the following code. There are 2 functions defined here returnint and returnvoid. I need to replace returnint with giveint and returnvoid with givevoid only in the function... (1 Reply)
Discussion started by: i.srini89
1 Replies

7. Shell Programming and Scripting

How to conditionally replace a pattern?

Hi, How to replace only the function calls with a new name and skip the function definition and declarations. consider the following code. There are 2 functions defined here returnint and returnvoid. I need to replace returnint with giveint and returnvoid with givevoid only in the function... (2 Replies)
Discussion started by: i.srini89
2 Replies

8. Shell Programming and Scripting

Urgent ...pls Sorting files based on timestamp and picking the latest file

Hi Friends, Newbie to shell scripting. Currently i have used the below to sort data based on filenames and datestamp $ printf '%s\n' *.dat* | sort -t. -k3,4 filename_1.dat.20120430.Z filename_2.dat.20120430.Z filename_3.dat.20120430.Z filename_1.dat.20120501.Z filename_2.dat.20120501.Z... (1 Reply)
Discussion started by: robertbrown624
1 Replies

9. UNIX for Dummies Questions & Answers

Sorting files based on timestamp and picking the latest file

Hi Friends, Newbie to shell scripting Currently i have used the below to sort data based on filenames and datestamp $ printf '%s\n' *.dat* | sort -t. -k3,4 filename_1.dat.20120430.Z filename_2.dat.20120430.Z filename_3.dat.20120430.Z filename_1.dat.20120501.Z filename_2.dat.20120501.Z... (12 Replies)
Discussion started by: robertbrown624
12 Replies

10. Shell Programming and Scripting

Picking up last two specific files, for transfer

Hi I need to transfer files with csv extension from one server to another (all running solaris). But the files to be picked are in a directory that also has: /directory deposit_698.csv deposit_698.checksum deposit_699.csv deposit_699.checksum deposit_700.csv deposit_700.checksum... (15 Replies)
Discussion started by: fretagi
15 Replies
PAPS(1) 						      General Commands Manual							   PAPS(1)

NAME
paps - UTF-8 to PostScript converter using Pango SYNOPSIS
paps [options] files... DESCRIPTION
paps reads a UTF-8 encoded file and generates a PostScript language rendering of the file. The rendering is done by creating outline curves through the pango ft2 backend. OPTIONS
These programs follow the usual GNU command line syntax, with long options starting with two dashes (`-'). A summary of options is included below. --landscape Landscape output. Default is portrait. --columns=cl Number of columns output. Default is 1. --font=desc Set the font description. Default is Monospace 12. --rtl Do rtl layout. --paper ps Choose paper size. Known paper sizes are legal, letter, a4. Default is A4. --bottom-margin=bm Set bottom margin in postscript points (1/72 inch). Default is 36. --top-margin=tm Set top margin. Default is 36. --left-margin=lm Set left margin. Default is 36. --right-margin=rm Set right margin. Default is 36. --help Show summary of options. --header Draw page header for each page. --markup Interpret the text as pango markup. --encoding=ENCODING Assume the documentation encoding is ENCODING. --lpi Set the lines per inch. This determines the line spacing. --cpi Set the characters per inch. This is an alternative method of specifying the font size. --stretch-chars Indicates that characters should be stretched in the y-direction to fill up their vertical space. This is similar to the texttops behaviour. AUTHOR
paps was written by Dov Grobgeld <dov.grobgeld@gmail.com>. This manual page was written by Lior Kaplan <kaplan@debian.org>, for the Debian project (but may be used by others). April 17, 2006 PAPS(1)
All times are GMT -4. The time now is 07:12 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy