I like implicit restartable processes -- no code to mess up when first it is stressed!
Move the input files to another dir when they are done, or rename them (mv)? This is cheap and easy. Putting status and time stamps on the file name is a relatively reliable way to signal. Even if ftp'ing files, ftp to one name and if it is OK, do a rename to the final name, which tells the other end that, by its very existence, it is ok to use. Using dirs means there is less conflict over the locks on the directory inode, and everything in there says work needs to be done! As long as a mv does not change volumes/mounts/devices, it is cheap, just two writes to dir inodes, at least on most UNIX file systems and WinDos.
---------- Post updated at 04:41 PM ---------- Previous update was at 04:22 PM ----------
On a more cooperative vein, find all the files of each type without conflicting path, use sed to remove the extension differences, sort them, and use "comm -13 in_file_list_file out_file_list_file" to tell you what is undone. With Solaris you can pipe it all:
Thanks DGP but I'm looking for what's wrong with the syntax to the sed command (I'm pretty certain the logic is OK) else I'll end up having to debug other code.
If someone could point out what I seem to be missing I learn from my mistake for next time.
Agree on using $(), definitely easier to spot command substituions that way and if you're missing the closing end.
But replace this:
with this:
When the literal '/' is a part of your address pattern, you can change the delimeters by using any other character as long as the first one is preceded with '\'
Last edited by rwuerth; 10-07-2010 at 06:21 PM..
Reason: grammar/spelling fixes
I'd forgotten that you need the backslash in front of the substitution, mainly because I usually do just substitution where the syntax structure is different.
Hi, In my previous post ( How to print lines from a files with specific start and end patterns and pick only the last lines? ), i have got a help to get the last select statement from a file, now i need to remove/exclude the output from main file:
Input File format:
SELECT
ABCD,
DEFGH,... (2 Replies)
I wish to search and delete all lines in /app/Jenkins/deploy.txt having this filename string /app/Jenkins/file2.mrt as entry:
I'm using : colon as delimiter in sed command as I'm dealing with file paths.
Below is the command I was expecting to work.
sed -i ":/app/Jenkins/file2.mrt:d"... (1 Reply)
There are many matching blocks of text in one file that need to be deleted. This example below is one block that needs to be either deleted or replaced with an empty line.
This text below is the input file. The ouput file should be empty
Searching Checks. Based on search criteria
name: Value :... (2 Replies)
Hi,
I have a file called abc.txt with the following dates
2016-01-27
2016-01-28
2016-01-29
2016-01-30
2016-01-31
2016-02-01
2016-02-02
2016-02-03
I would like to print all lines below if 2016-01-31 is found, excluding that date.
I use this command --> sed '1,/2016-01-31/d' abc.txt
If... (4 Replies)
I have a very large file (over 700 million lines) that has some lines that I need to delete. An example of 5 lines of the file:
HS4_80:8:2303:19153:193032 153 k80:138891
HS4_80:8:2105:5544:43174 89 k88:81949
165 k88:81949 323 0 * = 323 0 ... (6 Replies)
Hi
Im trying to do the following in sed. I want to delete any blank line at the start of a file until it matches a pattern and then stops. for example:
Input
output:
I have got it to work within a range of two patterns with the following:
sed '/1/,/pattern/{/^]*$/d}'
The... (2 Replies)
Sample file:
This is line one,
this is another line,
this is the PRIMARY INDEX line
l ;
This is another line
The command should find the line with “PRIMARY INDEX” and remove the last character from the line preceding it (in this case , comma) and remove the first character from the line... (5 Replies)
I have a very large results file, and a list of filters
grep -wf filterlist.txt datafile.txt > outfile.txt
The above line works but is very slow and I'm wondering how to make it faster.
The items in my filterlist are only relevant to the first column. I don't care if any item in the... (1 Reply)
hi all
i want to delete a line upto a particular character. here is example.
cp cms/images/wifi-zone.png
i want to delete the line till . (cp cms/images/wifi-zone.) so the output wud be "png" only
how can i do it?
also please note down that dot (.) can also occur multiple... (12 Replies)
I have a text file, a sample of which is as follows:
r/- * 0: WINDOWS/Microsoft.NET/Framework/v2.0.50727/ASP.NETWebAdminFiles/Images/headerGRADIENT_Tall.gif
r/- * 0: WINDOWS/SoftwareDistribution/Download/cf8ec753e88561d2ddb53e183dc05c3e/backoff.jpg
r/- * 0: ... (2 Replies)