Sponsored Content
Top Forums Shell Programming and Scripting Running rename command on large files and make it faster Post 302549413 by shoaibjameel123 on Tuesday 23rd of August 2011 03:33:38 AM
Old 08-23-2011
Running rename command on large files and make it faster

Hi All,

I have some 80,000 files in a directory which I need to rename. Below is the command which I am currently running and it seems, it is taking fore ever to run this command. This command seems too slow. Is there any way to speed up the command. I have have GNU Parallel installed on my system but I am not sure how to run this command using GNU Parallel.

What this script is doing is to rename all .xml.dat files to .xml

Code:
for f in *.xml.dat; do f2=`basename $f .dat`; echo "$f -> $f2"; mv $f $f2; done

I am using Linux with BASH.
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Error when running the make command

Hi, Not really sure whether this question should go to this forum but am giving it a shot. I have compiled a simple C program test.c. #include <stdio.h> #include <stdlib.h> #include <string.h> static int a; int test() { a=a+1; return a; } When I run a make command, I get this: ... (2 Replies)
Discussion started by: nattynatty
2 Replies

2. Shell Programming and Scripting

Can anyone make this script run faster?

One of our servers runs Solaris 8 and does not have "ls -lh" as a valid command. I wrote the following script to make the ls output easier to read and emulate "ls -lh" functionality. The script works, but it is slow when executed on a directory that contains a large number of files. Can anyone make... (10 Replies)
Discussion started by: shew01
10 Replies

3. Red Hat

Re:How to make the linux pc faster

Hi, Can any one help me out in solving the problem i have a linux database server it is tooo slow that i am unable to open even the terminial is there any solution to get rid of this problem.How to make this server faster. Thanks & Regards Venky (0 Replies)
Discussion started by: venky_vemuri
0 Replies

4. Emergency UNIX and Linux Support

Help to make awk script more efficient for large files

Hello, Error awk: Internal software error in the tostring function on TS1101?05044400?.0085498227?0?.0011041461?.0034752266?.00397045?0?0?0?0?0?0?11/02/10?09/23/10???10?no??0??no?sct_det3_10_20110516_143936.txt What it is It is a unix shell script that contains an awk program as well as... (4 Replies)
Discussion started by: script_op2a
4 Replies

5. Shell Programming and Scripting

Make script faster

Hi all, In bash scripting, I use to read files: cat $file | while read line; do ... doneHowever, it's a very slow way to read file line by line. E.g. In a file that has 3 columns, and less than 400 rows, like this: I run next script: cat $line | while read line; do ## Reads each... (10 Replies)
Discussion started by: AlbertGM
10 Replies

6. UNIX for Dummies Questions & Answers

Rename a large number of files in subdirectories

Hi, I have a large number of subdirectories (>200), and in each of these directories there is a file with a name like "opp1234.dat". I'd like to know how I could change the names of these files to say "out.dat" in all these subdirectories in one go. Thanks! (5 Replies)
Discussion started by: lost.identity
5 Replies

7. Shell Programming and Scripting

Faster command to remove headers for files in a directory

Good evening Im new at unix shell scripting and im planning to script a shell that removes headers for about 120 files in a directory and each file contains about 200000 lines in average. i know i will loop files to process each one and ive found in this great forum different solutions... (5 Replies)
Discussion started by: alexcol
5 Replies

8. Shell Programming and Scripting

awk changes to make it faster

I have script like below, who is picking number from one file and and searching in another file, and printing output. Bu is is very slow to be run on huge file.can we modify it with awk #! /bin/ksh while read line1 do echo "$line1" a=`echo $line1` if then echo "$num" cat file1|nawk... (6 Replies)
Discussion started by: mirwasim
6 Replies

9. Shell Programming and Scripting

How to make awk command faster?

I have the below command which is referring a large file and it is taking 3 hours to run. Can something be done to make this command faster. awk -F ',' '{OFS=","}{ if ($13 == "9999") print $1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12 }' ${NLAP_TEMP}/hist1.out|sort -T ${NLAP_TEMP} |uniq>... (13 Replies)
Discussion started by: Peu Mukherjee
13 Replies

10. Shell Programming and Scripting

How to make awk command faster for large amount of data?

I have nginx web server logs with all requests that were made and I'm filtering them by date and time. Each line has the following structure: 127.0.0.1 - xyz.com GET 123.ts HTTP/1.1 (200) 0.000 s 3182 CoreMedia/1.0.0.15F79 (iPhone; U; CPU OS 11_4 like Mac OS X; pt_br) These text files are... (21 Replies)
Discussion started by: brenoasrm
21 Replies
IO::AtomicFile(3)					User Contributed Perl Documentation					 IO::AtomicFile(3)

NAME
IO::AtomicFile - write a file which is updated atomically SYNOPSIS
use IO::AtomicFile; ### Write a temp file, and have it install itself when closed: my $FH = IO::AtomicFile->open("bar.dat", "w"); print $FH "Hello! "; $FH->close || die "couldn't install atomic file: $!"; ### Write a temp file, but delete it before it gets installed: my $FH = IO::AtomicFile->open("bar.dat", "w"); print $FH "Hello! "; $FH->delete; ### Write a temp file, but neither install it nor delete it: my $FH = IO::AtomicFile->open("bar.dat", "w"); print $FH "Hello! "; $FH->detach; DESCRIPTION
This module is intended for people who need to update files reliably in the face of unexpected program termination. For example, you generally don't want to be halfway in the middle of writing /etc/passwd and have your program terminate! Even the act of writing a single scalar to a filehandle is not atomic. But this module gives you true atomic updates, via rename(). When you open a file /foo/bar.dat via this module, you are actually opening a temporary file /foo/bar.dat..TMP, and writing your output there. The act of closing this file (either explicitly via close(), or implicitly via the destruction of the object) will cause rename() to be called... therefore, from the point of view of the outside world, the file's contents are updated in a single time quantum. To ensure that problems do not go undetected, the "close" method done by the destructor will raise a fatal exception if the rename() fails. The explicit close() just returns undef. You can also decide at any point to trash the file you've been building. AUTHOR
Primary Maintainer David F. Skoll (dfs@roaringpenguin.com). Original Author Eryq (eryq@zeegee.com). President, ZeeGee Software Inc (http://www.zeegee.com). REVISION
$Revision: 1.2 $ perl v5.12.1 2005-02-10 IO::AtomicFile(3)
All times are GMT -4. The time now is 08:41 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy