03-15-2019
Parallel is not a go-faster button for files. Unless your CPU is maxing out, there's no benefit.
GNU parallel is just doing individual files like you were doing anyway. It has to, lacking magic mechanisms to predict future filesize and move things where they belong.
If your CPU is maxing out,
pigz may work faster one-at-a-time than you were trying to do parallel.
Last edited by Corona688; 03-15-2019 at 05:19 PM..
This User Gave Thanks to Corona688 For This Post:
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
I am searching some rather large text files using grep and or awk.
What I would like to know is if there is a way (either with grep, awk, or realy any other unix tool) to stop the search when a predifined number of results are returned.
I would like to do this for speed purpuses. When i get... (6 Replies)
Discussion started by: evan108
6 Replies
2. UNIX for Advanced & Expert Users
Hello All,
I am maintaining a server and I use pine as MUA and sendmail as MTA. Suddenly many users in the network face the problem of not being able to attach files using pine. I checked the sendmail.cf file and found a variable "MaxMessageSize = 1000000". Eventhough the message size... (2 Replies)
Discussion started by: maybemedic
2 Replies
3. Shell Programming and Scripting
if test.dat is the file
cat test.dat|uuencode test.dat|mailx -s "subject" mailid
can be used for attaching test.dat
how can i attach more than one file to a mail using mailx (2 Replies)
Discussion started by: anumkoshy
2 Replies
4. Shell Programming and Scripting
I am trying to send multiple files as attachment in one email, I tried to search but couldn't find. Please let me know if any solutions. (2 Replies)
Discussion started by: mgirinath
2 Replies
5. Shell Programming and Scripting
Hi all,
I have a list of xml file. I need to split the files to a different files when see the <ko> tag.
The list of filename are
B20090908.1100-20090908.1200_CDMA=1,NO=2,SITE=3.xml
B20090908.1200-20090908.1300_CDMA=1,NO=2,SITE=3.xml
B20090908.1300-20090908.1400_CDMA=1,NO=2,SITE=3.xml
... (3 Replies)
Discussion started by: natalie23
3 Replies
6. Shell Programming and Scripting
Hi,
My script has to send 2 files as a separate attachment(Note : files to be sent without zipping) to the specified email id.
Below code was used but it is not attaching the file as expected instead the file contents are displayed in the body of the email.
Kindly,help with your... (22 Replies)
Discussion started by: meva
22 Replies
7. UNIX for Dummies Questions & Answers
Hi, I have 7 terabytes of tar files, one for every single day since 1980. Inside these tar files are GRIB files, each with 100+ variables. There's 8 GRIBs in each tar, corresponding to different times of the day. I need 6 friggin variables..., and it takes TWO WEEKS TO EXTRACT ALL THE TAR FILES... (3 Replies)
Discussion started by: sammysoil
3 Replies
8. Shell Programming and Scripting
I am trying to display the output of ls and also print the number of characters in EVERY file name. This is what I have so far:
#!/bin/sh
for x in `ls`; do
echo The number of characters in x | wc -m
done
Any help appreciated (1 Reply)
Discussion started by: LinuxNubBrah
1 Replies
9. Shell Programming and Scripting
Hi guys,
I'm struggling with this one, any help is appreciated.
I have File1 with hundreds of unique words, like this:
word1
word2
word3
I want to count each word from file1 in file2 and return how many times each word is found.
I tried something like this:
for i in $(cat file1); do... (13 Replies)
Discussion started by: demmel
13 Replies
10. Shell Programming and Scripting
Hi all,
I have 20 files (file001.txt upto file020.txt) and I want to read them from 3rd line upto end of file (line 1002). But in the final file they should appear to start from line 1.
I need following kind of output in a single file:
Filename Line number 2ndcolumn 4thcolumn
I... (14 Replies)
Discussion started by: bioinfo
14 Replies
wc(1) General Commands Manual wc(1)
NAME
wc - Counts the lines, words, characters, and bytes in a file
SYNOPSIS
wc [-c | -m] [-lw] [file...]
The wc command counts the lines, words, and characters or bytes in a file, or in the standard input if you do not specify any files, and
writes the results to standard output. It also keeps a total count for all named files.
STANDARDS
Interfaces documented on this reference page conform to industry standards as follows:
wc: XCU5.0
Refer to the standards(5) reference page for more information about industry standards and associated tags.
OPTIONS
Counts bytes in the input. Counts lines in the input. Counts characters in the input. Counts words in the input.
OPERANDS
Specifies the pathname of the input file. If this operand is omitted, standard input is used.
DESCRIPTION
A word is defined as a string of characters delimited by white space as defined in the X/Open Base Definitions for XCU4.
The wc command counts lines, words, and bytes by default. Use the appropriate options to limit wc output. Specifying wc without options
is the equivalent of specifying wc -lwc. If any options are specified, only the requested information is output.
The order in which counts appear in the output line is lines, words, bytes. If an option is omitted, then the corresponding field in the
output is omitted. If the -m option is used, then character counts replace byte counts.
When you specify one or more files, wc displays the names of the files along with the counts. If standard input is used, then no file name
is displayed.
EXIT STATUS
The following exit values are returned: Successful completion. An error occurred.
EXAMPLES
To display the number of lines, words, and bytes in the file text, enter: wc text
This results in the following output: 27 185 722 text
The numbers 27, 185, and 722 are the number of lines, words, and bytes, respectively, in the file text. To display only one or two
of the three counts include the appropriate options. For example, the following command displays only line and byte counts: wc -cl
text
27 722 text To count lines, words, and bytes in more than one file, use wc with more than one input file or with a file name pat-
tern. For example, the following command can be issued in a directory containing the files text, text1, and text2: wc -l text*
27 text 112 text1 5 text2 144 total
The numbers 27, 112, and 5 are the numbers of lines in the files text, text1, and text2, respectively, and 144 is the total number
of lines in the three files. The file name is always appended to the output. To obtain a pure number for things like reporting
purposes, pipe all input to the wc command using cat. For example, the following command will report the total count of characters
in all files in a directory. echo There are `cat *.c | wc -c` characters in *.c files
There are 1869 characters in *.c files
ENVIRONMENT VARIABLES
The following environment variables affect the execution of wc: Provides a default value for the internationalization variables that are
unset or null. If LANG is unset or null, the corresponding value from the default locale is used. If any of the internationalization vari-
ables contain an invalid setting, the utility behaves as if none of the variables had been defined. If set to a non-empty string value,
overrides the values of all the other internationalization variables. Determines the locale for the interpretation of sequences of bytes
of text data as characters (for example, single-byte as opposed to multibyte characters in arguments and input files) and which characters
are defined as white space characters. Determines the locale for the format and contents of diagnostic messages written to standard error
and informative messages written to standard output. Determines the location of message catalogues for the processing of LC_MESSAGES.
SEE ALSO
Commands: cksum(1), ls(1)
Standards: standards(5)
wc(1)