Sponsored Content
Full Discussion: Add 1 million columns
Top Forums UNIX for Dummies Questions & Answers Add 1 million columns Post 302916213 by RudiC on Monday 8th of September 2014 07:37:36 AM
Old 09-08-2014
Quote:
Originally Posted by SriniShoo
@RudiC code with better performance

Code:
awk 'BEGIN {for(i=1; i<=1000000; i++) t = (t " 0")} {printf "%s%s\n", $0, t}' file1 > file2

Sure?
Code:
time awk 'BEGIN {for(i=1;i<=100000;i++) t=t " 0"} {printf "%s%s\n", $0, t}' file1 | wc -w
1000060

real    0m4.763s
user    0m4.016s
sys    0m0.750s
time awk '{printf "%s", $0; for(i=1;i<=100000;i++) printf " 0"; printf "\n"}' file1 | wc -w
1000060

real    0m0.277s
user    0m0.328s
sys    0m0.004s

and, even faster:
Code:
time awk '{printf "%s", $0; for(i=1;i<=10000;i++) printf " 0 0 0 0 0 0 0 0 0 0"; printf "\n"}' file1 | wc -w 
1000060

real    0m0.092s
user    0m0.137s
sys    0m0.008s

 

8 More Discussions You Might Find Interesting

1. What is on Your Mind?

Pick a Number Between 0 and 20 for 1 Million Bits

Here is an easy game! I wrote a number between 0 and 20 (that can include 0 and 20) on a piece of paper. I am staring at it now, imagining the number so you can read my mind ;) Reply once, and only once, with a number from 0 to 20 and the first person to guess it wins 1,000,000 Bits. ... (24 Replies)
Discussion started by: Neo
24 Replies

2. Shell Programming and Scripting

Single command for add 2 columns and remove 2 columns in unix/performance tuning

Hi all, I have created a script which adding two columns and removing two columns for all files. Filename: Cust_information_1200_201010.txt Source Data: "1","Cust information","123","106001","street","1-203 high street" "1","Cust information","124","105001","street","1-203 high street" ... (0 Replies)
Discussion started by: onesuri
0 Replies

3. Shell Programming and Scripting

Matching 10 Million file records with 10 Million in other file

Dear All, I have two files both containing 10 Million records each separated by comma(csv fmt). One file is input.txt other is status.txt. Input.txt-> contains fields with one unique id field (primary key we can say) Status.txt -> contains two fields only:1. unique id and 2. status ... (8 Replies)
Discussion started by: vguleria
8 Replies

4. Shell Programming and Scripting

Evaluate 2 columns, add sum IF two columns match on two rows

Hi all, I know this sounds suspiciously like a homework course; but, it is not. My goal is to take a file, and match my "ID" column to the "Date" column, if those conditions are true, add the total number of minutes worked and place it in this file, while not printing the original rows that I... (6 Replies)
Discussion started by: mtucker6784
6 Replies

5. UNIX for Dummies Questions & Answers

Deleting a million of files ..

Hi, Which way is faster rm -rf /path/ or find / -name -exec rm {} \; and why? (7 Replies)
Discussion started by: cain82
7 Replies

6. Shell Programming and Scripting

Evaluate 2 columns, add sum IF two columns satisfy the condition

HI All, I'm embedding SQL query in Script which gives following output: Assignee Group Total ABC Group1 17 PQR Group2 5 PQR Group3 6 XYZ Group1 10 XYZ Group3 5 I have saved the above output in a file. How do i sum up the contents of this output so as to get following output: ... (4 Replies)
Discussion started by: Khushbu
4 Replies

7. Shell Programming and Scripting

Add new columns based on existing columns

Hi all, I am kind of stuck with printing my desired output. Please help me if you know how it can work. My input file(tab separated): NW_0068.1 41,16 100,900 NW_0699.1 4,2,19 200,700,80 My Output file (desired): NW_0068.1 41,16 100,900 100 - 141 NW_0068.1 41,16 100,900 ... (3 Replies)
Discussion started by: sam_2921
3 Replies

8. UNIX for Beginners Questions & Answers

Group by columns and add sum in new columns

Dear Experts, I have input file which is comma separated, has 4 columns like below, BRAND,COUNTRY,MODEL,COUNT NIKE,USA,DUMMY,5 NIKE,USA,ORIGINAL,10 PUMA,FRANCE,DUMMY,20 PUMA,FRANCE,ORIGINAL,15 ADIDAS,ITALY,DUMMY,50 ADIDAS,ITALY,ORIGINAL,50 SPIKE,CHINA,DUMMY,1O And expected output add... (2 Replies)
Discussion started by: ricky1991
2 Replies
English(3pm)						 Perl Programmers Reference Guide					      English(3pm)

NAME
English - use nice English (or awk) names for ugly punctuation variables SYNOPSIS
use English; use English qw( -no_match_vars ) ; # Avoids regex performance penalty # in perl 5.16 and earlier ... if ($ERRNO =~ /denied/) { ... } DESCRIPTION
This module provides aliases for the built-in variables whose names no one seems to like to read. Variables with side-effects which get triggered just by accessing them (like $0) will still be affected. For those variables that have an awk version, both long and short English alternatives are provided. For example, the $/ variable can be referred to either $RS or $INPUT_RECORD_SEPARATOR if you are using the English module. See perlvar for a complete list of these. PERFORMANCE
NOTE: This was fixed in perl 5.20. Mentioning these three variables no longer makes a speed difference. This section still applies if your code is to run on perl 5.18 or earlier. This module can provoke sizeable inefficiencies for regular expressions, due to unfortunate implementation details. If performance matters in your application and you don't need $PREMATCH, $MATCH, or $POSTMATCH, try doing use English qw( -no_match_vars ) ; . It is especially important to do this in modules to avoid penalizing all applications which use them. perl v5.18.2 2014-01-06 English(3pm)
All times are GMT -4. The time now is 04:31 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy