Sponsored Content
Full Discussion: Add 1 million columns
Top Forums UNIX for Dummies Questions & Answers Add 1 million columns Post 302916213 by RudiC on Monday 8th of September 2014 07:37:36 AM
Old 09-08-2014
Quote:
Originally Posted by SriniShoo
@RudiC code with better performance

Code:
awk 'BEGIN {for(i=1; i<=1000000; i++) t = (t " 0")} {printf "%s%s\n", $0, t}' file1 > file2

Sure?
Code:
time awk 'BEGIN {for(i=1;i<=100000;i++) t=t " 0"} {printf "%s%s\n", $0, t}' file1 | wc -w
1000060

real    0m4.763s
user    0m4.016s
sys    0m0.750s
time awk '{printf "%s", $0; for(i=1;i<=100000;i++) printf " 0"; printf "\n"}' file1 | wc -w
1000060

real    0m0.277s
user    0m0.328s
sys    0m0.004s

and, even faster:
Code:
time awk '{printf "%s", $0; for(i=1;i<=10000;i++) printf " 0 0 0 0 0 0 0 0 0 0"; printf "\n"}' file1 | wc -w 
1000060

real    0m0.092s
user    0m0.137s
sys    0m0.008s

 

8 More Discussions You Might Find Interesting

1. What is on Your Mind?

Pick a Number Between 0 and 20 for 1 Million Bits

Here is an easy game! I wrote a number between 0 and 20 (that can include 0 and 20) on a piece of paper. I am staring at it now, imagining the number so you can read my mind ;) Reply once, and only once, with a number from 0 to 20 and the first person to guess it wins 1,000,000 Bits. ... (24 Replies)
Discussion started by: Neo
24 Replies

2. Shell Programming and Scripting

Single command for add 2 columns and remove 2 columns in unix/performance tuning

Hi all, I have created a script which adding two columns and removing two columns for all files. Filename: Cust_information_1200_201010.txt Source Data: "1","Cust information","123","106001","street","1-203 high street" "1","Cust information","124","105001","street","1-203 high street" ... (0 Replies)
Discussion started by: onesuri
0 Replies

3. Shell Programming and Scripting

Matching 10 Million file records with 10 Million in other file

Dear All, I have two files both containing 10 Million records each separated by comma(csv fmt). One file is input.txt other is status.txt. Input.txt-> contains fields with one unique id field (primary key we can say) Status.txt -> contains two fields only:1. unique id and 2. status ... (8 Replies)
Discussion started by: vguleria
8 Replies

4. Shell Programming and Scripting

Evaluate 2 columns, add sum IF two columns match on two rows

Hi all, I know this sounds suspiciously like a homework course; but, it is not. My goal is to take a file, and match my "ID" column to the "Date" column, if those conditions are true, add the total number of minutes worked and place it in this file, while not printing the original rows that I... (6 Replies)
Discussion started by: mtucker6784
6 Replies

5. UNIX for Dummies Questions & Answers

Deleting a million of files ..

Hi, Which way is faster rm -rf /path/ or find / -name -exec rm {} \; and why? (7 Replies)
Discussion started by: cain82
7 Replies

6. Shell Programming and Scripting

Evaluate 2 columns, add sum IF two columns satisfy the condition

HI All, I'm embedding SQL query in Script which gives following output: Assignee Group Total ABC Group1 17 PQR Group2 5 PQR Group3 6 XYZ Group1 10 XYZ Group3 5 I have saved the above output in a file. How do i sum up the contents of this output so as to get following output: ... (4 Replies)
Discussion started by: Khushbu
4 Replies

7. Shell Programming and Scripting

Add new columns based on existing columns

Hi all, I am kind of stuck with printing my desired output. Please help me if you know how it can work. My input file(tab separated): NW_0068.1 41,16 100,900 NW_0699.1 4,2,19 200,700,80 My Output file (desired): NW_0068.1 41,16 100,900 100 - 141 NW_0068.1 41,16 100,900 ... (3 Replies)
Discussion started by: sam_2921
3 Replies

8. UNIX for Beginners Questions & Answers

Group by columns and add sum in new columns

Dear Experts, I have input file which is comma separated, has 4 columns like below, BRAND,COUNTRY,MODEL,COUNT NIKE,USA,DUMMY,5 NIKE,USA,ORIGINAL,10 PUMA,FRANCE,DUMMY,20 PUMA,FRANCE,ORIGINAL,15 ADIDAS,ITALY,DUMMY,50 ADIDAS,ITALY,ORIGINAL,50 SPIKE,CHINA,DUMMY,1O And expected output add... (2 Replies)
Discussion started by: ricky1991
2 Replies
BEGIN(7)						  PostgreSQL 9.2.7 Documentation						  BEGIN(7)

NAME
BEGIN - start a transaction block SYNOPSIS
BEGIN [ WORK | TRANSACTION ] [ transaction_mode [, ...] ] where transaction_mode is one of: ISOLATION LEVEL { SERIALIZABLE | REPEATABLE READ | READ COMMITTED | READ UNCOMMITTED } READ WRITE | READ ONLY [ NOT ] DEFERRABLE DESCRIPTION
BEGIN initiates a transaction block, that is, all statements after a BEGIN command will be executed in a single transaction until an explicit COMMIT(7) or ROLLBACK(7) is given. By default (without BEGIN), PostgreSQL executes transactions in "autocommit" mode, that is, each statement is executed in its own transaction and a commit is implicitly performed at the end of the statement (if execution was successful, otherwise a rollback is done). Statements are executed more quickly in a transaction block, because transaction start/commit requires significant CPU and disk activity. Execution of multiple statements inside a transaction is also useful to ensure consistency when making several related changes: other sessions will be unable to see the intermediate states wherein not all the related updates have been done. If the isolation level, read/write mode, or deferrable mode is specified, the new transaction has those characteristics, as if SET TRANSACTION (SET_TRANSACTION(7)) was executed. PARAMETERS
WORK, TRANSACTION Optional key words. They have no effect. Refer to SET TRANSACTION (SET_TRANSACTION(7)) for information on the meaning of the other parameters to this statement. NOTES
START TRANSACTION (START_TRANSACTION(7)) has the same functionality as BEGIN. Use COMMIT(7) or ROLLBACK(7) to terminate a transaction block. Issuing BEGIN when already inside a transaction block will provoke a warning message. The state of the transaction is not affected. To nest transactions within a transaction block, use savepoints (see SAVEPOINT(7)). For reasons of backwards compatibility, the commas between successive transaction_modes can be omitted. EXAMPLES
To begin a transaction block: BEGIN; COMPATIBILITY
BEGIN is a PostgreSQL language extension. It is equivalent to the SQL-standard command START TRANSACTION (START_TRANSACTION(7)), whose reference page contains additional compatibility information. The DEFERRABLE transaction_mode is a PostgreSQL language extension. Incidentally, the BEGIN key word is used for a different purpose in embedded SQL. You are advised to be careful about the transaction semantics when porting database applications. SEE ALSO
COMMIT(7), ROLLBACK(7), START TRANSACTION (START_TRANSACTION(7)), SAVEPOINT(7) PostgreSQL 9.2.7 2014-02-17 BEGIN(7)
All times are GMT -4. The time now is 06:48 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy