Sponsored Content
Top Forums Shell Programming and Scripting Generating millions of record using shell script Post 302379556 by Rahil2k9 on Friday 11th of December 2009 04:28:16 AM
Old 12-11-2009
Generating millions of record using shell script

Hi All,

My requirement is like this.
I want to generate records of 1 million lines. If I say lines it means one line will contain some string or numbers like
Code:
AA,3,4,45,+223424234,Tets,Ghdj,+33434,345453434,........................ upto length lets say 41. ( 41 comma sepearted aplha numneric values ).
Now i want 5, 7 and 23 field should be unique or in some incremental order for ex.  if in first line value is 33 then next line, same fields position value should be 34 and so on. All other fields in all the lines can be same
Now for reference this script should refer one sample file and based of that it will create a file with 1 million lines. In my case following is my reference file

1,10,+91900000000,+9112323230000,,1,60,25/1/200923:10:10,3534671234500001,40591912000000001,1001001,1,1,10,3,1,1,1,0,20,+91900000000,329,ss,in_flag,digits,IC_TRUN,OG_TRUNK,NET_IND,realer_st,switch,source,plmn,1,cug_code,ff_call,corela,service3g,conf,op1,op2,op3

I want 3,4 and 21 field as as unique rest filed can be same.

Please help me out in this as am new to shell scripting.

rd-
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Parsing record into multiple records in Shell Script

Hi, I am trying to parse a very long record in a text file into multiple records by checking ADD, DELETE, or MODIFY field value in a shell script. Input # File name xyz.txt ADD|N000|8015662|DELETE|N001|9915662|MODIFY|N999|85678 Output ADD|N000|8015662| DELETE|N001|9915662|... (8 Replies)
Discussion started by: naveed
8 Replies

2. UNIX for Dummies Questions & Answers

A shell script or software for generating random passwords

Hi, Is there an shell script/batch file to genarate random passwords which expires after a stipulated time period? Please suggest a software which does this for AIX and windows both else. Thanks. (5 Replies)
Discussion started by: dwiravi
5 Replies

3. Shell Programming and Scripting

Need script to remove millions of tmp files in /html/cache/ directory

Hello, I just saw that on my vps (centOS) my oscommerce with a seo script has created millions of tmp files inside the /html/cache/ directory. I would need to remove all those files (millions), I tried via shell but the vps loads goes to very high and it hangs, is there some way to do a... (7 Replies)
Discussion started by: andymc1
7 Replies

4. Shell Programming and Scripting

Extract record in shell script,so far using awk without success

hello, I require help in the following. There is a report similar to the ones shown here. I need to do couple of things 10/07 12:47 0003210042 R TENN ANVISER0 DELF96A.V1.O.TENRREG 120710 124657 U 00000 DELFOR 1 4331 10/07 12:47 0003210043 S TENN... (2 Replies)
Discussion started by: rakeshv
2 Replies

5. Shell Programming and Scripting

Millions of record using shell script

I wanna create a millions of record using shell scripts.. the scenario is somehow like this,,,i want to create the record for these fields in my database..id,name,addrs,sales. So can you please give a overall idea for how to achieve it? (9 Replies)
Discussion started by: shankarpanda003
9 Replies

6. Shell Programming and Scripting

Print record count of a file using shell script

HI, I need to print the record count of a file using shell script. If the record count of a file excluding header and trailer record if greater than zero then print 'Record count of a file is xxxx records'. If the record count is zero print 'zero records' Thanks Mahendra (1 Reply)
Discussion started by: mmeda
1 Replies

7. Shell Programming and Scripting

Shell Script for generating NTIDs(Usernames) from the email ids which exists in MSAD Directory

Hi Everyone, I just need a shell script which automatically gives the list of NT IDs mean the Usernames from the list of email ids. We have around 140 users from AMERICAS,ASIAPACIFIC and EMEA User Directories and we have their email ids.For ex. i have email id called naveen-kumar.dasu@hp.com... (7 Replies)
Discussion started by: naveen.dasu
7 Replies

8. Shell Programming and Scripting

Generating & executing the SELECT and UPDATE clause dynamically using shell script

Hi All, I need to write one shell script. The requirement is as follows. a) I am having list the employee names in a file stored in /tmp directory location as below /tmp/emp.txt b) and the contents are as below cat emp.txt ravi raj ram arun c) I need to connect to sybase... (1 Reply)
Discussion started by: Gowtham_giri
1 Replies

9. Shell Programming and Scripting

Need code for updating second record to first record in shell scripting

Hi,, I have requirement that i need to get DISTINCT values from a table and if there are two records i need to update it to one record and then need to submit INSERT statements by using the updated value as a parameter. Here is the example follows.. SELECT DISTINCT ID FROM OFFER_GROUP WHERE... (1 Reply)
Discussion started by: Samah
1 Replies

10. Shell Programming and Scripting

Is there a way to handle commas inside the data when generating a csv file from shell script?

I am extracting data via sql query and some of the data has commas. Output File must be csv and I cannot update the data in the db (as it is used by other application). Example table FavoriteThings Person VARCHAR2(25), Favorite VARCHAR2(100) Sample Data Greta rain drop on... (12 Replies)
Discussion started by: patk625
12 Replies
UNIQ(1) 						    BSD General Commands Manual 						   UNIQ(1)

NAME
uniq -- report or filter out repeated lines in a file SYNOPSIS
uniq [-cdu] [-f fields] [-s chars] [input_file [output_file]] DESCRIPTION
The uniq utility reads the standard input comparing adjacent lines, and writes a copy of each unique input line to the standard output. The second and succeeding copies of identical adjacent input lines are not written. Repeated lines in the input will not be detected if they are not adjacent, so it may be necessary to sort the files first. The following options are available: -c Precede each output line with the count of the number of times the line occurred in the input, followed by a single space. -d Don't output lines that are not repeated in the input. -f fields Ignore the first fields in each input line when doing comparisons. A field is a string of non-blank characters separated from adja- cent fields by blanks. Field numbers are one based, i.e. the first field is field one. -s chars Ignore the first chars characters in each input line when doing comparisons. If specified in conjunction with the -f option, the first chars characters after the first fields fields will be ignored. Character numbers are one based, i.e. the first character is character one. -u Don't output lines that are repeated in the input. If additional arguments are specified on the command line, the first such argument is used as the name of an input file, the second is used as the name of an output file. The uniq utility exits 0 on success, and >0 if an error occurs. COMPATIBILITY
The historic +number and -number options have been deprecated but are still supported in this implementation. SEE ALSO
sort(1) STANDARDS
The uniq utility is expected to be IEEE Std 1003.2 (``POSIX.2'') compatible. BSD
January 6, 2007 BSD
All times are GMT -4. The time now is 08:41 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy