08-05-2009
[Solved] Find Specific records from file and add totals into variables
Hi Eveyone,
I am working on one shell script to find the specific records from data file and add the totals into variables and print them. you can find the sample data file below for more clarification.
Sample Data File:
PXSTYL00__20090803USA
CHCART00__20090803IND
SSTRAN00__20090803USA
Records Sent: 100 Sum1: 10 20090803 1234567890 1234567890 123456789 123456789 123456789
Records Sent: 200 Sum1: 20 20090803 1234567890 1234567890 123456789 123456789 123456789
Records Sent: 300 Sum1: 30 20090803 1234567890 1234567890 123456789 123456789 123456789
In the above sample data file, I want to read all Records with word Records Sent and then get the totals and sums for all records together.
Out Put Should be like this
Total Records Sent: 100+200+300 = 600
Total Sum : 10+20+30 = 60
I tried the following code:
total = 0
sum = 0
grep "Records Sent:" filename > filename.tmp
cat filename.tmp |\
while read line
do
tot = `$line | cut -f3 -d " "`
su = `$line | cut -f5 -d " "`
total ='expr $total + $total`
sum =`expr $sum +su`
done
echo $total
echo $sum
I am having some issues while running this code. Those issues are
when i am using while read line so it will try to goby line by line but basically it should go by record by record because the Record which has Records Sent: will be 2 lines.
I had another error saying cannot create. I have no idea why this error is coming.
Can anybody write above script in different way.Please advise me on this
Best Regards
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I have a file which is 5 million records. And each records has 412 fields has delimited by "|". So that makes each records to be 2923 bytes long. I wanted to extract specific records like top 100 or 2500 - 5000, 50001 - 10000 etc. from this file.
I tried using head command for top 100 records,... (1 Reply)
Discussion started by: acheepi
1 Replies
2. Shell Programming and Scripting
I am trying to cut the first 10 characters from a file only if the file has 'xyz' in field 185-188.
I tried this
cat filename | cut -c1-10
but this gives me all the records regardless of what is in field 185-188.
Is this doable ?
Thanks in advance for responses. (2 Replies)
Discussion started by: jxh461
2 Replies
3. UNIX for Dummies Questions & Answers
Hi,
I am looking for a method to get column13 to column 50 data from the 1st line of a multiline reord. The records are stored in a large file and are separated by newline. sample format is
(data in red is to be extracted)
<header>
A001dfhskhfkdsh hajfhksdhfjh... (3 Replies)
Discussion started by: sunayana3112
3 Replies
4. UNIX for Dummies Questions & Answers
Hi everyone.
I am a newbie to Linux stuff. I have this kind of problem which couldn't solve alone. I have a text file with records separated by empty lines like this:
ID: 20
Name: X
Age: 19
ID: 21
Name: Z
ID: 22
Email: xxx@yahoo.com
Name: Y
Age: 19
I want to grep records that... (4 Replies)
Discussion started by: Atrisa
4 Replies
5. Shell Programming and Scripting
<Update>
I have the solution:
sed 's/\{3\}/&;&;---;4/'
The thread can be marked as solved!
</Update>
Hi There,
I'm working on a script processing some data from a website into cvs format. There is only one final problem left I can't find a solution.
I've processed my file... (0 Replies)
Discussion started by: lolworlds
0 Replies
6. Shell Programming and Scripting
Hi,
I look for a awk one liner for below issue.
input file
ABC 1234 abc 12345
ABC 4567 678 XYZ
xyz ght 678
ABC 787 yyuu
ABC 789 7890 777
zxr hyip hyu
mno uii 678 776
ABC ty7 888
All lines should be started with ABC as first field. If a record has another value for 1st... (7 Replies)
Discussion started by: ratheesh2011
7 Replies
7. Shell Programming and Scripting
Dear all, today I'm scratching my head with a simple (I believe) issue.
Working with date is quite simple, so if I Need to add some seconds to current time, I'll use:
date --date='+30 seconds' +"%Y-%m-%d %H:%M:%S"But, how to pass the value to add from a variable? I tried the following without... (2 Replies)
Discussion started by: Lord Spectre
2 Replies
8. Shell Programming and Scripting
I have a file data_1.out which contains:
1|abc mail|mail subject|mail body
2|def mail|mail subject|def mail body
I am trying to read the variables from data_1.out and use them to print to 2 different files based on the id (first_column)
The problem is I am not able to read the file... (8 Replies)
Discussion started by: sol_nov
8 Replies
9. Shell Programming and Scripting
Hi All,
I have the below text file from which I have to cut particular section starting from PTR_Security_Rpeorting.cpf to PTR_Security_Reporting_Env93_export.
Report Model............: "D:\Cognos_Publishing\tmp.a2R94KLQec"\PTR_Security_Reporting.cpf
Report Output Script....:... (4 Replies)
Discussion started by: Vikram_Tanwar12
4 Replies
10. UNIX for Beginners Questions & Answers
Hi ,
I have a requirement to read a file ( 5 fields , ~ delimited) and find the records which contain anything other than Alphabets, Numbers , comma ,space and dot . ie a-z and A-Z and 0-9 and . and " " and , in 2nd field. Once I do that i would want the result to have field1|<flag>
flag can... (2 Replies)
Discussion started by: ashwin3086
2 Replies
LEARN ABOUT PHP
mongogridfs
MONGOGRIDFS(3) 1 MONGOGRIDFS(3)
The MongoGridFS class
INTRODUCTION
Utilities for storing and retrieving files from the database.
GridFS is a storage specification all supported drivers implement. Basically, it defines two collections: files, for file metadata, and
chunks, for file content. If the file is large, it will automatically be split into smaller chunks and each chunk will be saved as a docu-
ment in the chunks collection.
Each document in the files collection contains the filename, upload date, and md5 hash. It also contains a unique _id field, which can be
used to query the chunks collection for the file's content. Each document in the chunks collection contains a chunk of binary data, a
files_id field that matches its file's _id, and the position of this chunk in the overall file.
For example, the files document is something like:
<?php
array("_id" => 123456789, "filename" => "foo.txt", "chunkSize" => 3, "length" => 12);
?>
<?php
array("files_id" => 123456789, "n" => 0, "data" => new MongoBinData("abc"));
array("files_id" => 123456789, "n" => 1, "data" => new MongoBinData("def"));
array("files_id" => 123456789, "n" => 2, "data" => new MongoBinData("ghi"));
array("files_id" => 123456789, "n" => 3, "data" => new MongoBinData("jkl"));
?>
INTER-LANGUAGE COMPATIBILITY
You should be able to use any files created by MongoGridFS with any other drivers, and vice versa. However, some drivers expect that all
metadata associated with a file be in a "metadata" field. If you're going to be using other languages, it's a good idea to wrap info you
might want them to see in a "metadata" field. For example, instead of:
<?php
$grid->storeFile("somefile.txt", array("date" => new MongoDate()));
?>
use something like:
<?php
$grid->storeFile("somefile.txt", array("metadata" => array("date" => new MongoDate())));
?>
THE MongoGridFS" FAMILY"
MongoGridFS represents the files and chunks collections. MongoGridFS extends MongoCollection, and an instance of MongoGridFS has access to
all of MongoCollection methods, which act on the files collection:
<?php
$grid = $db->getGridFS();
$grid->update(array("filename" => "foo"), $newObj); // update on the files collection
?>
Another example of manipulating metadata:
<?php
// save a file
$id = $grid->storeFile("game.tgz");
$game = $grid->findOne();
// add a downloads counter
$game->file['downloads'] = 0;
$grid->save($game->file);
// increment the counter
$grid->update(array("_id" => $id), array('$inc' => array("downloads" => 1)));
?>
You can also access the chunks collection from an instance of MongoGridFS:
<?php
$chunks = $grid->chunks; // $chunks is a normal MongoCollection
$chunks->insert(array("x" => 4));
?>
There are some methods for MongoGridFS with the same name as MongoCollection methods, that behave slightly differently. For example, Mon-
goGridFS.remove(3) will remove any objects that match the criteria from the files collection and their content from the chunks collection.
To store something new in GridFS, there are a couple options. If you have a filename, you can say:
<?php
$grid->storeFile($filename, array("whatever" => "metadata", "you" => "want"));
?>
If you have a string of bytes that isn't a file, you can also store that using MongoGridFS.storeBytes(3):
<?php
$grid->storeBytes($bytes, array("whatever" => "metadata", "you" => "want"));
?>
Querying a MongoGridFS collection returns a MongoGridFSCursor, which behaves like a normal MongoCursor except that it returns MongoGridFS-
Files instead of associative arrays.
MongoGridFSFiles can be written back to disc using MongoGridFSFile.write(3) or retrieved in memory using MongoGridFSFile.getBytes(3). There
is currently no method that automatically streams chunks, but it would be fairly easy to write by querying the $grid->chunks collection.
MongoGridFSFile objects contain a field file which contains any file metadata.
CLASS SYNOPSIS
MongoGridFS
extends
MongoCollection
Fields
o public MongoCollection$chunks NULL
o protected string$filesName NULL
o protected string$chunksName NULL
Methods
o public MongoGridFS::__construct (MongoDB $db, [string $prefix = "fs"], [mixed $chunks = "fs"])
o public bool MongoGridFS::delete (mixed $id)
o public array MongoGridFS::drop (void )
o public MongoGridFSCursor MongoGridFS::find ([array $query = array()], [array $fields = array()])
o public MongoGridFSFile MongoGridFS::findOne ([mixed $query = array()], [mixed $fields = array()])
o public MongoGridFSFile MongoGridFS::get (mixed $id)
o public mixed MongoGridFS::put (string $filename, [array $metadata = array()])
o public bool MongoGridFS::remove ([array $criteria = array()], [array $options = array()])
o public mixed MongoGridFS::storeBytes (string $bytes, [array $metadata = array()], [array $options = array()])
o public mixed MongoGridFS::storeFile (string $filename, [array $metadata = array()], [array $options = array()])
o public mixed MongoGridFS::storeUpload (string $name, [array $metadata])
SEE ALSO
oMongoDB core docs on GridFS
oLightCube Solutions blog post on saving user uploads
oLightCube Solutions blog post on adding metadata to files
PHP Documentation Group MONGOGRIDFS(3)