Sponsored Content
Full Discussion: AWS S3 with Bash Script
Top Forums Shell Programming and Scripting AWS S3 with Bash Script Post 302936907 by RavinderSingh13 on Monday 2nd of March 2015 01:05:49 AM
Old 03-02-2015
Quote:
Originally Posted by krux_rap
Hello All,

I am trying few stuff on Amazon S3 move files to EC2 (Ubuntu Server)

I am receiving daily 2 files in S# bucket

Code:
XYZ_2015-02-26_ 200.csv
ABC_2015-02-26_ 200.csv

I want to move daily received files to local EC2(Ubuntu server) and rename files to
Code:
XYZ_2015-02-26.csv
ABC_2015-02-26.csv

Instead of Renaming files after moving to EC2 , if possible i want to rename files on S3 itself.

and send an email notification with files received or not on daily basis

Is this possible using Bash? If yes, please guide.

Thanks
Krux
Hello Krux_rap,

Could you please try following.
Code:
ls -lhtr *200.csv | awk '{A=$(NF-1);gsub(/\_/,X,$(NF-1));print "mv " A $NF OFS $(NF-1)".csv"}'

It will print only the move command with new file and old file names, if happy with results you can try following.
Code:
ls -lhtr *200.csv | awk '{A=$(NF-1);gsub(/\_/,X,$(NF-1));print "mv " A $NF OFS $(NF-1)".csv"}' | sh

Thanks,
R. Singh
 

4 More Discussions You Might Find Interesting

1. Virtualization and Cloud Computing

Computing in the Clouds with AWS

Tim Bass 07-25-2008 02:34 AM The admin*team at The UNIX Forums*have been considering moving the UNIX and*Linux*Forums to the clouds - the Amazon Web Services (AWS) cloud.* Amazon EC2 is one option to scale the forums, which is a*LAMP application.* Amazon EC2 allows*us to rent dedicated... (3 Replies)
Discussion started by: Linux Bot
3 Replies

2. Virtualization and Cloud Computing

How to grow an EBS Array at AWS

This process was collected by trial and error (mostly error) since most of the documents I found on the 'net were incomplete. The cookbook/walkthrough on how to grow an Array on EBS at AWS... # walkthrough by John Jones (jjones at cirrhus9 dot com) umount /dev/md0 mdadm --stop /dev/md0... (0 Replies)
Discussion started by: Habitual
0 Replies

3. Shell Programming and Scripting

AWS Cloud EBS limit script!

Hi, I am trying to correct the following script to get total EBS usage from Amazon cloud. Can someone please help in correcting it. It appears that its not working properly. t=0; for i in `ec2-describe-volumes | grep VOLUME | grep standard | awk '{print $3}'`; do t=`expr $t + $i`; done;... (2 Replies)
Discussion started by: amar0000
2 Replies

4. Shell Programming and Scripting

How to block first bash script until second bash script script launches web server/site?

I'm new to utilities like socat and netcat and I'm not clear if they will do what I need. I have a "compileDeployStartWebServer.sh" script and a "StartBrowser.sh" script that are started by emacs/elisp at the same time in two different processes. I'm using Cygwin bash on Windows 10. My... (3 Replies)
Discussion started by: siegfried
3 Replies
Amazon::SQS::Simple::Queue(3pm) 			User Contributed Perl Documentation			   Amazon::SQS::Simple::Queue(3pm)

NAME
Amazon::SQS::Simple::Queue - OO API for representing queues from the Amazon Simple Queue Service. SYNOPSIS
use Amazon::SQS::Simple; my $access_key = 'foo'; # Your AWS Access Key ID my $secret_key = 'bar'; # Your AWS Secret Key my $sqs = new Amazon::SQS::Simple($access_key, $secret_key); my $q = $sqs->CreateQueue('queue_name'); $q->SendMessage('Hello world!'); my $msg = $q->ReceiveMessage(); print $msg->MessageBody() # Hello world! $q->DeleteMessage($msg->MessageId()); INTRODUCTION
Don't instantiate this class directly. Objects of this class are returned by various methods in "Amazon::SQS::Simple". See Amazon::SQS::Simple for more details. METHODS
Endpoint() Get the endpoint for the queue. Delete([%opts]) Deletes the queue. Any messages contained in the queue will be lost. SendMessage($message, [%opts]) Sends the message. The message can be up to 8KB in size and should be plain text. ReceiveMessage([%opts]) Get the next message from the queue. Returns an "Amazon::SQS::Simple::Message" object. See Amazon::SQS::Simple::Message for more details. If MaxNumberOfMessages is greater than 1, the method returns an array of "Amazon::SQS::Simple::Message" objects. Options for ReceiveMessage: o MaxNumberOfMessages => NUMBER Maximum number of messages to return. Value should be an integer between 1 and 10 inclusive. Default is 1. DeleteMessage($receipt_handle, [%opts]) Delete the message with the specified receipt handle from the queue ChangeMessageVisibility($receipt_handle, $timeout, [%opts]) NOT SUPPORTED IN APIs EARLIER THAN 2009-01-01 Changes the visibility of the message with the specified receipt handle to $timeout seconds. $timeout must be in the range 0..43200. AddPermission($label, $account_actions, [%opts]) NOT SUPPORTED IN APIs EARLIER THAN 2009-01-01 Sets a permissions policy with the specified label. $account_actions is a reference to a hash mapping 12-digit AWS account numbers to the action(s) you want to permit for those account IDs. The hash value for each key can be a string (e.g. "ReceiveMessage") or a reference to an array of strings (e.g. ["ReceiveMessage", "DeleteMessage"]) RemovePermission($label, [%opts]) NOT SUPPORTED IN APIs EARLIER THAN 2009-01-01 Removes the permissions policy with the specified label. GetAttributes([%opts]) Get the attributes for the queue. Returns a reference to a hash mapping attribute names to their values. Currently the following attribute names are returned: o VisibilityTimeout o ApproximateNumberOfMessages SetAttribute($attribute_name, $attribute_value, [%opts]) Sets the value for a queue attribute. Currently the only valid attribute name is "VisibilityTimeout". AUTHOR
Copyright 2007-2008 Simon Whitaker <swhitaker@cpan.org> This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. perl v5.10.1 2009-05-16 Amazon::SQS::Simple::Queue(3pm)
All times are GMT -4. The time now is 12:32 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy