well I wound up writing one, I know it's probably a terrific hack, but here it is in case it helps someone else, this took me awhile to figure out
Code:
#!/bin/bash
# pipe ls of remote folder to file
# to compare to figure out which files
# are newer
b=`sftp user@ftp.domain.com <<EOF
cd folder_with_hourly_dataset
ls
quit
EOF`
echo $b > remotelist.txt
# sanitize that file, so that all we
# see are the .csv entries
list=$(cat remotelist.txt)
for i in $list
do
echo $i | grep \.csv$ >> sortedremotelist.txt
done
# create local list of files
ls *.csv >> locallist.txt
# compare local/remote list and then
# sanitize the output file into a file
# with one .csv per line so we can
# use a for loop to sftp get it
diff locallist.txt sortedremotelist.txt > get
cut -c3- get > get2
grep \.csv get2 > get3
# now go get that list of files
c=$(cat get3)
for i in $c
do
d=`user@domain.com <<EOF
cd directory_with_hourly_data
mget $i
quit
EOF`
echo "-------------"
echo "getting file: "$i
done
# clean up files for next time
cat /dev/null > remotelist.txt
cat /dev/null > sortedremotelist.txt
cat /dev/null > get
cat /dev/null > get2
cat /dev/null > get3
hope that helps someone else. Of course I could've done the whole thing with a single rsync -auv -e ssh user@example.com:/folder_with_updates/ ./ but I couldn't get a system user account on the box to do it
As part of a NIS implementation, (I think) I want to create a group (in /etc/group) that has users that do not exist on the target machine.
What effect will this have? Will it cause any problems?
Thanks,
Gary Cooper (1 Reply)
Hello,
I want to run one (not multiple) action if an element doesn't exist in array.
for example:
@array = (1..10);
foreach $el (@array)
{
if ($el != 11)
{
print "number not found\n";
}
}
the output of this simple script:
number not found (3 Replies)
Hey,
I'm currently getting into some kernel module progamming. As a little exercise I want to read the headers out of an ELF file. My code is very simple, here is the important part:
struct file *fp;
/* ... */
fp = filp_open("some/file/on/my/pc", O_RDONLY, 0);
if(fp == NULL) {
... (15 Replies)
Hi Forums,
I got a little problem, I made a few modifications to the code of the launch script of a testing server(minecraft) and now updating is broken aswell as the automatic directory creation.
These Lines somehow create an endless symlink that refers to itself and I don't know how to fix... (0 Replies)
Hii all,
i create the user
useradd -d /home/kk kk
passwd kk
when i tried to login to kk
i get a error user directory doesn't exist
then i tried
useradd kkk
passwd kkkwhen i tried to login to kkk
i get the same error user directory doesn't exist. (4 Replies)
Our SAN administrator decided to unpresent then destroy LUN's we were actively using as a volume group (all PV's in said volume group). Now every time I do a pvscan or whatever it complains about I/O errors trying to access those PV's. How do I get it to forget the VG existed completely? vgreduce... (7 Replies)
Need to use dd to generate a large file from a sample file of random data. This is because I don't have /dev/urandom.
I create a named pipe then:
dd if=mynamed.fifo do=myfile.fifo bs=1024 count=1024
but when I cat a file to the fifo that's 1024 random bytes:
cat randomfile.txt >... (7 Replies)
Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted!
1. The problem statement, all variables and given/known data:
I'm able to create a group but when I'm trying to delete the group it keeps stating Group Doesn't Exist. I know... (2 Replies)
hi Team,
I am connecting from one (A) linux server to another(C)/any linux server by sftp
on A linux server:
sftp userid@C
password:
mget is Not working fine
I am using mget to pull the files. it shows mget as invalid command.
But from (B) Linux server to (C) /to Any server Linux... (15 Replies)
Hi ,
we are not able to get file name is different casesensitive in sftp server.
for example, if the files are any case that means upper or lower case ,i need to get the filenames using the mget command in sfp.
sftp>mget Test.txt
Thanks (5 Replies)
Discussion started by: bmk123
5 Replies
LEARN ABOUT DEBIAN
git-ftp
GIT-FTP(1) General Commands Manual GIT-FTP(1)NAME
Git-ftp - FTP done the Git way
SYNOPSIS
git-ftp [actions] [options] [url]...
DESCRIPTION
This manual page documents briefly the git-ftp program.
Git-ftp is a FTP client using Git to determine which local files to upload or which files should be deleted on the remote host.
It saves the deployed state by uploading the SHA1 hash in the .git-ftp.log file. There is no need for Git (http://git-scm.org) to be
installed on the remote host.
Even if you play with different branches, git-ftp knows which files are different and only handles those files. No ordinary FTP client can
do this and it saves time and bandwith.
Another advantage is Git-ftp only handles files which are tracked with Git (http://git-scm.org).
ACTIONS
init Initializes the first upload to remote host.
push Uploads files which have changed since last upload.
catchup
Uploads the .git-ftp.log file only. We have already uploaded the files to remote host with a different program and want to remember
its state by uploading the .git-ftp.log file.
show Downloads last uploaded SHA1 from log and hooks `git show`.
add-scope <scope>
Creates a new scope (e.g. dev, production, testing, foobar). This is a wrapper action over git-config. See SCOPES section for
more information.
remove-scope <scope>
Remove a scope.
help Prints a usage help.
OPTIONS -u [username], --user [username]
FTP login name. If no argument is given, local user will be taken.
-p [password], --passwd [password]
FTP password. If no argument is given, a password prompt will be shown.
-k [[user]@[account]], --keychain [[user]@[account]]
FTP password from KeyChain (Mac OS X only).
-a, --all
Uploads all files of current Git checkout.
-c, --commit
Sets the SHA1 hash of last deployed commit by option.
-A, --active
Uses FTP active mode.
-s <scope>, --scope <scope>
Using a scope (e.g. dev, production, testing, foobar). See SCOPE and DEFAULTS section for more information.
-l, --lock
Enable remote locking.
-D, --dry-run
Does not upload or delete anything, but tries to get the .git-ftp.log file from remote host.
-f, --force
Does not ask any questions, it just does.
-n, --silent
Be silent.
-h, --help
Prints some usage information.
-v, --verbose
Be verbose.
-vv Be as verbose as possible.
--syncroot
Specifies a directory to sync from as if it were the git project root path.
--connections
Number of simultanious connections (Linux only).
--version
Prints version.
URL
The scheme of an URL is what you would expect
protocol://host.domain.tld:port/path
Below a full featured URL to host.exmaple.com on port 2121 to path mypath using protocol ftp:
ftp://host.example.com:2121/mypath
But, there is not just FTP. Supported protocols are:
ftp://...
FTP (default if no protocol is set)
sftp://...
SFTP
ftps://...
FTPS
ftpes://...
FTP over explicit SSL (FTPES) protocol
DEFAULTS
Don't repeat yourself. Setting defaults for git-ftp in .git/config
$ git config git-ftp.<(url|user|password)> <value>
Everyone likes examples
$ git config git-ftp.user john
$ git config git-ftp.url ftp.example.com
$ git config git-ftp.password secr3t
$ git config git-ftp.connections 10
$ git config git-ftp.syncroot path/dir
After setting those defaults, push to john@ftp.example.com is as simple as
$ git ftp push
SCOPES
Need different defaults per each system or environment? Use the so called scope feature.
Useful if you use multi environment development. Like a development, testing and a production environment.
$ git config git-ftp.<scope>.<(url|user|password)> <value>
So in the case below you would set a testing scope and a production scope.
Here we set the params for the scope "testing"
$ git config git-ftp.testing.url ftp.testing.com:8080/foobar-path
$ git config git-ftp.testing.password simp3l
Here we set the params for the scope "production"
$ git config git-ftp.production.user manager
$ git config git-ftp.production.url live.example.com
$ git config git-ftp.production.password n0tThatSimp3l
Pushing to scope testing alias john@ftp.testing.com:8080/foobar-path using password simp3l
$ git ftp push -s testing
Note: The SCOPE feature can be mixed with the DEFAULTS feature. Because we didn't set the user for this scope, git-ftp uses john as user
as set before in DEFAULTS.
Pushing to scope production alias manager@live.example.com using password n0tThatSimp3l
$ git ftp push -s production
You can also create scopes using the add-scope action. All settings can be defined in the URL. Here we create the production scope using
add-scope
$ git ftp add-scope production ftp://manager:n0tThatSimp3l@live.example.com/foobar-path
Deleting scopes is easy using the remove-scope action.
$ git ftp remove-scope production
IGNORING FILES
Add file names to .git-ftp-ignore to be ignored.
Ignoring all in Directory config:
config/*
Ignoring all files having extension .txt in ./ :
*.txt
This ignores a.txt and b.txt but not dir/c.txt
Ingnoring a single file called gargantubrain.txt:
gargantubrain.txt
EXIT CODES
There are a bunch of different error codes and their corresponding error messages that may appear during bad conditions. At the time of
this writing, the exit codes are:
1 Unknown error
2 Wrong Usage
3 Missing arguments
4 Error while uploading
5 Error while downloading
6 Unknown protocol
7 Remote locked
8 Not a Git project
KNOWN ISSUES &; BUGS
The upstream BTS can be found at <http://github.com/resmo/git-ftp/issues>.
AUTHORS
Rene Moser <mail@renemoser.net>.
git-ftp User Manual December 23, 2010 GIT-FTP(1)
03-05-2009
