Sponsored Content
Top Forums UNIX for Dummies Questions & Answers Rsync whole folder and head of each file Post 302877305 by DGPickett on Friday 29th of November 2013 04:06:31 PM
Old 11-29-2013
rsync is not interested in your short or empty files, but you may be able to ask it to not sync large files, let's look: https://www.unix.com/man-page/linux/1/rsync/ Yes:
--max-size=SIZE don't transfer any file larger than SIZE
Now, for the big files, you need to script up a find and process to make the exerp or empty files under the right relative paths.

If you make a local clone tree of the original directory tree, modified with exerpts, you can rsync that all, so your script does not have to ssh for every file it creates. A clone tree is a dir on the same device where you make all the same sub-directories and then hard-link all the files of the original tree (and any other inodes including any relative withing subtree symbolic links, but not absolute or relative outside subtree symolic links). When you want a file to be different, you must delete the hard link and replace it with a new, real file. (Only such files, with one link, should be written, else you destroy the original.)
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

head command with more than one file

Hi, I have the following problem. I have files with one column of data (let's say file1.dat, file2.dat...file6.dat), and I would like to record the first value of the column of each file into another file (let's name it fileall.dat), which would have the the six values, one in each column. I use to... (4 Replies)
Discussion started by: josegr
4 Replies

2. UNIX for Advanced & Expert Users

does head create new file

hi, Here my doubt is can we create a file using head/tail command in unix. for example: my file consists 2000 lines , i want to cut it into 2. does this work please give correct command line. head -1000 abc.txt > acd.txt tail -1000 abc.txt > bdc.txt hi, Here my doubt is can we... (5 Replies)
Discussion started by: chittisri
5 Replies

3. Shell Programming and Scripting

How can i get the head of file while using awk?

Hi All, I'm a newbie here, I'm just wondering how can i get the head of my file while using awk? input data: nik1,nik2,nik3 nik2,nik3,nik4 nik3,nik4,nik5 expected output is: nik1 because it is in the top and it is in the first delimeted. i tried awk -F "," '{print $1}' but i... (2 Replies)
Discussion started by: nikki1200
2 Replies

4. Shell Programming and Scripting

rsync delete single file from the target folder

Hi We want to delete a single file from the destiantion directory using rsync. Is it possible to do this ? or Do we have any alternate approaches in rsync( for ex applying some filters ..etc) For ex: ----------------------------------------------- Source (Folder) ... (3 Replies)
Discussion started by: MVEERA
3 Replies

5. Shell Programming and Scripting

rsync delete specific files - from different target folder

Hi, I need to use rsync to delete multiple files(only specified files not all) using --delete option, these files are located in different target folders. Instead of running rsync command multiple times for each file, can we achieve this with one time execution? your help is much... (0 Replies)
Discussion started by: MVEERA
0 Replies

6. Shell Programming and Scripting

Script to tar/rsync/rm multiple folder names

hi all, i attach a link with what im trying to do automatically via script but i have some questions i need answering please, bear in mind i am really new to bash scripting, the only thing i know how to do is commands in scripts like cd rm tar rsync cp stuff like that i have mutiple project... (48 Replies)
Discussion started by: robertkwild
48 Replies

7. Shell Programming and Scripting

Rsync - how to copy hidden folder or hidden files when using full path

Hello. I use this command : rsync -av --include=".*" --dry-run "$A_FULL_PATH_S" "$A_FULL_PATH_D"The data comes from the output of a find command. And no full source directories are in use, only some files. Source example... (2 Replies)
Discussion started by: jcdole
2 Replies

8. Shell Programming and Scripting

Rsync Error: rsync: link_stat failed: No such file or directory (2)

I wish to copy all the files & folder under /web/Transfer_Files/data/ on mymac1 (Linux) to remote server mybank.intra.com (Solaris 10) /tmp/ location I am using Ansible tool synchronize module which triggers the unix rsync command as below:rsync --delay-updates -F --compress --archive --rsh=ssh... (2 Replies)
Discussion started by: mohtashims
2 Replies

9. Shell Programming and Scripting

Help Needed for creating the folder by checking today's date and, take backup using rsync command

How to create a shell script to create a folder by using the today's date to take backup using rsync command on every evening around 7 pm. Kindly help. Thanks. To be more precise, I want to create a script which matches the today's date with server's date format, if matches then creates the... (2 Replies)
Discussion started by: bakula10
2 Replies
ln(1)							      General Commands Manual							     ln(1)

Name
       ln - link to a file

Syntax
       ln [ -f ] [ -i ] [ -s ] name1 [name2]
       ln [ -f ] [ -i ] [ -s ] name ... directory

Description
       A  link is a directory entry referring to a file.  A file, together with its size and all its protection information may have several links
       to it.  There are two kinds of links: hard links and symbolic links.

       By default makes hard links.  A hard link to a file is indistinguishable from the original directory entry.  Any  changes  to  a  file  are
       effective independent of the name used to reference the file.  Hard links may not span file systems and may not refer to directories.

       Given  one or two arguments, creates a link to an existing file name1.  If name2 is given, the link has that name.  The name2 may also be a
       directory in which to place the link.  Otherwise it is placed in the current directory.	If only the directory is specified,  the  link	is
       made to the last component of name1.

       Given  more  than two arguments, makes links to all the named files in the named directory.  The links made have the same name as the files
       being linked to.

Options
       -f   Forces existing destination pathnames to be removed before linking without prompting for confirmation.

       -i   Write a prompt to standard output requesting information for each link that would overwrite an existing file.  If  the  response  from
	    standard input is affirmative, and if permissions allow, the link is done. The -i option has this effect even if the standard input is
	    not a terminal.

       -s   Creates a symbolic link.

	    A symbolic link contains the name of the file to which it is linked.  The referenced file is used when an operation  is  performed	on
	    the  link.	 A  on a symbolic link returns the linked-to file.  An must be done to obtain information about the link.  The call may be
	    used to read the contents of a symbolic link.  Symbolic links may span file systems and may refer to directories.

See Also
       cp(1), mv(1), rm(1), link(2), readlink(2), stat(2), symlink(2)

																	     ln(1)
All times are GMT -4. The time now is 03:14 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy