Unable to preserve hard links. Why?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Unable to preserve hard links. Why?
# 1  
Old 08-26-2010
Question Unable to preserve hard links. Why?

Hi,

I'm trying to create a Makefile that would automate remastering Knoppix distribution.

As a part of the process I am mounting using linux cloop device a compressed filesystem and copy the content out of it to separate dir. However during that process I need to preserve hard links and it seems it is not working.

I've tried using:
  • cp -a --preserve=all
  • rsync -a -H
  • tar | tar version

All of them are making original hard-linked files a separate files.

The lines in the code are:
Code:
	#sudo nice tar -C $(NAME).cloop -cf - . | sudo nice tar -C $(NAME).tree -xvpf -
	sudo cp -av --preserve=all $(NAME).cloop $(NAME).tree
	#sudo rsync -x -a -H --progress --delete $(NAME).cloop/ $(NAME).tree

Can you please advise what can be wrong?

Below is the full content of the Makefile

Code:
SITE=ftp://sunsite.icm.edu.pl/pub/Linux/distributions/knoppix/
NAME=KNOPPIX_V6.2.1CD-2010-01-31-EN
PWD=$(shell pwd)

.PHONY: all clean test test-orig

all: $(NAME)-cherry88.iso

$(NAME).iso:
	wget -c -O $(NAME).iso.download $(SITE)/$(NAME).iso && mv $(NAME).iso.download $(NAME).iso

clean:
	if [ ! -z "`mount | grep "$(PWD)/$(NAME).cloop"`" ]; then sudo umount $(NAME).cloop && rm -f $(NAME).cloop.mount || test 1=1; fi
	if [ ! -z "`mount | grep "$(PWD)/$(NAME)"`" ]; then sudo umount $(NAME) && rm -f $(NAME).mount || test 1=1; fi

$(NAME).mount: $(NAME).iso
	mkdir -p $(NAME)
	if [ -z "`mount | grep "$(PWD)/$(NAME)"`" ]; then sudo mount -o loop $(NAME).iso $(NAME); fi
	touch $(NAME).mount

$(NAME).cloop.iso: $(NAME).mount
	nice extract_compressed_fs $(NAME)/KNOPPIX/KNOPPIX - > $(NAME).cloop.iso
	touch $(NAME).cloop.iso

$(NAME).cloop.mount: $(NAME).cloop.iso
	mkdir -p $(NAME).cloop
	if [ -z "`mount | grep "$(PWD)/$(NAME).cloop"`" ]; then sudo mount -o loop $(NAME).cloop.iso $(NAME).cloop; fi
	touch $(NAME).cloop.mount

$(NAME).cdtree.touch: $(NAME).mount
	sudo mkdir -p $(NAME).cdtree
	sudo rm -Rf $(NAME).cdtree
	#sudo nice tar -C $(NAME) --exclude=KNOPPIX/KNOPPIX -cf - . | sudo nice tar -C $(NAME).cdtree -xvpf -
	sudo nice cp -av --preserve=all $(NAME) $(NAME).cdtree
	#sudo rsync -x -a -H --progress --delete $(NAME)/ $(NAME).cdtree
	touch $(NAME).cdtree.touch

$(NAME).tree.touch: $(NAME).cloop.mount
	sudo mkdir -p $(NAME).tree
	sudo rm -Rf $(NAME).tree
	#sudo nice tar -C $(NAME).cloop -cf - . | sudo nice tar -C $(NAME).tree -xvpf -
	sudo cp -av --preserve=all $(NAME).cloop $(NAME).tree
	#sudo rsync -x -a -H --progress --delete $(NAME).cloop/ $(NAME).tree
	touch $(NAME).tree.touch

$(NAME).cdtree/KNOPPIX/KNOPPIX: $(NAME).tree.touch $(NAME).cdtree.touch $(NAME).cdtree.mods
	#nice -n 5 create_compressed_fs - 65536 | \
	#
	sudo rm -f $(NAME).cdtree/KNOPPIX/KNOPPIX || test 1=1
	nice mkisofs -R -U -V "KNOPPIX.net filesystem" -publisher "KNOPPIX www.knoppix.net" -v  \
		-hide-rr-moved -cache-inodes -pad $(NAME).tree | \
	sudo nice -n 5 create_compressed_fs -B 65536 -f tmp - $(NAME).cdtree/KNOPPIX/KNOPPIX

$(NAME)-cherry88.iso: $(NAME).cdtree/KNOPPIX/KNOPPIX
	sudo chmod a+w $(NAME).cdtree/KNOPPIX/sha1sums
	cd $(NAME).cdtree; sudo find -type f -not -name sha1sums -not -name boot.cat -not -name isolinux.bin \
		   -exec sha1sum '{}' \; > KNOPPIX/sha1sums
	sudo chmod a-w $(NAME).cdtree/KNOPPIX/sha1sums
	sudo nice mkisofs -pad -l -r -J -v -V "KNOPPIX" -no-emul-boot -boot-load-size 4 \
		-boot-info-table -b boot/isolinux/isolinux.bin -c boot/isolinux/boot.cat \
		-hide-rr-moved -o $(NAME)-cherry88.iso $(NAME).cdtree

$(NAME).cloop.md5sums: $(NAME).cloop.mount
	cd $(NAME).cloop && sudo nice find . -type f -print0 | nice sort -z | sudo nice xargs -0 ls -l | awk '{ print $$1" "$$2" "$$3" "$$4" "$$8 }' > ../$(NAME).cloop.ls
	cd $(NAME).cloop && sudo nice find . -type f -print0 | nice sort -z | sudo nice xargs -0 md5sum > ../$(NAME).cloop.md5sums

$(NAME).md5sums: $(NAME).mount
	cd $(NAME) && sudo nice find . -type f -print0 | nice sort -z | sudo nice xargs -0 ls -l | awk '{ print $$1" "$$2" "$$3" "$$4" "$$8 }' > ../$(NAME).ls
	cd $(NAME) && sudo nice find . -type f -print0 | nice sort -z | sudo nice xargs -0 md5sum > ../$(NAME).md5sums

$(NAME).cdtree.md5sums: $(NAME).cdtree.touch
	cd $(NAME).cdtree && sudo nice find . -type f -print0 | nice sort -z | sudo nice xargs -0 ls -l | awk '{ print $$1" "$$2" "$$3" "$$4" "$$8 }'> ../$(NAME).cdtree.ls
	cd $(NAME).cdtree && sudo nice find . -type f -print0 | nice sort -z | sudo nice xargs -0 md5sum > ../$(NAME).cdtree.md5sums

$(NAME).tree.md5sums: $(NAME).tree.touch
	cd $(NAME).tree && sudo nice find . -type f -print0 | nice sort -z | sudo nice xargs -0 ls -l | awk '{ print $$1" "$$2" "$$3" "$$4" "$$8 }' > ../$(NAME).tree.ls
	cd $(NAME).tree && sudo nice find . -type f -print0 | nice sort -z | sudo nice xargs -0 md5sum > ../$(NAME).tree.md5sums


# MODYFIKACJE
$(NAME).cdtree.mods: $(NAME).cdtree.touch
	# no 3d
	#	sudo sed -i '' -e 's/window_manager=compiz/window_manager=openbox-lxde/g' $(NAME).tree/etc/xdg/lxsession/LXDE/desktop.conf
	# touch
	touch $(NAME).cdtree.mods

test: $(NAME)-cherry88.iso
	qemu -m 512 -cdrom $(NAME)-cherry88.iso -boot d

test-orig:
	qemu -m 512 -cdrom $(NAME).iso -boot d

changes: $(NAME).cloop.md5sums $(NAME).tree.md5sums
	diff -u $(NAME).cloop.md5sums $(NAME).tree.md5sums > changes; test 1=1
	diff -u $(NAME).cloop.ls $(NAME).tree.ls > changes-ls; test 1=1

changes-cd: $(NAME).md5sums $(NAME).cdtree.md5sums
	diff -u $(NAME).md5sums $(NAME).cdtree.md5sums >  changes-cd
	diff -u $(NAME).ls $(NAME).cdtree.ls > changes-cd-ls

The command:
Code:
make changes

will work for a longer moment and produce changes file which is a diff of mounted cloop device and copyied tree content.
# 2  
Old 08-26-2010
tar preserves hardlinked files fine:

Code:
$ touch a
$ ln a b
$ tar -cf hard.tar [ab]
$ mkdir wtf
$ tar -C wtf -xf ./hard.tar
$ ls -li wtf/
total 0
1920716 -rw-r--r-- 2 tyler users 0 Aug 26 12:23 a
1920716 -rw-r--r-- 2 tyler users 0 Aug 26 12:23 b

I think part of your problem with tar may be you are trying to use the -C option when creating the tarball. -C does nothing when creating the file, you have to cd. Meaning, your hardlinks may be ending up a directory or two deeper than you thought they were.
# 3  
Old 08-26-2010
actually hard link is like directory record..so inode nums can change when in new dir..

Code:
 
touch a
ln a b

Code:
tar -cvf hard.tar *
a
b

Code:
# ls -li *
65194 -rw-r--r-- 2 root root 10240 Jun 25 03:20 a
65194 -rw-r--r-- 2 root root 10240 Jun 25 03:20 b
65423 -rw-r--r-- 1 root root 20480 Jun 25 03:22 hard.tar

Code:
rsync -H -avve "ssh localhost" /xx/dirrss/test1 /xx/dirrss/oba/*

building file list ...
done
delta-transmission disabled for local transfer or --whole-file
"a" is a hard link
b
hard.tar
test1/
a => b
total: matches=0  hash_hits=0  false_alarms=0 data=30720
...............
....................

Code:
ls -li test1/
total 48
65485 -rw-r--r-- 2 root root 10240 Jun 25 03:20 a
65485 -rw-r--r-- 2 root root 10240 Jun 25 03:20 b
65497 -rw-r--r-- 1 root root 20480 Jun 25 03:22 hard.tar
65484 drwxr-xr-x 2 root root  4096 Jun 25 03:22 test1

at the result pair of files are same data and same inode and hard link preserve Smilie
# 4  
Old 08-26-2010
Guys, I know how it should work, but it ain't happening. Possibly due to the fact that the source filesystem is RO cloop device based.

You may want to copy a Makefile, run make changes yourself and see wha I mean.
# 5  
Old 08-26-2010
The way you were using tar to create archives was completely wrong: It would create an archive with files in the wrong place, or even the wrong files. Have you tried it using my suggestions?

Could you give an ls -li listing of the input files you want preserved, and the output files not properly linked?
# 6  
Old 08-26-2010
I have three methods that should work. Non of them is. I've just rerun the whole script from the scratch with tar method enabled and it does work as I want except preserving the hardlinks.


Code:
$ ls -li */bin/dnsdomainname
   10419 -rwxr-xr-x 4 root root 12992 2010-01-19 10:40 KNOPPIX_V6.2.1CD-2010-01-31-EN.cloop/bin/dnsdomainname
10641687 -rwxr-xr-x 1 root root 12992 2010-01-19 10:40 KNOPPIX_V6.2.1CD-2010-01-31-EN.tree/bin/dnsdomainname

# 7  
Old 08-27-2010
You're trying to create hardlinks between different filesystems!?

That won't ever work, period. You'll have to find a different way to do whatever you're trying to do.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Timestamp change for hard and soft links

Hi team, i am writing a purge script to delete softlinks and hardlinks on linux system which are 3/10/30 days old. To test the script i need to create links with old timestamp, i am able to cange timestamp for files but not for links. i tried touch -h option but this option is not available on... (1 Reply)
Discussion started by: Satyak
1 Replies

2. UNIX for Dummies Questions & Answers

Simple example for soft and hard links

Hai, give me a simple example for soft and hard links. this will work for soft link ?? ln -s (2 Replies)
Discussion started by: Ramesh M
2 Replies

3. AIX

List all the soft links and hard links

Hi I'm logged in as root in an aix box Which command will list all the soft links and hard links present in the server ? (2 Replies)
Discussion started by: newtoaixos
2 Replies

4. Solaris

Hard Links and Soft or Sym links

When loooking at files in a directory using ls, how can I tell if I have a hard link or soft link? (11 Replies)
Discussion started by: Harleyrci
11 Replies

5. Shell Programming and Scripting

Hard Links Help

Ksh newbie here, so please bear with me. I'm currently writing a script that searches through a directory and displays files with multiple hard links. The way I have it set up, is that it displays the i-node number and then each of the link names. In addition to this, I need to know if there... (7 Replies)
Discussion started by: Unknown50862
7 Replies

6. UNIX for Advanced & Expert Users

Hard links for directories.

Hard links for directories are not permitted by default. But in some flavor of Unix, super user can create hard links for directories by some other way? Is that true? Is it possible in latest version of BSD or other unix? (2 Replies)
Discussion started by: bbala
2 Replies

7. UNIX for Dummies Questions & Answers

Deleting Symbolic and/or Hard links

From what I understand a symbolic link is alot like a shortcut where it points to another file. if the original file is deleted the symbolic link is rendered useless but a symbolic link can be deleted without any problem. A hard link is like a copy of the file itself but pointing to the same... (3 Replies)
Discussion started by: cue
3 Replies

8. UNIX for Dummies Questions & Answers

hard links in unix

hi i have a hardlink how can i find the source of it (2 Replies)
Discussion started by: jpriyank
2 Replies

9. UNIX for Dummies Questions & Answers

links.... soft or hard.. not sure?

hi, i am in a directory, have 2 files as below then do a ls -l gives the below lrwxrwxrwx 1 root system 23 Mar 08 2001 filea -> /adir/filea lrwxrwxrwx 1 root system 23 Mar 08 2001 filea -> /adir/fileb now, when i do a cd /adir, the system said, adir not... (5 Replies)
Discussion started by: yls177
5 Replies

10. UNIX for Dummies Questions & Answers

links: (soft, hard? symbolic??) inode

Hi, what is link? and soft link? how about hard one and symbolic link. and inode. i get confuse about this links. could anyone help me with full explainsion? thks Gusla (5 Replies)
Discussion started by: gusla
5 Replies
Login or Register to Ask a Question