UNIX and HDFS - file systems on same partition.

 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers UNIX and HDFS - file systems on same partition.
# 1  
Old 04-24-2016
UNIX and HDFS - file systems on same partition.

I am learning Hadoop. As a part of that, Hdfs - Hadoop distributed file system has commands similar to unix where we can create,copy,move files from unix/linux file system to HDFS.
My question is
1) how two file systems (unix and hdfs) can coexist on thr same partition..
2)What if block used by one FS is overwritten/ modified by another FS.
3) Who does this block handling mechanism inorder to avoid unwanted results.
4) what happens to inodes(unix) pointing to data blocks being used/blocked by HDFS.
Please explain what happens at atomic level when mutiple file systems coexists on same partition.
# 2  
Old 04-24-2016
Multiple filesystems DO NOT coexist on a partition.

Hadoop either creates a file on an underlying UNIX filesystem and uses UNIX system calls to allocate blocks to that file, read blocks from the file, write blocks to the file, etc; or it creates a HDFS filesystem on a block special UNIX file and/or character special UNIX file that is a partition where no UNIX filesystem is allocated (and still uses UNIX read and write system calls to read and write HDFS blocks to that underlying UNIX file). The fact that the data written to that file is treated as a filesystem by Hadoop and may contain many files that Hadoop serves to users of the HDFS stored (entirely or partially) inside that UNIX file is invisible to the underlying UNIX system.
This User Gave Thanks to Don Cragun For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

8 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

To get latest hdfs file system

Hi All, I am having below hdfs file system /user/home/dte=2019_01_30/part_1 /user/home/dte=2019_01_30/part_2 /user/home/dte=2019_01_31/part_1 I need to take the latest month hdfs folder while passing date as parameter. For eg . if i pass as Feb month i.e. 20190201(YYYYMMDD), then... (0 Replies)
Discussion started by: Master_Mind
0 Replies

2. Shell Programming and Scripting

Read CSV file and delete hdfs, hive and hbase tables

I have a CSV file with hdfs directories, hive tables and hbase tables. 1. first column - hdfs directories 2. second column - hive tables 3. third column - hbase tables I have to check the csv file and look for the first column and delete the hdfs directory from the hdfs path, now... (2 Replies)
Discussion started by: shivamayam
2 Replies

3. UNIX for Advanced & Expert Users

UNIX and HDFS - file systems on same partition.

I am learning Hadoop. As a part of that, Hdfs - Hadoop distributed file system has commands similar to unix where we can create,copy,move files from unix/linux file system to HDFS. My question is 1) how two file systems (unix and hdfs) can coexist on thr same partition.. 2)What if block used... (0 Replies)
Discussion started by: Narendra Eliset
0 Replies

4. Shell Programming and Scripting

Need help how to copy few records from hdfs to UNIX

Hi All , I am facing one issue here...I have a huge file in hadoop file system.Some disk space issues are thr ,thatswhy I want to copy 1st 100 records from hdfs to local unix.I tried below command but it is no working .Its giving error like cat: Unable to write to output stream.if any one can... (2 Replies)
Discussion started by: STCET22
2 Replies

5. Shell Programming and Scripting

Downloading hdfs file to local UNIX through UNIX script

Hi All , I am very new to unix script.I am aware of unix commands but never put together in unix script level.If any one can suggest me technical guidance in the below scenario that will highly beneficial. Data have been already migrated from mainframe to Hadoop file system(HDFS).HDFS server... (15 Replies)
Discussion started by: STCET22
15 Replies

6. UNIX for Dummies Questions & Answers

Monitoring Unix systems

I am looking for a commercial tool that will give me -UNIX Monitoring performance solution+ reports on CCV format. (as perfmon on windows machines). The tool must have following counters per PROCESS: Page Faults/sec Virtual Bytes % Processor Time Handles count Threads count... (19 Replies)
Discussion started by: gen4ik
19 Replies

7. UNIX for Dummies Questions & Answers

file systems for unix

please someone give me 3 file systems for unix HP-UX version !!! thnks (2 Replies)
Discussion started by: androc
2 Replies

8. SCO

Sharing unix drives from two unix systems

I have two SCO openserver systems, 1 in the US and 1 in the UK. I am setting up a vpn to connect the two local networks that also have windows pc's on them. Is there a way that either unix system can see the hard drive on the other unix system so that I can share data between them. I run a cobol... (1 Reply)
Discussion started by: rongrout
1 Replies
Login or Register to Ask a Question