Login or Register to Ask a Question and Join Our Community


UNIX for Beginners Questions & Answers

Help with HDFS Linux permission

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Help with HDFS Linux permission
# 1  
Old 04-18-2017
Help with HDFS Linux permission
Hi,

I am unable to change the permissions for a directory in HDFS.
from what i understand acl's supersede all other permissions. even if a directory is not owned by me, but there is an acl for me with rwx then i must be able to change the permissions of that directory.

Please find the permissions and the error i get when changing.

Code:
desind@test:~#> hdfs dfs -ls -d /adhoc/desind
drwxrwx---+  - hdfs supergroup          0 2017-04-17 15:01 /adhoc/desind

desind@test:~#> hdfs dfs -getfacl /adhoc/desind
# file: /adhoc/desind
# owner: hdfs
# group: supergroup
user::rwx
user:desind:rwx
group::---
mask::rwx
other::---

desind@test:~#> hdfs dfs -ls /adhoc/desind
Found 3 items
drwxrwxrwt+  - hdfs   supergroup          0 2017-04-17 14:37 /adhoc/desind/.Trash
-rw-rw----   3 desind supergroup          0 2017-04-17 14:50 /adhoc/desind/test.txt
-rw-rw----   3 desind supergroup          0 2017-04-17 15:01 /adhoc/desind/test1.txt



desind@test:~#> hdfs dfs -chmod 777 /adhoc/desind
chmod: changing permissions of '/adhoc/desind': Permission denied. user=desind is not the owner of inode=desind


Moderator's Comments:
Mod Comment Please use CODE tags as required by forum rules!
Last edited by RudiC; 04-18-2017 at 11:19 AM.. Reason: Added CODE tags.
# 2  
Old 04-18-2017
I am not familiar with hdfs in particular, but based on standard filesystems you will not be able to change ownership/perms on someone elses file. If you want your desind user to be your filesystem administrator you should set up sudo rules for that user to manipulate the files and directories.


Something in /etc/sudoers like the following that will let the desind user modify permissions on files/directories within the /adhoc filesystem:
Code:
User_Alias FILEADMIN = desind
Host_Alias HDFSHOSTS = test
Command_Alias HDFSCOM = hdfs dfs -chmod * /adhoc/*
FILEADMIN HDFSHOSTS=(root) HDFSCOM

Then at the command line you execute
Code:
$ sudo hdfs dfs -chmod 777 /adhoc/desind



Note that ideally you would put the full path to the hdfs command in the sudoers command alias, but I don't know what the full path is on your system. You can use the following command to determine the location
Code:
which hdfs

# 3  
Old 04-18-2017
Quote:
Originally Posted by Padow1
Then at the command line you execute
Code:
$ sudo hdfs dfs -chmod 777 /adhoc/desind

777 is not the magic sledgehammer to fix all permissions problems. Should this folder really be world-editable!?
# 4  
Old 04-18-2017
I agree that i cannot change others home directory. But should'nt i be able to change the permissions of my home directory ?

This desind directory in HDFS is my own directory.

Code:
/adhoc/desind

# 5  
Old 04-18-2017
You are absolutely right. I was just showing the original poster how to execute the command they were trying to execute without commenting on whether or not they should.
# 6  
Old 04-18-2017
Quote:
Originally Posted by desind
I agree that i cannot change others home directory. But should'nt i be able to change the permissions of my home directory ?
Depends what its ownership and group is, and whether you are either of them.

You definitely don't want to 777 your own home directory...
# 7  
Old 04-18-2017
Quote:
Originally Posted by desind
.
.
.
But should'nt i be able to change the permissions of my home directory ?
.
.
.
Not necessarily, depending on the policy of your company / provider / host admin. And, why should you? And, while you own most of the files in it, it might even be that there are, say, configuration files that are locked by the system admin (although not too probable, though).
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

To get latest hdfs file system

Hi All, I am having below hdfs file system /user/home/dte=2019_01_30/part_1 /user/home/dte=2019_01_30/part_2 /user/home/dte=2019_01_31/part_1 I need to take the latest month hdfs folder while passing date as parameter. For eg . if i pass as Feb month i.e. 20190201(YYYYMMDD), then... (0 Replies)
Discussion started by: Master_Mind
0 Replies

2. Shell Programming and Scripting

Shell Script for HDFS Ingestion Using JDBC

Peers, I was in process of building a script that connects to salesforce using jdbc and pull the data using spark and process in hive table. During this process I have encountered a problem where and variable assigned with hadoop command that list files in Azure Data lake is not parsing the... (2 Replies)
Discussion started by: jg355187
2 Replies

3. Shell Programming and Scripting

How to check total files size in hdfs directory?

Is there a way to calculate the total file size of HDFS file directory in GB or MB? I dont want to use du/df command. Without that is there a way HDFS Directory - /test/my_dir (1 Reply)
Discussion started by: rohit_shinez
1 Replies

4. Shell Programming and Scripting

Read CSV file and delete hdfs, hive and hbase tables

I have a CSV file with hdfs directories, hive tables and hbase tables. 1. first column - hdfs directories 2. second column - hive tables 3. third column - hbase tables I have to check the csv file and look for the first column and delete the hdfs directory from the hdfs path, now... (2 Replies)
Discussion started by: shivamayam
2 Replies

5. Shell Programming and Scripting

Strange permission issue on Linux server.

Starting sendmail throws this error: I decided to see the permission on the files under /etc/mail and this is the output: # ls -ltr total 284 -rwxr-xr-x. 1 root root 1847 Jan 27 2014 virtusertable -rwxr-xr-x. 1 root root 127 Jan 27 2014 trusted-users -rwxr-xr-x. 1 root root 92... (3 Replies)
Discussion started by: mohtashims
3 Replies

6. UNIX for Beginners Questions & Answers

UNIX and HDFS - file systems on same partition.

I am learning Hadoop. As a part of that, Hdfs - Hadoop distributed file system has commands similar to unix where we can create,copy,move files from unix/linux file system to HDFS. My question is 1) how two file systems (unix and hdfs) can coexist on thr same partition.. 2)What if block... (1 Reply)
Discussion started by: Narendra Eliset
1 Replies

7. UNIX for Advanced & Expert Users

UNIX and HDFS - file systems on same partition.

I am learning Hadoop. As a part of that, Hdfs - Hadoop distributed file system has commands similar to unix where we can create,copy,move files from unix/linux file system to HDFS. My question is 1) how two file systems (unix and hdfs) can coexist on thr same partition.. 2)What if block used... (0 Replies)
Discussion started by: Narendra Eliset
0 Replies

8. Shell Programming and Scripting

Change file permission of mounted drive Linux

I got a problem with the permission of mounted 2TB drive in my Linux/Mint system. All the files in any folder are with 777, which is not what I want. my fstab line for this disk is: UUID=90803E0C803DF974 /media/grape/Workspace1_ntfs ntfs auto,users,permissions 0 0 and blkid gave me: $> blkid ... (4 Replies)
Discussion started by: yifangt
4 Replies

9. Red Hat

SE Linux write permission denied

Hi, In my server I am getting below errors in "/var/log/messages": Oct 8 14:45:44 LKOGOMEEMM01 kernel: type=1400 audit(1444295744.792:15818): avc: denied { write } for pid=53421 comm="ip" path="/var/VRTSvcs/log/tmp/IPMultiNIC-8" dev=dm-0 ino=2754879 scontext=system_u:system_r:ifconfig_t:s0... (4 Replies)
Discussion started by: rochitsharma
4 Replies

10. Shell Programming and Scripting

Need help how to copy few records from hdfs to UNIX

Hi All , I am facing one issue here...I have a huge file in hadoop file system.Some disk space issues are thr ,thatswhy I want to copy 1st 100 records from hdfs to local unix.I tried below command but it is no working .Its giving error like cat: Unable to write to output stream.if any one can... (2 Replies)
Discussion started by: STCET22
2 Replies
Login or Register to Ask a Question