Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Extract file name based on the pattern Post 302986452 by Don Cragun on Friday 25th of November 2016 01:50:18 PM
Old 11-25-2016
As a first wild guess, I would try:
Code:
cd /user/cloudera/prod/SMS
for filename in *DATA_*_F*
do	ftmp=${filename##*DATA_}	# gives "BAD5A_RO_F_20161104.CSV"
	result=${ftmp%%_F*}		# gives "BAD5A_RO"
	printf '%s\n' "$result"
done > list.txt

and if that doesn't work, and assuming that the command:
Code:
hdfs dfs -ls /user/cloudera/prod/SMS

gives you a list of filenames separated by sequences of spaces, tabs, and/or newline characters and that none of your filenames contain any space, tab, or newline characters, I would also try:
Code:
for filename in $(hdfs dfs -ls /user/cloudera/prod/SMS/*DATA_*_F*)
do	ftmp=${filename##*DATA_}	# gives "BAD5A_RO_F_20161104.CSV"
	result=${ftmp%%_F*}		# gives "BAD5A_RO"
	printf '%s\n' "$result"
done > list.txt

I have absolutely no experience with hadoop filesystems or utilities, so I have no confidence that either of these will work.
This User Gave Thanks to Don Cragun For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Extract specific pattern from a file

Hi All, I am a newbie to Shell Scripting. I have a File The Server Name XXX002 ------------------------- 2.1 LAPD Iface Id Link MTU Side ecc_3_1 4 Up 512 User ecc_3_2 5 Up 512 User The Server Name XXX003 ------------------------- 2.1 LAPD (4 Replies)
Discussion started by: athreyavc
4 Replies

2. Shell Programming and Scripting

extract based on pattern

I have a mail log file and I want to extract some lines belonging to one domain. For example Input File: Dec 12 03:15:28 postfix/smtpd: 3F481EB0295: client=unknown, sasl_method=PLAIN, sasl_username=abcd@xyz.com Dec 12 03:22:08 postfix/smtpd: 60B56EE001D: client=5ad9b9ba.com,... (7 Replies)
Discussion started by: Bijayant Kumar
7 Replies

3. Shell Programming and Scripting

Extract a pattern from file

In my file I have a pattern ri="234" value of ri can be any i want to find this pattern in file replace this value of ri with another value stored in some variable say newri Please tell how to do it? Thanks in Advance (10 Replies)
Discussion started by: pasricha.kunal
10 Replies

4. Shell Programming and Scripting

Extract rows from file based on row numbers stored in another file

Hi All, I have a file which is like this: rows.dat 1 2 3 4 5 6 3 4 5 6 7 8 7 8 9 0 4 3 2 3 4 5 6 7 1 2 3 4 5 6 I have another file with numbers like these (numbers.txt): 1 3 4 5 I want to read numbers.txt file line by line. The extract the row from rows.dat based on the... (3 Replies)
Discussion started by: shoaibjameel123
3 Replies

5. Shell Programming and Scripting

extract a pattern from a xml file

Hello All, I want to write a shell script for extracting a content from a xml file the xml file looks like this: <Variable name="moreAxleInfo"> <type> <Table> <type> <NamedType> <type> <TypeRef... (11 Replies)
Discussion started by: suvendu4urs
11 Replies

6. Shell Programming and Scripting

Splitting textfile based on pattern and name new file after pattern

Hi there, I am pretty new to those things, so I couldn't figure out how to solve this, and if it is actually that easy. just found that awk could help:(. so i have a textfile with strings and numbers (originally copy pasted from word, therefore some empty cells) in the following structure: SC... (9 Replies)
Discussion started by: luja
9 Replies

7. Shell Programming and Scripting

Extract pattern from file

hi, file is having entries like below NRTMRC1=some value NRTSMO1=some value NRTMAA1=some value NRTSCO1=some value NRTSSMA1=some value NRTMRC11=some value NRTSMO11=some value NRTMAA11=some value NRTSCO11=some value NRTSSMA11=some alue NRTSSMMAA11=some value MNRMRC1=some value... (4 Replies)
Discussion started by: deepakiniimt
4 Replies

8. Shell Programming and Scripting

Extract sentence and its details from a text file based on another file of sentences

Hi I have two text files. The first file is TEXTFILEONE.txt as given below: <Text Text_ID="10155645315851111_10155645333076543" From="460350337461111" Created="2011-03-16T17:05:37+0000" use_count="123">This is the first text</Text> <Text Text_ID="10155645315851111_10155645317023456"... (7 Replies)
Discussion started by: my_Perl
7 Replies

9. Shell Programming and Scripting

Extract date from files based on file pattern

I want to extract dates from the files and i have different types of files with pattern. I have list file with the patterns and want to date extract based on it in a sh script Files in the directory : file1_20160101.txt file2_20160101_abc.txt filexyz20160101.txt list file with... (2 Replies)
Discussion started by: lijjumathew
2 Replies

10. UNIX for Beginners Questions & Answers

Extract some characters from lines based on pattern

Hi All, i would like to get some help regarding extracting certain characters from a line grepped. blahblah{1:F01IRVTUS30XXXX0000000001}{2:I103IRVTDEF0XXXXN}{4:blah blahblah{1:F01IRVTUS30XXXX0000000001}{2:I103IRVTDEF0XXXXN}{4:blah... (10 Replies)
Discussion started by: mad man
10 Replies
Graphics::Primitive::Insets(3pm)			User Contributed Perl Documentation			  Graphics::Primitive::Insets(3pm)

NAME
Graphics::Primitive::Insets - Space between things DESCRIPTION
Graphics::Primitive::Insets represents the amount of space that surrounds something. This object can be used to represent either padding or margins (in the CSS sense, one being inside the bounding box, the other being outside) SYNOPSIS
use Graphics::Primitive::Insets; my $insets = Graphics::Primitive::Insets->new({ top => 5, bottom => 5, left => 5, right => 5 }); METHODS
Constructor new Creates a new Graphics::Primitive::Insets. Instance Methods as_array Return these insets as an array in the form of top, right, bottom and left. bottom Set/Get the inset from the bottom. equal_to Determine if these Insets are equal to another. left Set/Get the inset from the left. right Set/Get the inset from the right. top Set/Get the inset from the top. zero Sets all the insets (top, left, bottom, right) to 0. AUTHOR
Cory Watson, "<gphat@cpan.org>" SEE ALSO
perl(1) COPYRIGHT &; LICENSE Copyright 2008-2010 by Cory G Watson. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. perl v5.12.3 2010-08-21 Graphics::Primitive::Insets(3pm)
All times are GMT -4. The time now is 01:14 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy