Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

read string, check string length and cut

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting read string, check string length and cut
# 1  
Old 03-20-2007
Error read string, check string length and cut
Hello All,

Plz help me with:

I have a csv file with data separated by ',' and optionally enclosed by "". I want to check each of these values to see if they exceed the specified string length, and if they do I want to cut just that value to the max length allowed and keep the csv format as it is.

Example:

csv file:

1,Test Name,"This is a test and is funny",,,1234

Value1: max(10)
Value2: max(8)
Value3: max(21)
Value4: max(5)
Value5: max(5)
Value6: max(5)

and the expected result is:

1,Test Nam,This is a test and is,,,1234

Plz help!

Thnx in advance!
~Ozzy
# 2  
Old 03-20-2007
Quote:
Originally Posted by ozzy80
Hello All,

Plz help me with:

I have a csv file with data separated by ',' and optionally enclosed by "". I want to check each of these values to see if they exceed the specified string length, and if they do I want to cut just that value to the max length allowed and keep the csv format as it is.

Example:

csv file:

1,Test Name,"This is a test and is funny",,,1234

Value1: max(10)
Value2: max(8)
Value3: max(21)
Value4: max(5)
Value5: max(5)
Value6: max(5)

and the expected result is:

1,Test Nam,This is a test and is,,,1234

Plz help!

Thnx in advance!
~Ozzy
Code:
echo '1,Test Name,"This is a test and is funny",,,1234' | sed -e 's/"//g' -e 's/\([^,]\{0,10\}\)[^,]*,\([^,]\{0,8\}\)[^,]*,\([^,]\{0,21\}\)[^,]*,\([^,]\{0,5\}\)[^,]*,\([^,]\{0,5\}\)[^,]*,\([^,]\{0,5\}\)[^,]*/\1,\2,\3,\4,\5,\6/'

Code:
sed -e 's/"//g' -e 's/\([^,]\{0,10\}\)[^,]*,\([^,]\{0,8\}\)[^,]*,\([^,]\{0,21\}\)[^,]*,\([^,]\{0,5\}\)[^,]*,\([^,]\{0,5\}\)[^,]*,\([^,]\{0,5\}\)[^,]*/\1,\2,\3,\4,\5,\6/' <file>

# 3  
Old 03-20-2007
if you have Python, here's an alternative
Code:
#!/usr/bin/python
for line in open("csvfile"):
  line = line.strip().split(',')
  print "%s,%s,%s,%s,%s,%s" % (line[0][0:10] ,line[1][0:8],line[2].strip('"')[0:21],line[3][0:5],line[4][0:5],line[5][0:5])

output:
Code:
1,Test Nam,This is a test and is,,,1234

# 4  
Old 03-21-2007
Quote:
Originally Posted by ozzy80
Hello All,

Plz help me with:

I have a csv file with data separated by ',' and optionally enclosed by "". I want to check each of these values to see if they exceed the specified string length, and if they do I want to cut just that value to the max length allowed and keep the csv format as it is.

Example:

csv file:

1,Test Name,"This is a test and is funny",,,1234

Value1: max(10)
Value2: max(8)
Value3: max(21)
Value4: max(5)
Value5: max(5)
Value6: max(5)

and the expected result is:

1,Test Nam,This is a test and is,,,1234

Code:
awk -v lengths=10,8,21,5,5,5 '
BEGIN { FS = OFS = ","
        split(lengths,len,FS)
      }
      { n = 0
        while ( ++n <= NF ) $n = substr($n,1,len[n])
        print
      }' "$FILE"

# 5  
Old 03-21-2007
Quote:
Originally Posted by sb008
Code:
echo '1,Test Name,"This is a test and is funny",,,1234' | sed -e 's/"//g' -e 's/\([^,]\{0,10\}\)[^,]*,\([^,]\{0,8\}\)[^,]*,\([^,]\{0,21\}\)[^,]*,\([^,]\{0,5\}\)[^,]*,\([^,]\{0,5\}\)[^,]*,\([^,]\{0,5\}\)[^,]*/\1,\2,\3,\4,\5,\6/'

Code:
sed -e 's/"//g' -e 's/\([^,]\{0,10\}\)[^,]*,\([^,]\{0,8\}\)[^,]*,\([^,]\{0,21\}\)[^,]*,\([^,]\{0,5\}\)[^,]*,\([^,]\{0,5\}\)[^,]*,\([^,]\{0,5\}\)[^,]*/\1,\2,\3,\4,\5,\6/' <file>

Thank you so much! But one last question probably...

my script looks like this....

Code:
#!/bin/ksh

echo "started at " $(date);

while read record
do
   echo $record | sed -e 's/"//g' -e 's/\([^,]\{0,12\}\)[^,]*,\([^,]\{0,35\}\)[^,]*,\([^,]\{0,35\}\)[^,]*,\([^,]\{0,35\}\)[^,
]*,\([^,]\{0,20\}\)[^,]*,\([^,]\{0,10\}\)[^,]*,\([^,]\{0,13\}\)[^,]*,\([^,]\{0,5\}\)[^,]*,\([^,]\{0,35\}\)[^,]*,\([^,]\{0,35\
}\)[^,]*,\([^,]\{0,13\}\)[^,]*,\([^,]\{0,5\}\)[^,]*,\([^,]\{0,31\}\)[^,]*,\([^,]\{0,75\}\)[^,]*,\([^,]\{0,180\}\)[^,]*/\1,\2,
\3,\4,\5,\6,\7,\8,\9,\"10",\"11",\"12",\"13",\"14",\"15"/' >> test_data_2.dat

done < test_email

echo "ended at " $(date)

exit;

and data in tets_email file is ...

Code:
97	Metro Packaging	160 Fornelius Ave	Clifton	NJ	7013	(973) 709-9100	289	Jack Bhohj	Steven Neal	(973) 709-9100	218			Call for appt between 0600 and 1400 M-F.  Leave MSG if get voicemail. 973-777-3999 Warehouse direct line for emergencies POC William Toro. Shipping 0800 to 1600 M to F only.
98	Anchor Glass & Container	151 East McCanns Blvd.	Elmira	NY	14903	(607) 737-1933	324	Bill Weston	Mike Sopp	(607) 737-1933	300			Shipping hours 0800 to 2200 M to F.
278	Tate & Lyle #0278	Rt. 4 950 Morning Star Rd.	Houlton	ME	4730	(207) 532-9523								Load Hours: 7AM-2:00PM (EST)      Requires Appt.
509	QUINCY PLANT	4551 SQUIRES ROAD	QUINCY	MI	49082	(517) 689-2391		Ed Loftis	Charlotte Laws	(517) 689-2391			edward.loftis@conagrafoods.com#http://edward.loftis@conagrafoods.com#	Charlotte Laws (2nd Shift Lead Person)  charlotte.laws@conagrafoods.com  (517-689-2391)
786	Tate & Lyle / Specialty Warehouse #0786	333 Blair Bend Dr.	Loudon	TN	37774	(865) 458-9585								Load Hours: 8AM-3:00PM (EST)      First come, first serve.
2243	Tate & Lyle / Distribution Center #2243	4464 E. 350 South	Lafayette	IN	47905	(765) 474-2512		Shipping						Load Hours: 7AM-6:30PM (EST)      First come, first serve.
2247	Tate & Lyle / McLeod Warehouse #2247	4988 Cundiff Circle	Decatur	IL	62526	(217) 877-9626

When I execute it, I get "sed garbled error".. plz help!!!

Thnx
~Ozzy
# 6  
Old 03-21-2007
Quote:
Originally Posted by ozzy80
When I execute it, I get "sed garbled error".. plz help!!!

Even if it worked, that sed command is still garbled. How can you realistically expect to debug something like that?

For complex scripts, do not use sed; see the awk script I provided eariler in the thread.

# 7  
Old 03-21-2007
Quote:
Originally Posted by cfajohnson

Even if it worked, that sed command is still garbled. How can you realistically expect to debug something like that?

For complex scripts, do not use sed; see the awk script I provided eariler in the thread.

The awk is not working... says syntaz error!!!
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Script to count matched string then check again from last read position

Hi all, I am currently trying to figure out how can i capture a particular string from a log file, then search again but from the last line it read before onward. Basically let's say that the check runs every 5 mins via cron, counting the number of matched strings "Cannot assign requested... (5 Replies)
Discussion started by: nms
5 Replies

2. Shell Programming and Scripting

How to read file, and replace certain string with another string?

Hi all, the value in the following file is just an example. It could be a different value/network addresses. Here is my example of initial output in a file name net.txt Initial Output, net.txt The goal is to produce the following format which is to convert from CIDR to Netmask... (6 Replies)
Discussion started by: type8code0
6 Replies

3. UNIX for Dummies Questions & Answers

I need a Script to read Log string and check date at the same time

I need to check 1 log file, which is logging: 2014-08-18T09:10:39+02:00 user: XXXXX START FEATURE 2014-08-18T09:10:39+02:00 user: XXXXX FINISH FEATURE I first need to check that the START FEATURE starts and finish on the same time/date for the same user, which is different each time START... (2 Replies)
Discussion started by: TheBest43
2 Replies

4. Shell Programming and Scripting

Check file for string existence before appending it with string

I want to append file with a string but before doing that i want to check if this string already exist in that file.I tried with grep on Solaris 10 but unsuccessful.Man pages from grep seems to suggest if the string is found command status will be 0 and if not 1.But i am not finding it.May be i... (2 Replies)
Discussion started by: sahil_shine
2 Replies

5. Shell Programming and Scripting

Read File and check records for length

I need a script that will run in unix to: 1) Read and input file with 1 column that contains for ex: 0123456789 1234567890 ...etc 2) Checks the first column if it is: a. Numeric from 0 - 9 b. if it is not less... (4 Replies)
Discussion started by: mrn6430
4 Replies

6. Shell Programming and Scripting

how to cut a string from a string

now I have a string .a/b/c/done I want to get "done" from it. That is the result should be "done" sorry for my language before. Thanks in advance (5 Replies)
Discussion started by: Henryyy
5 Replies

7. UNIX for Dummies Questions & Answers

Read a string with leading spaces and find the length of the string

HI In my script, i am reading the input from the user and want to find the length of the string. The input may contain leading spaces. Right now, when leading spaces are there, they are not counted. Kindly help me My script is like below. I am using the ksh. #!/usr/bin/ksh echo... (2 Replies)
Discussion started by: dayamatrix
2 Replies

8. Shell Programming and Scripting

Read string from a file,plz help me to check

#!/usr/bin/ksh exec 0<property while read newReceiveDir do if then sed -e 's//home/joshua/bodi/data/receive/{$newReceiveDir:25}/g/' mp_validate.sh >| mp_validate.sh elif then sed -e 's//home/joshua/bodi/data/temp/{$newReceiveDir:22}/g/' mp_validate.sh >| mp_validate.sh ... (1 Reply)
Discussion started by: joshuaduan
1 Replies

9. Shell Programming and Scripting

sed problem - replacement string should be same length as matching string.

Hi guys, I hope you can help me with my problem. I have a text file that contains lines like this: 78 ANGELO -809.05 79 ANGELO2 -5,000.06 I need to find all occurences of amounts that are negative and replace them with x's 78 ANGELO xxxxxxx 79... (4 Replies)
Discussion started by: amangeles
4 Replies
Login or Register to Ask a Question