From stack overflow, I got the standard reg expression
to eliminate -
a) special characters
b) digits
c) more than 2 spaces to single space
to include -
d) - (hyphen)
e) ' (single quote)
SELECT ID, REGEXP_REPLACE(REGEXP_REPLACE(forenames, '[^A-Za-z-]', ' '),'\s{2,}',' ') , REGEXP_REPLACE(REGEXP_REPLACE(surname, '[^A-Za-z-]', ' '),'\s{2,}',' ') , forenames, surname from table1;
Instead of 2 functions how to get the result in single function?
to include '(single quote) \' is not working in regexp_replace.
Thanks.
Oracle Setup:
CREATE TABLE test_data ( id, value ) AS
SELECT 1, '123a45b£$- ''c45d#{e''' FROM DUAL
Query:
SELECT ID,
REGEXP_REPLACE(
value,
'[^a-zA-Z'' -]| +( )',
'\1'
)
FROM test_data
Output:
ID | REGEXP_REPLACE(VALUE,'[^A-ZA-Z''-]|+()','\1')
-: | :--------------------------------------------
1 | ab- 'cde'
db<>fiddle here
Related
Hi I want to replace following examples with the following:
(LTRIM) Replace any space at the beginning before first Alphanumerical character
(RTRIM) Replace any space at the end after last Alphanumerical character
Replace any non-alpha numerical with space
Replace any empty (not null) with word "UNKNOWN"
In case of only space replace all spaces with word "UNKNOWN"
' abc ' -> 'abc'
'abc ' -> 'abc'
' abc ' -> 'abc'
'!ab c ? ' -> 'ab c'
' a b c ' -> 'a b c'
'a!b?c $ ' -> 'a b c'
' ' -> 'UNKNOWN'
'' -> 'UNKNOWN'
null -> null
These are the queries I'm working on but I'm not getting anywhere:
select
'-|&#/,.‘“<>():;' as default1,
translate(SUBSTR(MAX ('-|&#/,.‘“<>():;'),1,70), '-|&#/,.‘“<>():;', ' ') as formatted1
from dual;
select trim(regexp_replace(regexp_replace(' a b cdefgh ' , '[[:space:]]*',''), '(.)', '\1 UNK' )) as formatted from dual;
SELECT LTRIM(RTRIM(' NEXT LEVEL EMPLOYEE ')) from dual;
Replace any empty (not null) with word "UNKNOWN"
This is impossible as, in Oracle, there are no empty strings as an empty string is stored as NULL.
Apart from that impossibility, you can use:
SELECT value,
CASE
WHEN value IS NULL
THEN NULL
ELSE COALESCE(
TRIM(
BOTH ' ' FROM
REGEXP_REPLACE(
value,
'[^A-Za-z0-9]',
' '
)
),
'UNKNOWN'
)
END AS updated_value
FROM table_name;
Which, for the sample data:
CREATE TABLE table_name (value) AS
SELECT ' abc ' FROM DUAL UNION ALL
SELECT 'abc ' FROM DUAL UNION ALL
SELECT ' abc ' FROM DUAL UNION ALL
SELECT '!ab c ? ' FROM DUAL UNION ALL
SELECT ' a b c ' FROM DUAL UNION ALL
SELECT 'a!b?c $ ' FROM DUAL UNION ALL
SELECT ' ' FROM DUAL UNION ALL
SELECT '' FROM DUAL UNION ALL
SELECT null FROM DUAL;
Outputs:
VALUE
UPDATED_VALUE
abc
abc
abc
abc
abc
abc
!ab c ?
ab c
a b c
a b c
a!b?c $
a b c
UNKNOWN
null
null
null
null
fiddle
I have a long string.
message := 'I loooove my pet animal';
This string in 23 chars long. If message is greater that 15 chars, I need to find the length of message where I can break the string into 2 strings. For example, in this case,
message1 := 'I loove my'
message2 := 'pet animal'
Essentially it should find the position of a whole word at the previous to 15 chars and the break the original string into 2 at that point.
Please give me ideas how I can do this.
Thank you.
Here is a general solution - with possibly more than one input string, and with inputs of any length. The only assumption is that no single word may be more than 15 characters, and that everything between two spaces is considered a word. If a "word" can be more than 15 characters, the solution can be adapted, but the requirement itself would need to state what the desired result is in such a case.
I make up two input strings in a CTE (at the top) - that is not part of the solution, it is just for testing and illustration. I also wrote this in plain SQL - there is no need for PL/SQL code for this type of problem. Set processing (instead of one row at a time) should result in much better execution.
The approach is to identify the location of all spaces (I append and prepend a space to each string, too, so I won't have to deal with exceptions for the first and last substring); then I decide, in a recursive subquery, where each "maximal" substring should begin and where it should end; and then outputting the substrings is trivial. I used a recursive query, that should work in Oracle 11.1 (or 11.2 with the syntax I used, with column names in CTE declarations - it can be changed easily to work in 11.1). In Oracle 12, it would be easier to rewrite the same idea using MATCH_RECOGINZE.
with
inputs ( id, str ) as (
select 101, 'I loooove my pet animal' from dual union all
select 102, '1992 was a great year for - make something up here as needed' from dual
),
positions ( id, pos ) as (
select id, instr(' ' || str || ' ', ' ', 1, level)
from inputs
connect by level <= length(str) - length(replace(str, ' ')) + 2
and prior id = id
and prior sys_guid() is not null
),
r ( id, str, line_number, pos_from, pos_to ) as (
select id, ' ' || str || ' ', 0, null, 1
from inputs
union all
select r.id, r.str, r.line_number + 1, r.pos_to,
( select max(pos)
from positions
where id = r.id and pos - r.pos_to between 1 and 16
)
from r
where pos_to is not null
)
select id, line_number, substr(str, pos_from + 1, pos_to - pos_from - 1) as line_text
from r
where line_number > 0 and pos_to is not null
order by id, line_number
;
Output:
ID LINE_NUMBER LINE_TEXT
---- ----------- ---------------
101 1 I loooove my
101 2 pet animal
102 1 1992 was a
102 2 great year for
102 3 - make
102 4 something up
102 5 here as needed
7 rows selected.
First you reverse string.
SELECT REVERSE(strField) FROM DUAL;
Then you calculate length i = length(strField).
Then find the first space after the middle
j := INSTR( REVERSE(strField), ' ', i / 2, i)`
Finally split by i - j (maybe +/- 1 need to test it)
DEMO
WITH parameter (id, strField) as (
select 101, 'I loooove my pet animal' from dual union all
select 102, '1992 was a great year for - make something up here as needed' from dual union all
select 103, 'You are Supercalifragilisticexpialidocious' from dual
), prepare (id, rev, len, middle) as (
SELECT id, reverse(strField), length(strField), length(strField) / 2
FROM parameter
)
SELECT p.*, l.*,
SUBSTR(strField, 1, len - INSTR(rev, ' ', middle)) as first,
SUBSTR(strField, len - INSTR(rev, ' ', middle) + 2, len) as second
FROM parameter p
JOIN prepare l
ON p.id = l.id
OUTPUT
I have a string like below.
'comp' as "COMPUTER",'ms' as "MOUSE" ,'keybr' as "KEYBOARD",'MONT' as "MONITOR",
Is it possible to write a query so that i will get the result as
'comp' ,'ms' ,'keybr' ,'MONT' ,
I can replace the string "as" with empty string using REPLACE query.
But how do I remove the string inside double quote?
Can anyone help me doing this?
Thanks in advance.
select replace(
regexp_replace('''comp'' as "COMPUTER"'
, '(".*")'
,null)
,' as '
,null)
from dual
The regex '(".*")' selects text whatever in double quotes.
EDIT:
the regex replaced the entire length of the matching pattern. So, we might need to tokenise the string first using comma as delimiter and apply the regex. Later join it.(LISTAGG)
WITH str_tab(str1, rn) AS
(SELECT regexp_substr(str, '[^,]+', 1, LEVEL), -- delimts
LEVEL
FROM (SELECT '''comp'' as "computer",''comp'' as "computer"' str
FROM dual) tab
CONNECT BY LEVEL <= LENGTH(str) - LENGTH(REPLACE(str, ',')) + 1)
SELECT listagg(replace(
regexp_replace(str1
, '(".*")'
,null)
,' as '
,null), ',') WITHIN GROUP (ORDER BY rn) AS new_text
FROM str_tab;
EDIT2:
A cleaner approach from #EatAPeach
with x(y) as (
select q'<'comp' as "COMPUTER",'ms' as "MOUSE" ,'keybr' as "KEYBOARD",'MONT' as "MONITOR",>'
from dual
)
select y,
regexp_replace(y, 'as ".*?"' ,null)
from x;
I have the following string my trying get out the 1111111 and 33333333333 with out the |
character
SELECT regexp_substr('7|1111111|2222222|33333333333|0||20140515|||false|0|0|0|0|0|','*[|]*[|][0-9]*')FROM dual
Using REGEXP_REPLACE may be a bit simpler;
SELECT REGEXP_REPLACE('7|1111111|2222222|33333333333|0||20140515|||false|0|0|0|0|0|',
'^([^|]*[|]){1}([^|]*).*$', '\2') FROM dual;
> 1111111
SELECT REGEXP_REPLACE('7|1111111|2222222|33333333333|0||20140515|||false|0|0|0|0|0|',
'^([^|]*[|]){3}([^|]*).*$', '\2') FROM dual;
> 33333333333
You can choose column by choosing how many pipes to skip in the {1} part.
A simple SQLfiddle to test with.
A short explanation of the regexp;
([^|]+[|]){3} -- Matches 3 groups of {optional characters}{pipe}
(\d*) -- Matches the next digit group (the one we want)
.* -- Matches the rest of the expression
What we want is the second paranthesized group, that is, we replace the whole string by the back reference \2.
Because "|" separators always present it's simpler to extract fields with simple substring function rather than using regular expressions.
Just find positions of corresponding separators in source string and extract content between them:
with test_data as (
select
'7|1111111|2222222|33333333333|0||20140515|||false|0|0|0|0|0|ABC' as s,
8 as field_number -- test 1, 3, 8, 10 and 16
from dual
)
select
field_number,
substr(
s,
decode( field_number,
1,1,
instr(s,'|',1,field_number - 1) + 1
),
(
decode( instr(s,'|',1,field_number),
0, length(s)+ 1,
instr(s,'|',1,field_number)
)
-
decode( field_number,
1, 1,
instr(s,'|',1,field_number - 1) + 1
)
)
) as field_value
from
test_data
SQLFiddle
This variant works with empty fields, non-numeric fields and so on.
Possible simplification with appending additional separators to the start and the end of the string:
with test_data as (
select
(
'|' ||
'7|1111111|2222222|33333333333|0||20140515|||false|0|0|0|0|0|ABC' ||
'|'
) as s, -- additional separators appended before and after original string
10 as field_number -- test 1, 3, 8, 10 and 16
from dual
)
select
field_number,
substr(
s,
instr(s, '|', 1, field_number) + 1,
(
instr(s, '|', 1, field_number + 1)
-
(instr(s, '|', 1, field_number) + 1)
)
) as field_value
from
test_data
;
SQLFiddle
Background
A locution is a noun-phrase consisting of at least two words, such as:
black olive
hot pepper sauce
rose finn apple potato
The separate words black and olive are an adjective (black - JJ) and a noun (olive - NN). However, humans know that black olive is a noun (that differentiates it from, say, a green olive).
The problem here is how to most efficiently transform a list of normalized ingredient names (such as the list above) into a specific format for a natural language processor (NLP).
Example Data
The table can be created as follows:
CREATE TABLE ingredient_name (
id bigserial NOT NULL, -- Uniquely identifies the ingredient.
label character varying(30) NOT NULL
);
The following SQL statements show actual database records:
insert into ingredient_name (label) values ('alfalfa sprout');
insert into ingredient_name (label) values ('almond extract');
insert into ingredient_name (label) values ('concentrated apple juice');
insert into ingredient_name (label) values ('black-eyed pea');
insert into ingredient_name (label) values ('rose finn apple potato');
Data Format
The general format is:
lexeme1_lexeme2_<lexemeN> lexeme1_lexeme2_lexemeN NN
Given the list of words above, the NLP expects:
black_<olive> black_olive NN
hot_pepper_<sauce> hot_pepper_sauce NN
rose_finn_apple_<potato> rose_finn_apple_potato NN
The database has a table (recipe.ingredient_name) and a column (label). The labels are normalized (e.g., single space, lower case).
SQL Statement
The code that produces the expected results:
CREATE OR REPLACE VIEW ingredient_locutions_vw AS
SELECT
t.id,
-- Replace spaces with underscores
translate( t.prefix, ' ', '_' )
|| '<' || t.suffix || '>' || ' ' ||
translate( t.label, ' ', '_' )
|| ' NN' AS locution_nlp
FROM (
SELECT
id,
-- Ingredient name
label,
-- All words except the last word
left( label, abs( strpos( reverse( label ), ' ' ) - length( label ) ) + 1 ) AS prefix,
-- Just the last word
substr( label,
length( label ) - strpos( reverse( label ), ' ' ) + 2
) AS suffix
FROM
ingredient_name
WHERE
-- Limit set to ingredient names having at least one space
strpos( label, ' ' ) > 0
) AS t;
Question
What is a more efficient (or elegant) way to split the prefix (all words except the first) and suffix (just the last word) in the above code?
The system is PostgreSQL 9.1.
Thank you!
CREATE OR REPLACE VIEW ingredient_locutions_vw AS
SELECT
t.id,
format('%s_<%s> %s NN',
array_to_string(t.prefix, '_'),
t.suffix,
array_to_string(t.label, '_')
) AS locution_nlp
FROM (
SELECT
id,
-- Ingredient name
label,
-- All words except the last word
label[1:array_length(label, 1) - 1] AS prefix,
-- Just the last word
label[array_length(label, 1)] AS suffix
FROM (
select id, string_to_array(label, ' ') as label
from ingredient_name
) s
WHERE
-- Limit set to ingredient names having at least one space
array_length(label, 1) > 1
) AS t;
select * from ingredient_locutions_vw ;
id | locution_nlp
----+--------------------------------------------------------
1 | alfalfa_<sprout> alfalfa_sprout NN
2 | almond_<extract> almond_extract NN
3 | concentrated_apple_<juice> concentrated_apple_juice NN
4 | black-eyed_<pea> black-eyed_pea NN
5 | rose_finn_apple_<potato> rose_finn_apple_potato NN
(5 rows)