Functions for Working with Strings
Functions for searching in strings and for replacing in strings are described separately.
empty
Checks whether the input string is empty. A string is considered non-empty if it contains at least one byte, even if this byte is a space or the null byte.
The function is also available for arrays and UUIDs.
Syntax
Arguments
x
— Input value. String.
Returned value
- Returns
1
for an empty string or0
for a non-empty string. UInt8.
Example
Result:
notEmpty
Checks whether the input string is non-empty. A string is considered non-empty if it contains at least one byte, even if this byte is a space or the null byte.
The function is also available for arrays and UUIDs.
Syntax
Arguments
x
— Input value. String.
Returned value
- Returns
1
for a non-empty string or0
for an empty string string. UInt8.
Example
Result:
length
Returns the length of a string in bytes rather than in characters or Unicode code points. The function also works for arrays.
Alias: OCTET_LENGTH
Syntax
Parameters
Returned value
- Length of the string or array
s
in bytes. UInt64.
Example
Query:
Result:
Query:
Result:
lengthUTF8
Returns the length of a string in Unicode code points rather than in bytes or characters. It assumes that the string contains valid UTF-8 encoded text. If this assumption is violated, no exception is thrown and the result is undefined.
Aliases:
CHAR_LENGTH
CHARACTER_LENGTH
Syntax
Parameters
s
— String containing valid UTF-8 encoded text. String.
Returned value
- Length of the string
s
in Unicode code points. UInt64.
Example
Query:
Result:
left
Returns a substring of string s
with a specified offset
starting from the left.
Syntax
Parameters
s
— The string to calculate a substring from. String or FixedString.offset
— The number of bytes of the offset. (U)Int*.
Returned value
- For positive
offset
: A substring ofs
withoffset
many bytes, starting from the left of the string. - For negative
offset
: A substring ofs
withlength(s) - |offset|
bytes, starting from the left of the string. - An empty string if
length
is 0.
Example
Query:
Result:
Query:
Result:
leftUTF8
Returns a substring of a UTF-8 encoded string s
with a specified offset
starting from the left.
Syntax
Parameters
s
— The UTF-8 encoded string to calculate a substring from. String or FixedString.offset
— The number of bytes of the offset. (U)Int*.
Returned value
- For positive
offset
: A substring ofs
withoffset
many bytes, starting from the left of the string. - For negative
offset
: A substring ofs
withlength(s) - |offset|
bytes, starting from the left of the string. - An empty string if
length
is 0.
Example
Query:
Result:
Query:
Result:
leftPad
Pads a string from the left with spaces or with a specified string (multiple times, if needed) until the resulting string reaches the specified length
.
Syntax
Alias: LPAD
Arguments
string
— Input string that should be padded. String.length
— The length of the resulting string. UInt or Int. If the value is smaller than the input string length, then the input string is shortened tolength
characters.pad_string
— The string to pad the input string with. String. Optional. If not specified, then the input string is padded with spaces.
Returned value
- A left-padded string of the given length. String.
Example
Result:
leftPadUTF8
Pads the string from the left with spaces or a specified string (multiple times, if needed) until the resulting string reaches the given length. Unlike leftPad which measures the string length in bytes, the string length is measured in code points.
Syntax
Arguments
string
— Input string that should be padded. String.length
— The length of the resulting string. UInt or Int. If the value is smaller than the input string length, then the input string is shortened tolength
characters.pad_string
— The string to pad the input string with. String. Optional. If not specified, then the input string is padded with spaces.
Returned value
- A left-padded string of the given length. String.
Example
Result:
right
Returns a substring of string s
with a specified offset
starting from the right.
Syntax
Parameters
s
— The string to calculate a substring from. String or FixedString.offset
— The number of bytes of the offset. (U)Int*.
Returned value
- For positive
offset
: A substring ofs
withoffset
many bytes, starting from the right of the string. - For negative
offset
: A substring ofs
withlength(s) - |offset|
bytes, starting from the right of the string. - An empty string if
length
is 0.
Example
Query:
Result:
Query:
Result:
rightUTF8
Returns a substring of UTF-8 encoded string s
with a specified offset
starting from the right.
Syntax
Parameters
s
— The UTF-8 encoded string to calculate a substring from. String or FixedString.offset
— The number of bytes of the offset. (U)Int*.
Returned value
- For positive
offset
: A substring ofs
withoffset
many bytes, starting from the right of the string. - For negative
offset
: A substring ofs
withlength(s) - |offset|
bytes, starting from the right of the string. - An empty string if
length
is 0.
Example
Query:
Result:
Query:
Result:
rightPad
Pads a string from the right with spaces or with a specified string (multiple times, if needed) until the resulting string reaches the specified length
.
Syntax
Alias: RPAD
Arguments
string
— Input string that should be padded. String.length
— The length of the resulting string. UInt or Int. If the value is smaller than the input string length, then the input string is shortened tolength
characters.pad_string
— The string to pad the input string with. String. Optional. If not specified, then the input string is padded with spaces.
Returned value
- A left-padded string of the given length. String.
Example
Result:
rightPadUTF8
Pads the string from the right with spaces or a specified string (multiple times, if needed) until the resulting string reaches the given length. Unlike rightPad which measures the string length in bytes, the string length is measured in code points.
Syntax
Arguments
string
— Input string that should be padded. String.length
— The length of the resulting string. UInt or Int. If the value is smaller than the input string length, then the input string is shortened tolength
characters.pad_string
— The string to pad the input string with. String. Optional. If not specified, then the input string is padded with spaces.
Returned value
- A right-padded string of the given length. String.
Example
Result:
compareSubstrings
Compare two strings lexicographically.
Syntax
Arguments
string1
— The first string to compare. Stringstring2
- The second string to compare.Stringstring1_offset
— The position (zero-based) instring1
from which the comparison starts. UInt*.string2_offset
— The position (zero-based index) instring2
from which the comparison starts. UInt*.num_bytes
— The maximum number of bytes to compare in both strings. Ifstring_offset
+num_bytes
exceeds the end of an input string,num_bytes
will be reduced accordingly. UInt*.
Returned value
- -1 — If
string1
[string1_offset
:string1_offset
+num_bytes
] <string2
[string2_offset
:string2_offset
+num_bytes
]. - 0 — If
string1
[string1_offset
:string1_offset
+num_bytes
] =string2
[string2_offset
:string2_offset
+num_bytes
]. - 1 — If
string1
[string1_offset
:string1_offset
+num_bytes
] >string2
[string2_offset
:string2_offset
+num_bytes
].
Example
Query:
Result:
lower
Converts the ASCII Latin symbols in a string to lowercase.
Syntax*
Alias: lcase
Parameters
input
: A string type String.
Returned value
- A String data type value.
Example
Query:
upper
Converts the ASCII Latin symbols in a string to uppercase.
Syntax
Alias: ucase
Parameters
input
— A string type String.
Returned value
- A String data type value.
Examples
Query:
lowerUTF8
Converts a string to lowercase, assuming that the string contains valid UTF-8 encoded text. If this assumption is violated, no exception is thrown and the result is undefined.
Does not detect the language, e.g. for Turkish the result might not be exactly correct (i/İ vs. i/I). If the length of the UTF-8 byte sequence is different for upper and lower case of a code point (such as ẞ
and ß
), the result may be incorrect for this code point.
Syntax
Parameters
input
— A string type String.
Returned value
- A String data type value.
Example
Query:
Result:
upperUTF8
Converts a string to uppercase, assuming that the string contains valid UTF-8 encoded text. If this assumption is violated, no exception is thrown and the result is undefined.
Does not detect the language, e.g. for Turkish the result might not be exactly correct (i/İ vs. i/I). If the length of the UTF-8 byte sequence is different for upper and lower case of a code point (such as ẞ
and ß
), the result may be incorrect for this code point.
Syntax
Parameters
input
— A string type String.
Returned value
- A String data type value.
Example
Query:
Result:
isValidUTF8
Returns 1, if the set of bytes constitutes valid UTF-8-encoded text, otherwise 0.
Syntax
Parameters
input
— A string type String.
Returned value
- Returns
1
, if the set of bytes constitutes valid UTF-8-encoded text, otherwise0
.
Query:
Result:
toValidUTF8
Replaces invalid UTF-8 characters by the �
(U+FFFD) character. All running in a row invalid characters are collapsed into the one replacement character.
Syntax
Arguments
input_string
— Any set of bytes represented as the String data type object.
Returned value
- A valid UTF-8 string.
Example
repeat
Concatenates a string as many times with itself as specified.
Syntax
Alias: REPEAT
Arguments
s
— The string to repeat. String.n
— The number of times to repeat the string. UInt* or Int*.
Returned value
A string containing string s
repeated n
times. If n
<= 0, the function returns the empty string. String.
Example
Result:
space
Concatenates a space (
) as many times with itself as specified.
Syntax
Alias: SPACE
.
Arguments
n
— The number of times to repeat the space. UInt* or Int*.
Returned value
The string containing string
repeated n
times. If n
<= 0, the function returns the empty string. String.
Example
Query:
Result:
reverse
Reverses the sequence of bytes in a string.
reverseUTF8
Reverses a sequence of Unicode code points in a string. Assumes that the string contains valid UTF-8 encoded text. If this assumption is violated, no exception is thrown and the result is undefined.
concat
Concatenates the given arguments.
Syntax
Arguments
Values of arbitrary type.
Arguments which are not of types String or FixedString are converted to strings using their default serialization. As this decreases performance, it is not recommended to use non-String/FixedString arguments.
Returned values
The String created by concatenating the arguments.
If any of arguments is NULL
, the function returns NULL
.
Example
Query:
Result:
Query:
Result:
||
operatorUse the || operator for string concatenation as a concise alternative to concat()
. For example, 'Hello, ' || 'World!'
is equivalent to concat('Hello, ', 'World!')
.
concatAssumeInjective
Like concat but assumes that concat(s1, s2, ...) → sn
is injective. Can be used for optimization of GROUP BY.
A function is called injective if it returns for different arguments different results. In other words: different arguments never produce identical result.
Syntax
Arguments
Values of type String or FixedString.
Returned values
The String created by concatenating the arguments.
If any of argument values is NULL
, the function returns NULL
.
Example
Input table:
Result:
concatWithSeparator
Concatenates the given strings with a given separator.
Syntax
Alias: concat_ws
Arguments
- sep — separator. Const String or FixedString.
- exprN — expression to be concatenated. Arguments which are not of types String or FixedString are converted to strings using their default serialization. As this decreases performance, it is not recommended to use non-String/FixedString arguments.
Returned values
The String created by concatenating the arguments.
If any of the argument values is NULL
, the function returns NULL
.
Example
Result:
concatWithSeparatorAssumeInjective
Like concatWithSeparator
but assumes that concatWithSeparator(sep, expr1, expr2, expr3...) → result
is injective. Can be used for optimization of GROUP BY.
A function is called injective if it returns for different arguments different results. In other words: different arguments never produce identical result.
substring
Returns the substring of a string s
which starts at the specified byte index offset
. Byte counting starts from 1. If offset
is 0, an empty string is returned. If offset
is negative, the substring starts pos
characters from the end of the string, rather than from the beginning. An optional argument length
specifies the maximum number of bytes the returned substring may have.
Syntax
Aliases:
substr
mid
byteSlice
Arguments
s
— The string to calculate a substring from. String, FixedString or Enumoffset
— The starting position of the substring ins
. (U)Int*.length
— The maximum length of the substring. (U)Int*. Optional.
Returned value
A substring of s
with length
many bytes, starting at index offset
. String.
Example
Result:
substringUTF8
Returns the substring of a string s
which starts at the specified byte index offset
for Unicode code points. Byte counting starts from 1
. If offset
is 0
, an empty string is returned. If offset
is negative, the substring starts pos
characters from the end of the string, rather than from the beginning. An optional argument length
specifies the maximum number of bytes the returned substring may have.
Assumes that the string contains valid UTF-8 encoded text. If this assumption is violated, no exception is thrown and the result is undefined.
Syntax
Arguments
s
— The string to calculate a substring from. String, FixedString or Enumoffset
— The starting position of the substring ins
. (U)Int*.length
— The maximum length of the substring. (U)Int*. Optional.
Returned value
A substring of s
with length
many bytes, starting at index offset
.
Implementation details
Assumes that the string contains valid UTF-8 encoded text. If this assumption is violated, no exception is thrown and the result is undefined.
Example
substringIndex
Returns the substring of s
before count
occurrences of the delimiter delim
, as in Spark or MySQL.
Syntax
Alias: SUBSTRING_INDEX
Arguments
- s — The string to extract substring from. String.
- delim — The character to split. String.
- count — The number of occurrences of the delimiter to count before extracting the substring. If count is positive, everything to the left of the final delimiter (counting from the left) is returned. If count is negative, everything to the right of the final delimiter (counting from the right) is returned. UInt or Int
Example
Result:
substringIndexUTF8
Returns the substring of s
before count
occurrences of the delimiter delim
, specifically for Unicode code points.
Assumes that the string contains valid UTF-8 encoded text. If this assumption is violated, no exception is thrown and the result is undefined.
Syntax
Arguments
s
— The string to extract substring from. String.delim
— The character to split. String.count
— The number of occurrences of the delimiter to count before extracting the substring. If count is positive, everything to the left of the final delimiter (counting from the left) is returned. If count is negative, everything to the right of the final delimiter (counting from the right) is returned. UInt or Int
Returned value
A substring String of s
before count
occurrences of delim
.
Implementation details
Assumes that the string contains valid UTF-8 encoded text. If this assumption is violated, no exception is thrown and the result is undefined.
Example
appendTrailingCharIfAbsent
Appends character c
to string s
if s
is non-empty and does not end with character c
.
Syntax
convertCharset
Returns string s
converted from the encoding from
to encoding to
.
Syntax
base32Encode
Encodes a string using Base32.
Syntax
Arguments
plaintext
— String column or constant.
Returned value
- A string containing the encoded value of the argument. String or FixedString.
Example
Result:
base32Decode
Accepts a string and decodes it using Base32 encoding scheme.
Syntax
Arguments
encoded
— String or FixedString. If the string is not a valid Base32-encoded value, an exception is thrown.
Returned value
- A string containing the decoded value of the argument. String.
Example
Result:
tryBase32Decode
Like base32Decode
but returns an empty string in case of error.
Syntax
Parameters
encoded
: String or FixedString. If the string is not a valid Base32-encoded value, returns an empty string in case of error.
Returned value
- A string containing the decoded value of the argument.
Examples
Query:
base58Encode
Encodes a string using Base58 in the "Bitcoin" alphabet.
Syntax
Arguments
plaintext
— String column or constant.
Returned value
- A string containing the encoded value of the argument. String or FixedString.
Example
Result:
base58Decode
Accepts a string and decodes it using Base58 encoding scheme using "Bitcoin" alphabet.
Syntax
Arguments
encoded
— String or FixedString. If the string is not a valid Base58-encoded value, an exception is thrown.
Returned value
- A string containing the decoded value of the argument. String.
Example
Result:
tryBase58Decode
Like base58Decode
but returns an empty string in case of error.
Syntax
Parameters
encoded
: String or FixedString. If the string is not a valid Base58-encoded value, returns an empty string in case of error.
Returned value
- A string containing the decoded value of the argument.
Examples
Query:
base64Encode
Encodes a String or FixedString as base64, according to RFC 4648.
Alias: TO_BASE64
.
Syntax
Arguments
plaintext
— String column or constant.
Returned value
- A string containing the encoded value of the argument.
Example
Result:
base64URLEncode
Encodes an URL (String or FixedString) as base64 with URL-specific modifications, according to RFC 4648.
Syntax
Arguments
url
— String column or constant.
Returned value
- A string containing the encoded value of the argument.
Example
Result:
base64Decode
Accepts a String and decodes it from base64, according to RFC 4648. Throws an exception in case of an error.
Alias: FROM_BASE64
.
Syntax
Arguments
encoded
— String column or constant. If the string is not a valid Base64-encoded value, an exception is thrown.
Returned value
- A string containing the decoded value of the argument.
Example
Result:
base64URLDecode
Accepts a base64-encoded URL and decodes it from base64 with URL-specific modifications, according to RFC 4648. Throws an exception in case of an error.
Syntax
Arguments
encodedURL
— String column or constant. If the string is not a valid Base64-encoded value with URL-specific modifications, an exception is thrown.
Returned value
- A string containing the decoded value of the argument.
Example
Result:
tryBase64Decode
Like base64Decode
but returns an empty string in case of error.
Syntax
Arguments
encoded
— String column or constant. If the string is not a valid Base64-encoded value, returns an empty string.
Returned value
- A string containing the decoded value of the argument.
Examples
Query:
tryBase64URLDecode
Like base64URLDecode
but returns an empty string in case of error.
Syntax
Parameters
encodedURL
— String column or constant. If the string is not a valid Base64-encoded value with URL-specific modifications, returns an empty string.
Returned value
- A string containing the decoded value of the argument.
Examples
Query:
endsWith
Returns whether string str
ends with suffix
.
Syntax
endsWithUTF8
Returns whether string str
ends with suffix
, the difference between endsWithUTF8
and endsWith
is that endsWithUTF8
match str
and suffix
by UTF-8 characters.
Syntax
Example
Result:
startsWith
Returns whether string str
starts with prefix
.
Syntax
Example
startsWithUTF8
Returns whether string str
starts with prefix
, the difference between startsWithUTF8
and startsWith
is that startsWithUTF8
match str
and suffix
by UTF-8 characters.
Example
Result:
trim
Removes the specified characters from the start or end of a string. If not specified otherwise, the function removes whitespace (ASCII-character 32).
Syntax
Arguments
Returned value
A string without leading and/or trailing specified characters. String.
Example
Result:
trimLeft
Removes the consecutive occurrences of whitespace (ASCII-character 32) from the start of a string.
Syntax
Alias: ltrim
.
Arguments
input_string
— The string to trim. String.trim_characters
— The characters to trim. Optional. String. If not specified,' '
( single whitespace) is used as trim character.
Returned value
A string without leading common whitespaces. String.
Example
Result:
trimRight
Removes the consecutive occurrences of whitespace (ASCII-character 32) from the end of a string.
Syntax
Alias: rtrim
.
Arguments
input_string
— The string to trim. String.trim_characters
— The characters to trim. Optional. String. If not specified,' '
( single whitespace) is used as trim character.
Returned value
A string without trailing common whitespaces. String.
Example
Result:
trimBoth
Removes the consecutive occurrences of whitespace (ASCII-character 32) from both ends of a string.
Syntax
Alias: trim
.
Arguments
input_string
— The string to trim. String.trim_characters
— The characters to trim. Optional. String. If not specified,' '
( single whitespace) is used as trim character.
Returned value
A string without leading and trailing common whitespaces. String.
Example
Result:
CRC32
Returns the CRC32 checksum of a string using CRC-32-IEEE 802.3 polynomial and initial value 0xffffffff
(zlib implementation).
The result type is UInt32.
CRC32IEEE
Returns the CRC32 checksum of a string, using CRC-32-IEEE 802.3 polynomial.
The result type is UInt32.
CRC64
Returns the CRC64 checksum of a string, using CRC-64-ECMA polynomial.
The result type is UInt64.
normalizeUTF8NFC
Converts a string to NFC normalized form, assuming the string is valid UTF8-encoded text.
Syntax
Arguments
words
— UTF8-encoded input string. String.
Returned value
- String transformed to NFC normalization form. String.
Example
Result:
normalizeUTF8NFD
Converts a string to NFD normalized form, assuming the string is valid UTF8-encoded text.
Syntax
Arguments
words
— UTF8-encoded input string. String.
Returned value
- String transformed to NFD normalization form. String.
Example
Result:
normalizeUTF8NFKC
Converts a string to NFKC normalized form, assuming the string is valid UTF8-encoded text.
Syntax
Arguments
words
— UTF8-encoded input string. String.
Returned value
- String transformed to NFKC normalization form. String.
Example
Result:
normalizeUTF8NFKD
Converts a string to NFKD normalized form, assuming the string is valid UTF8-encoded text.
Syntax
Arguments
words
— UTF8-encoded input string. String.
Returned value
- String transformed to NFKD normalization form. String.
Example
Result:
encodeXMLComponent
Escapes characters with special meaning in XML such that they can afterwards be place into a XML text node or attribute.
The following characters are replaced: <
, &
, >
, "
, '
.
Also see the list of XML and HTML character entity references.
Syntax
Arguments
x
— An input string. String.
Returned value
- The escaped string. String.
Example
Result:
decodeXMLComponent
Un-escapes substrings with special meaning in XML. These substrings are: "
&
'
>
<
This function also replaces numeric character references with Unicode characters. Both decimal (like ✓
) and hexadecimal (✓
) forms are supported.
Syntax
Arguments
x
— An input string. String.
Returned value
- The un-escaped string. String.
Example
Result:
decodeHTMLComponent
Un-escapes substrings with special meaning in HTML. For example: ℏ
>
♦
♥
<
etc.
This function also replaces numeric character references with Unicode characters. Both decimal (like ✓
) and hexadecimal (✓
) forms are supported.
Syntax
Arguments
x
— An input string. String.
Returned value
- The un-escaped string. String.
Example
Result:
extractTextFromHTML
This function extracts plain text from HTML or XHTML.
It does not conform 100% to the HTML, XML or XHTML specification but the implementation is reasonably accurate and fast. The rules are the following:
- Comments are skipped. Example:
<!-- test -->
. Comment must end with-->
. Nested comments are disallowed. Note: constructions like<!-->
and<!--->
are not valid comments in HTML but they are skipped by other rules. - CDATA is pasted verbatim. Note: CDATA is XML/XHTML-specific and processed on a "best-effort" basis.
script
andstyle
elements are removed with all their content. Note: it is assumed that closing tag cannot appear inside content. For example, in JS string literal has to be escaped like"<\/script>"
. Note: comments and CDATA are possible insidescript
orstyle
- then closing tags are not searched inside CDATA. Example:<script><![CDATA[</script>]]></script>
. But they are still searched inside comments. Sometimes it becomes complicated:<script>var x = "<!--"; </script> var y = "-->"; alert(x + y);</script>
Note:script
andstyle
can be the names of XML namespaces - then they are not treated like usualscript
orstyle
elements. Example:<script:a>Hello</script:a>
. Note: whitespaces are possible after closing tag name:</script >
but not before:< / script>
.- Other tags or tag-like elements are skipped without inner content. Example:
<a>.</a>
Note: it is expected that this HTML is illegal:<a test=">"></a>
Note: it also skips something like tags:<>
,<!>
, etc. Note: tag without end is skipped to the end of input:<hello
- HTML and XML entities are not decoded. They must be processed by separate function.
- Whitespaces in the text are collapsed or inserted by specific rules.
- Whitespaces at the beginning and at the end are removed.
- Consecutive whitespaces are collapsed.
- But if the text is separated by other elements and there is no whitespace, it is inserted.
- It may cause unnatural examples:
Hello<b>world</b>
,Hello<!-- -->world
- there is no whitespace in HTML, but the function inserts it. Also consider:Hello<p>world</p>
,Hello<br>world
. This behavior is reasonable for data analysis, e.g. to convert HTML to a bag of words.
- Also note that correct handling of whitespaces requires the support of
<pre></pre>
and CSSdisplay
andwhite-space
properties.
Syntax
Arguments
x
— input text. String.
Returned value
- Extracted text. String.
Example
The first example contains several tags and a comment and also shows whitespace processing.
The second example shows CDATA
and script
tag processing.
In the third example text is extracted from the full HTML response received by the url function.
Result:
ascii
Returns the ASCII code point (as Int32) of the first character of string s
.
If s
is empty, the result is 0. If the first character is not an ASCII character or not part of the Latin-1 supplement range of UTF-16, the result is undefined.
Syntax
soundex
Returns the Soundex code of a string.
Syntax
Arguments
val
— Input value. String
Returned value
- The Soundex code of the input value. String
Example
Result:
punycodeEncode
Returns the Punycode representation of a string. The string must be UTF8-encoded, otherwise the behavior is undefined.
Syntax
Arguments
val
— Input value. String
Returned value
- A Punycode representation of the input value. String
Example
Result:
punycodeDecode
Returns the UTF8-encoded plaintext of a Punycode-encoded string. If no valid Punycode-encoded string is given, an exception is thrown.
Syntax
Arguments
val
— Punycode-encoded string. String
Returned value
- The plaintext of the input value. String
Example
Result:
tryPunycodeDecode
Like punycodeDecode
but returns an empty string if no valid Punycode-encoded string is given.
idnaEncode
Returns the ASCII representation (ToASCII algorithm) of a domain name according to the Internationalized Domain Names in Applications (IDNA) mechanism. The input string must be UTF-encoded and translatable to an ASCII string, otherwise an exception is thrown. Note: No percent decoding or trimming of tabs, spaces or control characters is performed.
Syntax
Arguments
val
— Input value. String
Returned value
- A ASCII representation according to the IDNA mechanism of the input value. String
Example
Result:
tryIdnaEncode
Like idnaEncode
but returns an empty string in case of an error instead of throwing an exception.
idnaDecode
Returns the Unicode (UTF-8) representation (ToUnicode algorithm) of a domain name according to the Internationalized Domain Names in Applications (IDNA) mechanism.
In case of an error (e.g. because the input is invalid), the input string is returned.
Note that repeated application of idnaEncode()
and idnaDecode()
does not necessarily return the original string due to case normalization.
Syntax
Arguments
val
— Input value. String
Returned value
- A Unicode (UTF-8) representation according to the IDNA mechanism of the input value. String
Example
Result:
byteHammingDistance
Calculates the hamming distance between two byte strings.
Syntax
Examples
Result:
Alias: mismatches
stringJaccardIndex
Calculates the Jaccard similarity index between two byte strings.
Syntax
Examples
Result:
stringJaccardIndexUTF8
Like stringJaccardIndex but for UTF8-encoded strings.
editDistance
Calculates the edit distance between two byte strings.
Syntax
Examples
Result:
Alias: levenshteinDistance
editDistanceUTF8
Calculates the edit distance between two UTF8 strings.
Syntax
Examples
Result:
Alias: levenshteinDistanceUTF8
damerauLevenshteinDistance
Calculates the Damerau-Levenshtein distance between two byte strings.
Syntax
Examples
Result:
jaroSimilarity
Calculates the Jaro similarity between two byte strings.
Syntax
Examples
Result:
jaroWinklerSimilarity
Calculates the Jaro-Winkler similarity between two byte strings.
Syntax
Examples
Result:
initcap
Convert the first letter of each word to upper case and the rest to lower case. Words are sequences of alphanumeric characters separated by non-alphanumeric characters.
Because initCap
converts only the first letter of each word to upper case you may observe unexpected behaviour for words containing apostrophes or capital letters. For example:
will return
This is a known behaviour, with no plans currently to fix it.
Syntax
Arguments
val
— Input value. String.
Returned value
val
with the first letter of each word converted to upper case. String.
Example
Query:
Result:
initcapUTF8
Like initcap, initcapUTF8
converts the first letter of each word to upper case and the rest to lower case. Assumes that the string contains valid UTF-8 encoded text.
If this assumption is violated, no exception is thrown and the result is undefined.
This function does not detect the language, e.g. for Turkish the result might not be exactly correct (i/İ vs. i/I). If the length of the UTF-8 byte sequence is different for upper and lower case of a code point, the result may be incorrect for this code point.
Syntax
Arguments
val
— Input value. String.
Returned value
val
with the first letter of each word converted to upper case. String.
Example
Query:
Result:
firstLine
Returns the first line from a multi-line string.
Syntax
Arguments
val
— Input value. String
Returned value
- The first line of the input value or the whole value if there is no line separators. String
Example
Result:
stringCompare
Compare two strings lexicographically.
Syntax
Arguments
string1
— The first string to compare. Stringstring2
- The second string to compare.Stringstring1_offset
— The position (zero-based) instring1
from which the comparison starts. Optional, positive number.string2_offset
— The position (zero-based index) instring2
from which the comparison starts. Optional, positive number.num_bytes
— The maximum number of bytes to compare in both strings. Ifstring_offset
+num_bytes
exceeds the end of an input string,num_bytes
will be reduced accordingly.
Returned value
- -1 — If
string1
[string1_offset
:string1_offset
+num_bytes
] <string2
[string2_offset
:string2_offset
+num_bytes
] andstring1_offset
< len(string1
) andstring2_offset
< len(string2
). Ifstring1_offset
>= len(string1
) andstring2_offset
< len(string2
). - 0 — If
string1
[string1_offset
:string1_offset
+num_bytes
] =string2
[string2_offset
:string2_offset
+num_bytes
] andstring1_offset
< len(string1
) andstring2_offset
< len(string2
). Ifstring1_offset
>= len(string1
) andstring2_offset
>= len(string2
). - 1 — If
string1
[string1_offset
:string1_offset
+num_bytes
] >string2
[string2_offset
:string2_offset
+num_bytes
] andstring1_offset
< len(string1
) andstring2_offset
< len(string2
). Ifstring1_offset
< len(string1
) andstring2_offset
>= len(string2
).
Example
Result:
Result:
sparseGrams
Finds all substrings of a given string that have a length of at least n
,
where the hashes of the (n-1)-grams at the borders of the substring
are strictly greater than those of any (n-1)-gram inside the substring.
Uses crc32 as a hash function.
Syntax
Arguments
s
— An input string. Stringmin_ngram_length
— The minimum length of extracted ngram. The default and minimal value is 3.max_ngram_length
— The maximum length of extracted ngram. The default value is 100. Should be not less than 'min_ngram_length'
Returned value
Example
Result:
sparseGramsUTF8
Finds all substrings of a given string that have a length of at least n
,
where the hashes of the (n-1)-grams at the borders of the substring
are strictly greater than those of any (n-1)-gram inside the substring.
Uses crc32 as a hash function.
Expects UTF-8 string, throws an exception in case of invalid UTF-8 sequence.
Syntax
Arguments
s
— An input string. Stringmin_ngram_length
— The minimum length of extracted ngram. The default and minimal value is 3.max_ngram_length
— The maximum length of extracted ngram. The default value is 100. Should be not less than 'min_ngram_length'
Returned value
Example
Result:
sparseGramsHashes
Finds hashes of all substrings of a given string that have a length of at least n
,
where the hashes of the (n-1)-grams at the borders of the substring
are strictly greater than those of any (n-1)-gram inside the substring.
Uses crc32 as a hash function.
Syntax
Arguments
s
— An input string. Stringmin_ngram_length
— The minimum length of extracted ngram. The default and minimal value is 3.max_ngram_length
— The maximum length of extracted ngram. The default value is 100. Should be not less than 'min_ngram_length'
Returned value
Example
Result:
sparseGramsHashesUTF8
Finds hashes of all substrings of a given string that have a length of at least n
,
where the hashes of the (n-1)-grams at the borders of the substring
are strictly greater than those of any (n-1)-gram inside the substring.
Uses crc32 as a hash function.
Expects UTF-8 string, throws an exception in case of invalid UTF-8 sequence.
Syntax
Arguments
s
— An input string. Stringmin_ngram_length
— The minimum length of extracted ngram. The default and minimal value is 3.max_ngram_length
— The maximum length of extracted ngram. The default value is 100. Should be not less than 'min_ngram_length'
Returned value
Example
Result:
stringBytesUniq
Counts the number of distinct bytes in a string.
Syntax
Arguments
s
— The string to analyze. String.
Returned value
- The number of distinct bytes in the string. UInt16.
Example
Result:
stringBytesEntropy
Calculates Shannon's entropy of byte distribution in a string.
Syntax
Arguments
s
— The string to analyze. String.
Returned value
- Shannon's entropy of byte distribution in the string. Float64.
Example
Result:
conv
Converts numbers between different number bases, supporting bases from 2 to 36. This function is compatible with MySQL's CONV() function and handles partial invalid inputs by processing valid digits until the first invalid character.
Syntax
Arguments
number
— The number to convert. String, FixedString or Integerfrom_base
— The source base (2-36). Integerto_base
— The target base (2-36). Integer
Returned value
- Returns the string representation of the number in the target base. String
Example
Result:
CRC32
Introduced in: v20.1
Calculates the CRC32 checksum of a string using the CRC-32-IEEE 802.3 polynomial and initial value 0xffffffff
(zlib implementation).
Syntax
Arguments
s
— String to calculate CRC32 for.String
Returned value
Returns the CRC32 checksum of the string. UInt32
Examples
Usage example
CRC32IEEE
Introduced in: v20.1
Calculates the CRC32 checksum of a string using the CRC-32-IEEE 802.3 polynomial.
Syntax
Arguments
s
— String to calculate CRC32 for.String
Returned value
Returns the CRC32 checksum of the string. UInt32
Examples
Usage example
CRC64
Introduced in: v20.1
Calculates the CRC64 checksum of a string using the CRC-64-ECMA polynomial.
Syntax
Arguments
s
— String to calculate CRC64 for.String
Returned value
Returns the CRC64 checksum of the string. UInt64
Examples
Usage example
appendTrailingCharIfAbsent
Introduced in: v1.1
Appends character c
to string s
if s
is non-empty and does not end with character c
.
Syntax
Arguments
Returned value
Returns string s
with character c
appended if s
does not end with c
. String
Examples
Usage example
ascii
Introduced in: v22.11
Returns the ASCII code point of the first character of string s
as an Int32
.
Syntax
Arguments
s
— String input.String
Returned value
Returns the ASCII code point of the first character. If s
is empty, the result is 0
. If the first character is not an ASCII character or not part of the Latin-1 supplement range of UTF-16, the result is undefined. Int32
Examples
Usage example
base32Decode
Introduced in: v25.6
Decodes a Base32 (RFC 4648) string. If the string is not valid Base32-encoded, an exception is thrown.
Syntax
Arguments
encoded
— String column or constant.String
Returned value
Returns a string containing the decoded value of the argument. String
Examples
Usage example
base32Encode
Introduced in: v25.6
Encodes a string using Base32.
Syntax
Arguments
plaintext
— Plaintext to encode.String
Returned value
Returns a string containing the encoded value of the argument. String
or FixedString
Examples
Usage example
base58Decode
Introduced in: v22.7
Decodes a Base58 string. If the string is not valid Base58-encoded, an exception is thrown.
Syntax
Arguments
encoded
— String column or constant to decode.String
Returned value
Returns a string containing the decoded value of the argument. String
Examples
Usage example
base58Encode
Introduced in: v22.7
Encodes a string using Base58 encoding.
Syntax
Arguments
plaintext
— Plaintext to encode.String
Returned value
Returns a string containing the encoded value of the argument. String
Examples
Usage example
base64Decode
Introduced in: v18.16
Decodes a string from Base64 representation, according to RFC 4648. Throws an exception in case of error.
Syntax
Arguments
encoded
— String column or constant to decode. If the string is not valid Base64-encoded, an exception is thrown.String
Returned value
Returns the decoded string. String
Examples
Usage example
base64Encode
Introduced in: v18.16
Encodes a string using Base64 representation, according to RFC 4648.
Syntax
Arguments
plaintext
— Plaintext column or constant to decode.String
Returned value
Returns a string containing the encoded value of the argument. String
Examples
Usage example
base64URLDecode
Introduced in: v24.6
Decodes a string from Base64 representation using URL-safe alphabet, according to RFC 4648. Throws an exception in case of error.
Syntax
Arguments
encoded
— String column or constant to encode. If the string is not valid Base64-encoded, an exception is thrown.String
Returned value
Returns a string containing the decoded value of the argument. String
Examples
Usage example
base64URLEncode
Introduced in: v18.16
Encodes a string using Base64 (RFC 4648) representation using URL-safe alphabet.
Syntax
Arguments
plaintext
— Plaintext column or constant to encode.String
Returned value
Returns a string containing the encoded value of the argument. String
Examples
Usage example
basename
Introduced in: v20.1
Extracts the tail of a string following its last slash or backslash. This function is often used to extract the filename from a path.
Syntax
Arguments
expr
— A string expression. Backslashes must be escaped.String
Returned value
Returns the tail of the input string after its last slash or backslash. If the input string ends with a slash or backslash, the function returns an empty string. Returns the original string if there are no slashes or backslashes. String
Examples
Extract filename from Unix path
Extract filename from Windows path
String with no path separators
byteHammingDistance
Introduced in: v23.9
Calculates the hamming distance between two byte strings.
Syntax
Arguments
Returned value
Returns the Hamming distance between the two strings. UInt64
Examples
Usage example
compareSubstrings
Introduced in: v25.2
Compares two strings lexicographically.
Syntax
Arguments
s1
— The first string to compare.String
s2
— The second string to compare.String
s1_offset
— The position (zero-based) ins1
from which the comparison starts.UInt*
s2_offset
— The position (zero-based index) ins2
from which the comparison starts.UInt*
num_bytes
— The maximum number of bytes to compare in both strings. Ifs1_offset
(ors2_offset
) +num_bytes
exceeds the end of an input string,num_bytes
will be reduced accordingly.UInt*
Returned value
Returns:
-1
ifs1
[s1_offset
:s1_offset
+num_bytes
] <s2
[s2_offset
:s2_offset
+num_bytes
].0
ifs1
[s1_offset
:s1_offset
+num_bytes
] =s2
[s2_offset
:s2_offset
+num_bytes
].1
ifs1
[s1_offset
:s1_offset
+num_bytes
] >s2
[s2_offset
:s2_offset
+num_bytes
].Int8
Examples
Usage example
concat
Introduced in: v1.1
Concatenates the given arguments.
Arguments which are not of types String
or FixedString
are converted to strings using their default serialization.
As this decreases performance, it is not recommended to use non-String/FixedString arguments.
Syntax
Arguments
s1, s2, ...
— Any number of values of arbitrary type.Any
Returned value
Returns the String created by concatenating the arguments. If any of arguments is NULL
, the function returns NULL
. If there are no arguments, it returns an empty string. Nullable(String)
Examples
String concatenation
Number concatenation
concatAssumeInjective
Introduced in: v1.1
Like concat
but assumes that concat(s1, s2, ...) → sn
is injective,
i.e, it returns different results for different arguments.
Can be used for optimization of GROUP BY
.
Syntax
Arguments
s1, s2, ...
— Any number of values of arbitrary type.String
orFixedString
Returned value
Returns the string created by concatenating the arguments. If any of argument values is NULL
, the function returns NULL
. If no arguments are passed, it returns an empty string. String
Examples
Group by optimization
concatWithSeparator
Introduced in: v22.12
Concatenates the provided strings, separating them by the specified separator.
Syntax
Arguments
sep
— The separator to use.const String
orconst FixedString
exp1, exp2, ...
— Expression to be concatenated. Arguments which are not of typeString
orFixedString
are converted to strings using their default serialization. As this decreases performance, it is not recommended to use non-String/FixedString arguments.Any
Returned value
Returns the String created by concatenating the arguments. If any of the argument values is NULL
, the function returns NULL
. String
Examples
Usage example
concatWithSeparatorAssumeInjective
Introduced in: v22.12
Like concatWithSeparator
but assumes that concatWithSeparator(sep[,exp1, exp2, ... ]) → result
is injective.
A function is called injective if it returns different results for different arguments.
Can be used for optimization of GROUP BY
.
Syntax
Arguments
sep
— The separator to use.const String
orconst FixedString
exp1, exp2, ...
— Expression to be concatenated. Arguments which are not of typeString
orFixedString
are converted to strings using their default serialization. As this decreases performance, it is not recommended to use non-String/FixedString arguments.String
orFixedString
Returned value
Returns the String created by concatenating the arguments. If any of the argument values is NULL
, the function returns NULL
. String
Examples
Usage example
conv
Introduced in: v1.1
Converts numbers between different number bases.
The function converts a number from one base to another. It supports bases from 2 to 36. For bases higher than 10, letters A-Z (case insensitive) are used to represent digits 10-35.
This function is compatible with MySQL's CONV() function.
Syntax
Arguments
number
— The number to convert. Can be a string or numeric type. -from_base
— The source base (2-36). Must be an integer. -to_base
— The target base (2-36). Must be an integer.
Returned value
String representation of the number in the target base.
Examples
Convert decimal to binary
Convert hexadecimal to decimal
Convert with negative number
Convert binary to octal
convertCharset
Introduced in: v1.1
Returns string s
converted from the encoding from
to encoding to
.
Syntax
Arguments
s
— Input string.String
from
— Source character encoding.String
to
— Target character encoding.String
Returned value
Returns string s
converted from encoding from
to encoding to
. String
Examples
Usage example
damerauLevenshteinDistance
Introduced in: v24.1
Calculates the Damerau-Levenshtein distance between two byte strings.
Syntax
Arguments
Returned value
Returns the Damerau-Levenshtein distance between the two strings. UInt64
Examples
Usage example
decodeHTMLComponent
Introduced in: v23.9
Decodes HTML entities in a string to their corresponding characters.
Syntax
Arguments
s
— String containing HTML entities to decode.String
Returned value
Returns the string with HTML entities decoded. String
Examples
Usage example
decodeXMLComponent
Introduced in: v21.2
Decodes XML entities in a string to their corresponding characters.
Syntax
Arguments
s
— String containing XML entities to decode.String
Returned value
Returns the provided string with XML entities decoded. String
Examples
Usage example
editDistance
Introduced in: v23.9
Calculates the edit distance between two byte strings.
Syntax
Arguments
Returned value
Returns the edit distance between the two strings. UInt64
Examples
Usage example
editDistanceUTF8
Introduced in: v24.6
Calculates the edit distance between two UTF8 strings.
Syntax
Arguments
Returned value
Returns the edit distance between the two UTF8 strings. UInt64
Examples
Usage example
encodeXMLComponent
Introduced in: v21.1
Escapes characters to place string into XML text node or attribute.
Syntax
Arguments
s
— String to escape.String
Returned value
Returns the escaped string. String
Examples
Usage example
endsWith
Introduced in: v1.1
Checks whether a string ends with the provided suffix.
Syntax
Arguments
Returned value
Returns 1
if s
ends with suffix
, otherwise 0
. UInt8
Examples
Usage example
endsWithCaseInsensitive
Introduced in: v25.9
Checks whether a string ends with the provided case-insensitive suffix.
Syntax
Arguments
Returned value
Returns 1
if s
ends with case-insensitive suffix
, otherwise 0
. UInt8
Examples
Usage example
endsWithCaseInsensitiveUTF8
Introduced in: v25.9
Returns whether string s
ends with case-insensitive suffix
.
Assumes that the string contains valid UTF-8 encoded text.
If this assumption is violated, no exception is thrown and the result is undefined.
Syntax
Arguments
Returned value
Returns 1
if s
ends with case-insensitive suffix
, otherwise 0
. UInt8
Examples
Usage example
endsWithUTF8
Introduced in: v23.8
Returns whether string s
ends with suffix
.
Assumes that the string contains valid UTF-8 encoded text.
If this assumption is violated, no exception is thrown and the result is undefined.
Syntax
Arguments
Returned value
Returns 1
if s
ends with suffix
, otherwise 0
. UInt8
Examples
Usage example
extractTextFromHTML
Introduced in: v21.3
Extracts text content from HTML or XHTML.
This function removes HTML tags, comments, and script/style elements, leaving only the text content. It handles:
- Removal of all HTML/XML tags
- Removal of comments (
<!-- -->
) - Removal of script and style elements with their content
- Processing of CDATA sections (copied verbatim)
- Proper whitespace handling and normalization
Note: HTML entities are not decoded and should be processed with a separate function if needed.
Syntax
Arguments
html
— String containing HTML content to extract text from.String
Returned value
Returns the extracted text content with normalized whitespace. String
Examples
Usage example
firstLine
Introduced in: v23.7
Returns the first line of a multi-line string.
Syntax
Arguments
s
— Input string.String
Returned value
Returns the first line of the input string or the whole string if there are no line separators. String
Examples
Usage example
idnaDecode
Introduced in: v24.1
Returns the Unicode (UTF-8) representation (ToUnicode algorithm) of a domain name according to the Internationalized Domain Names in Applications (IDNA) mechanism.
In case of an error (e.g. because the input is invalid), the input string is returned.
Note that repeated application of idnaEncode()
and idnaDecode()
does not necessarily return the original string due to case normalization.
Syntax
Arguments
s
— Input string.String
Returned value
Returns a Unicode (UTF-8) representation of the input string according to the IDNA mechanism of the input value. String
Examples
Usage example
idnaEncode
Introduced in: v24.1
Returns the ASCII representation (ToASCII algorithm) of a domain name according to the Internationalized Domain Names in Applications (IDNA) mechanism. The input string must be UTF-encoded and translatable to an ASCII string, otherwise an exception is thrown.
No percent decoding or trimming of tabs, spaces or control characters is performed.
Syntax
Arguments
s
— Input string.String
Returned value
Returns an ASCII representation of the input string according to the IDNA mechanism of the input value. String
Examples
Usage example
initcap
Introduced in: v23.7
Converts the first letter of each word to upper case and the rest to lower case. Words are sequences of alphanumeric characters separated by non-alphanumeric characters.
Because initcap
converts only the first letter of each word to upper case you may observe unexpected behaviour for words containing apostrophes or capital letters.
This is a known behaviour and there are no plans to fix it currently.
Syntax
Arguments
s
— Input string.String
Returned value
Returns s
with the first letter of each word converted to upper case. String
Examples
Usage example
Example of known behavior for words containing apostrophes or capital letters
initcapUTF8
Introduced in: v23.7
Like initcap
, initcapUTF8
converts the first letter of each word to upper case and the rest to lower case.
Assumes that the string contains valid UTF-8 encoded text.
If this assumption is violated, no exception is thrown and the result is undefined.
This function does not detect the language, e.g. for Turkish the result might not be exactly correct (i/İ vs. i/I). If the length of the UTF-8 byte sequence is different for upper and lower case of a code point, the result may be incorrect for this code point.
Syntax
Arguments
s
— Input string.String
Returned value
Returns s
with the first letter of each word converted to upper case. String
Examples
Usage example
isValidASCII
Introduced in: v25.9
Returns 1 if the input String or FixedString contains only ASCII bytes (0x00–0x7F), otherwise 0.
Syntax
Arguments
- None. Returned value
Examples
isValidASCII
isValidUTF8
Introduced in: v20.1
Checks if the set of bytes constitutes valid UTF-8-encoded text.
Syntax
Arguments
s
— The string to check for UTF-8 encoded validity.String
Returned value
Returns 1
, if the set of bytes constitutes valid UTF-8-encoded text, otherwise 0
. UInt8
Examples
Usage example
jaroSimilarity
Introduced in: v24.1
Calculates the Jaro similarity between two byte strings.
Syntax
Arguments
Returned value
Returns the Jaro similarity between the two strings. Float64
Examples
Usage example
jaroWinklerSimilarity
Introduced in: v24.1
Calculates the Jaro-Winkler similarity between two byte strings.
Syntax
Arguments
Returned value
Returns the Jaro-Winkler similarity between the two strings. Float64
Examples
Usage example
left
Introduced in: v22.1
Returns a substring of string s
with a specified offset
starting from the left.
Syntax
Arguments
s
— The string to calculate a substring from.String
orFixedString
offset
— The number of bytes of the offset.(U)Int*
Returned value
Returns:
- For positive
offset
, a substring ofs
withoffset
many bytes, starting from the left of the string. - For negative
offset
, a substring ofs
withlength(s) - |offset|
bytes, starting from the left of the string. - An empty string if
length
is0
.String
Examples
Positive offset
Negative offset
leftPad
Introduced in: v21.8
Pads a string from the left with spaces or with a specified string (multiple times, if needed) until the resulting string reaches the specified length
.
Syntax
Arguments
string
— Input string that should be padded.String
length
— The length of the resulting string. If the value is smaller than the input string length, then the input string is shortened tolength
characters.(U)Int*
pad_string
— Optional. The string to pad the input string with. If not specified, then the input string is padded with spaces.String
Returned value
Returns a left-padded string of the given length. String
Examples
Usage example
leftPadUTF8
Introduced in: v21.8
Pads a UTF8 string from the left with spaces or a specified string (multiple times, if needed) until the resulting string reaches the given length.
Unlike leftPad
which measures the string length in bytes, the string length is measured in code points.
Syntax
Arguments
string
— Input string that should be padded.String
length
— The length of the resulting string. If the value is smaller than the input string length, then the input string is shortened tolength
characters.(U)Int*
pad_string
— Optional. The string to pad the input string with. If not specified, then the input string is padded with spaces.String
Returned value
Returns a left-padded string of the given length. String
Examples
Usage example
leftUTF8
Introduced in: v22.1
Returns a substring of a UTF-8-encoded string s
with a specified offset
starting from the left.
Syntax
Arguments
s
— The UTF-8 encoded string to calculate a substring from.String
orFixedString
offset
— The number of bytes of the offset.(U)Int*
Returned value
Returns:
- For positive
offset
, a substring ofs
withoffset
many bytes, starting from the left of the string.\n" - For negative
offset
, a substring ofs
withlength(s) - |offset|
bytes, starting from the left of the string.\n" - An empty string if
length
is 0.String
Examples
Positive offset
Negative offset
lengthUTF8
Introduced in: v1.1
Returns the length of a string in Unicode code points rather than in bytes or characters. It assumes that the string contains valid UTF-8 encoded text. If this assumption is violated, no exception is thrown and the result is undefined.
Syntax
Arguments
s
— String containing valid UTF-8 encoded text.String
Returned value
Length of the string s
in Unicode code points. UInt64
Examples
Usage example
lower
Introduced in: v1.1
Converts an ASCII string to lowercase.
Syntax
Arguments
s
— A string to convert to lowercase.String
Returned value
Returns a lowercase string from s
. String
Examples
Usage example
lowerUTF8
Introduced in: v1.1
Converts a string to lowercase, assuming that the string contains valid UTF-8 encoded text. If this assumption is violated, no exception is thrown and the result is undefined.
Syntax
Arguments
input
— Input string to convert to lowercase.String
Returned value
Returns a lowercase string. String
Examples
first
normalizeUTF8NFC
Introduced in: v21.11
Normalizes a UTF-8 string according to the NFC normalization form.
Syntax
Arguments
str
— UTF-8 encoded input string.String
Returned value
Returns the NFC normalized form of the UTF-8 string. String
Examples
Usage example
normalizeUTF8NFD
Introduced in: v21.11
Normalizes a UTF-8 string according to the NFD normalization form.
Syntax
Arguments
str
— UTF-8 encoded input string.String
Returned value
Returns the NFD normalized form of the UTF-8 string. String
Examples
Usage example
normalizeUTF8NFKC
Introduced in: v21.11
Normalizes a UTF-8 string according to the NFKC normalization form.
Syntax
Arguments
str
— UTF-8 encoded input string.String
Returned value
Returns the NFKC normalized form of the UTF-8 string. String
Examples
Usage example
normalizeUTF8NFKD
Introduced in: v21.11
Normalizes a UTF-8 string according to the NFKD normalization form.
Syntax
Arguments
str
— UTF-8 encoded input string.String
Returned value
Returns the NFKD normalized form of the UTF-8 string. String
Examples
Usage example
punycodeDecode
Introduced in: v24.1
Returns the UTF8-encoded plaintext of a Punycode-encoded string. If no valid Punycode-encoded string is given, an exception is thrown.
Syntax
Arguments
s
— Punycode-encoded string.String
Returned value
Returns the plaintext of the input value. String
Examples
Usage example
punycodeEncode
Introduced in: v24.1
Returns the Punycode representation of a string. The string must be UTF8-encoded, otherwise the behavior is undefined.
Syntax
Arguments
s
— Input value.String
Returned value
Returns a Punycode representation of the input value. String
Examples
Usage example
repeat
Introduced in: v20.1
Concatenates a string as many times with itself as specified.
Syntax
Arguments
Returned value
A string containing string s
repeated n
times. If n
is negative, the function returns the empty string. String
Examples
Usage example
reverseUTF8
Introduced in: v1.1
Reverses a sequence of Unicode code points in a string. Assumes that the string contains valid UTF-8 encoded text. If this assumption is violated, no exception is thrown and the result is undefined.
Syntax
Arguments
s
— String containing valid UTF-8 encoded text.String
Returned value
Returns a string with the sequence of Unicode code points reversed. String
Examples
Usage example
right
Introduced in: v22.1
Returns a substring of string s
with a specified offset
starting from the right.
Syntax
Arguments
s
— The string to calculate a substring from.String
orFixedString
offset
— The number of bytes of the offset.(U)Int*
Returned value
Returns:
- For positive
offset
, a substring ofs
withoffset
many bytes, starting from the right of the string. - For negative
offset
, a substring ofs
withlength(s) - |offset|
bytes, starting from the right of the string. - An empty string if
length
is0
.String
Examples
Positive offset
Negative offset
rightPad
Introduced in: v21.8
Pads a string from the right with spaces or with a specified string (multiple times, if needed) until the resulting string reaches the specified length
.
Syntax
Arguments
string
— Input string that should be padded.String
length
— The length of the resulting string. If the value is smaller than the input string length, then the input string is shortened tolength
characters.(U)Int*
pad_string
— Optional. The string to pad the input string with. If not specified, then the input string is padded with spaces.String
Returned value
Returns a right-padded string of the given length. String
Examples
Usage example
rightPadUTF8
Introduced in: v21.8
Pads the string from the right with spaces or a specified string (multiple times, if needed) until the resulting string reaches the given length.
Unlike rightPad
which measures the string length in bytes, the string length is measured in code points.
Syntax
Arguments
string
— Input string that should be padded.String
length
— The length of the resulting string. If the value is smaller than the input string length, then the input string is shortened tolength
characters.(U)Int*
pad_string
— Optional. The string to pad the input string with. If not specified, then the input string is padded with spaces.String
Returned value
Returns a right-padded string of the given length. String
Examples
Usage example
rightUTF8
Introduced in: v22.1
Returns a substring of UTF-8 encoded string s
with a specified offset
starting from the right.
Syntax
Arguments
s
— The UTF-8 encoded string to calculate a substring from.String
orFixedString
offset
— The number of bytes of the offset.(U)Int*
Returned value
Returns:
- For positive
offset
, a substring ofs
withoffset
many bytes, starting from the right of the string. - For negative
offset
, a substring ofs
withlength(s) - |offset|
bytes, starting from the right of the string. - An empty string if
length
is0
.String
Examples
Positive offset
Negative offset
soundex
Introduced in: v23.4
Returns the Soundex code of a string.
Syntax
Arguments
s
— Input string.String
Returned value
Returns the Soundex code of the input string. String
Examples
Usage example
space
Introduced in: v23.5
Concatenates a space (
) as many times with itself as specified.
Syntax
Arguments
n
— The number of times to repeat the space.(U)Int*
Returned value
Returns astring containing a space repeated n
times. If n <= 0
, the function returns the empty string. String
Examples
Usage example
sparseGrams
Introduced in: v25.5
Finds all substrings of a given string that have a length of at least n
,
where the hashes of the (n-1)-grams at the borders of the substring
are strictly greater than those of any (n-1)-gram inside the substring.
Uses CRC32
as a hash function.
Syntax
Arguments
s
— An input string.String
min_ngram_length
— Optional. The minimum length of extracted ngram. The default and minimal value is 3.UInt*
max_ngram_length
— Optional. The maximum length of extracted ngram. The default value is 100. Should be not less thanmin_ngram_length
.UInt*
Returned value
Returns an array of selected substrings. Array(String)
Examples
Usage example
sparseGramsHashes
Introduced in: v25.5
Finds hashes of all substrings of a given string that have a length of at least n
,
where the hashes of the (n-1)-grams at the borders of the substring
are strictly greater than those of any (n-1)-gram inside the substring.
Uses CRC32
as a hash function.
Syntax
Arguments
s
— An input string.String
min_ngram_length
— Optional. The minimum length of extracted ngram. The default and minimal value is 3.UInt*
max_ngram_length
— Optional. The maximum length of extracted ngram. The default value is 100. Should be not less thanmin_ngram_length
.UInt*
Returned value
Returns an array of selected substrings CRC32 hashes. Array(UInt32)
Examples
Usage example
sparseGramsHashesUTF8
Introduced in: v25.5
Finds hashes of all substrings of a given UTF-8 string that have a length of at least n
, where the hashes of the (n-1)-grams at the borders of the substring are strictly greater than those of any (n-1)-gram inside the substring.
Expects UTF-8 string, throws an exception in case of invalid UTF-8 sequence.
Uses CRC32
as a hash function.
Syntax
Arguments
s
— An input string.String
min_ngram_length
— Optional. The minimum length of extracted ngram. The default and minimal value is 3.UInt*
max_ngram_length
— Optional. The maximum length of extracted ngram. The default value is 100. Should be not less thanmin_ngram_length
.UInt*
Returned value
Returns an array of selected UTF-8 substrings CRC32 hashes. Array(UInt32)
Examples
Usage example
sparseGramsUTF8
Introduced in: v25.5
Finds all substrings of a given UTF-8 string that have a length of at least n
, where the hashes of the (n-1)-grams at the borders of the substring are strictly greater than those of any (n-1)-gram inside the substring.
Expects a UTF-8 string, throws an exception in case of an invalid UTF-8 sequence.
Uses CRC32
as a hash function.
Syntax
Arguments
s
— An input string.String
min_ngram_length
— Optional. The minimum length of extracted ngram. The default and minimal value is 3.UInt*
max_ngram_length
— Optional. The maximum length of extracted ngram. The default value is 100. Should be not less thanmin_ngram_length
.UInt*
Returned value
Returns an array of selected UTF-8 substrings. Array(String)
Examples
Usage example
startsWith
Introduced in: v1.1
Checks whether a string begins with the provided string.
Syntax
Arguments
Returned value
Returns 1
if s
starts with prefix
, otherwise 0
. UInt8
Examples
Usage example
startsWithCaseInsensitive
Introduced in: v25.9
Checks whether a string begins with the provided case-insensitive string.
Syntax
Arguments
Returned value
Returns 1
if s
starts with case-insensitive prefix
, otherwise 0
. UInt8
Examples
Usage example
startsWithCaseInsensitiveUTF8
Introduced in: v25.9
Checks if a string starts with the provided case-insensitive prefix. Assumes that the string contains valid UTF-8 encoded text. If this assumption is violated, no exception is thrown and the result is undefined.
Syntax
Arguments
Returned value
Returns 1
if s
starts with case-insensitive prefix
, otherwise 0
. UInt8
Examples
Usage example
startsWithUTF8
Introduced in: v23.8
Checks if a string starts with the provided prefix. Assumes that the string contains valid UTF-8 encoded text. If this assumption is violated, no exception is thrown and the result is undefined.
Syntax
Arguments
Returned value
Returns 1
if s
starts with prefix
, otherwise 0
. UInt8
Examples
Usage example
stringBytesEntropy
Introduced in: v25.6
Calculates Shannon's entropy of byte distribution in a string.
Syntax
Arguments
s
— The string to analyze.String
Returned value
Returns Shannon's entropy of byte distribution in the string. Float64
Examples
Usage example
stringBytesUniq
Introduced in: v25.6
Counts the number of distinct bytes in a string.
Syntax
Arguments
s
— The string to analyze.String
Returned value
Returns the number of distinct bytes in the string. UInt16
Examples
Usage example
stringJaccardIndex
Introduced in: v23.11
Calculates the Jaccard similarity index between two byte strings.
Syntax
Arguments
Returned value
Returns the Jaccard similarity index between the two strings. Float64
Examples
Usage example
stringJaccardIndexUTF8
Introduced in: v23.11
Like stringJaccardIndex
but for UTF8-encoded strings.
Syntax
Arguments
Returned value
Returns the Jaccard similarity index between the two UTF8 strings. Float64
Examples
Usage example
substring
Introduced in: v1.1
Returns the substring of a string s
which starts at the specified byte index offset
.
Byte counting starts from 1 with the following logic:
- If
offset
is0
, an empty string is returned. - If
offset
is negative, the substring startspos
characters from the end of the string, rather than from the beginning.
An optional argument length
specifies the maximum number of bytes the returned substring may have.
Syntax
Arguments
s
— The string to calculate a substring from.String
orFixedString
orEnum
offset
— The starting position of the substring ins
.(U)Int*
length
— Optional. The maximum length of the substring.(U)Int*
Returned value
Returns a substring of s
with length
many bytes, starting at index offset
. String
Examples
Basic usage
substringIndex
Introduced in: v23.7
Returns the substring of s
before count
occurrences of the delimiter delim
, as in Spark or MySQL.
Syntax
Arguments
s
— The string to extract substring from.String
delim
— The character to split.String
count
— The number of occurrences of the delimiter to count before extracting the substring. If count is positive, everything to the left of the final delimiter (counting from the left) is returned. If count is negative, everything to the right of the final delimiter (counting from the right) is returned.UInt
orInt
Returned value
Returns a substring of s
before count
occurrences of delim
. String
Examples
Usage example
substringIndexUTF8
Introduced in: v23.7
Returns the substring of s
before count
occurrences of the delimiter delim
, specifically for Unicode code points.
Assumes that the string contains valid UTF-8 encoded text.
If this assumption is violated, no exception is thrown and the result is undefined.
Syntax
Arguments
s
— The string to extract substring from.String
delim
— The character to split.String
count
— The number of occurrences of the delimiter to count before extracting the substring. If count is positive, everything to the left of the final delimiter (counting from the left) is returned. If count is negative, everything to the right of the final delimiter (counting from the right) is returned.UInt
orInt
Returned value
Returns a substring of s
before count
occurrences of delim
. String
Examples
UTF8 example
substringUTF8
Introduced in: v1.1
Returns the substring of a string s
which starts at the specified byte index offset
for Unicode code points.
Byte counting starts from 1
with the following logic:
- If
offset
is0
, an empty string is returned. - If
offset
is negative, the substring startspos
characters from the end of the string, rather than from the beginning.
An optional argument length
specifies the maximum number of bytes the returned substring may have.
This function assumes that the string contains valid UTF-8 encoded text. If this assumption is violated, no exception is thrown and the result is undefined.
Syntax
Arguments
s
— The string to calculate a substring from.String
orFixedString
orEnum
offset
— The starting position of the substring ins
.Int
orUInt
length
— The maximum length of the substring. Optional.Int
orUInt
Returned value
Returns a substring of s
with length
many bytes, starting at index offset
. String
Examples
Usage example
toValidUTF8
Introduced in: v20.1
Converts a string to valid UTF-8 encoding by replacing any invalid UTF-8 characters with the replacement character �
(U+FFFD).
When multiple consecutive invalid characters are found, they are collapsed into a single replacement character.
Syntax
Arguments
s
— Any set of bytes represented as the String data type object.String
Returned value
Returns a valid UTF-8 string. String
Examples
Usage example
trimBoth
Introduced in: v20.1
Removes the specified characters from the start and end of a string. By default, removes common whitespace (ASCII) characters.
Syntax
Arguments
s
— String to trim.String
trim_characters
— Optional. Characters to trim. If not specified, common whitespace characters are removed.String
Returned value
Returns the string with specified characters trimmed from both ends. String
Examples
Usage example
trimLeft
Introduced in: v20.1
Removes the specified characters from the start of a string. By default, removes common whitespace (ASCII) characters.
Syntax
Arguments
input
— String to trim.String
trim_characters
— Optional. Characters to trim. If not specified, common whitespace characters are removed.String
Returned value
Returns the string with specified characters trimmed from the left. String
Examples
Usage example
trimRight
Introduced in: v20.1
Removes the specified characters from the end of a string. By default, removes common whitespace (ASCII) characters.
Syntax
Arguments
s
— String to trim.String
trim_characters
— Optional characters to trim. If not specified, common whitespace characters are removed.String
Returned value
Returns the string with specified characters trimmed from the right. String
Examples
Usage example
tryBase32Decode
Introduced in: v25.6
Accepts a string and decodes it using Base32 encoding scheme.
Syntax
Arguments
encoded
— String column or constant to decode. If the string is not valid Base32-encoded, returns an empty string in case of error.String
Returned value
Returns a string containing the decoded value of the argument. String
Examples
Usage example
tryBase58Decode
Introduced in: v22.10
Like base58Decode
, but returns an empty string in case of error.
Syntax
Arguments
encoded
— String column or constant. If the string is not valid Base58-encoded, returns an empty string in case of error.String
Returned value
Returns a string containing the decoded value of the argument. String
Examples
Usage example
tryBase64Decode
Introduced in: v18.16
Like base64Decode
, but returns an empty string in case of error.
Syntax
Arguments
encoded
— String column or constant to decode. If the string is not valid Base64-encoded, returns an empty string in case of error.String
Returned value
Returns a string containing the decoded value of the argument. String
Examples
Usage example
tryBase64URLDecode
Introduced in: v18.16
Like base64URLDecode
, but returns an empty string in case of error.
Syntax
Arguments
encoded
— String column or constant to decode. If the string is not valid Base64-encoded, returns an empty string in case of error.String
Returned value
Returns a string containing the decoded value of the argument. String
Examples
Usage example
tryIdnaEncode
Introduced in: v24.1
Returns the Unicode (UTF-8) representation (ToUnicode algorithm) of a domain name according to the Internationalized Domain Names in Applications (IDNA) mechanism. In case of an error it returns an empty string instead of throwing an exception.
Syntax
Arguments
s
— Input string.String
Returned value
Returns an ASCII representation of the input string according to the IDNA mechanism of the input value, or empty string if input is invalid. String
Examples
Usage example
tryPunycodeDecode
Introduced in: v24.1
Like punycodeDecode
but returns an empty string if no valid Punycode-encoded string is given.
Syntax
Arguments
s
— Punycode-encoded string.String
Returned value
Returns the plaintext of the input value, or empty string if input is invalid. String
Examples
Usage example
upper
Introduced in: v1.1
Converts the ASCII Latin symbols in a string to uppercase.
Syntax
Arguments
s
— The string to convert to uppercase.String
Returned value
Returns an uppercase string from s
. String
Examples
Usage example
upperUTF8
Introduced in: v1.1
Converts a string to uppercase, assuming that the string contains valid UTF-8 encoded text. If this assumption is violated, no exception is thrown and the result is undefined.
This function doesn't detect the language, e.g. for Turkish the result might not be exactly correct (i/İ vs. i/I).
If the length of the UTF-8 byte sequence is different for upper and lower case of a code point (such as ẞ
and ß
), the result may be incorrect for that code point.
Syntax
Arguments
s
— A string type.String
Returned value
A String data type value. String
Examples
Usage example