Data types reference

Identifier

Count Unique, First, Last

Yes

matches/does not match
is empty/is not empty

No

Integer* / Decimal

Count Unique, First, Last, Min, Max, Sum, Average, Median, Percentile

Defaults to No, can be configured to Yes by admin

is one of/is not one of
is less than (<)
is greater than (>)
is less than or equal to (<=)
is greater than or equal to (>=)

No

Integer Set / Decimal Set

Defaults to No, can be configured to Yes by admin

matches/does not match [column/value]
is empty/is not empty

No

String

Count Unique, First, Last

Yes

matches/does not match
text contains/does not contain
starts with/does not start with
ends with/does not end with
is empty/is not empty
text matches regular expression

Yes

String Set

Count Unique

Yes

string value matches/does not match
set contains/does not contain text
starts with/does not start with
ends with/does not end with
is empty/is not empty
text matches regular expression

Yes

Time

Count Unique, First, Last, Min, Max, Sum, Average, Median, Percentile 

No

is one of, is not one of, is less than (<), is greater than (>), is less than or equal to (<=), is greater than or equal to (>=)

No

IP Address

String

city, region, country, continent

URL

N/A (not loaded into Scuba by default)

scheme, hostname, path, filename, query, params, fragment

User Agent

N/A (not loaded into Scuba)

browser, device, platform, browser_majorver, browser_minorver, browser_patchver

Flatten Nested JSON Objects

{"column": {"a": 1, "b": "xxx"}}

{"column.a": 1, "column.b": "xxx"}

Shred Arrays of JSON Objects

{"column": [{"a": 1,"b": "zzz"}, {"a": 2,"b": "yyy"}]}

{"column.a": [1,2], "column.b": ["zzz","yyy"]}

"1234"

Integer values

"abc123"

Integer (converted to base 10), for values up to 16 hex digits

"abcdef-124"

Identifier, for values of 17 digits or more. The value can include hyphens. 

Detect Time

{"abc" : 1448933490}

Time

Scuba will attempt to interpret JSON ints as epoch timestamps in one of (microseconds, milliseconds, seconds). 

Detect Integer

{"abc" : 12345}

Integer

Simple JSON ints are interpreted as Integers by Scuba.

Detect Decimal

{"abc" : 12345.98}

Decimal

Detect Integer Set

{"abc" : [12345, 245, 99834]}

Integer Set

Detect Decimal Set

{"abc" : [12345.98, 245.2, 99834]}

Decimal Set

Detect Time From JSON String

{"abc" : "2015-11-30 08:09:12"}

Time

Scuba will attempt to interpret JSON strings as timestamps using approximately 40 different format strings (including ISO-8601).  

Detect Identifier from JSON String

{"abc" : "e41249ed-2398-4c29-a6fa-ee81116dd302"}

Identifier

Scuba will attempt to interpret JSON strings containing 16 or more characters as hexadecimal identifiers, including some common uuid formats. Note that non-hex characters (like hyphens or dots) are stripped out of the resulting data.

Detect Integer From JSON String

{"abc" : "12345"}

Integer

Detect Decimal From JSON String

{"abc" : "12345.98"}

Decimal

Detect Decimal From JSON String With Dollar Sign

{"abc" : "$12,345.98"}

Decimal

Note that dollar signs and commas are stripped out of the resulting data.

Detect String Set

{"abc" : ["hello", "goodbye", "nice", "to", "see", "you"]}

String Set

Detect URL

{"abc" : "http://www.site.com/landing/"}

URL

Detect IP Address

{"abc" : "127.0.0.1"}

IP Address

Detect User Agent

{"abc" : "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_0)"}

User Agent

Scuba will attempt to interpret JSON strings as User Agents using a regex matching scheme.

strict_number_detection

1

Do not interpret JSON strings as Integer or Decimal. 

force_url_to_string

1

Do not interpret JSON strings as URLs.

add_full_and_parsed_url

1

In addition to expanding the pieces of the URL, also store the original full URL as a String column. (This takes extra space.)

force_geo_to_string

1

Do not interpret JSON strings as IP addresses.

force_useragent_to_string

1

Do not interpret JSON strings as User Agents.

force_hexn_to_string

1

Do not interpret JSON strings as hex identifiers.