zotonic_stdlib

Zotonic standard library - a library with hopefully useful functions

Extracted from Zotonic, the Erlang CMS http://zotonic.com/

Installation

Due to historical reasons the library is called z_stdlib. It should be checked out as zotonic_stdlib:

git clone https://github.com/zotonic/z_stdlib.git zotonic_stdlib

The library is also on Hex https://hex.pm/packages/zotonic_stdlib

In your rebar.config file, use Hex (for rebar3):

{deps, [
    zotonic_stdlib
]}.

Or, directly git (rebar2):

{deps, [
    {zotonic_stdlib, ".*", {git, "https://github.com/zotonic/z_stdlib.git", "master"}}
]}.

Modules

The following modules are provided:

z_convert

Conversion routines for:

• all common data types (binary, boolean, integer, etc.).
• dates: to_utc, to_local, iso8601 parsing
• json, convert to simple json constructs
• IP address conversions

z_css

Strict CSS parser and sanitizer.

z_cssmin

CSS minifier

z_dateformat

Routines for formatting dates. Example:

z_dateformat:format({{2008,12,10},{15,30,0}}, "Y-m-d H:i:s", [ {tz, "GMT"} ]).

z_email_dnsbl

Check IP addresses for their presence on DNS block- or allowlists.

z_email_utils

Test if an email address is syntactically valid. Extract email address from a text.

z_filelib

Convenience function to make nested directories.

z_html

HTML routines like escape, unescape, sanitize, etc.

z_ip_address

Match an IP address against lists, check if an IP address is a non routable LAN address.

z_jsmin

JavaScript minimizer, removes spaces, comments and newlines.

z_string

String routines. To uppercase, lowercase, truncate, trim, utf8 checks, and more.

z_svg

SVG sanitizer.

z_tempfile

Temporary file routines. Make tempfiles, including watch dog process to clean up the file.

z_ubf

UBF-A encoding and decoding routines. Safe for atoms and maximum memory size.

Maps are encoded as a UBF-A list of {Key, Value} tuples followed by the type tag `map`. For example, #{a => 1, b => 2} is encoded as #{'b',2}&{'a',1}&`map`$. The pair order is canonicalized by Erlang term order before encoding. Decoding accepts the pair list and converts it with maps:from_list/1; duplicate keys follow the same last-value-wins semantics.

z_url

URL encode, decode and more.

z_url_fetch

Fetch the first N bytes of a URL. Protection against too large return body.

Redirects are followed by default. Use {autoredirect, false} to return the redirect response without contacting its destination. Cross-origin redirects (scheme, host or effective port changes) default to {authorization_redirect, none}: caller credentials are removed and the callback receives {authorization, none}. The callback can replace this default with destination-specific credentials. Use {authorization_redirect, error} to reject cross-origin redirects carrying explicit authorization with {error, authorization_redirect}. Same-origin redirects with explicit authorization still require HTTPS. An explicit caller option {authorization, none} remains anonymous throughout the request and is never sent as an HTTP header.

For destination-specific credentials, supply {options_fun, Fun}. Before each HTTP request, Fun(Method, Url, Options) receives the method, normalized binary URL and the sanitized caller options without options_fun. Return the request options for that destination; callback-generated credentials are not carried into the next request. The caller's explicit {authorization, none} takes precedence over callback results. Keep transfer settings (device and max_length) in the original options. Relative redirects, redirect limits and streamed responses are handled by the fetcher.

OptionsFun = fun(_Method, Url, Options) ->
    case uri_string:parse(Url) of
        #{scheme := <<"https">>, host := <<"api.example.com">>} ->
            [{authorization, <<"Bearer ", Token/binary>>} | Options];
        _ -> Options
    end
end,
z_url_fetch:fetch(Url, [{options_fun, OptionsFun}]).

z_url_metadata

Fetch URL, extract metadata like mime type, title, description, images etc.

The returned metadata can be queried with z_url_metadata:p/2, for example:

{ok, Metadata} = z_url_metadata:fetch(Url),
Title = z_url_metadata:p(title, Metadata),
Image = z_url_metadata:p(image, Metadata).

JSON-LD data is available through the json_ld property:

JsonLDs = z_url_metadata:p(json_ld, Metadata).

This list contains decoded JSON-LD maps from:

• <script type="application/ld+json"> blocks.
• HTML microdata using itemscope, itemtype, itemid, and itemprop.

Microdata is normalized to JSON-LD-style maps. Schema.org item types use <<"https://schema.org">> as <<"@context">> and the local type name as <<"@type">>. Scoped items with an itemid are added as separate maps in the list; scoped items without an itemid are embedded as property values of their parent item. Repeated properties are returned as lists.

Tests

To run the test set:

make test

All tests should pass.