Module markup

Convert wrapped plain text to unwrapped.

Two or more newlines are considered as paragraph boundaries and left in, while all other newlines are removed. Whitespace in the text is simplified throughout.

Parameters:

• text (string) - text to unwrap

Returns: string

unwrapped text

Convert any XML-like markup to plain text.

By default, all tags in the text are replaced with a single space; entities, unless one of the XML default (<, >, &, ", '), are left untouched; all whitespace groups are simplified to single space and leading and trailing removed.

If only a particular subset of tags should be taken into account, it can be specified by the tags parameter, as a sequence of tag names (the sequence is internally converted to set before processing).

If a tag should be replaced with a special sequence of characters (either opening or closing tag), or the text wrapped by it replaced too, this can be specified by the subs parameter. It is a dictionary of 3-tuples by tag name, which tells what to replace with the opening tag, the closing tag, and the wrapped text. For example, to replace <i>foobar</i> with /foobar/, the dictionary entry would be {"i": ("/", "/", None)} (where final None states not to touch the wrapped text); to replace <code>...</code> with @@@ (i.e. remove code segment completely but leave in a marker that there was something), the entry is {"code": ("", "", "@@@")}. The replacement for the wrapped text can also be a function, taking a string and returning a string. Note that whitespace is automatically simplified, so if whitespace given by the replacements should be exactly preserved, use WS_* string constants in place of corresponding whitespace characters.

To have some entities other than the XML default replaced with proper values, a dictionary of known entities with values may be provided using the ents parameter.

Whitespace can be preserved within some elements, as given by their tags in the keepws sequence.

Some elements may be completely removed, as given by the ignels sequence. Each element of the sequence should either be a tag, or a (tag, type) tuple, where type is the value of the type argument to element, if any.

It is assumed that the markup is well-formed, and if it is not the result is undefined; but best attempt at conversion is made.

There are several other functions in this module which deal with well known markups, such that it is not necessary to use this function with tags, subs, or ents manually specified.

If you only want to resolve entities from a known set, instead of calling this function with empty tags and entities given in ents, consider using the more powerfull pology.resolve.resolve_entities.

Parameters:

• text (string) - markup text to convert to plain
• tags (sequence of strings) - known tags
• subs (dictionary of 3-tuples) - replacement specification
• ents (dictionary) - known entities and their values
• keepws (sequence of strings) - tags of elements in which to preserve whitespace
• ignels (sequence of strings and (string, string) tuples) - tags or tag/types or elements to completely remove

Returns: string

plain text version

Convert HTML markup to plain text.

Parameters:

• text (string) - HTML text to convert to plain

Returns: string

plain text version

Convert Qt rich-text markup to plain text.

Parameters:

• text (string) - Qt rich text to convert to plain

Returns: string

plain text version

Convert KUIT markup to plain text.

Parameters:

• text (string) - KUIT text to convert to plain

Returns: string

plain text version

Convert KDE4 GUI markup to plain text.

KDE4 GUI texts may contain both Qt rich-text and KUIT markup, even mixed in the same text. Note that the conversion cannot be achieved, in general, by first converting Qt rich-text, and then KUIT, or vice versa. For example, if the text has < entity, after first conversion it will become plain <, and interfere with second conversion.

Parameters:

• text (string) - KDE4 text to convert to plain

Returns: string

plain text version

Convert Docbook 4.x markup to plain text.

Parameters:

• text (string) - Docbook text to convert to plain

Returns: string

plain text version

Collect lightweight XML format specification, level 1.

Level 1 specification is the dictionary of all known tags, with allowed attributes and subtags for each.

File of the level 1 specification is in the following format:

   # A comment.
   # Tag with unconstrained attributes and subtags:
   tagA;
   # Tag with constrained attributes and unconstrained subtags:
   tagF : attr1 attr2 ...;
   # Tag with unconstrained attributes and constrained subtags:
   tagF > stag1 stag2 ...;
   # Tag with constrained attributes and subtags:
   tagF : attr1 attr2 ... > stag1 stag2 ...;
   # Tag with no attributes and unconstrained subtags:
   tagA :;
   # Tag with unconstrained attributes and no subtags:
   tagA >;
   # Tag with no attributes and no subtags:
   tagA :>;
   # Attribute value constrained by a regular expression:
   .... attr1=/^(val1|val2|val3)$/i ...
   # Reserved dummy tag specifying attributes common to all tags:
   pe-common-attrib : attrX attrY;

The specification can contain a dummy tag named pe-common-attrib, stating attributes which are common to all tags, instead of having to list them with each and every tag. To make an attribute mandatory, it's name should be prefixed by exclamation sign (!).

Specification file must be UTF-8 encoded.

Parameters:

• specpath (string) - path to level 1 specification file

Returns: dict

level 1 specification

Validate XML markup in text against level1 specification.

Text is not required to have a top tag; if it does not, a dummy one will be assigned to assure that the check passes.

If spec is None, text is only checked to be well-formed.

If ents are None, entities in the text are ignored by the check; otherwise, an entity not belonging to the known set is considered erroneous. Default XML entities (<, >, &, ", ') are automatically added to the set of known entities.

Tag and attribute names can be made case-insensitive by setting casesens to False.

If text is a part of user interface, and the environment may use the literal ampersand as accelerator marker, it can be allowed to pass the check by setting accelamp to True.

Text can be one or more entity definitions of the form <!ENTITY ...>, when special check is applied.

The result of the check is list of erroneous spans in the text, each given by start and end index (in Python standard semantics), and the error description, packed in a tuple. If there are no errors, empty list is returned. Reported spans need not be formally complete with respect to the error location, but are heuristically determined to be short and provide good visual indication of what triggers the error.

Parameters:

• text (string) - text to check
• spec (level1 specification) - markup definition
• xmlfmt (string) - name of the particular XML format (for error messages)
• ents (sequence) - set of known entities
• casesens (bool) - whether tag names are case-insensitive
• accelamp (bool) - whether to allow ampersand as accelerator marker

Returns: list of (int, int, string) tuples

erroneous spans in the text

Check general XML markup in translation [hook factory].

Text is only checked to be well-formed XML, and possibly also whether encountered entities are defined. Markup errors are reported to stdout.

msgstr can be either checked only if the msgid is valid itself, or regardless of the validity of the original. This is governed by the strict parameter.

Entities in addition to XML's default (<, etc.) may be provided using the entities parameter. Several types of values with different semantic are possible:

• if entities is None, unknown entities are ignored on checking
• if string, it is understood as a general function evaluation request, and its result expected to be (name, value) dictionary-like object
• otherwise, entities is considered to be a (name, value) dictionary

If a message has sieve flag no-check-markup, the check is skipped for that message. If one or several markup keywords are given as mkeyw parameter, check is skipped for all messages in a catalog which does not report one of the given keywords by its markup() method. See set_markup() for list of markup keywords recognized at the moment.

Parameters:

• strict (bool) - whether to require valid msgstr even if msgid is not
• entities (None, dict, or string) - additional entities to consider as known
• mkeyw (string or list of strings) - markup keywords for taking catalogs into account

Returns: (msgstr, msg, cat) -> numerr

type S3C hook

Like check_xml, except that erroneous spans are returned instead of reporting problems to stdout [hook factory].

Returns: (msgstr, msg, cat) -> spans

type V3C hook

Validate Docbook 4.x markup in text against level1 specification.

Markup definition is extended to include <placeholder-N/> elements, which xml2po uses to segment text when extracting markup documents into PO templates.

See validate_xml_l1 for description of the ents parameter and the return value.

Parameters:

• text (string) - text to check
• ents (sequence) - set of known entities (in addition to default)

Returns: list of (int, int, string) tuples

erroneous spans in the text

Check XML markup in translations of Docbook 4.x catalogs [hook factory].

See check_xml for description of parameters.

Returns: (msgstr, msg, cat) -> numerr

type S3C hook

Like check_docbook4, except that erroneous spans are returned instead of reporting problems to stdout [hook factory].

Returns: (msgstr, msg, cat) -> spans

type V3C hook

Check for any known problem in translation in messages in Docbook 4.x catalogs [hook factory].

Currently performed checks:

• Docbook markup
• cross-message insertion placeholders

See check_xml for description of parameters.

Returns: (msg, cat) -> parts

type V4A hook

Validate HTML markup in text against level1 specification.

At the moment, this function can only check HTML markup if well-formed in the XML sense, although HTML allows omission of some closing tags.

See validate_xml_l1 for description of the ents parameter and the return value.

Parameters:

• text (string) - text to check
• ents (sequence) - set of known entities (in addition to default)

Returns: list of (int, int, string) tuples

erroneous spans in the text

Check HTML markup in translations [hook factory].

See check_xml for description of parameters. See notes on checking HTML markup to validate_html_l1.

Returns: (msgstr, msg, cat) -> numerr

type S3C hook

Like check_html, except that erroneous spans are returned instead of reporting problems to stdout [hook factory].

Returns: (msgstr, msg, cat) -> spans

type V3C hook

Validate Qt rich-text markup in text against level1 specification.

At the moment, this function can only check Qt rich-text if well-formed in the XML sense, although Qt rich-text allows HTML-type omission of closing tags.

See validate_xml_l1 for description of the ents parameter and the return value.

Parameters:

• text (string) - text to check
• ents (sequence) - set of known entities (in addition to default)

Returns: list of (int, int, string) tuples

erroneous spans in the text

Check Qt rich-text markup in translations [hook factory].

See check_xml for description of parameters. See notes on checking Qt rich-text to validate_qtrich_l1.

Returns: (msgstr, msg, cat) -> numerr

type S3C hook

Like check_qtrich, except that erroneous spans are returned instead of reporting problems to stdout [hook factory].

Returns: (msgstr, msg, cat) -> spans

type V3C hook

Validate KUIT markup in text against level1 specification.

KUIT is the semantic markup for user interface in KDE4.

See validate_xml_l1 for description of the ents parameter and the return value.

Parameters:

• text (string) - text to check
• ents (sequence) - set of known entities (in addition to default)

Returns: list of (int, int, string) tuples

erroneous spans in the text

Validate markup in texts used in KDE4 GUI.

KDE4 GUI texts may contain both Qt rich-text and KUIT markup, even mixed in the same text.

See validate_xml_l1 for description of the ents parameter and the return value.

Parameters:

• text (string) - text to check
• ents (sequence) - set of known entities (in addition to default)

Returns: list of (int, int, string) tuples

erroneous spans in the text

Check XML markup in translations of KDE4 UI catalogs [hook factory].

See check_xml for description of parameters.

Returns: (msgstr, msg, cat) -> numerr

type S3C hook

Like check_kde4, except that erroneous spans are returned instead of reporting problems to stdout [hook factory].

Returns: (msgstr, msg, cat) -> spans

type V3C hook

Validate Pango markup in text against level1 specification.

See validate_xml_l1 for description of the ents parameter and the return value.

Parameters:

• text (string) - text to check
• ents (sequence) - set of known entities (in addition to default)

Returns: list of (int, int, string) tuples

erroneous spans in the text

Check XML markup in translations of Pango UI catalogs [hook factory].

See check_xml for description of parameters.

Returns: (msgstr, msg, cat) -> numerr

type S3C hook

Like check_pango, except that erroneous spans are returned instead of reporting problems to stdout [hook factory].

Returns: (msgstr, msg, cat) -> spans

type V3C hook

Convert numeric XML entity to character.

Numeric XML entities can be decimal, &#DDDD;, or hexadecimal, &#xHHHH;, where D and H stand for number system's digits. 4 digits is the maximum, but there can be less.

If the entity cannot be converted to a character, for whatever reason, None is reported.

Parameters:

• nument (string) - numeric entity, with or without & and ;

Returns: string or None

character represented by the entity

Check whether XML-like entities in the text are among known.

The text does not have to be XML markup as such. No XML parsing is performed, only the raw search for XML-like entities.

Parameters:

• text (string) - text with entities to check
• ents (sequence) - known entities
• default (bool) - whether default XML entities are allowed (&, etc.)
• numeric (bool) - whether numeric character entities are allowed

Returns: list of (int, int, string) tuples

erroneous spans in the text

Check existence of XML entities in translations [hook factory].

See check_xml for description of parameters strict, entities, and mkeyw. See validate_xmlents for parameters default and numeric, and for general notes on checking entities.

Returns: (msgstr, msg, cat) -> numerr

type S3C hook

Like check_xmlents, except that erroneous spans are returned instead of reporting problems to stdout [hook factory].

Returns: (msgstr, msg, cat) -> spans

type V3C hook

Check if sets of <placeholder-N/> elements are matching between original and translated text.

<placeholder-N/> elements are added into text by xml2po, for finer segmentation of markup documents extracted into PO templates.

See validate_xml_l1 for description of the return value.

Parameters:

• orig (string) - original text
• trans (string) - translated text

Returns: list of (int, int, string) tuples

erroneous spans in translation

xml_entities

Value:

{'amp': '&', 'apos': '\'', 'gt': '>', 'lt': '<', 'quot': '"'}

html_entities

Value:

{u'AElig': u'Æ', u'Aacute': u'Á', u'Acirc': u'Â', u'Agrave': u'À', u'Aring': u'Å', u'Atilde': u'Ã', u'Auml': u'Ä', u'Ccedil': u'Ç', ...