o -$`r@sdZddlmZzddlZddlmZddlmZdZWn e y'dZYnwgdZ Gd d d eZ Gd d d e Z Gd dde ZdS)z@Lexical analysis of formal languages (i.e. code) using Pygments.)ApplicationErrorN)get_lexer_by_name)_get_ttype_classTF)tokentextc@s eZdZdS) LexerErrorN)__name__ __module__ __qualname__r r >/usr/lib/python3/dist-packages/docutils/utils/code_analyzer.pyrsrc@s*eZdZdZd ddZddZddZd S) LexeraParse `code` lines and yield "classified" tokens. Arguments code -- string of source code to parse, language -- formal language the code is written in, tokennames -- either 'long', 'short', or 'none' (see below). Merge subsequent tokens of the same token-type. Iterating over an instance yields the tokens as ``(tokentype, value)`` tuples. The value of `tokennames` configures the naming of the tokentype: 'long': downcased full token type name, 'short': short name defined by pygments.token.STANDARD_TYPES (= class argument used in pygments html output), 'none': skip lexical analysis. shortcCsj||_||_||_d|_|dvs|dkrdStstdz t|j|_WdStjj y4td|w)zE Set up a lexical analyzer for `code` in `language`. N)rrnonez0Cannot analyze code. Pygments package not found.z6Cannot analyze code. No Pygments lexer found for "%s".) codelanguage tokennameslexer with_pygmentsrrpygmentsutil ClassNotFound)selfrrrr r r __init__0szLexer.__init__ccsvt|}t|\}}|D]\}}||ur||7}q ||fV||}}q |dr0|dd}|r9||fVdSdS)zrMerge subsequent tokens of same token-type. Also strip the final newline (added by pygments).  N)iternextendswith)rtokenslasttypelastvalttypevaluer r r mergeKs       z Lexer.mergeccs|jdurg|jfVdSt|j|j}||D]$\}}|jdkr.t|d}nt |g}dd|D}||fVqdS)z7Parse self.code and yield "classified" tokens. Nlong.cSsg|]}|tvr|qSr )unstyled_tokens).0clsr r r isz"Lexer.__iter__..) rrrlexr%rstrlowersplitr)rr tokentyper$classesr r r __iter__]s     zLexer.__iter__N)r)r r r __doc__rr%r2r r r r rs   rc@s eZdZdZddZddZdS) NumberLinesatInsert linenumber-tokens at the start of every code line. Arguments tokens -- iterable of ``(classes, value)`` tuples startline -- first line number endline -- last line number Iterating over an instance yields the tokens with a ``(['ln'], '')`` token added for every code line. Multi-line tokens are splitted.cCs"||_||_dtt||_dS)Nz%%%dd )r startlinelenr-fmt_str)rr r5endliner r r rzszNumberLines.__init__ccs|j}dg|j|fV|jD]-\}}|d}|ddD]}||dfV|d7}dg|j|fVq||dfVqdS)Nlnrr)r5r7r r/)rlinenor#r$linesliner r r r2s zNumberLines.__iter__N)r r r r3rr2r r r r r4ms r4)r3docutilsrrpygments.lexersrpygments.formatters.htmlrr ImportErrorr(robjectrr4r r r r s    Q