KJi[9 vdZddlZddlmZddlmZddlmZddlZ ddl m Z ddl m Z d Ze jZe jjZeiZGd d ZGd d eZGddeZGddZdZdZdZd$dZGddZeeedZeeZ edeejdeejdeejddddZ ee Z! e"e#e$e%ej&ej'ej(ej)ej*ej+d Z,ee,Z- d!Z.d%d"Z/d#Z0dS)&a>Piculet is a module for scraping XML and HTML documents using XPath queries. It consists of this single source file with no dependencies other than the standard library, which makes it very easy to integrate into applications. For more information, please refer to the documentation: https://piculet.readthedocs.io/ N)partial) itemgetter)MappingProxyType)etree)jselz1.2b2c>eZdZdZddZdZd dZedZdS) Extractorz?Abstract base extractor for getting data out of an XML element.NcF||_ |t|nd|_dS)zInitialize this extractor. :param transform: Function to transform the extracted value. :param foreach: Path to apply for generating a collection of values. N) transformXPathforeach)selfr rs X/var/www/html/auto_sub_bot/venv/lib/python3.11/site-packages/imdb/parser/http/piculet.py__init__zExtractor.__init__5s* #8)0)z&Extractor.from_map..ts$555!T]]1%%555rsection)r(r r)get transformers ValueErrorreducersPathRules) item transformerr rrreducerr extractorrruless rr$zExtractor.from_mapUs$hh{++  II$((55I  !6777((9%%xx  hhx((G!g..>$%6777T6YPPPIIHHW%%E55u555EeTXXi-@-@(17DDDIr)NN)T) __name__ __module__ __qualname____doc__rrr staticmethodr$r!rrr r 2swII C C C CTTT J J J J""\"""rr c*eZdZdZdfd ZdZxZS)r-z4An extractor for getting text out of an XML element.Nct||t||_ | tj}||_dS)a3Initialize this extractor. :param path: Path to apply to get the data. :param reduce: Function to reduce selected texts into a single string. :param transform: Function to transform extracted value. :param foreach: Path to apply for generating a collection of data. rN)superrr rr,concatr)rrrr r __class__s rrz Path.__init__~sK 9g>>>$KK 7 >_F EErc||}t|dkrd}n||}|S)zApply this extractor to an element. :param element: Element to apply this extractor to. :return: Extracted text. rN)rlenr)rrselectedrs rrz Path.applysC 99W%% x==A  EEKK))E rNNNr4r5r6r7rr __classcell__r=s@rr-r-{sX>>FFFFFF&       rr-c*eZdZdZdfd ZdZxZS)r.z:An extractor for getting data items out of an XML element.Nct||||_ |t|nd|_dS)aInitialize this extractor. :param rules: Rules for generating the data items. :param section: Path for setting the root of this section. :param transform: Function to transform extracted value. :param foreach: Path for generating multiple items. rN)r;rr3r r()rr3r(r rr=s rrzRules.__init__sH 9g>>> 2)0).sDHHH"##n44Q%4HHHHHrc(g|]}||tu |SN)r)r%vs rr'z Rule.extract..s"YYYAMQW!rcDg|]}tj|jSr!)rselectrQ)r%r[rs rr'z Rule.extract..s(RRRadk!T-@AARRRrr)rrSrPrTrr2rQrr]rr?r listmap) rrrKrJrIrPr raw_valuesvaluess ` rrz Rule.extracts  $ 4G99$,,w:O:O @ @G(377V$((TX=M=Mg=V=VC{~%-..w77&D:eS+A+AD Kt/BCCEMu!S HHHH'+~'='=g'F'FHHH YYZYYY&SRRRR6RRRFv;;!##&*n&>&FFFT^5v>>??S  rrZ)r4r5r6r7rr8r$rr!rrr#r#s_IIGGGG& K K\ Krr#ct||}t|dkr,|D]+}||*dSdS)zRemove selected elements from the tree. :param root: Root element of the tree. :param path: XPath to select the elements to remove. rN)r r? getparentremove)rootrelementsrs rremove_elementsrg sn uT{{4  H 8}}q 0 0G      & &w / / / / 0 0rcpt||}|D]}t|tr|n,t||}|It|tr|n,t||}|||j|<dS)aSet an attribute for selected elements. :param root: Root element of the tree. :param path: XPath to select the elements to set attributes for. :param name: Description for name generation. :param value: Description for value generation. N)r rSrTr r$rattrib)rernamerrfr attr_name attr_values rset_element_attrrmsuT{{4  H / /&tS116DD   t $ $ , ,W 5 5    (447UU   u % % - -g 6 6    $.y!! / /rct||}|D]M}t|tr|n,t||}||_NdS)zSet the text for selected elements. :param root: Root element of the tree. :param path: XPath to select the elements to set attributes for. :param text: Description for text generation. N)r rSrTr r$rtext)rerrorfr element_texts rset_element_textrq0swuT{{4  H$$)$446tt   t $ $ , ,W 5 5 $ $$rFcl|rtj|Stj|S)zBuild a tree from an XML document. :param document: XML document to build the tree from. :param force_html: Force to parse from HTML without converting. :return: Root element of the XML tree. )lxmlhtml fromstring ElementTree)document force_htmls r build_treery?s3.y##H---  !( + ++rc$eZdZdZdZdZdZdS)Registryz$A simple, attribute-based namespace.c:|j|dS)z\Initialize this registry. :param entries: Entries to add to this registry. N)__dict__rH)rentriess rrzRegistry.__init__Ns W%%%%%rc6|j|S)zGet the value of an entry from this registry. :param item: Entry to get the value for. :return: Value of entry. )r}r))rr/s rr)z Registry.getUs }  &&&rc||j|<dS)zRegister a new entry in this registry. :param key: Key to search the entry in this registry. :param value: Value to store for the entry. N)r})rrPrs rregisterzRegistry.register]s # crN)r4r5r6r7rr)rr!rrr{r{KsG..&&&'''#####rr{)rdset_attrset_text |ctjddd|ddS)Nz\s+rr )resubjoinreplacestripxss rrus:vsBGGBKK,?,?,L,LMMSSUUrctjddd|ddS)Nz [^a-z0-9_]rr_)rrrlowerrrs rrrvs<BF<RWWR[[5F5F5H5H5P5PQTVY5Z5Z[[r)firstr<r pipe_joinclean normalize) intfloatboolr?rupper capitalizelstriprstriprc|D]}|d}|dkrt||d'|dkr&t||d|d|dS|dkrt||d|d  xtd d S) zProcess a tree before starting extraction. :param root: Root of tree to process. :param pre: Descriptions for processing operations. oprdrrrjr)rjrrro)rozUnknown preprocessing operationN)rgrmrqr+)represteprs r preprocessrs  @ @ $Z >> D$v, / / / / :   T4?? ? @ @rcbtd|D|}||S)zExtract data from an XML element. :param element: Element to extract the data from. :param items: Descriptions for extracting items. :param section: Path to select the root element for these items. :return: Extracted data. cBg|]}t|Sr!r")r%r/s rr'zextract..s$99944==&&999rr()r.r)rrr(r3s rrrs7 9959997 K K KE == ! !!rct|}|d}|t||t||d|d}|S)zExtract data from a document after optionally preprocessing it. :param document: Document to scrape. :param spec: Extraction specification. :return: Extracted data. rNrr(r)ryr)rr)rwspecrerrKs rscrapersf h  D ((5//C 4 4'**DHHY4G4G H H HD Kr)FrZ)1r7r functoolsroperatorrtypesr lxml.htmlrsrrvrr __version__r _Elementxpathrr r-r.r#rgrmrqryr{_PREPROCESSORS preprocessorsrTr _REDUCERSr,rrrr?rrrrrr _TRANSFORMERSr*rrrr!rrrs  """"""%%%%%%  "  "  FFFFFFFFR!!!!!9!!!H'1'1'1'1'1I'1'1'1T@@@@@@@@F 0 0 0///. $ $ $ , , , ,########8   (( Z]]gch## GCHc " "3'' U U[[     8I       Y Y.jj Y   x && @@@$ " " " "     r