o a0@s<dZddlZddlZddlZddlZddlmZddlZddlmZddlm Z ddlm Z ddlm Z ddl m Z dd l mZdd lmZdd lmZdd lmZdd lmZddlmZddlmZddlmZddlmZddlmZddlmZddlmZddlmZddlmZddlmZddlmZddlmZddlm Z ddlm!Z!ddlm"Z"ddlm#Z#ddlm$Z$ddlm%Z%dd lm&Z&erd!d"Z!d#d$Z'Gd%d&d&eZ(e"e&)ed'e&)eoe d(Gd)d*d*e(Z*e&)ed+Gd,d-d-e*Z+Gd.d/d/e(Z,e-d0krdd1l.m/Z/e/e0dSdS)2a: Notes about unicode handling in psutil ====================================== Starting from version 5.3.0 psutil adds unicode support, see: https://github.com/giampaolo/psutil/issues/1040 The notes below apply to *any* API returning a string such as process exe(), cwd() or username(): * all strings are encoded by using the OS filesystem encoding (sys.getfilesystemencoding()) which varies depending on the platform (e.g. "UTF-8" on macOS, "mbcs" on Win) * no API call is supposed to crash with UnicodeDecodeError * instead, in case of badly encoded data returned by the OS, the following error handlers are used to replace the corrupted characters in the string: * Python 3: sys.getfilesystemencodeerrors() (PY 3.6+) or "surrogatescape" on POSIX and "replace" on Windows * Python 2: "replace" * on Python 2 all APIs return bytes (str type), never unicode * on Python 2, you can go back to unicode by doing: >>> unicode(p.exe(), sys.getdefaultencoding(), errors="replace") For a detailed explanation of how psutil handles unicode see #1040. Tests ===== List of APIs returning or dealing with a string: ('not tested' means they are not tested to deal with non-ASCII strings): * Process.cmdline() * Process.connections('unix') * Process.cwd() * Process.environ() * Process.exe() * Process.memory_maps() * Process.name() * Process.open_files() * Process.username() (not tested) * disk_io_counters() (not tested) * disk_partitions() (not tested) * disk_usage(str) * net_connections('unix') * net_if_addrs() (not tested) * net_if_stats() (not tested) * net_io_counters() (not tested) * sensors_fans() (not tested) * sensors_temperatures() (not tested) * users() (not tested) * WindowsService.binpath() (not tested) * WindowsService.description() (not tested) * WindowsService.display_name() (not tested) * WindowsService.name() (not tested) * WindowsService.status() (not tested) * WindowsService.username() (not tested) In here we create a unicode path with a funky non-ASCII name and (where possible) make psutil return it back (e.g. on name(), exe(), open_files(), etc.) and make sure that: * psutil never crashes with UnicodeDecodeError * the returned path matches N)closing)BSD)OPENBSD)POSIX)WINDOWS)PY3)u)APPVEYOR)ASCII_FS) CI_TESTING)HAS_CONNECTIONS_UNIX) HAS_ENVIRON)HAS_MEMORY_MAPS)INVALID_UNICODE_SUFFIX)PYPY TESTFN_PREFIX)UNICODE_SUFFIX)PsutilTestCase)bind_unix_socket)chdir)copyload_shared_lib) create_exe) get_testfn) safe_mkdir safe_rmpath) serialrun)skip_on_access_denied)spawn_testproc) terminate)unittestcCs4ddlm}z||WStytYdSw)Nrr) psutil.testsr WindowsError traceback print_exc)pathrmr(;/usr/lib/python3/dist-packages/psutil/tests/test_unicode.pyrss   rc Csd}t|d}zGzt|t|t|gd}t||dt|dWnttfy?YW|dur9t|t|dSwW|durIt|t|dS|durWt|t|w)z`Return True if both the fs and the subprocess module can deal with a unicode file name. Nsuffixcmdz-2FT) rrrrshutilcopyfileUnicodeEncodeErrorIOErrorr )r+sproctestfnr(r(r) try_unicodes*     r4c@seZdZdZddZdS)BaseUnicodeTestNcCs&|jdurt|js|ddSdS)Nzcan't handle unicode str) funky_suffixr4skipTest)selfr(r(r)setUps   zBaseUnicodeTest.setUp)__name__ __module__ __qualname__r6r9r(r(r(r)r5s r5zASCII fsztoo much trouble on PYPY2c@seZdZdZeZeddZeddZddZ dd Z d d Z d d Z ddZ eeo-edddZee dddZee dee deddZddZee dee deedddZd S)! TestFSAPIsz1Test FS APIs with a funky, valid, UTF8 path name.cCst|jd|_t|jdS)Nr*)rr6 funky_namerclsr(r(r) setUpClassszTestFSAPIs.setUpClasscCst|jdSN)rr>r?r(r(r) tearDownClassszTestFSAPIs.tearDownClasscCs^t|jtrdntd}ttd|jt|vWdS1s(wYdS)N.ignore) isinstancer>strrwarningscatch_warnings simplefilteroslistdir)r8herer(r(r)expect_exact_path_matchs   $z"TestFSAPIs.expect_exact_path_matchcCs^|j|jgd}t|j}|}||t|r-| t j |t j |jdSdSNr,) rr>psutilProcesspidexeassertIsInstancerGrN assertEqualrKr&normcase)r8subpprSr(r(r) test_proc_exes   zTestFSAPIs.test_proc_execCsR|j|jgd}t|j}||t|r'| |t j |jdSdSrO) rr>rPrQrRnamerTrGrNrUrKr&basename)r8rWrZr(r(r)test_proc_names  zTestFSAPIs.test_proc_namecCsZ|j|jgd}t|j}|}|D]}||tq|r+| ||jgdSdSrO) rr>rPrQrRcmdlinerTrGrNrU)r8rWrXr]partr(r(r)test_proc_cmdlines zTestFSAPIs.test_proc_cmdlinecCs|jd}|t|t|t|t}|}Wdn1s&wY||t | r?| ||dSdSN2) r> addCleanuprrrrPrQcwdrTrGrNrU)r8dnamerXrcr(r(r) test_proc_cwds    zTestFSAPIs.test_proc_cwdzfails on PYPY + WINDOWScCst}t|}t|jdt|}Wdn1s!wY||j}||t t r<|s<| dS| rQ| tj|tj|jdSdS)Nrbzopen_files on BSD is broken)rPrQset open_filesopenr>popr&rTrGrr7rNrUrKrV)r8rXstartnewr&r(r(r)test_proc_open_filess    zTestFSAPIs.test_proc_open_filesz POSIX onlycCs|j|jd}zt|}Wntytrtdwt|*t dd}| |j t tsC||j |WddSWddS1sNwYdS)Nr* not supportedunixr)rr6rr0rr!SkipTestrrPrQ connectionsrTladdrrGrrU)r8rZsockconnr(r(r)test_proc_connectionss     "z TestFSAPIs.test_proc_connectionszcan't list UNIX socketscCsdd}|j|jd}zt|}Wnty trtdwt|+tj dd}t sH||}| |j t ||j |WddSWddS1sSwYdS)NcSs.|D]}tj|jtr|Sqtd)Nzconnection not found)rKr&r[rr startswithr ValueError)consrtr(r(r) find_sock s z2TestFSAPIs.test_net_connections..find_sockr*rnro)kind)rr6rr0rr!rprrPnet_connectionsrrTrrrGrU)r8ryrZrsrxrtr(r(r)test_net_connectionss$     "zTestFSAPIs.test_net_connectionscCs,|jd}|t|t|t|dSr`)r>rbrrrP disk_usage)r8rdr(r(r)test_disk_usage"s  zTestFSAPIs.test_disk_usagernz&ctypes does not support unicode on PY2zunstable on PYPYcst|jd4}ddfddtD}dd|D}||||D]}||tq)WddS1s=wYdS)Nr*cSstjtj|SrB)rKr&realpathrV)rXr(r(r)normpath/sz-TestFSAPIs.test_memory_maps..normpathcsg|]}|jqSr()r&.0xrr(r) 1sz/TestFSAPIs.test_memory_maps..cSsg|]}t|vr|qSr(rrr(r(r)r4s)rr6rPrQ memory_mapsassertInrTrG)r8 funky_pathlibpathsr&r(rr)test_memory_maps(s  "zTestFSAPIs.test_memory_mapsN)r:r;r<__doc__rr6 classmethodrArCrNrYr\r_rer!skipIfrrrmrrur rr|r~rrrr(r(r(r)r=s2             r=zunreliable on CIc@s eZdZdZeZeddZdS)TestFSAPIsWithInvalidPathz-Test FS APIs with a funky, invalid path name.cCsdS)NTr(r?r(r(r)rN?sz1TestFSAPIsWithInvalidPath.expect_exact_path_matchN)r:r;r<rrr6rrNr(r(r(r)r:s rc@sBeZdZdZer endZee dee oe dddZ dS) TestNonFSAPISz&Unicode tests for non fs-related APIs.èrnzsegfaults on PYPY + WINDOWScCsttj}|j|d<|j|d}t|j}|}|D]\}}| |t | |t q| |d|jdS)N FUNNY_ARG)env) rKenvironcopyr6rrPrQrRitemsrTrGrU)r8rr2rXkvr(r(r)test_proc_environNs     zTestNonFSAPIS.test_proc_environN) r:r;r<rrrr6r!rr rrrr(r(r(r)rJs   r__main__) run_from_name)1rrKr.r$rH contextlibrrPrrrrpsutil._compatrrr"r r r r r rrrrrrrrrrrrrrrrr r!r4r5rr=rrr:psutil.tests.runnerr__file__r(r(r(r)shD