o Q_1@sdZddlZddlZddlZddlZddlZddlZddlmZddl m Z m Z m Z ddl mZddlmZddlmZddlZddlmZmZmZejrSdd lmZejZd efd d Zdd dZda ddeedeed efddZd eefddZGddde Z!dS)zUtilities for working with multiple processes, including both forking the server into multiple processes and managing subprocesses. N)hexlify)Future"future_set_result_unless_cancelled%future_set_exception_unless_cancelled)ioloop) PipeIOStream)gen_log)OptionalAnyCallable)Listreturnc Cs\tdurdSztWStyYnwztdWSttfy&YnwtddS)z1Returns the number of processors on this machine.NSC_NPROCESSORS_CONFz1Could not detect number of processors; assuming 1) multiprocessing cpu_countNotImplementedErrorossysconfAttributeError ValueErrorrerrorrr1/usr/lib/python3/dist-packages/tornado/process.pyr0s    rcCshdtjvrdSddl}z tttdd}Wnty,ttdt A}Ynw| |dS)Nrandomri) sysmodulesrintrrurandomrtimegetpidseed)rr"rrr_reseed_random@s  r# num_processes max_restartsc sNtjdkr td|durd}tdusJ|dus|dkr t}td|idtdttffd d }t |D]}||}|durG|Sq9d}rt \}}|vrWqJ |}t |rmtd ||t |nt |dkrtd ||t |ntd ||qJ|d7}||krtd||}|dur|SsLtddS)aqStarts multiple worker processes. If ``num_processes`` is None or <= 0, we detect the number of cores available on this machine and fork that number of child processes. If ``num_processes`` is given and > 0, we fork that specific number of sub-processes. Since we use processes and not threads, there is no shared memory between any server code. Note that multiple processes are not compatible with the autoreload module (or the ``autoreload=True`` option to `tornado.web.Application` which defaults to True when ``debug=True``). When using multiple processes, no IOLoops can be created or referenced until after the call to ``fork_processes``. In each child process, ``fork_processes`` returns its *task id*, a number between 0 and ``num_processes``. Processes that exit abnormally (due to a signal or non-zero exit status) are restarted with the same id (up to ``max_restarts`` times). In the parent process, ``fork_processes`` calls ``sys.exit(0)`` after all child processes have exited normally. max_restarts defaults to 100. Availability: Unix win32zfork not available on windowsNdrzStarting %d processesir cs*t}|dkrt|a|S||<dSNr)rforkr#_task_id)r(pidchildrenrr start_child~sz#fork_processes..start_childz1child %d (pid %d) killed by signal %d, restartingz3child %d (pid %d) exited with status %d, restartingz!child %d (pid %d) exited normallyrz"Too many child restarts, giving up)rplatform Exceptionr+rrinforr rangerwaitpop WIFSIGNALEDwarningWTERMSIG WEXITSTATUS RuntimeErrorexit) r$r%r/r(id num_restartsr,statusnew_idrr-rfork_processesRsZ        r@cCstS)zpReturns the current task id, if any. Returns None if this process was not created by `fork_processes`. )r+rrrrtask_idsrAc@seZdZdZeZdZiZdZde de ddfddZ d e e gdfddfd d Z dd eddfddZedddZedddZedddZede ddfddZde ddfddZdS) Subprocessa Wraps ``subprocess.Popen`` with IOStream support. The constructor is the same as ``subprocess.Popen`` with the following additions: * ``stdin``, ``stdout``, and ``stderr`` may have the value ``tornado.process.Subprocess.STREAM``, which will make the corresponding attribute of the resulting Subprocess a `.PipeIOStream`. If this option is used, the caller is responsible for closing the streams when done with them. The ``Subprocess.STREAM`` option and the ``set_exit_callback`` and ``wait_for_exit`` methods do not work on Windows. There is therefore no reason to use this class instead of ``subprocess.Popen`` on that platform. .. versionchanged:: 5.0 The ``io_loop`` argument (deprecated since version 4.1) has been removed. FNargskwargsr c Osptj|_g}g}|dtjur-t\}}||d<| ||f| |t ||_ |dtjurPt\}}||d<| ||f| |t ||_ |dtjurst\} } | |d<| | | f| | t | |_z tj|i||_Wn|D]} t| q|D]} t| q|jj|_dD]} t|| st|| t|j| qd|_d|_dS)Nstdinstdoutstderr)rErFrG)rIOLoopcurrentio_loopgetrBSTREAMrpipeextendappendrrErFrG subprocessPopenproccloser,hasattrsetattrgetattr_exit_callback returncode) selfrCrDpipe_fdsto_closein_rin_wout_rout_werr_rerr_wfdattrrrr__init__sH               zSubprocess.__init__callbackcCs*||_t|tj|j<t|jdS)aRuns ``callback`` when this process exits. The callback takes one argument, the return code of the process. This method uses a ``SIGCHLD`` handler, which is a global setting and may conflict if you have other libraries trying to handle the same signal. If you are using more than one ``IOLoop`` it may be necessary to call `Subprocess.initialize` first to designate one ``IOLoop`` to run the signal handlers. In many cases a close callback on the stdout or stderr streams can be used as an alternative to an exit callback if the signal handler is causing a problem. Availability: Unix N)rWrB initialize_waitingr,_try_cleanup_process)rYrerrrset_exit_callbacks zSubprocess.set_exit_callbackT raise_errorz Future[int]cs,tdtddffdd }||S)a2Returns a `.Future` which resolves when the process exits. Usage:: ret = yield proc.wait_for_exit() This is a coroutine-friendly alternative to `set_exit_callback` (and a replacement for the blocking `subprocess.Popen.wait`). By default, raises `subprocess.CalledProcessError` if the process has a non-zero exit status. Use ``wait_for_exit(raise_error=False)`` to suppress this behavior and return the exit status without raising. .. versionadded:: 4.2 Availability: Unix retr Ncs.|dkrrtt|ddSt|dS)Nrunknown)rCalledProcessErrorr)rkfuturerjrrre(s  z*Subprocess.wait_for_exit..callback)rrri)rYrjrerrnr wait_for_exits zSubprocess.wait_for_exitcs8jrdStjttjfdd_d_dS)aInitializes the ``SIGCHLD`` handler. The signal handler is run on an `.IOLoop` to avoid locking issues. Note that the `.IOLoop` used for signal handling need not be the same one used by individual Subprocess objects (as long as the ``IOLoops`` are each running in separate threads). .. versionchanged:: 5.0 The ``io_loop`` argument (deprecated since version 4.1) has been removed. Availability: Unix Ncs jSN)add_callback_from_signal_cleanup)sigframeclsrJrrHs z'Subprocess.initialize..T) _initializedrrHrIsignalSIGCHLD _old_sigchldrwrrvrrf4s   zSubprocess.initializecCs$|jsdSttj|jd|_dS)z Removes the ``SIGCHLD`` handler.NF)ryrzr{r|r}rrr uninitializeLs zSubprocess.uninitializecCs"t|jD]}||qdSrq)listrgkeysrh)rwr,rrrrsTs zSubprocess._cleanupr,cCsdz t|tj\}}Wn tyYdSw|dkrdS||ks"J|j|}|j|j|dSr)) rwaitpidWNOHANGChildProcessErrorrgr5rJrr_set_returncode)rwr,ret_pidr>subprocrrrrhYs   zSubprocess._try_cleanup_processr>cCsttjdkr d|_nt|rt| |_n t|sJt||_|j|j_|j r8|j }d|_ ||jdSdS)Nr&) rr0rXrr6r8 WIFEXITEDr9rRrW)rYr>rerrrres    zSubprocess._set_returncode)Tr N)__name__ __module__ __qualname____doc__objectrLryrgr|r rdr rriboolrp classmethodrfr~rsrhrrrrrrBs$'    rBrrq)"rrrrzrPrr binasciirtornado.concurrentrrrtornadortornado.iostreamr tornado.logrtypingr r r TYPE_CHECKINGr rmrrr#r+r@rArrBrrrrs<       `