o bc\@sdZddlmZddlmZmZmZddlmZddl m Z ddl m Z ddl mZmZdZGd d d eZGd d d eZGd ddeZGdddZGdddee ZgdZdS)ap Memcache client protocol. Memcached is a caching server, storing data in the form of pairs key/value, and memcache is the protocol to talk with it. To connect to a server, create a factory for L{MemCacheProtocol}:: from twisted.internet import reactor, protocol from twisted.protocols.memcache import MemCacheProtocol, DEFAULT_PORT d = protocol.ClientCreator(reactor, MemCacheProtocol ).connectTCP("localhost", DEFAULT_PORT) def doSomething(proto): # Here you call the memcache operations return proto.set("mykey", "a lot of data") d.addCallback(doSomething) reactor.run() All the operations of the memcache protocol are present, but L{MemCacheProtocol.set} and L{MemCacheProtocol.get} are the more important. See U{http://code.sixapart.com/svn/memcached/trunk/server/doc/protocol.txt} for more information about the protocol. )deque)Deferred TimeoutErrorfail) LineReceiver) TimeoutMixin)log) nativeString networkStringi+c@eZdZdZdS) NoSuchCommandzA Exception raised when a non existent command is called. N__name__ __module__ __qualname____doc__rrZ!d?d@Z"dAdBZ#dUdCdDZ$dUdEdFZ%dGdHZ&dVdJdKZ'dLdMZ(dNdOZ)dPdQZ*dIS)WMemCacheProtocola1 MemCache protocol: connect to a memcached server to store/retrieve values. @ivar persistentTimeOut: the timeout period used to wait for a response. @type persistentTimeOut: L{int} @ivar _current: current list of requests waiting for an answer from the server. @type _current: L{deque} of L{Command} @ivar _lenExpected: amount of data expected in raw mode, when reading for a value. @type _lenExpected: L{int} @ivar _getBuffer: current buffer of data, used to store temporary data when reading in raw mode. @type _getBuffer: L{list} @ivar _bufferLength: the total amount of bytes in C{_getBuffer}. @type _bufferLength: L{int} @ivar _disconnected: indicate if the connectionLost has been called or not. @type _disconnected: L{bool} F<cCs*t|_d|_d|_d|_||_|_dS)z Create the protocol. @param timeOut: the timeout to wait before detecting that the connection is dead and close it. It's expressed in seconds. @type timeOut: L{int} N)r_current _lenExpected _getBuffer _bufferLengthpersistentTimeOuttimeOut)rr0rrrr s zMemCacheProtocol.__init__cCs(|jr|j}|||jsdSdS)zW Cancel all the outstanding commands, making them fail with C{reason}. N)r+popleftr)rreasoncmdrrr_cancelCommandss  z MemCacheProtocol._cancelCommandscCs|td|jdS)z: Close the connection in case of timeout. zConnection timeoutN)r4r transportloseConnectionrrrrtimeoutConnectionsz"MemCacheProtocol.timeoutConnectioncCs d|_||t||dS)z9 Cause any outstanding commands to fail. TN) _disconnectedr4rconnectionLost)rr2rrrr:s zMemCacheProtocol.connectionLostcCs"|js ||jt||dS)zA Override sendLine to add a timeout to response. N)r+ setTimeoutr/rsendLine)rlinerrrr<s zMemCacheProtocol.sendLinecCs||j||jt|7_|j|jdkr`d|j}|d|j}||jdd}|}d|_d|_d|_|jd}|jrV|j |j \}}|||f|j |j <n||_ | |dSdS)z) Collect data for a get. Nr) resetTimeoutr-appendr.lenr,joinr+multiplevalues currentKeyr# setLineMode)rdatabufremvalr3flagscasrrrrawDataReceiveds$   z MemCacheProtocol.rawDataReceivedcC|jddS)z? Manage a success response to a set operation. TNr+r1r$r7rrr cmd_STOREDzMemCacheProtocol.cmd_STOREDcCrO)z Manage a specific 'not stored' response to a set operation: this is not an error, but some condition wasn't met. FNrPr7rrrcmd_NOT_STOREDszMemCacheProtocol.cmd_NOT_STOREDcCs|j}|jdkr)|jrdd|jD}||dS||j|jfdS|jdkrF|jr9||jdS||j|j |jfdS|jdkrS||jdSt d t |j)zB This the end token to a get or a stat operation. getcSs i|] \}}||dddqS)Nr>r).0keyrKrrr s z,MemCacheProtocol.cmd_END..getsstatsz%Unexpected END response to {} commandN) r+r1rrDrErr$rLr#rM RuntimeErrorformatr )rr3rErrrcmd_ENDs"    zMemCacheProtocol.cmd_ENDcCrO)z= Manage error response for incr/decr/delete. FNrPr7rrr cmd_NOT_FOUNDrRzMemCacheProtocol.cmd_NOT_FOUNDcCs|jd}|jdkr|\}}}d}n|\}}}}t||_g|_d|_|jr@||jvr3t d||_ t||g|j |<n|j |krIt dt||_ ||_|dS)z: Prepare the reading a value after a get. rrTr?zUnexpected commands answer.N)r+rsplitintr,r-r.rDkeysrZrFrErVrLrM setRawMode)rr=r3rVrLlengthrMrrr cmd_VALUEs$       zMemCacheProtocol.cmd_VALUEcCs(|jd}|dd\}}||j|<dS)z- Reception of one stat line. r N)r+r^rE)rr=r3rVrKrrrcmd_STAT s zMemCacheProtocol.cmd_STATcCs|j|dS)z% Read version token. NrP)r versionDatarrr cmd_VERSIONrRzMemCacheProtocol.cmd_VERSIONcCs$td|j}|tdS)z7 A non-existent command has been sent. zNon-existent command sent.N)rerrr+r1rr )rr3rrr cmd_ERRORs  zMemCacheProtocol.cmd_ERRORcC2t|}td||j}|t|dS)z0 An invalid input as been sent. zInvalid input: N)reprrrir+r1rrrerrTextr3rrrcmd_CLIENT_ERROR z!MemCacheProtocol.cmd_CLIENT_ERRORcCrk)z4 An error has happened server-side. zServer error: N)rlrrir+r1rrrmrrrcmd_SERVER_ERROR(rpz!MemCacheProtocol.cmd_SERVER_ERRORcCrO)z> A delete command has completed successfully. TNrPr7rrr cmd_DELETED1rRzMemCacheProtocol.cmd_DELETEDcCrO)z6 The last command has been completed. TNrPr7rrrcmd_OK7rRzMemCacheProtocol.cmd_OKcCrO)z5 A C{checkAndSet} update has failed. FNrPr7rrr cmd_EXISTS=rRzMemCacheProtocol.cmd_EXISTScCs||ddd}t|dt|d}|dur1|dddd}|r-||dn*|n&|dd}t|dt|d}|durI|n|j}t|}|||jsa| ddSdS)z8 Receive line commands from the server. rdrercmd_N_) r@r^getattrr replacer+r1r_r$r;)rr=tokenr3argsrKrrr lineReceivedCs$   zMemCacheProtocol.lineReceivedrecC|d||S)a Increment the value of C{key} by given value (default to 1). C{key} must be consistent with an int. Return the new value. @param key: the key to modify. @type key: L{bytes} @param val: the value to increment. @type val: L{int} @return: a deferred with will be called back with the new value associated with the key (after the increment). @rtype: L{Deferred} sincr _incrdecrrrVrKrrr increment`szMemCacheProtocol.incrementcCr|)a Decrement the value of C{key} by given value (default to 1). C{key} must be consistent with an int. Return the new value, coerced to 0 if negative. @param key: the key to modify. @type key: L{bytes} @param val: the value to decrement. @type val: L{int} @return: a deferred with will be called back with the new value associated with the key (after the decrement). @rtype: L{Deferred} sdecrr}rrrr decrementqszMemCacheProtocol.decrementcCs|jr ttdSt|tsttdt|dSt||jkr'ttdSd ||dt |fg}| |t ||d}|j ||jS)z1 Internal wrapper for incr/decr. not connectedInvalid type for key: , expecting bytes Key too longrds%drV)r9rrZ isinstancebytesrtyperBMAX_KEY_LENGTHrCr_r<rr+rAr)rr3rVrKfullcmdcmdObjrrrr~s      zMemCacheProtocol._incrdecrrcC|d||||dS)a Replace the given C{key}. It must already exist in the server. @param key: the key to replace. @type key: L{bytes} @param val: the new value associated with the key. @type val: L{bytes} @param flags: the flags to store with the key. @type flags: L{int} @param expireTime: if different from 0, the relative time in seconds when the key will be deleted from the store. @type expireTime: L{int} @return: a deferred that will fire with C{True} if the operation has succeeded, and C{False} with the key didn't previously exist. @rtype: L{Deferred} sreplacer?_setrrVrKrL expireTimerrrrxzMemCacheProtocol.replacecCr)a Add the given C{key}. It must not exist in the server. @param key: the key to add. @type key: L{bytes} @param val: the value associated with the key. @type val: L{bytes} @param flags: the flags to store with the key. @type flags: L{int} @param expireTime: if different from 0, the relative time in seconds when the key will be deleted from the store. @type expireTime: L{int} @return: a deferred that will fire with C{True} if the operation has succeeded, and C{False} with the key already exists. @rtype: L{Deferred} saddr?rrrrraddrzMemCacheProtocol.addcCr)a9 Set the given C{key}. @param key: the key to set. @type key: L{bytes} @param val: the value associated with the key. @type val: L{bytes} @param flags: the flags to store with the key. @type flags: L{int} @param expireTime: if different from 0, the relative time in seconds when the key will be deleted from the store. @type expireTime: L{int} @return: a deferred that will fire with C{True} if the operation has succeeded. @rtype: L{Deferred} ssetr?rrrrrsetrzMemCacheProtocol.setcCs|d|||||S)am Change the content of C{key} only if the C{cas} value matches the current one associated with the key. Use this to store a value which hasn't been modified since last time you fetched it. @param key: The key to set. @type key: L{bytes} @param val: The value associated with the key. @type val: L{bytes} @param cas: Unique 64-bit value returned by previous call of C{get}. @type cas: L{bytes} @param flags: The flags to store with the key. @type flags: L{int} @param expireTime: If different from 0, the relative time in seconds when the key will be deleted from the store. @type expireTime: L{int} @return: A deferred that will fire with C{True} if the operation has succeeded, C{False} otherwise. @rtype: L{Deferred} scasr)rrVrKrMrLrrrr checkAndSetszMemCacheProtocol.checkAndSetc Cs|jr ttdSt|tsttdt|dSt||jkr'ttdSt|ts8ttdt|dS|r>d|}t|}d ||t d|||fg|}| || |t ||||d} |j | | jS) z6 Internal wrapper for setting values. rrrrzInvalid type for value: rdz%d %d %d)rVrLrb)r9rrZrrrrrBrrCr r<rr+rAr) rr3rVrKrLrrMrbrrrrrrs4       zMemCacheProtocol._setcC|d||dddS)a Append given data to the value of an existing key. @param key: The key to modify. @type key: L{bytes} @param val: The value to append to the current value associated with the key. @type val: L{bytes} @return: A deferred that will fire with C{True} if the operation has succeeded, C{False} otherwise. @rtype: L{Deferred} sappendrr?rrrrrrAzMemCacheProtocol.appendcCr)a Prepend given data to the value of an existing key. @param key: The key to modify. @type key: L{bytes} @param val: The value to prepend to the current value associated with the key. @type val: L{bytes} @return: A deferred that will fire with C{True} if the operation has succeeded, C{False} otherwise. @rtype: L{Deferred} sprependrr?rrrrrprepend'rzMemCacheProtocol.prependcCs||g|dS)a Get the given C{key}. It doesn't support multiple keys. If C{withIdentifier} is set to C{True}, the command issued is a C{gets}, that will return the current identifier associated with the value. This identifier has to be used when issuing C{checkAndSet} update later, using the corresponding method. @param key: The key to retrieve. @type key: L{bytes} @param withIdentifier: If set to C{True}, retrieve the current identifier along with the value and the flags. @type withIdentifier: L{bool} @return: A deferred that will fire with the tuple (flags, value) if C{withIdentifier} is C{False}, or (flags, cas identifier, value) if C{True}. If the server indicates there is no value associated with C{key}, the returned value will be L{None} and the returned flags will be C{0}. @rtype: L{Deferred} F_get)rrVwithIdentifierrrrget9szMemCacheProtocol.getcCs|||dS)a Get the given list of C{keys}. If C{withIdentifier} is set to C{True}, the command issued is a C{gets}, that will return the identifiers associated with each values. This identifier has to be used when issuing C{checkAndSet} update later, using the corresponding method. @param keys: The keys to retrieve. @type keys: L{list} of L{bytes} @param withIdentifier: If set to C{True}, retrieve the identifiers along with the values and the flags. @type withIdentifier: L{bool} @return: A deferred that will fire with a dictionary with the elements of C{keys} as keys and the tuples (flags, value) as values if C{withIdentifier} is C{False}, or (flags, cas identifier, value) if C{True}. If the server indicates there is no value associated with C{key}, the returned values will be L{None} and the returned flags will be C{0}. @rtype: L{Deferred} @since: 9.0 Tr)rr`rrrr getMultipleQszMemCacheProtocol.getMultiplec Cst|}|jr ttdS|D]$}t|ts$ttdt|dSt||j kr3ttdSq|r9d}nd}d |g|}| ||rZdd |D}t |||d d }n t ||d d d ddd}|j ||jS)z> Helper method for C{get} and C{getMultiple}. rrrrrXrTrdcSsi|]}|dqS))rr?Nr)rUrVrrrrWsz)MemCacheProtocol._get..T)r`rErDrNr?F)rVr#rLrMrD)listr9rrZrrrrrBrrCr<rr+rAr) rr`rrDrVr3rrErrrrrks0    zMemCacheProtocol._getNcCsL|rd|}nd}|jrttdS||tdid}|j||jS)a Get some stats from the server. It will be available as a dict. @param arg: An optional additional string which will be sent along with the I{stats} command. The interpretation of this value by the server is left undefined by the memcache protocol specification. @type arg: L{None} or L{bytes} @return: a deferred that will fire with a L{dict} of the available statistics. @rtype: L{Deferred} sstats rYr)rEr9rrZr<rr+rAr)rargr3rrrrstatss     zMemCacheProtocol.statscC6|jr ttdS|dtd}|j||jS)z Get the version of the server. @return: a deferred that will fire with the string value of the version. @rtype: L{Deferred} rsversionrrrrrrversion    zMemCacheProtocol.versioncCs`|jr ttdSt|tsttdt|dS|d|td|d}|j ||j S)a Delete an existing C{key}. @param key: the key to delete. @type key: L{bytes} @return: a deferred that will be called back with C{True} if the key was successfully deleted, or C{False} if not. @rtype: L{Deferred} rrrsdelete sdeleter) r9rrZrrrrr<rr+rAr)rrVrrrrdeletes    zMemCacheProtocol.deletecCr)z Flush all cached values. @return: a deferred that will be called back with C{True} when the operation has succeeded. @rtype: L{Deferred} rs flush_allrrrrrflushAllrzMemCacheProtocol.flushAll)r*)re)rr)F)N)+rrrrrr9r r4r8r:r<rNrQrSr\r]rcrfrhrjrorqrrrsrtr{rrr~rxrrrrrArrrrrrrrrrrrr(csR             r()r( DEFAULT_PORTr rrN)r collectionsrtwisted.internet.deferrrrtwisted.protocols.basicrtwisted.protocols.policiesrtwisted.pythonrtwisted.python.compatr r r Exceptionr rrrr(__all__rrrrs$    ) x