spot7.org logo
Home PHP C# C++ Android Java Javascript Python IOS SQL HTML Categories

Can you hook pyramid into twisted, skipping the wsgi part?


Twisted Web lets you interpret HTTP request bodies (regardless of content-type, HTML or otherwise) incrementally as they're received - but it doesn't make doing so very easy. There's a very old ticket that we never seem to make much progress on for improving this situation. Until it's resolved, there probably isn't a better answer than the one I'm about to give. This incremental HTTP request body delivery, I think, is what you're looking for here (because you said you expect requests to "be a big HTML chunk").

The hook for incremental request body handling is Request.handleContentChunk. You can see a complete demonstration of its use in my answer to Python server for streaming request body content.

This gives you the data as it arrives at the server. If you want to use Pyramid, you'll have to construct a Pyramid request that uses this data. Most of the initialization of the Pyramid request object should be straightforward (eg filling the environ dictionary with the request headers - you can take these from Request.requestHeaders). The slightly trickier part will be initializing the Pyramid request object's body - which is supposed to be a file-like object that provides synchronous access to the request body.

On the one hand, if you dispatch the request before the request body has been completely received then you avoid the cost of buffering the entire request body in memory. On the other hand, if you let application code begin to read the request body then you have to deal with the circumstance that it tries to read beyond the point in the data which has actually arrived at the server. This can be dealt with. The body file-like object is expected to present a blocking interface. All you have to do is block until the data is available.

Here's a brief (incomplete, not meant to actually work) sketch of what I mean:

# XXX Note: Queue is not actually
thread-safe.  Use a safer primitive.
from Queue import Queue

class Body(object):
    def __init__(self):
        self._buffer = Queue()
        self._pending = b""
        self._eof = False

    def read(self, how_many):
        if self._eof:
            return b""

        if self._pending == b"":
            data = self._buffer.get()
            if data is None:
                self._eof = True
                return b""
            else:
                self._pending = data

        if self._pending is None:

        result = self._pending[:how_many]
        self._pending = self._pending[how_many:]
        return result

    def _add_data(self, data):
        self._buffer.put(data)

You can create an instance of this type, initialize the Pyramid request object's body attribute with it, and then call _add_data on it in the Twisted Request class's handleContentChunk callback.

You could also implement this as an enhancement to Twisted's own WSGI server. For the sake of simplicity, Twisted's WSGI server does read the entire request body before dispatching the request to the WSGI application - but it doesn't have to. If this is the only problem with WSGI then it'd be better to improve the quality of the WSGI implementation and keep the interface rather than both implementing the improvement and stepping outside of the interface (tying you more closely to both Twisted and Pyramid - unnecessarily).

The second half of the problem, generating response bodies incrementally, shouldn't really be a problem. Twisted's WSGI container will write out response data as the WSGI application object yields it. Or if you use twisted.web.resource instead of the WSGI interface, you can call request.write as many times as you like, at any time you like (up until you call request.finish). The only trick is that if you want to do this you must return NOT_DONE_YET from the render method.


Categories : Python

Related to : Can you hook pyramid into twisted, skipping the wsgi part?
How to use dataReceived in Twisted?
I have implemented a server program using Twisted. I am using basic.lineReceiver with the method dataReceived to receive data from multiple clients. This is a mistake - an unfortunately common one brought on by the mistaken use of inheritance in many of Twisted's protocol implementations as the mechanism for building up more and more sophisticated behaviors. When you use twisted.protocols.ba

Categories : Python
Adding views using directives and actions in pyramid
This is what's happening: A register_status_view action is registered when your directive is called. When Pyramid executes actions, it executes this action. However, this action calls a directive that itself registers an action. But action execution is not recursive. Actions that are registered as the result of an action execution are dropped on the floor. We have some plans to resolve this

Categories : Python
How to use python-twisted-web2 in Ubuntu 14.04?
Twisted Web2 never progressed beyond the experimental stage. Development of the project was abandoned many years ago. The source was removed from the upstream Twisted repository many years ago. You do not want to use Twisted Web2. Twisted Web includes client and server implementations of the HTTP/1.1 protocol. You want to use Twisted Web.

Categories : Python
Twisted reverse proxy SSL backend
ReverseProxyResource does not support TLS. When you write ReverseProxyResource(host, 443, "/"+name) you're creating a resource which will establish a normal TCP connection to host on port 443. The TCP connection attempt will succeed but the TLS handshake will definitely fail - because the client won't even attempt one. This is a limitation of the current ReverseProxyResource: it doesn't support

Categories : Python
Twisted Python - If deferred goes out of scope, will it ever be fired?
The hypothetical connectToRemoteServer API is going to hold a reference to d internally, usually with a reference from the global reactor to an object which will fire d (by calling d.callback) when the operation represented by d is complete. So the reference goes from the stack, to the stack frame for reactor.run (because the reactor is running), to d.

Categories : Python
Recently Add
Adding json to new Django database
document clustering in python
Only read the last character in a .txt file
Is it possible to redirect to different domain retaining the trailing endpoint, from a route?
Python NameError: not defined
Numpy array loop
Turning off Tick Marks in Bokeh
Python: Number and operator concatenation not working (Euler's Method)
Python: Twitter API tweets/search: Flatten nested dictionary to columns
Python 2.7 cmd autocomplete readline buffers seem stale
Memory usage/efficiency for pandas dataframe versus lists versus tuples, etc.
How can i use multiple lists as arguments in a function and receive them differently?
PyQt - setText method of QTableWidget gets AttributeError
Matplotlib - get value of autoscale
Backwards axes in numpy.delete
Twilio - How to determine the length of a conference call?
Counting string using for loop
Automating creation of class instances in python for an undetermined amount of instances
Printing 2D-array in a grid
Load PreComputed Vectors Gensim
IPython _repr_html_
Reversing a number using recursion
Is there a configuration under which a numpy operation will work on more than a single core/thread?
Is there a better way to write this if-statement?
Python: Effective reading from a file using csv module
Django/Python: CSV for-in loop overriding first row each time through
How to read the pickled igraph graph object from old version by new version igraph
Tornado WebSocket with Django ORM with shared session
trying to plot contours of bivariate normal, won't work with a correlation term
Python split users input
© Copyright 2017 spot7.org Publishing Limited. All rights reserved.