Status of Asynchronous Support

classic Classic list List threaded Threaded
20 messages Options
Reply | Threaded
Open this post in threaded view
|

Status of Asynchronous Support

Tal Liron
Thought I've followed what I could of the progress on asynchronous
support, I have a few interdependent questions and suggestions.

1. On the one hand, I really like how simple asynchronicity is handled.
I can defer my response to later (from another thread). But, Restlet
provides no tools to help me decide if I should defer or not. What I
would want to know is -- Am I under load right now, meaning high
concurrency? Should I yield so that other requests can have a chance to
get in?

2. A related question: If I do defer, I do not necessarily want to
decide on my own when to finish the response. I might want to wait until
there is a free thread that can take me. It would be great if there were
1) a callback for this, or 2) a way for me to "signal" that I'm ready to
respond, as soon as the server is free. Or, at the very least a way to
poll the server load.

(To summarize these two issues: the Restlet server is very opaque right
now. For truly scalable applications (cloud, cloud, cloud) we need
realtime information from it so that the application can react dynamically.)

3. I know that the "default" connector supports commit/abort, but what
about Grizzly, Jetty and Netty? If they don't support it yet, is support
being planned for 2.0? This will strongly affect my deployment strategies.

4. Assuming only the "default" connector is asynchronous, could you
provide more details about it? Until now, frankly, I've ignored it as
something that just works and isn't fit for production. It seems like
this connector is very capable by now, but I know nothing about how it's
implemented. NIO? Thread pools? SSL? I think it would be very good if
there were a wiki page with an in-depth comparison of the various
available connectors, what features they provide and implementation
details -- both of the servers themselves, and of the features the
Restlet adapters leverages (or not). As it stands, it's hard for a
deployer to make an informed decision on which connector to use.

(For the record -- I think benchmarks are quite useless for this
comparison. Performance != scalability, and in any case benchmarks are
highly contextual. It's very easy to switch connectors in Restlet and
for people to do their own benchmarks.)

5. Might I suggest giving the "default" connector a name? It seems to be
a nice little engine of its own. If we're going to start comparing it to
the others as a viable alternative, it might be easier to name it. My
only name suggestion is Thier Engine, short for Thierry, and reminds me
of "tier." :)

-Tal

------------------------------------------------------
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=2611218
Reply | Threaded
Open this post in threaded view
|

RE: Status of Asynchronous Support

jlouvel
Administrator
Hi Tal,

1. This is a good point that needs to be explored. The current path we are
following is to deal with high-load directly inside the internal connector,
for example to stop accepting new connections at IO level if we are
overloaded, or giving a priority of responses sending over new requests
handling.

You might prefer to deal with that inside resources, which could make
perfect sense. We should explore this during the 2.1 development cycle. Feel
free to enter a RFE in the tracker.

2. There is a TaskService associated to your application that you could
leverage. It separates threads usage from tasks to process (such as
committing responses).

Your point about instrumentation is valid though, we need to work on that.
Creating a RFE would be a good step forward.

3. abort() should now work with other connectors AFAIK (check the changes
log). Regarding provisional responses and commit(), I think this will have
to wait for 2.1.

4. The internal connector is described here:
http://wiki.restlet.org/developers/172-restlet/354-restlet.html

We are working on the NIO enhancements to it and hope to have this ready for
2.0 in order to have a scalable/asynchronous/lightweight connector. It uses
a configurable thread pool, but HTTPS is not planned for 2.0 at all (too
much work).

We also have this page describing the various connectors available:
http://wiki.restlet.org/docs_2.0/13-restlet/27-restlet/325-restlet/37-restle
t.html

This should be improved I guess with addition details/criterias. We did
cover those details in the "Restlet in Action" book (early access).

5. Currently, we call this default connector the "internal" connector. I'm
not sure about having a code name for it at this point... I'd prefer people
to think about it as a natural part of the Restlet engine.

Best regards,
Jerome Louvel
--
Restlet ~ Founder and Technical Lead ~ http://www.restlet.org
Noelios Technologies ~ http://www.noelios.com




-----Message d'origine-----
De : Tal Liron [mailto:[hidden email]]
Envoyé : jeudi 20 mai 2010 11:00
À : [hidden email]
Objet : Status of Asynchronous Support

Thought I've followed what I could of the progress on asynchronous
support, I have a few interdependent questions and suggestions.

1. On the one hand, I really like how simple asynchronicity is handled.
I can defer my response to later (from another thread). But, Restlet
provides no tools to help me decide if I should defer or not. What I
would want to know is -- Am I under load right now, meaning high
concurrency? Should I yield so that other requests can have a chance to
get in?

2. A related question: If I do defer, I do not necessarily want to
decide on my own when to finish the response. I might want to wait until
there is a free thread that can take me. It would be great if there were
1) a callback for this, or 2) a way for me to "signal" that I'm ready to
respond, as soon as the server is free. Or, at the very least a way to
poll the server load.

(To summarize these two issues: the Restlet server is very opaque right
now. For truly scalable applications (cloud, cloud, cloud) we need
realtime information from it so that the application can react dynamically.)

3. I know that the "default" connector supports commit/abort, but what
about Grizzly, Jetty and Netty? If they don't support it yet, is support
being planned for 2.0? This will strongly affect my deployment strategies.

4. Assuming only the "default" connector is asynchronous, could you
provide more details about it? Until now, frankly, I've ignored it as
something that just works and isn't fit for production. It seems like
this connector is very capable by now, but I know nothing about how it's
implemented. NIO? Thread pools? SSL? I think it would be very good if
there were a wiki page with an in-depth comparison of the various
available connectors, what features they provide and implementation
details -- both of the servers themselves, and of the features the
Restlet adapters leverages (or not). As it stands, it's hard for a
deployer to make an informed decision on which connector to use.

(For the record -- I think benchmarks are quite useless for this
comparison. Performance != scalability, and in any case benchmarks are
highly contextual. It's very easy to switch connectors in Restlet and
for people to do their own benchmarks.)

5. Might I suggest giving the "default" connector a name? It seems to be
a nice little engine of its own. If we're going to start comparing it to
the others as a viable alternative, it might be easier to name it. My
only name suggestion is Thier Engine, short for Thierry, and reminds me
of "tier." :)

-Tal

------------------------------------------------------
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=26112
18

------------------------------------------------------
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=2611691
Reply | Threaded
Open this post in threaded view
|

Re: Status of Asynchronous Support

Tal Liron
Thanks for the detailed response!

RFE:
"Expose server instrumentation"
http://restlet.tigris.org/issues/show_bug.cgi?id=1117

So, I guess only the internal connector will support true asynchronous
handling now for 2.0.

The comparison charts talk about the internal connector being suitable
for development. But, what's your opinion about production environments?
It seems to be much improved since the early days of Restlet.  If
asynchronicity is a required feature, I do not have another choice for 2.0.

Or, perhaps we should consider asynchronicity as a "preview" feature in
2.0, and wait until 2.1 for leveraging it?

I think it was wise of you not to add SSL support. It's definitely a
pain to write, and there are simple solutions. (I recommend using
Perlbal as your load balancer, and handling SSL there.) Have you
considered, though, using code from Grizzly for SSL?

-Tal

On 05/21/2010 03:53 AM, Jerome Louvel wrote:

> Hi Tal,
>
> 1. This is a good point that needs to be explored. The current path we are
> following is to deal with high-load directly inside the internal connector,
> for example to stop accepting new connections at IO level if we are
> overloaded, or giving a priority of responses sending over new requests
> handling.
>
> You might prefer to deal with that inside resources, which could make
> perfect sense. We should explore this during the 2.1 development cycle. Feel
> free to enter a RFE in the tracker.
>
> 2. There is a TaskService associated to your application that you could
> leverage. It separates threads usage from tasks to process (such as
> committing responses).
>
> Your point about instrumentation is valid though, we need to work on that.
> Creating a RFE would be a good step forward.
>
> 3. abort() should now work with other connectors AFAIK (check the changes
> log). Regarding provisional responses and commit(), I think this will have
> to wait for 2.1.
>
> 4. The internal connector is described here:
> http://wiki.restlet.org/developers/172-restlet/354-restlet.html
>
> We are working on the NIO enhancements to it and hope to have this ready for
> 2.0 in order to have a scalable/asynchronous/lightweight connector. It uses
> a configurable thread pool, but HTTPS is not planned for 2.0 at all (too
> much work).
>
> We also have this page describing the various connectors available:
> http://wiki.restlet.org/docs_2.0/13-restlet/27-restlet/325-restlet/37-restle
> t.html
>
> This should be improved I guess with addition details/criterias. We did
> cover those details in the "Restlet in Action" book (early access).
>
> 5. Currently, we call this default connector the "internal" connector. I'm
> not sure about having a code name for it at this point... I'd prefer people
> to think about it as a natural part of the Restlet engine.
>
> Best regards,
> Jerome Louvel
> --
> Restlet ~ Founder and Technical Lead ~ http://www.restlet.org
> Noelios Technologies ~ http://www.noelios.com
>
>
>
>
> -----Message d'origine-----
> De : Tal Liron [mailto:[hidden email]]
> Envoyé : jeudi 20 mai 2010 11:00
> À : [hidden email]
> Objet : Status of Asynchronous Support
>
> Thought I've followed what I could of the progress on asynchronous
> support, I have a few interdependent questions and suggestions.
>
> 1. On the one hand, I really like how simple asynchronicity is handled.
> I can defer my response to later (from another thread). But, Restlet
> provides no tools to help me decide if I should defer or not. What I
> would want to know is -- Am I under load right now, meaning high
> concurrency? Should I yield so that other requests can have a chance to
> get in?
>
> 2. A related question: If I do defer, I do not necessarily want to
> decide on my own when to finish the response. I might want to wait until
> there is a free thread that can take me. It would be great if there were
> 1) a callback for this, or 2) a way for me to "signal" that I'm ready to
> respond, as soon as the server is free. Or, at the very least a way to
> poll the server load.
>
> (To summarize these two issues: the Restlet server is very opaque right
> now. For truly scalable applications (cloud, cloud, cloud) we need
> realtime information from it so that the application can react dynamically.)
>
> 3. I know that the "default" connector supports commit/abort, but what
> about Grizzly, Jetty and Netty? If they don't support it yet, is support
> being planned for 2.0? This will strongly affect my deployment strategies.
>
> 4. Assuming only the "default" connector is asynchronous, could you
> provide more details about it? Until now, frankly, I've ignored it as
> something that just works and isn't fit for production. It seems like
> this connector is very capable by now, but I know nothing about how it's
> implemented. NIO? Thread pools? SSL? I think it would be very good if
> there were a wiki page with an in-depth comparison of the various
> available connectors, what features they provide and implementation
> details -- both of the servers themselves, and of the features the
> Restlet adapters leverages (or not). As it stands, it's hard for a
> deployer to make an informed decision on which connector to use.
>
> (For the record -- I think benchmarks are quite useless for this
> comparison. Performance != scalability, and in any case benchmarks are
> highly contextual. It's very easy to switch connectors in Restlet and
> for people to do their own benchmarks.)
>
> 5. Might I suggest giving the "default" connector a name? It seems to be
> a nice little engine of its own. If we're going to start comparing it to
> the others as a viable alternative, it might be easier to name it. My
> only name suggestion is Thier Engine, short for Thierry, and reminds me
> of "tier." :)
>
> -Tal
>
> ------------------------------------------------------
> http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=26112
> 18
>
> ------------------------------------------------------
> http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=2611691

------------------------------------------------------
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=2612116
Reply | Threaded
Open this post in threaded view
|

Re: Status of Asynchronous Support

Tal Liron
In reply to this post by jlouvel
Hi Jerome,

> 2. There is a TaskService associated to your application that you could
> leverage. It separates threads usage from tasks to process (such as
> committing responses).

I've tried this, but there is a concurrency problem with this pattern in
ServerResource --

If I set autoCommiting to false during a ServerResource.get() and put my
task on a separate thread pool, then my own task might try to update the
response at the same time as the upper parts of the call stack
(Finder.handle(), for example) also try to update the response according
to my return value from get(). Both threads might be doing it at the
same time, and the response ends up corrupted (no concurrency
exceptions, but it's mismatched status/entity, for example).

What would be the correct way to defer a response from within a
ServerResource?

My thought is that there should be a way to return from get() while
signaling to the rest of the call stack that I am handling the response.
A null won't work, because it internally signifies an unavailable
entity. Perhaps a new kind of ResourceException?
DefferedResponseResourceException?

-Tal

------------------------------------------------------
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=2612147
Reply | Threaded
Open this post in threaded view
|

Re: Status of Asynchronous Support

Tim Peierls
I never saw a response to this, but it does seem like a problem.

I don't love the idea of a special exception used purely to handle control flow, but I can't think of anything better off-hand.

--tim

On Sat, May 22, 2010 at 8:05 PM, Tal Liron <[hidden email]> wrote:
Hi Jerome,

> 2. There is a TaskService associated to your application that you could
> leverage. It separates threads usage from tasks to process (such as
> committing responses).

I've tried this, but there is a concurrency problem with this pattern in
ServerResource --

If I set autoCommiting to false during a ServerResource.get() and put my
task on a separate thread pool, then my own task might try to update the
response at the same time as the upper parts of the call stack
(Finder.handle(), for example) also try to update the response according
to my return value from get(). Both threads might be doing it at the
same time, and the response ends up corrupted (no concurrency
exceptions, but it's mismatched status/entity, for example).

What would be the correct way to defer a response from within a
ServerResource?

My thought is that there should be a way to return from get() while
signaling to the rest of the call stack that I am handling the response.
A null won't work, because it internally signifies an unavailable
entity. Perhaps a new kind of ResourceException?
DefferedResponseResourceException?

-Tal

------------------------------------------------------
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=2612147

Reply | Threaded
Open this post in threaded view
|

RE: Status of Asynchronous Support

jlouvel
Administrator

Hi Tal and Tim,

 

There is a way to signal an asynchronous response on a per-call basis, calling Response#setCommitted(false).

 

Then, if you call Response#commit() before the initial request thread reaches back the connector, this should work fine as well. I’m not sure yet how it could get ‘corrupted’. Do you have a precise scenario?

 

Note that the NIO version of the asynchronous internal connector has been moved to version 2.1. This feature should indeed be considered as preview feature for 2.0.

 

Best regards,
Jerome Louvel
--
Restlet ~ Founder and Technical Lead ~ http://www.restlet.org
Noelios Technologies ~
http://www.noelios.com

 

 

 

 

 

De : [hidden email] [mailto:[hidden email]] De la part de Tim Peierls
Envoyé : vendredi 18 juin 2010 13:51
À : [hidden email]
Objet : Re: Status of Asynchronous Support

 

I never saw a response to this, but it does seem like a problem.

 

I don't love the idea of a special exception used purely to handle control flow, but I can't think of anything better off-hand.

 

--tim

On Sat, May 22, 2010 at 8:05 PM, Tal Liron <[hidden email]> wrote:

Hi Jerome,


> 2. There is a TaskService associated to your application that you could
> leverage. It separates threads usage from tasks to process (such as
> committing responses).

I've tried this, but there is a concurrency problem with this pattern in
ServerResource --

If I set autoCommiting to false during a ServerResource.get() and put my
task on a separate thread pool, then my own task might try to update the
response at the same time as the upper parts of the call stack
(Finder.handle(), for example) also try to update the response according
to my return value from get(). Both threads might be doing it at the
same time, and the response ends up corrupted (no concurrency
exceptions, but it's mismatched status/entity, for example).

What would be the correct way to defer a response from within a
ServerResource?

My thought is that there should be a way to return from get() while
signaling to the rest of the call stack that I am handling the response.
A null won't work, because it internally signifies an unavailable
entity. Perhaps a new kind of ResourceException?
DefferedResponseResourceException?

-Tal

------------------------------------------------------
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=2612147

 

Reply | Threaded
Open this post in threaded view
|

Re: Status of Asynchronous Support

Tal Liron
Jerome, even though setCommitted(false) gets called, there still the usual processing of handle() in the entire restlet chain (the Finder, delegating to the UniformResource, etc.). Within these routines, the return values from various callbacks are handled. Stuff that happens there on the response can override work being done in another thread. The response is thread safe, but it's still not the intended result.

I don't have a test case right now, but it would be easy to produce: simply call setCommitted(false) in a ServerResource annotated @Get implementation, and immediately move processing to a thread, where commit() is eventually called. The worker handler thread would overlap with the work done in the regular Restlet handling thread and you would get weird statuses and response entities.

The reason I suggest an exception (and I share Tim's unease with it) is that it's a way to stop the handling chain in the regular thread, so that only the new handler thread would have free reign on the response. But it doesn't have to be an exception. Another solution would be for ServerResource, UniformResource and Finder to be refactored specifically to handle the committed=false case. I worry, though, that future work or custom implementations would not handle the case properly.

-Tal

On Sun, Jul 11, 2010 at 9:00 PM, Jerome Louvel <[hidden email]> wrote:

Hi Tal and Tim,

 

There is a way to signal an asynchronous response on a per-call basis, calling Response#setCommitted(false).

 

Then, if you call Response#commit() before the initial request thread reaches back the connector, this should work fine as well. I’m not sure yet how it could get ‘corrupted’. Do you have a precise scenario?

 

Note that the NIO version of the asynchronous internal connector has been moved to version 2.1. This feature should indeed be considered as preview feature for 2.0.

 

Best regards,
Jerome Louvel
--
Restlet ~ Founder and Technical Lead ~ http://www.restlet.org
Noelios Technologies ~
http://www.noelios.com

 

 

 

 

 

De : [hidden email] [mailto:[hidden email]] De la part de Tim Peierls
Envoyé : vendredi 18 juin 2010 13:51

Objet : Re: Status of Asynchronous Support

 

I never saw a response to this, but it does seem like a problem.

 

I don't love the idea of a special exception used purely to handle control flow, but I can't think of anything better off-hand.

 

--tim

On Sat, May 22, 2010 at 8:05 PM, Tal Liron <[hidden email]> wrote:

Hi Jerome,


> 2. There is a TaskService associated to your application that you could
> leverage. It separates threads usage from tasks to process (such as
> committing responses).

I've tried this, but there is a concurrency problem with this pattern in
ServerResource --

If I set autoCommiting to false during a ServerResource.get() and put my
task on a separate thread pool, then my own task might try to update the
response at the same time as the upper parts of the call stack
(Finder.handle(), for example) also try to update the response according
to my return value from get(). Both threads might be doing it at the
same time, and the response ends up corrupted (no concurrency
exceptions, but it's mismatched status/entity, for example).

What would be the correct way to defer a response from within a
ServerResource?

My thought is that there should be a way to return from get() while
signaling to the rest of the call stack that I am handling the response.
A null won't work, because it internally signifies an unavailable
entity. Perhaps a new kind of ResourceException?
DefferedResponseResourceException?

-Tal

------------------------------------------------------
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=2612147

 


Reply | Threaded
Open this post in threaded view
|

RE: Re: Status of Asynchronous Support

Tal Liron
I have created the simplest example I could to prove my point, and hopefully to help find a solution to this important problem:

https://github.com/tliron/restlet-async-test

The current code works as expected -- but only because AsyncResource.delay is non-zero. If you set the value to zero, then the Restlet chain will do its work before the AsyncResource commits, and then the resource will not return the entity.

No exceptions get thrown -- the code is "thread safe" -- but it still shows that it's impossible to reliably handle async responses.

I hope other people can take a look and let me know if I missed something important.


> Jerome, even though setCommitted(false) gets called, there still the usual
> processing of handle() in the entire restlet chain (the Finder, delegating
> to the UniformResource, etc.). Within these routines, the return values from
> various callbacks are handled. Stuff that happens there on the response can
> override work being done in another thread. The response is thread safe, but
> it's still not the intended result.
>
> I don't have a test case right now, but it would be easy to produce: simply
> call setCommitted(false) in a ServerResource annotated @Get implementation,
> and immediately move processing to a thread, where commit() is eventually
> called. The worker handler thread would overlap with the work done in the
> regular Restlet handling thread and you would get weird statuses and
> response entities.
>
> The reason I suggest an exception (and I share Tim's unease with it) is that
> it's a way to stop the handling chain in the regular thread, so that only
> the new handler thread would have free reign on the response. But it doesn't
> have to be an exception. Another solution would be for ServerResource,
> UniformResource and Finder to be refactored specifically to handle the
> committed=false case. I worry, though, that future work or custom
> implementations would not handle the case properly.
>
> -Tal
>
> On Sun, Jul 11, 2010 at 9:00 PM, Jerome Louvel <jerome dot louvel at noelios dot com>wrote:
>
> >  Hi Tal and Tim,
> >
> >
> >
> > There is a way to signal an asynchronous response on a per-call basis,
> > calling Response#setCommitted(false).
> >
> >
> >
> > Then, if you call Response#commit() before the initial request thread
> > reaches back the connector, this should work fine as well. I’m not sure yet
> > how it could get ‘corrupted’. Do you have a precise scenario?
> >
> >
> >
> > Note that the NIO version of the asynchronous internal connector has been
> > moved to version 2.1. This feature should indeed be considered as preview
> > feature for 2.0.
> >
> >
> >
> > Best regards,
> > Jerome Louvel
> > --
> > Restlet ~ Founder and Technical Lead ~ http://www.restlet.org
> > Noelios Technologies ~ http://www.noelios.com
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > *De :* tpeierls at gmail dot com [mailto:tpeierls at gmail dot com] *De la part de* Tim
> > Peierls
> > *Envoyé :* vendredi 18 juin 2010 13:51
> >
> > *À :* discuss at restlet dot tigris dot org
> > *Objet :* Re: Status of Asynchronous Support
> >
> >
> >
> > I never saw a response to this, but it does seem like a problem.
> >
> >
> >
> > I don't love the idea of a special exception used purely to handle control
> > flow, but I can't think of anything better off-hand.
> >
> >
> >
> > --tim
> >
> > On Sat, May 22, 2010 at 8:05 PM, Tal Liron <tal dot liron at threecrickets dot com>
> > wrote:
> >
> > Hi Jerome,
> >
> >
> > > 2. There is a TaskService associated to your application that you could
> > > leverage. It separates threads usage from tasks to process (such as
> > > committing responses).
> >
> > I've tried this, but there is a concurrency problem with this pattern in
> > ServerResource --
> >
> > If I set autoCommiting to false during a ServerResource.get() and put my
> > task on a separate thread pool, then my own task might try to update the
> > response at the same time as the upper parts of the call stack
> > (Finder.handle(), for example) also try to update the response according
> > to my return value from get(). Both threads might be doing it at the
> > same time, and the response ends up corrupted (no concurrency
> > exceptions, but it's mismatched status/entity, for example).
> >
> > What would be the correct way to defer a response from within a
> > ServerResource?
> >
> > My thought is that there should be a way to return from get() while
> > signaling to the rest of the call stack that I am handling the response.
> > A null won't work, because it internally signifies an unavailable
> > entity. Perhaps a new kind of ResourceException?
> > DefferedResponseResourceException?
> >
> > -Tal
> >
> > ------------------------------------------------------
> >
> > http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=2612147
> >
> >
> >

------------------------------------------------------
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=2899008
Reply | Threaded
Open this post in threaded view
|

Re: Re: Status of Asynchronous Support

Tim Peierls
Jerome said earlier in this conversation:

The only way I see to solve this issue for now (2.1) would be to maintain a custom Restlet call stack containing the list of Filters that intercepted the inbound call and make sure that their afterHandle() method is invoked by the asynchronous response thread.

Trying to flesh this out in my head: So the same logic that calls beforeHandle would push the corresponding afterHandle call onto a stack associated with the response; setting committed false would prevent the normal afterHandle machinery from executing, and the commit() machinery would be given access to the stack so it could pop those afterHandles and call them?

Afaict, this isn't in yet, so I think it would be safer to say that asynch I/O is still in preview mode for 2.1. Am I being too pessimistic?

--tim



On Thu, Dec 22, 2011 at 12:56 AM, Tal Liron <[hidden email]> wrote:
I have created the simplest example I could to prove my point, and hopefully to help find a solution to this important problem:

https://github.com/tliron/restlet-async-test

The current code works as expected -- but only because AsyncResource.delay is non-zero. If you set the value to zero, then the Restlet chain will do its work before the AsyncResource commits, and then the resource will not return the entity.

No exceptions get thrown -- the code is "thread safe" -- but it still shows that it's impossible to reliably handle async responses.

I hope other people can take a look and let me know if I missed something important.


> Jerome, even though setCommitted(false) gets called, there still the usual
> processing of handle() in the entire restlet chain (the Finder, delegating
> to the UniformResource, etc.). Within these routines, the return values from
> various callbacks are handled. Stuff that happens there on the response can
> override work being done in another thread. The response is thread safe, but
> it's still not the intended result.
>
> I don't have a test case right now, but it would be easy to produce: simply
> call setCommitted(false) in a ServerResource annotated @Get implementation,
> and immediately move processing to a thread, where commit() is eventually
> called. The worker handler thread would overlap with the work done in the
> regular Restlet handling thread and you would get weird statuses and
> response entities.
>
> The reason I suggest an exception (and I share Tim's unease with it) is that
> it's a way to stop the handling chain in the regular thread, so that only
> the new handler thread would have free reign on the response. But it doesn't
> have to be an exception. Another solution would be for ServerResource,
> UniformResource and Finder to be refactored specifically to handle the
> committed=false case. I worry, though, that future work or custom
> implementations would not handle the case properly.
>
> -Tal
>
> On Sun, Jul 11, 2010 at 9:00 PM, Jerome Louvel <jerome dot louvel at noelios dot com>wrote:
>
> >  Hi Tal and Tim,
> >
> >
> >
> > There is a way to signal an asynchronous response on a per-call basis,
> > calling Response#setCommitted(false).
> >
> >
> >
> > Then, if you call Response#commit() before the initial request thread
> > reaches back the connector, this should work fine as well. I’m not sure yet
> > how it could get ‘corrupted’. Do you have a precise scenario?
> >
> >
> >
> > Note that the NIO version of the asynchronous internal connector has been
> > moved to version 2.1. This feature should indeed be considered as preview
> > feature for 2.0.
> >
> >
> >
> > Best regards,
> > Jerome Louvel
> > --
> > Restlet ~ Founder and Technical Lead ~ http://www.restlet.org
> > Noelios Technologies ~ http://www.noelios.com
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > *De :* tpeierls at gmail dot com [mailto:[hidden email] at gmail dot com] *De la part de* Tim
> > Peierls
> > *Envoyé :* vendredi 18 juin 2010 13:51
> >
> > *À :* discuss at restlet dot tigris dot org
> > *Objet :* Re: Status of Asynchronous Support
> >
> >
> >
> > I never saw a response to this, but it does seem like a problem.
> >
> >
> >
> > I don't love the idea of a special exception used purely to handle control
> > flow, but I can't think of anything better off-hand.
> >
> >
> >
> > --tim
> >
> > On Sat, May 22, 2010 at 8:05 PM, Tal Liron <tal dot liron at threecrickets dot com>
> > wrote:
> >
> > Hi Jerome,
> >
> >
> > > 2. There is a TaskService associated to your application that you could
> > > leverage. It separates threads usage from tasks to process (such as
> > > committing responses).
> >
> > I've tried this, but there is a concurrency problem with this pattern in
> > ServerResource --
> >
> > If I set autoCommiting to false during a ServerResource.get() and put my
> > task on a separate thread pool, then my own task might try to update the
> > response at the same time as the upper parts of the call stack
> > (Finder.handle(), for example) also try to update the response according
> > to my return value from get(). Both threads might be doing it at the
> > same time, and the response ends up corrupted (no concurrency
> > exceptions, but it's mismatched status/entity, for example).
> >
> > What would be the correct way to defer a response from within a
> > ServerResource?
> >
> > My thought is that there should be a way to return from get() while
> > signaling to the rest of the call stack that I am handling the response.
> > A null won't work, because it internally signifies an unavailable
> > entity. Perhaps a new kind of ResourceException?
> > DefferedResponseResourceException?
> >
> > -Tal
> >
> > ------------------------------------------------------
> >
> > http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=2612147
> >
> >
> >

------------------------------------------------------
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=2899008

Reply | Threaded
Open this post in threaded view
|

RE: Re: Status of Asynchronous Support

jlouvel
Administrator

Hi Tim,

 

Yes, this is a good summary of the API refactoring intent. Actually it would be a bit simpler as the stack would only contain Restlet instances part of the processing chain, not lower level Java methods.

 

Another key aspect is that a call to Restlet#handle() would not take care itself of invoking the next Restlet in the chain otherwise the stack wouldn’t be properly populated. Maybe the next Restlet should be the result of the call to Restlet#handle()…. “handle the call at your level and then return the control to the caller by indicating the next Restlet if any, and let the caller invoke this next Restlet after updating the Restlet stack”.

 

Best regards,

Jerome

--

http://www.restlet.org

http://twitter.com/#!/jlouvel

 

 

 

 

De : [hidden email] [mailto:[hidden email]] De la part de Tim Peierls
Envoyé : jeudi 22 décembre 2011 19:01
À : [hidden email]
Objet : Re: Re: Status of Asynchronous Support

 

Jerome said earlier in this conversation:

 

The only way I see to solve this issue for now (2.1) would be to maintain a custom Restlet call stack containing the list of Filters that intercepted the inbound call and make sure that their afterHandle() method is invoked by the asynchronous response thread.

 

Trying to flesh this out in my head: So the same logic that calls beforeHandle would push the corresponding afterHandle call onto a stack associated with the response; setting committed false would prevent the normal afterHandle machinery from executing, and the commit() machinery would be given access to the stack so it could pop those afterHandles and call them?

 

Afaict, this isn't in yet, so I think it would be safer to say that asynch I/O is still in preview mode for 2.1. Am I being too pessimistic?

 

--tim

 

 

 

On Thu, Dec 22, 2011 at 12:56 AM, Tal Liron <[hidden email]> wrote:

I have created the simplest example I could to prove my point, and hopefully to help find a solution to this important problem:

https://github.com/tliron/restlet-async-test

The current code works as expected -- but only because AsyncResource.delay is non-zero. If you set the value to zero, then the Restlet chain will do its work before the AsyncResource commits, and then the resource will not return the entity.

No exceptions get thrown -- the code is "thread safe" -- but it still shows that it's impossible to reliably handle async responses.

I hope other people can take a look and let me know if I missed something important.



> Jerome, even though setCommitted(false) gets called, there still the usual
> processing of handle() in the entire restlet chain (the Finder, delegating
> to the UniformResource, etc.). Within these routines, the return values from
> various callbacks are handled. Stuff that happens there on the response can
> override work being done in another thread. The response is thread safe, but
> it's still not the intended result.
>
> I don't have a test case right now, but it would be easy to produce: simply
> call setCommitted(false) in a ServerResource annotated @Get implementation,
> and immediately move processing to a thread, where commit() is eventually
> called. The worker handler thread would overlap with the work done in the
> regular Restlet handling thread and you would get weird statuses and
> response entities.
>
> The reason I suggest an exception (and I share Tim's unease with it) is that
> it's a way to stop the handling chain in the regular thread, so that only
> the new handler thread would have free reign on the response. But it doesn't
> have to be an exception. Another solution would be for ServerResource,
> UniformResource and Finder to be refactored specifically to handle the
> committed=false case. I worry, though, that future work or custom
> implementations would not handle the case properly.
>
> -Tal
>

> On Sun, Jul 11, 2010 at 9:00 PM, Jerome Louvel <jerome dot louvel at noelios dot com>wrote:

>
> >  Hi Tal and Tim,
> >
> >
> >
> > There is a way to signal an asynchronous response on a per-call basis,
> > calling Response#setCommitted(false).
> >
> >
> >
> > Then, if you call Response#commit() before the initial request thread
> > reaches back the connector, this should work fine as well. I’m not sure yet
> > how it could get ‘corrupted’. Do you have a precise scenario?
> >
> >
> >
> > Note that the NIO version of the asynchronous internal connector has been
> > moved to version 2.1. This feature should indeed be considered as preview
> > feature for 2.0.
> >
> >
> >
> > Best regards,
> > Jerome Louvel
> > --
> > Restlet ~ Founder and Technical Lead ~ http://www.restlet.org
> > Noelios Technologies ~ http://www.noelios.com
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >

> > *De :* tpeierls at gmail dot com [mailto:[hidden email] at gmail dot com] *De la part de* Tim
> > Peierls
> > *Envoyé :* vendredi 18 juin 2010 13:51
> >
> > *À :* discuss at restlet dot tigris dot org
> > *Objet :* Re: Status of Asynchronous Support

> >
> >
> >
> > I never saw a response to this, but it does seem like a problem.
> >
> >
> >
> > I don't love the idea of a special exception used purely to handle control
> > flow, but I can't think of anything better off-hand.
> >
> >
> >
> > --tim
> >

> > On Sat, May 22, 2010 at 8:05 PM, Tal Liron <tal dot liron at threecrickets dot com>

> > wrote:
> >
> > Hi Jerome,
> >
> >
> > > 2. There is a TaskService associated to your application that you could
> > > leverage. It separates threads usage from tasks to process (such as
> > > committing responses).
> >
> > I've tried this, but there is a concurrency problem with this pattern in
> > ServerResource --
> >
> > If I set autoCommiting to false during a ServerResource.get() and put my
> > task on a separate thread pool, then my own task might try to update the
> > response at the same time as the upper parts of the call stack
> > (Finder.handle(), for example) also try to update the response according
> > to my return value from get(). Both threads might be doing it at the
> > same time, and the response ends up corrupted (no concurrency
> > exceptions, but it's mismatched status/entity, for example).
> >
> > What would be the correct way to defer a response from within a
> > ServerResource?
> >
> > My thought is that there should be a way to return from get() while
> > signaling to the rest of the call stack that I am handling the response.
> > A null won't work, because it internally signifies an unavailable
> > entity. Perhaps a new kind of ResourceException?
> > DefferedResponseResourceException?
> >
> > -Tal
> >
> > ------------------------------------------------------
> >
> > http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=2612147
> >
> >
> >

------------------------------------------------------

http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=2899008

 

Reply | Threaded
Open this post in threaded view
|

RE: Re: Status of Asynchronous Support

Tal Liron
Jerome,

What about my idea of throwing a special exception, something like "ResponseCommitDeferredException"? Exceptions are an easy and natural way in the JVM to unwind a call stack. The exception would go up to the lowest possible level, where it explicitly caught and set to wait. Nothing along the way would have to know any specific about it.


> Hi Tim,
>
>  
>
> Yes, this is a good summary of the API refactoring intent. Actually it would
> be a bit simpler as the stack would only contain Restlet instances part of
> the processing chain, not lower level Java methods.
>
>  
>
> Another key aspect is that a call to Restlet#handle() would not take care
> itself of invoking the next Restlet in the chain otherwise the stack
> wouldn’t be properly populated. Maybe the next Restlet should be the result
> of the call to Restlet#handle()…. “handle the call at your level and then
> return the control to the caller by indicating the next Restlet if any, and
> let the caller invoke this next Restlet after updating the Restlet stack”.
>
>  
>
> Best regards,
>
> Jerome
>
> --
>
>  <http://www.restlet.org/> http://www.restlet.org
>
>  <http://twitter.com/#!/jlouvel> http://twitter.com/#!/jlouvel
>
>  
>
>  
>
>  
>
>  
>
> De : tpeierls at gmail dot com [mailto:tpeierls at gmail dot com] De la part de Tim
> Peierls
> Envoyé : jeudi 22 décembre 2011 19:01
> À : discuss at restlet dot tigris dot org
> Objet : Re: Re: Status of Asynchronous Support
>
>  
>
> Jerome said earlier in this conversation:
>
>  
>
> The only way I see to solve this issue for now (2.1) would be to maintain a
> custom Restlet call stack containing the list of Filters that intercepted
> the inbound call and make sure that their afterHandle() method is invoked by
> the asynchronous response thread.
>
>  
>
> Trying to flesh this out in my head: So the same logic that calls
> beforeHandle would push the corresponding afterHandle call onto a stack
> associated with the response; setting committed false would prevent the
> normal afterHandle machinery from executing, and the commit() machinery
> would be given access to the stack so it could pop those afterHandles and
> call them?
>
>  
>
> Afaict, this isn't in yet, so I think it would be safer to say that asynch
> I/O is still in preview mode for 2.1. Am I being too pessimistic?
>
>  
>
> --tim
>
>  
>
>  
>
>  
>
> On Thu, Dec 22, 2011 at 12:56 AM, Tal Liron <tal dot liron at threecrickets dot com>
> wrote:
>
> I have created the simplest example I could to prove my point, and hopefully
> to help find a solution to this important problem:
>
> https://github.com/tliron/restlet-async-test
>
> The current code works as expected -- but only because AsyncResource.delay
> is non-zero. If you set the value to zero, then the Restlet chain will do
> its work before the AsyncResource commits, and then the resource will not
> return the entity.
>
> No exceptions get thrown -- the code is "thread safe" -- but it still shows
> that it's impossible to reliably handle async responses.
>
> I hope other people can take a look and let me know if I missed something
> important.
>
>
>
> > Jerome, even though setCommitted(false) gets called, there still the usual
> > processing of handle() in the entire restlet chain (the Finder, delegating
> > to the UniformResource, etc.). Within these routines, the return values
> from
> > various callbacks are handled. Stuff that happens there on the response
> can
> > override work being done in another thread. The response is thread safe,
> but
> > it's still not the intended result.
> >
> > I don't have a test case right now, but it would be easy to produce:
> simply
> > call setCommitted(false) in a ServerResource annotated @Get
> implementation,
> > and immediately move processing to a thread, where commit() is eventually
> > called. The worker handler thread would overlap with the work done in the
> > regular Restlet handling thread and you would get weird statuses and
> > response entities.
> >
> > The reason I suggest an exception (and I share Tim's unease with it) is
> that
> > it's a way to stop the handling chain in the regular thread, so that only
> > the new handler thread would have free reign on the response. But it
> doesn't
> > have to be an exception. Another solution would be for ServerResource,
> > UniformResource and Finder to be refactored specifically to handle the
> > committed=false case. I worry, though, that future work or custom
> > implementations would not handle the case properly.
> >
> > -Tal
> >
>
> > On Sun, Jul 11, 2010 at 9:00 PM, Jerome Louvel <jerome dot louvel at
> noelios dot com>wrote:
>
> >
> > >  Hi Tal and Tim,
> > >
> > >
> > >
> > > There is a way to signal an asynchronous response on a per-call basis,
> > > calling Response#setCommitted(false).
> > >
> > >
> > >
> > > Then, if you call Response#commit() before the initial request thread
> > > reaches back the connector, this should work fine as well. I’m not sure
> yet
> > > how it could get ‘corrupted’. Do you have a precise scenario?
> > >
> > >
> > >
> > > Note that the NIO version of the asynchronous internal connector has
> been
> > > moved to version 2.1. This feature should indeed be considered as
> preview
> > > feature for 2.0.
> > >
> > >
> > >
> > > Best regards,
> > > Jerome Louvel
> > > --
> > > Restlet ~ Founder and Technical Lead ~ http://www.restlet.org
> > > Noelios Technologies ~ http://www.noelios.com
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
>
> > > *De :* tpeierls at gmail dot com [mailto:tpeierls at gmail dot com] *De
> la part de* Tim
> > > Peierls
> > > *Envoyé :* vendredi 18 juin 2010 13:51
> > >
> > > *À :* discuss at restlet dot tigris dot org
> > > *Objet :* Re: Status of Asynchronous Support
>
> > >
> > >
> > >
> > > I never saw a response to this, but it does seem like a problem.
> > >
> > >
> > >
> > > I don't love the idea of a special exception used purely to handle
> control
> > > flow, but I can't think of anything better off-hand.
> > >
> > >
> > >
> > > --tim
> > >
>
> > > On Sat, May 22, 2010 at 8:05 PM, Tal Liron <tal dot liron at
> threecrickets dot com>
>
> > > wrote:
> > >
> > > Hi Jerome,
> > >
> > >
> > > > 2. There is a TaskService associated to your application that you
> could
> > > > leverage. It separates threads usage from tasks to process (such as
> > > > committing responses).
> > >
> > > I've tried this, but there is a concurrency problem with this pattern in
> > > ServerResource --
> > >
> > > If I set autoCommiting to false during a ServerResource.get() and put my
> > > task on a separate thread pool, then my own task might try to update the
> > > response at the same time as the upper parts of the call stack
> > > (Finder.handle(), for example) also try to update the response according
> > > to my return value from get(). Both threads might be doing it at the
> > > same time, and the response ends up corrupted (no concurrency
> > > exceptions, but it's mismatched status/entity, for example).
> > >
> > > What would be the correct way to defer a response from within a
> > > ServerResource?
> > >
> > > My thought is that there should be a way to return from get() while
> > > signaling to the rest of the call stack that I am handling the response.
> > > A null won't work, because it internally signifies an unavailable
> > > entity. Perhaps a new kind of ResourceException?
> > > DefferedResponseResourceException?
> > >
> > > -Tal
> > >
> > > ------------------------------------------------------
> > >
> > > http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447
> <http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=2612
> 147> &dsMessageId=2612147
> > >
> > >
> > >
>
> ------------------------------------------------------
> http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447
> <http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=2899
> 008> &dsMessageId=2899008

------------------------------------------------------
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=2899300
Reply | Threaded
Open this post in threaded view
|

Re: Re: Status of Asynchronous Support

Tim Peierls
My understanding of Jerome's e-mail of 2010-Aug-30 (has it really been that long?) is that it's not sufficient to unwind the stack, whether by exception or by testing response state. The main problem is arranging for the calls to afterHandle  to occur in a different thread. These are calls that would have been executed on a committed response as part of the normal stack unwind.

While it might be a reasonable implementation choice to use an exception internally, it's not a real exception that can be allowed to propagate arbitrarily, because it's OK, not exceptional, for the original thread to return without a committed response.

This is a tough problem.

--tim

On Fri, Dec 23, 2011 at 3:23 AM, Tal Liron <[hidden email]> wrote:
Jerome,

What about my idea of throwing a special exception, something like "ResponseCommitDeferredException"? Exceptions are an easy and natural way in the JVM to unwind a call stack. The exception would go up to the lowest possible level, where it explicitly caught and set to wait. Nothing along the way would have to know any specific about it.


> Hi Tim,
>
>
>
> Yes, this is a good summary of the API refactoring intent. Actually it would
> be a bit simpler as the stack would only contain Restlet instances part of
> the processing chain, not lower level Java methods.
>
>
>
> Another key aspect is that a call to Restlet#handle() would not take care
> itself of invoking the next Restlet in the chain otherwise the stack
> wouldn’t be properly populated. Maybe the next Restlet should be the result
> of the call to Restlet#handle()…. “handle the call at your level and then
> return the control to the caller by indicating the next Restlet if any, and
> let the caller invoke this next Restlet after updating the Restlet stack”.
>
>
>
> Best regards,
>
> Jerome
>
> --
>
>  <http://www.restlet.org/> http://www.restlet.org
>
>  <http://twitter.com/#!/jlouvel> http://twitter.com/#!/jlouvel
>
>
>
>
>
>
>
>
>
> De : tpeierls at gmail dot com [mailto:[hidden email] at gmail dot com] De la part de Tim
> Peierls
> Envoyé : jeudi 22 décembre 2011 19:01
> À : discuss at restlet dot tigris dot org
> Objet : Re: Re: Status of Asynchronous Support
>
>
>
> Jerome said earlier in this conversation:
>
>
>
> The only way I see to solve this issue for now (2.1) would be to maintain a
> custom Restlet call stack containing the list of Filters that intercepted
> the inbound call and make sure that their afterHandle() method is invoked by
> the asynchronous response thread.
>
>
>
> Trying to flesh this out in my head: So the same logic that calls
> beforeHandle would push the corresponding afterHandle call onto a stack
> associated with the response; setting committed false would prevent the
> normal afterHandle machinery from executing, and the commit() machinery
> would be given access to the stack so it could pop those afterHandles and
> call them?
>
>
>
> Afaict, this isn't in yet, so I think it would be safer to say that asynch
> I/O is still in preview mode for 2.1. Am I being too pessimistic?
>
>
>
> --tim
>
>
>
>
>
>
>
> On Thu, Dec 22, 2011 at 12:56 AM, Tal Liron <tal dot liron at threecrickets dot com>
> wrote:
>
> I have created the simplest example I could to prove my point, and hopefully
> to help find a solution to this important problem:
>
> https://github.com/tliron/restlet-async-test
>
> The current code works as expected -- but only because AsyncResource.delay
> is non-zero. If you set the value to zero, then the Restlet chain will do
> its work before the AsyncResource commits, and then the resource will not
> return the entity.
>
> No exceptions get thrown -- the code is "thread safe" -- but it still shows
> that it's impossible to reliably handle async responses.
>
> I hope other people can take a look and let me know if I missed something
> important.
>
>
>
> > Jerome, even though setCommitted(false) gets called, there still the usual
> > processing of handle() in the entire restlet chain (the Finder, delegating
> > to the UniformResource, etc.). Within these routines, the return values
> from
> > various callbacks are handled. Stuff that happens there on the response
> can
> > override work being done in another thread. The response is thread safe,
> but
> > it's still not the intended result.
> >
> > I don't have a test case right now, but it would be easy to produce:
> simply
> > call setCommitted(false) in a ServerResource annotated @Get
> implementation,
> > and immediately move processing to a thread, where commit() is eventually
> > called. The worker handler thread would overlap with the work done in the
> > regular Restlet handling thread and you would get weird statuses and
> > response entities.
> >
> > The reason I suggest an exception (and I share Tim's unease with it) is
> that
> > it's a way to stop the handling chain in the regular thread, so that only
> > the new handler thread would have free reign on the response. But it
> doesn't
> > have to be an exception. Another solution would be for ServerResource,
> > UniformResource and Finder to be refactored specifically to handle the
> > committed=false case. I worry, though, that future work or custom
> > implementations would not handle the case properly.
> >
> > -Tal
> >
>
> > On Sun, Jul 11, 2010 at 9:00 PM, Jerome Louvel <jerome dot louvel at
> noelios dot com>wrote:
>
> >
> > >  Hi Tal and Tim,
> > >
> > >
> > >
> > > There is a way to signal an asynchronous response on a per-call basis,
> > > calling Response#setCommitted(false).
> > >
> > >
> > >
> > > Then, if you call Response#commit() before the initial request thread
> > > reaches back the connector, this should work fine as well. I’m not sure
> yet
> > > how it could get ‘corrupted’. Do you have a precise scenario?
> > >
> > >
> > >
> > > Note that the NIO version of the asynchronous internal connector has
> been
> > > moved to version 2.1. This feature should indeed be considered as
> preview
> > > feature for 2.0.
> > >
> > >
> > >
> > > Best regards,
> > > Jerome Louvel
> > > --
> > > Restlet ~ Founder and Technical Lead ~ http://www.restlet.org
> > > Noelios Technologies ~ http://www.noelios.com
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
>
> > > *De :* tpeierls at gmail dot com [mailto:[hidden email] at gmail dot com] *De
> la part de* Tim
> > > Peierls
> > > *Envoyé :* vendredi 18 juin 2010 13:51
> > >
> > > *À :* discuss at restlet dot tigris dot org
> > > *Objet :* Re: Status of Asynchronous Support
>
> > >
> > >
> > >
> > > I never saw a response to this, but it does seem like a problem.
> > >
> > >
> > >
> > > I don't love the idea of a special exception used purely to handle
> control
> > > flow, but I can't think of anything better off-hand.
> > >
> > >
> > >
> > > --tim
> > >
>
> > > On Sat, May 22, 2010 at 8:05 PM, Tal Liron <tal dot liron at
> threecrickets dot com>
>
> > > wrote:
> > >
> > > Hi Jerome,
> > >
> > >
> > > > 2. There is a TaskService associated to your application that you
> could
> > > > leverage. It separates threads usage from tasks to process (such as
> > > > committing responses).
> > >
> > > I've tried this, but there is a concurrency problem with this pattern in
> > > ServerResource --
> > >
> > > If I set autoCommiting to false during a ServerResource.get() and put my
> > > task on a separate thread pool, then my own task might try to update the
> > > response at the same time as the upper parts of the call stack
> > > (Finder.handle(), for example) also try to update the response according
> > > to my return value from get(). Both threads might be doing it at the
> > > same time, and the response ends up corrupted (no concurrency
> > > exceptions, but it's mismatched status/entity, for example).
> > >
> > > What would be the correct way to defer a response from within a
> > > ServerResource?
> > >
> > > My thought is that there should be a way to return from get() while
> > > signaling to the rest of the call stack that I am handling the response.
> > > A null won't work, because it internally signifies an unavailable
> > > entity. Perhaps a new kind of ResourceException?
> > > DefferedResponseResourceException?
> > >
> > > -Tal
> > >
> > > ------------------------------------------------------
> > >
> > > http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447
> <http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=2612
> 147> &dsMessageId=2612147
> > >
> > >
> > >
>
> ------------------------------------------------------
> http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447
> <http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=2899
> 008> &dsMessageId=2899008

------------------------------------------------------
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=2899300

Reply | Threaded
Open this post in threaded view
|

RE: Re: Re: Status of Asynchronous Support

Tal Liron
I see.

Well, how about this:

Perhaps setAutoCommit(false) might be the wrong approach. If anybody else along the chain tries to commit, it should fail somehow. How, then, would response.commit() know when it's time to really, truly commit?

My idea is to make a commitment to commit (ha!) via a kind of token, which would identify the committer. The flow would be something like this:

Object token = response.deferCommit();
...
From now on, the response has a flag telling it that commitment is deferred, and it will fail on all commit() calls, unless the token is provided.
...
Later, on some other thread:
...
response.commit(token);
...
The above will actually commit, because the token was provided.

Hopefully my test code can be turned into a unit test for whatever solution is found (though reliable unit tests for race conditions are notoriously hard to get right).



> My understanding of Jerome's e-mail of 2010-Aug-30 (has it really been that
> long?) is that it's not sufficient to unwind the stack, whether by
> exception or by testing response state. The main problem is arranging for
> the calls to afterHandle  to occur in a different thread. These are
> calls that would have been executed on a committed response as part of the
> normal stack unwind.
>
> While it *might *be a reasonable implementation choice to use an exception
> internally, it's not a real exception that can be allowed to propagate
> arbitrarily, because it's OK, not exceptional, for the original thread to
> return without a committed response.
>
> This is a tough problem.
>
> --tim

------------------------------------------------------
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=2899593
Reply | Threaded
Open this post in threaded view
|

Re: Re: Re: Status of Asynchronous Support

Tim Peierls
But doesn't that still leave the problem of arranging for all the afterHandle calls to happen in the thread that does that commit? That's what scares me.

On Sat, Dec 24, 2011 at 3:31 AM, Tal Liron <[hidden email]> wrote:
I see.

Well, how about this:

Perhaps setAutoCommit(false) might be the wrong approach. If anybody else along the chain tries to commit, it should fail somehow. How, then, would response.commit() know when it's time to really, truly commit?

My idea is to make a commitment to commit (ha!) via a kind of token, which would identify the committer. The flow would be something like this:

Object token = response.deferCommit();
...
From now on, the response has a flag telling it that commitment is deferred, and it will fail on all commit() calls, unless the token is provided.
...
Later, on some other thread:
...
response.commit(token);
...
The above will actually commit, because the token was provided.

Hopefully my test code can be turned into a unit test for whatever solution is found (though reliable unit tests for race conditions are notoriously hard to get right).



> My understanding of Jerome's e-mail of 2010-Aug-30 (has it really been that
> long?) is that it's not sufficient to unwind the stack, whether by
> exception or by testing response state. The main problem is arranging for
> the calls to afterHandle  to occur in a different thread. These are
> calls that would have been executed on a committed response as part of the
> normal stack unwind.
>
> While it *might *be a reasonable implementation choice to use an exception
> internally, it's not a real exception that can be allowed to propagate
> arbitrarily, because it's OK, not exceptional, for the original thread to
> return without a committed response.
>
> This is a tough problem.
>
> --tim

------------------------------------------------------
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=2899593

Reply | Threaded
Open this post in threaded view
|

RE: Re: Re: Re: Status of Asynchronous Support

Tal Liron
I would have two answers:

1) Isn't that the price to pay for asynchronicity, and the desire of the user? The whole point is to implicate a different set of threads into communication with the client. Of course, hybrid thread-pool-plus-event-driven servers come to mind, which queue up event responses but handle them on a single thread pool. There are advantages to that, but it didn't seem to be the design goal for Restlet.

2) It could be handled by the application's TaskService. Still, I would imagine that the user would not always want that. How about this:

response.commit(token)

...to commit in the current thread, and...

response.commitLater(token)

...to commit in the TaskService.

> But doesn't that still leave the problem of arranging for all the
> afterHandle calls to happen in the thread that does that commit? That's
> what scares me.

------------------------------------------------------
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=2899780
Reply | Threaded
Open this post in threaded view
|

Re: Re: Re: Re: Status of Asynchronous Support

Tim Peierls
I think we're in violent agreement. I'm just worried about the difficulties of arranging for the afterHandle calls to occur in a different thread. It's a huge change with lots of potential for hard-to-detect bugs. I'm not arguing against doing it.

--tim

On Sun, Dec 25, 2011 at 12:02 AM, Tal Liron <[hidden email]> wrote:
I would have two answers:

1) Isn't that the price to pay for asynchronicity, and the desire of the user? The whole point is to implicate a different set of threads into communication with the client. Of course, hybrid thread-pool-plus-event-driven servers come to mind, which queue up event responses but handle them on a single thread pool. There are advantages to that, but it didn't seem to be the design goal for Restlet.

2) It could be handled by the application's TaskService. Still, I would imagine that the user would not always want that. How about this:

response.commit(token)

...to commit in the current thread, and...

response.commitLater(token)

...to commit in the TaskService.

> But doesn't that still leave the problem of arranging for all the
> afterHandle calls to happen in the thread that does that commit? That's
> what scares me.

------------------------------------------------------
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=2899780

Reply | Threaded
Open this post in threaded view
|

RE: Re: Re: Re: Status of Asynchronous Support

jlouvel
Administrator

Hi Tim, Tal,

 

It seems that we all better see the implications. Regarding the exception idea, this looks similar to Jetty continuations mechanism, which is IMHO working but complex and unintuitive: http://wiki.eclipse.org/Jetty/Feature/Continuations

 

Can we do better/easier? If we step back a little, here is the list of design issues/goals we want to address:

·         Handling of HTTP transactions including one request, potentially several provisional responses and a final responses

·         The current Uniform interface strongly implies that there is only one request and one (final) response possible, corresponding to the objects provided as parameters

·         We want to allow several threads to process the messages of the same transaction in order to:

o   resume sending after a long request processing

o   progressively send a response entity (such as Server-Sent Events)

o   progressively retrieve a large request entity

·         Consistently behave on both client and server sides

·         Allow assembly of a processing Restlets chain including routers and filters

·         Ensure that the default scenario (synchronous processing with one request and one response) stays simple (as simple as today?)

·         Ensure that server-side request can be proxied on the client-side (using Redirector in outbound mode for example)

 

Looking at all those issues, I (still) think that the cleanest way is to view the Restlet processing chain as a directed and dynamic graph where Restlet instances are nodes processing requests coming in first and then coming back (unless it is a leaf node).

 

In term of pseudo-code, here is what I have in mind for the Restlet class:

·          [final] handle(Request, Response) : void                           // capable of sync or async handling, manages processing stack

o   calls “handle(req, resp, true)”

·         [final] handle(Request, Response, boolean incoming) : void

o   For incoming calls

·         add the Restlet at the top of the processing stack

·         invoke handleIn(…)

·         if a Restlet is returned, invoke handle(…) on it

·         if the response has been synchronously committed, invoke handleOut(…)

o   For outgoing calls

·         if the current Restlet isn’t at the top of the processing stack, throw an error

·         invoke handleOut(…)

·         remove the Restlet from the top of the processing stack

·         if the response has been asynchronously committed, invoke handle(req, resp, false) on the Restlet at the top of the stack

·         [protected] handleIn(Request, Response) : Restlet      // handle incoming calls, returning any next Restlet to call

o   Intended to be overridden

·         [protected] handleOut(Request, Response) : void                        // handle calls going out back to the client from further Restlets in the stack

o   Intended to be overridden

 

Makes sense? :)

 

BTW, I’ve updated the related wiki page with those notes:

http://wiki.restlet.org/developers/172-restlet/g1/297-restlet.html

 

Best regards,

Jerome

--

http://www.restlet.org

http://twitter.com/#!/jlouvel

 

 

 

De : [hidden email] [mailto:[hidden email]] De la part de Tim Peierls
Envoyé : dimanche 25 décembre 2011 14:13
À : [hidden email]
Objet : Re: Re: Re: Re: Status of Asynchronous Support

 

I think we're in violent agreement. I'm just worried about the difficulties of arranging for the afterHandle calls to occur in a different thread. It's a huge change with lots of potential for hard-to-detect bugs. I'm not arguing against doing it.

 

--tim

 

On Sun, Dec 25, 2011 at 12:02 AM, Tal Liron <[hidden email]> wrote:

I would have two answers:

1) Isn't that the price to pay for asynchronicity, and the desire of the user? The whole point is to implicate a different set of threads into communication with the client. Of course, hybrid thread-pool-plus-event-driven servers come to mind, which queue up event responses but handle them on a single thread pool. There are advantages to that, but it didn't seem to be the design goal for Restlet.

2) It could be handled by the application's TaskService. Still, I would imagine that the user would not always want that. How about this:

response.commit(token)

...to commit in the current thread, and...

response.commitLater(token)

...to commit in the TaskService.


> But doesn't that still leave the problem of arranging for all the
> afterHandle calls to happen in the thread that does that commit? That's
> what scares me.

------------------------------------------------------
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=2899780

 

Reply | Threaded
Open this post in threaded view
|

RE: Re: Re: Re: Status of Asynchronous Support

jlouvel
Administrator
In reply to this post by Tim Peierls

Hi Tim, Tal,

 

It seems that we all better see the implications. Regarding the exception idea, this looks similar to Jetty continuations mechanism, which is IMHO working but complex and unintuitive: http://wiki.eclipse.org/Jetty/Feature/Continuations

 

Can we do better/easier? If we step back a little, here is the list of design issues/goals we want to address:

·         Handling of HTTP transactions including one request, potentially several provisional responses and a final responses

·         The current Uniform interface strongly implies that there is only one request and one (final) response possible, corresponding to the objects provided as parameters

·         We want to allow several threads to process the messages of the same transaction in order to:

o   resume sending after a long request processing

o   progressively send a response entity (such as Server-Sent Events)

o   progressively retrieve a large request entity

·         Consistently behave on both client and server sides

·         Allow assembly of a processing Restlets chain including routers and filters

·         Ensure that the default scenario (synchronous processing with one request and one response) stays simple (as simple as today?)

·         Ensure that server-side request can be proxied on the client-side (using Redirector in outbound mode for example)

 

Looking at all those issues, I (still) think that the cleanest way is to view the Restlet processing chain as a directed and dynamic graph where Restlet instances are nodes processing requests coming in first and then coming back (unless it is a leaf node).

 

In term of pseudo-code, here is what I have in mind for the Restlet class:

·          [final] handle(Request, Response) : void                           // capable of sync or async handling, manages processing stack

o   calls “handle(req, resp, true)”

·         [final] handle(Request, Response, boolean incoming) : void

o   For incoming calls

·         add the Restlet at the top of the processing stack

·         invoke handleIn(…)

·         if a Restlet is returned, invoke handle(…) on it

·         if the response has been synchronously committed, invoke handleOut(…)

o   For outgoing calls

·         if the current Restlet isn’t at the top of the processing stack, throw an error

·         invoke handleOut(…)

·         remove the Restlet from the top of the processing stack

·         if the response has been asynchronously committed, invoke handle(req, resp, false) on the Restlet at the top of the stack

·         [protected] handleIn(Request, Response) : Restlet      // handle incoming calls, returning any next Restlet to call

o   Intended to be overridden

·         [protected] handleOut(Request, Response) : void                        // handle calls going out back to the client from further Restlets in the stack

o   Intended to be overridden

 

Makes sense? :)

 

BTW, I’ve updated the related wiki page with those notes:

http://wiki.restlet.org/developers/172-restlet/g1/297-restlet.html

 

Best regards,

Jerome

--

http://www.restlet.org

http://twitter.com/#!/jlouvel

 

 

De : [hidden email] [mailto:[hidden email]] De la part de Tim Peierls
Envoyé : dimanche 25 décembre 2011 14:13
À : [hidden email]
Objet : Re: Re: Re: Re: Status of Asynchronous Support

 

I think we're in violent agreement. I'm just worried about the difficulties of arranging for the afterHandle calls to occur in a different thread. It's a huge change with lots of potential for hard-to-detect bugs. I'm not arguing against doing it.

 

--tim

 

On Sun, Dec 25, 2011 at 12:02 AM, Tal Liron <[hidden email]> wrote:

I would have two answers:

1) Isn't that the price to pay for asynchronicity, and the desire of the user? The whole point is to implicate a different set of threads into communication with the client. Of course, hybrid thread-pool-plus-event-driven servers come to mind, which queue up event responses but handle them on a single thread pool. There are advantages to that, but it didn't seem to be the design goal for Restlet.

2) It could be handled by the application's TaskService. Still, I would imagine that the user would not always want that. How about this:

response.commit(token)

...to commit in the current thread, and...

response.commitLater(token)

...to commit in the TaskService.


> But doesn't that still leave the problem of arranging for all the
> afterHandle calls to happen in the thread that does that commit? That's
> what scares me.

------------------------------------------------------
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=2899780

 

Reply | Threaded
Open this post in threaded view
|

RE: Re: Re: Re: Status of Asynchronous Support

Tal Liron
In reply to this post by jlouvel
Jerome,

You make a lot of sense.

I agree with you that Jetty's solution is complicated, but I was 1) hoping we could do it more straightforwardly in Restlet (possibly a naive hope), and 2) a graph (and other) solutions would have their own complexity. There's really no simple way out of this. That said, the graph approach is programatically more comprehensible, and also more idiomatic to Java, than simulating continuations.

(I'd also add that functional languages would make this problem much more comprehensible, but I'm afraid that Tim would bite my head off, so I won't...)

I was also thinking of the problem in a far more local sense -- a single thread "committing" -- rather than the more complex scenarios you've listed, of chunked responses, which I imagine would be extremely useful in the new streaming applications to which Restlet is being applied, with Noelios' terrific new partnerships.

So, yes, I'm fully in agreement with the graph approach, it is comprehensive and can potentially address all needs. So, it'll be ready next week? ;)

Happy new year!

------------------------------------------------------
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=2903175
Reply | Threaded
Open this post in threaded view
|

Re: Re: Re: Re: Status of Asynchronous Support

Tim Peierls
On Sat, Dec 31, 2011 at 3:42 AM, Tal Liron <[hidden email]> wrote:
(I'd also add that functional languages would make this problem much more comprehensible, but I'm afraid that Tim would bite my head off, so I won't...)

No head-biter I, but while I agree that you could probably express things more neatly in a functional way, I don't see how that brings us any closer to a Java solution than Jerome's proposal already gets us.

Jerome, wouldn't your proposal break code that overrides handle(Req, Rsp)? I guess that's an acceptable incompatibility, but I wonder if there's a way that existing overrides could be detected and handled purely synchronously.

--tim