Transparent reverse proxying using org.restlet.routing.Redirector

classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Transparent reverse proxying using org.restlet.routing.Redirector

Primož Kokol
I am trying to create a transparent reverse proxy using org.restlet.routing.Redirector.

For the sake of simplicity let's say all I want to do is to redirect all requests pointing at

http://localhost:80

to be dispatched to another unrelated server:

http://localhost:8080

I've wrote a simple reverse proxy using Redirector (MODE_SERVER_OUTBOUND mode) and it actually works as expected in the sense that it dispatches requests properly and also handles redirects.

But now let's say that resources at http://localhost:8080 are protected with Digest authentication mechanism.

Now the problem is that headers in requests and responses when handled by Redirector are removed which makes Digest authentication mechanism unusable.

From docs WRT MODE_SERVER_OUTBOUND:
----
Note that in this mode, the headers of HTTP requests, stored in the request's attributes, are removed before dispatching. Also, when a HTTP response comes back the headers are also removed.
----

Is there any way to leave headers intact so that end server (http://localhost:8080) and clients could communicate as there is no proxy in between?

------------------------------------------------------
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=3076621
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Transparent reverse proxying using org.restlet.routing.Redirector

Tim Peierls
The code that does the header removal is here:


If you're confident that that's what you want, you can extend Redirector and override serverRedirect to do everything it normally except remove those headers.

--tim



On Wed, Apr 16, 2014 at 9:52 AM, Primož Kokol <[hidden email]> wrote:
I am trying to create a transparent reverse proxy using org.restlet.routing.Redirector.

For the sake of simplicity let's say all I want to do is to redirect all requests pointing at

http://localhost:80

to be dispatched to another unrelated server:

http://localhost:8080

I've wrote a simple reverse proxy using Redirector (MODE_SERVER_OUTBOUND mode) and it actually works as expected in the sense that it dispatches requests properly and also handles redirects.

But now let's say that resources at http://localhost:8080 are protected with Digest authentication mechanism.

Now the problem is that headers in requests and responses when handled by Redirector are removed which makes Digest authentication mechanism unusable.

From docs WRT MODE_SERVER_OUTBOUND:
----
Note that in this mode, the headers of HTTP requests, stored in the request's attributes, are removed before dispatching. Also, when a HTTP response comes back the headers are also removed.
----

Is there any way to leave headers intact so that end server (http://localhost:8080) and clients could communicate as there is no proxy in between?

------------------------------------------------------
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=3076621

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: Re: Transparent reverse proxying using org.restlet.routing.Redirector

Primož Kokol
Tim, thanks for advice.

That is actually one of the first things I've tried.

After extending Redirector and overriding serverRedirect  method so that line which clears the headers was removed 1002 Internal Connector Error was always returned as response (from proxy) so I thought this was not the right approach.

Any idea what could be cause of 1002 Internal Connector Error in this case?

------------------------------------------------------
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=3076674
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Re: Transparent reverse proxying using org.restlet.routing.Redirector

Tim Peierls
I'm guessing you have to very selective about which headers you allow to remain.


On Thu, Apr 17, 2014 at 3:08 PM, Primož Kokol <[hidden email]> wrote:
Tim, thanks for advice.

That is actually one of the first things I've tried.

After extending Redirector and overriding serverRedirect  method so that line which clears the headers was removed 1002 Internal Connector Error was always returned as response (from proxy) so I thought this was not the right approach.

Any idea what could be cause of 1002 Internal Connector Error in this case?

------------------------------------------------------
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=3076674

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: Re: Re: Transparent reverse proxying using org.restlet.routing.Redirector

Primož Kokol
Hi again,

My next try was to remove all original headers (like default implementation do) and then preserver only Authentication header.

So I replaced this line:

https://github.com/restlet/restlet-framework-java/blob/2.2/modules/org.restlet/src/org/restlet/routing/Redirector.java#L407

with call to this method:

        private void preserveHeaders(Request request)
        {
                Series<Header> headers = (Series<Header>) request.getAttributes().get(HeaderConstants.ATTRIBUTE_HEADERS);
                Series<Header> newHeaders = new Series<Header>(Header.class);

                for (String headerName : headersToPreserve) {
                        Header headerToPreserve = headers.getFirst(headerName);
                        if (headerToPreserve != null) {
                                newHeaders.add(headerToPreserve);
                        }
                }
                request.getAttributes().put(HeaderConstants.ATTRIBUTE_HEADERS, newHeaders);
        }

where

        headersToPreserve = Arrays.asList(HeaderConstants.HEADER_AUTHORIZATION)

After this change, the response from proxy is always:

HTTP/1.1 1002 Internal Connector Error
Date: Fri, 18 Apr 2014 08:01:10 GMT
Accept-Ranges: bytes
Server: Restlet-Framework/2.1.4
Content-Length: 484
Content-Type: text/html; charset=UTF-8

I am somehow abandoning the idea of using Restlet as a transparent reverse proxy, but anyway, I've attached a small example of test implementation if someone will have time and willpower to look at it.

Tim, thank you for your support!

Any advices on how could I achieve Digest/Basic authentication to work on backend server when in front of it is a proxy are still welcome.

------------------------------------------------------
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=3076691

RestletReverseProxyTest.rar (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Re: Re: Transparent reverse proxying using org.restlet.routing.Redirector

Tim Peierls
I don't have time to look at this, unfortunately. 

I have Basic auth working with outbound redirection. I don't change the way the headers are removed, but I do remove the challenge response.

    // From my Redirector subclass:
    @Override
    protected void outboundServerRedirect(
            Reference targetRef, Request request, Response response) {
            
// Unset challenge response that authentication puts in.
request.setChallengeResponse(null);
super.outboundServerRedirect(targetRef, request, response);
    }

Before I did this, I believe I had internal connector errors similar to what you describe.

--tim


On Fri, Apr 18, 2014 at 4:14 AM, Primož Kokol <[hidden email]> wrote:
Hi again,

My next try was to remove all original headers (like default implementation do) and then preserver only Authentication header.

So I replaced this line:

https://github.com/restlet/restlet-framework-java/blob/2.2/modules/org.restlet/src/org/restlet/routing/Redirector.java#L407

with call to this method:

        private void preserveHeaders(Request request)
        {
                Series<Header> headers = (Series<Header>) request.getAttributes().get(HeaderConstants.ATTRIBUTE_HEADERS);
                Series<Header> newHeaders = new Series<Header>(Header.class);

                for (String headerName : headersToPreserve) {
                        Header headerToPreserve = headers.getFirst(headerName);
                        if (headerToPreserve != null) {
                                newHeaders.add(headerToPreserve);
                        }
                }
                request.getAttributes().put(HeaderConstants.ATTRIBUTE_HEADERS, newHeaders);
        }

where

        headersToPreserve = Arrays.asList(HeaderConstants.HEADER_AUTHORIZATION)

After this change, the response from proxy is always:

HTTP/1.1 1002 Internal Connector Error
Date: Fri, 18 Apr 2014 08:01:10 GMT
Accept-Ranges: bytes
Server: Restlet-Framework/2.1.4
Content-Length: 484
Content-Type: text/html; charset=UTF-8

I am somehow abandoning the idea of using Restlet as a transparent reverse proxy, but anyway, I've attached a small example of test implementation if someone will have time and willpower to look at it.

Tim, thank you for your support!

Any advices on how could I achieve Digest/Basic authentication to work on backend server when in front of it is a proxy are still welcome.

------------------------------------------------------
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=3076691

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Transparent reverse proxying using org.restlet.routing.Redirector

Arjohn Kampman-2
I've debugged the problem to an infinite loop when trying to set the Via
header on the request that the proxy sends to the back-end server. This
infinite loop is not triggered when Redirector removes the headers
becuase HttpInboundRequest.getRecipientsInfo() has a "getHeaders() !=
null" check.

I've captured the stack trace below when when the loop has gone through
three cycles between setRecipientsInfo and getRecipientsInfo using
restlet 2.1.4. Is this a bug in HttpInboundRequest or are we doing
something wrong?

Daemon Thread [Restlet-21015178] (Suspended)
***** loop break point *****
     HttpInboundRequest.getRecipientsInfo() line: 579
HttpInboundRequest(Message).setRecipientsInfo(List<RecipientInfo>) line:
426
     HttpInboundRequest.setRecipientsInfo(List<RecipientInfo>) line: 804
     HttpInboundRequest.getRecipientsInfo() line: 579
HttpInboundRequest(Message).setRecipientsInfo(List<RecipientInfo>) line:
426
     HttpInboundRequest.setRecipientsInfo(List<RecipientInfo>) line: 804
     HttpInboundRequest.getRecipientsInfo() line: 579
HttpInboundRequest(Message).setRecipientsInfo(List<RecipientInfo>) line:
426
     HttpInboundRequest.setRecipientsInfo(List<RecipientInfo>) line: 804
     HttpInboundRequest.getRecipientsInfo() line: 579
***** loop start ******
     HeaderUtils.addGeneralHeaders(Message, Series<Header>) line: 264
     ClientAdapter.toSpecific(HttpClientHelper, Request) line: 163
     HttpClientHelper(HttpClientHelper).handle(Request, Response) line: 111
     Client.handle(Request, Response) line: 180
     ClientRoute(Filter).doHandle(Request, Response) line: 159
     ClientRoute(Filter).handle(Request, Response) line: 206
     ClientRouter(Router).doHandle(Restlet, Request, Response) line: 431
     ClientRouter(Router).handle(Request, Response) line: 648
     ComponentClientDispatcher.doHandle(Request, Response) line: 140
     ComponentClientDispatcher(Filter).handle(Request, Response) line: 206
     ChildClientDispatcher.parentHandle(Request, Response) line: 148
     ChildClientDispatcher.doHandle(Request, Response) line: 121
     ChildClientDispatcher(Filter).handle(Request, Response) line: 206
     ChildClientDispatcher.parentHandle(Request, Response) line: 148
     ChildClientDispatcher.doHandle(Request, Response) line: 121
     ChildClientDispatcher(Filter).handle(Request, Response) line: 206
     Decoder(Filter).doHandle(Request, Response) line: 159
     Decoder(Filter).handle(Request, Response) line: 206
     CustomRedirector.serverRedirect(Restlet, Reference, Request,
Response) line: 53
     CustomRedirector(Redirector).outboundServerRedirect(Reference,
Request, Response) line: 352
     CustomRedirector(Redirector).handle(Request, Response) line: 295
     TemplateRoute(Filter).doHandle(Request, Response) line: 159
     TemplateRoute(Filter).handle(Request, Response) line: 206
     Router.doHandle(Restlet, Request, Response) line: 431
     Router.handle(Request, Response) line: 648
     RangeFilter(Filter).doHandle(Request, Response) line: 159
     RangeFilter(Filter).handle(Request, Response) line: 206
     Decoder(Filter).doHandle(Request, Response) line: 159
     Decoder(Filter).handle(Request, Response) line: 206
     StatusFilter(Filter).doHandle(Request, Response) line: 159
     StatusFilter.doHandle(Request, Response) line: 155
     StatusFilter(Filter).handle(Request, Response) line: 206
     TunnelFilter(Filter).doHandle(Request, Response) line: 159
     TunnelFilter(Filter).handle(Request, Response) line: 206
     ApplicationHelper(CompositeHelper<T>).handle(Request, Response)
line: 211
     ApplicationHelper.handle(Request, Response) line: 84
     TransparentReverseProxyApp(Application).handle(Request, Response)
line: 381
     VirtualHost$1(Filter).doHandle(Request, Response) line: 159
     VirtualHost$1(Filter).handle(Request, Response) line: 206
     VirtualHost(Router).doHandle(Restlet, Request, Response) line: 431
     VirtualHost(Router).handle(Request, Response) line: 648
     HostRoute(Filter).doHandle(Request, Response) line: 159
     HostRoute(Filter).handle(Request, Response) line: 206
     ServerRouter(Router).doHandle(Restlet, Request, Response) line: 431
     ServerRouter(Router).handle(Request, Response) line: 648
     StatusFilter(Filter).doHandle(Request, Response) line: 159
     StatusFilter.doHandle(Request, Response) line: 155
     StatusFilter(Filter).handle(Request, Response) line: 206
     LogFilter(Filter).doHandle(Request, Response) line: 159
     LogFilter(Filter).handle(Request, Response) line: 206
     ComponentHelper(CompositeHelper<T>).handle(Request, Response) line:
211
     Component.handle(Request, Response) line: 392
     Server.handle(Request, Response) line: 516
     HttpServerHelper(ServerConnectionHelper).handle(Request, Response)
line: 257
HttpServerHelper(ServerConnectionHelper).doHandleInbound(Response) line:
186
     BaseHelper$2.run() line: 593
     ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) line: 1145
     ThreadPoolExecutor$Worker.run() line: 615
     Thread.run() line: 744

------------------------------------------------------
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=3076700
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Transparent reverse proxying using org.restlet.routing.Redirector

Arjohn Kampman-2
We've updated from restlet 2.1.4 to 2.2.0 now and to our surprise this
fixed the Redirector problems. In fact, Redirector works perfectly
out-of-the-box, including the digest authentication. No subclassing
required. So probably this was a bug in 2.1.4 that has been fixed
somewhere in the 2.2 development.

Regards,
Arjohn

------------------------------------------------------
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=3076835
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Transparent reverse proxying using org.restlet.routing.Redirector

Jerome Louvel-3
Thanks for the feed-back Arjohn! 

We have completely replaced the internal HTTP connector in 2.2 so that certainly helped a lot.
I don't think 2.1.7 would solve your issue but that could be interesting to confirm.

Best,
Jérôme



On Tue, Apr 22, 2014 at 1:45 AM, Arjohn Kampman <[hidden email]> wrote:
We've updated from restlet 2.1.4 to 2.2.0 now and to our surprise this
fixed the Redirector problems. In fact, Redirector works perfectly
out-of-the-box, including the digest authentication. No subclassing
required. So probably this was a bug in 2.1.4 that has been fixed
somewhere in the 2.2 development.

Regards,
Arjohn

------------------------------------------------------
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=3076835

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Transparent reverse proxying using org.restlet.routing.Redirector

Arjohn Kampman-2
Hi Jerome,

Before moving to 2.2.0 I first tried 2.1.7, but that has the same issue.

We do run into a new issue with the Redirector in 2.2.0 now though. The
problem appears related to entity encoding. Responses that are encoded
by the back-end server aren't properly delivered by the proxy to the
client. Typical errors messages on the client are unexpect
end-of-streams in converters and this one:

org.apache.http.ProtocolException: The server failed to respond with a
valid HTTP response
     at
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:151)

We haven't been able to pinpoint the source of this problem yet. Theories:
- Redirector encoding an already encoded entity a second time
- Redirector not copying/sending the proper encoding headers to the client

Any thoughts on what we can try?

Regards,
Arjohn


On 25/04/2014 03:19, Jerome Louvel wrote:

> Thanks for the feed-back Arjohn!
>
> We have completely replaced the internal HTTP connector in 2.2 so that
> certainly helped a lot.
> I don't think 2.1.7 would solve your issue but that could be
> interesting to confirm.
>
> Best,
> Jérôme
>

------------------------------------------------------
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=3077049
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Transparent reverse proxying using org.restlet.routing.Redirector

Arjohn Kampman-2
Hi Jerome, others,

We've figured out what is going wrong by monitoring the traffic with
Wireshark. The problem is indeed, as suspected, related to gzip
encoding. I'll try to explain with one of the traces that we've captured:

1. Client sends a request to the proxy with "Accept-Encoding: gzip".
2. The proxy forwards the request to the back-end server adding
"Accept-Charset: *" and "Accept-Language: *", but otherwise identical.
3. The backend server responds with 127 bytes of gzip encoded data,
expanding to 145 bytes when decompressed.
4. The proxy forwards the response to the client, but sends the 145
bytes of uncompressed data. The proxy did remove the "Content-Encoding:
gzip" header so that part is ok. HOWEVER: the response still indicates
"Content-Length: 127".
5. The client receives the response from the proxy and returns the first
127 bytes to the calling code, resulting in premature end-of-stream errors.

I think there are two options to fix this:
1) Let Redirector replace/remove the Content-Length header from the response
2) Let Redirector forward the compressed body as-is.

We're very much in favor of the second option as that would benefit both
the proxy (less processing overhead) and the client (less network
traffic). Is this something that can be addressed sometime soon? Not
sure where to start looking for supplying a patch myself.

Regards,
Arjohn

------------------------------------------------------
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=3077059
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Transparent reverse proxying using org.restlet.routing.Redirector

Tim Peierls
Could this be addressed by turning off the DecoderService? There's no need to perform the decompression in the first place, is there?


On Fri, Apr 25, 2014 at 8:13 AM, Arjohn Kampman <[hidden email]> wrote:
Hi Jerome, others,

We've figured out what is going wrong by monitoring the traffic with
Wireshark. The problem is indeed, as suspected, related to gzip
encoding. I'll try to explain with one of the traces that we've captured:

1. Client sends a request to the proxy with "Accept-Encoding: gzip".
2. The proxy forwards the request to the back-end server adding
"Accept-Charset: *" and "Accept-Language: *", but otherwise identical.
3. The backend server responds with 127 bytes of gzip encoded data,
expanding to 145 bytes when decompressed.
4. The proxy forwards the response to the client, but sends the 145
bytes of uncompressed data. The proxy did remove the "Content-Encoding:
gzip" header so that part is ok. HOWEVER: the response still indicates
"Content-Length: 127".
5. The client receives the response from the proxy and returns the first
127 bytes to the calling code, resulting in premature end-of-stream errors.

I think there are two options to fix this:
1) Let Redirector replace/remove the Content-Length header from the response
2) Let Redirector forward the compressed body as-is.

We're very much in favor of the second option as that would benefit both
the proxy (less processing overhead) and the client (less network
traffic). Is this something that can be addressed sometime soon? Not
sure where to start looking for supplying a patch myself.

Regards,
Arjohn

------------------------------------------------------
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=3077059

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Transparent reverse proxying using org.restlet.routing.Redirector

Arjohn Kampman-2
For whatsoever reason, disabling the decoder service results in a NPE
being thrown at
org.restlet.routing.Redirector.outboundServerRedirect(Redirector.java:349).
That's this line in 2.2.0:

             next = getContext().getClientDispatcher();

AFAICT, this can only happen if the context is <null>. Full stack trace
attached

------------------------------------------------------
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=3077064

Exception.txt (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Transparent reverse proxying using org.restlet.routing.Redirector

Arjohn Kampman-2
Any thoughts on the encoding related bug in Redirector?

------------------------------------------------------
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=3077354
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Transparent reverse proxying using org.restlet.routing.Redirector

Thierry Boileau-4
Hello all,

I'm having a look at it.

Best regards,
Thierry Boileau


2014-05-01 12:27 GMT+02:00 Arjohn Kampman <[hidden email]>:
Any thoughts on the encoding related bug in Redirector?

------------------------------------------------------
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=3077354

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Transparent reverse proxying using org.restlet.routing.Redirector

rameshsr
In reply to this post by Arjohn Kampman-2
Arjohn Kampman-2 wrote
We've updated from restlet 2.1.4 to 2.2.0 now and to our surprise this
fixed the Redirector problems. In fact, Redirector works perfectly
out-of-the-box, including the digest authentication. No subclassing
required. So probably this was a bug in 2.1.4 that has been fixed
somewhere in the 2.2 development.
Thanks for this info, which gives me some confidence that I can get this working too with some help. In order to set the authentication information for a MODE_SERVER_OUTBOUND redirection, I added a filter (code shown below), in front of the Redirector, to set the ChallengeResponse as shown below. But I could never get this authentication work successfully, since the server always fails authentication for the passed username/password. I would appreciate if you could share the details on how you passed the authentication details to the Redirector.

public class MyRedirectorAuthenticatorFilter extends Filter {

       public MyRedirectorAuthenticatorFilter(Context context) {
                this.setContext(context);

        }

        @Override
        protected int doHandle(Request request, Response response) {

                String username = "username";
                String password = "plaintext password";

                request.setProxyChallengeResponse(new ChallengeResponse(
                                ChallengeScheme.HTTP_DIGEST, username, password));

                return super.doHandle(request, response);

        }
}

And in my application, I have,

                String target2 = "http://localhost:8080/MyWebApp{rr}";
                Redirector redirector2 = new Redirector(getContext(), target2,
                                Redirector.MODE_SERVER_OUTBOUND);
                MyRedirectorAuthenticatorFilter myfilter = new MyRedirectorAuthenticatorFilter(getContext());
                myfilter.setNext(redirector2);
                router.attach("/myapp", myfilter);

Appreciate your help on this.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: Re: Transparent reverse proxying using org.restlet.routing.Redirector

Primož Kokol
Hi Ramesh,

Please take a look at the example I've attached.

After running it (mvn clean install exec:java) server will be started on port 8080 and reverse proxy on port 9999.

You will be able to access the server on 8080 but you will have to provide credentials (test/test) when accessing trough reverse proxy.

Hope this is what you're looking for.

------------------------------------------------------
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=3080062

RestletReverseProxyAuth.rar (4K) Download Attachment
Loading...