11.7 vs. 12.8 OpenEdge.Net.HTTP client via proxy

JamesBowen · Feb 21, 2025

I have existing code that has been working as expected on OE11.7. The code makes a simple POST request via a proxy server.
Since upgrading to 12.8 the POST request is different and responds with a 404 http error code.

OE 11.7 via a proxy, returns HTTP 200

Code:

POST /gateway/oauth/token HTTP/1.1
User-Agent: OpenEdge-HttpClient/0.4.0 (WIN32/64) OpenEdge/11.7.20.0.2288 Lib-ABLSockets/0.5.0
Host: www.randomhost.com
Content-Type: application/x-www-form-urlencoded
Content-Length: 185
Authorization: Basic xxxxxxxxxxx
Accept: */*

OE 12.8 via a proxy, returns HTTP 404

Code:

POST https://www.randomhost.com/gateway/oauth/token HTTP/1.1
User-Agent: OpenEdge-HttpClient/0.7.0 (WIN32/64) OpenEdge/12.8.3.0.1217 Lib-ABLSockets/0.7.0
Host: www.randomhost.com
Content-Type: application/x-www-form-urlencoded
Content-Length: 195
Authorization: Basic xxxxxxxxxxx
Accept: application/json

OE 12.8 direct (no proxy) returns HTTP 200

Code:

POST gateway/oauth/token HTTP/1.1
User-Agent: OpenEdge-HttpClient/0.7.0 (WIN32/64) OpenEdge/12.8.3.0.1217 Lib-ABLSockets/0.7.0
Host: www.randomhost.com
Content-Type: application/x-www-form-urlencoded
Content-Length: 195
Authorization: Basic xxxxxxxxxxx
Accept: application/json

The difference is that I can see is the resource location is the full URL path. https://www.randomhost.com/gateway/oauth/token vs. /gateway/oauth/token

I know when the HTTP client connects via a proxy, it should include the HOST & PORT in combination with a "CONNECT" and I wonder if this is a bug in the OE code.

Stefan · Feb 21, 2025

Have fun browsing through the differences: Comparing v11.7.21.0...v12.8.6.0 · progress/ADE

Are you using the same TLS version in both cases?

JamesBowen · Feb 21, 2025

Stefan said:
Have fun browsing through the differences: Comparing v11.7.21.0...v12.8.6.0 · progress/ADE

Are you using the same TLS version in both cases?

Thanks, I'm now eyeballs deep looking at GitHub Repos.

I'm using the same TLS 1.2 and the same ciphers.

JamesBowen · Feb 22, 2025

I've found this. HTTP/1.1: Request
And by my interpretation, It seems that OE 12.8 is working according the RFC.

If you make a request via a proxy server the URI can be an absolute URI. Servers MUST accept the absolute URI form in requests, even though HTTP/1.1 clients will only generate them in requests to proxies.

So, it not a bug, but rather than the webserver I'm connecting to (via a proxy) is not accepting the absolute URI in the POST request.

peterjudge · Feb 23, 2025

JamesBowen said:
I know when the HTTP client connects via a proxy, it should include the HOST & PORT in combination with a "CONNECT" and I wonder if this is a bug in the OE code.

The CONNECT request is (only) sent when proxying over https, so you should be seeing it in your case.

You are not going to see that request written into the request and response txt files (or rather, they will be overwritten), but if you set the LOG-MANAGER LOGGING-LEVEL to 5 then you will see a message in the log to the effect that a tunnel is being created via CONNECT.

JamesBowen said:
So, it not a bug, but rather than the webserver I'm connecting to (via a proxy) is not accepting the absolute URI in the POST request.

Sadly this kind of thing happens more than you'd expect ... servers not implementing the HTTP protocol correctly or completely.

JamesBowen · Feb 23, 2025

Thanks Peter. Would you a agree that it is a change in behavior between 11.7 and 12.8 when a client connects via a proxy server.

peterjudge · Feb 23, 2025

I would not have expected a behaviour change in this regard. There are some internal changes to the data structures used (and obviously other changes for various things) but the code that writes the start line URL has not changed between 11.7 and 12.8 as far as I can tell.

Code:

// In OpenEdge.Net.HTTP.Filter.Payload.DefaultRequestFilter
        if valid-object(oProxyRequest) and
           valid-object(oProxyRequest:ProxyURI) then
            /* for proxies, we want http://example.com/some/data?query=true */
            assign cRequestURI = poRequest:URI:Encode().
        else
            /* for normal request, we want /some/data?query=true */
            assign cRequestURI = poRequest:URI:Path + poRequest:URI:EncodeQuery().

Can you share the site that's impacted here?

JamesBowen · Feb 26, 2025

This is very odd. I've gone through the different revisions of OpenEdge.Net.HTTP.Filter.Payload.DefaultRequestFilter on GitHub and you are correct. There has been no changes in the logic of when to output a absolute URI vs. a relative URI when connecting via a proxy server.

So, I'm not sure how in my logging of the output Headers has managed to show two difference behaviors 11.7 vs. 12.8.

I've spoken to API provider and they have said that they have other consumers of the API connecting by proxy servers without an issue and they won't be making any changes to there service offering.

Unfortunately, the solution was to shell out to the command line an use cURL to overcome this issue.

peterjudge · Feb 26, 2025

It almost looks like there _was_ a bug in the 11.7 code which did not write the correct POST line. But it's hard to tell without some more debugging (I know that's just a variation on 'it depends'). Which exact 11.7 are you seeing the "good" behaviour in?

JamesBowen · Feb 27, 2025

peterjudge said:
It almost looks like there _was_ a bug in the 11.7 code which did not write the correct POST line. But it's hard to tell without some more debugging (I know that's just a variation on 'it depends'). Which exact 11.7 are you seeing the "good" behaviour in?

11.7.20.0.2288 was the version that was working.

It almost need to have an override option to not use absolute URI when connecting via a proxy Server.

peterjudge · Feb 27, 2025

JamesBowen said:
11.7.20.0.2288 was the version that was working.

It almost need to have an override option to not use absolute URI when connecting via a proxy Server.

A way to provide an override is as follows:
1. Create a class that extendes OpenEdge.Net.HTTP.Filter.Payload.DefaultRequestFilter (inherits from).
2. Override the WriteStartLine method. Copy & paste the original method in, and remove the lines that check for the existence of a proxy. DO NOT CALL SUPER in this method.
3. Before you make any calls, register this class for use by the HTTP client. In ABL,

Code:

OpenEdge.Net.HTTP.Filter.Writer.RequestWriterBuilder:Registry:Put('HTTP/1.1':u,
                                                                  get-class(Your.Type.Name)).

Now, all requests should use the "non proxy" URL. If you only want to use this for these particular requests, then you can reset to the default by

Code:

OpenEdge.Net.HTTP.Filter.Writer.RequestWriterBuilder:Registry:Put('HTTP/1.1':u,
                                                                  get-class(OpenEdge.Net.HTTP.Filter.Payload.DefaultRequestFilter)).

JamesBowen · Mar 1, 2025

This is my code and I think I've followed your instructions, but unfortunately I getting this error (See Bellow).

C#:

 /*------------------------------------------------------------------------
    File        : DefaultRequestFilterOverride
    Purpose     :
    Syntax      :
    Description :
    Author(s)   : james
    Created     : Sat Mar 01 21:10:06 NZDT 2025
    Notes       :
  ----------------------------------------------------------------------*/

USING Progress.Lang.*.
using OpenEdge.Core.ByteBucket.
using OpenEdge.Core.IAdaptable.
using OpenEdge.Core.StringConstant.
using OpenEdge.Net.HTTP.IHttpRequest.
using OpenEdge.Net.HTTP.ISupportProxy.
USING OpenEdge.Net.HTTP.Filter.Payload.DefaultRequestFilter.

BLOCK-LEVEL ON ERROR UNDO, THROW.

CLASS Classes.OAuth.DefaultRequestFilterOverride INHERITS DefaultRequestFilter:
    
    /*------------------------------------------------------------------------------
     Purpose:
     Notes:
    ------------------------------------------------------------------------------*/

    METHOD OVERRIDE PROTECTED VOID WriteStartLine( INPUT poRequest AS OpenEdge.Net.HTTP.IHttpRequest ):
        
        define variable cRequestURI as character no-undo.
        define variable oProxyRequest as ISupportProxy no-undo.
        
        
        if type-of(poRequest, IAdaptable) then
            assign oProxyRequest = cast(cast(poRequest, IAdaptable):GetAdapter(get-class(ISupportProxy)), ISupportProxy).
        else
        if type-of(poRequest, ISupportProxy) then
            assign oProxyRequest = cast(poRequest, ISupportProxy).
        
/*        if valid-object(oProxyRequest) and                                    */
/*           valid-object(oProxyRequest:ProxyURI) then                          */
/*            /* for proxies, we want http://example.com/some/data?query=true */*/
/*            assign cRequestURI = poRequest:URI:Encode().                      */
/*        else                                                                  */
            /* for normal request, we want /some/data?query=true */
            assign cRequestURI = poRequest:URI:Path + poRequest:URI:EncodeQuery().
            
        message cRequestURI
            view-as alert-box info.   
        
        cast(this-object:Entity, ByteBucket):PutString(substitute('&1 &2 &3':u,
                                caps(poRequest:Method),
                                cRequestURI,
                                poRequest:Version ) + StringConstant:CRLF).

    END METHOD.

END CLASS.

peterjudge · Mar 1, 2025

JamesBowen said:

This is my code and I think I've followed your instructions, but unfortunately I getting this error (See Bellow).

C#:

 /*------------------------------------------------------------------------
    File        : DefaultRequestFilterOverride
    Purpose     :
    Syntax      :
    Description :
    Author(s)   : james
    Created     : Sat Mar 01 21:10:06 NZDT 2025
    Notes       :
  ----------------------------------------------------------------------*/

USING Progress.Lang.*.
using OpenEdge.Core.ByteBucket.
using OpenEdge.Core.IAdaptable.
using OpenEdge.Core.StringConstant.
using OpenEdge.Net.HTTP.IHttpRequest.
using OpenEdge.Net.HTTP.ISupportProxy.
USING OpenEdge.Net.HTTP.Filter.Payload.DefaultRequestFilter.

BLOCK-LEVEL ON ERROR UNDO, THROW.

CLASS Classes.OAuth.DefaultRequestFilterOverride INHERITS DefaultRequestFilter:
   
    /*------------------------------------------------------------------------------
     Purpose:
     Notes:
    ------------------------------------------------------------------------------*/

    METHOD OVERRIDE PROTECTED VOID WriteStartLine( INPUT poRequest AS OpenEdge.Net.HTTP.IHttpRequest ):
       
        define variable cRequestURI as character no-undo.
        define variable oProxyRequest as ISupportProxy no-undo.
       
       
        if type-of(poRequest, IAdaptable) then
            assign oProxyRequest = cast(cast(poRequest, IAdaptable):GetAdapter(get-class(ISupportProxy)), ISupportProxy).
        else
        if type-of(poRequest, ISupportProxy) then
            assign oProxyRequest = cast(poRequest, ISupportProxy).
       
/*        if valid-object(oProxyRequest) and                                    */
/*           valid-object(oProxyRequest:ProxyURI) then                          */
/*            /* for proxies, we want http://example.com/some/data?query=true */*/
/*            assign cRequestURI = poRequest:URI:Encode().                      */
/*        else                                                                  */
            /* for normal request, we want /some/data?query=true */
            assign cRequestURI = poRequest:URI:Path + poRequest:URI:EncodeQuery().
           
        message cRequestURI
            view-as alert-box info.  
       
        cast(this-object:Entity, ByteBucket):PutString(substitute('&1 &2 &3':u,
                                caps(poRequest:Method),
                                cRequestURI,
                                poRequest:Version ) + StringConstant:CRLF).

    END METHOD.

END CLASS.

View attachment 3230

That was one of the structural changes I mentioned ... in 12.5 (I think) the type went from ByteBucket to RawMessage to support HAR tracing.

The principle's the same, just make sure you use the correct version. You may nee to typecheck before casting if you want to use this class in 11.7 and 12.8..

JamesBowen · Mar 1, 2025

peterjudge said:
That was one of the structural changes I mentioned ... in 12.5 (I think) the type went from ByteBucket to RawMessage to support HAR tracing.

The principle's the same, just make sure you use the correct version. You may nee to typecheck before casting if you want to use this class in 11.7 and 12.8..

My mistake... I was using the 11.7 code. The "WriteStartLine" override method worked!

BTW.. I also try using HAR tracing and crashed when output the when connecting to a proxy server. Separate issue to this.

11.7 vs. 12.8 OpenEdge.Net.HTTP client via proxy

JamesBowen

19+ years progress programming and still learning.

Stefan

Well-Known Member

JamesBowen

19+ years progress programming and still learning.

JamesBowen

19+ years progress programming and still learning.

peterjudge

Active Member

JamesBowen

19+ years progress programming and still learning.

peterjudge

Active Member

JamesBowen

19+ years progress programming and still learning.

peterjudge

Active Member

JamesBowen

19+ years progress programming and still learning.

peterjudge

Active Member

JamesBowen

19+ years progress programming and still learning.

peterjudge

Active Member

JamesBowen

19+ years progress programming and still learning.