Resolved Security Audit data capture of OE HTTP Client

JamesBowen

JamesBowen

19+ years progress programming and still learning.
After a security review, I'm needing to capture all the web requests that is being called by OE HTTP Client. Currently is just dumping to a file but will be eventually stored into the database.
I'm able to capture the HTTP header information, but I can find the correct method a capturing the POST form body of the request.
I'm assuming RequestObj:Entity contains the form body content? ¯\_(ツ)_/¯

How do I capture the form body when the content type is application/x-www-form-urlencoded ?

Code Fragment:
C#: 
numHeaders = RequestObj:GetHeaders(OUTPUT HTTPHeaderTX ).
            
            PUT STREAM sHTTPDump UNFORMATTED SUBSTITUTE("&1 &2 &3", RequestObj:method, RequestObj:URI:RelativeURI , RequestObj:Version) SKIP.
                
            DO iHeaderIndex = 1 TO numHeaders:
                PUT STREAM sHTTPDump UNFORMATTED HTTPHeaderTX[iHeaderIndex]:ToString() SKIP.
                DELETE OBJECT HTTPHeaderTX[iHeaderIndex].
            END.
            
            PUT STREAM sHTTPDump UNFORMATTED SKIP(1).
            
            OUTPUT stream sHTTPDump Close. // Important! Close the stream
            
            DEFINE VARIABLE HTMLFormPOST AS LONGCHAR NO-UNDO.
            
            HTMLFormPOST = CAST(RequestObj:Entity, OpenEdge.Core.String):value. // <----- Failing here


This is what I'm able to capture:
HTTP: 
POST /gateway/oauth/token HTTP/1.1
Accept: */*
Authorization: Basic xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Content-Length: 215
Content-Type: application/x-www-form-urlencoded
Host: api.somehostserver.com
User-Agent: OpenEdge-HttpClient/0.7.0 (WIN32/64) OpenEdge/12.8.6.0.1236 Lib-ABLSockets/0.7.0

Error:
Invalid cast from OpenEdge.Core.Collections.StringStringMap to OpenEdge.Core.String. (12869) (12869)
Click to expand...
 
Update:

I think I need loop through the StringStringMap object to extract each key entry pair. Not sure how to do this.
 
Last edited:
If you're on 12.5+, take a look at Progress Documentation .

Alternatively you can do what you're doing and also convert the body to bytes using the BodyWriterBuilder which will give you a bytebucket instance that you can dump to disk.
 
I'm trying use the BodyWriterBuilder class object. But, the oByteBucket:Size is always zero. The RequestObj:ContentLength returns 512.

What I'm I missing?

Code: 
MESSAGE "VALID-OBJECT(RequestObj)" VALID-OBJECT(RequestObj).
            MESSAGE "RequestObj:ContentType" RequestObj:ContentType.
            MESSAGE "RequestObj:ContentLength" RequestObj:ContentLength.  // <-- 512 bytes
            
            objMessageWriterBuilder = OpenEdge.Net.HTTP.Filter.Writer.BodyWriterBuilder:Build( RequestObj  ).
            
            oByteBucket = NEW OpenEdge.Core.ByteBucket( RequestObj:ContentLength + 2 ).
            //oByteBucket = NEW OpenEdge.Core.ByteBucket(  ).
            
            objMessageWriterBuilder:WriteTo(oByteBucket):Writer.
          
            MESSAGE "oByteBucket:Size" oByteBucket:Size.
            oResponseMemptrEntity = oByteBucket:GetBytes().
            MESSAGE "oResponseMemptrEntity:size" oResponseMemptrEntity:size.   
                
            IF oResponseMemptrEntity:size GT 0 THEN
            DO:
                MESSAGE "COPY-LOB FROM oResponseMemptrEntity:Value TO FILE outputFileName APPEND". //Debugging
                COPY-LOB FROM oResponseMemptrEntity:Value TO FILE outputFileName APPEND.
            END.
 
The bit you're missing is doing the writing of the Entity (payload) to the ByteBucket.

Passing the message to the Build() method is effectively only doing the set up for the kind of writer to use for that message (ie is it JSON or XML or something else).

Code: 
objMessageWriterBuilder:Open().
/* writes the message's entity contents into a byte bucket */
objMessageWriterBuilder:Write(RequestObj:Entity).
objMessageWriterBuilder:Close().
 
peterjudge said:
The bit you're missing is doing the writing of the Entity (payload) to the ByteBucket.

Passing the message to the Build() method is effectively only doing the set up for the kind of writer to use for that message (ie is it JSON or XML or something else).

Code: 
objMessageWriterBuilder:Open().
/* writes the message's entity contents into a byte bucket */
objMessageWriterBuilder:Write(RequestObj:Entity).
objMessageWriterBuilder:Close().
Click to expand...

Hi Petter, thank-you for your support...But, I might be misunderstanding something.
I can't see a Open(), Write() or Close() method for class object OpenEdge.Net.HTTP.Filter.Writer.MessageWriterBuilder.


1750739814606.png
 
Sorted it out with the help from DeepSeek. Not sure if this is the most efficient way to do it, but it's working.
MVP to meet the security requirements.

Code: 
 DEFINE VARIABLE objStringStringMap AS CLASS    OpenEdge.Core.Collections.StringStringMap NO-UNDO.
            DEFINE VARIABLE objIter            AS CLASS    OpenEdge.Core.Collections.IIterator       NO-UNDO.
            DEFINE VARIABLE cKey               AS CLASS    OpenEdge.Core.String                      NO-UNDO.
            DEFINE VARIABLE cValue             AS CLASS    OpenEdge.Core.String                      NO-UNDO.
            DEFINE VARIABLE lcFormEncode       AS longchar NO-UNDO.
            DEFINE VARIABLE objFormEncoder     AS CLASS    OpenEdge.Net.FormEncoder                  NO-UNDO.
           
            IF TYPE-OF(RequestObj:Entity, OpenEdge.Core.Collections.StringStringMap) THEN
            DO:
               
                //message "TYPE-OF(RequestObj:Entity, OpenEdge.Core.Collections.StringStringMap) TRUE".
               
                objFormEncoder = NEW OpenEdge.Net.FormEncoder().
                         
                objStringStringMap = CAST(RequestObj:Entity, OpenEdge.Core.Collections.StringStringMap ).
                objIter = objStringStringMap:KeySet:Iterator().
           
                DO WHILE objIter:HasNext():
               
                    cKey = CAST( objIter:Next(), OpenEdge.Core.String).
                    cValue =  objStringStringMap:Get( cKey ) .
                    //MESSAGE "Key: " string(cKey:value) " Value: " string(cValue:Value).
                   
                    lcFormEncode = lcFormEncode + SUBSTITUTE("&1=&2&&", cKey:Value, objFormEncoder:Encode( cValue:Value ) ).
                END.
               
                //message "length(outputFileName)" length(outputFileName).
               
                lcFormEncode = right-trim(lcFormEncode, '&').
                COPY-LOB FROM lcFormEncode TO FILE outputFileName APPEND.
            END.
 
JamesBowen said:
Sorted it out with the help from DeepSeek. Not sure if this is the most efficient way to do it, but it's working.
MVP to meet the security requirements.

Code: 
 DEFINE VARIABLE objStringStringMap AS CLASS    OpenEdge.Core.Collections.StringStringMap NO-UNDO.
            DEFINE VARIABLE objIter            AS CLASS    OpenEdge.Core.Collections.IIterator       NO-UNDO.
            DEFINE VARIABLE cKey               AS CLASS    OpenEdge.Core.String                      NO-UNDO.
            DEFINE VARIABLE cValue             AS CLASS    OpenEdge.Core.String                      NO-UNDO.
            DEFINE VARIABLE lcFormEncode       AS longchar NO-UNDO.
            DEFINE VARIABLE objFormEncoder     AS CLASS    OpenEdge.Net.FormEncoder                  NO-UNDO.
       
            IF TYPE-OF(RequestObj:Entity, OpenEdge.Core.Collections.StringStringMap) THEN
            DO:
           
                //message "TYPE-OF(RequestObj:Entity, OpenEdge.Core.Collections.StringStringMap) TRUE".
           
                objFormEncoder = NEW OpenEdge.Net.FormEncoder().
                     
                objStringStringMap = CAST(RequestObj:Entity, OpenEdge.Core.Collections.StringStringMap ).
                objIter = objStringStringMap:KeySet:Iterator().
       
                DO WHILE objIter:HasNext():
           
                    cKey = CAST( objIter:Next(), OpenEdge.Core.String).
                    cValue =  objStringStringMap:Get( cKey ) .
                    //MESSAGE "Key: " string(cKey:value) " Value: " string(cValue:Value).
               
                    lcFormEncode = lcFormEncode + SUBSTITUTE("&1=&2&&", cKey:Value, objFormEncoder:Encode( cValue:Value ) ).
                END.
           
                //message "length(outputFileName)" length(outputFileName).
           
                lcFormEncode = right-trim(lcFormEncode, '&').
                COPY-LOB FROM lcFormEncode TO FILE outputFileName APPEND.
            END.
Click to expand...
 
Last edited by a moderator:
Sorry, I thought you'd assigned the writer to a variable ...

Code: 
define variable oWriter as OpenEdge.Net.HTTP.Filter.Payload.MessageWriter no-undo.

oWriter = objMessageWriterBuilder:Writer.

oWriter:Open().
oWriter:Write(RequestObj:Entity).
oWriter:Close().
 
Last edited:
Thanks Peter, your code recommendation is working.

I've managed to condense the code into this:

Code: 
DEFINE VARIABLE oWriter AS OpenEdge.Net.HTTP.Filter.Payload.MessageWriter NO-UNDO.

oWriter = OpenEdge.Net.HTTP.Filter.Writer.BodyWriterBuilder:Build( RequestObj ):Writer.

oWriter:Open().
oWriter:Write(RequestObj:Entity).
oWriter:Close().

COPY-LOB FROM OBJECT CAST(oWriter:Entity, OpenEdge.Core.ByteBucket):Value TO FILE outputFileName APPEND.
 
Currently is just dumping to a file but will be eventually stored into the database.
Click to expand...

You don't mention the OE version, but the from 12.5 onwards, the ABL HTTP tracing feature I linked to can do all of this for you. You'd have to write a class to write the (JSON) trace data to a DB, but all the other bits are part of the OE product.
 
peterjudge said:
You don't mention the OE version, but the from 12.5 onwards, the ABL HTTP tracing feature I linked to can do all of this for you. You'd have to write a class to write the (JSON) trace data to a DB, but all the other bits are part of the OE product.
Click to expand...
OE 12.8.7. But, I might need to be backward compatible with 11.7.x
 
You must log in or register to reply here.
Back
Top