Background:
We’re shifting from an environment which used to use ldirectord for our load balancing to haProxy, predominantly because it can perform SSL termination, layer 7 filtering etc. It’s far more flexible in terms of how it can cater for multiple SSL sites behind it…
Problem:
This is very specific: when a public user visits a CF based webservice, hosted over SSL, with SSL being terminated on the haProxy load balancer, and with traffic between the LB and the web server over http (off-loaded SSL termination), the CF environment understandably believes that the URL which it ought to publish in the auto-generated WSDL is one over http. – If the haProxy LB is set to force traffic to use SSL, then the client fails to be able to invoke WebService methods as they are called against a HTTP, not HTTPS based url. The Method calls seem particularly sensitive to the LB forcing them to use SSL, perhaps some of the data which was being passed on the initial request does not get resent on a redirected request? – For whatever reason the WS client was then failing to obey the redirection request, and the mis-match in protocols was causing client’s use of the WS to fail.
As a temporary work-around we removed the force-ssl option from the LB, and permitted the method calls to use HTTP through it instead, but this was not a solution to the problem. Despite there being no sensitive data being passed through the WS, we felt that we really needed a solution to this and one which could be applied easily across a number of applications, or even server-wide on a back-end server basis.
Hope?:
We had found a TomCat ‘Valve’ – called ‘RemoteIpValve’ – https://tomcat.apache.org/tomcat-7.0-doc/api/org/apache/catalina/valves/RemoteIpValve.html – which gave us a glimmer of hope, thinking that we could affect the CF environment at an early enough stage (at the Connector), to let it know that it was over SSL, and therefore get it to change the CGI variable scope to indicate it was over https, and hoped that this too might affect the way in which the WSDL was created, changing the URL listed within it.
Nothing we tried worked. This Valve is intended to be used for something like this purpose, but it’s really unclear what the expected further extent of its influence ought to be, and there is almost no documentation on how its influence is expected to affect coldfusion’s server created variables. – It is documented that this Valve can be used to change the view of the ‘Remote IP address used within logs’, in combination with another Valve.
Anyway, that’s when we took to Facebook to ask for some advice. – Thank you very much to those who responded to use yesterday. You saved us chasing our tails for much longer, most of the comments posted encouraged us to manipulate the WSDL content by hand, and then serve the WS description using a separate end-point to the actual service itself (a static file). Other suggestions included using SSL between the LB and the web server, but that would negate the purpose of shifting the SSL termination point to the LB, so we were not keen to pursue that option.
Our solution:
We wanted an approach which would fit – in place, into existing applications without having to inform existing end-users to update their end-points. In short, we needed the ?wsdl request to return an appropriately modified service description, and to know when it needed to apply special considerations and change it from what it was seeing itself within CF.
Thanks to a useful post by Ben Nadel here: https://www.bennadel.com/blog/1878-ask-ben-blocking-wsdl-access-in-a-coldfusion-application.htm – Thank you Ben! – This provided us hope that we could intercept and change the behaviour of the ?wsdl request by modifying our application.cfc – which is what we did.
We included a heap of considerations about whether we ought to build a custom, static WSDL file or not, looking for the haProxy added headers which indicated that there would be a mismatch between the local CF server’s port and that requested by the end-user client. And in those scenarios looked for a local, static https based wsdl file from the local folder (with .XML extension to simplify the serving and filetype headers). Simply put, if it hadn’t been created for that service yet, make it and store it, if it is already there then use it.
Road-block:
It’s never quite that simple right? – The onRequestStart function can only return a boolean. If you start messing with the return type, 500 Server errors start happening. Noting that Ben’s example was sucessfully modifying the HTTP response headers, and knowing that is essentially exactly what a CFLOCATION also performs, we thought it was worth a try to use a CFLOCATION based redirect to the static XML file which we’d just written to the server. – The WS clients seemed to accept this redirect without any problem, and with the wsdlsoap:address location value changed to https:…. the subsequent ws calls made by the client were appropriately sent via the haProxy LB using HTTPS.
The only additional snag we found was that in order to inspect the http headers, using the getHTTPRequestData() function, if you attempt to use this function at all during a web service method request, it would bomb with a 500 Internal Server error (and the error content in the logs were useless and entirely irrelevant). We had to shift this function call inside of a CFIF which first checked that it was a call for the WSDL content. These ?wsdl calls never had this issue, it was only during the subsequent method based calls.
The following code was added into our application.cfc as a general approach on an application wide basis to handle this issue, this is a quick first draft and proof of concept which will be tidied up in due course, but you’ll get the idea from this hopefully:
<cffunction name="onRequestStart" access="public" returntype="boolean" output="false" hint="I initialize the request.">
<!--- Check to see if the WSDL flag is present in the URL and that it's locally over port 80..
If so, grab the headers and proceed with further checks... --->
<cfif structKeyExists( url, "wsdl" ) and cgi.SERVER_PORT eq 80>
<!--- Using getHTTPRequestData() for a ws method call was causing it to bomb, so we've had to shift that
inside of the above CFIF, then nest the further header checks, based on it being a normal HTTP based call
for the ?WSDL content. --->
<cfset reqHeaders = getHTTPRequestData()>
<cfif structKeyExists( reqHeaders, "headers" )
and structKeyExists( reqHeaders.headers, "x-forwarded-for" )
and structKeyExists( reqHeaders.headers, "x-forwarded-proto" )
and reqHeaders.headers['x-forwarded-proto'] is 'https' > --->
<!--- In this instance we will have a mis-match of soap webservice address, the WSDL would naturally
provide an HTTP and not an HTTPS based address --->
<!--- Flow:
1. Check to see if we have a locally saved static file, if not:
Fetch the WSDL, find the http portion: 'wsdlsoap:address location="http://' and replace it with
the same address, but https: 'wsdlsoap:address location="http://'
Save the file locally.
2. Use CFLOCATION redirection to send WSDL request to static .XML file version --->
<cftry>
<cffile action="read" file="#getTemplatePath()#.wsdl.https.xml" variable="staticWSDL">
<cfcatch>
<!--- Assume failure is due to missing file.. ? --->
<!--- Fetch it locally on the server, this will require a local host record! --->
<cfhttp url="http://#cgi.server_name##script_name#?wsdl" method="get" timeout="1">
<cfset outputcontent = replaceNoCase(cfhttp.filecontent
, 'wsdlsoap:address location="http:'
, 'wsdlsoap:address location="https:' )>
<cffile action="write" file="#getTemplatePath()#.wsdl.https.xml" nameconflict="overwrite" output="#outputcontent#">
</cfcatch>
</cftry>
<cflocation url="https://#cgi.server_name##script_name#.wsdl.https.xml" addtoken="false">
</cfif>
<cfelse>
<cfreturn true />
</cfif>
<!--- Return true to let page request process. --->
<cfreturn true />
</cffunction>