Utilizing Perl scripts in a web service app - Performance Considerations
Looking for some insight into the performance of using XML-RPC to run perl scripts via web service.
I am considering buying some perl scripts that have been developed to process large amounts of data as part of a backend ETL to a data warehouse. These scripts currently run as batch jobs via ksh/cron using standard filein/out. I would like to evaluate retooling them into a set of web services (e.g. SOAP, XML-RPC) to be used in an online/realtime transaction mode.
I have done some research and prototyping to confirm that technically, it can be done, but I am concerned about performance/scalability. Seems to me that parsing the XML is going to be the biggest bottleneck. I am sure I can throw hardware at it, but I am wondering if it will be cost effective. We are talking about processing 10Gb/client per day.
Does anyone here have any experience with perl via XML-RPC? Any insight would be appreciated.
Rewriting these perl scripts from the ground up is not an option (too expensive).