Building a web service with Perl - web-services

I need to build a server-side application (tiny web service) for testing proposes. What are some CPAN modules and Perl libraries for implementing such task?

Testing a tiny Web service with Plack::Test:
use Plack::Test;
use Test::More;
test_psgi(
app => sub {
my ($env) = #_;
return [200, ['Content-Type' => 'text/plain'], ["Hello World"]],
},
client => sub {
my ($cb) = #_;
my $req = HTTP::Request->new(GET => "http://localhost/hello");
my $res = $cb->($req);
like $res->content, qr/Hello World/;
},
);
done_testing;

There are a lot of possibilities
CGI - if you like to do everything like in the olden days
CGI::Application - a little more advanced
or you could use frameworks like
Catalyst
Dancer
Mojolicious
It depends on your skills and aims what solution you should choose.

A web service simply returns a HTTP status code and some data, perhaps serialized in JSON or XML. You can use the CGI module to do this, e.g.:
#!/usr/bin/perl -w
use strict;
use warnings;
use CGI;
use CGI::Pretty qw/:standard/;
use URI::Escape;
my $query = CGI->new;
my $jsonQueryValue = uri_unescape $query->param('helloWorld');
# let's say that 'helloWorld' is a uri_escape()-ed POST variable
# that contains the JSON object { 'hello' : 'world' }
print header(-type => "application/json", -status => "200 OK");
print "$jsonQueryValue";
You can, of course, print an HTTP response with other status codes and data. A web service might need to return a 404 error, for example, depending on what's being asked for. That sort of thing.

I like to use mojolicious. It's lightweight at first and can do the heavy lifting later too. Mojolicious::Lite in particular is good for quick and dirty.
use Mojolicious::Lite;
# Route with placeholder
get '/:foo' => sub {
my $self = shift;
my $foo = $self->param('foo');
$self->render(text => "Hello from $foo.");
};
# Start the Mojolicious command system
app->start;

Related

Stubbing/mocking a service in a Dancer2 application

Usually, I have a setup similar to this:
#!/usr/bin/env perl
package Demo;
use Dancer2;
use Moose;
sub get($self, $params) {
my $whatever = ...; # connect with db and do stuff
return $whatever;
}
my $service = Demo->new();
sub make_web_friendly($param) { # no $self, here!
return "Hello $param!";
}
get '/' => sub {
my $response = $service->get(query_parameters);
return make_web_friendly($response);
};
start;
This way, I can separate concerns pretty well and my services are testable without going over a bogus web request via Plack::Test;
However, whenever I test the request, I always also test the service, with goes against this separation.
I would want to provide the Plack Test with a stubbed service that would mock a service answer. So (I assume) either I can make the get action somehow $self-aware, or would have to pass the mocked service to Plack.
# this is put together to showcase the principle, it does not work
package Stub {
use Moose;
use Test::More;
use Data::Dumper;
sub get($self, $params) {
diag "I received:" . Dumper($params);
return "Bianca!";
}
}
my $app = Demo->to_app;
my $test = Plack::Test->create($app);
my $stub = Stub->new();
$test->service($stub);
my $request = HTTP::Request->new( GET => '/' );
my $response = $test->request($request);
is $response, 'Hello, Bianca', 'Bianca was here';
How can I achieve (something like) this?

Perl: can not post xml data to web service

The web service accepts the xml data and returns values back in xml again. I am trying to post the xml data to the web services, without any success, I need to do it using Perl. Following is the code I tried:
use SOAP::Lite ;
my $URL = "http://webservice.com:7011/webServices/HealthService.jws?WSDL=";
my $xml_data = '<Request>HealthCheck</Request>' ;
my $result = SOAP::Lite -> service($xml_data);
print $result ;
I tried another approach with proxy:
use SOAP::Lite +trace => 'debug';
my $URI = 'webServices/HealthService' ;
my $URL = "http://webservice.com:7011/webServices/HealthService.jws?WSDL=" ;
my $test = SOAP::Lite -> uri($URI)
-> proxy($URL) ;
my $xml_data = '<Request>HealthCheck</Request>' ;
my $result = $test -> healthRequest($xml_data);
print $result ;
However this is throwing the following error:
Can't locate class method "http://webservice.com:7011/healthRequest" via package "SOAP::Lite\" at 7.pl line 4. BEGIN failed--compilation aborted at 7.pl line 4.
The webservice provides only one method HealthRequest. I am not sure why it is trying to find out the class method in SOAP:Lite. I get the same error for both the approach.
Is there any other method to achieve the same using Perl?
Try something like this, I have not tested it so just test and see what happens, you should at least not get the PM error.
use strict;
use SOAP::Lite;
my $xml_data = '<Request>HealthCheck</Request>' ;
my $soap = SOAP::Lite
->uri("webServices/HealthService")
->proxy("http://webservice.com:7011/webServices/HealthService.jws?WSDL=");
print $soap->service($xml_data),"\n";
If you want to create the XML yourself and not delegate that task to SOAP::Lite, you need to let SOAP::Lite know what you are doing:
$soap = SOAP::Lite->ns( $URI )->proxy( $URL );
$soap->HealthCheck( SOAP::Data->type( xml => $xml_data ) );
I have my doubts, though, that this will work with your XML.
If your request really has no variable parameters, this may work:
$soap = SOAP::Lite->ns( $URI )->proxy( $URL );
$soap->HealthCheck;
PS: Are your sure that your webservice is a SOAP service?

soap lite pass string argument

I have a problem passing a string argument using Perl. The following code
#!/usr/bin/perl -w
use SOAP::Lite;
my $service = SOAP::Lite->service('http://localhost:8080/greeting?wsdl');
print $service->greetClient('perl wooooo'), "\n";
Results in
Greeting null! Have a nice day...
A similar python code
from suds.client import Client
client = Client('http://localhost:8080/greeting?wsdl')
print client.service.greetClient('python wooooo')
works perfectly
Greeting python wooooo! Have a nice day...
I tried to set different encodings
print $service->encoding('utf-8')->greetClient("perl wooooo"), "\n";
with the same result.
A SOAP Monitor shows that there is no arg0 in a case of Perl
<greetClient xsi:nil="true" xsi:type="tns:greetClient" />
which is present in a case of Python
<ns0:greetClient>
<arg0>python wooooo</arg0>
</ns0:greetClient>
What can be a problem?
Why it's so complicated to implement a SOAP client with Perl compared to Python?
EDIT:
SOLUTION
Finally the following solution is working
#!/usr/bin/perl -w
use strict;
use warnings;
use XML::Compile::SOAP11;
use XML::Compile::WSDL11;
use XML::Compile::Transport::SOAPHTTP;
my $soap = XML::Compile::WSDL11->new('c:/temp/greeting.wsdl');
my $call = $soap->compileClient('greetClient');
print $call->(arg0 => 'perl wooooo'){'greetClientResponse'}{'return'}, "\n";
SOAP::Lite can be infuriatingly bad. You might give XML::Compile::SOAP a try:
use strict;
use warnings;
use XML::Compile::SOAP11;
use XML::Compile::WSDL11;
use XML::Compile::Transport::SOAPHTTP;
my $soap = XML::Compile::WSDL11->new(
'http://localhost:8080/greeting?wsdl',
schema_dirs => [
'c:/soft/Perl/site/lib/XML/Compile/SOAP11/xsd'
'c:/soft/Perl/site/lib/XML/Compile/XOP/xsd'
'c:/soft/Perl/site/lib/XML/Compile/xsd'
]
);
$soap->compileCalls;
my ( $response, $trace ) = $soap->call( 'greetClient', arg0 => 'perl wooooo' );
$trace->printResponse;
$response will be the call response converted to a hashref via XML::Simple, which may be all you need. The $trace object is handy to see what the raw XML response looks like.
Unfortunately, I can't see your WSDL.
But in regards to SOAP::Lite, I don't see you setting up neither a proxy (endpoint) nor an uri.
You're also probably going to have to change the on_action behavior as well. By default, SOAP::Lite wants to use the '#' concatenation.
So something along these lines might work.
$service->proxy( $uri_of_my_end_point );
$service->uri( $schema_namespace );
$service->on_action( sub {
my ( $uri, $method ) = #_;
my $slash = $uri =~ m{/$} ? '' : '/';
return qq{"$uri$slash$method"};
});

Perl web API using Data::Dumper

We've developed an open web API using Apache and mod_perl, where you can pass text created by Data::Dumper to make requests.
Our data generally looks like this:
$VAR1 = {
'OurField' => 'OurValue'
};
Currently, I noticed we're using an eval to get the data back into a Perl hash server side:
my $VAR1;
eval $our_dumper_string;
#$VAR1 is now filled with hash value
The problem with this, is it is a major security issue. You can pass malicious perl code in there and it will run server side...
It there a better way to safely take a Data::Dumper string and turn it into a hash?
Yes. Use JSON::XS and use JSON rather than Data::Dumper format. That is much more compatible with other web APIs
If your data is simple and predictable you can even try to write a simple "parser" to read back the values in a data stricture
#!/usr/bin/perl
use strict;
use warnings;
use Data::Dumper;
my $data = { 'key1' => 'value' };
my $dumper = Dumper($data);
print $dumper;
my $data_2;
while( $dumper =~ /(.+)$/mg) {
if ( $1 =~ m/'(.*)' => '(.*)'/ ) {
$data_2->{$1} = $2;
}
}
print Dumper( $data_2 );
(this is just an example and wont work with integers or nested data structures)

Querying a website with Perl LWP::Simple to Process Online Prices

In my free time, I've been trying to improve my perl abilities by working on a script that uses LWP::Simple to poll one specific website's product pages to check the prices of products (I'm somewhat of a perl noob). This script also keeps a very simple backlog of the last price seen for that item (since the prices change frequently).
I was wondering if there was any way I could further automate the script so that I don't have to explicitly add the page's URL to the initial hash (i.e. keep an array of key terms and do a search query amazon to find the page or price?). Is there anyway way I could do this that doesn't involve me just copying Amazon's search URL and parsing in my keywords? (I'm aware that processing HTML with regex is generally bad form, I just used it since I only need one small piece of data).
#!usr/bin/perl
use strict;
use warnings;
use LWP::Simple;
my %oldPrice;
my %nameURL = (
"Archer Season 1" => "http://www.amazon.com/Archer-Season-H-Jon-Benjamin/dp/B00475B0G2/ref=sr_1_1?ie=UTF8&qid=1297282236&sr=8-1",
"Code Complete" => "http://www.amazon.com/Code-Complete-Practical-Handbook-Construction/dp/0735619670/ref=sr_1_1?ie=UTF8&qid=1296841986&sr=8-1",
"Intermediate Perl" => "http://www.amazon.com/Intermediate-Perl-Randal-L-Schwartz/dp/0596102062/ref=sr_1_1?s=books&ie=UTF8&qid=1297283720&sr=1-1",
"Inglorious Basterds (2-Disc)" => "http://www.amazon.com/Inglourious-Basterds-Two-Disc-Special-Brad/dp/B002T9H2LK/ref=sr_1_3?ie=UTF8&qid=1297283816&sr=8-3"
);
if (-e "backlog.txt"){
open (LOG, "backlog.txt");
while(){
chomp;
my #temp = split(/:\s/);
$oldPrice{$temp[0]} = $temp[1];
}
close(LOG);
}
print "\nChecking Daily Amazon Prices:\n";
open(LOG, ">backlog.txt");
foreach my $key (sort keys %nameURL){
my $content = get $nameURL{$key} or die;
$content =~ m{\s*\$(\d+.\d+)} || die;
if (exists $oldPrice{$key} && $oldPrice{$key} != $1){
print "$key: \$$1 (Was $oldPrice{$key})\n";
}
else{
print "\n$key: $1\n";
}
print LOG "$key: $1\n";
}
close(LOG);
Yes, the design can be improved. It's probably best to delete everything and start over with an existing full-featured web scraping application or framework, but since you want to learn:
The name-to-URL map is configuration data. Retrieve it from outside of the program.
Store the historic data in a database.
Learn XPath and use it to extract data from HTML, it's easy if you already grok CSS selectors.
Other stackers, if you want to amend my post with the rationale for each piece of advice, go ahead and edit it.
I made simple script to demonstate Amazon search automation. Search url for all departments was changed with escaped search term. The rest of code is simple parsing with HTML::TreeBuilder. Structure of HTML in question can be easily examined with dump method (see commented-out line).
use strict; use warnings;
use LWP::Simple;
use URI::Escape;
use HTML::TreeBuilder;
use Try::Tiny;
my $look_for = "Archer Season 1";
my $contents
= get "http://www.amazon.com/s/ref=nb_sb_noss?url=search-alias%3Daps&field-keywords="
. uri_escape($look_for);
my $html = HTML::TreeBuilder->new_from_content($contents);
for my $item ($html->look_down(id => qr/result_\d+/)) {
# $item->dump; # find out structure of HTML
my $title = try { $item->look_down(class => 'productTitle')->as_trimmed_text };
my $price = try { $item->look_down(class => 'newPrice')->find('span')->as_text };
print "$title\n$price\n\n";
}
$html->delete;