[dancer-users] High CPU usage on DB
Hermann Calabria
hermann at ivouch.com
Sat Dec 10 21:21:13 GMT 2016
If you're using Dancer::Plugin::Database (and I suspect D2 has the same
issue), you're using transactions, and a crash occurs mid-transaction, note
that there is no automatic rollback() on die(). I documented this issue
here, and bigpresh (the plugin author) was kind enough to offer some
suggestions:
https://github.com/bigpresh/Dancer-Plugin-Database/issues/76
With enough of these, you end up with a bunch of unfinished transactions,
and the obvious problems eventually start occurring -- memory leaks, high
cpu usage, hung processes, etc.
Unrelated, but may be helpful: We gave up on Starman. Just could not get it
stable in a high-traffic production environment. We switched to uwsgi (with
nginx) and haven't looked back since. uwsgi is fast, compact, has tons of
useful options. It is used by Python stacks, so it is well-supported and
updated, including commercial support if you need it.
Good luck on your project!
-----Original Message-----
From: Zahir Lalani
Sent: Saturday, December 10, 2016 1:52 AM
To: Perl Dancer users mailing list
Subject: Re: [dancer-users] High CPU usage on DB
Hi Warren
We are looking into the processes more - we are also looking at all the fine
tuning of PG parameters to make sure none of those are having an impact
On the Dancer side - maybe I need to check that we are doing things
correctly. We use Dancer2::Plugin::Database in the following way.
In each module that performs DB requests, we 'use' the above package
In a typical DB layer request we will do:
Package mmm;
use Dancer2 appname => 'APP';
Use Dancer2::Plugin::Database
.........
$sth = database->prepare("select xxxx from yyy(?)");
$sth->execute($id);
$result = $sth->fetchrow_hashref();
return $result;
............
There is no explicit closing of connections.
We use starman to run our app, and it looks like the number of starman
workers matches exactly the number of open connections to the DB - so it
seems that the plugin holds one open connection per instance.
Does the above all sound the correct use of the plugin?
Z
> -----Original Message-----
> From: dancer-users [mailto:dancer-users-bounces at dancer.pm] On Behalf
> Of Warren Young
> Sent: 08 December 2016 17:33
> To: Perl Dancer users mailing list <dancer-users at dancer.pm>
> Subject: Re: [dancer-users] High CPU usage on DB
>
> On Dec 8, 2016, at 10:27 AM, Zahir Lalani <ZahirLalani at oliver.agency>
> wrote:
> >
> > We use the Dancer database module to create a connection
>
> Are you caching those connections or re-opening a fresh one for every
> request?
>
> And if the latter, are you *closing* the connection before returning the
> queried data to the client? Are you sure, evidenced by monitoring the OS’s
> open file handles counter?
>
> You don’t say which OS this is, but if it’s Linux:
>
> ls /proc/$(pidof my-dancer-program)/fd | wc -l
>
> > so in this case we have 20 connections to the DB
>
> So you believe. Double-check.
>
> > a restart of Starman cures the problem
>
> That sure sounds like a leaked resource of some kind. I have you chasing
> DB
> file handles. You say it isn’t leaked memory. If not either of those,
> then it
> could be web sockets or any similar sort of thing.
>
> _______________________________________________
> dancer-users mailing list
> dancer-users at dancer.pm
> http://lists.preshweb.co.uk/mailman/listinfo/dancer-users
_______________________________________________
dancer-users mailing list
dancer-users at dancer.pm
http://lists.preshweb.co.uk/mailman/listinfo/dancer-users
More information about the dancer-users
mailing list