forum.vmspython.org Forum Index forum.vmspython.org
Forum system
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

DocXMLRPCServer buffered I/O overhead

 
Post new topic   Reply to topic    forum.vmspython.org Forum Index -> Python for OpenVMS
View previous topic :: View next topic  
Author Message
Colin Brown



Joined: 11 Oct 2012
Posts: 18
Location: Wellington, NZ

PostPosted: Sat Oct 13, 2012 7:35 am    Post subject: DocXMLRPCServer buffered I/O overhead Reply with quote

I am wondering why if I create a simple instance of a DocXMLRPCServer and then serve_forever do I get a buffered I/O count that increases faster than once per second? I can create a simple TCP socket, bind, listen, accept that does not use buffered I/O while waiting.
Back to top
View user's profile Send private message
Colin Brown



Joined: 11 Oct 2012
Posts: 18
Location: Wellington, NZ

PostPosted: Sun Oct 14, 2012 7:01 am    Post subject: Reply with quote

Solved! Crude SockerServer implementation of serve_forever is responsible (serve_forever takes a parameter that is the timeout value to select.select).
Setting a nice high number (like 1e6) stops the idle state buffered I/O's.

BaseServer.serve_forever(poll_interval=0.5)
Handle requests until an explicit shutdown() request. Polls for shutdown every poll_interval seconds.


def serve_forever(self, poll_interval=0.5):
"""Handle one request at a time until shutdown.

Polls for shutdown every poll_interval seconds. Ignores
self.timeout. If you need to do periodic tasks, do them in
another thread.
"""
self.__serving = True
self.__is_shut_down.clear()
while self.__serving:
# XXX: Consider using another file descriptor or
# connecting to the socket to wake this up instead of
# polling. Polling reduces our responsiveness to a
# shutdown request and wastes cpu at all other times.
r, w, e = select.select([self], [], [], poll_interval)
if r:
self._handle_request_noblock()
self.__is_shut_down.set()

select.select(rlist, wlist, xlist[, timeout])
This is a straightforward interface to the Unix select() system call. The first three arguments are sequences of ‘waitable objects’: either integers representing file descriptors or objects with a parameterless method named fileno() returning such an integer:

· rlist: wait until ready for reading
· wlist: wait until ready for writing
· xlist: wait for an “exceptional condition” (see the manual page for what your system considers such a condition)
Empty sequences are allowed, but acceptance of three empty sequences is platform-dependent. (It is known to work on Unix but not on Windows.) The optional timeout argument specifies a time-out as a floating point number in seconds. When the timeout argument is omitted the function blocks until at least one file descriptor is ready. A time-out value of zero specifies a poll and never blocks.
Back to top
View user's profile Send private message
jescab



Joined: 28 Jan 2008
Posts: 254

PostPosted: Mon Oct 15, 2012 12:08 pm    Post subject: Reply with quote

Hi.

I guess you do not need the shutdown feature (that is only polled at timeout from the I/O) ?

And besides, there is realy no need for a timeout that is longer then the mean time between "real" I/O's, at which time the I/O will complete anyway.
Most systems (that are not powered down Smile ) has 100s or 1000s of bufIOs per seconds anyway.
A few "empty" I/O's can hardly make that a big difference, can it?

Just trying to understand the nature of the problem you wanted to solve. Smile
Back to top
View user's profile Send private message
Colin Brown



Joined: 11 Oct 2012
Posts: 18
Location: Wellington, NZ

PostPosted: Mon Oct 15, 2012 12:55 pm    Post subject: Reply with quote

I guess my initial problem was understanding that "serve_forever" is actually implemented as "serve_until_shutdown". You are correct in that a timeout longer than the maximum application idle time is all that is required to not incur the repeated select call timeouts but if I have a server doing say a thousand jobs a day, I am still saving over 160,000 totally pointless buffered I/Os by stopping the half second timeouts when I "serve_forever".
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    forum.vmspython.org Forum Index -> Python for OpenVMS All times are GMT + 2 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group