c++ - Pool of extended OVERLAPPED objects in a multithreaded environment: where and how to use locking efficiently -

c++ - Pool of extended OVERLAPPED objects in a multithreaded environment: where and how to use locking efficiently -

in c++, i've stream object abstracts handle on windows, , i've various derivatives objects, such file, tcpsocket, udpsocket, pipe derives straight stream object, , i've requestio object own version of extended overlapped object, is, requestio straight inherits overlapped structure. now, saying requesio same saying overlapped.

in requestio object store several useful things couldn't stored in single overlapped structure, such flags, user pointers , on. there store pointer next requestio object, in order have intrusive linked list of these objects.

then, stream object has 2 heads of intrusive linked list, 1 requestio objects reading, , 1 writing. in way stream object can have little pool of these requestio objects, , don't have allocate/deallocate them @ every i/o operation, nor needs locking because 2 intrusive list separated readings writings 2 kind of operations may occur in 2 different threads simultaneously in iocps.

when i'll have stream-like objects (such sockets, or pipes) have 1 requestio reading (more 1 not needed) , 1 writing, not need locking because @ first socket.read() new requestio allocated, inserted in linked list, , reused on , on again, until socket closed , destroyed, same writings.

but, there not stream-like objects (such random access file, udp sockets) can issue more 1 requestio both reading or writing. let's consider udp socket can issue n pending requestio objects reading datagrams, or random access file can issue several requestio packets reading/writing to/from different parts of file.

here things complicated. if have linked list of requestio objects, have traverse list , see requestio not pending , issue new i/o operation one.

said seems easy, not: despite can set flag requestio says "its pending", problem not solved: shouldn't flag atomic integer? since flag unset other thread. , retrieving first requestio available linked list, when there multiple requestio instanced? shouldn't interlocked operation? , insertion on linked list? e.g. when allocate new requestio packet because others pending.

a possible solution thinking traverse linked list , check atomic integer in requestio object cas (compareandswap) instruction, if 0 means not pending, , set 1, thread see 1 pending , go next requestio object. if can't find requestio object allocates new one, here should lock linked list head...to insert new allocated requestio object!

so, fastest , effective way correctly manage pool of n overlapped (or requestio in case) objects, without incurring in massive locking degrade performance , purpose of multithreaded iocps?

keep linked list containing unused requestio objects. can pop object head of list whenever need one, , force each object onto list when finished it.

the initializeslisthead, interlockedpushentryslist, , interlockedpopentryslist functions provide efficient multiprocessor-safe linked list implementation.

c++ multithreading winapi atomic iocp