Thanks to ffiore I discovered a nice way for communicating data between processes and thread: Message Queues. MQs are a Linux feature that implement a kernel-based push-pop stack. Linux has two types of MQs: Posix and SystemV. This post is about SystemV.
Instead of reading/writing data to a standard pipe/socket where you have to manage the split of the messages and other issues, you can simply use MQs with the following pseudocode procedure:
# pusher thread
struct mq_msg msg;
msg.mtext = “my message”
mq = mq_create()
msgsnd(mq, msg, ..)
while the pop-er thread gets messages
# popper thread
msg = msgrcv(mq,..)
Linux provides some command line tools to manage the queue, while queue size is managed by sysctl and ulimit.
ananke # sysctl -a | grep msg
kernel.msgmnb = 65536 # max queue size = (msgsize*#msgs)
kernel.msgmni = 16
kernel.msgmax = 65536 #Â maximum Â number Â of Â messages Â in Â a Â queue.
fs.mqueue.msgsize_max = 8192 # maximum message size.
fs.mqueue.msg_max = 10
If we want to queue 10k messages of 12kbytes each, we should set:
# sysclt -w fs.mqueue.msgsize_max=12000
# sysclt -w kernel.msgmnb=120000000
You can view message queues with the #ipcs command:
ananke# sudo apt-get installÂ util-linux
an MQ follows, containing 12 messages summing up 390 bytes
ananke #Â Â ipcs -q
—— Message Queues ——–
key Â Â Â Â msqid Â Â Â owner Â Â Â perms Â Â Â used-bytes Â messages
0x00000000 0 Â Â Â Â Â rpolli Â Â 666 Â Â Â Â 390 Â Â Â Â Â 13
For further info on System V and Posix MQs
#man svipc mq_overview