Thanks to ffiore I discovered a nice way for communicating data between processes and thread: Message Queues. MQs are a Linux feature that implement a kernel-based push-pop stack. Linux has two types of MQs: Posix and SystemV. This post is about SystemV.
Instead of reading/writing data to a standard pipe/socket where you have to manage the split of the messages and other issues, you can simply use MQs with the following pseudocode procedure:
# pusher thread
struct mq_msg msg;
msg.mtext = “my message”
mq = mq_create()
msgsnd(mq, msg, ..)
while the pop-er thread gets messages
# popper thread
msg = msgrcv(mq,..)
print msg.mtext
Linux provides some command line tools to manage the queue, while queue size is managed by sysctl and ulimit.
ananke # sysctl -a | grep msg
kernel.msgmnb = 65536 # max queue size = (msgsize*#msgs)
kernel.msgmni = 16
kernel.msgmax = 65536 # maximum  number  of  messages  in  a  queue.
fs.mqueue.msgsize_max = 8192 # maximum message size.
fs.mqueue.msg_max = 10
If we want to queue 10k messages of 12kbytes each, we should set:
# sysclt -w fs.mqueue.msgsize_max=12000
# sysclt -w kernel.msgmnb=120000000
You can view message queues with the #ipcs command:
ananke# sudo apt-get install util-linux
an MQ follows, containing 12 messages summing up 390 bytes
ananke #Â Â ipcs -q
—— Message Queues ——–
key     msqid    owner    perms    used-bytes  messages
0x00000000 0      rpolli   666     390      13
For further info on System V and Posix MQs
#man svipc mq_overview