Skip to main content

Shared memory

The shmem node-type can be used to quickly exchange samples with a process on the same host using a POSIX shared memory object.

Prerequisites​

This node-type does not have any special library dependencies. It is always available.

Implementation​

The source code of the node-type is available here: https://github.com/VILLASframework/node/blob/master/lib/nodes/shmem.cpp

Configuration​

queuelen
integer
Default: "<DEFAULT_SHMEM_QUEUELEN>"

Length of the input and output queues in elements.

samplelen
integer
Default: "<DEFAULT_SHMEM_SAMPLELEN>"

Maximum number of data elements in a single `struct Sample`` for the samples handled by this node.

mode
string
Default: "pthread"
Enum: "pthread" "polling"

If set to pthread, POSIX condition variables (CV) are used to signal writes between processes. If set to polling, no CV's are used, meaning that blocking writes have to be implemented using polling, leading to performance improvements at a cost of unnecessary CPU usage.

exec
Array of strings

Optional name and command-line arguments (as passed to execve) of a command to be executed during node startup. This can be used to start the external program directly from VILLASNode. If unset, no command is executed.

object (Input configuration (received by VILLASnode))
object (Output configuration (sent out by VILLASnode))
vectorize
integer
Default: 1

This setting allows to send multiple samples in a single message to the destination nodes.

The value of this setting determines how many samples will be combined into one packet.

Array of Hook Object (object) or Hook Name (string) (hook_list)
builtin
boolean (Builtin hook functions)
Default: true

By default, each node and paths has a couple of default hooks attached to them. With this setting the attachment of built-in hooks can be disabled.

{
  • "queuelen": "<DEFAULT_SHMEM_QUEUELEN>",
  • "samplelen": "<DEFAULT_SHMEM_SAMPLELEN>",
  • "mode": "pthread",
  • "exec": [
    ],
  • "in": {
    },
  • "out": {
    },
  • "vectorize": 1,
  • "hooks": [
    ],
  • "builtin": true
}

Example​

node/etc/examples/nodes/shmem.conf
nodes = {
    shmem_node = {
        type = "shmem"

        in = {
            # Name of shared memory segment for receiving side
            name = "sn1_in"
        }

        out = {
            # Name of shared memory segment for sending side
            name = "sn1_in"
        }

        # Length of the queues
        queuelen = 1024

        # We can busy-wait or use pthread condition variables for synchronizations
        mode = "pthread"

        # Execute an external process when starting the node which
        # then starts the other side of this shared memory channel
        # Usually we also pass the shmem names as parameters
        exec = [ "villas-shmem", "sn1_in", "sn1_out" ]
    }
}

API for external programs​

The actual sharing of data is implemented by putting two shared struct queues (one per direction) and an associated struct pool in the shared memory region. Samples can be exchanged by simply writing to and reading from these queues.

External programs that want to use this interface must link against libvillas.so.

The interface for external programs is very simple: after opening the shared memory object with shmem_shared_open (passing the object name from the configuration file), samples can be read from and written to VILLASNode using shmem_shared_read and shmem_shared_write, respectively. Samples written to the node must be allocated by sample_alloc from the shared pool; samples read from the node should be freed with sample_put after they have been processed.

See the example client and the API for more details.