liboml2(3)
==========

NAME
----
liboml2 - OML2 client library

SYNOPSIS
--------
[verse]
*#include <oml2/omlc.h>*
[verse]
'int'    *omlc_init*('const char' \*appname, 'int' \*pargc, 'const char' \*argv[], 'o_log_fn' log_fn); +
'OmlMP'* *omlc_add_mp*('const char' \*name, 'OmlMPDef' \*definition); +
'int'    *omlc_start*('void'); +
'void'   *omlc_inject*('OmlMP' \*mp, OmlValueU \*values); +
'int'	 *omlc_inject_metadata*('OmlMP'* mp, 'const char'* key, 'const OmlValueU'* value, 'OmlValueT' type, 'const char'* fname); +
'oml_guid_t' *omlc_guid_generate*(); +
'int'    *omlc_close*('void'); +
[verse]
*#define OML_FROM_MAIN* '// enable oml_register_mps() and necessary storage; only in one file'
*#include "APPNAME_oml.h"*
[verse]
'inline void' *oml_register_mps*(); +
'inline void' *oml_inject_MPNAME*('OmlMP'* mp, ...); +

DESCRIPTION
-----------

*liboml2* is the client library for OML2.  It provides an API for
application writers to collect measurements from their applications via
user-defined 'Measurement Points' (MPs).  It also provides a flexible
filtering and collection mechanism that allows application users to
customise how measurements are processed and stored by an OML-enabled
application.

This man page summarises the API.  Each function in the API also has its
own detailed man page.  To learn how to configure an application that
uses the API from the command line, see linkoml:liboml2[1].  To learn
about the external configuration file that *liboml2* can use, see
linkoml:liboml2.conf[5].

A programmer who wants to use *liboml2* to perform measurements in their
application must first decide what to measure. OML lets the programmer
first define 'MPs' and then inject measurement tuples into them. The
core of the library then performs filtering on the injected measurements
and, at predefined times, emits the filtered samples to either a local
file or a measurement server (see linkoml:oml2-server[1]).

A program using OML can therefore be divided into two stages:

Initialisation:: First the library is initialised and measurement points
are defined.

Measurement::
Second, the filters are constructed and enabled by calling
linkoml:omlc_start[3]. Measurement can proceed, with as many calls to
linkoml:omlc_inject[3] as required.

In some cases, it is necessary to occasionaly inject metadata about one
'MP', or one of its fields (e.g., unit or quality index of the current
samples). The linkomlalias:omlc_inject_metadata[omlc_inject,3] function
can be used for this purpose.

It is also sometimes necessary to link measurement tuples together
(e.g., the destination IP of a packet in a network 'MP' and its
transport protocol and port in a transport 'MP'). OML supports the
concept of a Globally Unique ID (GUID) as a data type. It is a random
identifier which can be generated on demand using the
*omlc_guid_generate*() function. It returns an 'oml_guid_t', which can
then be injected into any 'MP' with a field of type ':guid' (see
linkoml:oml2-scaffold[1]). In case a GUID is not required for the
injection of a specific tuple (i.e., no linkage to other measurements),
the special value 'OMLC_GUID_NULL' can be used.

For convenience, it is recommended to use linkoml:oml2-scaffold[1] to
generate helper functions for MP registration and injection
(*oml_register_mps* and *oml_inject_MPNAME*) which take care of
preparing the data and passing it to linkoml:omlc_add_mp[3] and
linkoml:omlc_inject[3], respectively.

It is also best to call linkoml:omlc_close[3] to shut down the
measurement library cleanly once measurement tasks are completed, such
as when the application is terminating.

Initialisation
~~~~~~~~~~~~~~
When the application starts up, the programmer should make a call to
linkoml:omlc_init[3] to initialise *liboml2*.  The function accepts a
name for the application itself, together with the executable's command
line arguments (the C *main*() function's *argc* and *argv* parameters),
and a fourth argument which is not widely used and can safely be set to
NULL:

----
#include <oml2/omlc.h>

int main(int argc, const char **argv)
{
  int result = omlc_init ("MyOmlClient", &argc, argv, NULL);
  if (result == -1) {
    fprintf (stderr, "Could not initialise OML\n");
    exit (1);
  } else if (result == 1) {
    fprintf (stderr, "OML was disabled by the user, exiting\n");
    exit (0);
  }

  /* Do application stuff... */

  return 0;
}
----

The linkoml:omlc_init[3] function gathers configuration information from
the command line arguments, the environment, and possibly an external
configuration file, and sets internal configuration options of the
library.  It removes any command line options that it recognises, so
that the application itself does not get confused when it tries to
interpret the command line.  All OML options start with the prefix
*--oml-*.  An application should call linkoml:omlc_init[3] _before_
running its own command line option parser.

After initialising the library, the programmer must define the
application's 'MPs'.  Each 'MP' is defined as a typed tuple.  OML
supports basic integer types, a double-precision floating point type,
a string type and a blob type.

[[generated-register,]]
Using the registration helper
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

This is the recommended method as it is easiest.

The linkoml:oml2-scaffold[1] tool can generate the definition of the
desired measurement points, and a single registration function as part
of the OML header ('APPNAME_oml.h' generated with the *--oml* flag).
The registration function, *oml_register_mps*(), declares theses MPs to
the library. It takes no argument, and initialises a structure
*g_oml_mps_APPNAME* which contains named pointer to each measurement
point. Taking the example from linkoml:oml2-scaffold[1],

----
defApplication('oml:man:foo', 'foo') do |app|
  # ... Some needed code skipped here ...
  app.defMeasurement("memory") do |mp|
    # ... Some more code skipped here ...
  end

  app.defMeasurement("cpu") do |mp|
    # ... Some more code skipped here ...
  end
end
----

The *oml_register_mps*() function declares and initialises a structure
containing pointers to the 'OmlMP*'. In this example, where the
application name is 'foo', it will be accessible as 'g_oml_mps_foo', and
the 'OmlMP*' for 'memory' and 'cpu' as 'g_oml_mps_foo->memory' and
'g_oml_mps_foo->cpu', respectively.

This function is made available conditionally from 'APPNAME_oml.h'.
Indeed, it requires objects which need only be allocated once, and is
done at the same time. To make this function available, one must
'#define' linkman:cpp[1] macro 'OML_FROM_MAIN' prior to the '#include'
statement for 'APPNAME_oml.h'. This should be done *only in file using
oml_register_mps()*.

Registering MPs manually
^^^^^^^^^^^^^^^^^^^^^^^^
In case some or more 'MPs' cannot be known at compile time (e.g., a
plugin providing its own 'MPs'), the auto-generated registration code
cannot be used. These 'MPs' have to be defined and added manually in the
code. An 'MP' definition is represented as an array of 'OmlMPDef'
objects, terminated by a NULL object, as shown here:

----
OmlMPDef mp_def [] =
{
  { "source", OML_UINT32_VALUE },
  { "destination", OML_UINT32_VALUE },
  { "length", OML_UINT32_VALUE },
  { "weight", OML_DOUBLE_VALUE },
  { "protocol", OML_STRING_VALUE },
  { NULL, (OmlValueT)0 }
};
----

The first member of the 'OmlMPDef' struct is the name of the field of
the 'MP' that it represents.  The second member is its type.  The name
appears in the schema for the local file storage, and as part of the
database column name for the database created by linkoml:oml2-server[1].

To register the 'MP' definition with *liboml2*, the programmer must call
linkoml:omlc_add_mp[3]:

----
OmlMP* mp = omlc_add_mp ("packet_info", mp_def);

if (mp == NULL) {
 fprintf (stderr, "Error: could not register Measurement Point 'packet_info'");
 exit (1);
}
----

This function returns a handle to the internal 'MP' object, which should
be used in subsequent calls to linkoml:omlc_inject[3], and should be
treated as opaque.

Measurement
~~~~~~~~~~~

Once all measurement points have been defined and created using calls to
linkoml:omlc_add_mp[3], the programmer must start the measurement
collection process by calling linkoml:omlc_start[3].  This call creates
the internal filters according to the current configuration options, and
starts the filter output sampling threads for \'interval' style filters
(for more information see linkoml:liboml2.conf[5]).  Once
linkoml:omlc_start[3] has been called, no more 'MPs' can be added:
further calls to linkoml:omlc_add_mp[3] will be ignored.  Conversely,
once linkoml:omlc_start[3] has been called, calls can be made to perform
measurements, and inject there results into the OML reporting path.

Using injection helpers
^^^^^^^^^^^^^^^^^^^^^^^

This is the recommended method as it is easiest and much less error-prone. It
is also more future-proof as it shields the application from low-level API
changes.  These helpers also provide C-compiler type-checking and proper memory
management.

The linkoml:oml2-scaffold[1] tool can generate helper injection
functions as part of the OML header (*--oml*). Their prototype is  of
the form *oml_inject_MPNAME*('OmlMP*' mp, ...), where *MPNAME* is the
name of the 'MP' that is passed as *mp*. The remaining arguments are
C-typed variables constituting the sample to be injected in the 'MP'.

Continuing with the <<generated-register,auto-generated example from above>>,
injecting a sample in the 'memory' 'MP' can be done as

----
oml_inject_memory(g_oml_mps_foo->memory, ram, total, free, used, free_percent, used_percent);
----

where 'ram', 'total', 'free', 'used', 'free_percent' and 'used_percent' are
assumed to be variables updated by the rest of the application code.

[[manual-inject,]]
Implementing injection code manually
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The linkoml:omlc_inject[3] function accepts an 'MP' handle and a vector
of linkoml:OmlValueU[3] objects. These values MUST always be initialised
first with linkomlalias:omlc_zero[OmlValueU,3] or linkomlalias:omlc_zero_array[OmlValueU,3]. The
programmer should first load up the vector with new values to be
measured using the *omlc_set_** macros, and then call
linkoml:omlc_inject[3]. If string or blobs were affected to some of the
linkoml:OmlValueU[3], these have to be reset with
linkomlalias:omlc_reset_string[OmlValueU,3] or linkomlalias:omlc_reset_blob[OmlValueU,3] so any
allocated memory is properly cleared. For instance, here is how a
measurement injection might look for the 'MP' definition above:

----
...

OmlMP *mp = omlc_add_mp("packet_info", mp_def);

...

uint32_t source_id;
uint32_t destination_id;
uint32_t packet_length;
double weight;
char *protocol;

...

/* Some application-specific code to obtain new values for the variables above */

...

{
   OmlValueU values[5];

   omlc_zero_array(values, 5);

   omlc_set_uint32 (values[0], source_id);
   omlc_set_uint32 (values[1], destination_id);
   omlc_set_uint32 (values[2], packet_length);
   omlc_set_double (values[3], weight);
   omlc_set_string (values[4], protocol);

   omlc_inject (mp, values);

   omlc_reset_string (values[4]); /* Free potentially allocated space */
}
----

The vector must be loaded with values in the same order as the original
definition of the fields of the 'MP' in the call to
linkoml:omlc_add_mp[3]. The call to linkoml:omlc_inject[3] can be
repeated as many times as the program wants to make measurements for a
given 'MP'.

OML Types
+++++++++

OML uses an opaque data structure, linkoml:OmlValueU[3] to store
arbitrary data types. This structure can contain chunks of allocated
memory and should therefore be properly initialised before use (with the
linkomlalias:omlc_zero[OmlValueU,3] and linkomlalias:omlc_zero_array[OmlValueU,3] functions.

OML defines the following types and setters:

* 'OML_INT32_VALUE' (linkomlalias:omlc_set_int32[OmlValueU,3])
* 'OML_UINT32_VALUE' (linkomlalias:omlc_set_uint32[OmlValueU,3])
* 'OML_INT64_VALUE' (linkomlalias:omlc_set_int64[OmlValueU,3])
* 'OML_UINT64_VALUE' (linkomlalias:omlc_set_uint64[OmlValueU,3])
* 'OML_DOUBLE_VALUE' (linkomlalias:omlc_set_double[OmlValueU,3])
* 'OML_STRING_VALUE' (linkomlalias:omlc_set_string[OmlValueU,3], linkomlalias:omlc_reset_string[OmlValueU,3])
* 'OML_BLOB_VALUE' (linkomlalias:omlc_set_blob[OmlValueU,3], linkomlalias:omlc_reset_blob[OmlValueU,3])
* 'OML_GUID_VALUE' (linkomlalias:omlc_set_guid[OmlValueU,3])
* 'OML_BOOL_VALUE' (linkomlalias:omlc_set_bool[OmlValueU,3])

Each of the first four integer types maps to an underlying equivalent
type from the C standard header linkman:stdint.h[0].  These types should
be used in calls to the *omlc_set_** macros.  They are:

* 'int32_t'
* 'uint32_t'
* 'int64_t'
* 'uint64_t'

The 'OML_DOUBLE_VALUE' type maps to an underlying C double.

The 'OML_STRING_VALUE' maps to a nil-terminated C string. Memory will be
dynamically allocated (or reused) when a value of this type is set to an
linkoml:OmlValueU[3].

The 'OML_BLOB_VALUE' maps to an arbitrary block of binary data, of a given
size. It is handled in a way similar to 'OML_STRING_VALUE', save for the
nil-termination and automatic size calculation.

The 'OML_GUID_VALUE' type can be used to create logical groups of
samples, either within the same 'MP', or across 'MPs'. For example,
parameters from the IP and TCP headers of a TCP packet should be
reported in separate 'MPs' (as not all IP packets have TCP payload).
However, it is desirable to link these tuples together. Adding an
'OML_GUID_VALUE' field in both 'MPs', which can be filled with the same
GUID returned linkomlalias:omlc_guid_generate[liboml2,3] would allow to
join the relevant information about this packet at a later time, during
analysis.  Similarly, if multiple samples in an 'MP' belong to the same
logical group (e.g., an array), they can be linked using the same
mechanism, by adding an 'OML_GUID_VALUE' field to that 'MP', and
generating a new 'oml_guid_t' every time a new group needs to be
created. To fill 'OML_GUID_VALUE' fields where grouping is not needed, a
default 'NULL' value is available as 'OMLC_GUID_NULL'.

The 'OML_BOOL_VALUE' simply encapsulate a boolean value. Anything that is
not 0 is assumed to be true. There is however no guarantee that a
non-zero boolean will retain the actual (integral) value which was used
to set it, beyond its logical truth value.

*liboml2* currently also defines an 'OML_LONG_VALUE' that maps to a C
'long' type, but this type is deprecated because it can change size
between 32-bit and 64-bit platforms. It will be removed from the API at
some point in the future. The library internally clamps 'OML_LONG_VALUE'
'MP' fields to the most positive and negative values that will fit in an
'int32_t' object before sending them to the linkoml:oml2-server[1] or a
local measurement file. 'OML_LONG_VALUE' should not be used in new
applications.

Building
~~~~~~~~
Applications using *liboml2* must include the '-loml2' flag on the
linker command line to link against the client library.

[[example]]
EXAMPLE
-------

The following is a fully-functional example application that counts from
1 to 1000 and outputs the value of the counter as an unsigned integer, a
string, and a double at each step:

--------
#include <stdio.h>
#include <stdlib.h>
#include <oml2/omlc.h>

int main (int argc, const char **argv)
{
  int result = omlc_init ("Simple", &argc, argv, NULL);
  if (result == -1) {
    fprintf (stderr, "Could not initialise OML\n");
    exit (1);
  } else if (result == 1) {
    fprintf (stderr, "OML was disabled by the user, exiting\n");
    exit (0);
  }

  OmlMPDef mp_def [] = {
    { "count", OML_UINT32_VALUE },
    { "count_str", OML_STRING_VALUE },
    { "count_real", OML_DOUBLE_VALUE },
    { NULL, (OmlValueT)0 }
  };

  OmlMP *mp = omlc_add_mp ("counter", mp_def);

  if (mp == NULL) {
    fprintf (stderr, "Error: could not register Measurement Point 'counter'");
    exit (1);
  }

  omlc_start();

  int i = 0;
  for (i = 0; i < 1000; i++) {
    uint32_t count = (uint32_t)i;
    char count_str[16];
    double count_real = (double)i;
    OmlValueU values[3];

    omlc_zero_array(values, 3);

    snprintf(count_str, sizeof(count_str), "%d", i);

    omlc_set_uint32 (values[0], count);
    omlc_set_string (values[1], count_str);
    omlc_set_double (values[2], count_real);

    omlc_inject (mp, values);

    omlc_reset_string(values[1]);
  }


  omlc_close();

  return 0;
}
--------

The following command should be sufficient to compile this
program:

--------
$ gcc -o counter counter.c -loml2
--------

Here is an example command line to run this application with a default
set of filters:

--------
$ ./counter --oml-id myid --oml-domain count --oml-collect file:-
--------

The output from this command will appear on the terminal because we told
OML to use the standard output (*--oml-collect file:-*).  Here is what
it looks like:

--------
INFO   OML Client 2.x.y [OMSPv5] Copyright 2007-2014, NICTA
protocol: 5
domain: count
start-time: 1283160287
sender-id: myid
app-name: Counter
schema: 1 Counter_counter count:uint32 count_str:string count_real:double
content: text


0.797713	1	1	0	0	0.000000
0.797736	1	2	1	1	1.000000
0.797748	1	3	2	2	2.000000
0.797760	1	4	3	3	3.000000
0.797774	1	5	4	4	4.000000
0.797785	1	6	5	5	5.000000
0.797796	1	7	6	6	6.000000
...
--------

An easy way to create more complicated applications is to use
linkoml:oml2-scaffold[1].

PORTING PRE-2.9 APPLICATIONS
----------------------------

In between OML 2.8 and 2.9, a heavy refactoring of the lower layers
happened to better track the memory used, and reduce memory leaks.
Unfortunately, this change could not be hidden from the higher layers,
as the API was extended with the introduction of initialisers
destructors for linkoml:OmlValueU[3] variables, with proper management
of dynamic memory allocation.

The shared library's version has been bumped from 0.8.1 to 9.0.0, so old
binaries could still work properly (on distributions supporting multiple
library versions, such as Debian). However, this means these old
binaries will not benefit from the new features of OML 2.9 onwards.

Porting pre-2.9 instrumentations to OML 2.9 is however relatively easy.
Whenever linkoml:OmlValueU[3], or arrays thereof, are declared, they
MUST be properly initialised prior to any use. Not doing so might result
in cryptic segmentation faults, or weird ghostly data being collected.
Refer to <<manual-inject,Implementing injection code manually>> and the
<<example,example>> above to see where calls to
linkomlalias:omlc_zero[OmlValueU,3] (or
linkomlalias:omlc_zero_array[OmlValueU,3]), and
linkomlalias:omlc_reset_string[OmlValueU,3] and
linkomlalias:omlc_reset_blob[OmlValueU,3] should be added to your code.

Additionally, omlc_set_const_string() is deprecated, as it did not have
much significance. The string is now always duplicated into separate
storage to ensure integrity regardless of how the instrumented
application manipulates its original data after injection.

All new instrumentations should however avoid using
linkoml:omlc_inject[3] directly, and rely on the code generation
capabilities of linkoml:oml2-scaffold[1] instead. It is also recommended
that these changes be taken as an opportunity to port old applications
to linkoml:oml2-scaffold[1] too.

BUGS
----
include::bugs.txt[]

SEE ALSO
--------
Manual Pages
~~~~~~~~~~~~
linkoml:oml2-server[1], linkoml:liboml2[1], linkoml:liboml2.conf[5], linkoml:oml2-scaffold[1]

linkoml:omlc_init[3], linkoml:omlc_add_mp[3], linkoml:omlc_start[3], linkoml:omlc_inject[3], linkoml:omlc_close[3]

linkoml:OmlValueU[3]

include::manual.txt[]

// vim: ft=asciidoc:tw=72
