.. _`Introduction`:

############
Introduction
############


*******************************
Google Protocol Buffers for DDS
*******************************

*Vortex OpenSplice* is capable of using the **Google Protocol Buffer**
(*GPB*) system for publishing and subscribing GPB messages in a DDS
system. This makes it possible to use GPB as an alternative to OMG-IDL
for those who prefer to use GPB rather than IDL. With the
seamless integration of GPB and DDS technologies there is no need for
OMG-IDL knowledge or visibility when working with GPB data models, and
no OMG-DDS data-types are needed in the application (no explicit
type-mapping between GPB and DDS types is required).

This results in an easy migration of GPB users to DDS(-based
data-sharing) with data-centric GPB with support for keys, filters
and (future) QoS-annotations (ony a few DDS calls are needed).
Also easy migration of DDS applications to GPB(-based data-modeling),
only the field accessors change.

This Tutorial will describe how this is done for the language bindings
**Java5** and **ISO-C++** by defining a GPB message layout which is
compiled into proper interfaces for the Vortex DDS system.

GPB Installation and usage with DDS
===================================
Google Protocol Buffers (GPB) can be downloaded from the following locations:

Linux:
https://github.com/google/protobuf/releases/download/v2.6.1/protobuf-2.6.1.tar.gz

Windows:
https://github.com/google/protobuf/releases/download/v2.6.1/protobuf-2.6.1.zip

After unpacking follow the install instructions located in install.txt in the unpacked directory.
For windows there is a visual studio solution file that will build everything that is needed.

|info|
  | In order for GPB to work with DDS an environment variable **PROTOBUF_HOME** needs to be
  | set that points to the unpacked directory.
  | For windows also another environment variable **PROTOBUF_LIB_HOME** needs to be set that
  | points to the directory that contains the generated libprotobuf.lib.

IDL usage in a DDS system
=========================

In a Data Distributed System (DDS) as a Global DataSpace (GDS) for
ubiquitous information-sharing in distributed systems as specified by
the Object Management Group (OMG), the data is traditionally
captured in the platform- and language-independent OMG-IDL language.
The relational model of DDS is supported by the notion of
identifying key fields in these data structures
where structure/content-awareness by the middleware allows for
dynamic querying and filtering of data.

Google Protocol Buffers
=======================

Google Protocol Buffers (GPB) are a flexible, efficient, automated
mechanism for serializing structured data; think XML, but smaller,
faster, and simpler. One can define how data needs
to be structured once, after which language-specific source code can
be generated to easily write and read this structured data to and
from a variety of data streams using a variety of
languages. The information structure is defined in so-called *protocol
buffer* message types in ``.proto`` files. Each protocol buffer message is
a small logical record of information, containing a series of name-value
pairs. This approach is quite similar to using IDL for data modeling in
combination with an IDL compiler (as available in OpenSplice and DDS
implementations in general).

Additionally, the GPB data structure can be updated without breaking
deployed programs that are compiled against the 'old' format,
similar to the *xTypes* concept as defined for DDS.

Using a GPB data-model instead of an IDL data-model
---------------------------------------------------

For an IDL-OMG based application, the IDL file is compiled with the
IDL-PP compiler to generate the needed classes.

For Java as an example, ``Address.idl`` will (among others)
be compiled into:

  - ``Address.java``
  - ``AddressTypesSupport.java``
  - ``AddressDataWriter.java``
  - ``AddressDataReader.java``
  - ...

Using a GPB data-model, it is not necessary to create IDL files. The
``protoc_gen_ddsJava`` plug-in in OpenSplice will create them from the
given ``.proto`` data-model.

For the GPB ``.proto`` based application, the ``.proto`` file is first
compiled by the Google ``protoc`` compiler. This compiler will call the
``protoc_gen_ddsJava`` plug-in in OpenSplice with the
``.proto`` data parsed into a ``CodeGeneratorRequest`` protocol
buffer.

The OpenSplice plug-in will generate an IDL file from this data. Any
field member that is marked as key or filterable is explicitly mapped
to a member in the IDL type.

The complete serialized ``.proto`` message is stored in the generic
``ospl_protobuf_data`` attribute as a sequence of bytes (making it
opaque data for DDS). The mapping between data types is
given in the table `Mapping of GPB types to DDS types`_.

As the next step the IDL-PP compiler will generate the previously-named
files from the idl file needed for the DDS domain.
The Google ``protoc`` compiler will generate the classes needed for the
GPB domain.

The dds options for the proto file are given in the
`omg/dds/descriptor.proto`_ file listed below.
This proto file shows how the different dds options on the proto
file are interpreted, and gives the unique id ``1016`` to the dds types.

|info|
  | **Note that the id 1016 has officially been granted to the Vortex product by Google.**
  | This ensures these options are always unique and won't clash with any options used by users.

How mapping is done between the different languages is shown below in
the table `Mapping of GPB types to DDS types`_.


omg/dds/descriptor.proto
------------------------

.. literalinclude:: ../../../../src/tools/protobuf/protos/omg/dds/descriptor.proto
   :language: protobuf


Mapping of GPB types to DDS types
---------------------------------

.. tabularcolumns:: | p{1.6cm} | p{7.5cm} | p{1.5cm} | p{1.5cm} | p{1.5cm} |

+---------------+-------------------------------------+----------+-----------+---------------+
| .proto Type   | Notes                               | C++ Type | Java Type | DDS IDL Type  |
+===============+=====================================+==========+===========+===============+
| double        |                                     | double   | double    | double        |
+---------------+-------------------------------------+----------+-----------+---------------+
| float         |                                     | float    | float     | float         |
+---------------+-------------------------------------+----------+-----------+---------------+
| int32         | Uses variable-length encoding.      | int32    | int       | long          |
|               | Inefficient for encoding            |          |           |               |
|               | negative numbers; if your field     |          |           |               |
|               | is likely to have negative values,  |          |           |               |
|               | use sint32 instead                  |          |           |               |
+---------------+-------------------------------------+----------+-----------+---------------+
| int64         | Uses variable-length encoding.      | int64    | long      | long long     |
|               | Inefficient for encoding            |          |           |               |
|               | negative numbers; if your field     |          |           |               |
|               | is likely to have negative values,  |          |           |               |
|               | use sint64 instead                  |          |           |               |
+---------------+-------------------------------------+----------+-----------+---------------+
| uint32        | Uses variable-length encoding       | uint32   | int       | unsigned long |
+---------------+-------------------------------------+----------+-----------+---------------+
| uint64        | Uses variable-length encoding       | uint64   | long      | unsigned long |
|               |                                     |          |           | long          |
+---------------+-------------------------------------+----------+-----------+---------------+
| sint32        | Uses variable-length encoding.      | int32    | int       | long          |
|               | Signed int value.                   |          |           |               |
|               | These encode negative numbers       |          |           |               |
|               | more efficiently than regular       |          |           |               |
|               | int32s.                             |          |           |               |
+---------------+-------------------------------------+----------+-----------+---------------+
| sint64        | Uses variable-length encoding.      | int64    | long      | long long     |
|               | Signed int value.                   |          |           |               |
|               | These encode negative numbers       |          |           |               |
|               | more efficiently than regular       |          |           |               |
|               | int64s.                             |          |           |               |
+---------------+-------------------------------------+----------+-----------+---------------+
| fixed32       | Always four bytes.                  | uint32   | int       | unsigned long |
|               | More efficient than uint32 if       |          |           |               |
|               | values are often greater than 2^28. |          |           |               |
+---------------+-------------------------------------+----------+-----------+---------------+
| fixed64       | Always eight bytes.                 | uint64   | long      | unsigned long |
|               | More efficient than uint64 if       |          |           | long          |
|               | values are often greater than 2^56. |          |           |               |
+---------------+-------------------------------------+----------+-----------+---------------+
| sfixed32      | Always four bytes.                  | int32    | int       | long          |
+---------------+-------------------------------------+----------+-----------+---------------+
| sfixed64      | Always eight bytes.                 | int64    | long      | long long     |
+---------------+-------------------------------------+----------+-----------+---------------+
| bool          |                                     | bool     | boolean   | bool          |
+---------------+-------------------------------------+----------+-----------+---------------+
| string        | A string must always contain UTF-8  | string   | String    | string        |
|               | encoded or 7-bit ASCII text         |          |           |               |
+---------------+-------------------------------------+----------+-----------+---------------+



.. |caution| image:: ./images/icon-caution.*
            :height: 6mm
.. |info|   image:: ./images/icon-info.*
            :height: 6mm
.. |windows| image:: ./images/icon-windows.*
            :height: 6mm
.. |unix| image:: ./images/icon-unix.*
            :height: 6mm
.. |linux| image:: ./images/icon-linux.*
            :height: 6mm
.. |c| image:: ./images/icon-c.*
            :height: 6mm
.. |cpp| image:: ./images/icon-cpp.*
            :height: 6mm
.. |csharp| image:: ./images/icon-csharp.*
            :height: 6mm
.. |java| image:: ./images/icon-java.*
            :height: 6mm

