dm_collector_c and Message Parser

In this page, we will review dm_collector_c, a module which implements the key function of message parser in MobileInsight, as well as other auxiliary functions. We also gives a tutorial on how to develop a specific message parser.

What is a dm_collector_c

dm_collector_c is the module developed to facilitate MobileInsight monitor’s functions. The core function is to parse messages. Basically, a message parser translates raw low-level cellular messages (binary stream extracted from the phone’s cellular chipset by diag_revealer) into readable messages. It supports a variety of message parsers each for one type of message supported. MobileInsight supports a series of cellular-specific messages, including the control-plane protocol messages (NAS, RRC) and low-layer cellular-specific protocols (e.g, PHY, MAC, RLC, PDCP). It also support some vendor-specific messages (e.g, statistics) which are defined by cellular chipset vendors (like Qualcomm, Samsung, Intel, MediaTek, etc).

Each message has a pre-defined format, which is either regulated by 3GPP standard or defined by vendors. It consists of multiple elements (fields) in a certain order, and each element takes a fixed number of bits. A message parser reads the input binary stream bit by bit to retrieve values of each element and then organize those elements into a readable message.

Please note that, ``dm_collector_c`` is not just a message parser. . In addition to message parsing, dm_collector_c also supports logs filtering (selective message parsing) and generating configuration which determine message extracted in the mobile phone’s chipset.

dm_collector_c is chipset-specific. Currently, dm_collector_c supports Qualcomm Snapdragon chipsets. We plan to extend it to support more cellular chipsets (e.g., MediaTek, Intel XMM series).

Logic of dm_collector_c: A Big Picture

Here is the complete code for the module of dm_collector_c. Here we only discuss its high-level logics. The following figure shows the main architecture of dm_collector_c:

TODO:Figure is wrong and need to update it

+-------------+ Raw binary logs (via diag_revealer_fifo) +------------------+   Readable parsed messages (via dm_collector_c)   +---------+
|diag_revealer| ---------------------------------------> |AndroidDiagMonitor| ------------------------------------------> |Analyzers|
| (raw logs)  | <---------------------------------------       | (OnlineMonitor)  | <---------------------------------------   |mi2log|
+-------------+      DIAG_CFG (via dm_collector_c)       +------------------+        Filtered logs   (via dm_collector_c)      +---------+

As shown above, dm_collector_c implements three main functions: (1) generating diag_cfg (configuration for diag_revealer), (2) Log filtering, (3) Message parsing.

dm_collector_c is an extension module of python writing in C++. The source code of this module is located in $mobileInsight-desktop/dm_collector_c/, the compiled extension module file dm_collector_c.so will be copy to mobileInsight-desktop/mobile_insight/monitor/dm_collector/ and called by monitors. ``dm_collector_c``.cpp is the entry file which implement 8 functions mentioned above.

TODO/Question: why here 8 functions, but 3 functions in the big picture.

We next present how they are implemented and introduce some key .h/.c files in dm_collector_c for each function.

1. generating diag_cfg (dm_collector_c_disable_logs, dm_collector_c_enable_logs, dm_collector_c_generate_diag_cfg);

// Return: successful or not
static PyObject *
dm_collector_c_disable_logs (PyObject *self, PyObject *args) {
    IdVector empty;
    BinaryBuffer buf;
    PyObject *serial_port = NULL;
    if (!PyArg_ParseTuple(args, "O", &serial_port)) {
        return NULL;
    }
    Py_INCREF(serial_port);

    // Check arguments
    if (!check_serial_port(serial_port)) {
        PyErr_SetString(PyExc_TypeError, "\'port\' is not a serial port.");
        goto raise_exception;
    }

    buf = encode_log_config(DISABLE, empty);
    if (buf.first == NULL || buf.second == 0) {
        Py_DECREF(serial_port);
        Py_RETURN_FALSE;
    }
    (void) send_msg(serial_port, buf.first, buf.second);
    Py_DECREF(serial_port);
    delete [] buf.first;
    Py_RETURN_TRUE;

    raise_exception:
        Py_DECREF(serial_port);
        return NULL;
}

// Return: successful or not
static PyObject *
dm_collector_c_enable_logs (PyObject *self, PyObject *args) {
    PyObject *serial_port = NULL;
    PyObject *sequence = NULL;
    bool success = false;

    if (!PyArg_ParseTuple(args, "OO", &serial_port, &sequence)) {
        return NULL;
    }
    Py_INCREF(serial_port);
    Py_INCREF(sequence);

    // Check arguments
    if (!check_serial_port(serial_port)) {
        PyErr_SetString(PyExc_TypeError, "\'port\' is not a serial port.");
        goto raise_exception;
    }
    if (!PySequence_Check(sequence)) {
        PyErr_SetString(PyExc_TypeError, "\'type_names\' is not a sequence.");
        goto raise_exception;
    }

    success = generate_log_config_msgs(serial_port, sequence);
    if (!success) {
        goto raise_exception;
    }
    Py_DECREF(sequence);
    Py_DECREF(serial_port);
    Py_RETURN_TRUE;

    raise_exception:
        Py_DECREF(sequence);
        Py_DECREF(serial_port);
        return NULL;
}

// Return: successful or not
static PyObject *
dm_collector_c_generate_diag_cfg (PyObject *self, PyObject *args) {
    PyObject *file = NULL;
    PyObject *sequence = NULL;
    bool success = false;
    BinaryBuffer buf;
    IdVector empty;

    if (!PyArg_ParseTuple(args, "OO", &file, &sequence)) {
        return NULL;
    }
    Py_INCREF(file);
    Py_INCREF(sequence);

    // Check arguments
    if (!check_file(file)) {
        PyErr_SetString(PyExc_TypeError, "\'file\' is not a file object.");
        goto raise_exception;
    }
    if (!PySequence_Check(sequence)) {
        PyErr_SetString(PyExc_TypeError, "\'type_names\' is not a sequence.");
        goto raise_exception;
    }

    // Disable previous logs
    for (int k = 0; k < 2; k++) {
        if (k == 0) {   // Disable normal log msgs
            buf = encode_log_config(DISABLE, empty);
        } else {        // Disable debug msgs
            buf = encode_log_config(DISABLE_DEBUG, empty);
        }
        if (buf.first != NULL && buf.second != 0) {
            (void) send_msg(file, buf.first, buf.second);
            delete [] buf.first;
            buf.first = NULL;
        } else {
            PyErr_SetString(PyExc_RuntimeError, "Log config msg failed to encode.");
            goto raise_exception;
        }
    }

    success = generate_log_config_msgs(file, sequence);
    if (!success) {
        goto raise_exception;
    }
    Py_DECREF(sequence);
    Py_DECREF(file);
    Py_RETURN_TRUE;

    raise_exception:
        Py_DECREF(sequence);
        Py_DECREF(file);
        return NULL;
}

2. filtering logs (dm_collector_c_set_filtered_export, dm_collector_c_set_filtered);

// Return: successful or not
static PyObject *
dm_collector_c_set_filtered_export (PyObject *self, PyObject *args) {
    const char *path;
    PyObject *sequence = NULL;
    IdVector type_ids;
    bool success = false;

    if (!PyArg_ParseTuple(args, "sO", &path, &sequence)) {
        return NULL;
    }
    Py_INCREF(sequence);

    // Check arguments
    if (!PySequence_Check(sequence)) {
        PyErr_SetString(PyExc_TypeError, "\'type_names\' is not a sequence.");
        goto raise_exception;
    }

    success = map_typenames_to_ids(sequence, type_ids);
    if (!success) {
        PyErr_SetString(PyExc_ValueError, "Wrong type name.");
        goto raise_exception;
    }
    Py_DECREF(sequence);

    manager_change_config(&g_emanager, path, type_ids);
    Py_RETURN_TRUE;

    raise_exception:
        Py_DECREF(sequence);
        return NULL;
}

// Return: successful or not
static PyObject *
dm_collector_c_set_filtered (PyObject *self, PyObject *args) {
    PyObject *sequence = NULL;
    IdVector type_ids;
    bool success = false;

    if (!PyArg_ParseTuple(args, "O", &sequence)) {
        return NULL;
    }
    Py_INCREF(sequence);

    // Check arguments
    if (!PySequence_Check(sequence)) {
        PyErr_SetString(PyExc_TypeError, "\'type_names\' is not a sequence.");
        goto raise_exception;
    }

    success = map_typenames_to_ids(sequence, type_ids);
    if (!success) {
        PyErr_SetString(PyExc_ValueError, "Wrong type name.");
        goto raise_exception;
    }
    Py_DECREF(sequence);

    manager_change_config(&g_emanager, NULL, type_ids);
    Py_RETURN_TRUE;

    raise_exception:
        Py_DECREF(sequence);
        return NULL;
}

3. parsing messages (dm_collector_c_feed_binary, dm_collector_c_reset, dm_collector_c_receive_log_packet).

// Return: None
static PyObject *
dm_collector_c_feed_binary (PyObject *self, PyObject *args) {
    const char *b;
    int length;
    if (!PyArg_ParseTuple(args, "s#", &b, &length)){
        // printf("dm_collector_c_feed_binary returns NULL\n");
        return NULL;
    }
    feed_binary(b, length);
    Py_RETURN_NONE;
}

static PyObject *
dm_collector_c_reset (PyObject *self, PyObject *args) {
    reset_binary();
    Py_RETURN_NONE;
}


// Return: decoded_list or None
static PyObject *
dm_collector_c_receive_log_packet (PyObject *self, PyObject *args) {
    std::string frame;
    bool crc_correct = false;
    bool skip_decoding = false, include_timestamp = false;  // default values
    PyObject *arg_skip_decoding = NULL;
    PyObject *arg_include_timestamp = NULL;
    if (!PyArg_ParseTuple(args, "|OO:receive_log_packet",
                                &arg_skip_decoding, &arg_include_timestamp))
        return NULL;
    if (arg_skip_decoding != NULL) {
        Py_INCREF(arg_skip_decoding);
        skip_decoding = (PyObject_IsTrue(arg_skip_decoding) == 1);
        Py_DECREF(arg_skip_decoding);
    }
    if (arg_include_timestamp != NULL) {
        Py_INCREF(arg_include_timestamp);
        include_timestamp = (PyObject_IsTrue(arg_include_timestamp) == 1);
        Py_DECREF(arg_include_timestamp);
    }
    // printf("skip_decoding=%d, include_timestamp=%d\n", skip_decoding, include_timestamp);
    double posix_timestamp = (include_timestamp? get_posix_timestamp(): -1.0);

    bool success = get_next_frame(frame, crc_correct);
    // printf("success=%d crc_correct=%d is_log_packet=%d\n", success, crc_correct, is_log_packet(frame.c_str(), frame.size()));
    // if (success && crc_correct && is_log_packet(frame.c_str(), frame.size())) {
    if (success && crc_correct) {
        // manager_export_binary(&g_emanager, frame.c_str(), frame.size());
        if (!manager_export_binary(&g_emanager, frame.c_str(), frame.size())) {
            Py_RETURN_NONE;
        }
        else if(is_log_packet(frame.c_str(), frame.size())){
            const char *s = frame.c_str();
            // printf("%x %x %x %x\n",s[0],s[1],s[2],s[3]);
            PyObject *decoded = decode_log_packet(  s + 2,  // skip first two bytes
                                                    frame.size() - 2,
                                                    skip_decoding);
            if (include_timestamp) {
                PyObject *ret = Py_BuildValue("(Od)", decoded, posix_timestamp);
                Py_DECREF(decoded);
                return ret;
            } else {
                return decoded;
            }

        }
        else if(is_debug_packet(frame.c_str(), frame.size())){
            //Yuanjie: the original debug msg does not have header...

            unsigned short n_size = frame.size()+sizeof(char)*14;

            char tmp[14]={
                0xFF, 0xFF,
                0x00, 0x00, 0xeb, 0x1f,
                0x00, 0x00, 0x73, 0xB7,
                0xB8, 0x65, 0xDD, 0x00
            };
            // tmp[2]=(char)(n_size);
            *(tmp+2)=n_size;
            *(tmp)=n_size;
            char *s = new char[n_size];
            memmove(s,tmp,sizeof(char)*14);
            memmove(s+sizeof(char)*14,frame.c_str(),frame.size());
            PyObject *decoded = decode_log_packet_modem(s, n_size, skip_decoding);

            // char *s = new char[n_size];
            // memset(s,0,sizeof(char)*n_size);
            // *s = n_size;
            // *(s+2) = n_size;
            // *(s+4) = 0xeb;
            // *(s+5) = 0x1f;
            // memmove(s+sizeof(char)*14,frame.c_str(),frame.size());
            // PyObject *decoded = decode_log_packet(s, n_size, skip_decoding);



            // // The following code does not crash on Android.
            // // But if use s and frame.size(), it crashes
            // const char *s = frame.c_str();
            // PyObject *decoded = decode_log_packet(  s + 2,  // skip first two bytes
            //                                         frame.size() - 2,
            //                                         skip_decoding);

            if (include_timestamp) {
                PyObject *ret = Py_BuildValue("(Od)", decoded, posix_timestamp);
                Py_DECREF(decoded);
                // delete [] s; //Yuanjie: bug for it on Android, but no problem on laptop
                return ret;
            } else {
                // delete [] s; //Yuanjie: bug for it on Android, but no problem on laptop
                return decoded;
            }
        }
        else {
            Py_RETURN_NONE;
        }

    } else {
        Py_RETURN_NONE;
    }
}
  • export_manager.h/.cpp files: support function dm_collector_c_set_filtered_export and dm_collector_c_set_filtered. Only decode filtered packets.

  • hdlc.h/.cpp: calculate CRC, encode hdlc frame and decode hdlc frame. (hdlc frame: :raw-latex:`\x`7d...:raw-latex:`x`7e).

  • log_config.h/.cpp: encode log config message send to dev_diag, support function dm_collector_c_enable_logs, dm_collector_c_disable_logs and dm_collector_c_generate_diag_cfg.

  • utils.h/.cpp: some utility functions.

  • log_packet.h/.cpp: the entry of log packet decoding. Implement and call message parse functions.

To compile those codes, there is a setup.py file under mobileInsight-desktop/. To compile dm_collector_c module:

sudo python setup.py build_ext --inplace

How to develop a message parser?

To develop a message parser for a new message, we have the following four steps:

Step 1. Register the new message type and its parsing function in dm_collector_c.

Step 2. Define the format for the new message type.

Step 3. Match the defined format to python objects.

Step 4. Further polish your roughly matching.

During Step 3 and Step 4, we also need to pay attention about how to organize our result python objects. There are some conventional rules to be explained and a template file is provided for reference.

Add your new message type id and message name entry in consts.h.

There are two data structures in consts.h. In LogPacketType, we enumerate supported log packet type with their type id in hex. In the following example, LTE_LL1_PUSCH_CSF is the packet type and 0xB14E is the type id in hex. The entry function in parser will check this data structure to match type id read in binary stream with a specific packet type and call proper parsing function. The next one is LogPacketTypeID_To_Name, it is an array of ValueName which is defined in utils.h. ValueName is just a structure helps to organize entries with such types: __{int, const char*, bool}__. Here we match every supported packet type with two properties: type name in English, and a boolean to indicate whether we want to expose it to public. The type name in English will be used when monitor or analyzer call function enable_log().

enum LogPacketType{
    LTE_LL1_PUSCH_CSF = 0xB14E,
}
const ValueName LogPacketTypeID_To_Name [] = {
    {LTE_LL1_PUSCH_CSF,
        "LTE_PHY_PUSCH_CSF", false}, // the boolean value indicate whether you want this message to be published.
}

Use parser_template.h to create your new message parser file.

Just simply do: cp parser_template.h new_parser.h

Step 1. Include your new_parser.h in log_packet.cpp file. Register your parser functions in function decode_log_packet:

log_packet.h/.cpp and other parser.h files support the message parsing function in dm_collector_c. You need to first include your new_parser.h in log_packet.cpp, and later register your new parsing function inside the entry function. The entry function is defined in log_packet.cpp as decode_log_packet().

PyObject *decode_log_packect (const char *b, size_t length, bool skip_decoding) {
// Parse Header
    result = PyList_New(0);
    offset = 0;
    offset += _decode_by_fmt(LogPacketHeaderFmt, ARRAY_SIZE(LogPacketHeaderFmt, Fmt),
                                b, offset, length, result);
    PyObject *old_result = result;
    result = PyList_GetSlice(result, 1, 4); // remove the duplicate "len1" field
    Py_DECREF(old_result);
    old_result = NULL;

    // Differentiate using type ID
    LogPacketType type_id = (LogPacketType) _map_result_field_to_name(
                                result,
                                "type_id",
                                LogPacketTypeID_To_Name,
                                ARRAY_SIZE(LogPacketTypeID_To_Name, ValueName),
                                "Unsupported");

    if (skip_decoding) {    // skip further decoding
        return result;
    }

    switch (type_id) {
    case CDMA_Paging_Channel_Message:
        // Not decoded yet.
        break;
        ......
    }
    return result;
}

Here we convert type_id in int (enum) to readable string using LogPacketTypeID we mentioned before in consts.h and then call proper packet parsing functions. The return value is PyObject which should be a well organized python dictionary object. There is an example of decoding log packet header (the common packet header which includes QxDM time stamp, not packet specific) by using _decode_by_fmt function in codes above.

// Decode a binary string according to an array of field description (fmt[]).
// Decoded fields are appended to result
static int
_decode_by_fmt (const Fmt fmt [], int n_fmt,
                const char *b, int offset, int length,
                PyObject *result) {
    // ......
}

Function _decode_by_fmt is defined in log_packet_helper.h file. It consumes binary stream and convert it to proper python object depends on your defined format:

fmt: the array of Fmt you defined before, it will be used to match binary stream.

n_fmt: the array size of fmt

b: the binary stream

offset: the start offset of binary stream

length: the length of binary stream

return value: number of bytes consumed

Finally, you need to add the case of your new supported packet type.

case LTE_LL1_PUSCH_CSF:
        offset += _decode_by_fmt(LteLl1PuschCsf_Fmt,
                ARRAY_SIZE(LteLl1PuschCsf_Fmt, Fmt)
                b, offset, length, result);
        offset += _decode_lte_ll1_pusch_csf_payload(b, offset, length, result);
        break;

In your new_parser.h file you need to have defined two things. One is the packet header format (in most packects, this part is simply an integer of packet verion), for example LteLl1PuschCsf_Fmt here, and another one is the “main” function to parsing the payload of your new message, for example _decode_lte_ll1_pusch_csf_payload.

Step 2. Define the new message format in log_packet.c/.h

The most common supported format types are: UINT, BYTE_STREAM, SKIP and PLACEHOLDER. The format should be defined as an C++ array of class Fmt to indicate format type, element name and number of bytes, (note: the ordering of message field matters):

const Fmt someFmt [] = {
    {type, "name", number of bytes},
    {UINT, "Version", 1},
    {SKIP, NULL, 4},
    {UINT, "System Frame Number", 2},
    {PLACEHOLDER, "Sub-frame Number", 0},
}

UINT: convert binary stream to uint.

BYTE_SREAM: convert binary stream to string in hex

SKIP: In case you have several bytes of stream and you don’t know how to map it, you can use type SKIP to skip bytes in binary stream.

PLACEHOLDER: If you need to insert some elements but you don’t want to consume any bytes from stream, you can use type PLACEHOLDER to place a holder as a field without consuming bytes.

Step 3. Match your defined format to python objects

The message elements should be mapped to python objects, and those objects should be organized into a python dictionary. The value can be integer, float, string, and lists (each item in the lists should still be a dictionary). For example, UINT will be mapped to python integer and BYTE_STREAM will be mapped to python string. You can also create your own customized value type using Py_BuildValue. More info about Py_BuildValue: https://docs.python.org/2/c-api/arg.html.

Function _decode_by_fmt will match binary stream with defined format and build python objects respectively. Call function _replace_result to change value of elements or assign value to the holder later.

offset += _decode_by_fmt(LteLl1PuschCsf_Payload_v42,
                    ARRAY_SIZE(LteLl1PuschCsf_Payload_v42, Fmt),
                    b, offset, length, result);
PyObject *old_object = _replace_result_int(result,
                    "Start System Sub-frame Number", iSubFN);
Py_DECREF(old_object);

In the case of there are several elements and they are less than 1 byte, you can use UINT and PLACEHOLDER in packet format, and call function search_result_int to get the value of this UINT, use bit operation to get value of that little field and call function _replace_result to assign it back to placeholder. Note: if the first UINT type is larger than or equal to 4 bytes, use search_result_uint to get the value.

For example, define the format like this:

const Fmt LteLl1PuschCsf_Payload_v42 [] = {
    {UINT, "Start System Sub-frame Number", 2}, // 4 bits
    {PLACEHOLDER, "Start System Frame Number", 0},  // 10 bits
    {UINT, "PUSCH Reporting Mode", 1},  // last 2 bits in previous byte + 1 bit
    {PLACEHOLDER, "Rank Index", 0}, // 1 bit
    {PLACEHOLDER, "Csi Meas Set Index", 0},    // 1 bit
    {UINT, "Number of Subbands", 4},    // 5 bits
    {PLACEHOLDER, "WideBand CQI CW0", 0},   // 4 bits
    {PLACEHOLDER, "WideBand CQI CW1", 0},   // 4 bits
    {PLACEHOLDER, "SubBand Size (k)", 0},   // 7 bits
    {PLACEHOLDER, "Single WB PMI", 0},  // 4 bits
    {PLACEHOLDER, "Single MB PMI", 0},  // 4 bits
    {PLACEHOLDER, "CSF Tx Mode", 0},    // 4 bits
    {SKIP, NULL, 27},
    {UINT, "Carrier Index", 1}, // 4 bits
    {PLACEHOLDER, "Num CSIrs Ports", 0},    // 4 bits
};

And in the parsing function, search, do bit operation and replace the correct value like this:

uint utemp = _search_result_uint(result, "Number of Subbands");
int iNumSubbands = utemp & 31;
int iWideBandCQICW0 = (utemp >> 5) & 15;
int iWideBandCQICW1 = (utemp >> 9) & 15;
int iSubBandSize = (utemp >> 13) & 127;
int iSingleWBPMI = (utemp >> 20) & 15;
int iSingleMBPMI = (utemp >> 24) & 15;
int iCSFTxMode = (utemp >> 28) & 15;

old_object = _replace_result_int(result, "Number of Subbands",
                    iNumSubbands);
Py_DECREF(old_object);
old_object = _replace_result_int(result, "WideBand CQI CW0",
                    iWideBandCQICW0);
Py_DECREF(old_object);
old_object = _replace_result_int(result, "WideBand CQI CW1",
                    iWideBandCQICW1);
Py_DECREF(old_object);
old_object = _replace_result_int(result, "SubBand Size (k)",
                    iSubBandSize);
Py_DECREF(old_object);
old_object = _replace_result_int(result, "Single WB PMI",
                    iSingleWBPMI);
Py_DECREF(old_object);
old_object = _replace_result_int(result, "Single MB PMI",
                    iSingleMBPMI);
Py_DECREF(old_object);
old_object = _replace_result_int(result, "CSF Tx Mode",
                    iCSFTxMode);
Py_DECREF(old_object);
(void) _map_result_field_to_name(result, "CSF Tx Mode",
                    ValueNameCSFTxMode,
                    ARRAY_SIZE(ValueNameCSFTxMode, ValueName),
                    "(MI)Unknown");

Step 4 (optional). Make modification for your roughly matching

If there are fields which need to be matched to strings using integers, you don’t need to call _search_result_int() and _replace_reuslt(), using ValueName and function _map_result_field_to_name to do the match more conveniently.

Define your customized ValueName:

const ValueName ValueNameRNTIType [] = {
    // 4 bits
    {0, "C-RNTI"},
    {2, "P-RNTI"},
    {3, "RA-RNTI"},
    {4, "Temporary-C-RNTI"},
    {5, "SI-RNTI"}
};

In your parsing function, use _map_result_field_to_name

(void) _map_result_field_to_name(result_record_item,
        "RNTI Type",
        ValueNameRNTIType,
        ARRAY_SIZE(ValueNameRNTIType, ValueName),
        "(MI)Unknown");

Organize your translated python object in a good structure

The message content may be nested, for example, assume there is a type of message, in the payload of this message, there are several records; and in each records, there are several stream; and in each stream, there are several energy metric. Be careful to organize the dictionary with list. The parser_template file gives an example of 4 layers nested structure and should be helpful.

/*
 * This is the message parser template .h file
 * LTE ******
 */

#include "consts.h"
#include "log_packet.h"
#include "log_packet_helper.h"

const Fmt Lte******_Fmt [] = {
    {UINT, "Version", 1},
};

const Fmt Lte***_Payload_v* [] = {
};

const Fmt Lte***_Record_v* [] = {
};

const Fmt Lte***_Stream_v* [] = {
};

const Fmt Lte***_EnergyMetric_v* [] = {
};

static int _decode_lte_***_payload (const char *b,
        int offset, size_t length, PyObject *result) {
    int start = offset;
    int pkt_ver = _search_result_int(result, "Version");

    PyObject *old_object;
    PyObject *pyfloat;
    int temp;

    switch (pkt_ver) {
    case *: // check the message version
        {
            offset += _decode_by_fmt(Lte***_Payload_v*, // decode payload
                    ARRAY_SIZE(Lte***_Payload_v*, Fmt),
                    b, offset, length, result);

            PyObject *result_record = PyList_New(0); // build the list of records
            for (int i = 0; i < num_record; i++) {
                PyObject *result_record_item = PyList_New(0);
                offset += _decode_by_fmt(Lte***_Record_v*, // decode each record
                        ARRAY_SIZE(Lte***_Record_v*, Fmt),
                        b, offset, length, result_record_item);

                PyObject *result_record_stream = PyList_New(0); // build the list of streams
                for (int j = 0; j < num_stream; j++) {
                    PyObject *result_record_stream_item = PyList_New(0);
                    offset += _decode_by_fmt(Lte***_Stream_v*, // decode each stream
                            ARRAY_SIZE(Lte***_Stream_v*, Fmt),
                            b, offset, length, result_record_stream_item);

                    PyObject *result_energy_metric = PyList_New(0); // build the list of energy metrics
                    for (int k = 0; k < num_energy_metric; k++) {
                        PyObject *result_energy_metric_item = PyList_New(0);
                        offset += _decode_by_fmt(Lte***_EnergyMetric_v*, // decode each energy metric
                                ARRAY_SIZE(Lte***_EnergyMetric_v*, Fmt),
                                b, offset, length, result_energy_metric_item);

                        PyObject *t5 = Py_BuildValue("(sOs)", "Ignored",
                                result_energy_metric_item, "dict");
                        PyList_Append(result_energy_metric, t5); // add each energy metric back to list of energy metrics
                        Py_DECREF(t5);
                        Py_DECREF(result_energy_metric_item);
                    }
                    offset += (13 - num_energy_metric) * 4;

                    PyObject *t4 = Py_BuildValue("(sOs)", "Energy Metrics",
                            result_energy_metric, "list");
                    PyList_Append(result_record_stream_item, t4); // add list of energy metrics to single stream
                    Py_DECREF(t4);
                    Py_DECREF(result_energy_metric);

                    PyObject *t3 = Py_BuildValue("(sOs)", "Ignored",
                            result_record_stream_item, "dict");
                    PyList_Append(result_record_stream, t3); // add each stream back to list of streams
                    Py_DECREF(t3);
                    Py_DECREF(result_record_stream_item);
                }

                PyObject *t2 = Py_BuildValue("(sOs)", "Streams",
                        result_record_stream, "list");
                PyList_Append(result_record_item, t2); // add list of streams to single record
                Py_DECREF(t2);
                Py_DECREF(result_record_stream);

                PyObject *t1 = Py_BuildValue("(sOs)", "Ignored",
                        result_record_item, "dict");
                PyList_Append(result_record, t1); // add each record to list of records
                Py_DECREF(t1);
                Py_DECREF(result_record_item);
            }
            PyObject *t = Py_BuildValue("(sOs)", "Records",
                    result_record, "list");
            PyList_Append(result, t); // add list of records to result.
            Py_DECREF(t);
            Py_DECREF(result_record);
            return offset - start;
        }
    default:
        printf("(MI)Unknown LTE *** version: 0x%x\n", pkt_ver);
        return 0;
    }
}

FAQs

Q: How can I know the type id, format and element ground truth of a cellular message got from QualComm chipset?

A: QCAT, TreeView in QCAT.