mobileInsight-core Part-II: Analyzer¶

Overview¶

Given cellular messages from Monitors, MobileInsight further triggers analyzers to perform diverse cellular analytics. We have realized various analyzers, all of which are included in mobile_insight/analyzer, as shown below (this list still goes on and long):

./mobileInsight-desktop/mobile_insight/analyzer/
├── __init__.py
├── analyzer.py
├── wcdma_rrc_analyzer.py
├── log_analyzer.py
├── lte_mac_analyzer.py
├── lte_rrc_analyzer.py
├── lte_nas_analyzer.py
├── lte_pdcp_analyzer.py
├── lte_phy_analyzer.py
├── lte_rlc_analyzer.py
└── ...

Some common analyzers are summarized as below. Note that to develop a cellular protocol analyzer, it is usually required to have a comprehensive understanding of the 3G/4G cellular protocols. This requires to read 3GPP standards, as listed below. Instead of elaborating the protocol-specific analytics, this article will focus on the common APIs, conventions, and structures of an analyzer. We next elaborate it.

Base class: Analyzer¶

All MobileInsight analyzers should inherit from the base class Analyzers, which defines the basic interface for a monitor: - Declare its dependency on cellular message types: enable_log; - Declare its dependency on other analyzers: include_analyzer; - Bind to a monitor: set_source; - Callbacks in response to events from monitor (add_source_callback) or other analyzers (include_analyzer); - (Optional) pushes runtime cellular information to other mobile apps (with MI-APP API).

Each analyzer should overload these interfaces based on its specific environment. The following code snippet summaries the abstract interfaces (complete code in mobile_insight/monitor/analyzer.py):

from ..element import Element, Event

class Analyzer(Element):

    def __init__(self):
        Element.__init__(self)

        """
        Task lists
        1. Initialize empty monitor
        2. Initialize empty list of callbacks for monitor and other analyzers
        """
        # ...

    def set_source(self,source):
        """
        Bind this analyzer to a monitor (source)
        The messages from the source will drive the analysis.
        All parent analyzers this analyzer depends on will also be recursively bound to source
        """

    def add_source_callback(self,callback):
        """
        Add a callback function to the analyzer.
        When a message arrives, the analyzer will trigger the callbacks for analysis.
        """

    def rm_source_callback(self,callback):
        """
        Delete a callback function to the analyzer.
        """

    def include_analyzer(self,analyzer_name,callback_list,*args):
        """
        Declares the dependency from other analyzers.
        Once declared, the current analyzer will receive events
        from other analyzers, then trigger functions in callback_list
        """

    def recv(self,module,event):
        """
        Handle the received events.
        This is an overload member from Element
        """

In the following, we will use one example LtePhyAnalyzer to show how to develop your own analyzer, and overload above interfaces.

Declare its dependency on cellular message types: enable_log¶

An analyzer should declare the cellular messages it need for protocol analytics. This is realized by calling enable_log, which is typically called when binding the analyzer to the monitor. In this way, the monitor will activate the corresponding message collection at hardware, and pushes this message to analyzer by calling the source callbacks (registered via add_source_callback).

The following shows one example of how to achieve it. In writing a new analyzer, we should overload set_source to include supported message types, and register the callback in __init__ initialization function.

def set_source(self,source):
        """
        Set the trace source. Enable the cellular signaling messages

        :param source: the trace source (collector).
        """
        Analyzer.set_source(self,source)

        #Phy-layer logs
        source.enable_log("LTE_PHY_PDSCH_Packet")
        source.enable_log("LTE_PHY_PUSCH_CSF")

def __init__(self):
        Analyzer.__init__(self)

        self.add_source_callback(self.__msg_callback)
        # ...

Declare analyzer dependency include_analyzer¶

In many cases, your analyzer can be built on top of existing analyzers, thus reusing their analytical results. This modular approach simplifies the development, and makes code reusing possible. Typically, to declare the dependency of an analyzer (i.e., “parent analyzer”), your analyzer should call include_analyzer during the initialization (in __init__), and passes a callback function in response to these analyzer’s events. The following code shows how it works:

def __init__(self):
        Analyzer.__init__(self)
        #...
        #include analyzers
        self.include_analyzer("LteRrcAnalyzer",[self.__on_event])
        self.include_analyzer("WcdmaRrcAnalyzer",[self.__on_event])
        #...

def __on_event(self,msg):
        """callbacks for the 3G/4G RRC messages"""
        #...

Event-driven Callbacks¶

For both monitor and parent analyzer, an analyzer should provide a callback function in response to their cellular events. These callbacks are registered when binding to a monitor (add_source_callback), or declaring a parent analyzer (include_analyzer). In both cases, the callback takes one parameter msg as input, which is the event raised by monitor or parent analyzer. Inside the callback, you can decode this message, and perform specific analytics tasks.

The following shows the cellular message callback in LtePhyAnalyzer. This callback is triggered by cellular messages from Monitor. It first determines the message type, decode the messages, then take specific actions.

def __msg_callback(self,msg):
    if msg.type_id == "LTE_PHY_PDSCH_Packet":
        self.callback_pdsch(msg)
    elif msg.type_id == "LTE_PHY_PUSCH_CSF":
        self.callback_pusch(msg)
    elif msg.type_id ==  "LTE_MAC_UL_Tx_Statistics":
        self.callback_pusch_grant(msg)

def callback_pusch(self,msg):
    """
    Callback for LTE_PHY_PUSCH_CSF.
    Currently it updates CQI.

    :param msg: raw LTE_PHY_PUSCH_CSF packet
    """

    log_item = msg.data.decode()
    self.cur_cqi0 = log_item['WideBand CQI CW0']
    self.cur_cqi1 = log_item['WideBand CQI CW1']

Push notifications: MI-APP¶

Besides event-driven callbacks, the analyzer also provides another interface to push analytics results to other mobile apps. This is called MI-APP API. To learn how to use it, you can read the following tutorial:

How to use ``MI-APP API`? <mi-app>`__

Cellular message configuration and parsing: dm_collector_c¶

The introduction of this module can be found here:

How to write a message parser?

Wireshark helper: ws_dissector¶

ws_dissector is simply a wrapper of Wireshark’s parsing functions. It is called by DmLogPacket (which calls WSDissector in `mobile_insight/monitor/dm_collector/dm_endec/ws_dissector.py) to parse cellular messages. The structure is as follows:

.
├── Makefile # Makefile
├── WiresharkCMakeTarget.txt #CMake configuration for Wireshark
├── packet-aww.cpp
├── packet-aww.h
└── ws_dissector.cpp # Wrapper of wireshark parsers

Desktop-version packaging and installation: setup.py and MANIFEST.in¶

MobileInsight leverages the standardized Python setup scripts to package its desktop-version modules. It has two files:

• ``MANIFEST.in``: it specifies which files to be included, which files should not be included (such as dm_collector_c). It follows the standardized Python setup script format, as shown below (mobileInsight-desktop/MANIFEST.in):

include README.md
include LICENSE
include setup.py
recursive-include GUI *
recursive-include examples *.json *.py *.mi2log *.mi2app *.txt
recursive-include mobile_insight *.py  *.m *.pyd README
recursive-exclude examples *.db
recursive-exclude mobile_insight *.db
exclude mobile_insight/monitor/android_qmdl_monitor.py
exclude mobile_insight/analyzer/handoff_loop_analyzer.py
prune dm_collector_c
prune docs/docs

• ``setup.py``: this file is used to install MobileInsight on desktop. It is a standardized Python setup script, and implements the following installation logic:

1. It checks the desktop’s OS version (Win/OS X/Linux), and downloads the corresponding ws_dissector, ‘dm_collector_c.so/pyd` and wireshark libraries;

2. It copies all Python modules to Python-2.7’s default directory, and other binaries/libraries to system folders.

Desktop-version GUI¶

The details of this GUI can be found here:

Using MobileInsight GUI