Using MQTT to Minimize Data Consumption and Maximize Smartphone Battery Life

5 July 2018
Technology

An overview of how we implemented MQTT to minimize data consumption and maximize battery life, thereby enabling effective relay of a continuous stream of valuable data from our pilots’ smartphones.

 

Introduction

To ensure 100% data driven operations and enable advanced technology applications, our pilots (truck drivers) and vehicles are connected and tracked through IoT devices. RIVIGO pilots carry smartphone devices and use the pilot app which enables relay of a continuous stream of valuable data. There were two major challenges here – heavy draining of battery life and high data consumption. Of course, continuous charging is one of the hardware solutions, but this could have adverse effects on battery life in the long term. The solution had to be one that led to reduction in data packets ensuring reliability, security and high availability.

 

Objective

Our key objective was to build a solution that would maximize battery life and minimize data consumption on a pilot’s smartphone. This meant that the solution should enable the following.

  • Live tracking: For live tracking, periodically re-creating connection causes high battery drainage. We needed to build a solution that could maintain a live connection.
  • Connectivity: Our pilots work in remote area where access to good internet connectivity can be a challenge. Hence, reducing payload on each network call and saving mobile data become critical.
  • Reliability: High reliability with guaranteed message delivery.

After carefully evaluating alternatives, we decided to use MQTT as it maintains a live connection, reduces payload on each network call and is highly reliable as it works on a TCP layer.

 

What is MQTT ?

MQTT (Message Queue Telementory Transport) is a very lightweight and binary protocol, which excels when transferring data over the wire in comparison to protocols like HTTP. This is because MQTT has only a minimal packet overhead. Another important aspect is that MQTT is extremely easy to implement on the client side. This fit perfectly for constrained devices with limited resources.

MQTT is decoupled across three dimensions.

  • Space – Publisher and subscriber do not need to know each other (for example, by IP address and port)
  • Time – Publisher and subscriber do not need to run at the same time
  • Synchronization – Operations on both components are not halted during publishing or receiving 

Quality of service

Quality of service (QoS) refers to a network’s ability to achieve maximum bandwidth and deal with other network performance elements like latency, error rate and uptime. Quality of service also involves controlling and managing network resources by setting priorities for specific types of data (video, audio, files) on the network.

MQTT provides three levels of QoS. Below is the functionality of each of these levels:

  • QoS 0 – In QoS level 0, it is guaranteed that the message will be delivered at most once to the receiver. This means that a message won’t be acknowledged by the receiver or stored and re-delivered by the sender. This is often called “fire and forget” and provides the same guarantee as the underlying TCP protocol.
  • QoS 1 – When using QoS level 1, it is guaranteed that a message will be delivered at least once to the receiver. But the message can also be delivered more than once.
  • QoS 2 – The highest QoS is 2, it guarantees that each message is received only once by the counterpart. The guarantee is provided by two flows there and back between sender and receiver. 

RabbitMQ

We used RabbitMQ as the MQTT Broker. RabbitMQ is lightweight and easy to deploy on premises and in the cloud. It supports multiple messaging protocols. RabbitMQ can be deployed in distributed and federated configurations to meet high-scale, high-availability requirements.

Earlier, on HTTP,  we used to capture pilot’s GPS and other data periodically. While working on periodic pushing/polling task on the mobile side, we also needed to take care of the system performance. HTTP does not maintain live connection and hence affects system performance and does not sit well with our objectives.

Why did we prefer MQTT over HTTP ?

HTTP is the most popular and widely used protocol. But over the last few years, MQTT has gained significant traction. The following features of MQTT helped us with our objectives:

  • Simple to implement
  • QoS data delivery where every Qos level maximizes the reliability and reduces the failure rate
  • Lightweight and bandwidth efficient as it reduces payload size and creates dedicated channels for publishers and consumers which minimizes data consumption
  • Continuous session awareness as it maintains continuous sessions between client and broker

MQTT follows a publisher/subscriber pattern. A client can send a message to another client without knowing who the publisher or the subscriber is. This means that the publisher and subscriber are decoupled and do not know about each other’s existence. They are connected to the broker which is a third important component of the protocol. Broker filters all the incoming message and distributes them accordingly.

 

Implementation

MQTT works in two parts, namely, the publisher and the subscriber (or consumer).

Publisher

In our case, it is the pilot device which publishes the GPS and battery information. Whenever a pilot completes his login process, we establish a connection between the publisher and the broker. Once the connection is created, we do not recreate it periodically.

  • Client ID:  The client identifier (short Client ID) identifies the connecting client to the broker and serves as the pilot’s unique identification
  • Topic name: A topic is a UTF-8 string, which is used by the broker to filter messages for each connected client. A topic consists of one or more topic levels. Each topic level is separated by a forward slash (topic level separator).

In comparison to a message queue, a topic is very lightweight. There is no need for a client to create the desired topic before publishing or subscribing to it. This is because a broker accepts each valid topic without any prior initialization.

  • Message: This is the actual content of the message. Since MQTT is data-agnostic, it is possible to send images, text in any encoding, encrypted data and virtually also all data in binary format.

On QoS level 1, once successfully published, broker sends an acknowledgement to the client. Broker maintains a queue which stores data until consumption. Broker’s queue size depends on machine storage allocated size which is configurable.

In the above snippet (taken from our android client), MQTT publisher thread runs on runnable thread so it will not block any UI updates. This is why the user will not experience any lag during publishing the data. The mqttAndroidClient sends an encoded message to the broker on respective topic along with the corresponding identifier. Here we maintain local storage (SQLite database) on the client side as well so to prevent data loss during unavailability of internet.

Consumer

In our case, Chronos (data storage layer) acts as a consumer which subscribes to relevant topic with the targeted identifier. On the broker side, once data is successfully consumed, the broker gets an acknowledgement and releases the memory. Consumed data is further stored in Mongo.

  • Athena: Athena takes raw data and passes it to advance processing. Our pilot continuum and pilot auto allocation engines use processed data and convert it for application purpose.
  • Topic name: Once successfully published, the subscriber subscribes to the data with the corresponding identifier and topic name. Topic gives the information about what data does the consumer want to consume.
  • QoS level: When using QoS level 1, it is guaranteed that a message will be delivered at least once to the receiver. But the message can also be delivered more than once.

 

Tech Stack

  • Java (Spring), Mongo: Processing of raw data streams from pilot device sensors is done using Java and data is stored in the Mongo database. The sensors send data at a regular frequency and over time, raw data becomes huge in size and non-relational in nature. Hence, mongo best fits the requirements.
  • RabbitMQ: Broker which is used to filter data and distribute it in a systematic way. The advantages of Rabbit MQ were covered in an earlier section.
  • Android/SQLite: Batch data is stored in SQLite in android client to maintain reliability in case of connection loss. Upon active network connection, data is published.

 

Impact

We used battery historian for measuring the battery and data usage by the pilot app. This helped us accurately measure data usage and battery consumption.

 

Screenshot of impact of MQTT on battery and data consumption

 

The screenshot above shows how data consumption dropped to over half the original levels once we moved to MQTT. It also helped improve battery life by about 4%.

MQTT opens new areas for messaging use-case for billions of things connected through the Internet. As MQTT specializes in low-bandwidth, high-latency environments, it is considered to be an ideal protocol for machine-to-machine (M2M) communication. Most of time RIVIGO pilots are working in remote areas where they get limited access of resources. By implementing MQTT, we’ve solved for some key challenges in this regard and enabled effective data collection which fuels our pilot auto allocation engine and other applications.