MNBB – TCP Client, Parsing and Queries

From here forward I will be including technical posts about my new project; MNBB which stands for Mariners NMEA Black Box. I mentioned it a couple of posts back and have been hacking on it for the last week getting the foundation going so I can start working on the small parts.

First, a little background about how I decided to start this project.

When I first took up sailing, I had no idea about NMEA or even marine networks. It’s a technical topic with several layers of confusion. First, the NMEA standard is extremely out of date. When you have an instrument such as a barometer, you can send the barometer data to another instrument. NMEA experts call this talking. Any instrument can talk to several other instruments. When that instrument wants to receive data from different instrument, it is limited to one single source. This is where the problem lies. To make your marine navigation system play well with others you need to take all of the instrument NMEA wires and plug them into something called a multiplexer so it can listen to many instruments and then talk to many other instruments, multifunctional displays or networking devices. A multifunctional display would include your chartplotter and a networking device can transmit the nmea data to something like a computer or smartphone so the apps can use the data to assist you in navigating your vessel.

What does not exist already is a data logger or any kind of device that can recall what your instruments were outputting on a given voyage. If they do exist, they would cost exorbitant amounts of money and deliver not so promising capabilities. For a recreational consumer this cost tends to be prohibitive. The idea came up to develop this capability using a very inexpensive computer and completely open source software so the consumer can actually afford this kind of instrument. Since it’s just me working on MNBB this could take the better part of a year in my spare time. I’ve opened the project to anyone who is capable of contributing in the hopes that it will speed up development and deliver a beta version as soon as possible.

The benefit to this kind of product is mainly for those who would like to have a look at their instrument data and try to make better decisions about their current and future voyages. For example, if you could follow barometric pressure over a selected period of time, perhaps you could see that the barometer is dropping at an alarming rate. I like to be able to use technology to help me keep track of what has happened in the past so I can recall this information whenever I want. What about comparing wind speed and barometric pressure? Can you think of anything that might be useful in that comparison? I think there is quite a bit worth looking at.

Getting Started

The first step in this project was getting my Raspberry Pi computer up and running. This required just a few temporary peripherals. I needed a monitor, mouse and keyboard so I could install the operating system and boot the computer for the first time. After that, I no longer needed the peripherals because I use my laptop to login to the Pi and do everything I need. The second step was to decide what technology I can use to start writing the application. Since I am a JavaScript engineer and I like the simplicity of Node.js, I decided to use it for the entire architecture stack. The benefit is that the entire application will be written in one single language. We call this full-stack JavaScript. Don’t let anyone convince you that this is not a good application platform. It has been proven for it’s simplicity at working with networking, file systems, database communication and most other common needs of an application platform. Once I had Node running, I could use a simple client script to connect to the network and stream NMEA sentences from the Vesber XB wifi NMEA client. It took about an hour to get it working.

The next step after knowing my Raspberry Pi was receiving NMEA sentences was to start parsing the sentences to something meaningful. I had already seen a Node NMEA parser in Github and it is under the same license as the MNBB project so I forked the project and it worked right out of the box. Each NMEA sentence has an identifier that tells you what type of sentence it is. The parser already includes the following: GPGGA, GPRMC, GPGSA, GPGSV, GPVTG. To prove that the parsing library was built properly and easy to contribute to I decided to write my own parser for GPGLL as a proof of concept, which is a simple global positioning latitude and longitude sentence. To my surprise, within an hour I had developed a complete parser for an unsupported sentence. 

NMEA parser
NMEA parser
NMEA data storage at intervals
NMEA data storage at intervals

The next step is storing the data. This is called a data logger and so the next step is to build a simple database that can collect the parsed sentences and store them for later retrieval. This will take some time to complete because I am not completely certain the best way to store everything. An important step is ensuring that not every single sentence is stored. I only need a subset of the data because there will be a ton of duplicate data, which can cause storage capacity issues over a long period of time. One issue I can see is how some types of data may need more sentences to be stored than others. For example, AIS sentences provide real-time information about other vessels broadcasting their position, speed, distance, etc. These sentences need to be parsed differently than something like water temperature, where you might only need to log the single value once every five minutes.

NMEA data parsed into JSON
NMEA data parsed into JSON

What’s next?

At this point in time I am building a Mongo database for Linux on the Pi and should be able to begin storing all of the supported sentences at whatever intervals I want. I am able to connect to the NMEA client, receive a set of data and then send the data to a Mongo database. I can also run queries with a given talker type (e.g.- GPGLL) and respond with a set of results that can used in a web application. I am going to integrate the web application framework next so I can write the first web app and view it from any device I have connected to the same network. I am also hoping that by the time I have the first release that getting Mongo installed on a Pi will not take twenty-four hours to build from scratch.

The scale of some of the projects is beyond my capabilities at the moment. It’s a perfect way to understand how to gain skills to understand how some of the information is determined for marine navigation. For instance, comparing apparent wind and true wind requires an algorithm that will calculate true wind speed and direction from several input parameters (speed over water, heading, etc) . Once I have written the algorithm I can use this input as part of any wind instrument application.

2 Replies to “MNBB – TCP Client, Parsing and Queries”

    1. Josh,

      I took a short break to enjoy the summer but when it starts raining I’ll have more time to build the client application. I’ve decided to use Angular and build it as one page with calls to mongo using ajax or possibly web sockets, depending on the requirement. I’ve started using the promise pattern to avoid the callback antipattern, which should help with keeping code and architecture clean. Check out a short list of issues to see some of the next enhancements.

Leave a Reply

Your email address will not be published. Required fields are marked *